Jump to: navigation, search

Difference between revisions of "KeyManager"

(Caching Keys)
Line 128: Line 128:
'''Master-key''': a key-string used to encrypt the keys (key-strings) before saving in the key manager, saved in trusted storage at the service end-point.
'''Master-key''': a key-string used to encrypt the keys (key-strings) before saving in the key manager, saved in trusted storage at the service end-point.

Revision as of 11:48, 6 March 2013

Key Manager

Server side encryption with key management would make data protection more readily available, enable harnessing of any special hardware encryption support on the servers, make available a larger set of encryption algorithms and reduce client maintenance effort. Amazon and Google’s object storage systems provide transparent data encryption. Recently interest has grown in [[1]] to provide server side encryption in Cinder ( Volume ) [1], Swift ( Object ) [2], Glance ( Snapshot ).

Protecting data involves not only encryption support but also key management, the creating, storing, protecting, and providing ready access to the encryption keys. The keys would need to be stored on a device separate from that housing the data they seek to protect. Key management could be a separate OpenStack service or a sub-service of Keystone, OpenStack's identity service.

The keys themselves ideally would be random, of the desired length, with associated meta data such as ownership and themselves encrypted before being stored.

Security Model

  • Protection of data at rest: the encrypted data and the encryption keys are held in separate locations. Stealing the data disk still leaves the data protected.
  • Keys opaque: The keys themselves are encrypted using Master" keys.
  • Master Key Protection: Master keys are protected in Hardware using Trusted Platform Module (TPM) technology. Keys are released to only trusted host machines (BIOS and initial boot sequences that are measurably good ).
  • Secure Master Key Transmission: TPM technology is used to transfer master keys, from cooperating services and between sibling services (in the case of horizontal scaling).
  • Support Dual Locking: High value data could be protected with a user/project/domain specific key and a service key. This is akin to using two keys such as with bank safe-deposit boxes, a bank key and a customer key.
  • Limited Knowledge: Key Manager will not maintain mapping between keys to encrypted entities. Encrypted entities will maintain as meta data key-id (a pointer to a key to be used to unlock the same. But with dual keys, the customer key is not referenced, it is implicit as part of the authentication.
  • Limited Access: Authorization and access control mechanisms limit access to keys.
  • Protection from denial of service: multiple replicas of key manager.
  • Data Isolation: Should some audit or law enforcement authority demand access for a certain customer, data belonging to other customers not exposed because they use

different keys.


  1. API
  get <key-id>   <authorization-token>
  put <key-id> <encrypted-key-string> <authorization-token>
  delete<key-id> <authorization-token>
  update<key-id> <authorization-token>

Key Scope:

  • Per entity (entity could be a volume, an object, a VM image/snapshot)
  • Per user
  • Per project (within a domain)
  • Per domain

For strong encryption, typically a key is used in conjunction with an initialization vector (IV). The per-entity key would serve as an IV. It could be used alone or in conjunction with a wider scoped key, such as a domain scope key.


  • 128, 192, 256, .. 2048 .. longer or shorter (possibly used with padding).

Some algorithms require longer keys, so we support a wide range.

Keys in use

Swift (object storage) example: assume an object X is stored in encrypted for on the Swift object store. Let enc-object-x be the encrypted representation of object X. Then the Swift file system would contain: enc-object-x, with meta_data: <enc:true, algorithm:aes-cbc, key-id: 1234567899 >

Similarly, an encrypted Cinder volume might be represented as Volume<id>, meta_data: <enc:true, algorithm:aes-xts, key-id:abcdefghijklmnopqrstuvxyz>

Encryption Algorithm

These would be obtained by the OpenStack services by directly querying the libraries used to provide encryption support. The options would also be provided as options during user/project/domain creation, to set defaults. The options may further be offered with each entity creation (could get too chatty for high volume data such as objects). Typical options would be RSA, AES, DES etc.

Design Considerations

Key Manager Access

       Restricting access to the Key Manager to only OpenStack services other than the Compute Nodes (the least trusted of the hosts), and in so doing  increase security. No end user access.

Access Control

Keys inserted by a service only accessible by that service. User/Project/Domain, that is wider scope keys, owned by the Identity Service, which is part of keystone. Dual locking requires access to wider scope keys that do not belong to the service that provides the encryption support. This access is obtained using Keystone's Trust mechanism, available in the V3 API. It also needs access to the keystone master key used to encrypt the keys it owns.

Master Key

      Each OpenStack service that uses Key Manager to maintain its keys could have its own master key and use the same to encrypt a key string before passing it for storage to the Key Manager. The Master key could reside on a python key ring (currently it is included in common module in OpenStack and readily available to all packages).


1. Communication between the service and the key manager do not need to be further encrypted using ssl or https because they keys flying between them are at all times encrypted. The decrypted key string would at any time only reside on the service that seeks to save it or use.

2. Keys used by different open stack services could reside in a single storage system but if one service were to be compromised, the keys from other services would still be safe.

3. Further, should there be a desire to change a master key, only keys stored by that service need to be re-encrypted. The actual data that they were used to encrypt do not need to be re-encrypted. Key Manager in OpenStack

Key Flow

The figures below illustrate how the Key Manager fits into the regular flow of putting and getting an object in Swift. For simplicity, caching of keys and secondary key handling (for dual locking) is omitted.

Encrypted saving of an object

Decrypted retrieval of an object

Caching Keys

Key Manager’s keys need to be accessible at the same level as the objects they encrypt, to ensure ready access. The keys themselves could be cached at the service endpoint using them with an expiration equal to or less than that of the access token lifetime used to obtain them. Caching reduces network traffic and the load on the key manager. With dual keys, where the wider scope key is obtained through access delegation, the lifetime would be that of the delegation period.

Concerns and Questions

  1. With another service, Key Manager, in the picture, we have another component that could fail. But encryption will always need keys to be maintained, so this would be

a cost of the feature. Caching keys mitigates some of the problems, and using TPM protects the keys while they are being saved and transmitted by way of encryption, and only decrypted at point of use.

  1. Do we need to support KMIP in the key Manager. If the keys are just of end user encryption provision, perhaps not. However if we desire to use the key manager to save

private and public keys of the services in OpenStack, KMIP would be good to have.

  1. Data transfer overhead: Swift uses Rsync for file transfer during replication. Any encryption algorithm that uses some form of block cipher chaining or new initialization vector each time would result in the object representation changing drastically on each update. This would result in a larger network payload for transmission.
  1. Unauthorized key deletion: If we use a Swift based system to store keys and insert tombstone records to mimic a legitimate deletion after breaking into a Swift key storage node, yes, keys could indeed be deleted by a reaper task, but this would be no new security hazard from what Swift deals with today. Perhaps we could introduce a check that there was a logged request to delete a key before deleting a key.
  1. Wary of losing control of encryption key(s): Support the use case where the end user provides the encryption key (and stores a copy of their own key, and is responsible for maintaining safety of the key). The said key will not then be saved in the Key Manager.

7• Do we need an IV (initialization vector) for each object encrypted. Yes, if we take the common key for a project or domain approach. In this case the IV would need to be encrypted, and could be stored against a key-id. We could specify “compound-encryption” to imply use a master key in conjunction with the IV (accessed via the iv-id attached to the object meta-data).

8• No re-keying in phase-1. Not addressing background tasks of object re-keying such as that mentioned in Mirantis blog.

Implementation versions

Phase 1: Develop stub Key Manager service and specify encryption parameters in the url

Key manager could just be a hash table in the first version to get all the APIs specified and implement, to get the plumbing correct. Support a single most popular encryption algorithm. This would fully implement object encryption.

Phase-2: Make Key Manager is Swift instance, with multiple zones for storage. This would support true HA and fault tolerance.

Phase-3: Support multiple encryption algorithms. For instance, volume encryption may prefer XTS, an encryption strategy that uses sector address.

Phase-4: Reaper routine to change a master key for a service

Intel's Interest

Intel X86 hardware in Westmere and beyond provides AES-NI, hardware support for encryption/decryption, which provides performance improvements that can be as high as six or more times than pure software encryption. The hardware support also protects from certain side channel threats based on timing and memory analysis. In addition to hardware support, Intel's open source contributions for algorithm acceleration, that exploit context such common keys, data depencies, register widths to increase parallelism, and architectural features buy additional performance. [3] [4] [5]


Key-string: A string of bits used to encrypt data. Ideally auto-generated using a random number generator that exploits entropy. Intel's hardware random number generator is a high speed source of quality randomness.

Key-id: a unique ID used to index a key-string in the system. The key-id will be attached as meta data with the encrypted object/volume/.

Master-key: a key-string used to encrypt the keys (key-strings) before saving in the key manager, saved in trusted storage at the service end-point.

  1. https://blueprints.launchpad.net/nova/+spec/encrypt-cinder-volumes
  2. http://www.mirantis.com/blog/openstack-swift-encryption-architecture
  3. http://www.intel.com/content/www/us/en/communications/communications-ia-multi-buffer-paper.html
  4. http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/linux-secure-storage-performance-aes-ni-paper.pdf
  5. http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/ia-high-performance-storage-encryption-paper.pdf