Difference between revisions of "Swift/server-side-enc"

Revision as of 13:04, 6 November 2013

Server Side Encryption

Abstract

This wiki page summarizes design aspects and insights into Server Side Encryption for Swift. The general scheme is to create a middleware that will encrypt the object data during PUT and decrypt it during GET. The target is to create two domains - the user domain between the client and the middleware where the data is decrypted and the system domain between the middleware and the data at rest (on the device) where the data is encrypted.

A design goal is to extend swift as necessary but to not change existing swift behaviors.

First level design decissions

The design described here encrypts once at the proxy. To enable encryption, the admin needs to add the encryption middleware to the pipeline. Support is given to non greenfield installations by:

1. Once the middleware is added, the middleware would be able to encrypt new objects and keep old objects non encrypted
2. New objects would only be encrypted by the midleware if they are placed in a container marked for encryption. If an existing container is marked for encryption, only new objects (or new versions of existing objects) would be encrypted
3. If middleware is removed, keys would not be exposed, yet encrypted data would not be decrypted

Once added, the encryption middleware cretaes Encryption is enabled/disabled per object such that the same container may include encrypted and non encrypted objects.

The user cannot enable/disable encryption after the container was created (as this would require A container a user can specify a metadata value for that. The middleware stores state using container metadata on private keys of the container not accessible by the users.

Encryption is done on the proxy nodes.

Server Side Encryption

Etag issues

We call the MD5 of the stored object 'de-etag' and the MD5 of the encrypted object en-etag. The en-etg will be stored in in the etag-field of the object allowing auditors to continue working unchanged.

The de-etag, if provided by the client, needs to be examined before the data is stored to allow rejecting wrongly etagged objects. This can be done by the middleware or we may do so at the proxy core code if we include support for swift.encrypt callback at the proxy core code.

During a HEAD, the etag needs to be retrieved. This can be wastefully at the middleware by implementing HEAD using a GET, than decrypting the data and recalculating the de-etag. The same needs to be done for GET.

Alternatively the etag may be calculated before encryption and stored along side the object such that it may be retrieved in a later HEAD/GET. Yet, such support requires placing the de-etag as an object metadata. This cannot be done without changing the object server code to prevent a case where a client later use POST Object and replace all object metadata.

After encryption, the en-etag needs to be calculated and stored in the object's etag-filed. This can be done by the middleware not sending any etag resulting in the etag being calculated by the swift core and added as one today.

Keys

Each object is encrypted with its own encryption key ,which is stored encrypted under a container key as part of a private object meta data field. This requires us to add support for such key in the Object Server to avoid the key being deleted during an object POST.

The container key in turn is stored in its a private metadata field after being encrypted under an account key. The encrypted container key is therefore cached. Q: do we want to cache the unencrypted key to improve performance? If we do so we do not need to cache the account master key.

The account key in turn is stored in its a private metadata field after being encrypted under the account master key. The encrypted account key is therefore cached. Q: do we want to cache the unencrypted key to improve performance? If we do so we do not need to cache the account master key. Q: do we want to remove this extra level of indirection and have the container keys encrypted by the account master key?

The account master key is stored by barbican or an alternative key manager. If we do not cache the unencrypted account key or the unencrypted container key we would need to cache the account master key to achieve reasonable performance.

On account master key change, we never re-encrypt the bulk data, but only re-rencrypt the (account/container) keys.

Consistency and Signitures

When decrypting objects, we do consistency check to make sure we are using the right key and internal checksums match; TODO - add a proper encrypt-and-mac to prevent any modifications of the objects.

Blob Encryption

The M2Crypto library is used for the crypto operations.

Unresolved issues

1. Support in COPY- it appears that support in COPY requires either reimplementing COPY in middleware or to modify proxy implementation e.g. by having the proxy call the encryption middleware to decrypt and encrypt the data during COPY
2. ...

Observations

• Activating the optional signature feature results in each PUT REST request being translated by the middleware to two separate internal REST calls at the middleware: PUT + POST. If the cluster is configured with copy on POST enabled, this result in a copy of each encrypted object. If many objects are encrypted, this would effect the cluster performance as it would tripe the i/o used during each encrypted PUT operations. It is recommended to not use encryption and copy on POST on a system that is required to encrypt many of its objects.