Swift/server-side-enc

Server Side Encryption

Abstract

This wiki page summarizes design aspects and insights into Server Side Encryption for Swift. The general scheme is to create a middleware that will encrypt the object data during PUT and decrypt it during GET. The target is to create two domains - the user domain between the client and the middleware where the data is decrypted and the system domain between the middleware and the data at rest (on the device) where the data is encrypted.

A design goal is to extend swift as necessary but to not change existing swift behaviors.

First level design highlights

The design described here encrypts once at the proxy. To enable encryption, the admin needs to add the encryption middleware to the pipeline. Support is given to non greenfield installations by:

1. Once the middleware is added, the middleware would be able to encrypt new objects and keep old objects non encrypted
2. A PUT object would result in encrypting the object data by the middleware if the container is marked for encryption. If an existing container is marked for encryption, existing objects that area already in the system would remain unencrypted (unless a new version is uploaded with PUT).
3. Encrypted objects would be decrypted even if the container is not marked for encryption
4. If middleware is removed, keys would not be exposed, new objects would be stored unencrypted, encrypted objects would not be decrypted. Adding the middleware would resume the previous state.
5. Each object would have its own key for encrypting the data.
6. The object key of 'AUTH_myaccount/mycontainer/myobject' would be stored as system metadata x-object-sysmeta-key-XXX where XXX is base64(hash('AUTH_myaccount/mycontainer')) with a value of enc(containerkey, objkey) - i.e. objkey encrypted using the container key. This would allow COPY operation without decrypting objects as described below. The hash used would be the Swift system hash (MD5 be default).
7. Non encrypted objects would have no x-object-sysmeta-key-* metadata.
8. New objkey would be randomly chosen during PUT object
9. new containerkey, accountkey would be randomly chosen during the creation of the container/account respectively and will never be changed.
10. The container key is stored as system metadata x-conatiner-sysmeta-key with a value of enc(accountkey, containerkey) - i.e. containerkey encrypted using the account key.
11. The decrypted container key is cached together with the container metadata in memcache (requires enhancing Swift core)
12. The account key is stored as system metadata x-account-sysmeta-key with a value of enc(masterkey, accountkey) - i.e. accountkey encrypted using the master key (master key is stored per account by the key manager).
13. The decrypted account key is cached together with the account metadata in memcache (requires enhancing Swift core)

Server Side Encryption

Etag issues

We call the MD5 of the stored object 'de-etag' and the MD5 of the encrypted object en-etag. The en-etg will be stored in in the etag-field of the object allowing auditors to continue working unchanged.

The de-etag, if provided by the client, needs to be examined before the data is stored to allow rejecting wrongly etagged objects. This can be done by the middleware or we may do so at the proxy core code if we include support for swift.encrypt callback at the proxy core code.

During a HEAD, the etag needs to be retrieved. This can be wastefully at the middleware by implementing HEAD using a GET, than decrypting the data and recalculating the de-etag. The same needs to be done for GET.

Alternatively the etag may be calculated before encryption and stored along side the object such that it may be retrieved in a later HEAD/GET. Yet, such support requires placing the de-etag as an object metadata. This cannot be done without changing the object server code to prevent a case where a client later use POST Object and replace all object metadata.

After encryption, the en-etag needs to be calculated and stored in the object's etag-filed. This can be done by the middleware not sending any etag resulting in the etag being calculated by the swift core and added as one today.

Keys

Each object is encrypted with its own encryption key ,which is stored encrypted under a container key as part of a private object meta data field. This requires us to add support for such key in the Object Server to avoid the key being deleted during an object POST.

The container key in turn is stored in its a private metadata field after being encrypted under an account key. The encrypted container key is therefore cached. Q: do we want to cache the unencrypted key to improve performance? If we do so we do not need to cache the account master key.

The account key in turn is stored in its a private metadata field after being encrypted under the account master key. The encrypted account key is therefore cached. Q: do we want to cache the unencrypted key to improve performance? If we do so we do not need to cache the account master key. Q: do we want to remove this extra level of indirection and have the container keys encrypted by the account master key?

The account master key is stored by barbican or an alternative key manager. If we do not cache the unencrypted account key or the unencrypted container key we would need to cache the account master key to achieve reasonable performance.

On account master key change, we never re-encrypt the bulk data, but only re-rencrypt the (account/container) keys.

Consistency and Signitures

When decrypting objects, we do consistency check to make sure we are using the right key and internal checksums match; TODO - add a proper encrypt-and-mac to prevent any modifications of the objects.

Blob Encryption

The M2Crypto library is used for the crypto operations.

Unresolved issues

1. Support in COPY- it appears that support in COPY requires either reimplementing COPY in middleware or to modify proxy implementation e.g. by having the proxy call the encryption middleware to decrypt and encrypt the data during COPY
2. ...

Observations

• Activating the optional signature feature results in each PUT REST request being translated by the middleware to two separate internal REST calls at the middleware: PUT + POST. If the cluster is configured with copy on POST enabled, this result in a copy of each encrypted object. If many objects are encrypted, this would effect the cluster performance as it would tripe the i/o used during each encrypted PUT operations. It is recommended to not use encryption and copy on POST on a system that is required to encrypt many of its objects.