Blueprint-cinder-volume-driver-id

This is the design for the Cinder volume-driver-id blueprint.

Overview

This blueprint proposes the addition of a new driver_id field to the Cinder Volume table, which can be used to store a driver-specific reference to the backing storage used by a Volume.

The benefits of the blueprint are:

Cinder drivers that are capable of “naming” the resource associated with a Cinder Volume (e.g. a disk on an external storage controller) are no longer required to embed the Cinder Volume UUID into that name, freeing up the name to be used for a friendlier purpose, such as reflecting the Cinder Volume’s display name.
Higher performance and simpler code by allowing Cinder drivers to refer to their backing storage in a way that is most efficient and natural for them, rather than having to introduce a means of keying off the Cinder Volume UUID. For example, in the Storwize V7000 driver, disks are currently named based on the Volume’s “name” property (which is of the form prefix-uuid), rather than the preferred internal Storwize Volume ID.
Removing the requirement for backend storage to present itself in a certain way (e.g. for Storwize, with the correct disk name) in order to be consumable by OpenStack. This capability simplifies the volume-migration blueprint, which may require that during migrations a given Cinder Volume must refer to multiple disks (and on some storage controllers, no two disks may have the same name). It would also enable a possible future capability to “import” existing storage into Cinder without having to rename the storage.

Alternatives considered

Volume/Snapshot metadata is not suitable because it can be modified by the user. It would constitute a security hole to allow a user to change the backend storage that their Cinder Volume or Snapshot used.

The existing ec2_id and provider_location field illustrate the need for backend-specific data associated with a Volume, but in both cases these existing fields are not suitable, for the following reasons:

ec2_id is an integer, and thus too inflexible to be a generic driver reference.
provider_location is a suitable size, but is used to store iSCSI connection details, which would reasonably be in use at the same time as a driver_id field.

Requirement

The requirement is to introduce a new driver_id field to the Cinder Volume database table, which a Cinder driver may optionally make use of. Cinder drivers that take advantage of this feature are free to name their backing storage by any means; they do not need to use the formal name that is associated with a Cinder Volume today and derived from a configuration file template and the volume’s UUID.

The new driver_id field will default to NULL, with database upgrades adding the field. Database downgrades will error if any non-NULL driver_id field exists, because it would not be guaranteed that Cinder Volumes that used driver_id could be referenced correctly in earlier code versions.

Assuming that Cinder drivers may now be reflecting properties of the Cinder Volume in some way onto the backend storage (e.g. by naming disks based on Volume display name), any modification to the Cinder Volume’s properties must now be signalled down to the Cinder driver, which may implement an optional update_volume API that can reflect changes to a Cinder Volume on the backend storage.

Migration Considerations

It is important that a migration from an earlier release of OpenStack would continue to function correctly, which means that drivers should tolerate Volumes and Snapshots that are missing a driver_id, and fall back to the identification scheme that existed prior to the driver supporting the driver_id field.

To support future migration paths, should a driver decide to change its internal usage of driver_id, it would be sensible for the driver to embed not only a reference to the backend storage in the driver_id, but also an indication of the referencing scheme used. For example, the Storwize/SVC driver, using the vdisk ID as a reference, for a disk with a vdisk ID of “5” would be wise to set the driver_id to “vdisk:5” rather than just “5”, to support any future changes to the referencing scheme.

Downgrade of a database should be suppressed if any Volumes exist that use the driver_id field, because there would be no way to reliably reference those Volumes without the presence of that field.

Detailed Changes Required

cinder/db/sqlalchemy/models.py

Introduce driver_id property as a Column(255) to the Volume and Snapshot tables.

cinder/db/sqlalchemy/migrate_repo/versions/xxx_add_backend_id_columns.py (NEW)

This new file inserts the driver_id field to the Volume and Snapshot tables during database upgrade.

During database downgrade, the driver_id field is removed from the Volume and Snapshot tables only if it is unused in both tables. If a driver_id is in use, the downgrade fails.

cinder/api/v1/snapshots.py cinder/api/v1/volumes.py cinder/api/v2/snapshots.py cinder/api/v2/views/volumes.py

Modify the various view methods so that the driver_id field can be returned via the REST APIs.

cinder/volume/api.py

Modify the update and update_snapshot methods to notify the driver of a change of properties by calling into a new volume_rpcapi methods.

cinder/volume/rpcapi.py

Implement new update and update_snapshot methods to notify the driver of a change of properties by performing an RPC call to new methods on the volume manager.

cinder/volume/manager.py

Implement new update and update_snapshot methods to notify the driver of a change of properties by calling into new methods on the driver.

cinder/volume/driver.py

Provide an empty implementation of the new update and update_snapshot methods, each of which are passed a data structure describing the new state of the volume from Cinder’s perspective.

cinder/volume/drivers/storwize_svc.py

A reference implementation of this blueprint.

The driver_id will be used to store the Storwize/SVC vdisk ID when a new Volume is created, and the display_name will be used to build the disk name on the backing storage controller. If the specified name is not valid on the storage controller, the operation will be retried using the programmatically generated name property of the Volume, as is the case today.

The update and update_snapshot methods will be implemented to update the name of the disk on the storage controller in the event that the display_name of a Volume or Snapshot changes.

Staging

Code will be delivered in three stages:

Stage 1: Basic driver_id enablement

This deliverable will introduce the database modifications to upgrade/downgrade the driver_id field, and expose it through the REST APIs.

Stage 2: Storwize/SVC reference implementation

This deliverable will modify the Storwize/SVC driver to demonstrate how it can be used.

Stage 3: Volume and Snapshot update

This deliverable will perform the modifications through the stack that are necessary to propagate Volume and Snapshot updates down to the Cinder driver.

The following table indicates which files would be modified in which deliverables:

	1	2	3
cinder/db/sqlalchemy/models.py	X
cinder/db/sqlalchemy/migrate_repo/versions/xxx_add_backend_id_columns.py	X
cinder/api/v1/snapshots.py	X
cinder/api/v1/volumes.py	X
cinder/api/v2/snapshots.py	X
cinder/api/v2/views/volumes.py	X
cinder/volume/api.py			X
cinder/volume/rpcapi.py			X
cinder/volume/manager.py			X
cinder/volume/driver.py			X
cinder/volume/drivers/storwize_svc.py		X	X