There are currently three ways in Nova to create a snapshot of one, some, or all of the volumes attached to a particular instance. But none of these options allow a user to create volume snapshots after I/O is quiesced as a single transaction. This proposal explains how this missing feature could be implemented, thus substantially improving the snapshotting capabilities of Nova.
These are three ways of creating a snapshot-like thing in Nova:
create_image- takes a snapshot of the root volume and may take snapshots of the attached volumes depending on the volume type of the root volume. I/O is not quiesced.
create_backup- takes a snapshot of the root volume with options to specify how often to repeat and how many previous snapshots to keep around. I/O is not quiesced.
os-assisted-snapshot- takes a snapshot of a single cinder volume. The volume is first quiesced before the snapshot is initiated.
My general thesis is that I/O should be quiesced in all cases if the underlying driver supports it. Libvirt supports this feature and I would like to extend the existing functionality to take advantage of it.
It's not reasonable to change the names or behaviour of the existing public api calls. Instead I would like to create a new snapshot() call in the v3 API.
We only need a quiesce() call added to the driver and the rest of the implementation will live in the api layer. It was suggested that the conductor would be the correct place for this code. Once implemented, the existing snapshot calls (image, backup, os-assisted) could use the underlying snapshot routines to achieve their expected results. Leaving us with only one set of snapshot-related functions to maintain.
The new snapshot call would take at least one option: the drives that should be snapshotted:
Where a value of None implies all volumes.
This allows the user to snapshot only the root volume if a small bootable image is desired.
There will be no exclusion based on volume type, both glance and cinder volumes will be snapshotted respectively. Otherwise we introduce unexpected behaviour that would be confusing to the user and difficult to explain.
The flow will look like:
- call the compute node to quiesce
- call the compute node to snapshot each individual glance drive
- call the volume driver to snapshot each cinder volume
- package the whole thing
The packaged snapshot could be stored in either Swift or Glance. I think Glance is more appropriate since the resulting collection of snapshots could be used to spin up a new instance. There is a pending proposal for "artifacts" in Glance that would be perfect for this - an instance template artifact would contain metadata that references the volume snapshots for an instance. These references would point to either images stored within Glance or snapshots contained in Cinder. All together this data represents a perfect point-in-time snapshot of an entire virtual machine.
create_backup are updated to use this implementation, then the behaviour will appear unchanged to the user with the exception that I/O was quiesced during the snapshot(s) and they therefore have a more reliable and useful result.
Given this, I think it makes more sense to leave the implementation within the api layer of Nova so that existing functions can share in the implementation - as opposed to moving it into the client.
This feature will require a support from libvirt - the ability to quiesce an instance without taking a snapshot. At the moment these two operations are bound together. I will begin working with the libvirt community to add support for this feature.