Latest revision as of 17:55, 26 February 2014

There are currently three ways in Nova to create a snapshot of one, some, or all of the volumes attached to a particular instance. But none of these options allow a user to create volume snapshots after I/O is quiesced as a single transaction. This proposal explains how this missing feature could be implemented, thus substantially improving the snapshotting capabilities of Nova.

Existing Behaviour

These are three ways of creating a snapshot-like thing in Nova:

create_image - takes a snapshot of the root volume and may take snapshots of the attached volumes depending on the volume type of the root volume. I/O is not quiesced.
create_backup - takes a snapshot of the root volume with options to specify how often to repeat and how many previous snapshots to keep around. I/O is not quiesced.
os-assisted-snapshot - takes a snapshot of a single cinder volume. The volume is first quiesced before the snapshot is initiated.

Proposed Changes

My general thesis is that I/O should be quiesced in all cases if the underlying driver supports it. Libvirt supports this feature and I would like to extend the existing functionality to take advantage of it.

It's not reasonable to change the names or behaviour of the existing public api calls. Instead I would like to create a new snapshot() call in the v3 API.

We only need a quiesce() call added to the driver and the rest of the implementation will live in the api layer. It was suggested that the conductor would be the correct place for this code. Once implemented, the existing snapshot calls (image, backup, os-assisted) could use the underlying snapshot routines to achieve their expected results. Leaving us with only one set of snapshot-related functions to maintain.

The new snapshot call would take at least one option: the drives that should be snapshotted:

snapshot(devices=['vda', 'vdb'])

Where a value of None implies all volumes.

This allows the user to snapshot only the root volume if a small bootable image is desired.

There will be no exclusion based on volume type, both glance and cinder volumes will be snapshotted respectively. Otherwise we introduce unexpected behaviour that would be confusing to the user and difficult to explain.

The flow will look like:

call the compute node to quiesce
call the compute node to snapshot each individual glance drive
call the volume driver to snapshot each cinder volume
package the whole thing

The packaged snapshot could be stored in either Swift or Glance. I think Glance is more appropriate since the resulting collection of snapshots could be used to spin up a new instance. There is a pending proposal for "artifacts" in Glance that would be perfect for this - an instance template artifact would contain metadata that references the volume snapshots for an instance. These references would point to either images stored within Glance or snapshots contained in Cinder. All together this data represents a perfect point-in-time snapshot of an entire virtual machine.

If create_image and create_backup are updated to use this implementation, then the behaviour will appear unchanged to the user with the exception that I/O was quiesced during the snapshot(s) and they therefore have a more reliable and useful result.

Given this, I think it makes more sense to leave the implementation within the api layer of Nova so that existing functions can share in the implementation - as opposed to moving it into the client.

External Prerequisites

This feature will require a support from libvirt - the ability to quiesce an instance without taking a snapshot. At the moment these two operations are bound together. I will begin working with the libvirt community to add support for this feature.

@@ Line 1: / Line 1: @@
-= Nova Instance-Level Snapshots =
+There are currently three ways in Nova to create a snapshot of one, some, or all of the volumes attached to a particular instance. But none of these options allow a user to create volume snapshots after I/O is quiesced as a single transaction. This proposal explains how this missing feature could be implemented, thus substantially improving the snapshotting capabilities of Nova.
-In a nutshell, we want the ability to take a snapshot of a running instance along with all attached volumes as a single psuedo-atomic operation.  This gives the user a filesystem-consistent snapshot (with qemu-assisted) or application-consistent snapshot (with guest-assisted) of all local storage accessible to a particular virtual machine at that time.
+= Existing Behaviour =
-== Current Behaviour ==
+These are three ways of creating a snapshot-like thing in Nova:
-Today a user can request a snapshot of a VM and that snapshot will contain only the contents of the rool volume.  Both image-backed and volume-backed root volumes are supported.  For image-backed instances, the resulting snapshot will be contained by glance as an available image.  For volume-backed instances, the resulting snapshot is contained by cinder as an available volume.  At snapshot-time, any additional attached volumes must be snapshotted separately, leaving no option to capture them all at once.
+# <code>create_image</code> - takes a snapshot of the root volume and may take snapshots of the attached volumes depending on the volume type of the root volume. I/O is not quiesced.
+# <code>create_backup</code> - takes a snapshot of the root volume with options to specify how often to repeat and how many previous snapshots to keep around. I/O is not quiesced.
+# <code>os-assisted-snapshot</code> - takes a snapshot of a single cinder volume. The volume is first quiesced before the snapshot is initiated.
-== Desired Behaviour ==
+= Proposed Changes =
-A way to indicate to create_image() that all attached volume should be included in a snapshot.
+My general thesis is that I/O should be quiesced in all cases if the underlying driver supports it. Libvirt supports this feature and I would like to extend the existing functionality to take advantage of it.
-== Proposed Changes ==
+It's not reasonable to change the names or behaviour of the existing public api calls. Instead I would like to create a new snapshot() call in the v3 API.
-The entry point to the Nova API for this feature is create_image().  This method will be modified to look for an additional aoption 'all_volumes'.  If this option is presen and is set to True, then create_image() will ensure that a snapshot request is issued for all attached volumes.  The resulting snapshot will be a combination of the root volume snapshot along with additional cinder volume snapshots.  These snapshots will be named in such a way to make correlation by the user an easy operation.  The naming scheme will be covered later in this document.
+We only need a quiesce() call added to the driver and the rest of the implementation will live in the api layer. It was suggested that the conductor would be the correct place for this code. Once implemented, the existing snapshot calls (image, backup, os-assisted) could use the underlying snapshot routines to achieve their expected results. Leaving us with only one set of snapshot-related functions to maintain.
-The all_volumes flag will be passed to the compute_api, either snapshot() for image-backed instances or snapshot_volume_backed() for volume-backed instances.  Regardless of the different snapshot implementations, the modified logic will be similar to the following pseudocode:
+The new snapshot call would take at least one option: the drives that should be snapshotted:
-  * volumes = get_all_attached_volumes()
+<pre>snapshot(devices=['vda', 'vdb'])</pre>
-  * freeze_instance()
+Where a value of None implies all volumes.
-  * snapshot_root_volume()
-  * for volume in volumes:
-        snapshot_volume(volume, clever_name)             [volume_api.create_snapshot() via cinder]
-  * thaw_instance()
-  * collect metadata for response
-  * return response
-== Missing Pieces ==
+This allows the user to snapshot only the root volume if a small bootable image is desired.
-The primary missing piece is freeze/thaw.
+There will be no exclusion based on volume type, both glance and cinder volumes will be snapshotted respectively. Otherwise we introduce unexpected behaviour that would be confusing to the user and difficult to explain.
-  * Does this functionality exist currently?
+The flow will look like:
-  * If so, can we use it similarily to what is described above?
-  * What other pieces are missing?
-== Todo ==
+* call the compute node to quiesce
+* call the compute node to snapshot each individual glance drive
+* call the volume driver to snapshot each cinder volume
+* package the whole thing
-  * Cinder volume naming scheme for snapshot correlation
-== Resolved Questions ==
+The packaged snapshot could be stored in either Swift or Glance. I think Glance is more appropriate since the resulting collection of snapshots could be used to spin up a new instance. There is a pending proposal for &quot;artifacts&quot; in Glance that would be perfect for this - an instance template artifact would contain metadata that references the volume snapshots for an instance. These references would point to either images stored within Glance or snapshots contained in Cinder. All together this data represents a perfect point-in-time snapshot of an entire virtual machine.
-  * Instead of snapshotting the instance, could we get away with *only* the attached volumes?
+If <code>create_image</code> and <code>create_backup</code> are updated to use this implementation, then the behaviour will appear unchanged to the user with the exception that I/O was quiesced during the snapshot(s) and they therefore have a more reliable and useful result.
-I don't see any reason that a user would want all attached volumes without the root volume.  If a need does arise for this behaviour, it can be added without too much trouble.
+Given this, I think it makes more sense to leave the implementation within the api layer of Nova so that existing functions can share in the implementation - as opposed to moving it into the client.
+= External Prerequisites =
+This feature will require a support from libvirt - the ability to quiesce an instance without taking a snapshot. At the moment these two operations are bound together. I will begin working with the libvirt community to add support for this feature.

Difference between revisions of "Nova/InstanceLevelSnapshots"

Latest revision as of 17:55, 26 February 2014

Existing Behaviour

Proposed Changes

External Prerequisites