Jump to: navigation, search


Revision as of 10:05, 16 May 2013 by Sam Stoelinga (talk | contribs) (Command line syntax for block config)

Block Device Configuration

This proposal is tracked in blueprint flexible-block-device-config

Controls for block device configuration

There are a variety of independent attributes that it is desirable to control when configuring block devices for a virtual machine in OpenStack

Source type

  • Glance image
  • Cinder volume
  • Ephemeral file

Destination type

  • local file
  • Cinder volume

Guest format

  • ext4 (or other FS formats)
  • ISO
  • Swap (different format per guest OS)
  • None

Device type

  • Disk
  • Floppy
  • Flash (MMC)

Disk bus

  • USB
  • IDE
  • VirtIO
  • SCSI

Shutdown action

  • Delete
  • Preserve

Boot order

  • Index

In addition to the above parameters, it is neccesary to support a "disk path" parameter, to allow the user to request a specific block device path in the guest eg /dev/sda. In general this cannot be supported by all hypervisors and thus its usage should be discouraged in favour of allowing the guest OS / hypervisor to choose paths. For the sake of compatibility with existing tools / APIs though, it must be supported

Command line syntax for block config

The 'nova boot' command currently uses a combination of the '-image' and '-block-device-mapping' command line args to configure storage. The latter is mostly a clone of the same named arg from the EC2 commands, and is not a very well designed syntax from the POV of extensibility.

Given the extra features to be supported, it is desirable to introduce a new syntax for 'nova boot' and friends which is more expressive

  • -block type=glance,id=XXXXXXX,bus=ide,bootindex=2,dest=volume
  • -block type=cinder,id=XXXXXXX,bus=ide,device=cdrom,bootdex=1
  • -block type=ephemeral,format=swap,size=50,bus=ide,device=floppy

The only compulsory argument ought to be the type + id, eg the glance or cinder ID. Everything else should have sane defaults filled in by the hypervisor driver in Nova.

Possible parameters would be

  • type=glance|cinder|ephemeral
  • dest=local|volume
  • id=XXXXXX (a volume/image UUID if using type=glance or type=cinder)
  • format=swap|ext4|...|none (to format the image/volume/ephemeral file; defaults to 'none' if omitted)
  • bus=ide|usb|virtio|scsi (hypervisor driver chooses a suitable default if omitted)
  • device=disk|cdrom|floppy|mmc (defaults to 'disk' if omitted)
  • bootindex=N (where N is any number >= 0, controls the order in which disks are looked at for booting)
  • size=NN (where NN is number of GB to create type=emphemeral image, or the size to re-size to for type=glance|cinder)
  • shutdown=preserve|remove (where the default is remove for type=glance|ephemeral or preserve for type=cinder)

Feedback Sam

It may be better to have default shutdown preserve for dest=volume and default remove for dest=local. Take for example:

  • -block type=glance,id=XXXXXXX,bus=ide,bootindex=2,dest=volume (Should preserve default)
  • -block type=cinder,id=XXXXXXX,bus=ide,bootindex=2,dest=volume (Should preserve default)

Both examples should be preserved, because the destionation is a volume. We shouldn't care about what the source type is imo. If type is cinder the dest has to be volume also, so as long as dest=volume the storage should be preserved.

Would this be possible?

  • - block type=cinder,id=XXXXXXX,bus=ide,bootindex=2,dest=file

DB data model and migration

Currently the data that we keep associated with a Block Device is:

  • id
  • instance_uuid
  • device_name - See the following section.
  • delete_on_termination - can be kept too
  • virtual_name - See the following section.
  • snapshot_id - See the following section
  • volume_id - See the following section
  • volume_size
  • no_device - it was used to override the image supplied block devices. This seems like it is in need of re-engineering but is not high priority.
  • connection_info - json string stored as a result of volume_api.attach

The new fields we will add:

  • image_id - See the following section
  • source_type - See the following section
  • dest_type - See the following section
  • guest_format - See the following section
  • device_type - See the following section
  • disk_bus - See the following section
  • boot_order - See the following section
  • user_label - we may want to add this and expose it in the guest filesystem for convenience

Field details


This field used to keep values like 'ephemeral0' and 'swap' (all other values are currently disregarded, and you can only pass this through image mapping at the moment - cli does not support it). When migrating we will use this field to infer source_type/guest_format but will most likely drop the field itself.


We will likely keep this one, however the migration may need to reassign the value. There is an issue with device name that comes from the fact that not all hypervisors guarantee that they will respect the user defined device name. This in turn means that the device file on the instance and the device_name in the db may or may not match. However, there are places in Nova code where we try to use devices from the db to make decisions (for example when attaching). We should either

  1. Make device name the final decision of the hypervisor and expose a driver method that will assign devices to each block device and update the DB. This will make user assigned values merely suggestions, but will make sure the DB is in sync with reality and make this data safe to use outside of the driver/compute node
  2. Remove this field from the DB and treat it a s an internal implementation detail that can only be used in the virt driver. This might break backwards compatibility.

snapshot_id and volume_id

We will keep these and we will be adding an image_id field as part of this change. This will be tied in with the source_type field which we can maybe omit as it ce What is worth to note here is that a block device with a snapshot_id will result in a volume being created and thus will have both snapshot and volume ids.

New fields:


We seem to agree that this can be one of the following:

  • image
  • volume
  • snapshot
  • blank

The migration will have to add one image block device for each image backed instance. We may want to keep the image_ref in the instance table too. It will also need to change


This is one of:

  • local
  • volume

The migration can set this to volume if volume_id or snapshot_id is defined, and local in every other case, or the code could handle empty dest_type by the same rule.


The semantics of this would be to tell nova how/if to format the image on boot/attach, so we will migrate most to none apart from swap and ephemeral which we can default to CONF.default_ephemeral_format to be consistent


At the moment we can make distinction only between disk and cdrom. So if the instances was started from an image that has the meta 'disk_format' property set to 'iso'. This All others will default to disk.


We can use some sane defaults for the info we currently have - or we can leave empty and allow driver to update the DB


This parameter makes no sense for swap/ephemeral devices, during migration we will this to 1 for either the image if instance was image backed or the first volume/snap. We might wanna make this a nullable field.

API data model and backwards compat issues