Jump to: navigation, search


Revision as of 20:50, 21 April 2015 by Robert Esker (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
  • Launchpad Entry: NovaSpec:xenapi-sm-support
  • Created: 1 February 2011
  • Contributors: Renuka Apte, Armando Migliaccio


Design and implement a new driver for Nova-Volume, based on XenAPI Storage Manager. This will provide basic storage functionality (like volume creation, and destruction) on a number of different storage back-ends, but it will enable the capability of using more sophisticated storage back-ends for operations like cloning/snapshotting etc. To have an idea of the benefits of using XenAPI SM to provide back-end storage services, the list below shows some of the storage plugins already supported in XenServer/XCP:

  • NFS VHD: SR plugin which stores disks as VHD files on a remote NFS filesystem
  • Local VHD on LVM: SR plugin which represents disks as VHD disks on Logical Volumes within a locally-attached Volume Group
  • HBA LUN-per-VDI driver: SR plugin which represents LUNs as VDIs sourced by hardware HBA adapters, e.g. hardware-based iSCSI or FC support
  • NetApp: SR driver for mapping of LUNs to VDIs on a NETAPP server, providing use of fast snapshot and clone features on the filer
  • LVHD over FC: SR plugin which represents disks as VHDs on Logical Volumes within a Volume Group created on an HBA LUN, e.g. hardware-based iSCSI or FC support
  • iSCSI: Base ISCSI SR driver, provides a LUN-per-VDI. Does not support creation of VDIs but accesses existing LUNs on a target.
  • LVHD over iSCSI: SR plugin which represents disks as Logical Volumes within a Volume Group created on an iSCSI LUN
  • EqualLogic: SR driver for mapping of LUNs to VDIs on a EQUALLOGIC array group, providing use of fast snapshot and clone features on the array


  • XenServer: Commercial, supported product from Citrix
  • Xen Cloud Platform (XCP): Open-source equivalent of XenServer (and the development project for the toolstack). Everything said about XenServer below applies equally to XCP
  • XenAPI: The management API exposed by XenServer and XCP
  • xapi: The primary daemon on XenServer and Xen Cloud Platform; the one that exposes the XenAPI

Release Note

No direct impact on user consuming the external HTTP API services. This blueprint enables the use of several storage back-ends by writing a single nova-volume driver, rather than several ones, as many as the specific storage back-ends required to be supported are needed. In due course, leveraging capabilities of specific back-ends, will enable operations like volume snapshotting, cloning, etc. that can be exposed via official API.


This new driver will sit alongside drivers already developed like AoE, iSCSI and SAN (both running on commodity boxes running Linux). However, the latter is slightly different as it will still run on commodity hardware, but it access remote controllers on SAN hardware by some protocol like SSH, CLI or custom API. This is clearly undesirable because every time there is a need to support new hardware, a new driver must be designed and developed. The adoption of XenAPI SM will enable a number of storage back-end that are abstracted by a common API. Moreover, this becomes premium value to customers of service providers and may become differentiator for enterprise private clouds.

User stories

The system administrator of a medium sized company deploys a private cloud based on OpenStack. He had previously bought and deployed a NetApp filer to provide back-end storage to employees' virtual machines. He now uses XenAPISM volume driver to provide block storage to VMs created using the OpenStack private cloud.

A service provider wants to deliver premium storage to its customers. Storage back-ends like NetApp or EqualLogic can be easily supported using XenAPISM volume driver.


For the design and implementation of the new Nova-Volume driver, XenAPI for XenServer/XCP is going to be used.



  • Backend: A term for a particular storage backend. This could be iSCSI, NFS, Netapp etc.
  • Backend-config: All the parameters required to connect to a specific backend. For e.g. For NFS, this would be the server, path, etc.
  • Flavor: A user friendly term to specify some notion of quality of service. For example, "gold" might mean that the volumes will use a backend where backups are possible.

The concept of flavor has been introduced in keeping with the LUNR work. No API changes have been introduced to allow for creating a volume of a particular flavor. This will be deferred until the LUNR code is available.

A flavor can be associated with multiple backends. Some kind of policy will decide which backend will be used to create a volume of a particular flavor. Until we have a way of selecting flavor, we will use a simple "first-fit" policy, where the first backend that can successfully create this volume is the one that is used.

The entity relationship diagram is given below:



Using the nova-manage command detailed in the implementation, an admin can add flavors and backends.

One or more nova-volume service instances will be deployed per availability zone. When an instance is started, it will create storage repositories (SRs) to connect to the backends available within that zone. All nova-volume instances within a zone can see all the available backends. These instances are completely symmetric and hence should be able to service any create_volume request within the zone.



A category called “sm” has been added to nova-manage in the class StorageManagerCommands

The following actions will be added:

  1. flavor_list
  2. flavor_create
  3. flavor_delete
  4. backend_list
  5. backend_add
  6. backend_remove


nova-manage sm flavor_create <label> <description>

nova-manage sm flavor_delete<label>

nova-manage sm backend_add <flavor label> <SR type> [config connection parameters]

Note: SR type and config connection parameters are in keeping with the Xen Command Line Interface. http://support.citrix.com/article/CTX124887

nova-manage sm backend_delete <backend-id>


nova-manage sm flavor_create gold "Not all that glitters"

nova-manage sm flavor_delete gold

nova-manage sm backend_add gold nfs name_label=toybox-renuka server=myserver serverpath=/local/scratch/myname

nova-manage sm backend_remove 1

API Changes

As mentioned above, the API changes required to create/delete a volume of a particular flavor will be deferred till the LUNR code is available. It is therefore not possible to create a volume for a particular flavor yet, although the code/design exists.

The existing euca-create-volume and euca-delete-volume commands should be used.

New Tables

SM Flavor:

Field Type Null Key Default
created_at datetime YES NULL
updated_at datetime YES NULL
deleted_at datetime YES NULL
deleted tinyint(1) YES NULL
id int(11) NO PRI NULL
label varchar(255) YES NULL
description varchar(255) YES NULL


Field Type Null Key Default
created_at datetime YES NULL
updated_at datetime YES NULL
deleted_at datetime YES NULL
deleted tinyint(1) YES NULL
id int(11) NO PRI NULL
flavor_id int(11) NO MUL NULL
sr_uuid varchar(255) YES NULL
sr_type varchar(255) YES NULL
config_params varchar(2047) YES NULL

SM Volume:

Field Type Null Key Default
created_at datetime YES NULL
updated_at datetime YES NULL
deleted_at datetime YES NULL
deleted tinyint(1) YES NULL
id int(11) NO PRI NULL
backend_id int(11) NO MUL NULL
vdi_uuid varchar(255) YES NULL

Code Changes

A new volume driver called XenSMDriver has been added to nova/volume. It leverages the xenapi storage manager to control the available backends. Functions have been exposed to create/forget storage repositories and connect to them.

A do_setup function is added to volume driver in order to perform initialization (which includes creating backend repositories) when the service is started.

Test/Demo Plan

Unit tests will be provided as code is developed.

Unresolved issues

None at the time of writing

BoF agenda and discussion

Related topics:

  • Having VM's root devices running on block storage
  • Thin provisioning
  • Fast cloning
  • Advanced volume API (Volume as a Service)
  • Fibre channel and StorageLink
  • Dedup