Jump to: navigation, search

Difference between revisions of "NexentaVolumeDriver"

 
(2 intermediate revisions by 2 users not shown)
Line 1: Line 1:
__NOTOC__
+
{{OldDesignPage}}
 
* '''Launchpad Entry''': [https://blueprints.launchpad.net/nova/+spec/nexenta-volume-driver nexenta-volume-driver]
 
* '''Launchpad Entry''': [https://blueprints.launchpad.net/nova/+spec/nexenta-volume-driver nexenta-volume-driver]
 
* '''Created''': 11 Nov 2011
 
* '''Created''': 11 Nov 2011
Line 34: Line 34:
 
To get the volumes stored to the Nexenta appliance, it has to be configured appropriately. Configuration of appliance includes the following steps:
 
To get the volumes stored to the Nexenta appliance, it has to be configured appropriately. Configuration of appliance includes the following steps:
 
* install and configure [[NexentaStor]] software via management console and web interface using initial configuration wizard;
 
* install and configure [[NexentaStor]] software via management console and web interface using initial configuration wizard;
* use management console or web interface to create ZFS volume, for example, ‘nova’;
+
* use management console or web interface to create ZFS volume, for example, ‘nova-volumes’;
 
* specify base volume name in the nova.conf on the Openstack node that runs nova-volume.
 
* specify base volume name in the nova.conf on the Openstack node that runs nova-volume.
 +
 +
==== Example Configuration ====
 +
 +
For example, we have an appliance with 8 disks, two of which are used for system pool 'syspool'. This leaves us 6 disks to create storage pool:
 +
 +
<pre><nowiki>
 +
nmc@nexentastor:/$ show lun disk
 +
LUN ID      Device    Type        Size        Volume    Mounted Attach GUID
 +
c1t0d0      sd0      disk        147GB        syspool    no      mpt    00825b934e690f9000059a03000b
 +
c1t1d0      sd2      disk        500GB                  no      mpt    00c469d24e68c469000b4c830006
 +
c1t2d0      sd3      disk        500GB                  no      mpt    00c469d24e6a0070000e519b0011
 +
c1t3d0      sd4      disk        500GB                  no      mpt    00c469d24e6a053400070c500012
 +
c1t4d0      sd5      disk        500GB                  no      mpt    00c469d24e6f25ab000bb7770008
 +
c1t5d0      sd6      disk        500GB                  no      mpt    00c469d24e6f25ab000bcd6d0009
 +
c1t6d0      sd7      disk        500GB                  no      mpt    00c469d24e6f514500045e94000c
 +
c1t9d0      sd9      disk        147GB        syspool    no      mpt    00c469d24e79b0ca0007a8000004
 +
</nowiki></pre>
 +
 +
 +
We want to create a 'nova-volumes' volume from this disks as 2 three-way mirrors to improve data protection. The pool can be created by the following command:
 +
 +
<pre><nowiki>
 +
nmc@nexentastor:/$ create volume nova-volumes mirror c1t1d0 c1t2d0 c1t3d0 mirror c1t4d0 c1t5d0 c1t6d0 -y
 +
</nowiki></pre>
 +
 +
This will non-interactively create volume 'nova-volumes'. We can see its properties with the command:
 +
 +
<pre><nowiki>
 +
nmc@nexentastor:/$ show volume nova status
 +
volume: nova
 +
state: ONLINE
 +
scan: none requested
 +
config:
 +
 +
        NAME        STATE    READ WRITE CKSUM
 +
        nova        ONLINE      0    0    0
 +
          mirror-0  ONLINE      0    0    0
 +
            c1t1d0  ONLINE      0    0    0
 +
            c1t2d0  ONLINE      0    0    0
 +
            c1t3d0  ONLINE      0    0    0
 +
          mirror-1  ONLINE      0    0    0
 +
            c1t4d0  ONLINE      0    0    0
 +
            c1t5d0  ONLINE      0    0    0
 +
            c1t6d0  ONLINE      0    0    0
 +
 +
errors: No known data errors
 +
</nowiki></pre>
 +
  
 
=== [[OpenStack]] Nova configuration ===
 
=== [[OpenStack]] Nova configuration ===
Line 98: Line 146:
 
== Test/Demo Plan ==
 
== Test/Demo Plan ==
  
This need not be added or completed until the specification is nearing beta.
+
You can see the demonstration of the function of this driver in this video:
 +
http://www.youtube.com/watch?v=q76OONdWo8I
  
 
== Unresolved issues ==
 
== Unresolved issues ==

Latest revision as of 18:03, 15 May 2014

Warning.svg Old Design Page

This page was used to help design a feature for a previous release of OpenStack. It may or may not have been implemented. As a result, this page is unlikely to be updated and could contain outdated information. It was last updated on 2014-05-15

Summary

We would like to introduce the NexentaStor support for the nova-volume service. This blueprint proposes the driver based on the iSCSI driver already integrated into the OpenStack nova-volume service, with the basic capabilities to create, export and delete volumes and snapshots.

The driver can interact with the NexentaStor Appliance using its REST API, or using DBus API. The default API is REST, and the DBus requires patched libdbus to function properly.

The proposal is to add NexentaStor driver to the nova-volume service.

Release Note

NexentaStor Appliance is NAS/SAN software platform designed for building reliable and fast network storage arrays. The NexentaStor is based on the OpenSolaris and uses ZFS as a disk management system. NexentaStor can serve as a storage node for the OpenStack and provide block-level volumes for the virtual servers via iSCSI protocol.

Requirements

  • Openstack nova-volume service installed
  • NexentaStor Appliance installed and configured
  • Dedicated ‘volume’ object created on appliance

Rationale

NexentaStor Appliance provides reliable block-level volumes which are easy to manage via simple REST interface. It automates many common storage administration tasks. It is intended to be used with commodity hardware and basically is vendor-independent. This makes NexentaStor Appliance software useful as a base for storage nodes in IaaS clouds.

User stories

User installs nova-volume service on the Openstack node and configures volume_driver, specifies communication transport and IP address of the NexentaStor appliance. Additional parameters for volumes can be specified in the flag file, i.e. block size. User can then create volumes with standard Nova API calls.

Nexenta Appliance Configuration

To get the volumes stored to the Nexenta appliance, it has to be configured appropriately. Configuration of appliance includes the following steps:

  • install and configure NexentaStor software via management console and web interface using initial configuration wizard;
  • use management console or web interface to create ZFS volume, for example, ‘nova-volumes’;
  • specify base volume name in the nova.conf on the Openstack node that runs nova-volume.

Example Configuration

For example, we have an appliance with 8 disks, two of which are used for system pool 'syspool'. This leaves us 6 disks to create storage pool:

nmc@nexentastor:/$ show lun disk
LUN ID      Device    Type         Size         Volume     Mounted Attach GUID
c1t0d0      sd0       disk         147GB        syspool    no      mpt    00825b934e690f9000059a03000b
c1t1d0      sd2       disk         500GB                   no      mpt    00c469d24e68c469000b4c830006
c1t2d0      sd3       disk         500GB                   no      mpt    00c469d24e6a0070000e519b0011
c1t3d0      sd4       disk         500GB                   no      mpt    00c469d24e6a053400070c500012
c1t4d0      sd5       disk         500GB                   no      mpt    00c469d24e6f25ab000bb7770008
c1t5d0      sd6       disk         500GB                   no      mpt    00c469d24e6f25ab000bcd6d0009
c1t6d0      sd7       disk         500GB                   no      mpt    00c469d24e6f514500045e94000c
c1t9d0      sd9       disk         147GB        syspool    no      mpt    00c469d24e79b0ca0007a8000004


We want to create a 'nova-volumes' volume from this disks as 2 three-way mirrors to improve data protection. The pool can be created by the following command:

nmc@nexentastor:/$ create volume nova-volumes mirror c1t1d0 c1t2d0 c1t3d0 mirror c1t4d0 c1t5d0 c1t6d0 -y

This will non-interactively create volume 'nova-volumes'. We can see its properties with the command:

nmc@nexentastor:/$ show volume nova status
volume: nova
 state: ONLINE
 scan: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        nova        ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            c1t1d0  ONLINE       0     0     0
            c1t2d0  ONLINE       0     0     0
            c1t3d0  ONLINE       0     0     0
          mirror-1  ONLINE       0     0     0
            c1t4d0  ONLINE       0     0     0
            c1t5d0  ONLINE       0     0     0
            c1t6d0  ONLINE       0     0     0

errors: No known data errors


OpenStack Nova configuration

You can set up the driver by setting following flags in nova.conf:

  • volume_driver - should be set to nexenta.volume.NexentaDriver;
  • nouse_local_volumes - this is necessary for all SAN drivers;
  • nexenta_host - should be hostname or IP address of Nexenta appliance;
  • nexenta_volume - should be set to the name of ZFS volume that should contain all Nova volumes;
  • nexenta_user - name of user at Nexenta appliance with REST API access (e.g. admin);
  • nexenta_password - password for that user.

Example config following:

--volume_driver=nexenta.volume.NexentaDriver
--nouse_local_volumes
--nexenta_host=172.16.1.10
--nexenta_volume=nova
--nexenta_user=admin
--nexenta_password=secrete


User Experience

The NexentaStor appliance using SAS disk drives and connected to the Compute node by 1Gbps Ethernet provides following input and output rates:

  • up to 60 Mbytes per second on sequential writes
  • up to 132 Mbytes per second on random reads
  • up to 1000 volumes per appliance tested

Assumptions

This driver assumes user has installed NexentaStor software on at least one appliance and configured it to have a logical volume dedicated to Openstack volumes.

Design

Driver extends iSCSI driver in the same way as all SAN drivers. All operations are performed using one of two transport modules: either D-Bus or JSON RPC transport supported.

Volumes Back-End

ZFS zvols are used as base for exported volumes. Zvol is a ZFS entity that essentialy is virtual block device. Its flexible configuration allows block sizes from 512 bytes to 128 Kbytes, and use of advanced ZFS specific features like built-in integrity control, deduplication and caching. Zvols can be sparse and consume only so much disk space as the size of actually written data.

The driver supports the following features of underlying ZFS zvols:

  • block size can be specified in bytes or Kbytes with --nexenta_blocksize flag;
  • use of sparse volumes can be configured with --nexenta_sparse flag;

Volumes Export

Common iSCSI initiator-target scheme is used for export. Each volume is exported through dedicated target which is placed in separate target group to ensure correct iSCSI LUN numbering.

Implementation

Driver functionality is separated into three modules. Two modules implement transports: nexenta.jsonrpc for JSON RPC and nexenta.NZA for D-Bus. Latter requires patched libdbus installed in the system. The nexenta.volume module contains the driver code.

Minor exception handling issue was fixed in the main nova code to enhance the behavior of the driver.

Code Changes

Nexenta driver code should be included as package under nova.volume package.

Test/Demo Plan

You can see the demonstration of the function of this driver in this video: http://www.youtube.com/watch?v=q76OONdWo8I

Unresolved issues

  • Currently one cannot delete a snapshot if there is a volume created from it due to Nexenta limitations.

External links