Revision as of 20:24, 6 March 2014

New LVM-based driver for shared storage in Cinder

Related blueprints

• https://blueprints.launchpad.net/cinder/+spec/lvm-driver-for-shared-storage
• https://blueprints.launchpad.net/nova/+spec/lvm-driver-for-shared-storage

Goals

• The goal of this blue print is to implement a new LVM-based driver which supports shared block device on backend storage via Fibre Channel or iSCSI.
• This will improve I/O bandwidth and response of guest VM compared to standard LVM driver.
• Use a backend storage directly without software iSCSI target.

Overview

At the standard LVM driver, Cinder creates volumes from volume group and Cinder provides these volumes using software iSCSI target and Compute node recognizes these volumes using software iSCSI initiator remotely. In this method, Compute node is not necessary to be connected directly to backend storage which includes the volume group. Therefore cinder can provide volumes to Compute node easily. However, user can’t use a storage which is connected via SAN(Fibre Channel or iSCSI storage) directly by this driver because this LVM driver presupposes virtual storage using iSCSI targetd(or LIO) and this driver does not support a storage via SAN.

In contrast, the new LVM-based shared storage driver supports backend storages which is connected via SAN(Fibre Channel or iSCSI) directly. As a result, this driver will improve I/O bandwidth and response of guest VM compared to standard LVM driver.

(a) Standard iSCSI LVM driver

(b) LVM-based driver for shared storage

Benefits of this driver

1. Improve I/O bandwidth and response of guest VM by using LVM on backend storage directly
2. Less traffic of data transfer between Cinder Node and Compute nodes
3. Basic features of Cinder such as snapshot and backup etc. are appropriable from standard cinder LVM driver.

Basic Design

• This driver uses shared block storages between Cinder node and compute nodes via iSCSI or Fibre Channel.
• To create Volume Group or Logical Volume on a LVM of shared storage, multiple servers can refer Volume Group or Logical Volume.
• The driver can attach LV which is created from shared LVM to a virtual machine as a volume same as traditional LVM driver.
• LVM holds management region including metadata of LVM configuration information. If multiple servers update the metadata at the same time, metadata will be broken. Therefore, it is necessary that operation of updating metadata can permit Cinder node.
• The operations of updating metadata are followings.

1. Volume create
2. Volume delete
3. Volume extend
4. Snapshot create
5. Snapshot delete

• On the other hands, the operations without updating metadata are followings. These operations are permitted every compute node.

1. Volume attach
2. Volume detach

Prerequisites

• Use QEMU/KVM as a hypervisor (via libvirt compute driver)
• Shared block storages between Cinder node and compute nodes via iSCSI or Fibre Channel
• A volume group on the shared block storage
• Disable lvmetad

When compute node attaches created volume to a virtual machine, latest LVM metadata is necessary. However the lvmetad caches LVM metadata and this prevent to obtain latest LVM metadata.

Configuration

In order to enable Shared LVM driver, need to define theses values at /etc/cinder/cinder.conf

Example

[LVM_shared]
volume_group=cinder-volumes-shared
volume_driver=cinder.volume.drivers.lvm.LVMSharedDriver
volume_backend_name=LVM_shared