Mellanox-Cinder

Overview

iSER (iSCSI over RDMA) Mellanox OpenStack support to Cinder.
This can allow 5x faster bandwidth compared to using iSCSI TCP.

For example, over RAM device LUN I got ~1.3GBps Vs. ~5.5GBps (TCP Vs. iSER), and much lower CPU overhead.

Havana Release

Note: tgt target needs to be 1.0.39 and up, if you use RDO installation you should check tgt version. tgt RPM can be found in http://www.mellanox.com/downloads/solutions/tgt/

Installation

The Mellanox Cinder pacakge is embeded within the OpenStack Cinder, no patches or plugins are needed. Just follow the formal OpenStack Cinder installation

Configuration

In order to enable iSER, need to adjust these values at /etc/cinder/cinder.conf:

iser_ip_address = <ipoib/roce_address>
volume_driver = cinder.volume.drivers.lvm.LVMISERDriver

iser_ip_address is required to do a "discovery" over the IB/RoCE interface from the initiator side.

volume_driver points cinder to use the ISERDriver, instead the LVMISCSIDriver.

On nova-compute side change the following in /etc/nova/nova.conf:

libvirt_volume_drivers = iser=nova.virt.libvirt.volume.LibvirtISERVolumeDriver

Restart Cinder and Nova compute services

#/etc/init.d/openstack-cinder-volume restart
#/etc/init.d/openstack-nova-compute restart

Grizzly Release

Installation

Getting the code

wget https://github.com/mellanox-openstack/cinder-iser/archive/stable/grizzly.zip
cd cinder-iser-stable-grizzly

Applying the patches

You have two choices:

1. To apply this support, replace the files under "cinder/cinder/" And "nova/nova/" Respectively.
2. Apply the patches under "cinder/" And "nova/", don't forget to copy "cinder/cinder/volume/iser.py" if you choose this way.

Configuration

In order to enable iSER, need to adjust these values at /etc/cinder/cinder.conf:

iser_ip_address = <ipoib/roce_address>
volume_driver = cinder.volume.drivers.lvm.LVMISERDriver

iser_ip_address is required to do a "discovery" over the IB/RoCE interface from the initiator side.

volume_driver points cinder to use the ISERDriver, instead the LVMISCSIDriver.

On nova-compute side change the following in /etc/nova/nova.conf:

libvirt_volume_drivers = iser=nova.virt.libvirt.volume.LibvirtISERVolumeDriver

Restart Cinder and Nova compute services

#/etc/init.d/openstack-cinder-volume restart
#/etc/init.d/openstack-nova-compute restart

References

1. OpenStack solution page at Mellanox site

2. Source repository

3. Mellanox OFED web page

4. Cinder-controller

5. Cinder-node

6. Cinder-install

Return to Mellanox-OpenStack wiki page.

For more details, please refer any inquiries to openstack@mellanox.com.