How to deploy cinder with GlusterFS

Scope

This document will guide trough the basic steps needed to configure the GlusterFS driver (backend) in Cinder.

Setup

Ensure the glusterfs-fuse client is installed on the nodes running Cinder (the cinder-volume service) and the nodes running Nova.

It is not in the scope of this document to describe the options available when creating a new GlusterFS volume but assuming you just want a replica and have two GlusterFS storage nodes available, the following will do:

# gluster vol create cindervol1 replica 2 xx.yy.zz.aa:/path/to/brick1/cindervol1 xx.yy.zz.bb:/path/to/brick1/cindervol1
# gluster vol start cindervol1

Configuration

The GlusterFS backend driver is a particular implementation of the NFS driver, they have similar configuration options indeed. The basic changes needed for it to work are:

volume_driver=cinder.volume.drivers.glusterfs.GlusterfsDriver
glusterfs_shares_config=/etc/cinder/shares.conf

Populate the shares file by adding there the mount path to the GlusterFS volume:

xx.yy.zz.aa:/cindervol1 -o backupvolfile-server=xx.yy.zz.bb

Later restart the Cinder services:

# for i in api scheduler volume; do service openstack-cinder-${i} restart; done

Testing

# source ~/.keystonerc_admin
# cinder create --display-name test 2
# cinder list

After the volume appears to be in available state, you should also see a corresponding raw file on the remote GlusterFS share (which is now mounted on the Cinder node):

# ls -lah /var/lib/cinder/mnt/a3082d8779198266d36d261ff40ac12b/
total 12K
drwxr-xr-x. 3 root   root   4.0K Aug  5 12:12 .
drwxr-xr-x. 3 cinder cinder 4.0K Aug  3 03:42 ..
-rw-rw-rw-. 1 root   root   2.0G Aug  5 12:12 volume-05abeca8-9296-4493-8ea0-3e15a475dcaa