Revision as of 21:24, 6 June 2011

OpenStack Cloud Storage (Gluster)

We propose to create a unified cloud storage environment consisting of multi-protocol POSIX compatible elastic file storage, SWIFT-compatible object storage, VM storage, HDFS-compatible big data storage, and continuous data replication. The system will be based upon GlusterFS.

ELASTIC FILE AND OBJECT STORAGE

For application data, POSIX-compatible elastic file storage enables enterprise applications to seamlessly migrate into the cloud without requiring application re-writes. It is much easier to treat unstructured data as files and folders, thereby creating the platform for OpenStack to eliminate barriers to adoption. The proposed solution will be compatible with the SWIFT APIs, allowing files to be treated as objects, objects to be treated as files, folders as tags, and buckets as volumes.

VM STORAGE

For VM images, the proposed solution will bring VMware's vMotion like capabilities to OpenStack. Unlike VMWare's VMFS, GlusterFS can scale to petabytes and is easy to configure. GlusterFS is implemented as a pure software-based storage stack that does not require expensive SAN hardware, instead leveraging the enterprise's choice of DAS/JBOD/SAN. This allows OpenStack users and developers to build amazon-like cloud storage (both S3 and file) easily, seamlessly and cost-effectively. OpenStack Glance compatibility will be maintained.

BIG DATA STORAGE

The proposed solution's HDFS-compatible big data storage enables the Apache-Hadoop stack and all of the big data applications to be instantly compatible with OpenStack without any modifications. Unlike HDFS, GlusterFS allows you to run Mapreduce jobs without the metadata scalability and block size limitation inherent in HDFS, thereby giving OpenStack true big data support.

CONTINUOUS DATA REPLICATION

The system provides continuous geographic replication combined with the unified cloud storage environment, thus enabling OpenStack to unify private and public clouds.

CURRENT STATE OF DEVELOPMENT

As noted above, the solution will be based on GlusterFS which already delivers most of the functionality described above. However, this proposed project will include specific integrations and optimizations for OpenStack. The code for elastic file storage for VM and application data and geo replication functionality is available today. The code for unified file object will be available for review by mid-July. The code for big data storage will be available in late August.

LICENSE

Contributed code will be compatible with OpenStack license.

CONTACT US

openstack@gluster.com