Jump to: navigation, search

Swift for File Systems w/o Extended Attributes

Summary

Today, Swift assumes that the file system underneath supports extended attributes. This restricts the use of file systems without extended attributes with Swift.

Release Date

Post Havana

Rationale

If we can conquer the limitations of file systems without extended attributes, that would add flexibility for end users to choose their file system(s) and thereby increase adoption of Swift and Openstack.

User Stories

User wants to deploy Openstack swift on EXT2 file system.

Design

Alternative to extended attributes

Swift creates one file in file system per Swift object. Each file has a set of extended attributes associated with it. To remove dependency on extended attributes, it is proposed to have one shadow file per Swift created directory. The shadow file will contain all extended attributes in key value form for all the files in a given directory.

When Swift shuffles data on various nodes, rsync is used. Swift issues rsync commands on the directory and not files. So it is inherently ensured, along with Swift files, the shadow file also moves. This change ensures consistency of Swift data.

Extended attribute writing to file

The pickle package would be used to manipulate attribute value pairs (extended attributes).

Implementation

Code Changes

read_metadata and write_metadata functions in server.py

These are the key functions to change, to either invoke a system call to get and set extended attributes, or call pickle based on configuration. Swift will use standard system calls, if it was known when it started that the underlying filesystem supports extended attributes. Otherwise, all get and set extended attribute calls are converted to read and write from the shadow file.

Other changes

Wherever call to read_metadata and write_metadata are made, a check is required to pass the correct file handle. If extended attributes are supported, then file handle uses the target file, else file handle to shadow file needs to be passed.

Test Plan

Standard Swift tests should pass with this feature in. Swift should pass tests on file system with extended attribute support as well as file systems without extended attributes support. At given time, all file systems underneath should either support extended attribute or should not support extended attributes.

Demo Plan

Demo to be uploaded on Calsoft website (www.calsoftinc.com)