Difference between revisions of "Swift for File Systems w/o Extended Attributes"
m (K7AAY moved page Swift for File Systems w/o Exteneded Attributes to Swift for File Systems w/o Extended Attributes: Fix spelling error) |
m (Spelling and grammar fixes) |
||
Line 10: | Line 10: | ||
==Design== | ==Design== | ||
===Alternative to extended attributes=== | ===Alternative to extended attributes=== | ||
− | Swift creates one file in file system per | + | Swift creates one file in file system per Swift object. Each file has a set of extended attributes associated with it. To remove dependency on extended attributes, it is proposed to have one shadow file per Swift created directory. The shadow file will contain all extended attributes in key value form for all the files in a given directory. |
− | When | + | When Swift shuffles data on various nodes, rsync is used. Swift issues rsync commands on the directory and not files. So it is inherently ensured, along with Swift files, the shadow file also moves. This change ensures consistency of Swift data. |
===Extended attribute writing to file=== | ===Extended attribute writing to file=== |
Revision as of 19:45, 4 November 2013
Contents
Summary
Today, Swift assumes that the file system underneath supports extended attributes. This restricts the use of file systems without extended attributes with Swift.
Release Date
Post Havana
Rationale
If we can conquer the limitations of file systems without extended attributes, that would add flexibility for end users to choose their file system(s) and thereby increase adoption of Swift and Openstack.
User Stories
User wants to deploy Openstack swift on EXT2 file system.
Design
Alternative to extended attributes
Swift creates one file in file system per Swift object. Each file has a set of extended attributes associated with it. To remove dependency on extended attributes, it is proposed to have one shadow file per Swift created directory. The shadow file will contain all extended attributes in key value form for all the files in a given directory.
When Swift shuffles data on various nodes, rsync is used. Swift issues rsync commands on the directory and not files. So it is inherently ensured, along with Swift files, the shadow file also moves. This change ensures consistency of Swift data.
Extended attribute writing to file
Package pickle will be used to manipulate attribute value pairs (extended attributes).
Implementation
Code Changes
read_metadata and write_metadata functions in server.py
These are the key functions that would undergo change. These functions would either invoke system call to get and set extended attributes or call pickle based on configuration. If it was configured at the swift start time that underlying FS supports extended attributes then Swift will use standard system calls. Otherwise all get and set extended attribute calls are converted to read and write from shadow file.
Other changes
Wherever call to read_metadata and write_metadata are made, a check is required to pass right file handle. If extended attribute are supported then file handle to target file else file handle to shadow file needs to be passed.
Test Plan
Standard Swift tests should pass with this feature in. Swift should pass tests on file system with extended attribute support as well as file systems without extended attributes support. At given time, all file systems underneath should either support extended attribute or should not support extended attributes.
Demo Plan
Demo to be uploaded on Calsoft website (www.calsoftinc.com)