Difference between revisions of "Swift for File Systems w/o Extended Attributes"
Tejas Sumant (talk | contribs) m (→Code Changes) |
Tejas Sumant (talk | contribs) m |
||
| Line 26: | Line 26: | ||
Wherever call to read_metadata and write_metadata are made, a check is required to pass right file handle. | Wherever call to read_metadata and write_metadata are made, a check is required to pass right file handle. | ||
If extended attribute are supported then file handle to target file else file handle to shadow file needs to be passed. | If extended attribute are supported then file handle to target file else file handle to shadow file needs to be passed. | ||
| − | + | ==Test Plan== | |
| + | Standard Swift tests should pass with this feature in. | ||
| + | Swift should pass tests on file system with extended attribute support as well as file systems without extended attributes support. | ||
| + | At given time, all file systems underneath should either support extended attribute or should not support extended attributes. | ||
==Demo Plan== | ==Demo Plan== | ||
Demo to be uploaded on Calsoft website (www.calsoftinc.com) | Demo to be uploaded on Calsoft website (www.calsoftinc.com) | ||
Revision as of 13:00, 26 August 2013
Contents
Summary
Today swift assumes that file system underneath supports extended attributes. This restricts use of file systems w/o extended attributes with Swift.
Release Date
Post Havana
Rationale
More the flexibility given to user, the better it would be adoption of Openstack swift. If we can conquer limitations of file systems w/o extended attributes it would be more flexible for end use to choose and select file system.
User Stories
User wants to deploy Openstack swift on EXT2 file system.
Design
Alternative to extended attributes
Swift creates one file in file system per swift object. Each file has set of extended attributes associated with it. To get rid of dependency on extended attributes, it is proposed to have one shadow file per Swift created directory. The shadow file will contain all extended attributes in in key value form for all files in given directory.
When swift shuffles data on various node, it is noticed that rsync gets used. Swift issues Rsync commands on directory and not files. So it is inherently ensured that along with swift files shadow files also move. This action ensures consistency of swift data.
Extended attribute writing to file
Package pickle will be used to manipulate attribute value pairs (extended attributes).
Implementation
Code Changes
read_metadata and write_metadata functions in server.py
These are the key functions that would undergo change. These functions would either invoke system call to get and set extended attributes or call pickle based on configuration. If it was configured at the swift start time that underlying FS supports extended attributes then Swift will use standard system calls. Otherwise all get and set extended attribute calls are converted to read and write from shadow file.
Other changes
Wherever call to read_metadata and write_metadata are made, a check is required to pass right file handle. If extended attribute are supported then file handle to target file else file handle to shadow file needs to be passed.
Test Plan
Standard Swift tests should pass with this feature in. Swift should pass tests on file system with extended attribute support as well as file systems without extended attributes support. At given time, all file systems underneath should either support extended attribute or should not support extended attributes.
Demo Plan
Demo to be uploaded on Calsoft website (www.calsoftinc.com)
