Swift/ideas/third-party-transfers

Problem

A user who wants to transfer big amounts of data between two different Swift servers needs to download all the objects with this client, and re-upload them to the remote site. This is potentially slow (based on the client network bandwidth) and requires the client to be online for all the time of the transfer.

Proposed solution

A swift client should be able to initiate a 'third party transfers' between two different Swift installations. This would allow to transfer big amounts of data between Swift installations without having the user to download and re-upload the objects from his client. The third party transfer operation should be unattended. There is something similar called container sync https://docs.openstack.org/swift/latest/overview_container_sync.html but this duplicates all the data, which is not the wanted behavior. This should be similar to what can be done with other tools such as GridFTP.

Contact: mpasserini on IRC

[tdasilva] mpasserini, I haven't had a chance to catch you on IRC, but the 1space project might be helfpul to you: https://github.com/swiftstack/1space.

[mpasserini] tdasilva, thanks for the link! This seems to only work for Swift>S3 but not Swift>Swift.

[tdasilva] mpasserini. It works Swift > Swift too, but thanks for the feedback, we probably need to make that more clear. If you need help with testing it out you can find me in the Swift channel on IRC.