Difference between revisions of "Meetings/Storlets"
< Meetings
(Add some items about bugs) |
|||
| Line 19: | Line 19: | ||
*** Data obfuscation and anonymization at the source. Thus, preventing sensitive data from ever leaving the store. | *** Data obfuscation and anonymization at the source. Thus, preventing sensitive data from ever leaving the store. | ||
** In this talk we will present the project, discuss and demonstrate the use cases and present a PoC about an effective architecture for image processing using Storlets done at NTT. | ** In this talk we will present the project, discuss and demonstrate the use cases and present a PoC about an effective architecture for image processing using Storlets done at NTT. | ||
| + | |||
'''Swift Hackathon''' | '''Swift Hackathon''' | ||
| Line 38: | Line 39: | ||
*** To-be-fixed thing in swift | *** To-be-fixed thing in swift | ||
**** chunked transferring stuff: https://review.openstack.org/#/c/256201/ | **** chunked transferring stuff: https://review.openstack.org/#/c/256201/ | ||
| + | |||
| + | |||
| + | '''Bugs"'' | ||
| + | * Range handling | ||
| + | ** https://bugs.launchpad.net/storlets/+bug/1534414 | ||
| + | ** https://bugs.launchpad.net/storlets/+bug/1534455 | ||
| + | * FD leak | ||
| + | ** https://bugs.launchpad.net/storlets/+bug/1537982 | ||
Revision as of 09:42, 27 January 2016
Meeting Time: Every Wednesday at 13:00 UTC in #openstack-storlets
The meeting on Wed January 27th may start at 12 UTC. Please see the IRC for final schedule.
Agenda:
- Austing talk.
- Swift Hackathon
- bugs.
Austin:
- We currently have the following abstract
- Title: From Analytic to Image Processing in Swift with Openstack Storlets
- Openstack Stolets is an emerging Openstack project aimed at pushing down data centric compute jobs to the object store. Storlets offer a server-less programming model enabling developers to quickly develop and deploy data path computations without worrying about the underlying platform details.
- Storlets facilitate several interesting use cases such as:
- Performing image processing in the storage, allowing to process large objects without downloading it.
- Spark analytic push down to the store, saving on both bandwidth and memory in the analytic cluster.
- Data obfuscation and anonymization at the source. Thus, preventing sensitive data from ever leaving the store.
- In this talk we will present the project, discuss and demonstrate the use cases and present a PoC about an effective architecture for image processing using Storlets done at NTT.
Swift Hackathon
- The purpose of the session is to update the community on what's happening in the openstack-storlets project and get feedback on number of subjects (the intention is not to discuss pushing the middleware to Swift at this point).
- Updates:
- Unit tests (Thanks NTT)
- storlets GET flow: https://review.openstack.org/#/c/263533/
- storlets PUT flow: https://review.openstack.org/#/c/264520/
- uploading storlets/dependency: https://review.openstack.org/#/c/264638/
- Copy semantics (I got a working prototype)
- Storlets samples (thumbnail extraction, more to come), what else?
- Unit tests (Thanks NTT)
- Points for discussion:
- Adding a non voting gate job for testing Storlets
- Use footers for adding metadata
- How to pass header information like Content-Type to storlet applications
- Resource limitation
- Should we the max number about containers or processes running on one node?
- Can we stop unnecessary containers/processes?
- To-be-fixed thing in swift
- chunked transferring stuff: https://review.openstack.org/#/c/256201/
- Updates:
'Bugs"
