Difference between revisions of "Meetings/Storlets"
< Meetings
m |
|||
| Line 1: | Line 1: | ||
Meeting Time: Every Wednesday at 13:00 UTC in #openstack-storlets | Meeting Time: Every Wednesday at 13:00 UTC in #openstack-storlets | ||
| − | + | 'Agenda:' | |
| − | |||
| − | |||
| − | |||
* Austin talk. | * Austin talk. | ||
* Swift Hackathon | * Swift Hackathon | ||
* bugs. | * bugs. | ||
| − | + | ''Austin:'' | |
| − | |||
* We currently have the following abstract | * We currently have the following abstract | ||
** Title: From Analytic to Image Processing in Swift with Openstack Storlets | ** Title: From Analytic to Image Processing in Swift with Openstack Storlets | ||
| Line 21: | Line 17: | ||
| − | '''Swift | + | ''Swift Hackathon'' |
| + | '''Swift Community Session''' | ||
* The purpose of the session is to update the community on what's happening in the openstack-storlets project and get feedback on number of subjects (the intention is not to discuss pushing the middleware to Swift at this point). | * The purpose of the session is to update the community on what's happening in the openstack-storlets project and get feedback on number of subjects (the intention is not to discuss pushing the middleware to Swift at this point). | ||
| − | + | * Updates: | |
| − | + | ** Unit tests (Thanks NTT) | |
| − | + | *** storlets GET flow: https://review.openstack.org/#/c/263533/ | |
| − | + | *** storlets PUT flow: https://review.openstack.org/#/c/264520/ | |
| − | + | *** uploading storlets/dependency: https://review.openstack.org/#/c/264638/ | |
| − | *** Copy semantics (I got a working prototype) | + | ** Refactoring (Thanks Takashi) |
| − | + | ** Copy semantics (I got a working prototype) | |
| − | + | ** Storlets samples (thumbnail extraction, more to come), what else? | |
| − | + | * Points for discussion: | |
| − | + | ** Adding a non voting gate job for testing Storlets | |
| − | ** | + | ** Use footers for adding metadata |
| − | + | ** To-be-fixed in swift | |
| − | + | ||
| − | + | '''Storlets Community Session(s)''' | |
| − | + | * Resource limitation (just ideas) | |
| − | ** | + | ** Limit the number of containers or processes running on one node |
| + | ** Limit resource consumptions in each containers | ||
| + | ** stop unnecessary containers/processes? | ||
| + | * Can we also add system header like Content-Type? | ||
| + | * Documentation | ||
| + | |||
| + | |||
**** chunked transferring support in proxy-server: https://review.openstack.org/#/c/256201/ | **** chunked transferring support in proxy-server: https://review.openstack.org/#/c/256201/ | ||
Revision as of 12:53, 27 January 2016
Meeting Time: Every Wednesday at 13:00 UTC in #openstack-storlets
'Agenda:'
- Austin talk.
- Swift Hackathon
- bugs.
Austin:
- We currently have the following abstract
- Title: From Analytic to Image Processing in Swift with Openstack Storlets
- Openstack Stolets is an emerging Openstack project aimed at pushing down data centric compute jobs to the object store. Storlets offer a server-less programming model enabling developers to quickly develop and deploy data path computations without worrying about the underlying platform details.
- Storlets facilitate several interesting use cases such as:
- Performing image processing in the storage, allowing to process large objects without downloading it.
- Spark analytic push down to the store, saving on both bandwidth and memory in the analytic cluster.
- Data obfuscation and anonymization at the source. Thus, preventing sensitive data from ever leaving the store.
- In this talk we will present the project, discuss and demonstrate the use cases and present a PoC about an effective architecture for image processing using Storlets done at NTT.
Swift Hackathon
Swift Community Session
- The purpose of the session is to update the community on what's happening in the openstack-storlets project and get feedback on number of subjects (the intention is not to discuss pushing the middleware to Swift at this point).
- Updates:
- Unit tests (Thanks NTT)
- storlets GET flow: https://review.openstack.org/#/c/263533/
- storlets PUT flow: https://review.openstack.org/#/c/264520/
- uploading storlets/dependency: https://review.openstack.org/#/c/264638/
- Refactoring (Thanks Takashi)
- Copy semantics (I got a working prototype)
- Storlets samples (thumbnail extraction, more to come), what else?
- Unit tests (Thanks NTT)
- Points for discussion:
- Adding a non voting gate job for testing Storlets
- Use footers for adding metadata
- To-be-fixed in swift
Storlets Community Session(s)
- Resource limitation (just ideas)
- Limit the number of containers or processes running on one node
- Limit resource consumptions in each containers
- stop unnecessary containers/processes?
- Can we also add system header like Content-Type?
- Documentation
- chunked transferring support in proxy-server: https://review.openstack.org/#/c/256201/
Bugs
