Difference between revisions of "Meetings/Storlets"
< Meetings
(→Agenda:) |
(→Agenda:) |
||
| Line 6: | Line 6: | ||
** https://review.openstack.org/#/c/276545/ (Improve timeout handling) | ** https://review.openstack.org/#/c/276545/ (Improve timeout handling) | ||
** https://review.openstack.org/#/c/282135/ (Refactor unit tests about storlet docker gateway) | ** https://review.openstack.org/#/c/282135/ (Refactor unit tests about storlet docker gateway) | ||
| − | ** https://review.openstack.org/#/c/280570/ (waiting 282135) | + | ** https://review.openstack.org/#/c/280570/ (already has +2/+A, but waiting 282135) |
| + | ** https://review.openstack.org/#/c/282931/ (Prohibit storlet execution with range header) | ||
| + | ** https://review.openstack.org/#/c/283423/ (Add X-Storlet-Range header to specify input range) | ||
** https://review.openstack.org/#/c/285947/ (Refactor directory structure about python modules): Should be fixed | ** https://review.openstack.org/#/c/285947/ (Refactor directory structure about python modules): Should be fixed | ||
*** We should decide when we deal with directory refactoring, to avoid many merge conflicts caused by that change. [takashi] | *** We should decide when we deal with directory refactoring, to avoid many merge conflicts caused by that change. [takashi] | ||
Revision as of 07:12, 7 March 2016
Meeting Time: Every Wednesday at 13:00 UTC in #openstack-storlets
Contents
Agenda:
- Prioritize patches (suggestion):
- https://review.openstack.org/#/c/283486/ (COPY functionality)
- https://review.openstack.org/#/c/276545/ (Improve timeout handling)
- https://review.openstack.org/#/c/282135/ (Refactor unit tests about storlet docker gateway)
- https://review.openstack.org/#/c/280570/ (already has +2/+A, but waiting 282135)
- https://review.openstack.org/#/c/282931/ (Prohibit storlet execution with range header)
- https://review.openstack.org/#/c/283423/ (Add X-Storlet-Range header to specify input range)
- https://review.openstack.org/#/c/285947/ (Refactor directory structure about python modules): Should be fixed
- We should decide when we deal with directory refactoring, to avoid many merge conflicts caused by that change. [takashi]
- Storlet in Hackathon etherpad
- I would like to add another analytics related question.
- What timeframe do we think we need?
Austin:
- We currently have the following abstract
- Title: From Analytic to Image Processing in Swift with Openstack Storlets
- Openstack Stolets is an emerging Openstack project aimed at pushing down data centric compute jobs to the object store. Storlets offer a server-less programming model enabling developers to quickly develop and deploy data path computations without worrying about the underlying platform details.
- Storlets facilitate several interesting use cases such as:
- Performing image processing in the storage, allowing to process large objects without downloading it.
- Spark analytic push down to the store, saving on both bandwidth and memory in the analytic cluster.
- Data obfuscation and anonymization at the source. Thus, preventing sensitive data from ever leaving the store.
- In this talk we will present the project, discuss and demonstrate the use cases and present a PoC about an effective architecture for image processing using Storlets done at NTT.
Swift Hackathon
- See the following etherpad
Bugs
- Range handling: on going
- FD leak
- FD leak in Storlet GET: https://bugs.launchpad.net/storlets/+bug/1537982 (Fix Released)
- https://review.openstack.org/#/c/272379/ ... merged
- There are other possible FD leaks, which may happen in timeout/exception case.
- Use context manager to make sure all FDs are closed: https://review.openstack.org/#/c/282211/ (patch should be fixed)
- FD leak in Storlet GET: https://bugs.launchpad.net/storlets/+bug/1537982 (Fix Released)
