Difference between revisions of "Meetings/Storlets"
< Meetings
m (→Agenda:) |
(→Storlets Community Session(s)) |
||
Line 35: | Line 35: | ||
* Resource limitation (just ideas) | * Resource limitation (just ideas) | ||
** Limit the number of containers or processes running on one node | ** Limit the number of containers or processes running on one node | ||
− | ** Limit resource consumptions in each containers | + | ** Limit resource consumptions in each containers |
− | ** stop unnecessary containers/processes | + | *** ex. core binding of docker container |
+ | ** stop unnecessary containers/processes | ||
+ | ** Set resource parameter about jvm: https://review.openstack.org/#/c/275083/ (patch should be fixed) | ||
* Can we also add system header like Content-Type? | * Can we also add system header like Content-Type? | ||
− | * Unit tests for Java modules | + | * Unit tests |
− | ** How can we integrate them into openstack gate testing? | + | ** for python modules |
+ | ** for Java modules | ||
+ | *** How can we integrate them into openstack gate testing? | ||
* Documentation | * Documentation | ||
+ | * Refactoring ideas | ||
=Bugs= | =Bugs= |
Revision as of 03:30, 24 February 2016
Meeting Time: Every Wednesday at 13:00 UTC in #openstack-storlets
Contents
Agenda:
- Prioritize patches (suggestion):
* https://review.openstack.org/#/c/277322/ (refactor test_storlet_handler)...merged * https://review.openstack.org/#/c/280488/ (refactor validation methods)...merged * https://review.openstack.org/#/c/276543/ (use logger exception).............merged * https://review.openstack.org/#/c/280038/ (change info to debug logs).....merged * COPY functionality - Still need to post the patch * https://review.openstack.org/#/c/276545/ (Improve timeout handling) * https://review.openstack.org/#/c/282135/ (Refactor unit tests about storlet docker gateway) * https://review.openstack.org/#/c/280570/ (Implement readline/readlines in IterLike, depends on 282135) * Putting the last two after COPY does not mean I will not review them before.
- Storlet in Hackathon etherpad
- I would like to add another analytics related question.
- What timeframe do we think we need?
Austin:
- We currently have the following abstract
- Title: From Analytic to Image Processing in Swift with Openstack Storlets
- Openstack Stolets is an emerging Openstack project aimed at pushing down data centric compute jobs to the object store. Storlets offer a server-less programming model enabling developers to quickly develop and deploy data path computations without worrying about the underlying platform details.
- Storlets facilitate several interesting use cases such as:
- Performing image processing in the storage, allowing to process large objects without downloading it.
- Spark analytic push down to the store, saving on both bandwidth and memory in the analytic cluster.
- Data obfuscation and anonymization at the source. Thus, preventing sensitive data from ever leaving the store.
- In this talk we will present the project, discuss and demonstrate the use cases and present a PoC about an effective architecture for image processing using Storlets done at NTT.
Swift Hackathon
Swift Community Session
- See the following etherpad
Storlets Community Session(s)
- Resource limitation (just ideas)
- Limit the number of containers or processes running on one node
- Limit resource consumptions in each containers
- ex. core binding of docker container
- stop unnecessary containers/processes
- Set resource parameter about jvm: https://review.openstack.org/#/c/275083/ (patch should be fixed)
- Can we also add system header like Content-Type?
- Unit tests
- for python modules
- for Java modules
- How can we integrate them into openstack gate testing?
- Documentation
- Refactoring ideas
Bugs
- Range handling: on going
- FD leak : Fix released