Jump to: navigation, search

Meetings/Storlets

< Meetings
Revision as of 05:16, 23 February 2016 by Takashi Kajinami (talk | contribs) (Agenda:)

Meeting Time: Every Wednesday at 13:00 UTC in #openstack-storlets

Agenda:

  • Prioritize patches (suggestion):
* https://review.openstack.org/#/c/277322/ (refactor test_storlet_handler)...merged
* https://review.openstack.org/#/c/280488/ (refactor validation methods)...merged
* https://review.openstack.org/#/c/276543/ (use logger exception).............merged
* https://review.openstack.org/#/c/280038/ (change info to debug logs).....merged
* COPY functionality - Still need to post the patch
* https://review.openstack.org/#/c/276545/ (Improve timeout handling)
* https://review.openstack.org/#/c/282135/ (Refactor unit tests about storlet docker gateway)
* https://review.openstack.org/#/c/280570/ (Implement readline/readlines in IterLike, depends on 282135)
* Putting the last two after COPY does not mean I will not review them before.
  • Storlet in Hackathon etherpad
* I would like to add another analytics related question.
* What timeframe do we think we need?

Austin:

  • We currently have the following abstract
    • Title: From Analytic to Image Processing in Swift with Openstack Storlets
    • Openstack Stolets is an emerging Openstack project aimed at pushing down data centric compute jobs to the object store. Storlets offer a server-less programming model enabling developers to quickly develop and deploy data path computations without worrying about the underlying platform details.
    • Storlets facilitate several interesting use cases such as:
      • Performing image processing in the storage, allowing to process large objects without downloading it.
      • Spark analytic push down to the store, saving on both bandwidth and memory in the analytic cluster.
      • Data obfuscation and anonymization at the source. Thus, preventing sensitive data from ever leaving the store.
    • In this talk we will present the project, discuss and demonstrate the use cases and present a PoC about an effective architecture for image processing using Storlets done at NTT.

Swift Hackathon

Swift Community Session

  • The purpose of the session is to update the community on what's happening in the openstack-storlets project and get feedback on number of subjects (the intention is not to discuss pushing the middleware to Swift at this point).
  • Updates:
  • Points for discussion:
    • Adding a non voting gate job for testing Storlets
    • Use footers / Multipart MIME for adding metadata
    • To-be-fixed in swift:

Storlets Community Session(s)

  • Resource limitation (just ideas)
    • Limit the number of containers or processes running on one node
    • Limit resource consumptions in each containers
    • stop unnecessary containers/processes?
  • Can we also add system header like Content-Type?
  • Documentation

Bugs