Jump to: navigation, search

Difference between revisions of "Meetings/Storlets"

 
(124 intermediate revisions by 4 users not shown)
Line 1: Line 1:
Meeting Time: Every Wednesday at 13:00 UTC in #openstack-storlets
+
Past Meeting Time: Every Thursday at 07:00 UTC in #openstack-meeting
 +
* 15:00 JST
 +
* 10:00 IST
  
'''Agenda:'''
+
Because of inactivity we don't hold regular meetings now.
* Austing talk.
+
Please reach out to PTL or other members if you have any topic for discussion.
* Swift Hackathon
 
* bugs.
 
  
'''Austin'''
+
== Meeting Name: storlets (#startmeeting storlets) ==
* We currently have the following abstract
+
== Meeting Logs:  http://eavesdrop.openstack.org/meetings/storlets ==
** Title: From Analytic to Image Processing in Swift with Openstack Storlets
+
== Sorry, but cannot attend ==
** Openstack Stolets is an emerging Openstack project aimed at pushing down data centric compute jobs to the object store. Storlets offer a server-less programming model enabling developers to quickly develop and deploy data path computations without worrying about the underlying platform details.
+
Please fill below date, nick for dates you know you cannot attend.
  
Storlets facilitate several interesting use cases such as:
+
=Agenda:=
1. Performing image processing in the storage, allowing to process large objects without downloading it.
+
* Discuss patches
2. Spark analytic push down to the store, saving on both bandwidth and memory in the analytic cluster.
+
* Hot links (To be updated in the IRC channel topic as soon as I get permission :-)
3. Data obfuscation and anonymization at the source. Thus, preventing sensitive data from ever leaving the store.
+
# PTG: https://etherpad.openstack.org/p/storlets-pike-design-summit
 
+
# https://etherpad.openstack.org/p/storlets-review-priorities
In this talk we will present the project, discuss and demonstrate the use cases and present a PoC about an effective architecture for image processing using Storlets done at NTT.
 
 
 
'''Swift Hackathon'''
 
* The purpose of the session is to update the community on what's happening in the openstack-storlets project and get feedback on number of subjects (the intention is not to discuss pushing the middleware to Swift at this point).
 
** Updates:
 
*** Unit tests (Thanks NTT)
 
**** storlets GET flow: https://review.openstack.org/#/c/263533/
 
**** storlets PUT flow: https://review.openstack.org/#/c/264520/
 
**** uploading storlets/dependency: https://review.openstack.org/#/c/264638/
 
*** Copy semantics (I got a working prototype)
 
*** Storlets samples (thumbnail extraction, more to come), what else?
 
** Points for discussion:
 
*** Adding a non voting gate job for testing Storlets
 
*** Use footers for adding metadata
 
**** How to pass header information like Content-Type to storlet applications
 
*** Resource limitation
 
**** Should we the max number about containers or processes running on one node?
 
**** Can we stop unnecessary containers/processes?
 
*** To-be-fixed thing in swift
 
**** chunked transferring stuff: https://review.openstack.org/#/c/256201/
 
 
 
 
 
 
 
 
 
** https://review.openstack.org/#/c/261397/
 
* Session in Openstack Hackathon
 
***** anything else in master?
 
**** more?
 
* Next Openstack Summit
 
** Here is a first shot at a talk abstract:
 
*** Title: From Analytic to Image Processing in Swift with Openstack Storlets
 
*** Openstack Stolets is an emerging project aimed at pushing down data centric compute jobs to the object store. In this talk we will present the project, discuss and demonstrate use cases varying from spark analytic push down to image processing and present a PoC about an effective system architecture for image processing using Storlets done at NTT.
 

Latest revision as of 15:59, 31 May 2021

Past Meeting Time: Every Thursday at 07:00 UTC in #openstack-meeting

  • 15:00 JST
  • 10:00 IST

Because of inactivity we don't hold regular meetings now. Please reach out to PTL or other members if you have any topic for discussion.

Meeting Name: storlets (#startmeeting storlets)

Meeting Logs: http://eavesdrop.openstack.org/meetings/storlets

Sorry, but cannot attend

Please fill below date, nick for dates you know you cannot attend.

Agenda:

  • Discuss patches
  • Hot links (To be updated in the IRC channel topic as soon as I get permission :-)
  1. PTG: https://etherpad.openstack.org/p/storlets-pike-design-summit
  2. https://etherpad.openstack.org/p/storlets-review-priorities