Jump to: navigation, search


< Storlets
Revision as of 20:08, 19 December 2016 by Eranrom (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

Echo System

  • Docker all in one
  • Develop an Eclipse plugin that would deploy the storlets straight to docker all in one for testing


  • Add non-voting py35 gate job
  • Add ability to control resources. See initial design thoughts [here|https://etherpad.openstack.org/p/swift-hackathon-feb-2016-storlets]
  • Add resource monitoring. This requires:
    • "Per Storlet Daemons" to log to host syslog.
    • Log CPU utilization per invocation
    • Log memory consumption per invocation
      • In python this can be done easily
      • In Java we first need to move to a process pool. Which in turn requires more control over JVM resources.
  • Move storlets infra from docker to a mounted directory
  • port storlets installation to devstack
  • Long running Storlets
  • Random access storlets (facilitates sampling / scikit learn file operations)
  • Allow swift run time user to run docker commands, thus getting rid of the "c" code for managing docker
  • Use user namespaces to get rid of 777 on daemon factory pipe permission
  • Replace 14.04 in s2aio with 16.04 and get rid of jre8 installation
  • Add timeout to Java SBUS (as suggested for python in https://review.openstack.org/#/c/410296/). A great way to garbadge collect unused containers and storlet daemon

Future Thoughts

  • Look at seccomp as either augmentation to Docker gateway or a new gateway for running storlets (Idea coming from Saggi)
  • Look at enabling a synchronous Lambda independent from objects: Run a function on provided data and get back answer (Idea coming from Doron)