Meteos/Meteos2.0

Meteos 2.0

Mission

To host & provide infrastructure for machine learning technologies and expose the machine learning technologies via robust & opaque APIs for consumers to consume the machine learning intelligence and also for producers to provide machine learning algorithms

Meteos 1.0 Vs 2.0

Meteos 1.0 concentrated on wrapping Apache SparkML inside Openstack wrapper and offered the same as Openstack Service. In that extent, ML intelligence is itself implemented via Openstack. Hence as a side effect, this made Meteos very much tightly coupled with one single ML technology. It also complicated the roadmap of adding multiple ML technologies like TF, PyTorch etc.

In reality, Openstack is best suited to provide infrastructure, rather than the intelligence. Hence in Meteos 2.0, we are re-imagining the project to do what Openstack is good at - Providing infrastructure in a super efficient way.

Scope

• Infrastructure ( CPU , GPU, Memory, NICs ) for
  • Serving pre-trained ML models
  • Training ML models
  • Executing Data Ingestion, Filtering & Data processing pipelines
• Expose APIs & SDKs for
  • Connecting user programs ( a.k.a ML Apps ) to ML models served via openstack
  • Uploading pre-trained ML models for serving prediction / classification requests
  • Uploading ML tasks / jobs as part of distributed cloud based training
  • Uploading / Downloading datasets, weights etc for utility functions

No Scope

• Tight integration to any specific ML technology
• Design & Development of any ML algorithm
• Algorithm marketplace

Highlevel Components

Meteos 2.0 Components

Component	Description
ML Stack Store	A store maintaining ML Technology stacks, based on HOT files. Any new technology stack ( TF, PyTorch etc ) can be introduced as a ML Stack and provisioned via Metes 2.0
Butter	Ice cream