Difference between revisions of "Roadmap (Marconi)"

Revision as of 20:24, 23 July 2014

Let's Do This

For Juno, let's ship a service that is super fast, has a polished API, and is well-documented. We can't decide the future of the project without getting more feedback based on real-world usage. If we need to totally reinvent the API based on that feedback, we can do that in K.

Juno Themes

• Speed
• Polish
• Docs

Major Goals

• Complete a polished version of the 1.0 API, v1.1
• Ship a driver for high-throughput use cases
  • 10ms for any operation, per individual client
  • 10K/sec aggregate rate per tenant
• Ship a driver for a non-AGPL backend that has feature parity with MongoDB
  • Durability is required, so can't simply be in-memory
  • Can be the same driver as for high-throughput use cases, if possible
• Expanded user guide, published to RTD

Upcoming milestones

Juno-3 (Sept 4)

Graduation

• Meet on G+ with TC rep to review graduation plan and get it blessed
• Create a feature-complete alternative to MongoDB, with similar durability and perf
• Rename the project
• Document all config options in user guide

Quality Engineering

• MongoDB gate
• Performance tuning
• Threat model for minimal HA deployment
• Publish benchmarks for minimal HA deployment for each production-ready driver
• End-game bug fixing and stabilization
• Decouple control plane backend drivers from data plane backend drivers to facilitate testing and maintenance

Drivers

• Finish foundational Redis message store driver
• Finish foundational AMQP transport driver

API v1.1

• Finish work on flavors
• Homedoc changes
• Response document changes
• MessagePack support (req. moving to storing bodies as )
• Finish up server implementation
• Implement in python-marconiclient
• Document in user guide
• Functional tests
• Tempest tests

NOTE on API v1.1

For the Juno release of Marconi, we will be positioning the API v1.1 as a community preview. Before freezing the spec and finalizing the implementation, we need to allow some time for feedback, esp. from SDK authors. Then, during k-1, we can implement that feedback and release the final API at the end of that milestone.

Past Milestones

Juno-2 (July 24)

• Finish tempest tests for API v1.0
• Basic marconi-bench tool (will be merged into Rally eventually)
• Finish most of API 1.1 (incl. msgpack support)
• Finish POCs for: Redis, AMQP, Elasticsearch (?), LevelDB (?)
• Start converting POCs into "real" drivers

Juno-1 (June 12)

• Start coding AMQP
• Start coding Redis
• Continue work on API v1.1
• Rename "shard" to "pool" and note the name change on wiki and in the user manual
• Start coding KPIs (/health endpoint)
• Fill out OSSG security FAQ template
• Revert read-only queue metadata from API 1.1 (don't forget to update the spec)

Postponed Work

Here are some things we would love to do this cycle, but don't have enough people to work on them. If you are interested in helping out, please get in touch via the openstack-dev mailing list (using the prefix "[marconi]" in the subject), or via IRC in #openstack-marconi on Freenode.

• Delayed message feature? (delay message visibility by x seconds, would help with hybrid/auditing use case)
• Live queue migration
• Rename the project
• AuthZ for Admin API
• Benchmark gate (3rd-party)
• Publish User Guide and API Reference on RTD
• User Guide 2.0
• Cross-service Request ID
• Basic security test suite
• Redis gate?
• DDoS testing and mitigation
• Research notifications (meet with 3rd-parties, create a design)
• Start coding notifications
• Integrate notifications with 1-2 other programs
• Demo videos
• Kafka driver
• Long polling or WebSocket support in the API
• Snappy or LZ4 compression of message bodies before storing them
• Structured logging
• Temp URL style endpoints with ACL, to give to untrusted parties for limited interactions with a queue
• Message signing