Zaqar
OpenStack Message Bus ("Marconi")
Marconi is a new OpenStack project to create a multi-tenant, light-weight, fast, scale-out alternative to other popular queuing systems. The service will support 100's of thousands (or even millions) of clients and channels in a cost-effective manner, without sacrificing performance, durability, or HA.
This project's objective is not to supplant intra-cluster/LAN message buses, such as those based on AMQP and DDS. Rather, Marconi provides a complementary service that is optimized for web app messaging, is multi-tenant, and that scales to hundreds of thousands of Internet-connected agents per tenant.
Marconi will define a clean, RESTful API, use a modular architecture, and will support both eventing and job-queuing semantics. Users will be able to customize Marconi to achieve a wide range of performance, durability, availability, and efficiency goals.
Design
Marconi aims to be pragmatic, building upon the real-world experiences of teams who have solid track records running and supporting web-scale message queueing systems. The project's overarching design philosophy is derived from Donald A. Norman's work regarding The Design of Everyday Things:
The value of a well-designed object is when it has such a rich set of affordances that the people who use it can do things with it that the designer never imagined.
Goals related to the above:
- Emergent functionality, utility
- Modular, pluggable code base
- REST architectural style
Principles to live by:
- DRY
- YAGNI
- KISS
Use Cases
- Distribute tasks among multiple workers (transactional job queues)
- Forward events to data collectors (transactional event queues)
- Publish events to any number of subscribers (pub-sub)
- Send commands to one or more agents (point-to-point or pub-sub)
- Request an action or get information from an agent (RPC)
Out of Scope
Marconi may be used as the foundation for other services to support the following use cases, but will not support them directly within its code base.
- Forwarding notifications to email, SMS, Twitter, etc.
- Forwarding notifications to web hooks
- Forwarding notifications to APNS, GCM, etc.
- Scheduling-as-a-service
- Metering usage
Resources
| Grizzly Spec |
| API Blueprint |
| Other Blueprints |
| Milestones |
| Developer Docs |
| Source code |
| Bug tracker |
