Difference between revisions of "Zaqar"
Sebasmagri (talk | contribs) m (→Resources: Fix broken link to python-marconiclient GitHub repo) |
(→Resources) |
||
Line 132: | Line 132: | ||
|- | |- | ||
| API Blueprint | | API Blueprint | ||
− | | [[Marconi/specs/api/v1]] | + | | [[Marconi/specs/api/v1]]<br/> |
− | [[Marconi/specs/zmq/api/v1]] | + | [[Marconi/specs/zmq/api/v1]]<br/> |
+ | [[Marconi/specs/sharding/v1|Sharding]] | ||
|- | |- | ||
| Other Blueprints | | Other Blueprints |
Revision as of 14:49, 30 December 2013
Contents
OpenStack Queuing and Notification Service
As OpenStack matures, it needs to support bigger, more complex web applications. Such applications require a robust, web-scale message queuing service to support the distributed nature of large web applications. Marconi is a new OpenStack project designed to fill this need. Our aim is to create an open alternative to SQS (producer-consumer) and SNS (pub-sub), for use in applications that run on OpenStack clouds. A message queueing service is vital component of big, complex web applications
The project will define a clean, RESTful API, use a modular architecture, and will support unified pub-sub and job-queuing semantics. Users will be able to customize Marconi to achieve a wide range of performance, durability, availability, and efficiency goals.
Core Team
Kurt Griffiths (kgriffs) is the current PTL. Other core team members include:
- Flavio Percoco (flaper87)
- Alejandro Cabrera (cpp-cabrera)
- [your name could be here!]
Design
Marconi aims to be pragmatic, building upon the real-world experiences of teams who have solid track records running and supporting web-scale message queueing systems. The project's overarching design philosophy is derived from Donald A. Norman:
The value of a well-designed object is when it has such a rich set of affordances that the people who use it can do things with it that the designer never imagined.
Goals related to the above:
- Emergent functionality, utility
- Modular, pluggable code base
- REST architectural style
Principles to live by:
- DRY
- YAGNI
- KISS
Architecture
Use Cases
- Distribute tasks among multiple workers (transactional job queues)
- Forward events to data collectors (transactional event queues)
- Publish events to any number of subscribers (pub-sub)
- Send commands to one or more agents (point-to-point or pub-sub)
- Request an action or get information from an agent (RPC)
Out of Scope
Marconi may be used as the foundation for other services to support the following use cases, but will not support them directly within its code base.
- Forwarding notifications to email, SMS, Twitter, etc.
- Forwarding notifications to web hooks
- Forwarding notifications to APNS, GCM, etc.
- Scheduling-as-a-service
- Metering usage
Etherpads
- Marconi Client Brainstorm #1
- Common Code Improvements
- ZMQ Brainstorm #1
- FIFO Etc.
- Tests Refactoring
- Placement Service
- Marconi Proxy Issues
Presentations
- Alejandro Cabrera. Rackspace Atlanta. Introducing Openstack Marconi. July 17, 2013. Youtube Speaker Deck
- Flavio Percoco. EuroPython 2013. Marconi: Queuing and Notification Service for Openstack. July 2, 2013. YouTube
- Kurt Griffiths, Allan Metts. Openstack Summit April 2013. Project Overview: OpenStack Queuing and Notification Service ("Marconi""). April 2013. YouTube
- Kurt Griffiths, Flavio Percoco, Allan Metts. Openstack Summit November 2013. Openstack Queuing and Notification Service Marconi. November 2013. YouTube
Articles
- Oz Akan. Rackspace Devops Blog. July 25, 2013. Openstack Marconi API.
FAQ
Will Marconi work with AMQP?
- Planned as a backend for v2 API
- Transport TBD
- Talk to us about use cases
- Need contributors
How does Marconi compare to AWS (SQS/SNS)?
- Targets similar workloads
- Marconi will provide a unified API to handle notifications and queuing
- Marconi is highly customizable
- FIFO and once-and-only-once guaranteed (depending on storage backend)
How mature is the project?
- Marconi is used in production: marconi_users
- Number of contributors is growing
What's next for marconi?
- Releasing API v1.1 for Icehouse
- Additional storage drivers
- Storage sharding: scaling horizontally and heterogeneous storage
- Notifications
- Message signing
- Additional ops features
- Reference client library
See also the Icehouse Roadmap.
How easy is it to contribute/get up and running?
- You don't need devstack
- You decide what you want to work on: storage, transport, client-side
- Very decoupled
- Easy: choose a bug and submit a patch
Resources
Meetings | Meetings/Marconi |
IRC | #openstack-marconi on Freenode |
Trello | https://trello.com/b/7NLODgbr (deprecated in favor of Launchpad) |
Havana Spec | Marconi/specs/havana |
API Blueprint | Marconi/specs/api/v1 |
Other Blueprints | https://blueprints.launchpad.net/marconi |
Milestones | https://launchpad.net/marconi/+milestones |
Developer Docs | Python Client Bindings Drivers Guarantees |
Incubation | Marconi/Incubation |
Source code | https://github.com/openstack/marconi |
Client source code | https://github.com/openstack/python-marconiclient |
Code Reviews | https://review.openstack.org/#/q/status:open+project:stackforge/marconi,n,z |
Bug tracker | https://bugs.launchpad.net/marconi |