Messaging Service ("Marconi")
Marconi is a cloud messaging and notification service for developers building applications on top of OpenStack. The service features a web-friendly HTTP API, which developers can use to send messages between the various components of their SaaS and mobile applications, using a variety of communication patterns. Underlying the API is an efficient messaging engine designed with scalability and security in mind.
Donald A. Norman:
The value of a well-designed object is when it has such a rich set of affordances that the people who use it can do things with it that the designer never imagined.
At first glance, Marconi's RESTful API may seem a bit out of the ordinary. Message brokers typically rely on a custom binary protocol and often assume a reliable, 1st-party network between server and client. However, many of today's developers prefer an HTTP-based API. They value the simplicity and transparency of the protocol, its firewall-friendly nature, and its huge ecosystem of tools, load balancers and proxies.
Marconi caters to developers looking for this type of experience.
Marconi's API provides a basic set of semantics that, when combined, afford a variety of messaging patterns, such as pub-sub, task distribution, and point-to-point. Messages can be consumed as feeds, queues, or a combination of the two. This can be a little confusing at first, since the API uses the term "queues" to represent a hybrid feeds-queues resource.
When interacting with the API, a client can choose to read a queue in a similar manner to an Atom feed, where any client can read any message, and is responsible for keeping track of its own marker (its position in the feed). This provides for messaging patterns such as pub-sub and point-to-point.
Alternatively, an application can implement task distribution by creating a pool of workers that simply claim messages off the front of the queue. Once a message is claimed, it becomes invisible to other workers in the pool to prevent messages from being processed more than once.
Finally, a client can use both feed and claim semantics simultaneously to create hybrid messaging patterns. For example, while workers are busy claiming messages from a task queue, an auditor can passively sample those same messages as they flow by.
Marconi is an incubated OpenStack program, preparing to graduate at the end of the Juno release cycle (fall 2014). With the close of the Icehouse cycle, the team achieved a number of exciting milestones:
- Marconi's first official, production-ready "1.0" release is now available for download. This first release includes a battle-tested MongoDB driver, and production-ready drivers for additional backends are in the works.
- Marconi's v1.0 API is stable and ready to code against.
- User and operator docs are now available in the Marconi repo, and we will be publishing them soon
- A reference client library (written in Python) is now available on PyPI. This library supports the entire v1.0 API. Support for other languages is available through Rackspace-supported SDKs.
The Marconi team hangs out in #openstack-marconi on Freenode, and we are always happy to hear your ideas and answer any questions you might have. If you run into a bug, you can report it using on our issue tracker. Source code is available on GitHub for both the server and client.
- 2014.1 (Icehouse) - download
- 2014.2 (Juno) - Plans
- Alejandro Cabrera. Rackspace Atlanta. Introducing Openstack Marconi. July 17, 2013. Youtube Speaker Deck
- Flavio Percoco. EuroPython 2013. Marconi: Queuing and Notification Service for Openstack. July 2, 2013. YouTube
- Kurt Griffiths, Allan Metts. Openstack Summit April 2013. Project Overview: OpenStack Queuing and Notification Service ("Marconi""). April 2013. YouTube
- Kurt Griffiths, Flavio Percoco, Allan Metts. Openstack Summit November 2013. Openstack Queuing and Notification Service Marconi. November 2013. YouTube
- Oz Akan. Rackspace Devops Blog. July 25, 2013. Openstack Marconi API.
- Flavio Percoco. Developer's Black Hole. March 2014. People don't like to queue up.
- Coming soon...*
- Coming soon...*
- Coming soon...*