Jump to: navigation, search

Difference between revisions of "Zaqar"

m (Key features)
(Status)
 
(22 intermediate revisions by 6 users not shown)
Line 1: Line 1:
 +
== Projects ==
 +
 +
=== Zaqar ===
 +
 +
{| border="1" cellpadding="2"
 +
| Official name
 +
| '''OpenStack Messaging'''
 +
|-
 +
| Source code
 +
| https://github.com/openstack/zaqar
 +
|-
 +
| Bug tracker
 +
| https://bugs.launchpad.net/zaqar
 +
|-
 +
| Feature tracker
 +
| https://blueprints.launchpad.net/zaqar
 +
|-
 +
| Contributor documentation
 +
| http://docs.openstack.org/developer/zaqar/
 +
|}
 +
 +
=== Python Zaqar Client ===
 +
{| border="1" cellpadding="2"
 +
| Source code
 +
| https://github.com/openstack/python-zaqarclient
 +
|-
 +
| Bug tracker
 +
| https://bugs.launchpad.net/python-zaqarclient
 +
|-
 +
| Feature tracker
 +
| https://blueprints.launchpad.net/python-zaqarclient
 +
|}
 +
 
== Overview ==
 
== Overview ==
  
Zaqar is a multi-tenant cloud service for web developers. It combines the ideas pioneered by Amazon's SQS product with additional semantics to support event broadcasting.
+
Zaqar is a multi-tenant cloud messaging service for web and mobile developers. It combines the ideas pioneered by Amazon's SQS product with additional semantics to support event broadcasting.
  
 
The service features a fully RESTful API, which developers can use to send messages between various components of their SaaS and mobile applications, by using a variety of communication patterns. Underlying this API is an efficient messaging engine designed with scalability and security in mind.  
 
The service features a fully RESTful API, which developers can use to send messages between various components of their SaaS and mobile applications, by using a variety of communication patterns. Underlying this API is an efficient messaging engine designed with scalability and security in mind.  
Line 7: Line 40:
 
Other OpenStack components can integrate with Zaqar to surface events to end users and to communicate with guest agents that run in the "over-cloud" layer. Cloud operators can leverage Zaqar to provide equivalents of SQS and SNS to their customers.
 
Other OpenStack components can integrate with Zaqar to surface events to end users and to communicate with guest agents that run in the "over-cloud" layer. Cloud operators can leverage Zaqar to provide equivalents of SQS and SNS to their customers.
  
=== Key features ===
+
==== Key features ====
  
 
Zaqar provides the following key features:
 
Zaqar provides the following key features:
  
* Firewall-friendly, HTTP-based API with Keystone support
+
* Choice between two communication transports. Both with Keystone support:
 +
** Firewall-friendly, HTTP-based RESTful API
 +
*** Many of today's developers prefer a more web-friendly HTTP API. They value the simplicity and transparency of the protocol, its firewall-friendly nature, and its huge ecosystem of tools, load balancers and proxies. In addition, cloud operators appreciate the scalability aspects of the REST architectural style.
 +
** Websocket-based API for persistent connections.
 +
*** Websocket protocol provides communication over persistent connections. Unlike HTTP, where new connections are opened for each request/response pair, Websocket can transfer multiple requests/responses over single TCP connection. It saves much network traffic and minimizes delays.
 
* Multi-tenant queues based on Keystone project IDs
 
* Multi-tenant queues based on Keystone project IDs
* Support for [[Use_Cases_(Marconi)|several common patterns]] including event broadcasting, task distribution, and point-to-point messaging
+
* Support for several common patterns including event broadcasting, task distribution, and point-to-point messaging (See [[Zaqar/Use_Cases|Zaqar Use Cases]])
 
* Component-based architecture with support for custom backends and message filters
 
* Component-based architecture with support for custom backends and message filters
 
* Efficient reference implementation with an eye toward low latency and high throughput (dependent on backend)
 
* Efficient reference implementation with an eye toward low latency and high throughput (dependent on backend)
* Highly-available and horizontally scalable
+
* Highly-available and horizontally scalable (See [[Server Architecture (Marconi)|Server Architecture]])
 +
* Support for subscriptions to queues. Several notification types are available:
 +
** Email notifications
 +
** Webhook notifications
 +
** Websocket notifications
  
=== Project scope ===
+
==== Project scope ====
  
 
The Zaqar API is data-oriented. That is, it does not provision message brokers and expose those directly to clients. Instead, the API acts as a bridge between the client and one or more backends. A provisioning service for message brokers—however useful—serves a somewhat different market from what Zaqar is targeting today. With that in mind, if users are interested in a broker provisioning service, the community should consider starting a new project to address that need.
 
The Zaqar API is data-oriented. That is, it does not provision message brokers and expose those directly to clients. Instead, the API acts as a bridge between the client and one or more backends. A provisioning service for message brokers—however useful—serves a somewhat different market from what Zaqar is targeting today. With that in mind, if users are interested in a broker provisioning service, the community should consider starting a new project to address that need.
  
=== Design principles ===
+
==== Design principles ====
  
 
Zaqar, as with all OpenStack projects, is designed with the following guidelines in mind:
 
Zaqar, as with all OpenStack projects, is designed with the following guidelines in mind:
Line 32: Line 73:
 
* '''Open standards.''' Be a reference implementation for a community-driven
 
* '''Open standards.''' Be a reference implementation for a community-driven
  
== The API ==
+
==== Status ====
 +
 
 +
Zaqar is production-ready for small and medium-sized workloads. The project is in production at Rackspace and other companies are in the process of deploying it in their own clouds. Support for the v2 APIs is available for Python in [https://github.com/openstack/python-zaqarclient python-zaqarclient], and for other languages through [http://developer.rackspace.com/#home-sdks Rackspace-supported SDKs].
 +
 
 +
== Documentation ==
 +
 
 +
Currently (during Mitaka release cycle) Zaqar team is reworking Zaqar documentation.
  
At first glance, Zaqar's RESTful API may seem a bit out of the ordinary. Message brokers typically rely on a custom binary protocol and long-lived connections. However, many of today's developers prefer a more web-friendly HTTP API. They value the simplicity and transparency of the protocol, its firewall-friendly nature, and its huge ecosystem of tools, load balancers and proxies. In addition, cloud operators appreciate the scalability aspects of the REST architectural style.
+
==== Zaqar Configuration Reference ====
  
[[Zaqar/specs/api/v1|Version 1.0]] of the API has been stable for some time. We are working on a [[Zaqar/specs/api/v1.1|1.1 version]] with minor changes to remove some of the rough edges in 1.0, and we welcome community feedback. Note that some proposals by the community, such as '''removing random access to individual messages by ID''', will be taken care of in version 2.0 when we will have more freedom to make major changes.
+
[http://docs.openstack.org/newton/config-reference/message.html Message service Configuration Reference]
  
== Status ==
+
==== Zaqar Contributor Guide ====
  
With the close of the Icehouse cycle, the program has achieved a number of important milestones:
+
[http://docs.openstack.org/developer/zaqar/ Zaqar Contributor Guide]
  
* Zaqar's first official, production-ready "1.0" release is now [http://tarballs.openstack.org/marconi/ available for download]. This first release includes a battle-tested MongoDB driver, and production-ready drivers for additional backends are in the works.
+
==== Zaqar RESTful API Specs ====
* Zaqar's v1.0 API is stable and ready to code against.
 
* Basic user and operator docs are [http://docs.openstack.org/developer/marconi/ now available], and we will be adding tons of new content during Juno.
 
* A reference client library (written in Python) is now [https://pypi.python.org/pypi/python-marconiclient/ available on PyPI] which supports the entire v1.0 API. Support for other languages is available through [http://developer.rackspace.com/#home-sdks Rackspace-supported SDKs].
 
  
 +
* [[Zaqar/specs/api/v1|API v1.0]] ('''Released in 2014.1''', '''Deprecated''')
 +
* [[Zaqar/specs/api/v1.1|API v1.1]] ('''Released in 2014.2''', '''Deprecated''')
 +
* [http://developer.openstack.org/api-ref/messaging/ API v2.0] ('''Current API Version''')
  
The program has attracted a growing community of contributors from all over the world. We are particularly proud of our interns:
+
==== Other documentation ====
  
* 10+ organizations represented
+
"Getting Started Guide",  "Zaqar RESTful API v1.0 Guide and Reference" are currently not hosted anywhere, but you can generate these documents from Zaqar source code repository. Here's how to do it:
* 5 core reviewers
+
# Ensure you have '''maven''' and '''git''' programs installed on your system.
* 6 interns for Juno (representing GSoC, GNOME OPW, Rackspace and Red Hat)
+
# Clone Zaqar repository in the preferred directory on your machine by executing '''git clone <nowiki>https://github.com/openstack/zaqar.git</nowiki>'''
 +
# Inside cloned repository in ''doc/user-guide'' directory execute '''mvn clean generate-sources''' to generate documentation html files from sources.
 +
# You can view documentation by opening generated html files in your browser:
 +
## Getting Started Guide: ''doc/user-guide/zaqar-get-started/target/docbkx/webhelp/zaqar-get-started/index.html''
 +
## Zaqar Configuration Reference: ''doc/user-guide/zaqar-config-ref/target/docbkx/webhelp/zaqar-config-ref/index.html''
 +
## Zaqar API v1.0 Guide and Reference: ''doc/user-guide/zaqar-api-ref/target/docbkx/webhelp/zaqar-api-ref/index.html''
  
 
== Resources ==
 
== Resources ==
  
The Zaqar team hangs out in '''#openstack-zaqar''' on Freenode. Feel free to stop in anytime to chat with us; the crew is always happy to hear your ideas and answer questions. If you run into a bug, please report it using our [https://bugs.launchpad.net/marconi issue tracker].  
+
The Zaqar team hangs out in '''#openstack-zaqar''' on Freenode. Feel free to stop in anytime to chat with us; the crew is always happy to hear your ideas and answer questions. If you run into a bug, please report it using our [https://bugs.launchpad.net/zaqar issue tracker].
  
==== Docs ====
+
==== FAQ ====
  
* [http://docs.openstack.org/developer/marconi/ Zaqar User Guide]
+
* [[Zaqar/Frequently asked questions|Frequently asked questions]]
* [https://ask.openstack.org/en/questions/ Ask OpenStack (Community Q&A)]
 
* [[Marconi/docs/about|About the Program]]
 
  
 
==== Releases ====
 
==== Releases ====
Line 79: Line 129:
 
==== Articles ====
 
==== Articles ====
  
* Oz Akan. Rackspace Devops Blog. July 25, 2013. [http://developer.rackspace.com/blog/openstack-marconi-api.html Openstack Marconi API].
+
* Oz Akan. Rackspace Devops Blog. July 25, 2013. [http://developer.rackspace.com/blog/openstack-marconi-api.html Openstack Zaqar API v1.0].
 
* Flavio Percoco. Developer's Black Hole. March 2014. [http://blog.flaper87.com/post/531cd585d987d24e83f082a5 People don't like to queue up].
 
* Flavio Percoco. Developer's Black Hole. March 2014. [http://blog.flaper87.com/post/531cd585d987d24e83f082a5 People don't like to queue up].
  
== Contributor Guide ==
+
== Community ==
  
''Note: The Marconi team is reworking this section of the wiki with tons of updated content, so some pages are missing or incomplete at the moment. If you can't find the information you are looking for, please drop us a line in IRC (#openstack-marconi) and we'll be happy to help.''
+
* [https://webchat.freenode.net/?channels=openstack-zaqar #openstack-zaqar] (IRC)
 
+
* [[Zaqar/Feedback|Feedback]]
==== Getting Started ====
+
* [https://ask.openstack.org/en/questions/ Ask OpenStack (Community Q&A)]
  
* [[Welcome New Contributors (Marconi)|Welcome New Contributors]]
+
== Information for Contributors ==
* [[Give Marconi a Try (Marconi)|Give Zaqar a Try]]
 
* [[Your First Patch (Marconi)|Your First Patch]]
 
* [[Your First Review (Marconi)|Your First Review]]
 
* [[Write the Tests! (Marconi)|Write the Tests!]]
 
* [[Write the Docs! (Marconi)|Write the Docs!]]
 
<!-- TODO(kgriffs): screencasts -->
 
  
==== API Specs ====
+
[http://docs.openstack.org/developer/zaqar/ Zaqar Contributor Guide]
  
* [[Marconi/specs/api/v1|API v1.0]] ('''released in 2014.1''')
+
''Note: If you can't find the information you are looking for, please drop us a line in IRC (#openstack-zaqar) and we'll be happy to help.''
* [[Marconi/specs/api/v1.1|API v1.1]]
 
* [[Marconi/specs/api/v2.0|API v2.0]]
 
  
 
==== Design ====
 
==== Design ====
  
* [[Use_Cases_(Marconi)|Marconi Use Cases]]
+
* [[Zaqar/Use_Cases|Zaqar Use Cases]]
 
* [[Server Architecture (Marconi)|Server Architecture]]
 
* [[Server Architecture (Marconi)|Server Architecture]]
 
* Reference Clients
 
* Reference Clients
Line 119: Line 161:
 
* [[Integration Testing (Marconi)|Integration Testing]]
 
* [[Integration Testing (Marconi)|Integration Testing]]
 
<!-- TODO(kgriffs): What about user experience design/testing? -->
 
<!-- TODO(kgriffs): What about user experience design/testing? -->
 
==== Resources ====
 
 
* [[About (Marconi)|About the Program]]
 
* [[Frequently Asked Questions (Marconi)|Frequently Asked Questions]]
 
* [[Development Process (Marconi)|Development Process]]
 
* [[Blueprinting Process (Marconi)|Blueprinting Process]]
 
* [[Programming Style Guide (Marconi)|Programming Style Guide]]
 
* [[Reviewer Guide (Marconi)|Reviewer Guide]]
 
* [[Getting Help (Marconi)|Getting Help]]
 
  
 
==== Community ====
 
==== Community ====
  
* [https://webchat.freenode.net/?channels=openstack-marconi #openstack-zaqar] (IRC)
+
* [https://webchat.freenode.net/?channels=openstack-zaqar #openstack-zaqar] (IRC)
* [[Meetings/Marconi|Meetings]]
+
* [[Meetings/Zaqar|Meetings]]
* [https://launchpad.net/marconi/+milestones Milestones]
+
* [[Zaqar/Feedback|Feedback]]
* [https://blueprints.launchpad.net/marconi Blueprints]
+
* [https://ask.openstack.org/en/questions/ Ask OpenStack (Community Q&A)]
* [https://bugs.launchpad.net/marconi Bugs]
 
* [https://github.com/openstack/marconi Server Code]
 
* [https://github.com/openstack/python-marconiclient Client Code]
 
 
 
== Third party ==
 
 
 
* [[Driver development guide (Marconi)|Driver development guide]]
 
  
== Administrivia ==
+
==== Administrivia ====
  
* [[Roadmap (Marconi)|Roadmap]]
+
* [[Roadmap (Marconi)|Roadmap (Juno)]]
 
* [https://etherpad.openstack.org/p/naav-graduation Incubation Status]
 
* [https://etherpad.openstack.org/p/naav-graduation Incubation Status]
 
* [[How to Release (Marconi)|How to Release]]
 
* [[How to Release (Marconi)|How to Release]]
* [[Security/Juno/Marconi|OSSG Q&A for Marconi (Juno)]]
+
* [[Security/Juno/Marconi|OSSG Q&A for Zaqar (Juno)]]

Latest revision as of 02:17, 7 November 2016

Projects

Zaqar

Official name OpenStack Messaging
Source code https://github.com/openstack/zaqar
Bug tracker https://bugs.launchpad.net/zaqar
Feature tracker https://blueprints.launchpad.net/zaqar
Contributor documentation http://docs.openstack.org/developer/zaqar/

Python Zaqar Client

Source code https://github.com/openstack/python-zaqarclient
Bug tracker https://bugs.launchpad.net/python-zaqarclient
Feature tracker https://blueprints.launchpad.net/python-zaqarclient

Overview

Zaqar is a multi-tenant cloud messaging service for web and mobile developers. It combines the ideas pioneered by Amazon's SQS product with additional semantics to support event broadcasting.

The service features a fully RESTful API, which developers can use to send messages between various components of their SaaS and mobile applications, by using a variety of communication patterns. Underlying this API is an efficient messaging engine designed with scalability and security in mind.

Other OpenStack components can integrate with Zaqar to surface events to end users and to communicate with guest agents that run in the "over-cloud" layer. Cloud operators can leverage Zaqar to provide equivalents of SQS and SNS to their customers.

Key features

Zaqar provides the following key features:

  • Choice between two communication transports. Both with Keystone support:
    • Firewall-friendly, HTTP-based RESTful API
      • Many of today's developers prefer a more web-friendly HTTP API. They value the simplicity and transparency of the protocol, its firewall-friendly nature, and its huge ecosystem of tools, load balancers and proxies. In addition, cloud operators appreciate the scalability aspects of the REST architectural style.
    • Websocket-based API for persistent connections.
      • Websocket protocol provides communication over persistent connections. Unlike HTTP, where new connections are opened for each request/response pair, Websocket can transfer multiple requests/responses over single TCP connection. It saves much network traffic and minimizes delays.
  • Multi-tenant queues based on Keystone project IDs
  • Support for several common patterns including event broadcasting, task distribution, and point-to-point messaging (See Zaqar Use Cases)
  • Component-based architecture with support for custom backends and message filters
  • Efficient reference implementation with an eye toward low latency and high throughput (dependent on backend)
  • Highly-available and horizontally scalable (See Server Architecture)
  • Support for subscriptions to queues. Several notification types are available:
    • Email notifications
    • Webhook notifications
    • Websocket notifications

Project scope

The Zaqar API is data-oriented. That is, it does not provision message brokers and expose those directly to clients. Instead, the API acts as a bridge between the client and one or more backends. A provisioning service for message brokers—however useful—serves a somewhat different market from what Zaqar is targeting today. With that in mind, if users are interested in a broker provisioning service, the community should consider starting a new project to address that need.

Design principles

Zaqar, as with all OpenStack projects, is designed with the following guidelines in mind:

  • Component-based architecture. Quickly add new behaviors
  • Highly available and scalable. Scale to very serious workloads
  • Fault tolerant. Isolated processes avoid cascading failures
  • Recoverable. Failures should be easy to diagnose, debug, and rectify
  • Open standards. Be a reference implementation for a community-driven

Status

Zaqar is production-ready for small and medium-sized workloads. The project is in production at Rackspace and other companies are in the process of deploying it in their own clouds. Support for the v2 APIs is available for Python in python-zaqarclient, and for other languages through Rackspace-supported SDKs.

Documentation

Currently (during Mitaka release cycle) Zaqar team is reworking Zaqar documentation.

Zaqar Configuration Reference

Message service Configuration Reference

Zaqar Contributor Guide

Zaqar Contributor Guide

Zaqar RESTful API Specs

Other documentation

"Getting Started Guide", "Zaqar RESTful API v1.0 Guide and Reference" are currently not hosted anywhere, but you can generate these documents from Zaqar source code repository. Here's how to do it:

  1. Ensure you have maven and git programs installed on your system.
  2. Clone Zaqar repository in the preferred directory on your machine by executing git clone https://github.com/openstack/zaqar.git
  3. Inside cloned repository in doc/user-guide directory execute mvn clean generate-sources to generate documentation html files from sources.
  4. You can view documentation by opening generated html files in your browser:
    1. Getting Started Guide: doc/user-guide/zaqar-get-started/target/docbkx/webhelp/zaqar-get-started/index.html
    2. Zaqar Configuration Reference: doc/user-guide/zaqar-config-ref/target/docbkx/webhelp/zaqar-config-ref/index.html
    3. Zaqar API v1.0 Guide and Reference: doc/user-guide/zaqar-api-ref/target/docbkx/webhelp/zaqar-api-ref/index.html

Resources

The Zaqar team hangs out in #openstack-zaqar on Freenode. Feel free to stop in anytime to chat with us; the crew is always happy to hear your ideas and answer questions. If you run into a bug, please report it using our issue tracker.

FAQ

Releases

Talks

  • Alejandro Cabrera. Rackspace Atlanta. Introducing Openstack Marconi. July 17, 2013. Youtube Speaker Deck
  • Flavio Percoco. EuroPython 2013. Marconi: Queuing and Notification Service for Openstack. July 2, 2013. YouTube
  • Kurt Griffiths, Allan Metts. Openstack Summit April 2013. Project Overview: OpenStack Queuing and Notification Service ("Marconi"). April 2013. YouTube
  • Kurt Griffiths, Flavio Percoco, Allan Metts. Openstack Summit November 2013. Openstack Queuing and Notification Service. November 2013. YouTube
  • Yeela Kaplan. EuroPython 2014. OpenStack Queuing and Notification Service ("Marconi"). July 22, 2014. YouTube

Articles

Community

Information for Contributors

Zaqar Contributor Guide

Note: If you can't find the information you are looking for, please drop us a line in IRC (#openstack-zaqar) and we'll be happy to help.

Design

Quality Engineering

Community

Administrivia