Jump to: navigation, search

Difference between revisions of "Zaqar/Incubation-Old"

m (moved vicky from previous contributor to current)
(Future Release Plans: Added message tags to future release plans)
Line 66: Line 66:
 
* MySQL Support
 
* MySQL Support
 
* Celery Integration
 
* Celery Integration
 +
* Message tags
  
 
 
==Location of project source code==
 
==Location of project source code==
  

Revision as of 18:48, 15 July 2013

Notice

We still have a lot of information to add, but do check back often and share your feedback in #openstack-marconi.

See also: Governance/Incubation Request


Go to start of metadata

Project Codename

Marconi

Summary

Marconi is a messaging and notifications queue for the OpenStack product portfolio, supporting both producer-consumer and publish-subscribe modes. Marconi is designed to perform and scale in a multi-tenant environment.

Detailed Description

As the OpenStack product line grows, the need for a messaging and notification service is growing in order to support more complex, distributed application architectures. Marconi is designed to run on Nova servers, behind OpenStack load balancers, and in coordination with Keystone auth.

Marconi provides an interface for users to create queues, producers add messages to those queues, workers to claims messages from the queues, and listeners to get feeds from queue activity. Marconi is not designed to provide workflow management, but will guarantee first in, first out (FIFO) order for single producer models.

Backed by a clean, simple REST API, the Marconi API is designed to run with two primary server types: a simple single-instance web application server, for development and prototyping, plus a Mongo DB-based cluster designed for high availability and horizontal scalability. Messages are stored in Mongo DB clusters, written to a primary and two secondary servers. If a server is unavailable, the Marconi code will force a retry and direct traffic to one of the secondary servers. High availability for the API itself is achieved via standard HTTP server best practices.

The Marconi API exposes the following calls:

Queues:

Create Queue
List Queues
Set Queue Metadata
Get Queue Metadata
Get Queue Stats
Delete Queue

Messages

Post Message(s)
Get Messages
Get a Specific Message
Claim Messages

Claims

Query Claim
Update Claim
Release Claim


Basic roadmap for the project

Version 1.0

  • REST API for queue/messages/claim management
  • Multi-tenant
  • Integrated with Keystone for authentication
  • Command Line Interface
  • Python Bindings
  • Error handling, edge cases
  • Ruggedization/packaging for production deployments
  • Operator Documentation
  • Performance Tuning

Future Release Plans

  • Non-Python language bindings
  • CI/CD, Complete System Tests
  • Message Compression
  • MySQL Support
  • Celery Integration
  • Message tags

Location of project source code

Marconi on Launchpad Marconi at StackForge python-marconiclient at StackForge

Programming language, required technology dependencies

Language: Python

Dependencies: Falcon, MongoDB, Oslo, Stevedore

Is project currently open sourced? What license?

Yes, under the Apache 2.0 license.

Level of maturity of software and team

The Marconi project has been under active development at both Rackspace and Red Hat for 6 months and recently reached feature-complete status. The current code base contains numerous unit and functional tests. A comprehensive load and security test suite is currently under development. The Marconi team is ramping up on client library development, and will be incubating the common OpenStack API library as part of the project.

Proposed project technical lead and qualifications

Kurt Griffiths is a Principal Architect at Rackspace Hosting, currently working with other technical leaders inside and outside the company on cloud architecture, strategy and evangelism. Kurt focuses on web service APIs, agents, cryptography, and performance. He works with other senior technologists to provide day-to-day and long-term technical guidance for products developed at Rackspace.

Other project developers and qualifications

Current Code Contributors

  • Flavio Percoco is a Software Developer at Red Hat. Active in the OpenStack community, Flavio has contributed and reviewed code on OpenStack’s Block Storage, Image, Oslo, and Stable Maintainers teams.
  • Malini Kamalambalis a Quality Engineer at Rackspace, with over 11 years of industry experience. In addition to test planning and functional execution, Malini has history in software development.
  • Oz Akan is a Cloud Engineering Manager at Rackspace and has been working in technology since 1997. Oz is an expert in database management, having spent time working with MySQL, Oracle, and Mongo DB.
  • Allan Metts is the Director of Engineering for the Rackspace Atlanta office, which is the site responsible for the company's Cloud Backup, Jungle Disk, and Cloud Queuing products. Prior to Rackspace, Allan led the development of several large-scale systems in the telecommunications, transportation, and financial sectors. He specializes in distributed systems that crunch large amounts of data at high rates of speed.
  • Victoria Martínez de la Cruz is a Licenciate in Computer Science student and one of the interns of GNOME OPW for OpenStack.

Previous Code Contributors

  • Jamie Painter is a Software Manager at Rackspace, as well as a member of Rackspace's Technical Leadership Team. He specializes in architecture/design, C/C++, Python, Linux, shell scripting, encryption and security.
  • Zhihao Yuan is a Software Developer at Rackspace, and a member of the Technical Leadership Team. Zihihao has experience with FreeBSD, C++, Python, and more.
  • Bryan Davidson is a Software Developer at Rackspace. He has over 5 years of industry experience building web-scale applications.


API Design Contributors (no code submitted)

  • John Hopper is a Software Develop at Rackspace and has over 11 years of industry experience. John specializes in tackling large, complex models and prototyping solutions.
  • Hong Yuan is a Senior Software Engineer at HP, working on projects such as Messaging as a Service and Database as a Service. Hong specializes in software architecture and development with high scalability, high availability, and low latency.
  • Travis Reeder is the a Co-Founder and CTO at Iron.io. Travis specializes in building highly scalable applications, Go (golang), Ruby on Rails, and Java.


Infrastructure requirements (testing, etc)

Currently on StackForge using Gerrit and Jenkins to run the unit test suite and flake8. A QA cluster is nearing completion which will facilitate automated load and security testing, with infrastructure donated by Rackspace. The QA cluster will serve as a reference configuration for operators wishing to deploy a Marconi cluster. No additional infrastructure requirements are expected.

Have all current contributors agreed to the OpenStack CLA?

Yes.

Related Links

Documentation Marconi on OpenStack Wiki

Status