Difference between revisions of "ZeroMQ"
Line 19: | Line 19: | ||
! Title !! Priority !! Bug Link !! Review Link | ! Title !! Priority !! Bug Link !! Review Link | ||
|- | |- | ||
− | | | + | | RuntimeError: dictionary changed size during iteration || Medium || https://bugs.launchpad.net/oslo.messaging/+bug/1432966 || https://review.openstack.org/164972 |
|- | |- | ||
| zmq driver broken with cinder multi-backend || Medium || https://bugs.launchpad.net/oslo.messaging/+bug/1381972 || https://review.openstack.org/129185 | | zmq driver broken with cinder multi-backend || Medium || https://bugs.launchpad.net/oslo.messaging/+bug/1381972 || https://review.openstack.org/129185 | ||
Line 66: | Line 66: | ||
=== DevStack Support === | === DevStack Support === | ||
ZeroMQ driver has been supported by DevStack. The configuration is as follows: | ZeroMQ driver has been supported by DevStack. The configuration is as follows: | ||
− | + | ENABLED_SERVICES+=,-rabbit,-qpid,zeromq | |
− | + | ZEROMQ_MATCHMAKER=redis | |
=== Discussion === | === Discussion === |
Revision as of 07:08, 17 March 2015
Contents
ZeroMQ Revival
Why ZeroMQ
0MQ (also known as ZeroMQ or zmq) looks like an embeddable networking library but acts like a concurrency framework. It gives you sockets that carry atomic messages across various transports like in-process, inter-process, TCP, and multicast. You can connect sockets N-to-N with patterns like fan-out, pub-sub, task distribution, and request-reply. It's fast enough to be the fabric for clustered products. Its asynchronous I/O model gives you scalable multi-core applications, built as asynchronous message-processing tasks. It has a score of language APIs and runs on most operating systems. 0MQ is from iMatix and is LGPLv3 open source.
Originally the zero in 0MQ was meant as "zero broker" and (as close to) "zero latency" (as possible). Since then, it has come to encompass different goals: zero administration, zero cost, and zero waste. More generally, "zero" refers to the culture of minimalism that permeates the project. We add power by removing complexity rather than by exposing new functionality.
Current Status
In Folsom, OpenStack introduced an optional messaging system using ZeroMQ. For some deployments, especially for those super-large scenarios, it may be desirable to use a broker-less RPC mechanism to scale out. Currently (in Juno cycle), ZeroMQ is one of the RPC backend drivers in oslo.messaging.
Other than AMQP-based drivers, like RabbitMQ or Qpid, ZeroMQ doesn't have any central brokers in oslo.messaging, instead, each host (running OpenStack services) is both ZeroMQ client and server. As a result, each host needs to listen to a certain TCP port for incoming connections and directly connect to other hosts simultaneously.
But unfortunately, ZeroMQ driver lacks of maintenance for a while that causes lots of bugs unhandled. So, welcome to anyone who is willing to contribute to this driver.
Current Bugs & Reviews
Blueprints & Future Plan
Title | Link | Status |
---|---|---|
DevStack support | https://blueprints.launchpad.net/devstack/+spec/zeromq | In Progress |
Create zmq context once per driver instance | https://blueprints.launchpad.net/oslo.messaging/+spec/zmq-context-per-driver-instance | In Discussion |
Control and manipulate ZeroMQ topic from a single object | https://blueprints.launchpad.net/oslo.messaging/+spec/zmq-topic-object | In Discussion |
Make ZeroMQ driver work without eventlet | https://blueprints.launchpad.net/oslo.messaging/+spec/zmq-work-without-eventlet | In Discussion |
DevStack Support
ZeroMQ driver has been supported by DevStack. The configuration is as follows: ENABLED_SERVICES+=,-rabbit,-qpid,zeromq ZEROMQ_MATCHMAKER=redis
Discussion
Title | Link |
---|---|
nova-rpc-zmq-receiver bottleneck | http://lists.openstack.org/pipermail/openstack-dev/2014-March/030411.html |
Running devstack with zeromq | http://lists.openstack.org/pipermail/openstack-dev/2014-June/038188.html |
zeromq work for kilo | http://lists.openstack.org/pipermail/openstack-dev/2014-September/046594.html |
Zero MQ remove central broker. Architecture change | http://lists.openstack.org/pipermail/openstack-dev/2014-November/050701.html |
ZeroMQ driver maintenance next steps | http://lists.openstack.org/pipermail/openstack-dev/2014-November/050896.html |
Resources
Title | Link |
---|---|
Presentation: Going Brokerless: The Transition from Qpid to 0mq | https://www.openstack.org/summit/openstack-summit-hong-kong-2013/session-videos/presentation/going-brokerless-the-transition-from-qpid-to-0mq |
Slide: Openstack Messaging with ZMQ (Distributed RPC) | http://www.slideshare.net/yatinkumbhare/zmq-in-context-of-openstack-31975405 |
Book: ZeroMQ Guide | http://zguide.zeromq.org |