Jump to: navigation, search

ZeroMQ

Why ZeroMQ

0MQ (also known as ZeroMQ or zmq) looks like an embeddable networking library but acts like a concurrency framework. It gives you sockets that carry atomic messages across various transports like in-process, inter-process, TCP, and multicast. You can connect sockets N-to-N with patterns like fan-out, pub-sub, task distribution, and request-reply. It's fast enough to be the fabric for clustered products. Its asynchronous I/O model gives you scalable multi-core applications, built as asynchronous message-processing tasks. It has a score of language APIs and runs on most operating systems. 0MQ is from iMatix and is LGPLv3 open source.

Originally the zero in 0MQ was meant as "zero broker" and (as close to) "zero latency" (as possible). Since then, it has come to encompass different goals: zero administration, zero cost, and zero waste. More generally, "zero" refers to the culture of minimalism that permeates the project. We add power by removing complexity rather than by exposing new functionality.

Current Status

In Folsom, OpenStack introduced an optional messaging system using ZeroMQ. For some deployments, especially for those super-large scenarios, it may be desirable to use a broker-less RPC mechanism to scale out. Currently (in Juno cycle), ZeroMQ is one of the RPC backend drivers in oslo.messaging.

Other than AMQP-based drivers, like RabbitMQ or Qpid, ZeroMQ doesn't have any central brokers in oslo.messaging, instead, each host (running OpenStack services) is both ZeroMQ client and server. As a result, each host needs to listen to a certain TCP port for incoming connections and directly connect to other hosts simultaneously.

But unfortunately, ZeroMQ driver lacks of maintenance for a while that causes lots of bugs unhandled. So, welcome to anyone who is willing to contribute to this driver.

Current Bugs & Reviews

Title Priority Bug Link Review Link
zmq driver broken with cinder multi-backend High https://bugs.launchpad.net/oslo.messaging/+bug/1381972 https://review.openstack.org/129185
Unconditional usage of permissive import Low https://bugs.launchpad.net/oslo.messaging/+bug/1373350 https://review.openstack.org/123691
impl_zmq driver. Global CONF object without explicit options registration Low https://bugs.launchpad.net/oslo.messaging/+bug/1373382 https://review.openstack.org/123711
ZMQ IPC socket paths limited to 107 characters in length Low https://bugs.launchpad.net/oslo.messaging/+bug/1367614 https://review.openstack.org/#/c/124099
ZeroMQ driver doesn't support transport urls Low https://bugs.launchpad.net/oslo.messaging/+bug/1400323 None
Zmq does not default to sending rpc envelope Low https://bugs.launchpad.net/oslo.messaging/+bug/1187036 None
ZeroMQ driver fails to send casts if processing during exit Low https://bugs.launchpad.net/oslo.messaging/+bug/1097927 None
zmq driver includes passwords in its debug logs Low https://bugs.launchpad.net/oslo.messaging/+bug/1171556 None
ZeroMQ cast timeout ineffective Low https://bugs.launchpad.net/oslo.messaging/+bug/1193439 None

Blueprints & Future Plan

Title Link Status
DevStack support https://blueprints.launchpad.net/devstack/+spec/zeromq In Progress
Create zmq context once per driver instance https://blueprints.launchpad.net/oslo.messaging/+spec/zmq-context-per-driver-instance In Discussion
Control and manipulate ZeroMQ topic from a single object https://blueprints.launchpad.net/oslo.messaging/+spec/zmq-topic-object In Discussion
Make ZeroMQ driver work without eventlet https://blueprints.launchpad.net/oslo.messaging/+spec/zmq-work-without-eventlet In Discussion

DevStack Support

ZeroMQ driver has been supported by DevStack. The configuration is as follows:

ENABLED_SERVICES+=,-rabbit,-qpid,zeromq
ZEROMQ_MATCHMAKER=redis

Discussion

Title Link
nova-rpc-zmq-receiver bottleneck http://lists.openstack.org/pipermail/openstack-dev/2014-March/030411.html
Running devstack with zeromq http://lists.openstack.org/pipermail/openstack-dev/2014-June/038188.html
zeromq work for kilo http://lists.openstack.org/pipermail/openstack-dev/2014-September/046594.html
Zero MQ remove central broker. Architecture change http://lists.openstack.org/pipermail/openstack-dev/2014-November/050701.html
ZeroMQ driver maintenance next steps http://lists.openstack.org/pipermail/openstack-dev/2014-November/050896.html

Resources

Title Link
Presentation: Going Brokerless: The Transition from Qpid to 0mq https://www.openstack.org/summit/openstack-summit-hong-kong-2013/session-videos/presentation/going-brokerless-the-transition-from-qpid-to-0mq
Blog: Neutron & ZeroMQ: Network messaging gets in the fast lane and scales there https://www.mirantis.com/openstack-portal/external-tutorials/neutron-zeromq-network-messaging-gets-fast-lane-scales/
Slide: Openstack Messaging with ZMQ (Distributed RPC) http://www.slideshare.net/yatinkumbhare/zmq-in-context-of-openstack-31975405
Slide: Distributed RPC in Nova with ZeroMQ http://www.slideshare.net/randybias/distributed-rpc-in-nova-with-zeromq
Book: ZeroMQ Guide http://zguide.zeromq.org