|
|
| (3 intermediate revisions by 3 users not shown) |
| Line 1: |
Line 1: |
| − | __NOTOC__
| + | |
| | + | |
| | + | <pre><nowiki>#!wiki caution |
| | + | '''This page is outdated''' |
| | + | |
| | + | The content of this page has not been updated for a very long time. The Burrow project has not been updated for the current release. |
| | + | </nowiki></pre> |
| | + | |
| | + | |
| | = Queue Service = | | = Queue Service = |
| | | | |
| | This document is a proposal and plan for a queue service, it is not an official [[OpenStack]] project yet. Once there is an implementation to evaluate, it will be submitted to the [[OpenStack]] [[Governance/PPB|Project Policy Board]] for approval. | | This document is a proposal and plan for a queue service, it is not an official [[OpenStack]] project yet. Once there is an implementation to evaluate, it will be submitted to the [[OpenStack]] [[Governance/PPB|Project Policy Board]] for approval. |
| | | | |
| − | This project has been started on Launchpad under the name "Burrow". You can find it at: https://launchpad.net/burrow | + | This project has been started on Launchpad under the name "Burrow". You can find the source code, current tasks, and other information at: https://launchpad.net/burrow |
| | | | |
| − | Most of the docs that were once here are now located on the new burrow documentation site at: http://burrow.openstack.org/
| + | The docs that were once here are now located on the new burrow documentation site at: http://burrow.openstack.org/ |
| | | | |
| | == Possible Features in the Future == | | == Possible Features in the Future == |
| Line 13: |
Line 21: |
| | * Allow secondary indexes on user-defined metadata. | | * Allow secondary indexes on user-defined metadata. |
| | * Support in-process router "workers" for advanced routing and replication configurations. This will require API additions to manage the routing rules. | | * Support in-process router "workers" for advanced routing and replication configurations. This will require API additions to manage the routing rules. |
| − |
| |
| − | == Examples ==
| |
| − |
| |
| − | === Basic Asynchronous Queue ===
| |
| − |
| |
| − | Multiple workers long-poll for messages until a client inserts one. Both workers tell the server to hide the message once it is read, so only one worker will be able to see the message. The POST request from a worker is an atomic get/set operation.
| |
| − |
| |
| − |
| |
| − | <pre><nowiki>
| |
| − | Worker1: POST /account/queue?wait=60&hide=60&detail=all (long-polling worker, request blocks until a message is ready)
| |
| − | Worker2: POST /account/queue?wait=60&hide=60&detail=all (long-polling worker, request blocks until a message is ready)
| |
| − | Client: PUT /account/queue (message inserted, returns unique id that was created)
| |
| − | Worker1: Return from the blocking POST request with the new message and process it
| |
| − | Worker1: DELETE /account/queue/id
| |
| − | </nowiki></pre>
| |
| − |
| |
| − |
| |
| − | === Fast Asynchronous Queue ===
| |
| − |
| |
| − | In this example message loss is acceptable and workers have the ability to do batch-processing (process multiple messages at once). Workers grab up to 100 messages at a time and tell the server to remove the messages once they are returned. If a worker crashes after the server returns them but before they were successfully processed, they are lost. The DELETE request from a worker is an atomic get/delete operation.
| |
| − |
| |
| − |
| |
| − | <pre><nowiki>
| |
| − | Worker1: DELETE /account/queue?limit=100&wait=60&detail=all (long-polling worker, request blocks until a message is ready)
| |
| − | Worker2: DELETE /account/queue?limit=100&wait=60&detail=all (long-polling worker, request blocks until a message is ready)
| |
| − | Client: PUT /account/queue (message inserted, returns unique id that was created)
| |
| − | ...
| |
| − | Worker1: Return from the blocking DELETE request with all new messages and process them
| |
| − | </nowiki></pre>
| |
| − |
| |
| − |
| |
| − | === Multi-cast Event Notifications ===
| |
| − |
| |
| − | This allows multiple workers to read the same message since the message is not hidden or removed by any worker. The server will automatically remove the message once the message TTL has expired.
| |
| − |
| |
| − |
| |
| − | <pre><nowiki>
| |
| − | Worker1: GET /account/queue?wait=60
| |
| − | Worker2: GET /account/queue?wait=60
| |
| − | Client: Put /account/queue/id1?ttl=60
| |
| − | Worker1: Return from blocking GET request with message id1
| |
| − | Worker2: Return from blocking GET request with message id1
| |
| − | Worker1: GET /account/queue?wait=60&marker=id1
| |
| − | Worker2: GET /account/queue?wait=60&marker=id1
| |
| − | </nowiki></pre>
| |
| − |
| |
| − |
| |
| − | == Example Deployment Configurations ==
| |
| − |
| |
| − | Configuration of the queue service will be flexible and this section describes three sample deployments. Each of the queue servers (green boxes) can optionally have persistence configured for them. Note that queue servers should not share the same backend storage unless the backend-storage is suitable for the HA needs of the deployment. The queue servers will not know of or coordinate with one another in any way, the clients and workers (or proxy, in the last example) are responsible for balancing the load between available queue servers and failing over to another server if one goes down.
| |
| − |
| |
| − | === Developer ===
| |
| − |
| |
| − | In this deployment, a single queue server is run, and all clients and workers connect to it.
| |
| − |
| |
| − | [[Image:QueueService$queue_service_dev.png]]
| |
| − |
| |
| − | === Simple HA ===
| |
| − |
| |
| − | In this deployment, multiple queue servers are run and all clients and workers are given the list of IP addresses for the available queue workers. Clients should either connect to the first available server, or if it wants to distribute load amongst all three, should use some algorithm depending on message. If all messages are unique, a simple round-robin distribution may be sufficient. For messages with client-side IDs that could possibly have duplicates, the client should use a hashing algorithm on the ID to ensure the same ID hits the same server. This allows the queue services to coalesce all messages with the same ID. Should a queue server go down, the clients can simply re-balance to the new server set. Workers should poll or remain connected to all queue servers to ensure it can pull a message no mater where it is inserted.
| |
| − |
| |
| − | [[Image:QueueService$queue_service_ha.png]]
| |
| − |
| |
| − | === Public Cloud Service ===
| |
| − |
| |
| − | In this deployment, proxy servers are placed in front of each cluster of queue servers. The proxy servers manage spreading the load across the queue cluster instead of relying on the clients and workers to manage multiple connections. This is only suitable when your proxy servers are configured in a redundantly (such as when using a HA load balancer). For a given account ID, all proxy servers in a zone should hash to the same subset of queue workers (with a default max of three), and use that set to spread load across. This is similar to how Swift spreads objects based on the placement in the hash ring. Once the account ID determines the set of queue servers to use, the queue name and message ID (other components of the unique message ID) will determine which queue server in the set to use. The algorithm used in the proxy should be modular, so you can easily alter how many queue servers to use for an account, and how to distribute to them within that set.
| |
| − |
| |
| − | [[Image:QueueService$queue_service_pub.png]]
| |
| − |
| |
| − | For example, if a client PUTs a message with ID /v1.0/account_1/queue_A/msg_123, the proxy server will parse out the "account_1" component and use that in the hashing algorithm to get a set of queue servers (lets say it returns the set qs1, qs4, qs5). With this set, the proxy then hashes the rest of the ID "queue_A/msg_123" to determine which queue server to proxy to (lets say it maps to qs4). If a message comes in with the exact same ID, the same algorithm is used to proxy it to the same queue server, possibly allowing the queue server to coalesces the message so it is processed by a worker only once (eliminating the thundering herd problem). If a queue server in the returned set should fail, it can either run with two servers or choose a third server until the original node comes back up.
| |
| − |
| |
| − | When the proxy is handling worker requests it will use the same hashing algorithms. When a worker GETs a queue name to read messages, the account portion is parsed and a connection is made to all queue servers. It will then aggregate messages from all queue servers handling that account into one view for the worker to consume. The proxy and queue servers may need to use a more efficient multiplexing protocol that can keep state for multiple accounts and requests rather than simple REST based calls to keep the number of connections reasonable.
| |
| | | | |
| | ---- | | ---- |
| | [[Category:RelatedProjects]] | | [[Category:RelatedProjects]] |
This project has been started on Launchpad under the name "Burrow". You can find the source code, current tasks, and other information at: https://launchpad.net/burrow
The docs that were once here are now located on the new burrow documentation site at: http://burrow.openstack.org/
