Zaqar/specs/proxy/v1
Contents
Marconi Proxy: v1 Blueprint
Overview
Marconi's proxy service was written to enable scaling out a Marconi deployment at the partition level. It is controlled through an HTTP API.
Reference
For authoritative header/general details, please refer to the Marconi Queues API v1 Spec.
Endpoints Synopsis
# Proxy-specific endpoints ## Partition operations GET /partitions GET /partitions/{partition} PUT /partitions/{partition} DELETE /partitions/{partition} ## Catalogue operations GET /catalogue GET /catalogue/{queue} GET /health # Queues API intercepting endpoints ## Implicitly modify catalogue GET /queues GET /queues/{queue} PUT /queues/{queue} DELETE /queues/{queue} # Operations forwarded to Queues API ## Informative GET /v1 ## Claims-forwarding POST /queues/{queue}/claims GET /queues/{queue}/claims/{claim_id} PATCH /queues/{queue}/claims/{claim_id} DELETE /queues/{queue}/claims/{claim_id} ## Messages forwarding GET /queues/{queue}/messages POST /queues/{queue}/messages DELETE /queues/{queue}/messages GET /queues/{queue}/messages/{message_id} DELETE /queues/{queue}/messages/{message_id} ## Metadata handling - also implicitly modifies catalogue GET /queues/{queue}/metadata PUT /queues/{queue}/metadata ## Queue stats GET /queues/{queue}/stats
Handling Partitions
A partition maps a name to a list of nodes associated with that partition and a weight. The weight is used to determine how likely a partition is to be selected as the storage location for a particular queue.
Register a Partition
Template
PUT /v1/partitions/{partition_name}
Request
PUT /v1/partitions/wat HTTP/1.1 Host: marconi.proxy.example.com { "hosts": ["http://marconi1.example.com", "http://marconi2.example.com"], "weight": 100 }
Response
HTTP/1.1 201 Created Location: /v1/partitions/wat
Discussion
Registers a partition.
partition_name
is the name to give the queue. The name MUST NOT exceed 64 bytes in length, and is limited to US-ASCII letters, digits, underscores and hyphens.
Read a Partition
Template
GET /v1/partitions/{partition_name}
Request
GET /v1/partitions/wat HTTP/1.1 Host: marconi.proxy.example.com
Response
HTTP/1.1 200 OK Location: /v1/partitions/wat { "hosts": ["http://marconi1.example.com", "http://marconi2.example.com"], "weight": 100 }
Discussion
Returns information on a registered partition.
Delete a Partition
Template
DELETE /v1/partitions/{partition_name}
Request
DELETE /v1/partitions/wat HTTP/1.1 Host: marconi.proxy.example.com
Response
HTTP/1.1 204 No Content
Discussion
Removes a partition from the registry.
List Partitions
Template
GET /v1/partitions
Request
GET /v1/partitions HTTP/1.1 Host: marconi.proxy.example.com
Response
HTTP/1.1 200 OK Location: /v1/partitions { "wat": { "hosts": [ "http://marconi1.example.com", "http://marconi2.example.com" ], "weight": 256 }, "wat2": { "hosts": [ "http://marconi3.example.com", "http://marconi4.example.com" "http://marconi5.example.com", "http://marconi6.example.com" ], "weight": 500 }, "wat3": { "hosts": [ ], "http://marconi7.example.com" "weight": 25 } }
Discussion
Removes a partition from the registry.
Handling the Catalogue
The catalogue associates a queue + project_id to a partition name. It also tracks metadata for that particular queue.
The catalogue is immutable, as far as the direct API allows. Modifications to the catalogue occur as a side effect to forwarding operations to a Marconi queues partition.
Read a Catalogue Entry
Template
GET /v1/catalogue/{queue_name}
Request
GET /v1/catalogue/taco HTTP/1.1 Host: marconi.proxy.example.com
Response
HTTP/1.1 200 OK Location: /v1/catalogue/taco { "name": "taco", "metadata": {}, "partition": "wat", "host": "http://marconi1.example.com" }
Discussion
Returns information on a captured catalogue entry.
NOTE
: should a host
be returned in the representation?
Listing the Catalogue
Template
GET /v1/catalogue
Request
GET /v1/catalogue HTTP/1.1 Host: marconi.proxy.example.com
Response
HTTP/1.1 200 OK Location: /v1/catalogue { "taco": { "name": "taco", "metadata": {}, "partition": "wat", "host": "http://marconi1.example.com" }, "lettuce": {...}, "rice": {...} }
Discussion
Lists the entries stored in the catalogue. Paging is not supported at this time.
Checking Health
If you need to determine whether a particular proxy is up, this endpoint exists for that sole purpose with no side effects to the system.
Check Node Health
Template
GET /v1/health
or
HEAD /v1/health
Request
GET /v1/health HTTP/1.1 Host: example.marconi.proxy.com ...
Response
HTTP/1.1 204 No Content
or
HTTP/1.1 503 Service Unavailable
Discussion
Use this request to check on the Marconi proxy node status. If the node is down, this endpoint will not be reachable and will act as a signal to the load balancer to take the node out of rotation. Alternatively, the endpoint may be reachable but the service is temporarily unavailable due to storage driver failure or some other error.
Forwarding Endpoints
The remaining endpoints all forward to a Marconi partition to receive an authoritative response. The following sections describe endpoints that trigger special behavior in the proxy.
Metadata Management
PUT on /v1/queues/{queue}/metadata updates the corresponding catalogue entry.
Queue Creation/Deletion
PUT|DELETE on /v1/queues/{queue} respectively create the queue on and delete the queue from the catalogue.
Queue Listing
GET /v1/queues requires that the proxy make the authoritative response, since no individual partition has the complete answer to the question, "What queues have I created?".