Zaqar/specs/proxy/v1

= Marconi Proxy: v1 Blueprint =

Overview
Marconi's proxy service was written to enable scaling out a Marconi deployment at the partition level. It is controlled through an HTTP API.

Reference
For authoritative header/general details, please refer to the Marconi Queues API v1 Spec.

Endpoints Synopsis
GET /partitions GET /partitions/{partition} PUT /partitions/{partition} DELETE /partitions/{partition}
 * 1) Proxy-specific endpoints
 * 2) Partition operations

GET /catalogue GET /catalogue/{queue} GET /health
 * 1) Catalogue operations

GET /queues GET /queues/{queue} PUT /queues/{queue} DELETE /queues/{queue}
 * 1) Queues API intercepting endpoints
 * 2) Implicitly modify catalogue

GET /v1
 * 1) Operations forwarded to Queues API
 * 2) Informative

POST /queues/{queue}/claims GET /queues/{queue}/claims/{claim_id} PATCH /queues/{queue}/claims/{claim_id} DELETE /queues/{queue}/claims/{claim_id}
 * 1) Claims-forwarding

GET /queues/{queue}/messages POST /queues/{queue}/messages DELETE /queues/{queue}/messages GET /queues/{queue}/messages/{message_id} DELETE /queues/{queue}/messages/{message_id}
 * 1) Messages forwarding

GET /queues/{queue}/metadata PUT /queues/{queue}/metadata
 * 1) Metadata handling - also implicitly modifies catalogue

GET /queues/{queue}/stats
 * 1) Queue stats

Handling Partitions
A partition maps a name to a list of nodes associated with that partition and a weight. The weight is used to determine how likely a partition is to be selected as the storage location for a particular queue.

Register a Partition
Template

PUT /v1/partitions/{partition_name}

Request

PUT /v1/partitions/wat HTTP/1.1 Host: marconi.proxy.example.com

{ "hosts": ["http://marconi1.example.com", "http://marconi2.example.com"], "weight": 100 }

Response

HTTP/1.1 201 Created Location: /v1/partitions/wat

Discussion

Registers a partition.

is the name to give the queue. The name MUST NOT exceed 64 bytes in length, and is limited to US-ASCII letters, digits, underscores and hyphens.

is the likelihood that this partition will be selected for the next queue allocation. It must be greater than 0.

is a list of marconi-queues endpoints associated with this partition.

Read a Partition
Template

GET /v1/partitions/{partition_name}

Request

GET /v1/partitions/wat HTTP/1.1 Host: marconi.proxy.example.com

Response

HTTP/1.1 200 OK Location: /v1/partitions/wat

{ "hosts": ["http://marconi1.example.com", "http://marconi2.example.com"], "weight": 100 }

Discussion

Returns information on a registered partition.

Delete a Partition
Template

DELETE /v1/partitions/{partition_name}

Request

DELETE /v1/partitions/wat HTTP/1.1 Host: marconi.proxy.example.com

Response

HTTP/1.1 204 No Content

Discussion

Removes a partition from the registry.

List Partitions
Template

GET /v1/partitions

Request

GET /v1/partitions HTTP/1.1 Host: marconi.proxy.example.com

Response

HTTP/1.1 200 OK Location: /v1/partitions

{   "wat": { "hosts": [ "http://marconi1.example.com", "http://marconi2.example.com" ],        "weight": 256 },    "wat2": { "hosts": [ "http://marconi3.example.com", "http://marconi4.example.com" "http://marconi5.example.com", "http://marconi6.example.com" ],        "weight": 500 },    "wat3": { "hosts": [ ],            "http://marconi7.example.com" "weight": 25 } }

Discussion

Removes a partition from the registry.

Handling the Catalogue
The catalogue associates a queue + project_id to a partition name. It also tracks metadata for that particular queue.

The catalogue is immutable, as far as the direct API allows. Modifications to the catalogue occur as a side effect to forwarding operations to a Marconi queues partition.

Read a Catalogue Entry
Template

GET /v1/catalogue/{queue_name}

Request

GET /v1/catalogue/taco HTTP/1.1 Host: marconi.proxy.example.com

Response

HTTP/1.1 200 OK Location: /v1/catalogue/taco

{ "name": "taco", "metadata": {}, "partition": "wat", "host": "http://marconi1.example.com" }

Discussion

Returns information on a captured catalogue entry.



Listing the Catalogue
Template

GET /v1/catalogue

Request

GET /v1/catalogue HTTP/1.1 Host: marconi.proxy.example.com

Response

HTTP/1.1 200 OK Location: /v1/catalogue

{ "taco": { "name": "taco", "metadata": {}, "partition": "wat", "host": "http://marconi1.example.com" }, "lettuce": {...}, "rice": {...} }

Discussion

Lists the entries stored in the catalogue. Paging is not supported at this time.

Checking Health
If you need to determine whether a particular proxy is up, this endpoint exists for that sole purpose with no side effects to the system.

Check Node Health
Template

GET /v1/health

or

HEAD /v1/health

Request

GET /v1/health HTTP/1.1 Host: example.marconi.proxy.com

...

Response

HTTP/1.1 204 No Content

or

HTTP/1.1 503 Service Unavailable

Discussion

Use this request to check on the Marconi proxy node status. If the node is down, this endpoint will not be reachable and will act as a signal to the load balancer to take the node out of rotation. Alternatively, the endpoint may be reachable but the service is temporarily unavailable due to storage driver failure or some other error.

Forwarding Endpoints
The remaining endpoints all forward to a Marconi partition to receive an authoritative response. The following sections describe endpoints that trigger special behavior in the proxy.

Metadata Management
PUT on /v1/queues/{queue}/metadata updates the corresponding catalogue entry.

Queue Creation/Deletion
PUT|DELETE on /v1/queues/{queue} respectively create the queue on and delete the queue from the catalogue.

Queue Listing
GET /v1/queues requires that the proxy make the authoritative response, since no individual partition has the complete answer to the question, "What queues have I created?".

Everything Else
The request is forwarded through untouched. The client connecting through the proxy receives the response as given by the Marconi partition. The behavior should be as if the proxy did not exist.