Jump to: navigation, search

Difference between revisions of "Zaqar/specs/proxy/v1"

(Register a Partition: Add parameters descriptions.)
(Register a Partition)
Line 94: Line 94:
  
 
<code><nowiki>partition_name</nowiki></code> is the name to give the queue. The name MUST NOT exceed 64 bytes in length, and is limited to US-ASCII letters, digits, underscores and hyphens.
 
<code><nowiki>partition_name</nowiki></code> is the name to give the queue. The name MUST NOT exceed 64 bytes in length, and is limited to US-ASCII letters, digits, underscores and hyphens.
 +
 
<code><nowiki>weight</nowiki></code> is the likelihood that this partition will be selected for the next queue allocation. It must be greater than 0.
 
<code><nowiki>weight</nowiki></code> is the likelihood that this partition will be selected for the next queue allocation. It must be greater than 0.
 +
 
<code><nowiki>hosts</nowiki></code> is a list of marconi-queues endpoints associated with this partition.
 
<code><nowiki>hosts</nowiki></code> is a list of marconi-queues endpoints associated with this partition.
  

Revision as of 14:31, 26 September 2013

Marconi Proxy: v1 Blueprint

Overview

Marconi's proxy service was written to enable scaling out a Marconi deployment at the partition level. It is controlled through an HTTP API.

Reference

For authoritative header/general details, please refer to the Marconi Queues API v1 Spec.

Endpoints Synopsis

# Proxy-specific endpoints
## Partition operations
GET /partitions
GET /partitions/{partition}
PUT /partitions/{partition}
DELETE /partitions/{partition}

## Catalogue operations
GET /catalogue
GET /catalogue/{queue}
GET /health

# Queues API intercepting endpoints
## Implicitly modify catalogue
GET /queues
GET /queues/{queue}
PUT /queues/{queue}
DELETE /queues/{queue}

# Operations forwarded to Queues API
## Informative
GET /v1

## Claims-forwarding
POST /queues/{queue}/claims
GET /queues/{queue}/claims/{claim_id}
PATCH /queues/{queue}/claims/{claim_id}
DELETE /queues/{queue}/claims/{claim_id}

## Messages forwarding
GET /queues/{queue}/messages
POST /queues/{queue}/messages
DELETE /queues/{queue}/messages
GET /queues/{queue}/messages/{message_id}
DELETE /queues/{queue}/messages/{message_id}

## Metadata handling - also implicitly modifies catalogue
GET /queues/{queue}/metadata
PUT /queues/{queue}/metadata

## Queue stats
GET /queues/{queue}/stats

Handling Partitions

A partition maps a name to a list of nodes associated with that partition and a weight. The weight is used to determine how likely a partition is to be selected as the storage location for a particular queue.

Register a Partition

Template

PUT /v1/partitions/{partition_name}

Request

PUT /v1/partitions/wat HTTP/1.1
Host: marconi.proxy.example.com

{
  "hosts": ["http://marconi1.example.com", "http://marconi2.example.com"],
  "weight": 100
}


Response

HTTP/1.1 201 Created
Location: /v1/partitions/wat


Discussion

Registers a partition.

partition_name is the name to give the queue. The name MUST NOT exceed 64 bytes in length, and is limited to US-ASCII letters, digits, underscores and hyphens.

weight is the likelihood that this partition will be selected for the next queue allocation. It must be greater than 0.

hosts is a list of marconi-queues endpoints associated with this partition.

Read a Partition

Template

GET /v1/partitions/{partition_name}

Request

GET /v1/partitions/wat HTTP/1.1
Host: marconi.proxy.example.com


Response

HTTP/1.1 200 OK
Location: /v1/partitions/wat

{
  "hosts": ["http://marconi1.example.com", "http://marconi2.example.com"],
  "weight": 100
}


Discussion

Returns information on a registered partition.

Delete a Partition

Template

DELETE /v1/partitions/{partition_name}

Request

DELETE /v1/partitions/wat HTTP/1.1
Host: marconi.proxy.example.com


Response

HTTP/1.1 204 No Content

Discussion

Removes a partition from the registry.

List Partitions

Template

GET /v1/partitions

Request

GET /v1/partitions HTTP/1.1
Host: marconi.proxy.example.com


Response

HTTP/1.1 200 OK
Location: /v1/partitions

{
    "wat": {
        "hosts": [
            "http://marconi1.example.com", 
            "http://marconi2.example.com"
        ], 
        "weight": 256
    }, 
    "wat2": {
        "hosts": [
            "http://marconi3.example.com", 
            "http://marconi4.example.com"
            "http://marconi5.example.com", 
            "http://marconi6.example.com"
        ], 
        "weight": 500
    }, 
    "wat3": {
        "hosts": [
        ], 
            "http://marconi7.example.com"
        "weight": 25
    }
}

Discussion

Removes a partition from the registry.

Handling the Catalogue

The catalogue associates a queue + project_id to a partition name. It also tracks metadata for that particular queue.

The catalogue is immutable, as far as the direct API allows. Modifications to the catalogue occur as a side effect to forwarding operations to a Marconi queues partition.

Read a Catalogue Entry

Template

GET /v1/catalogue/{queue_name}

Request

GET /v1/catalogue/taco HTTP/1.1
Host: marconi.proxy.example.com


Response

HTTP/1.1 200 OK
Location: /v1/catalogue/taco

{
  "name": "taco",
  "metadata": {},
  "partition": "wat",
  "host": "http://marconi1.example.com"
}


Discussion

Returns information on a captured catalogue entry.

NOTE: should a host be returned in the representation?

Listing the Catalogue

Template

GET /v1/catalogue

Request

GET /v1/catalogue HTTP/1.1
Host: marconi.proxy.example.com


Response

HTTP/1.1 200 OK
Location: /v1/catalogue

{
  "taco": {
    "name": "taco",
    "metadata": {},
    "partition": "wat",
    "host": "http://marconi1.example.com"
  },
  "lettuce": {...},
  "rice": {...}
}


Discussion

Lists the entries stored in the catalogue. Paging is not supported at this time.

Checking Health

If you need to determine whether a particular proxy is up, this endpoint exists for that sole purpose with no side effects to the system.

Check Node Health

Template

GET /v1/health

or

HEAD /v1/health


Request

GET /v1/health HTTP/1.1
Host: example.marconi.proxy.com

...


Response

HTTP/1.1 204 No Content

or

HTTP/1.1 503 Service Unavailable


Discussion

Use this request to check on the Marconi proxy node status. If the node is down, this endpoint will not be reachable and will act as a signal to the load balancer to take the node out of rotation. Alternatively, the endpoint may be reachable but the service is temporarily unavailable due to storage driver failure or some other error.

Forwarding Endpoints

The remaining endpoints all forward to a Marconi partition to receive an authoritative response. The following sections describe endpoints that trigger special behavior in the proxy.

Metadata Management

PUT on /v1/queues/{queue}/metadata updates the corresponding catalogue entry.

Queue Creation/Deletion

PUT|DELETE on /v1/queues/{queue} respectively create the queue on and delete the queue from the catalogue.

Queue Listing

GET /v1/queues requires that the proxy make the authoritative response, since no individual partition has the complete answer to the question, "What queues have I created?".

Everything Else

The request is forwarded through untouched. The client connecting through the proxy receives the response as given by the Marconi partition. The behavior should be as if the proxy did not exist.