Zaqar/specs/sharding/v1

= Zaqar Storage Pooling: v1 Blueprint =

Overview
This proposal introduces another approach to tackle making Zaqar highly scalable without losing any of its flexibility. This particular document covers the administrative API for registering storage pools.

Reference
For authoritative header/general details, please refer to the Zaqar Queues API v1 Spec.

Endpoints synopsis
GET /pools?detailed=False&marker=None&limit=10 GET /pools/{pool}?detailed=False

PUT /pools/{pool} DELETE /pools/{pool} PATCH /pools/{pool}

GET /health

Pooling with Zaqar
Zaqar supports heterogeneous pooling through the use of:


 * stevedore for dynamic storage driver loading
 * node weights

As long as an entry point is defined in an installed module that matches the scheme of a pool connection URI, then Zaqar will be able to use that pool. For example, a pool entry might look like:

{ "weight": 100, "uri": "mongodb://localhost:27017", "options": { "max_retry_sleep": 1 } }

Register a pool
Template

PUT /v1/pools/{pool}

Request

PUT /v1/pools/wat HTTP/1.1 Host: zaqar.example.com

{ "weight": 100, "uri": "mongodb://localhost:27017", "options": { "max_retry_sleep": 1, "partitions": 8 } }

Response

HTTP/1.1 201 Created Location: /v1/pools/wat

Discussion

Register a pool.

is the name to give the pool entry. The name MUST NOT exceed 64 bytes in length, and is limited to US-ASCII letters, digits, underscores and hyphens.

is the likelihood that this pool will be selected for the next queue allocation. It must be an integer greater than -1.

is a connection string compatible with a storage client (e.g., pymongo) attempting to connect to that pool.

An optional request component that gives storage-specific options used by storage driver implementations. Valid parameters come from the registered options for a given storage backend, for example: mongodb, sqlite

Read a pool
Template

GET /v1/pools/{pool}?detailed=True

Request

GET /v1/pools/wat HTTP/1.1 Host: zaqar.example.com

Response

HTTP/1.1 200 OK Content-Location: /v1/pools/wat

{ "uri": "mongodb://zaqar1.example.com:27017", "weight": 100 }

Discussion

Returns information on a registered pool.

Delete a pool
Template

DELETE /v1/pool/{pool}

Request

DELETE /v1/pools/wat HTTP/1.1 Host: zaqar.example.com

Response

HTTP/1.1 204 No Content

Discussion

Removes a pool from the registry.

Update a pool
Template

PATCH /v1/pools/{pool}

Request

PATCH /v1/pools/wat HTTP/1.1 Host: zaqar.example.com

{ "uri": "mongodb://zaqar3.example.com:27018", "weight": 120 }

Response

HTTP/1.1 204 No Content

Discussion

Allows one to update any or all of: `weight`, `uri`, `options`. At least one of these fields must be specified, else, a HTTP 400 is returned.

List pools
Template

GET /v1/pools?detailed=True&limit=10&marker=taco

Request

GET /v1/pools HTTP/1.1 Host: zaqar.proxy.example.com

Response

HTTP/1.1 200 OK Content-Location: /v1/pools

[ {"href": "/v1/pools/wat", "weight": 100, "uri": "mongodb://zaqar1.example.com:27017"}, {"href": "/v1/pools/wot", "weight": 50, "uri": "redis://zaqar2.example.com:6379"} ]

Discussion

Lists the registered pools.

if True, returns the options field in the listing. used for pagination - what pool do we start listing from? how many entries to return per request?