Difference between revisions of "Zaqar/specs/sharding/v1"
m (Malini moved page Marconi/specs/sharding/v1 to Zaqar/specs/sharding/v1: Project Rename) |
(Replace shards for pools and rename Marconi to Zaqar) |
||
Line 1: | Line 1: | ||
− | = | + | = Zaqar Storage Pooling: v1 Blueprint = |
== Overview == | == Overview == | ||
− | This proposal introduces another approach to tackle making | + | This proposal introduces another approach to tackle making Zaqar highly scalable without losing any of its flexibility. This particular document covers the administrative API for registering storage pools. |
== Reference == | == Reference == | ||
− | For authoritative header/general details, please refer to the [https://wiki.openstack.org/wiki/ | + | For authoritative header/general details, please refer to the [https://wiki.openstack.org/wiki/Zaqar/specs/api/v1 Zaqar Queues API v1 Spec]. |
− | == Endpoints | + | == Endpoints synopsis == |
<pre><nowiki> | <pre><nowiki> | ||
− | GET / | + | GET /pools?detailed=False&marker=None&limit=10 |
− | GET / | + | GET /pools/{pool}?detailed=False |
− | PUT / | + | PUT /pools/{pool} |
− | DELETE / | + | DELETE /pools/{pool} |
− | PATCH / | + | PATCH /pools/{pool} |
GET /health | GET /health | ||
</nowiki></pre> | </nowiki></pre> | ||
− | == | + | == Pooling with Zaqar == |
− | + | Zaqar supports heterogeneous pooling through the use of: | |
* stevedore for dynamic storage driver loading | * stevedore for dynamic storage driver loading | ||
* node weights | * node weights | ||
− | As long as an entry point is defined in an installed module that matches the scheme of a | + | As long as an entry point is defined in an installed module that matches the scheme of a pool connection URI, then Zaqar will be able to use that pool. For example, a pool entry might look like: |
<pre><nowiki> | <pre><nowiki> | ||
Line 41: | Line 41: | ||
</nowiki></pre> | </nowiki></pre> | ||
− | === Register a | + | === Register a pool === |
'''Template''' | '''Template''' | ||
<pre><nowiki> | <pre><nowiki> | ||
− | PUT /v1/ | + | PUT /v1/pools/{pool} |
</nowiki></pre> | </nowiki></pre> | ||
Line 52: | Line 52: | ||
<pre><nowiki> | <pre><nowiki> | ||
− | PUT /v1/ | + | PUT /v1/pools/wat HTTP/1.1 |
− | Host: | + | Host: zaqar.example.com |
{ | { | ||
Line 70: | Line 70: | ||
<pre><nowiki> | <pre><nowiki> | ||
HTTP/1.1 201 Created | HTTP/1.1 201 Created | ||
− | Location: /v1/ | + | Location: /v1/pools/wat |
</nowiki></pre> | </nowiki></pre> | ||
Line 76: | Line 76: | ||
'''Discussion''' | '''Discussion''' | ||
− | + | Register a pool. | |
− | <code><nowiki> | + | <code><nowiki>pool</nowiki></code> is the name to give the pool entry. The name MUST NOT exceed 64 bytes in length, and is limited to US-ASCII letters, digits, underscores and hyphens. |
− | <code><nowiki>weight</nowiki></code> is the likelihood that this | + | <code><nowiki>weight</nowiki></code> is the likelihood that this pool will be selected for the next queue allocation. It must be an integer greater than -1. |
− | <code><nowiki>uri</nowiki></code> is a connection string compatible with a storage client (e.g., pymongo) attempting to connect to that | + | <code><nowiki>uri</nowiki></code> is a connection string compatible with a storage client (e.g., pymongo) attempting to connect to that pool. |
<code><nowiki>options</nowiki></code> An optional request component that gives storage-specific options used by storage driver implementations. Valid parameters come from the registered options for a given storage backend, for example: [https://github.com/openstack/marconi/blob/master/marconi/queues/storage/mongodb/options.py mongodb], [https://github.com/openstack/marconi/blob/master/marconi/queues/storage/sqlite/driver.py#L29 sqlite] | <code><nowiki>options</nowiki></code> An optional request component that gives storage-specific options used by storage driver implementations. Valid parameters come from the registered options for a given storage backend, for example: [https://github.com/openstack/marconi/blob/master/marconi/queues/storage/mongodb/options.py mongodb], [https://github.com/openstack/marconi/blob/master/marconi/queues/storage/sqlite/driver.py#L29 sqlite] | ||
− | === Read a | + | === Read a pool === |
'''Template''' | '''Template''' | ||
<pre><nowiki> | <pre><nowiki> | ||
− | GET /v1/ | + | GET /v1/pools/{pool}?detailed=True |
</nowiki></pre> | </nowiki></pre> | ||
Line 97: | Line 97: | ||
<pre><nowiki> | <pre><nowiki> | ||
− | GET /v1/ | + | GET /v1/pools/wat HTTP/1.1 |
− | Host: | + | Host: zaqar.example.com |
</nowiki></pre> | </nowiki></pre> | ||
Line 106: | Line 106: | ||
<pre><nowiki> | <pre><nowiki> | ||
HTTP/1.1 200 OK | HTTP/1.1 200 OK | ||
− | Content-Location: /v1/ | + | Content-Location: /v1/pools/wat |
{ | { | ||
− | "uri": "mongodb:// | + | "uri": "mongodb://zaqar1.example.com:27017", |
"weight": 100 | "weight": 100 | ||
} | } | ||
Line 117: | Line 117: | ||
'''Discussion''' | '''Discussion''' | ||
− | Returns information on a registered | + | Returns information on a registered pool. |
− | === Delete a | + | === Delete a pool === |
'''Template''' | '''Template''' | ||
<pre><nowiki> | <pre><nowiki> | ||
− | DELETE /v1/ | + | DELETE /v1/pool/{pool} |
</nowiki></pre> | </nowiki></pre> | ||
Line 130: | Line 130: | ||
<pre><nowiki> | <pre><nowiki> | ||
− | DELETE /v1/ | + | DELETE /v1/pools/wat HTTP/1.1 |
− | Host: | + | Host: zaqar.example.com |
</nowiki></pre> | </nowiki></pre> | ||
Line 143: | Line 143: | ||
'''Discussion''' | '''Discussion''' | ||
− | Removes a | + | Removes a pool from the registry. |
− | === Update a | + | === Update a pool === |
'''Template''' | '''Template''' | ||
<pre><nowiki> | <pre><nowiki> | ||
− | PATCH /v1/ | + | PATCH /v1/pools/{pool} |
</nowiki></pre> | </nowiki></pre> | ||
Line 156: | Line 156: | ||
<pre><nowiki> | <pre><nowiki> | ||
− | PATCH /v1/ | + | PATCH /v1/pools/wat HTTP/1.1 |
− | Host: | + | Host: zaqar.example.com |
{ | { | ||
− | "uri": "mongodb:// | + | "uri": "mongodb://zaqar3.example.com:27018", |
"weight": 120 | "weight": 120 | ||
} | } | ||
Line 170: | Line 170: | ||
<pre><nowiki> | <pre><nowiki> | ||
HTTP/1.1 204 No Content | HTTP/1.1 204 No Content | ||
− | |||
</nowiki></pre> | </nowiki></pre> | ||
Line 177: | Line 176: | ||
Allows one to update any or all of: `weight`, `uri`, `options`. At least one of these fields must be specified, else, a HTTP 400 is returned. | Allows one to update any or all of: `weight`, `uri`, `options`. At least one of these fields must be specified, else, a HTTP 400 is returned. | ||
− | === List | + | === List pools === |
'''Template''' | '''Template''' | ||
<pre><nowiki> | <pre><nowiki> | ||
− | GET /v1/ | + | GET /v1/pools?detailed=True&limit=10&marker=taco |
</nowiki></pre> | </nowiki></pre> | ||
Line 188: | Line 187: | ||
<pre><nowiki> | <pre><nowiki> | ||
− | GET /v1/ | + | GET /v1/pools HTTP/1.1 |
− | Host: | + | Host: zaqar.proxy.example.com |
</nowiki></pre> | </nowiki></pre> | ||
Line 197: | Line 196: | ||
<pre><nowiki> | <pre><nowiki> | ||
HTTP/1.1 200 OK | HTTP/1.1 200 OK | ||
− | Content-Location: /v1/ | + | Content-Location: /v1/pools |
[ | [ | ||
− | {"href": "/v1/ | + | {"href": "/v1/pools/wat", "weight": 100, "uri": "mongodb://zaqar1.example.com:27017"}, |
− | {"href": "/v1/ | + | {"href": "/v1/pools/wot", "weight": 50, "uri": "redis://zaqar2.example.com:6379"} |
] | ] | ||
</nowiki></pre> | </nowiki></pre> | ||
Line 207: | Line 206: | ||
'''Discussion''' | '''Discussion''' | ||
− | Lists the registered | + | Lists the registered pools. |
<code><nowiki>detailed</nowiki></code> if True, returns the options field in the listing. | <code><nowiki>detailed</nowiki></code> if True, returns the options field in the listing. | ||
− | <code><nowiki>marker</nowiki></code> used for pagination - what | + | <code><nowiki>marker</nowiki></code> used for pagination - what pool do we start listing from? |
<code><nowiki>limit</nowiki></code> how many entries to return per request? | <code><nowiki>limit</nowiki></code> how many entries to return per request? |
Latest revision as of 16:41, 21 August 2014
Contents
Zaqar Storage Pooling: v1 Blueprint
Overview
This proposal introduces another approach to tackle making Zaqar highly scalable without losing any of its flexibility. This particular document covers the administrative API for registering storage pools.
Reference
For authoritative header/general details, please refer to the Zaqar Queues API v1 Spec.
Endpoints synopsis
GET /pools?detailed=False&marker=None&limit=10 GET /pools/{pool}?detailed=False PUT /pools/{pool} DELETE /pools/{pool} PATCH /pools/{pool} GET /health
Pooling with Zaqar
Zaqar supports heterogeneous pooling through the use of:
- stevedore for dynamic storage driver loading
- node weights
As long as an entry point is defined in an installed module that matches the scheme of a pool connection URI, then Zaqar will be able to use that pool. For example, a pool entry might look like:
{ "weight": 100, "uri": "mongodb://localhost:27017", "options": { "max_retry_sleep": 1 } }
Register a pool
Template
PUT /v1/pools/{pool}
Request
PUT /v1/pools/wat HTTP/1.1 Host: zaqar.example.com { "weight": 100, "uri": "mongodb://localhost:27017", "options": { "max_retry_sleep": 1, "partitions": 8 } }
Response
HTTP/1.1 201 Created Location: /v1/pools/wat
Discussion
Register a pool.
pool
is the name to give the pool entry. The name MUST NOT exceed 64 bytes in length, and is limited to US-ASCII letters, digits, underscores and hyphens.
weight
is the likelihood that this pool will be selected for the next queue allocation. It must be an integer greater than -1.
uri
is a connection string compatible with a storage client (e.g., pymongo) attempting to connect to that pool.
options
An optional request component that gives storage-specific options used by storage driver implementations. Valid parameters come from the registered options for a given storage backend, for example: mongodb, sqlite
Read a pool
Template
GET /v1/pools/{pool}?detailed=True
Request
GET /v1/pools/wat HTTP/1.1 Host: zaqar.example.com
Response
HTTP/1.1 200 OK Content-Location: /v1/pools/wat { "uri": "mongodb://zaqar1.example.com:27017", "weight": 100 }
Discussion
Returns information on a registered pool.
Delete a pool
Template
DELETE /v1/pool/{pool}
Request
DELETE /v1/pools/wat HTTP/1.1 Host: zaqar.example.com
Response
HTTP/1.1 204 No Content
Discussion
Removes a pool from the registry.
Update a pool
Template
PATCH /v1/pools/{pool}
Request
PATCH /v1/pools/wat HTTP/1.1 Host: zaqar.example.com { "uri": "mongodb://zaqar3.example.com:27018", "weight": 120 }
Response
HTTP/1.1 204 No Content
Discussion
Allows one to update any or all of: `weight`, `uri`, `options`. At least one of these fields must be specified, else, a HTTP 400 is returned.
List pools
Template
GET /v1/pools?detailed=True&limit=10&marker=taco
Request
GET /v1/pools HTTP/1.1 Host: zaqar.proxy.example.com
Response
HTTP/1.1 200 OK Content-Location: /v1/pools [ {"href": "/v1/pools/wat", "weight": 100, "uri": "mongodb://zaqar1.example.com:27017"}, {"href": "/v1/pools/wot", "weight": 50, "uri": "redis://zaqar2.example.com:6379"} ]
Discussion
Lists the registered pools.
detailed
if True, returns the options field in the listing.
marker
used for pagination - what pool do we start listing from?
limit
how many entries to return per request?