Difference between revisions of "Trove/Replication-And-Clustering-With-Nodes"
< Trove
m (→Summary of New Fields for Instance-Show) |
m |
||
Line 8: | Line 8: | ||
| cluster.size || int || size of the new cluster || mongdb, cassandra, couchbase, redis-cluster | | cluster.size || int || size of the new cluster || mongdb, cassandra, couchbase, redis-cluster | ||
|- | |- | ||
− | | cluster.slave | + | | cluster.slave || complex-field || see below for sub-fields || mysql, redis, redis-cluster |
|- | |- | ||
− | | cluster.slave.read_only || boolean || whether this slave should be read-only || mysql, redis, redis-cluster | + | | -- cluster.slave.of || string (uuid) || instance-id of the master of which to be a slave || mysql, redis, redis-cluster |
+ | |- | ||
+ | | -- cluster.slave.read_only || boolean || whether this slave should be read-only || mysql, redis, redis-cluster | ||
|} | |} | ||
<br> | <br> |
Revision as of 21:52, 28 April 2014
Summary
Summary of New Fields for Instance-Create
Field | Type | Description | Applicable Datastore(s) |
---|---|---|---|
cluster.size | int | size of the new cluster | mongdb, cassandra, couchbase, redis-cluster |
cluster.slave | complex-field | see below for sub-fields | mysql, redis, redis-cluster |
-- cluster.slave.of | string (uuid) | instance-id of the master of which to be a slave | mysql, redis, redis-cluster |
-- cluster.slave.read_only | boolean | whether this slave should be read-only | mysql, redis, redis-cluster |
Summary of New Fields for Instance-Show
Field | Type | Description | Applicable Datastore(s) |
---|---|---|---|
cluster.size | int | size of the cluster (excluding slaves) | mongdb, cassandra, couchbase, redis-cluster |
cluster.nodes | array | instance-ids of nodes | mongdb, cassandra, couchbase, redis-cluster |
cluster.slaves | array | instance-ids of slaves | mysql, redis, redis-cluster |
cluster.slave | complex-field | see below for sub-fields | mysql, redis, redis-cluster |
-- cluster.slave.of | string (uuid) | instance-id of master (if a slave) | mysql, redis, redis-cluster |
-- cluster.slave.read_only | boolean | if a slave, whether it's read-only | mysql, redis, redis-cluster |
Summary of Route Changes
New Route:
GET /instances/<id>/node/<node-id>
{ "node": { "status": "<status>", "id": "<node-id>", "name": "<name>", "created": "<timestamp>", "updated": "<timestamp>", "links": [{...}], "ip": ["<ip>"], //or hostname, depending on conf "volume": { "size": <size>, "used": <used> } } }
New Route:
POST /instances/<id>/node/<node-id>/action
{ "<action>": { <action-specific fields> } }
Summary
- the term "instance" now represents either a single standalone instance or a cluster of nodes.
- each node in a cluster must initially be homogenous in respect to flavor, volume-size, configuration-group, datastore/datastore-version, security-groups, etc.
- in this approach, a slave is not considered a node, and therefore the two terms and concepts are exclusive.
- the instance.name now acts as the cluster name, both in trove as well as the datastore (e.g. cluster_name/replSet)
- node.name is the instance.name with "-<num>" as a suffix (counter for the cluster, starting at 1)
- the semantic meaning of instance.status, and the possible states, will change
- the node.status will likely mirror the state values and transitions seen in instance.status prior to the clustering implementation.
- the cluster.nodes[].ids are generated uuids
Features for First Iteration
Supported:
- Create Multi-Node Cluster
- Restart Multi-Node Cluster
- Terminate Multi-Node Cluster
- Increase Cluster Size
Not Supported:
- No 'POST /instances/<id>/node/<node-id>/action' implementations will land in the first iteration. This means no restart-node, delete-node, reboot-node, promote-node, repair-node, etc.
- Decreasing Cluster Size
- Resizing Flavor/Volume
- Backups
- Altering/Adding/Dropping Security Groups
Example: Cassandra
To illustrate the approach, Cassandra is used in the examples below. The eccentricities of each Datastore will be explained in their own sections.
Create Instance
Request:
POST /instances { "instance": { "name": "products", "datastore": { "type": "cassandra", "version": "2.0.6" }, "configuration": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6", "flavorRef": "7", "volume": { "size": 1 }, "cluster": { "size": 3 } } }
Response:
{ "instance": { "status": "BUILD", "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998", "name": "products", "created": "2014-04-25T20:19:23", "updated": "2014-04-25T20:19:23", "links": [{...}], "datastore": { "type": "cassandra", "version": "2.0.6" }, "configuration": { "id": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6", "links": [{...}], }, "flavor": { "id": "7", "links": [{...}], }, "volume": { "size": 1 }, "cluster": { "size": 3, "nodes": [ {"id": "416b0b16-ba55-4302-bbd3-ff566032e1c1"}, {"id": "7f52e4f9-3fa6-4238-ac08-1ce15197329a"}, {"id": "ff9d680c-fde3-49c6-a84e-76173b6df39d"} ] } } }
Notes:
- Cassandra-specific fields that are required to construct the initial cluster (num_tokens, endpoint_snitch, seed ip-list, etc.) are to be determined/calculated based on configuration file values and common-sense.
Show Instance
Request:
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998
Response:
{ "instance": { "status": "ACTIVE", "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998", "name": "products", "created": "2014-04-25T20:19:23", "updated": "2014-04-25T20:19:23", "links": [{...}], "datastore": { "type": "cassandra", "version": "2.0.6" }, "configuration": { "id": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6", "links": [{...}], }, "flavor": { "id": "7", "links": [{...}], }, "volume": { "size": 1 }, "cluster": { "size": 3, "nodes": [ {"id": "416b0b16-ba55-4302-bbd3-ff566032e1c1"}, {"id": "7f52e4f9-3fa6-4238-ac08-1ce15197329a"}, {"id": "ff9d680c-fde3-49c6-a84e-76173b6df39d"} ] } } }
Notes:
- Change: instance.volume.used, instance.ip[], and instance.hostname will never be returned
- It's possible that instance.ip[] can remain if it only returns the seed ips.
- It's possible that instance.hostname can remain if it's converted to an array and only contains the seed hostnames.
Show Node
Request:
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/node/416b0b16-ba55-4302-bbd3-ff566032e1c1
Response:
{ "node": { "status": "ACTIVE", "id": "416b0b16-ba55-4302-bbd3-ff566032e1c1", "name": "products-1", "created": "2014-04-25T20:19:23", "updated": "2014-04-25T20:19:23", "links": [{...}], "ip": ["10.0.0.1"], "volume": { "size": 2, "used": 0.17 } } }
Scale Nodes
Request:
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/action { "resize_cluster": { "size": 6 } }
Response:
HTTP 202 (Empty Body)
All Other Operations
Notes:
- As mentioned in the Summary, other operations like: removing a node, replacing a running node, replacing a dead node, repairing a node, etc. are not supported in the first iteration.
Example: MySQL
Create Instance
Request:
POST /instances { "instance": { "name": "products", "datastore": { "type": "mysql", "version": "5.5" }, "configuration": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6", "flavorRef": "7", "volume": { "size": 1 } } }
Response:
{ "instance": { "status": "BUILD", "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998", "name": "products", "created": "2014-04-25T20:19:23", "updated": "2014-04-25T20:19:23", "links": [{...}], "datastore": { "type": "mysql", "version": "5.5" }, "configuration": { "id": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6", "links": [{...}], }, "flavor": { "id": "7", "links": [{...}], }, "volume": { "size": 1 } } }
Create Slave
Request:
POST /instances { "instance": { "name": "products-slave", "datastore": { "type": "mysql", "version": "5.5" }, "configuration": "fc318e00-3a6f-4f93-af99-146b44912188", "flavorRef": "7", "volume": { "size": 1 }, "cluster": { "slave": { "of": "dfbbd9ca-b5e1-4028-adb7-f78643e17998", "read_only": true } } } }
Response:
{ "instance": { "status": "BUILD", "id": "061aaf4c-3a57-411e-9df9-2d0f813db859", "name": "products", "created": "2014-04-25T20:19:23", "updated": "2014-04-25T20:19:23", "links": [{...}], "datastore": { "type": "mysql", "version": "5.5" }, "configuration": { "id": "fc318e00-3a6f-4f93-af99-146b44912188", "links": [{...}], }, "flavor": { "id": "7", "links": [{...}], }, "volume": { "size": 1 }, "cluster": { "slave": { "of": "dfbbd9ca-b5e1-4028-adb7-f78643e17998", "read_only": true } } } }
Show Master
Request:
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998
Response:
{ "instance": { "status": "ACTIVE", "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998", "name": "products", "created": "2014-04-25T20:19:23", "updated": "2014-04-25T20:19:23", "links": [{...}], "datastore": { "type": "mysql", "version": "5.5" }, "configuration": { "id": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6", "links": [{...}], }, "flavor": { "id": "7", "links": [{...}], }, "volume": { "size": 1 }, "cluster": { "slaves": [ {"id": "061aaf4c-3a57-411e-9df9-2d0f813db859"} ] } } }
Show Slave
Request:
GET /instances/061aaf4c-3a57-411e-9df9-2d0f813db859
Response:
{ "instance": { "status": "ACTIVE", "id": "061aaf4c-3a57-411e-9df9-2d0f813db859", "name": "products", "created": "2014-04-25T20:19:23", "updated": "2014-04-25T20:19:23", "links": [{...}], "datastore": { "type": "mysql", "version": "5.5" }, "configuration": { "id": "fc318e00-3a6f-4f93-af99-146b44912188", "links": [{...}], }, "flavor": { "id": "7", "links": [{...}], }, "volume": { "size": 1 }, "cluster": { "slave": { "of": "dfbbd9ca-b5e1-4028-adb7-f78643e17998", "read_only": true } } } }
MongoDB
Notes:
- instance.name will function as the replica-set name
- The smallest viable deployment pattern for a replica-set is 3 nodes. A three member replica-set can either be (a) a primary and two secondaries or (b) a primary, a secondary, and an arbiter. option (b) provides less redundancy and fault tolerance, so it will not be supported in this first iteration. Therefore, size=3 by default creates a three member replica-set with one primary and two secondaries.
- instance.cluster.size will only support '3', '5', and '7' as values for MongoDB in this first iteration. Why? (a) an odd number of members ensures the replica-set is always able to elect a primary (b) only 7 members can vote at a time.
- Scaling the Nodes from 3 to 5, or 5 to 7 will be supported.
- A replica-set can have up to 12 members, but only 7 voting members. To avoid having to implement and handle 'priority'/'hidden' in the first iteration, the maximum number of nodes is capped at 7.
- Changing the priority, hidden, slaveDelay, or adding an Arbiter will not be supported in this first iteration.
Couchbase
Notes:
- instance.cluster.size can be any positive number
- Scaling the Nodes by increasing the number will be supported.
- Removing a Node will not be supported in the first iteration.