https://wiki.openstack.org/w/api.php?action=feedcontributions&user=Drewford&feedformat=atomOpenStack - User contributions [en]2024-03-29T02:27:26ZUser contributionsMediaWiki 1.28.2https://wiki.openstack.org/w/index.php?title=Trove/scheduled-tasks&diff=52147Trove/scheduled-tasks2014-05-12T19:58:11Z<p>Drewford: </p>
<hr />
<div><br />
* '''Created''': 13 Aug 2013<br />
* '''Contributors''': Craig Vyvial (cp16net)<br />
<br />
__FORCETOC__<br />
<br />
= Automated/Scheduled Backups Design =<br />
== Overview ==<br />
The goal of this blueprint is to implement capabilities in Trove to support automated/scheduled backups. This will be supported in the Trove API as well as the guest agent to use the existing [[Snapshot-design|snapshot-design]] in an automated way. The automated backups shall allow a user to restore or clone to a new instance at a point in time.<br />
<br />
There are many different concerns when attempting to automate the backup of instances in a deployment, i.e. Stagger Backups, [[Trove/scheduled-tasks#Maintenance_Time_Window|Maintenance Time Window]], and others. In the design of this automated system each of these concerns should be capable of being addressed. This shall lead to a pluggable or configurable interface for allowing a deployer to chose what type of strategy they would like to use without a code change.<br />
<br />
The user shall define a [[Trove/scheduled-tasks#Maintenance_Time_Window|Maintenance Time Window]] of when the automated backup maintenance shall occur. The [[Trove/scheduled-tasks#Maintenance_Time_Window|Maintenance Time Window]] can be generalized a maintenance window of when a user would like to have scheduled backups, updates or other maintenance related operations to occur.<br />
<br />
In order for the strategy(s) of an automated backup to be handled in a pluggable way, it will require a trove scheduler that can determine the time from the general maintenance window when the backup must occur with simple or complex logic. The scheduler shall send a message to the guest agent to run the automated backup.<br />
<br />
[[User:Cp16net|cp16net]] ([[User talk:Cp16net|talk]]) 15:07, 15 August 2013 (UTC) There is a debate on where the scheduled tasks will be run from. Sent from the scheduler to the guest or the guest handles the runs itself. Either way there will be messages sent back and forth on the bus.<br />
<br />
==Design Goals==<br />
The following are the design goals:<br />
<br />
* Pluggable interface to determine the type of strategy(s) to support. (Stagger Backups, [[Trove/scheduled-tasks#Maintenance_Time_Window|Maintenance Time Window]], and others)<br />
** Manage network bandwidth and connections from backups<br />
* Database support for the schedule set and listing of the backups<br />
** Maintenance window of when to run scheduled tasks like backups<br />
<br />
==Maintenance Time Window==<br />
The Maintenance Time Window can be set by the user for when they would like package updates to occur, backup or their data, restart of service, system maintenance, or other related tasks. This shall allow users to determine when a disruption of the service occurs rather than the deployer determining a time.<br />
<br />
The user shall be able to create multiple time windows with different types that allow for specific types of maintenance or tasks to occur when the user specifies.<br />
<br />
==New Objects==<br />
* [[Trove/scheduled-tasks#Type of Scheduled Tasks|Type of Scheduled Tasks]]<br />
* [[Trove/scheduled-tasks#Create_a_Scheduled_Task|Scheduled Tasks]]<br />
<br />
==API Spec==<br />
User shall be able to create a new scheduled task for an instance and have basic CRUD operations on the scheduled task.<br />
<br />
===Type of Scheduled Tasks===<br />
The Schedule type needs to be configurable from the mgmt side but visible to the user.<br />
<br />
<pre><br />
GET /scheduledtasktypes<br />
</pre><br />
<br />
'''Response:'''<br />
<pre><br />
{<br />
"scheduledtasktypes": [<br />
{<br />
"type" : "backup",<br />
"description" : "backup description",<br />
"enabled": True<br />
},<br />
{<br />
"type" : "restart",<br />
"description" : "restart description",<br />
"enabled": False<br />
},<br />
{<br />
"type" : "update",<br />
"description" : "update description",<br />
"enabled": True<br />
}<br />
]<br />
}<br />
</pre><br />
[[User:Cp16net|cp16net]] ([[User talk:Cp16net|talk]]) 20:54, 22 August 2013 (UTC) May need some extra fields about each task type here. (optional)<br />
<br />
Enabled flag could be controlled by the deployer if they would like to stop certain tasks to stop from happening during a maintenance period.<br />
<br />
===Create a Scheduled Task===<br />
User shall be able to create a new scheduled task on an instance.<br />
<br />
<pre><br />
POST /scheduledtasks <br />
</pre><br />
Creates a new scheduled task for an instance.<br />
<br />
'''Parameters'''<br /><br />
name - optional - if a name is not specified one will be autopopulated<br /><br />
instance_id - the instance against which to run the task<br /><br />
enabled - True if the task is enabled False if not.<br /><br />
type - ID of the type of schedule<br /><br />
frequency - Frequency of the task. (hourly|daily|weekly|monthly)<br /><br />
time_window - Time window of when task can occur<br /><br />
description - optional - description of the schedule<br /><br />
metadata - optional - a dictionary of metadata<br />
<br />
[[User:Cp16net|cp16net]] ([[User talk:Cp16net|talk]]) 21:49, 20 August 2013 (UTC) Some of this data needs to be ironed out still. The frequency could be moved to a metadata block. Notification recipient could be a list instead of a single email address. <br />
<br />
[[User:Jimbobhickville|Jimbobhickville]] ([[User talk:Jimbobhickville|talk]]) 19:46, 19 December 2013 (UTC) I disagree on the frequency being moved to metadata, and I don't think notifications belongs there either. Anything that describes the task belongs in the main body, the metadata should be things that are only applicable to a specific task type. Rather than a single field for time_window, shouldn't we use two fields so we can still use date math?<br />
<br />
[[User:Jimbobhickville|Jimbobhickville]] ([[User talk:Jimbobhickville|talk]]) 19:42, 15 January 2014 (UTC) After some discussion, the notification parameters were removed, to be replaced by broadcast messages that external programs like ceilometer can listen for and handle notifying the customer.<br />
<br />
Request:<br /><br />
<pre><br />
{<br />
"scheduledtask" : {<br />
"name":"My Auto Backup",<br />
"enabled": "True|False",<br />
"type":"1",<br />
"frequency" : "hourly|daily|weekly|monthly",<br />
"time_window":"2012-03-28T22:00Z/2012-03-28T23:00Z",<br />
"description" : "Auto Backup for my production database."<br />
"metadata" : {<br />
"retentionPeriod" : "7|14|21|28",<br />
}<br />
}<br />
}<br />
</pre><br />
Response:<br />
<pre><br />
{<br />
"scheduledtask": {<br />
"id" : "56b80958-cf34-4e9e-b5b0-b84fbe5e4ecc"<br />
"name":"My Auto Backup",<br />
"enabled": "True|False",<br />
"type":"1",<br />
"frequency" : "hourly|daily|weekly|monthly",<br />
"time_window":"2012-03-28T22:00Z/2012-03-28T23:00Z",<br />
"description" : "Auto Backup for my production database.",<br />
"metadata": {<br />
"retentionPeriod" : "7|14|21|28",<br />
}<br />
}<br />
}<br />
</pre><br />
<br />
===Update a Scheduled Task===<br />
User shall be able to update an existing scheduled task on an instance. i.e.<br />
<br />
<pre><br />
PUT /scheduledtasks/{id} <br />
</pre><br />
Updates an existing scheduled task.<br />
<br />
'''Parameters'''<br /><br />
name - optional - if a name is not specified one will be autopopulated<br /><br />
enabled - optional - True if the task is enabled False if not.<br /><br />
frequency - optional - Frequency of the task. (hourly|daily|weekly|monthly)<br /><br />
time_window - optional - Time window of when task can occur<br /><br />
description - optional - description of the schedule<br /><br />
metadata - optional - metadata used by the task that runs<br /><br />
<br />
Request:<br /><br />
<pre><br />
{<br />
"scheduledtask" : {<br />
"name":"My Auto Backup",<br />
"enabled": "True|False",<br />
"type":"1",<br />
"frequency" : "hourly|daily|weekly|monthly",<br />
"time_window":"2012-03-28T22:00Z/2012-03-28T23:00Z",<br />
"description" : "Auto Backup for my production database.",<br />
"metadata": {<br />
"retentionPeriod" : "7|14|21|28",<br />
}<br />
}<br />
}<br />
</pre><br />
Response:<br />
<pre><br />
{<br />
"scheduledtask": {<br />
"id" : "56b80958-cf34-4e9e-b5b0-b84fbe5e4ecc"<br />
"name":"My Auto Backup",<br />
"enabled": "True|False",<br />
"type":"1",<br />
"frequency" : "hourly|daily|weekly|monthly",<br />
"time_window":"2012-03-28T22:00Z/2012-03-28T23:00Z",<br />
"description" : "Auto Backup for my production database."<br />
"metadata": {<br />
"retentionPeriod" : "7|14|21|28",<br />
}<br />
}<br />
}<br />
</pre><br />
<br />
===Get a Scheduled Task===<br />
User shall be able to get details of a scheduled task.<br />
<br />
<pre><br />
GET /scheduledtasks/{id} <br />
</pre><br />
Shows the scheduled task.<br />
<br />
Response:<br />
<pre><br />
{<br />
"scheduledtask": {<br />
"id" : "56b80958-cf34-4e9e-b5b0-b84fbe5e4ecc"<br />
"name":"My Auto Backup",<br />
"enabled": "True|False",<br />
"type":"1",<br />
"frequency" : "hourly|daily|weekly|monthly",<br />
"time_window":"2012-03-28T22:00Z/2012-03-28T23:00Z",<br />
"description" : "Auto Backup for my production database."<br />
"metadata": {<br />
"retentionPeriod" : "7|14|21|28",<br />
}<br />
}<br />
}<br />
</pre><br />
<br />
===Get list of Scheduled Tasks===<br />
User shall be able to list all of the scheduled tasks on an instance.<br />
<br />
<pre><br />
GET /instance/{id}/scheduledtasks <br />
</pre><br />
Shows a list of scheduled task for an instance.<br />
<br />
Response:<br />
<pre><br />
{<br />
"scheduledtasks": [<br />
{<br />
"id" : "11111111-cf34-4e9e-b5b0-b84fbe5e4ecc",<br />
"name":"My Auto Backup",<br />
"enabled": "True|False",<br />
"type":"1",<br />
"frequency" : "hourly|daily|weekly|monthly",<br />
"time_window":"2012-03-28T22:00Z/2012-03-28T23:00Z",<br />
"description" : "Auto Backup for my production database."<br />
"metadata": {<br />
"retentionPeriod" : "7|14|21|28",<br />
}<br />
},<br />
{<br />
"id" : "22222222-cf34-4e9e-b5b0-b84fbe5e4ecc",<br />
"name":"My Update Period",<br />
"enabled": "True|False",<br />
"type":"3",<br />
"frequency" : "hourly|daily|weekly|monthly",<br />
"time_window":"2012-03-28T23:00Z/2012-03-28T24:00Z",<br />
"description" : "Updates can be applied to my production database."<br />
"metadata": {<br />
"retentionPeriod" : "7|14|21|28",<br />
}<br />
}<br />
]<br />
}<br />
</pre><br />
<br />
===Delete a Scheduled Task===<br />
User shall be able to delete a scheduled task.<br />
<br />
<pre><br />
DELETE /scheduledtasks/{id} <br />
</pre><br />
Deletes a scheduled task.<br />
<br />
Response:<br />
202 Accepted<br />
<br />
==RPC Message Communication==<br />
This is the contract defined that the scheduler will have with the guest to run tasks and get replies from the guest on tasks being run.<br />
<br />
===Backup===<br />
<br />
====Run a Backup====<br />
This is the message payload that will be sent to the guest to excute a backup job.<br />
<br />
<pre><br />
{<br />
"backup": {<br />
"id" : "11111111-cf34-4e9e-b5b0-b84fbe5e4ecc",<br />
"auth_token": "0a9sdf809sdu09sjfaisduf098ublahblahblah",<br />
"destination_url": "customerbackupdir",<br />
"type": "xtrabackup|mysqldump|other"<br />
}<br />
}<br />
</pre><br />
<br />
====Guest Update on Backup====<br />
<br />
<pre><br />
{<br />
"backup": {<br />
"id" : "11111111-cf34-4e9e-b5b0-b84fbe5e4ecc",<br />
"status": "(Success|Failure)",<br />
"reason": "Some explaination of the failure or success",<br />
"size_mb": 123,<br />
"md5": "some-md5-check-sum"<br />
}<br />
}<br />
</pre><br />
<br />
===Update===<br />
====Run an Update====<br />
This is the message payload that will be sent to the guest to excute a backup job.<br />
<br />
<pre><br />
{<br />
"update": {<br />
"type": "packages"<br />
}<br />
}<br />
</pre><br />
<br />
====Guest Update on Update====<br />
<br />
<pre><br />
{<br />
"update": {<br />
"status": "(Success|Failure)",<br />
"reason": "Some explaination of the failure or success"<br />
}<br />
}<br />
</pre><br />
<br />
==Scheduled Task Schema==<br />
One new entity will be created in the trove database: scheduled_tasks. This entity will store the scheduled task data that is supplied by the user.<br />
<br />
Scheduled Task (scheduled_tasks)<br />
<br />
This table will contain the id, name, description, tenant id, instance id, task type, and enabled the task belongs to.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Name !! Data Type !! Length !! Nullable !! Details<br />
|-<br />
| id || VARCHAR || 36 || False || Primary Key, Generated UUID<br />
|-<br />
| name || VARCHAR || 255 || False || -<br />
|-<br />
| description || VARCHAR || 512 || True || -<br />
|-<br />
| tenant_id || VARCHAR || 36 || False || -<br />
|-<br />
| instance_id || VARCHAR || 36 || False || -<br />
|-<br />
| task_type || VARCHAR || 36 || False || -<br />
|-<br />
| enabled || BOOLEAN || default || False || -<br />
|-<br />
| deleted || BOOLEAN || default || False || -<br />
|-<br />
| created || DATETIME || default || False || -<br />
|-<br />
| updated || DATETIME || default || False || -<br />
|-<br />
| deleted_at || DATETIME || default || False || -<br />
|}<br />
<br />
<br />
== Feedback ==<br />
<br />
==== API Spec - Scheduled Task Creation ====<br />
*drewford: Time Window value is shown as a timestamp. Does the granularity of the "time_window" value change depending on your choice for frequency?</div>Drewfordhttps://wiki.openstack.org/w/index.php?title=Trove-Replication-And-Clustering-API-Single&diff=52136Trove-Replication-And-Clustering-API-Single2014-05-12T19:16:37Z<p>Drewford: Adding some notes from a design perspective</p>
<hr />
<div>== MySQL Master/Slave ==<br />
<br><br />
==== Create Master ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
POST /instances<br />
{<br />
"instance": {<br />
"availability_zone": "us-west-1",<br />
"name": "product-a",<br />
"datastore": {<br />
"type": "mysql",<br />
"version": "mysql-5.5"<br />
},<br />
"configuration": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6",<br />
"flavorRef": "7",<br />
"volume": {<br />
"size": 1<br />
}<br />
}<br />
}<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
{<br />
"instance": {<br />
"status": "BUILD",<br />
"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",<br />
"name": "product-a",<br />
"configuration": {<br />
"id": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6",<br />
"name": "config-a",<br />
"links": [{...}]<br />
},<br />
...<br />
}<br />
}<br />
</pre><br />
<br><br />
==== Create Slave ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
POST /instances<br />
{<br />
"instance": {<br />
"availability_zone": "us-west-2",<br />
"name": "product-b",<br />
"datastore": {<br />
"type": "mysql",<br />
"version": "mysql-5.5"<br />
},<br />
"topology": {<br />
"mysql": {<br />
"slave_of": [{"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"}]<br />
"read_only": true<br />
}<br />
},<br />
"configuration": "fc318e00-3a6f-4f93-af99-146b44912188",<br />
"flavorRef": "7",,<br />
"volume": {<br />
"size": 1<br />
}<br />
}<br />
}<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
{<br />
"instance": {<br />
"status": "BUILD",<br />
"id": "061aaf4c-3a57-411e-9df9-2d0f813db859",<br />
"name": "product-b",<br />
"configuration": {<br />
"id": "fc318e00-3a6f-4f93-af99-146b44912188",<br />
"name": "config-b",<br />
"links": [{...}]<br />
},<br />
...<br />
}<br />
}<br />
</pre><br />
<br />
Notes:<br />
* For master/slave wirings, the 'server_id' must differ between master and slave, and optionally the slave can specify whether it is read-only or not to avoid accidental writes.<br />
* Update: Agreed on 03/14/14 that the user should not have to specify 'server_id'. Instead, trove will be responsible for setting it and ensuring that a slave does not have the same server_id as its master, or any sibling slaves. As a part of this agreement, this requires removing 'server_id' from configuration-groups overrides (to avoid the user meddling with our bookkeeping).<br />
* 'read_only' was removed from configuration-groups and moved to topology.mysql{} because forcing the user to create a configuration-group for every MySQL slave is arduous and a poor user experience.<br />
* 'slave_of' and 'read_only' will be the only supported input fields for topology.mysql{}<br />
* 'read_only' is only permitted if 'slave_of' is set, otherwise the request will be failed.<br />
* Opinion: 'read_only' should default to true<br />
* 'slave_of' is an array to properly represent multi-source replication in the future (coming in MySQL 5.7)<br />
* For now, 'slave_of' requires a vanilla trove instance uuid, but will inevitably need to be prefixed with namespacing to support multiple dcs and sources (e.g. trove:us-west:tenant_id:instance:dfbbd9ca-b5e1-4028-adb7-f78643e17998)<br />
* Why topology.<datastore>{} vs. topology.metadata{}? This should be more evident as you walk through the rest of the datatores, but in short: it provides strong(er) typing, allows for apischema validation, easier to document, simpler to maintain, etc.<br />
<br><br />
==== Show Instance ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
{<br />
"instance": {<br />
"status": "ACTIVE",<br />
"updated": "2014-02-16T03:38:49"<br />
"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",<br />
"name": "product-a",<br />
"datastore": {<br />
"version": "mysql-5.5",<br />
"type": "mysql",<br />
},<br />
"flavor": {<br />
"id": "7",<br />
"links": [{...}]<br />
},<br />
"configuration": {<br />
"id": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6",<br />
"name": "config-a",<br />
"links": [{...}]<br />
}<br />
}<br />
}<br />
</pre><br />
<br><br />
<br />
==== Show Topology ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
{<br />
"topology": {<br />
"members": [<br />
{<br />
"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",<br />
"name": "product-a"<br />
},<br />
{<br />
"id": "061aaf4c-3a57-411e-9df9-2d0f813db859",<br />
"name": "product-b",<br />
"mysql": {<br />
"slave_of": [{"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"}],<br />
"read_only": true<br />
}<br />
}<br />
]<br />
}<br />
}<br />
</pre><br />
<br><br />
<br />
==== Remove Replication (aka "Promote" to Standalone) ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/action<br />
<br />
{<br />
"mysql": {<br />
"promote": {<br />
"id": "061aaf4c-3a57-411e-9df9-2d0f813db859"<br />
}<br />
}<br />
}<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
TBD<br />
</pre><br />
<br><br />
<br />
== MongoDB ==<br />
<br><br />
==== Create Replica-Set ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
POST /instances<br />
{<br />
"instance": {<br />
"name": "product-a",<br />
...<br />
"datastore": {<br />
"type": "mongodb",<br />
"version": "mongodb-2.0.4"<br />
},<br />
"topology": {<br />
"mongodb": {<br />
"type": "member",<br />
"replSet": "products",<br />
"join": false<br />
}<br />
},<br />
...<br />
}<br />
}<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
{<br />
"instance": {<br />
"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",<br />
...<br />
}<br />
}<br />
</pre><br />
<br><br />
Notes:<br />
* Enforce 'replSet' field to be provided for MongoDB, even in the case of a standalone/single instance. See http://www.mongodb.com/blog/post/dont-let-your-standalone-mongodb-server-stand-alone for reasoning.<br />
* 'join' indicates whether you're joining an existing replica-set, or creating a new one. If 'join' is false, and an active replica-set by that name for the tenant already exists, the request will be failed. 'join' by default will be false, but was included above for illustrative purposes.<br />
* 'type' is 'member' vs. 'primary' because in a replica-set, the primary is dynamic and can change in an election.<br />
<br><br />
==== Add Member to Replica-Set ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
POST /instances<br />
{<br />
"instance": {<br />
"name": "product-b",<br />
...<br />
"topology": {<br />
"mongodb": {<br />
"type": "member",<br />
"replSet": "products",<br />
"join": true<br />
}<br />
},<br />
...<br />
}<br />
}<br />
</pre><br />
<br><br />
Response:<br />
<pre><br />
{<br />
"instance": {<br />
"status": "BUILD",<br />
"id": "061aaf4c-3a57-411e-9df9-2d0f813db859",<br />
...<br />
}<br />
}<br />
</pre><br />
<br><br />
Notes:<br />
* If 'join' is true, and there is no existing replica-set for the tenant matching the 'replSet' value, the request will be failed.<br />
* Will have to use 'db.isMaster()' to determine the current primary to execute replica-set commands against (since it can be dynamic due to elections)<br />
* Will use http://docs.mongodb.org/manual/tutorial/expand-replica-set/#configure-and-add-a-member<br />
* Should protect against adding more than 12 members to a replica-set<br />
* Should protect against adding more than 7 voting members to a replica-set<br />
* Should return warning when number of voting members is even and there is no arbiter<br />
<br><br />
==== Add Another Member to Replica-Set ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
POST /instances<br />
{<br />
"instance": {<br />
"name": "product-c",<br />
...<br />
"topology": {<br />
"mongodb": {<br />
"type": "member",<br />
"replSet": "products",<br />
"join": true<br />
}<br />
},<br />
...<br />
}<br />
}<br />
</pre><br />
<br><br />
Response:<br />
<pre><br />
{<br />
"instance": {<br />
"status": "BUILD",<br />
"id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",<br />
...<br />
}<br />
}<br />
</pre><br />
<br><br />
==== Show Instance ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
{<br />
"instance": {<br />
...<br />
"topology": {<br />
"mongodb": {<br />
"type": "member",<br />
"replSet": "products"<br />
}<br />
},<br />
...<br />
}<br />
}<br />
</pre><br />
<br><br />
==== Show Topology ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
{<br />
"topology": {<br />
"members": [<br />
{<br />
"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",<br />
"name": "product-a",<br />
...<br />
"mongodb": {<br />
"type": "member",<br />
"replSet": "products"<br />
}<br />
},<br />
{<br />
"id": "061aaf4c-3a57-411e-9df9-2d0f813db859",<br />
"name": "product-b",<br />
...<br />
"mongodb": {<br />
"type": "member",<br />
"replSet": "products"<br />
}<br />
},<br />
{<br />
"id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",<br />
"name": "product-c",<br />
...<br />
"mongodb": {<br />
"type": "member",<br />
"replSet": "products"<br />
}<br />
}<br />
]<br />
}<br />
}<br />
</pre><br />
<br><br />
==== Add Arbiter ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
POST /instances<br />
{<br />
"instance": {<br />
"name": "product-arbiter",<br />
...<br />
"topology": {<br />
"mongodb": {<br />
"type": "arbiter",<br />
"replSet": "products",<br />
"join": true<br />
}<br />
},<br />
...<br />
}<br />
}<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
{<br />
"instance": {<br />
"status": "BUILD",<br />
"id": "a1b62aaa-7863-4384-8250-59024141c1f8",<br />
...<br />
}<br />
}<br />
</pre><br />
<br><br />
==== Add a Delayed Member ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
POST /instances<br />
{<br />
"instance": {<br />
"name": "product-delayed",<br />
...<br />
"topology": {<br />
"mongodb": {<br />
"type": "member",<br />
"replSet": "products",<br />
"join": true,<br />
"priority": 0,<br />
"hidden": true,<br />
"slaveDelay": 3600<br />
}<br />
},<br />
...<br />
}<br />
}<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
{<br />
"instance": {<br />
"status": "BUILD",<br />
"id": "7d8eb019-931b-4b2a-88d2-4c9f0ca1b29e",<br />
...<br />
}<br />
}<br />
</pre><br />
<br><br />
Notes:<br />
* 'type', 'replSet', 'join', 'priority', 'hidden', and 'slaveDelay' are the only fields supported in topology.mongodb{}. All other configuration values must be set via a configuration-group. After more thought, consider supporting 'hostname' and 'votes' as well.<br />
* Why isn't 'priority', 'hidden' and 'slaveDelay' in a configuration-group you ask? This is explained in "Modifying a Replica-Set" below.<br />
<br><br />
==== Modifying a Replica-Set ====<br />
<br><br />
Thus far we've been able to model building a replica-set, adding an arbiter, adding a delayed secondary member, etc. Let's continue with how to modify a replica-set.<br />
<br><br />
<br><br />
Example:<br />
<br />
<pre><br />
# from http://docs.mongodb.org/manual/tutorial/configure-secondary-only-replica-set-member/#example<br />
cfg = rs.conf()<br />
cfg.members[0].priority = 2<br />
cfg.members[1].priority = 1<br />
cfg.members[2].priority = 0.5<br />
cfg.members[3].priority = 0<br />
rs.reconfig(cfg)<br />
</pre><br />
<br />
Executing these priority changes one at a time can have catastrophic results, so it must be done as a transaction (with rs.reconfig() commiting). However, without the ability to address the cluster (i.e. multiple members at once), this becomes impossible. The only backdoor solution would be to guarantee that the MongoDB user(s) presented to the cloud tenant all have the clusterAdmin role, as this would allow them to connect to the primary and execute such transactions themselves via the native client. Obviously however, granting clusterAdmin to every DBaaS user in MongoDB is unacceptable in most deployments.<br />
<br><br />
<br><br />
'''Option #1: PATCH /instances/:id/topology'''<br />
<br><br />
<br><br />
Request:<br />
<br />
<pre><br />
PATCH /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology<br />
<br />
{<br />
"topology": {<br />
"members": [<br />
{<br />
"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",<br />
"mongodb": {<br />
"priority": 2<br />
}<br />
},<br />
{<br />
"id": "061aaf4c-3a57-411e-9df9-2d0f813db859",<br />
"mongodb": {<br />
"priority": 1<br />
}<br />
},<br />
{<br />
"id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",<br />
"mongodb": {<br />
"priority": 0.5<br />
}<br />
}<br />
]<br />
}<br />
}<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
{<br />
"topology": {<br />
"members": [<br />
{<br />
"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",<br />
"name": "product-a",<br />
...<br />
"mongodb": {<br />
"type": "member",<br />
"replSet": "products"<br />
}<br />
},<br />
{<br />
"id": "061aaf4c-3a57-411e-9df9-2d0f813db859",<br />
"name": "product-b",<br />
...<br />
"mongodb": {<br />
"type": "member",<br />
"replSet": "products"<br />
}<br />
},<br />
{<br />
"id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",<br />
"name": "product-c",<br />
...<br />
"mongodb": {<br />
"type": "member",<br />
"replSet": "products"<br />
}<br />
}<br />
]<br />
}<br />
}<br />
</pre><br />
<br><br />
Notes:<br />
* An HTTP PATCH vs. PUT because the omission of a field or structure should not be an indication to drop/delete it.<br />
* All modified fields in a request will be changed transactionally in a single rs.reconfig().<br />
* It should now be clear why 'priority', 'hidden' and 'slaveDelay' are in topology.mongodb{} vs. a configuration-group: when a configuration-group is changed, an event is immediately triggered to update any attached trove instances. Therefore, if you have a heterogeneous mixture of configuration-groups in a replica-set, there is no way to coordinate a consolidated rs.reconfig().<br />
* Downside: if topology.mongodb{} may have fields returned on a GET that you cannot change in a PATCH/PUT; re-worded, the granularity of what is permissible to change in a PATCH becomes complicated to check and validate.<br />
* TBD on what should be returned in the mongodb{} on a GET /instance/:id and GET /instance/:id/topology. It's a question of whether we should persist anything beyond the 'type' and 'replSet'. If say the 'priority' is stored, you introduce the possibility of drift from the truth, but can easily return it on a GET; if it's not stored, do we prompt MongoDB for the truth on a GET, or is that too computationally expensive?<br />
<br><br />
<br><br />
'''Option #2: POST /instances/:id/topology/action'''<br />
<br><br />
<br><br />
Request:<br />
<br />
<pre><br />
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/action<br />
<br />
{<br />
"mongodb": {<br />
"update_members": {<br />
"members": [<br />
{<br />
"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",<br />
"priority": 2<br />
},<br />
{<br />
"id": "061aaf4c-3a57-411e-9df9-2d0f813db859",<br />
"priority": 1<br />
},<br />
{<br />
"id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",<br />
"priority": 0.5<br />
}<br />
]<br />
}<br />
}<br />
}<br />
</pre><br />
<br><br />
Notes:<br />
* 'update_members.members[]' elements will only permit 'priority', 'hidden', and 'slaveDelay' (possibly 'votes' and 'hostname' as mentioned earlier).<br />
* Due to the limited field-set, this approach is much more fine-grained than the PATCH approach in Option #1.<br />
<br><br />
<br><br />
Decision: Option #2 is more fine-grained, easier to reason about, and less error-prone. As you'll see in later operations (like Remove a Member), the action vs. PUT/PATCH approach is also more appropriate.<br />
<br><br />
<br><br />
<br />
==== Remove a Member ====<br />
<br><br />
Removing a member is not the same as deleting one, therefore DELETE /instances/:id is not appropriate.<br />
<br><br />
<br><br />
'''Option #1: PUT /instances/:id/topology'''<br />
<br><br />
<br><br />
Request:<br />
<br />
<pre><br />
PUT /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology<br />
<br />
{<br />
"instance": {<br />
"topology": {<br />
"members": [<br />
{<br />
"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",<br />
},<br />
{<br />
"id": "061aaf4c-3a57-411e-9df9-2d0f813db859",<br />
}<br />
]<br />
}<br />
}<br />
</pre><br />
<br><br />
Notes:<br />
* By omitting a member{} for id=3a72ee87-cf3e-40f1-a1e1-fe8c7263a782 in a PUT operation, this indicates the member should be removed from the replica-set.<br />
* It's possible that one might want to modify the 'priority', 'hidden', 'votes', etc. fields of the remaining members while dropping a member. So although the example above does not show it, mongodb{} can be included in a member to indicate other changes, *BUT*, since it's a PUT the expectation of what happens to omitted fields in mongodb{} becomes unclear.<br />
<br><br />
Summary: Not very clean, mildly confusing, and very error-prone (nowhere is a "remove" action ever explicitly implied).<br />
<br><br />
<br><br />
'''Option #2: POST /instances/:id/topology/action'''<br />
<br><br />
<br><br />
Request:<br />
<br />
<pre><br />
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/action<br />
<br />
{<br />
"mongodb": {<br />
"remove_member": {<br />
"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"<br />
}<br />
}<br />
}<br />
</pre><br />
<br><br />
Notes:<br />
* mongodb{} wrapper isn't necessary, but provides the benefit of schema validation + declaration of intention/understanding.<br />
* The 'remove_member' action is explicit here, vs. implicit as seen in the PUT option.<br />
* 'remove_member' has a strict set of fields that are supported, so there is no question as to what can be provided and what will be honored (as compared to the PUT).<br />
<br><br />
Summary: Fairly clean, with no real drawbacks.<br />
<br><br />
<br><br />
'''Option #3: POST /instances/:id/topology/remove'''<br />
<br><br />
<br><br />
<pre><br />
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/remove<br />
<br />
{<br />
"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"<br />
}<br />
</pre><br />
<br><br />
Notes:<br />
* Differs from Option #2 in that the action is in the URI vs. the payload.<br />
* One drawback of this approach is that not every action will be supported across all datastores. So for example, a POST /instances/:id/topology/changeoplogsize (http://docs.mongodb.org/manual/tutorial/change-oplog-size/) makes absolutely no sense to any datastore other than MongoDB.<br />
<br><br />
Summary: At first glance is cleaner than Option #2 from a payload-perspective, but the URI discoverability and expansion is awful.<br />
<br><br />
<br><br />
'''Option #4: POST /instances/:id/action'''<br />
<br><br />
<br><br />
<pre><br />
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/action<br />
<br />
{<br />
"join": false<br />
}<br />
</pre><br />
<br><br />
Notes:<br />
* Executed against the instance you wish to remove itself from the cluster, so providing the 'id' in the payload is unnecessary.<br />
* Drawback: There are actions that are replica-set-wide (or against a subset of the replica-set), meaning Option #1 or #2 or #3 would have to co-exist with this option anyway.<br />
* Drawback: Increase the number of ways to accomplish the same thing (could unjoin against /instances/:id/action, or against /instances/:id/topology)<br />
<br><br />
Summary: For this very specific example it looks great, but isn't expressive enough for other actions.<br />
<br><br />
<br><br />
'''Option #5: POST /instances/:id/topology/:id/action'''<br />
<br><br />
<br><br />
<pre><br />
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/dfbbd9ca-b5e1-4028-adb7-f78643e17998/action<br />
<br />
{<br />
"mongodb": {<br />
"remove": {}<br />
}<br />
}<br />
</pre><br />
<br><br />
Notes:<br />
* The /instances/:id is an arbitrary member in the replica-set, it doesn't matter which one; the topology/:id is then a member of said replica-set that this action will be applied to.<br />
* Executed against the instance you wish to remove itself from the cluster, so providing the 'id' in the payload is unnecessary.<br />
* Needs More Thought: Could conceivably allow only specific operations here (like remove/unjoin), but not others that could be accomplished in a PATCH against /instances/:id/topology (like 'priority', 'hidden', etc.)<br />
<br><br />
Summary: Fairly clean with no real drawbacks.<br />
<br><br />
<br><br />
Decision: Option #2; it's extremely fine-grained, easy to reason about, and matches the approach taken in Remove A Member.<br />
<br />
<br />
==== MongoDB TokuMX ====<br />
<br />
* TokuMUX will require a new datastore-version and *possibly* a new manager class (same reasoning as why Tungsten/Galera will have their own datastore-version for MySQL)<br />
<br />
== Cassandra ==<br />
<br><br />
==== Create Cluster ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
POST /instances<br />
{<br />
"instance": {<br />
"name": "product-a",<br />
...<br />
"datastore": {<br />
"type": "cassandra",<br />
"version": "cassandra-2.0.5"<br />
},<br />
"topology": {<br />
"cassandra": {<br />
"cluster_name": "products",<br />
"num_tokens": 256,<br />
"is_seed": true,<br />
"endpoint_snitch": "RackInferringSnitch",<br />
"join": false<br />
}<br />
},<br />
...<br />
}<br />
}<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
{<br />
"instance": {<br />
"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",<br />
...<br />
}<br />
}<br />
</pre><br />
<br><br />
Notes:<br />
* Unlike in MongoDB, the 'type' field is not required (because all members of the cluster are of the same type)<br />
* 'cluster_name', 'num_tokens', and 'is_seed' are always required, with 'endpoint_snitch' being required if 'join' is false (if 'join' is true, the endpoint_snitch is inherited) and 'auto_bootstrap' required if 'join' is true.<br />
* 'seed_provider' can optionally be provided, but conveniently defaults to 'org.apache.cassandra.locator.SimpleSeedProvider'<br />
* 'join' indicates whether you're joining an existing cluster, or creating a new one. If 'join' is false, and an active cluster by that name for the tenant already exists, the request will be failed. 'join' by default will be false, but was included above for illustrative purposes.<br />
* Likely the Keystone region and the availability-zone inherent to the trove request can be used for the data-center and rack (cassandra-topology.properties), but if it turns out that the naming schemes are incompatible, 'data_center' and 'rack' can be introduced.<br />
* 'is_seed' is used vs. a seed list of ip-addresses because (1) the ip-address is not yet known and (2) when additional seeds are added, each node in the cluster must be notified and updated. More on this later.<br />
<br />
<br><br />
==== Add Node to Cluster ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
POST /instances<br />
{<br />
"instance": {<br />
"name": "product-b",<br />
...<br />
"topology": {<br />
"cassandra": {<br />
"cluster_name": "products",<br />
"num_tokens": 256,<br />
"is_seed": false,<br />
"auto_bootstrap": false,<br />
"join": true<br />
}<br />
},<br />
...<br />
}<br />
}<br />
</pre><br />
<br><br />
Response:<br />
<pre><br />
{<br />
"instance": {<br />
"status": "BUILD",<br />
"id": "061aaf4c-3a57-411e-9df9-2d0f813db859",<br />
...<br />
}<br />
}<br />
</pre><br />
<br><br />
Notes:<br />
* If 'join' is true, and there is no existing cluster for the tenant matching the 'cluster_name' value, the request will be failed.<br />
* If 'endpoint_snitch' is provided, and the value does not match that of the existing node(s) in the cluster, the request will be failed.<br />
<br><br />
==== Add Another Node to Cluster ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
POST /instances<br />
{<br />
"instance": {<br />
"name": "product-c",<br />
...<br />
"topology": {<br />
"cassandra": {<br />
"cluster_name": "products",<br />
"num_tokens": 256,<br />
"is_seed": false,<br />
"auto_bootstrap": false,<br />
"join": true<br />
}<br />
},<br />
...<br />
}<br />
}<br />
</pre><br />
<br><br />
Response:<br />
<pre><br />
{<br />
"instance": {<br />
"status": "BUILD",<br />
"id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",<br />
...<br />
}<br />
}<br />
</pre><br />
<br><br />
==== Show Instance ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
{<br />
"instance": {<br />
...<br />
"topology": {<br />
"cassandra": {<br />
"cluster_name": "products",<br />
"num_tokens": 256,<br />
"is_seed": true<br />
}<br />
},<br />
...<br />
}<br />
}<br />
</pre><br />
<br><br />
==== Show Topology ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
{<br />
"topology": {<br />
"members": [<br />
{<br />
"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",<br />
"name": "product-a",<br />
...<br />
"cassandra": {<br />
"cluster_name": "products",<br />
"num_tokens": 256,<br />
"is_seed": true<br />
}<br />
},<br />
{<br />
"id": "061aaf4c-3a57-411e-9df9-2d0f813db859",<br />
"name": "product-b",<br />
...<br />
"cassandra": {<br />
"cluster_name": "products",<br />
"num_tokens": 256,<br />
"is_seed": false<br />
}<br />
},<br />
{<br />
"id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",<br />
"name": "product-c",<br />
...<br />
"cassandra": {<br />
"cluster_name": "products",<br />
"num_tokens": 256,<br />
"is_seed": false<br />
}<br />
}<br />
]<br />
}<br />
}<br />
</pre><br />
<br><br />
==== Modifying a Cluster ====<br />
<br><br />
Example: Drain (http://www.datastax.com/documentation/cassandra/2.0/cassandra/tools/toolsDrain.html)<br />
<br />
<pre><br />
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/action<br />
<br />
{<br />
"cassandra": {<br />
"drain_node": {<br />
"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"<br />
}<br />
}<br />
}<br />
</pre><br />
<br />
<br />
==== Remove a Member ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/action<br />
<br />
{<br />
"cassandra": {<br />
"remove_node": {<br />
"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"<br />
}<br />
}<br />
}<br />
</pre><br />
<br><br />
<br />
== Couchbase ==<br />
<br><br />
==== Create Cluster ====<br />
<br><br />
'''Create Initial Cluster'''<br />
<br><br />
<br><br />
Request:<br />
<br />
<pre><br />
POST /instances<br />
{<br />
"instance": {<br />
"name": "product-a",<br />
...<br />
"datastore": {<br />
"type": "couchbase",<br />
"version": "couchbase-2.2"<br />
},<br />
"topology": {<br />
"couchbase": {<br />
"cluster_name": "products",<br />
"join": false<br />
}<br />
},<br />
...<br />
}<br />
}<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
{<br />
"instance": {<br />
"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",<br />
...<br />
}<br />
}<br />
</pre><br />
<br><br />
Notes:<br />
* Couchbase does not support a "cluster name", but does require a xdcr-cluster-name when using XDCR.<br />
* Tip: Summary of operations at http://docs.couchbase.com/couchbase-manual-2.2/#xdcr-replicate-options<br />
<br><br />
==== Add Node to Cluster ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
POST /instances<br />
{<br />
"instance": {<br />
"name": "product-b",<br />
...<br />
"topology": {<br />
"couchbase": {<br />
"cluster_name": "products",<br />
"join": true<br />
}<br />
},<br />
...<br />
}<br />
}<br />
</pre><br />
<br><br />
Response:<br />
<pre><br />
{<br />
"instance": {<br />
"status": "BUILD",<br />
"id": "061aaf4c-3a57-411e-9df9-2d0f813db859",<br />
...<br />
}<br />
}<br />
</pre><br />
<br><br />
Notes:<br />
* If 'join' is true, and there is no existing cluster for the tenant matching the 'cluster_name' value, the request will be failed.<br />
* In Couchbase, a new node can join the cluster by referencing any existing member, which is discoverable by trove via the cluster_name (unique per tenant)<br />
<br><br />
==== Add Another Node to Cluster ====<br />
<br><br />
Request/Response omitted due to a lack of any special considerations required<br />
<br><br />
<br><br />
==== Show Instance ====<br />
<br><br />
Request/Response omitted due to a lack of any special considerations required<br />
<br><br />
<br><br />
==== Show Topology ====<br />
<br><br />
Request/Response omitted due to a lack of any special considerations required<br />
<br><br />
<br><br />
==== Modifying a Cluster ====<br />
<br><br />
'''POST /instances/:id/topology/action'''<br />
<br><br />
<br><br />
Example: Create Bucket<br />
<br />
<pre><br />
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/action<br />
<br />
{<br />
"couchbase": {<br />
"create_bucket": {<br />
"bucket": "test_bucket",<br />
"bucket_type": "couchbase",<br />
"bucket_port": 11222,<br />
"bucket_ramsize": 200,<br />
"bucket_replica": 1<br />
}<br />
}<br />
}<br />
</pre><br />
<br />
<br />
==== Remove a Member ====<br />
<br><br />
'''POST /instances/:id/topology/action'''<br />
<br><br />
<br><br />
Request:<br />
<br />
<pre><br />
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/action<br />
<br />
{<br />
"couchbase": {<br />
"rebalance": {<br />
"server_remove": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"<br />
}<br />
}<br />
}<br />
</pre><br />
<br><br />
Notes:<br />
* Note how removing a node is done in the context of a rebalance for Couchbase. This is an example of how homogenizing actions across trove, like "remove node", might not be appropriate.<br />
<br />
== Redis ==<br />
<br><br />
==== Create Master ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
POST /instances<br />
{<br />
"instance": {<br />
"name": "product-a",<br />
...<br />
"datastore": {<br />
"type": "redis",<br />
"version": "redis-2.8.6"<br />
},<br />
"topology": {<br />
"redis": {}<br />
},<br />
...<br />
}<br />
}<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
{<br />
"instance": {<br />
"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",<br />
...<br />
}<br />
}<br />
</pre><br />
<br><br />
Notes:<br />
* 'join' field is not required for redis. also notice the lack of a 'cluster_name' of sorts (see 'Add Slave' for reasoning)<br />
<br />
<br><br />
==== Add Slave ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
POST /instances<br />
{<br />
"instance": {<br />
"name": "product-b",<br />
...<br />
"datastore": {<br />
"type": "redis",<br />
"version": "redis-2.8.6"<br />
},<br />
"topology": {<br />
"redis": {<br />
"slave_of": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"<br />
}<br />
},<br />
...<br />
}<br />
}<br />
</pre><br />
<br><br />
Response:<br />
<pre><br />
{<br />
"instance": {<br />
"status": "BUILD",<br />
"id": "061aaf4c-3a57-411e-9df9-2d0f813db859",<br />
...<br />
}<br />
}<br />
</pre><br />
<br><br />
Notes:<br />
* redis supports daisy-chaining slaves, therefore the 'slave_of' value needs to be a specific trove instance uuid vs. a manufactured 'cluster_name' of sorts.<br />
* whether it is a master or slave can be inferred from the presence (or lack thereof) of 'slave_of'.<br />
<br><br />
==== Show Instance ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
{<br />
"instance": {<br />
...<br />
"topology": {<br />
"redis": {}<br />
},<br />
...<br />
}<br />
}<br />
</pre><br />
<br><br />
==== Show Topology ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
{<br />
"topology": {<br />
"members": [<br />
{<br />
"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",<br />
"name": "product-a",<br />
...<br />
"redis": {}<br />
},<br />
{<br />
"id": "061aaf4c-3a57-411e-9df9-2d0f813db859",<br />
"name": "product-b",<br />
...<br />
"redis": {<br />
"slave_of": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"<br />
}<br />
}<br />
]<br />
}<br />
}<br />
</pre><br />
<br><br />
==== Promote/Disconnect Slave ====<br />
<br><br />
'''POST /instances/:id/topology/action'''<br />
<br><br />
<br><br />
Request:<br />
<br />
<pre><br />
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/action<br />
<br />
{<br />
"redis": {<br />
"slaveof_no_one": {<br />
"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"<br />
}<br />
}<br />
}<br />
</pre><br />
<br><br />
<br />
== Redis Cluster ==<br />
<br><br />
==== Create Cluster ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
POST /instances<br />
{<br />
"instance": {<br />
"name": "product-a",<br />
...<br />
"datastore": {<br />
"type": "redis",<br />
"version": "redis-3.0.0-beta1"<br />
},<br />
"topology": {<br />
"redis_cluster": {<br />
"cluster_name": "products",<br />
"cluster_timeout": 5000,<br />
"join": false<br />
}<br />
},<br />
...<br />
}<br />
}<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
{<br />
"instance": {<br />
"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",<br />
...<br />
}<br />
}<br />
</pre><br />
<br><br />
Notes:<br />
* Will require datastore-version to topology.<namespace> validation (e.g. if redis 3+, permit topology.redis_cluster{} and topology.redis{}, but if redis 2.x only permit topology.redis{})<br />
* 'cluster_name' and 'cluster_timeout' are unique to topology.redis_cluster{} (not supported in topology.redis{})<br />
* As seen in other datastore examples, uses 'join' and 'cluster_name'<br />
<br><br />
==== Add Another Master to Cluster ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
POST /instances<br />
{<br />
"instance": {<br />
"name": "product-b",<br />
...<br />
"datastore": {<br />
"type": "redis",<br />
"version": "redis-3.0.0-beta1"<br />
},<br />
"topology": {<br />
"redis_cluster": {<br />
"cluster_name": "products",<br />
"join": true<br />
}<br />
},<br />
...<br />
}<br />
}<br />
</pre><br />
<br><br />
Response:<br />
<pre><br />
{<br />
"instance": {<br />
"status": "BUILD",<br />
"id": "061aaf4c-3a57-411e-9df9-2d0f813db859",<br />
...<br />
}<br />
}<br />
</pre><br />
<br><br />
==== Add Slave to Cluster ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
POST /instances<br />
{<br />
"instance": {<br />
"name": "product-c",<br />
...<br />
"datastore": {<br />
"type": "redis",<br />
"version": "redis-3.0.0-beta1"<br />
},<br />
"topology": {<br />
"redis_cluster": {<br />
"cluster_name": "products",<br />
"slave_of": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"<br />
}<br />
},<br />
...<br />
}<br />
}<br />
</pre><br />
<br><br />
Response:<br />
<pre><br />
{<br />
"instance": {<br />
"status": "BUILD",<br />
"id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",<br />
...<br />
}<br />
}<br />
</pre><br />
<br><br />
Notes:<br />
* Despite redis clusters supporting the ability to add a slave without designating the master, we will require it to avoid unoptimal geographical relationships.<br />
<br><br />
==== Show Instance ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
{<br />
"instance": {<br />
...<br />
"topology": {<br />
"redis_cluster": {<br />
"cluster_name": "products",<br />
"cluster_timeout": 5000<br />
}<br />
},<br />
...<br />
}<br />
}<br />
</pre><br />
<br><br />
==== Show Topology ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
{<br />
"topology": {<br />
"members": [<br />
{<br />
"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",<br />
"name": "product-a",<br />
...<br />
"redis_cluster": {<br />
"cluster_name": "products",<br />
"cluster_timeout": 5000<br />
}<br />
},<br />
{<br />
"id": "061aaf4c-3a57-411e-9df9-2d0f813db859",<br />
"name": "product-b",<br />
...<br />
"redis_cluster": {<br />
"cluster_name": "products",<br />
"cluster_timeout": 5000<br />
}<br />
},<br />
{<br />
"id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",<br />
"name": "product-c",<br />
...<br />
"redis_cluster": {<br />
"cluster_name": "products",<br />
"cluster_timeout": 5000,<br />
"slave_of": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"<br />
}<br />
}<br />
]<br />
}<br />
}<br />
</pre><br />
<br><br />
==== Promote/Disconnect Slave ====<br />
<br><br />
Work in Progress<br />
<br />
== Feedback ==<br />
<br />
==== Masters/Slaves ====<br />
*drewford: Can one make a master/slave out of an existing instance?<br />
*drewford: Type and version should follow the same schema as listing datastore types<br />
<br />
==== Arbiters ====<br />
*drewford: Could/should arbiters be managed by the system automatically? Is it worth putting that responsibility on the API user, or can it be automated by the system for an easier, less technical experience? From reading the article noted in the link, it seems like it's a good enough idea to automate it to reduce the need for users to research and decide for themselves.<br />
<br />
==== Delayed Members ====<br />
*drewford: Could/should delayed members be managed by the system automatically? Can it just be fully automated for a better, less technical experience?<br />
<br />
==== Topology ====<br />
*drewford: Members of the topology in the mongo example all have the same "mongodb" object. Unless this could also be "mysql" or "redis" in the same topology, it might make sense to move up "type" and "replSet" attributes as siblings to "name".<br />
*drewford: The "type" attribute also seems redundant to the name of the array. Are there attributes other than "members" on a "topology"? If not, can "topology" be the array with each object having a "type"?<br />
<br />
==== Cassandra Create Cluster ====<br />
*drewford: can "num_tokens", "endpoint_snitch" or "auto_bootstrap" be automated in any way so they receive a valid default value even if the user does not specify one? Looking for ways in which we can take the decision workload off the user, and put it on the system in a healthy way.<br />
<br />
==== Promote/Disconnect Redis Slave ====<br />
*drewford: Seems like the promotion/disconnection json should be structured the same across all datastores if possible - with the backend handling any special manipulation to get the job done. Standardize in favor of the user.<br />
<br />
==== Redis Cluster - Create Cluster ====<br />
*drewford: "redis_cluster" here is labelled differently than the same object in Cassandra and Couchbase - they don't use "_cluster".</div>Drewfordhttps://wiki.openstack.org/w/index.php?title=Trove/Replication-And-Clustering-With-Nodes-3&diff=52117Trove/Replication-And-Clustering-With-Nodes-32014-05-12T15:20:05Z<p>Drewford: /* Feedback */</p>
<hr />
<div>== Example: Cassandra ==<br />
<br><br />
To illustrate the approach, Cassandra is used in the examples below. The eccentricities of each Datastore will be explained in their own sections.<br />
<br><br />
<br><br />
==== Create Cluster ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
POST /instances<br />
{<br />
"instance": {<br />
"name": "products",<br />
"datastore": {<br />
"type": "cassandra",<br />
"version": "2.0.6"<br />
},<br />
"configuration": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6",<br />
"flavorRef": "7",<br />
"volume": {<br />
"size": 1<br />
},<br />
"cluster": {<br />
"size": 3,<br />
"nodes": [<br />
{"region": "phx"},<br />
{"region": "slc"},<br />
{"region": "lvs"}<br />
]<br />
}<br />
}<br />
}<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
{<br />
"instance": {<br />
"status": "BUILD",<br />
"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",<br />
"name": "products",<br />
"created": "2014-04-25T20:19:23",<br />
"updated": "2014-04-25T20:19:23",<br />
"links": [{...}],<br />
"datastore": {<br />
"type": "cassandra",<br />
"version": "2.0.6"<br />
},<br />
"cluster": {<br />
"size": 3,<br />
"nodes": [<br />
{"id": "416b0b16-ba55-4302-bbd3-ff566032e1c1", "region": "phx"},<br />
{"id": "7f52e4f9-3fa6-4238-ac08-1ce15197329a", "region": "slc"},<br />
{"id": "ff9d680c-fde3-49c6-a84e-76173b6df39d", "region": "lvs"}<br />
]<br />
}<br />
}<br />
}<br />
</pre><br />
<br><br />
Notes:<br />
* For Phase One:<br />
** cluster.nodes{} will not be supported.<br />
** cluster.nodes[].region will not be returned.<br />
* For Phase Two:<br />
** if cluster.allocations{} is not provided, the current region is assumed.<br />
** cluster.nodes[].region will always be returned.<br />
* Cassandra-specific fields that are required to construct the initial cluster (num_tokens, endpoint_snitch, seed ip-list, etc.) are to be determined/calculated based on configuration file values and common-sense.<br />
<br />
<br><br />
==== Show Cluster ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
{<br />
"instance": {<br />
"status": "ACTIVE",<br />
"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",<br />
"name": "products",<br />
"created": "2014-04-25T20:19:23",<br />
"updated": "2014-04-25T20:19:23",<br />
"links": [{...}],<br />
"datastore": {<br />
"type": "cassandra",<br />
"version": "2.0.6"<br />
},<br />
"cluster": {<br />
"size": 3,<br />
"nodes": [<br />
{"id": "416b0b16-ba55-4302-bbd3-ff566032e1c1", "region": "phx"},<br />
{"id": "7f52e4f9-3fa6-4238-ac08-1ce15197329a", "region": "slc"},<br />
{"id": "ff9d680c-fde3-49c6-a84e-76173b6df39d", "region": "lvs"}<br />
]<br />
}<br />
}<br />
}<br />
</pre><br />
<br><br />
Notes:<br />
* In Phase One:<br />
** cluster.nodes[].region will not be returned.<br />
* Change: instance.volume.used, instance.ip[], and instance.hostname will never be returned<br />
** It's possible that instance.ip[] can remain if it only returns the seed ips.<br />
** It's possible that instance.hostname can remain if it's converted to an array and only contains the seed hostnames.<br />
<br><br />
==== Show Node ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/nodes/416b0b16-ba55-4302-bbd3-ff566032e1c1<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
{<br />
"node": {<br />
"status": "ACTIVE",<br />
"id": "416b0b16-ba55-4302-bbd3-ff566032e1c1",<br />
"name": "products-1",<br />
"created": "2014-04-25T20:19:23",<br />
"updated": "2014-04-25T20:19:23",<br />
"links": [{...}],<br />
"ip": ["10.0.0.1"],<br />
"configuration": {<br />
"id": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6",<br />
"links": [{...}],<br />
},<br />
"flavor": {<br />
"id": "7",<br />
"links": [{...}],<br />
},<br />
"volume": {<br />
"size": 2,<br />
"used": 0.17<br />
}<br />
}<br />
}<br />
</pre><br />
<br><br />
==== Add Node(s) ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/nodes<br />
{<br />
"nodes": {<br />
"num": 2,<br />
"allocations": [<br />
{"region": "phx"},<br />
{"region": "phx"}<br />
]<br />
}<br />
}<br />
</pre><br />
<br><br />
Response:<br />
<pre><br />
HTTP 202 (Empty Body)<br />
</pre><br />
<br><br />
Notes:<br />
* For Phase One:<br />
** nodes.num will be the only supported field (nodes.allocations will not)<br />
* For Phase Two:<br />
** if nodes.allocations[] is not provided, the region of every existing node must match, otherwise the request is failed.<br />
<br><br />
==== Replace Node ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/action<br />
<br />
{<br />
"replace_node": {<br />
"id": "7f52e4f9-3fa6-4238-ac08-1ce15197329a"<br />
}<br />
}<br />
</pre><br />
<br><br />
Response:<br />
<pre><br />
HTTP 202 (Empty Body)<br />
</pre><br />
<br><br />
Notes:<br />
* http://www.datastax.com/documentation/cassandra/2.0/cassandra/operations/ops_replace_live_node.html<br />
* http://www.datastax.com/documentation/cassandra/2.0/cassandra/operations/ops_replace_seed_node.html<br />
* http://www.datastax.com/documentation/cassandra/2.0/cassandra/operations/ops_replace_node_t.html<br />
<br><br />
<br />
==== Remove Node ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
DELETE /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/nodes/7f52e4f9-3fa6-4238-ac08-1ce15197329a<br />
</pre><br />
<br><br />
Response:<br />
<pre><br />
HTTP 202 (Empty Body)<br />
</pre><br />
<br><br />
Notes:<br />
* http://www.datastax.com/documentation/cassandra/2.0/cassandra/operations/ops_remove_node_t.html<br />
<br><br />
<br />
== Example: MongoDB ==<br />
<br><br />
==== Create Cluster ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
POST /instances<br />
{<br />
"instance": {<br />
"name": "products",<br />
"datastore": {<br />
"type": "mongodb",<br />
"version": "2.4.10"<br />
},<br />
"configuration": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6",<br />
"flavorRef": "7",<br />
"volume": {<br />
"size": 1<br />
},<br />
"cluster": {<br />
"size": 5,<br />
"nodes": [<br />
{"region": "phx"},<br />
{"region": "phx"},<br />
{"region": "phx"},<br />
{"region": "slc"},<br />
{"region": "slc"}<br />
]<br />
}<br />
}<br />
}<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
{<br />
"instance": {<br />
"status": "BUILD",<br />
"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",<br />
"name": "products",<br />
"created": "2014-04-25T20:19:23",<br />
"updated": "2014-04-25T20:19:23",<br />
"links": [{...}],<br />
"datastore": {<br />
"type": "mongodb",<br />
"version": "2.4.10"<br />
},<br />
"cluster": {<br />
"size": 5,<br />
"nodes": [<br />
{"id": "416b0b16-ba55-4302-bbd3-ff566032e1c1", "region": "phx"},<br />
{"id": "965ef811-7c1d-47fc-89f2-a89dfdd23ef2", "region": "phx"},<br />
{"id": "3642f41c-e8ad-4164-a089-3891bf7f2d2b", "region": "phx"},<br />
{"id": "7f52e4f9-3fa6-4238-ac08-1ce15197329a", "region": "slc"},<br />
{"id": "ff9d680c-fde3-49c6-a84e-76173b6df39d", "region": "slc"}<br />
]<br />
}<br />
}<br />
}<br />
</pre><br />
<br><br />
<br />
==== Show Cluster ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
{<br />
"instance": {<br />
"status": "ACTIVE",<br />
"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",<br />
"name": "products",<br />
"created": "2014-04-25T20:19:23",<br />
"updated": "2014-04-25T20:19:23",<br />
"links": [{...}],<br />
"datastore": {<br />
"type": "mongodb",<br />
"version": "2.4.10"<br />
},<br />
"cluster": {<br />
"size": 5,<br />
"nodes": [<br />
{"id": "416b0b16-ba55-4302-bbd3-ff566032e1c1", "region": "phx"},<br />
{"id": "965ef811-7c1d-47fc-89f2-a89dfdd23ef2", "region": "phx"},<br />
{"id": "3642f41c-e8ad-4164-a089-3891bf7f2d2b", "region": "phx"},<br />
{"id": "7f52e4f9-3fa6-4238-ac08-1ce15197329a", "region": "slc"},<br />
{"id": "ff9d680c-fde3-49c6-a84e-76173b6df39d", "region": "slc"}<br />
]<br />
}<br />
}<br />
}<br />
</pre><br />
<br><br />
<br />
==== Show Node ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/nodes/416b0b16-ba55-4302-bbd3-ff566032e1c1<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
{<br />
"node": {<br />
"status": "ACTIVE",<br />
"id": "416b0b16-ba55-4302-bbd3-ff566032e1c1",<br />
"name": "products-1",<br />
"created": "2014-04-25T20:19:23",<br />
"updated": "2014-04-25T20:19:23",<br />
"links": [{...}],<br />
"ip": ["10.0.0.1"],<br />
"configuration": {<br />
"id": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6",<br />
"links": [{...}],<br />
},<br />
"flavor": {<br />
"id": "7",<br />
"links": [{...}],<br />
},<br />
"volume": {<br />
"size": 2,<br />
"used": 0.17<br />
}<br />
}<br />
}<br />
</pre><br />
<br><br />
==== Create Arbiter(s) ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/nodes<br />
<br />
{<br />
"nodes": {<br />
"num": 2,<br />
"allocations": [<br />
{"region": "lvs", "type": "arbiter"},<br />
{"region": "lvs", "type": "arbiter"}<br />
]<br />
}<br />
}<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
HTTP 202 (Empty Body)<br />
</pre><br />
<br><br />
<br />
==== Show Cluster (After Arbiters) ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
{<br />
"instance": {<br />
"status": "ACTIVE",<br />
"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",<br />
"name": "products",<br />
"created": "2014-04-25T20:19:23",<br />
"updated": "2014-04-25T20:19:23",<br />
"links": [{...}],<br />
"datastore": {<br />
"type": "mongodb",<br />
"version": "2.4.10"<br />
},<br />
"cluster": {<br />
"size": 7,<br />
"nodes": [<br />
{"id": "416b0b16-ba55-4302-bbd3-ff566032e1c1", "region": "phx"},<br />
{"id": "965ef811-7c1d-47fc-89f2-a89dfdd23ef2", "region": "phx"},<br />
{"id": "3642f41c-e8ad-4164-a089-3891bf7f2d2b", "region": "phx"},<br />
{"id": "7f52e4f9-3fa6-4238-ac08-1ce15197329a", "region": "slc"},<br />
{"id": "ff9d680c-fde3-49c6-a84e-76173b6df39d", "region": "slc"},<br />
{"id": "77032c55-4496-4e35-8c0d-6cd1c18e1a9c", "region": "lvs", "type": "arbiter"},<br />
{"id": "1fd054ed-221f-4c99-8d17-570bcff4c1d2", "region": "lvs", "type": "arbiter"}<br />
]<br />
}<br />
}<br />
}<br />
</pre><br />
<br><br />
<br />
== Example: MySQL ==<br />
<br><br />
==== Create Master ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
POST /instances<br />
{<br />
"instance": {<br />
"name": "products",<br />
"datastore": {<br />
"type": "mysql",<br />
"version": "5.5"<br />
},<br />
"configuration": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6",<br />
"flavorRef": "7",<br />
"volume": {<br />
"size": 1<br />
}<br />
}<br />
}<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
{<br />
"instance": {<br />
"status": "BUILD",<br />
"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",<br />
"name": "products",<br />
"created": "2014-04-25T20:19:23",<br />
"updated": "2014-04-25T20:19:23",<br />
"links": [{...}],<br />
"datastore": {<br />
"type": "mysql",<br />
"version": "5.5"<br />
},<br />
"configuration": {<br />
"id": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6",<br />
"links": [{...}],<br />
},<br />
"flavor": {<br />
"id": "7",<br />
"links": [{...}],<br />
},<br />
"volume": {<br />
"size": 1<br />
}<br />
}<br />
}<br />
</pre><br />
<br><br />
<br />
==== Create Slave ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
POST /instances<br />
{<br />
"instance": {<br />
"name": "products-slave",<br />
"datastore": {<br />
"type": "mysql",<br />
"version": "5.5"<br />
},<br />
"configuration": "fc318e00-3a6f-4f93-af99-146b44912188",<br />
"flavorRef": "7",<br />
"volume": {<br />
"size": 1<br />
},<br />
"slave": {<br />
"of": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",<br />
"read_only": true<br />
}<br />
}<br />
}<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
{<br />
"instance": {<br />
"status": "BUILD",<br />
"id": "061aaf4c-3a57-411e-9df9-2d0f813db859",<br />
"name": "products",<br />
"created": "2014-04-25T20:19:23",<br />
"updated": "2014-04-25T20:19:23",<br />
"links": [{...}],<br />
"datastore": {<br />
"type": "mysql",<br />
"version": "5.5"<br />
},<br />
"configuration": {<br />
"id": "fc318e00-3a6f-4f93-af99-146b44912188",<br />
"links": [{...}],<br />
},<br />
"flavor": {<br />
"id": "7",<br />
"links": [{...}],<br />
},<br />
"volume": {<br />
"size": 1<br />
},<br />
"slave": {<br />
"of": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",<br />
"read_only": true<br />
}<br />
}<br />
}<br />
</pre><br />
<br><br />
==== Show Master ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
{<br />
"instance": {<br />
"status": "ACTIVE",<br />
"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",<br />
"name": "products",<br />
"created": "2014-04-25T20:19:23",<br />
"updated": "2014-04-25T20:19:23",<br />
"links": [{...}],<br />
"datastore": {<br />
"type": "mysql",<br />
"version": "5.5"<br />
},<br />
"configuration": {<br />
"id": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6",<br />
"links": [{...}],<br />
},<br />
"flavor": {<br />
"id": "7",<br />
"links": [{...}],<br />
},<br />
"volume": {<br />
"size": 1<br />
},<br />
"slave": {<br />
"list": [<br />
{"id": "061aaf4c-3a57-411e-9df9-2d0f813db859"}<br />
]<br />
}<br />
}<br />
}<br />
</pre><br />
<br><br />
==== Show Slave ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
GET /instances/061aaf4c-3a57-411e-9df9-2d0f813db859<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
{<br />
"instance": {<br />
"status": "ACTIVE",<br />
"id": "061aaf4c-3a57-411e-9df9-2d0f813db859",<br />
"name": "products",<br />
"created": "2014-04-25T20:19:23",<br />
"updated": "2014-04-25T20:19:23",<br />
"links": [{...}],<br />
"datastore": {<br />
"type": "mysql",<br />
"version": "5.5"<br />
},<br />
"configuration": {<br />
"id": "fc318e00-3a6f-4f93-af99-146b44912188",<br />
"links": [{...}],<br />
},<br />
"flavor": {<br />
"id": "7",<br />
"links": [{...}],<br />
},<br />
"volume": {<br />
"size": 1<br />
},<br />
"slave": {<br />
"of": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",<br />
"read_only": true<br />
}<br />
}<br />
}<br />
</pre><br />
<br><br />
==== Detach Slave ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
POST /instances/061aaf4c-3a57-411e-9df9-2d0f813db859/action<br />
<br />
{<br />
"detach": {}<br />
}<br />
</pre><br />
<br />
Response:<br />
<br />
<pre><br />
HTTP 202 (Empty Body)<br />
</pre><br />
<br><br />
<br />
==== Delete Master ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
DELETE /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998<br />
</pre><br />
<br><br />
Response:<br />
<pre><br />
HTTP 202 (Empty Body)<br />
</pre><br />
<br><br />
Notes:<br />
* How to handle situation in which a slave is attached to a master, and the user attempts to delete the master?<br />
<br><br />
==== Delete Slave ====<br />
<br><br />
Request:<br />
<br />
<pre><br />
DELETE /instances/061aaf4c-3a57-411e-9df9-2d0f813db859<br />
</pre><br />
<br><br />
Response:<br />
<pre><br />
HTTP 202 (Empty Body)<br />
</pre><br />
<br><br />
<br />
== Data Model Changes ==<br />
=== Nodes Table ===<br />
Create a new 'nodes' Table:<br />
<pre><br />
CREATE TABLE "nodes" (<br />
"id" varchar(36) NOT NULL,<br />
"instance_id" varchar(36) NOT NULL,<br />
"created" datetime DEFAULT NULL,<br />
"updated" datetime DEFAULT NULL,<br />
"name" varchar(255) DEFAULT NULL,<br />
"hostname" varchar(255) DEFAULT NULL,<br />
"compute_instance_id" varchar(36) DEFAULT NULL,<br />
"task_id" int(11) DEFAULT NULL,<br />
"task_description" varchar(32) DEFAULT NULL,<br />
"task_start_time" datetime DEFAULT NULL,<br />
"volume_id" varchar(36) DEFAULT NULL,<br />
"flavor_id" int(11) DEFAULT NULL,<br />
"volume_size" int(11) DEFAULT NULL,<br />
"tenant_id" varchar(36) DEFAULT NULL,<br />
"server_status" varchar(64) DEFAULT NULL,<br />
"deleted" tinyint(1) DEFAULT NULL,<br />
"deleted_at" datetime DEFAULT NULL,<br />
"datastore_version_id" varchar(36) NOT NULL,<br />
"configuration_id" varchar(36) DEFAULT NULL,<br />
PRIMARY KEY ("id"),<br />
KEY "instance_id" ("instance_id"),<br />
KEY "datastore_version_id" ("datastore_version_id"),<br />
KEY "configuration_id" ("configuration_id"),<br />
KEY "instances_tenant_id" ("tenant_id"),<br />
KEY "instances_deleted" ("deleted"),<br />
CONSTRAINT "nodes_ibfk_3" FOREIGN KEY ("instance_id") REFERENCES "instances" ("id"),<br />
CONSTRAINT "nodes_ibfk_2" FOREIGN KEY ("configuration_id") REFERENCES "configurations" ("id"),<br />
CONSTRAINT "nodes_ibfk_1" FOREIGN KEY ("datastore_version_id") REFERENCES "datastore_versions" ("id")<br />
) ENGINE=InnoDB DEFAULT CHARSET=utf8;<br />
</pre><br />
<br><br />
aka the same table as instances, except:<br />
* addition of: "instance_id" varchar(36) NOT NULL<br />
* addition of: KEY "instance_id" ("instance_id")<br />
* addition of: CONSTRAINT "instances_ibfk_3" FOREIGN KEY ("instance_id") REFERENCES "instances" ("id"),<br />
* TODO: changing of 'DEFAULT NULL' to 'NOT NULL' whenever possible (ex: things like CREATED should never be NULL)<br />
* TODO: addition of removal of indexes as deemed necessary<br />
<br><br />
=== Alter Instances Table ===<br />
Add slave_of Column to Instances Table (+ Constraint + Index):<br />
<pre><br />
ALTER TABLE instances ADD COLUMN slave_of VARCHAR(36) DEFAULT NULL;<br />
KEY "slave_of" ("slave_of"),<br />
CONSTRAINT "instances_ibfk_3" FOREIGN KEY ("slave_of") REFERENCES "instances" ("id")<br />
</pre><br />
<br><br />
<br />
=== Alter Other Tables ===<br />
Add node_id column to the following tables:<br />
* agent_heartbeats<br />
* backups<br />
* conductor_lastseen<br />
* root_enabled_history<br />
* security_group_instance_associations<br />
* service_statuses<br />
* usage_events<br />
<br><br />
<pre><br />
ALTER TABLE <table> ADD COLUMN node_id VARCHAR(36) DEFAULT NULL;<br />
KEY "node_id" ("node_id"),<br />
CONSTRAINT "<table>_ibfk_<num>" FOREIGN KEY ("node_id") REFERENCES "nodes" ("id")<br />
</pre><br />
<br><br />
== TaskManager ==<br />
<br />
* add node_id to /etc/guest_info (if it's a node in a cluster). guest_id remains as-is.<br />
* for-loop create each node.<br />
* poll until all nodes are active.<br />
* for each node: use trove/nova to get ip/hostname<br />
* for couchbase:<br />
** send ip/hostname list via rpc cast to guest<br />
* for cassandra:<br />
** send seed ip list via rpc cast to guest seed nodes, one by one (polling on REBOOT => ACTIVE), then to rest of nodes.<br />
* for mongodb:<br />
** send ip/hostname list via rpc cast to guest that is the db.isMaster()<br />
<br><br />
<br />
== Guest ==<br />
<br />
* update heartbeat payload ( heartbeat(guest_id, payload, sent) ) from {"service_status": "<status>"} to {"service_status": "<status>", "node_id": "<node-id>"}<br />
* add method to each datastore guest manager for handling ip/hostname list<br />
<br><br />
<br />
== Conductor ==<br />
<br />
* update heartbeat logic to update the nodes table (node status)<br />
<br><br />
<br />
== Capabilities ==<br />
<br><br />
A capability might be supported for a datastore-version for standalone instances, but not for clusters.<br />
Therefore, the capability tables must be amended to include a cluster-enabled flag.<br />
<br />
<pre><br />
ALTER TABLE capabilities ADD COLUMN enabled_cluster TINYINT(1) DEFAULT NULL;<br />
</pre><br />
<pre><br />
ALTER TABLE capability_overrides ADD COLUMN enabled_cluster TINYINT(1) DEFAULT NULL;<br />
</pre><br />
The following capabilities should have enabled_cluster set to false for the first iteration of clusters:<br />
* backup-create + list-instance<br />
* configuration-attach + detach + instances<br />
* resize-<all><br />
* database-<all><br />
* root-<all><br />
* secgroup-<all><br />
* user-<all><br />
<br><br />
== Configuration Groups ==<br />
<br><br />
=== Introduce read_only and hidden Parameters ===<br />
Need to introduce two additional attributes for configuration group parameters: read_only and hidden.<br />
* read_only fields include cluster_name, num_tokens, seed_provider, seeds, endpoint_snitch (cassandra) + replSet (mongodb) + server_id, log_bin (mysql).<br />
* depending on the provider, some of the read_only fields should also be hidden from the user on a configuration-show.<br />
* once read_only + hidden are available, a parallel effort should move configuration-default to configurations-show if a configuration-group is attached.<br />
* [[User:Amcrn|amcrn]] ([[User talk:Amcrn|talk]]) 20:43, 8 May 2014 (UTC): update: mysql master/slave will not be required to do this because the overrides.cnf functionality handles this nicely. to be determined as to how clustering will handle this. it could be this, or it could be a copy of the original conf, or a mixture thereof.<br />
<br><br />
<br />
=== Auto-Create and Attach ===<br />
* cassandra & mongodb need to have configuration-groups automatically created and attached to each node (for cluster_name, replset, etc.) during provisioning.<br />
* unique configuration-group per node.<br />
* auto-created+attached configuration-groups need to not be detachable from the instance.<br />
* dependency: configuration-group support for mongodb + cassandra<br />
* [[User:Amcrn|amcrn]] ([[User talk:Amcrn|talk]]) 20:43, 8 May 2014 (UTC): update: mysql master/slave will not be required to do this because the overrides.cnf functionality handles this nicely. to be determined as to how clustering will handle this. it could be this, or it could be a copy of the original conf, or a mixture thereof.<br />
<br><br />
<br />
== Feedback ==<br />
<br />
==== Create Slave ====<br />
<br />
*glucas: Replication will require capturing a snapshot of the master's state and passing that to the slave. It would be preferable to create multiple slaves from a single snapshot and then clean up, rather than repeating the snapshot process multiple times. For that reason we propose a 'replicate' action that can create N slaves from an existing master in one call. I believe this approach works with the schema changes proposed here, i.e. adding the ''slave_of'' reference to the instance table. <br />
* [[User:Amcrn|amcrn]] ([[User talk:Amcrn|talk]]) 17:49, 8 May 2014 (UTC): this is not usually cost-effective, depending on the provider. ex: if you have at least one slave in two or more regions, it makes sense to seed additional slaves with a backup from their home region. Here's a question: the backup of the master, will it stay put in Swift even after the replication has been setup? Re-worded, does the user have to delete the backup that's an artifact of setting up replication, or will Trove take care of it?<br />
*mwj: I thought the point of the topology design was to not introduce properties to the instance table that would not be relevant to all (i.e., non-replication) instances. I take it this is no longer an issue?<br />
* [[User:Amcrn|amcrn]] ([[User talk:Amcrn|talk]]) 17:49, 8 May 2014 (UTC): don't understand the question because the instances table has not been modified at all, except the addition of the slave_of column, and no clustered nodes will be inserted in the instances table.<br />
<br />
==== Capabilities ====<br />
<br />
*glucas: We should use capabilities to indicate whether a datastore supports replication (and potentially read-only vs. read-write replication). In the first iteration, replication will not be supported for clusters. mwj: Is it really necessary to have a capability for replication? We are designing replication to use backup/restore functionality, so the backup/restore capability should be enough, no?<br />
* [[User:Amcrn|amcrn]] ([[User talk:Amcrn|talk]]) 17:36, 8 May 2014 (UTC): there definitely needs to be a capability flag for clusters when it comes to features, as to whether master/slave needs it, that's an open question. does the logic for creating a backup wildly change when targeted against a slave vs. a standalone, and if so, are you guaranteeing you'll support this in your first iteration? you'll need to answer this for configuration-groups, users, databases, security-groups, root-enable/history, etc.<br />
<br />
==== Configuration Groups ====<br />
*dougshelley66: What if the end user wants to specify some configuration parameters for nodes or replicas? It appears (from the description above) that each node will have an auto generated group attached. Given an instance can only have one config group, this would allow the user to specify one?<br />
* [[User:Amcrn|amcrn]] ([[User talk:Amcrn|talk]]) 17:33, 8 May 2014 (UTC): here's how it'd work: you want a master/slave setup, with 2 slaves. all three nodes would get their own configuration-group automatically created and attached (3 unique ids). if the user wishes to configure some parameters, they can do so. if the user provides a configuration-id on provisioning, then we'd add the read_only/hidden parameters to their configuration-group automatically.<br />
<br />
==== Promote Slave ====<br />
*mwj: I would rather call this "detach slave" as I expect will may want to have "promote slave" be used in future fail-over designs.<br />
* [[User:Amcrn|amcrn]] ([[User talk:Amcrn|talk]]) 17:30, 8 May 2014 (UTC): agreed, just went with the industry standard as a default. changed it to "detach" because i agree with you.<br />
<br />
==== Delete Slave ====<br />
*mwj: Shouldn't "delete instance" just work, even for slaves?<br />
* [[User:Amcrn|amcrn]] ([[User talk:Amcrn|talk]]) 17:29, 8 May 2014 (UTC): didn't have the example above, but yes it should. added an example.<br />
<br />
==== Clusters ====<br />
*drewford: Should there be a "List Clusters" call?<br />
*drewford: "size" and "num" attributes seem like workarounds when dealing with nodes and allocations. Using "num"/"size" to tell the API how many objects you are adding in an array adds the dependency that the "num" value must match the length of the array. What if they are not the same? Unless there is a long-term use case for "size" and "num", they seem like they will become artifacts.<br />
*drewford: "allocations" adds one more level to the create object that seems like it doesn't need to be there. Why not get straight to the point when adding nodes, like: <br /><br />
<br />
"nodes": [<br />
{"region":"phx"},<br />
{"region":"phx"}<br />
]<br />
<br />
*drewford: When dealing with lists of child objects, preventing individual "detail" calls for each child is a good thing. Example - when getting details on a cluster - in a UI you would most likely also want to see a list of the nodes in the cluster. It would be good to prevent implementors from having to call the API for each individual node just to show some important details in a nodes list. Here are a couple options:<br />
# Use the complete node detail object in the nodes array on the "Show Cluster" call<br />
# Add a "List Cluster Nodes" call that returns a list of detailed node objects.</div>Drewford