Jump to: navigation, search

Difference between revisions of "Trove-Replication-And-Clustering-API-Single"

(MongoDB)
(Adding some notes from a design perspective)
 
(11 intermediate revisions by one other user not shown)
Line 1: Line 1:
== MySQL ==
+
== MySQL Master/Slave ==
<br>
 
==== MySQL Master/Slave ====
 
<br>
 
For Master/Slave, the ''server_id'' must differ, and optionally the slave can specify whether it is ''read_only'' or not to avoid accidental writes.
 
 
 
'''Create Master'''
 
 
<br>
 
<br>
 +
==== Create Master ====
 
<br>
 
<br>
 
Request:
 
Request:
Line 19: Line 14:
 
       "type": "mysql",
 
       "type": "mysql",
 
       "version": "mysql-5.5"
 
       "version": "mysql-5.5"
    },
 
    "topology": {
 
      "mysql": {
 
        "server_id": 1
 
      }
 
 
     },
 
     },
 
     "configuration": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6",
 
     "configuration": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6",
Line 52: Line 42:
 
</pre>
 
</pre>
 
<br>
 
<br>
'''Create Slave'''
+
==== Create Slave ====
<br>
 
 
<br>
 
<br>
 
Request:
 
Request:
Line 69: Line 58:
 
     "topology": {
 
     "topology": {
 
       "mysql": {
 
       "mysql": {
        "server_id": 2,
 
 
         "slave_of": [{"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"}]
 
         "slave_of": [{"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"}]
         "slave_read_only": true
+
         "read_only": true
 
       }
 
       }
 
     },
 
     },
Line 102: Line 90:
  
 
Notes:
 
Notes:
* 'server_id', 'slave_of', and 'slave_read_only' will be the only supported input fields for topology.mysql{}
+
* For master/slave wirings, the 'server_id' must differ between master and slave, and optionally the slave can specify whether it is read-only or not to avoid accidental writes.
* If 'slave_read_only' is present, but 'slave_of' is not, the request will be failed.
+
* Update: Agreed on 03/14/14 that the user should not have to specify 'server_id'. Instead, trove will be responsible for setting it and ensuring that a slave does not have the same server_id as its master, or any sibling slaves. As a part of this agreement, this requires removing 'server_id' from configuration-groups overrides (to avoid the user meddling with our bookkeeping).
* 'server_id' will be validated to be greater than zero.
+
* 'read_only' was removed from configuration-groups and moved to topology.mysql{} because forcing the user to create a configuration-group for every MySQL slave is arduous and a poor user experience.
* In the 'slave_of' scenario, 'server_id' will be validated to ensure it does not equal the master's server_id or any of the master's existing slaves' server_ids.
+
* 'slave_of' and 'read_only' will be the only supported input fields for topology.mysql{}
 +
* 'read_only' is only permitted if 'slave_of' is set, otherwise the request will be failed.
 +
* Opinion: 'read_only' should default to true
 
* 'slave_of' is an array to properly represent multi-source replication in the future (coming in MySQL 5.7)
 
* 'slave_of' is an array to properly represent multi-source replication in the future (coming in MySQL 5.7)
 
* For now, 'slave_of' requires a vanilla trove instance uuid, but will inevitably need to be prefixed with namespacing to support multiple dcs and sources (e.g. trove:us-west:tenant_id:instance:dfbbd9ca-b5e1-4028-adb7-f78643e17998)
 
* For now, 'slave_of' requires a vanilla trove instance uuid, but will inevitably need to be prefixed with namespacing to support multiple dcs and sources (e.g. trove:us-west:tenant_id:instance:dfbbd9ca-b5e1-4028-adb7-f78643e17998)
 +
* Why topology.<datastore>{} vs. topology.metadata{}? This should be more evident as you walk through the rest of the datatores, but in short: it provides strong(er) typing, allows for apischema validation, easier to document, simpler to maintain, etc.
 
<br>
 
<br>
'''Show Instance'''
+
==== Show Instance ====
<br>
 
 
<br>
 
<br>
 
Request:
 
Request:
Line 130: Line 120:
 
       "version": "mysql-5.5",
 
       "version": "mysql-5.5",
 
       "type": "mysql",
 
       "type": "mysql",
    },
 
    "topology": {
 
      "mysql": {
 
        "server_id": 1
 
      }
 
 
     },
 
     },
 
     "flavor": {
 
     "flavor": {
Line 149: Line 134:
 
</pre>
 
</pre>
 
<br>
 
<br>
'''Show Topology'''
+
 
<br>
+
==== Show Topology ====
 
<br>
 
<br>
 
Request:
 
Request:
Line 162: Line 147:
 
<pre>
 
<pre>
 
{
 
{
   "instance": {
+
   "topology": {
     "status": "ACTIVE",
+
     "members": [
    "updated": "2014-02-16T03:38:49"
+
      {
    "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
+
        "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
    "name": "product-a",
+
        "name": "product-a"
    "datastore": {
+
       },
      "version": "mysql-5.5",
+
       {
       "type": "mysql",
+
         "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
    },
+
        "name": "product-b",
    "topology": {
+
        "mysql": {
       "members": [
+
          "slave_of": [{"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"}],
        {
+
          "read_only": true
          "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
 
          "name": "product-a",
 
          "mysql": {
 
            "server_id": 1
 
          }
 
        },
 
         {
 
          "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
 
          "name": "product-b",
 
          "mysql": {
 
            "server_id": 2,
 
            "slave_of": [{"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"}],
 
            "slave_read_only": true
 
          }
 
 
         }
 
         }
       ]
+
       }
    }
+
     ]
    "flavor": {
+
   }
      "id": "7",
+
}
      "links": [{...}]
+
</pre>
     },
+
<br>
    "configuration": {
+
 
      "id": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6",
+
==== Remove Replication (aka "Promote" to Standalone) ====
      "name": "config-a",
 
      "links": [{...}]
 
    }
 
   }
 
}
 
</pre>
 
<br>
 
'''Remove Replication (aka "Promote" to Standalone)'''
 
<br>
 
 
<br>
 
<br>
 
Request:
 
Request:
Line 228: Line 190:
 
<br>
 
<br>
  
==== MySQL Master/Master ====
+
== MongoDB ==
 
<br>
 
<br>
For Master/Master, the ''server_id'' must differ and the increments be offset so as to avoid collisions.
+
==== Create Replica-Set ====
<br>
 
<br>
 
'''Create Configuration-Group for Master A'''
 
 
<br>
 
<br>
<br>
 
 
 
Request:
 
Request:
  
 
<pre>
 
<pre>
POST /configurations
+
POST /instances
 
{
 
{
   "configuration": {
+
   "instance": {
 +
    "name": "product-a",
 
     ...
 
     ...
     "values": {
+
     "datastore": {
       "server_id": 1,
+
       "type": "mongodb",
       "auto_increment_increment": 2,
+
       "version": "mongodb-2.0.4"
      "auto_increment_offset": 1
 
 
     },
 
     },
     ...
+
    "topology": {
 +
      "mongodb": {
 +
        "type": "member",
 +
        "replSet": "products",
 +
        "join": false
 +
      }
 +
    },
 +
     ...
 
   }
 
   }
 
}
 
}
Line 258: Line 222:
 
<pre>
 
<pre>
 
{
 
{
   "configuration": {
+
   "instance": {
     ...
+
     "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
    "values": {
 
      "server_id": 1,
 
      "auto_increment_increment": 2,
 
      "auto_increment_offset": 1
 
    },
 
 
     ...
 
     ...
 
   }
 
   }
Line 270: Line 229:
 
</pre>
 
</pre>
 
<br>
 
<br>
'''Create Configuration-Group for Master B'''
+
Notes:
 +
* Enforce 'replSet' field to be provided for MongoDB, even in the case of a standalone/single instance. See http://www.mongodb.com/blog/post/dont-let-your-standalone-mongodb-server-stand-alone for reasoning.
 +
* 'join' indicates whether you're joining an existing replica-set, or creating a new one. If 'join' is false, and an active replica-set by that name for the tenant already exists, the request will be failed. 'join' by default will be false, but was included above for illustrative purposes.
 +
* 'type' is 'member' vs. 'primary' because in a replica-set, the primary is dynamic and can change in an election.
 
<br>
 
<br>
 +
==== Add Member to Replica-Set ====
 
<br>
 
<br>
 
Request:
 
Request:
  
 
<pre>
 
<pre>
POST /configurations
+
POST /instances
 
{
 
{
   "configuration": {
+
   "instance": {
 +
    "name": "product-b",
 
     ...
 
     ...
     "values": {
+
     "topology": {
       "server_id": 2,
+
       "mongodb": {
      "auto_increment_increment": 2,
+
        "type": "member",
      "auto_increment_offset": 2
+
        "replSet": "products",
 +
        "join": true
 +
      }
 
     },
 
     },
 
     ...
 
     ...
Line 289: Line 255:
 
}
 
}
 
</pre>
 
</pre>
 
+
<br>
 
Response:
 
Response:
 
 
<pre>
 
<pre>
 
{
 
{
   "configuration": {
+
   "instance": {
    ...
+
     "status": "BUILD",
     "values": {
+
    "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
      "server_id": 2,
 
      "auto_increment_increment": 2,
 
      "auto_increment_offset": 2
 
    },
 
 
     ...
 
     ...
 
   }
 
   }
Line 306: Line 267:
 
</pre>
 
</pre>
 
<br>
 
<br>
'''Create Master A'''
+
Notes:
<br>
+
* If 'join' is true, and there is no existing replica-set for the tenant matching the 'replSet' value, the request will be failed.
<br>
+
* Will have to use 'db.isMaster()' to determine the current primary to execute replica-set commands against (since it can be dynamic due to elections)
Request:
+
* Will use http://docs.mongodb.org/manual/tutorial/expand-replica-set/#configure-and-add-a-member
 
+
* Should protect against adding more than 12 members to a replica-set
* Same as seen in Master/Slave scenario
+
* Should protect against adding more than 7 voting members to a replica-set
<br>
+
* Should return warning when number of voting members is even and there is no arbiter
Response:
 
 
 
* Same as seen in Master/Slave scenario
 
<br>
 
'''Create Master B'''
 
 
<br>
 
<br>
 +
==== Add Another Member to Replica-Set ====
 
<br>
 
<br>
 
Request:
 
Request:
Line 326: Line 283:
 
{
 
{
 
   "instance": {
 
   "instance": {
 +
    "name": "product-c",
 
     ...
 
     ...
 
     "topology": {
 
     "topology": {
       "mysql": {
+
       "mongodb": {
         "type": "master",
+
         "type": "member",
         "join": [{"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"}]
+
         "replSet": "products",
 +
        "join": true
 
       }
 
       }
 
     },
 
     },
Line 337: Line 296:
 
}
 
}
 
</pre>
 
</pre>
 
+
<br>
 
Response:
 
Response:
 
 
<pre>
 
<pre>
 
{
 
{
 
   "instance": {
 
   "instance": {
     "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
+
    "status": "BUILD",
 +
     "id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",
 
     ...
 
     ...
 
   }
 
   }
Line 349: Line 308:
 
</pre>
 
</pre>
 
<br>
 
<br>
'''Show Instance'''
+
==== Show Instance ====
<br>
 
 
<br>
 
<br>
 
Request:
 
Request:
  
* Same as in Master/Slave scenario
+
<pre>
<br>
+
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998
 +
</pre>
 +
 
 
Response:
 
Response:
  
Line 363: Line 323:
 
     ...
 
     ...
 
     "topology": {
 
     "topology": {
       "mysql": {
+
       "mongodb": {
         "type": "master",
+
         "type": "member",
         "join": [{"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"}]
+
         "replSet": "products"
 
       }
 
       }
 
     },
 
     },
Line 373: Line 333:
 
</pre>
 
</pre>
 
<br>
 
<br>
'''Show Topology'''
+
==== Show Topology ====
<br>
 
 
<br>
 
<br>
 
Request:
 
Request:
  
* Same as in Master/Slave scenario
+
<pre>
<br>
+
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology
 +
</pre>
 +
 
 
Response:
 
Response:
  
Line 385: Line 346:
 
{
 
{
 
   "topology": {
 
   "topology": {
    "id": "377d54bb-9e89-4ac3-bf29-f78c2fd4faca",
 
    ...
 
 
     "members": [
 
     "members": [
 
       {
 
       {
 
         "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
 
         "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
 +
        "name": "product-a",
 
         ...
 
         ...
         "topology": {
+
         "mongodb": {
           "mysql": {
+
           "type": "member",
            "type": "master",
+
          "replSet": "products"
            "join": [{"id": "061aaf4c-3a57-411e-9df9-2d0f813db859"}]
 
          }
 
 
         }
 
         }
 
       },
 
       },
 
       {
 
       {
 
         "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
 
         "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
 +
        "name": "product-b",
 +
        ...
 +
        "mongodb": {
 +
          "type": "member",
 +
          "replSet": "products"
 +
        }
 +
      },
 +
      {
 +
        "id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",
 +
        "name": "product-c",
 
         ...
 
         ...
         "topology": {
+
         "mongodb": {
           "mysql": {
+
           "type": "member",
            "type": "master",
+
          "replSet": "products"
            "join": [{"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"}]
 
          }
 
 
         }
 
         }
 
       }
 
       }
Line 413: Line 379:
 
</pre>
 
</pre>
 
<br>
 
<br>
'''Remove Replication (aka "Promote")'''
+
==== Add Arbiter ====
<br>
 
 
<br>
 
<br>
 
Request:
 
Request:
  
 
<pre>
 
<pre>
POST /instances/061aaf4c-3a57-411e-9df9-2d0f813db859/action
+
POST /instances
 
{
 
{
   "update_topology": {
+
   "instance": {
 +
    "name": "product-arbiter",
 +
    ...
 
     "topology": {
 
     "topology": {
       "mysql": {
+
       "mongodb": {
         "type": "master",
+
         "type": "arbiter",
         "join": []
+
        "replSet": "products",
 +
         "join": true
 
       }
 
       }
     }
+
     },
 +
    ...
 
   }
 
   }
 
}
 
}
 +
</pre>
  
*or*
+
Response:
  
PUT /instances/061aaf4c-3a57-411e-9df9-2d0f813db859
+
<pre>
 
{
 
{
 
   "instance": {
 
   "instance": {
     "topology": {
+
     "status": "BUILD",
      "mysql": {
+
    "id": "a1b62aaa-7863-4384-8250-59024141c1f8",
        "type": "master",
+
     ...
        "join": []
 
      }
 
     }
 
 
   }
 
   }
 
}
 
}
 
+
</pre>
*or*
 
 
 
POST /instances/061aaf4c-3a57-411e-9df9-2d0f813db859/topology
 
{
 
  "promote": {}
 
}
 
</pre>
 
 
 
Response:
 
 
 
<pre>
 
TBD
 
</pre>
 
 
 
== MongoDB ==
 
 
<br>
 
<br>
==== Create Replica-Set ====
+
==== Add a Delayed Member ====
 
<br>
 
<br>
 
Request:
 
Request:
Line 469: Line 421:
 
{
 
{
 
   "instance": {
 
   "instance": {
     "name": "product-a",
+
     "name": "product-delayed",
 
     ...
 
     ...
    "datastore": {
 
      "type": "mongodb",
 
      "version": "mongodb-2.0.4"
 
    },
 
 
     "topology": {
 
     "topology": {
 
       "mongodb": {
 
       "mongodb": {
 
         "type": "member",
 
         "type": "member",
         "replica_set": "products",
+
         "replSet": "products",
         "join": false
+
         "join": true,
 +
        "priority": 0,
 +
        "hidden": true,
 +
        "slaveDelay": 3600
 
       }
 
       }
 
     },
 
     },
Line 492: Line 443:
 
{
 
{
 
   "instance": {
 
   "instance": {
     "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
+
    "status": "BUILD",
 +
     "id": "7d8eb019-931b-4b2a-88d2-4c9f0ca1b29e",
 
     ...
 
     ...
 
   }
 
   }
Line 499: Line 451:
 
<br>
 
<br>
 
Notes:
 
Notes:
* Enforce 'replica_set' field to be provided for MongoDB, even in the case of a standalone/single instance. See http://www.mongodb.com/blog/post/dont-let-your-standalone-mongodb-server-stand-alone for reasoning.
+
* 'type', 'replSet', 'join', 'priority', 'hidden', and 'slaveDelay' are the only fields supported in topology.mongodb{}. All other configuration values must be set via a configuration-group. After more thought, consider supporting 'hostname' and 'votes' as well.
* 'join' indicates whether you're joining an existing replica-set, or creating a new one. If 'join' is false, and an active replica-set by that name for the tenant already exists, the request will be failed. 'join' by default will be false, but was included above for illustrative purposes.
+
* Why isn't 'priority', 'hidden' and 'slaveDelay' in a configuration-group you ask? This is explained in "Modifying a Replica-Set" below.
* 'type' is 'member' vs. 'primary' because in a replica-set, the primary is dynamic and can change in an election.
+
<br>
 +
==== Modifying a Replica-Set ====
 +
<br>
 +
Thus far we've been able to model building a replica-set, adding an arbiter, adding a delayed secondary member, etc. Let's continue with how to modify a replica-set.
 
<br>
 
<br>
==== Add Member to Replica-Set ====
 
 
<br>
 
<br>
Request:
+
Example:
  
 
<pre>
 
<pre>
POST /instances
+
# from http://docs.mongodb.org/manual/tutorial/configure-secondary-only-replica-set-member/#example
{
+
cfg = rs.conf()
  "instance": {
+
cfg.members[0].priority = 2
    "name": "product-b",
+
cfg.members[1].priority = 1
    ...
+
cfg.members[2].priority = 0.5
    "topology": {
+
cfg.members[3].priority = 0
      "mongodb": {
+
rs.reconfig(cfg)
        "type": "member",
+
</pre>
        "replica_set": "products",
+
 
        "join": true
+
Executing these priority changes one at a time can have catastrophic results, so it must be done as a transaction (with rs.reconfig() commiting). However, without the ability to address the cluster (i.e. multiple members at once), this becomes impossible. The only backdoor solution would be to guarantee that the MongoDB user(s) presented to the cloud tenant all have the clusterAdmin role, as this would allow them to connect to the primary and execute such transactions themselves via the native client. Obviously however, granting clusterAdmin to every DBaaS user in MongoDB is unacceptable in most deployments.
      }
+
<br>
    },
+
<br>
    ...
+
'''Option #1: PATCH /instances/:id/topology'''
  }
+
<br>
}
 
</pre>
 
 
<br>
 
<br>
Response:
+
Request:
 +
 
 
<pre>
 
<pre>
 +
PATCH /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology
 +
 
{
 
{
   "instance": {
+
   "topology": {
     "status": "BUILD",
+
     "members": [
    "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
+
      {
     ...
+
        "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
 +
        "mongodb": {
 +
          "priority": 2
 +
        }
 +
      },
 +
      {
 +
        "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
 +
        "mongodb": {
 +
          "priority": 1
 +
        }
 +
      },
 +
      {
 +
        "id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",
 +
        "mongodb": {
 +
          "priority": 0.5
 +
        }
 +
      }
 +
     ]
 
   }
 
   }
 
}
 
}
 
</pre>
 
</pre>
<br>
+
 
Notes:
+
Response:
* If 'join' is true, and there is no existing replica-set for the tenant matching the 'replica_set' value, the request will be failed.
 
* Will have to use 'db.isMaster()' to determine the current primary to execute replica-set commands against (since it can be dynamic due to elections)
 
* Will use http://docs.mongodb.org/manual/tutorial/expand-replica-set/#configure-and-add-a-member
 
* Should protect against adding more than 12 members to a replica-set
 
* Should protect against adding more than 7 voting members to a replica-set
 
* Should return warning when number of voting members is even and there is no arbiter
 
<br>
 
==== Add Another Member to Replica-Set ====
 
<br>
 
Request:
 
  
 
<pre>
 
<pre>
POST /instances
 
 
{
 
{
   "instance": {
+
   "topology": {
     "name": "product-c",
+
     "members": [
    ...
+
      {
    "topology": {
+
        "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
       "mongodb": {
+
        "name": "product-a",
        "type": "member",
+
        ...
         "replica_set": "products",
+
        "mongodb": {
         "join": true
+
          "type": "member",
 +
        "replSet": "products"
 +
        }
 +
      },
 +
       {
 +
        "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
 +
        "name": "product-b",
 +
        ...
 +
        "mongodb": {
 +
          "type": "member",
 +
          "replSet": "products"
 +
        }
 +
      },
 +
      {
 +
         "id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",
 +
        "name": "product-c",
 +
         ...
 +
        "mongodb": {
 +
          "type": "member",
 +
          "replSet": "products"
 +
        }
 
       }
 
       }
     },
+
     ]
    ...
 
 
   }
 
   }
 
}
 
}
 
</pre>
 
</pre>
 
<br>
 
<br>
Response:
+
Notes:
<pre>
+
* An HTTP PATCH vs. PUT because the omission of a field or structure should not be an indication to drop/delete it.
{
+
* All modified fields in a request will be changed transactionally in a single rs.reconfig().
  "instance": {
+
* It should now be clear why 'priority', 'hidden' and 'slaveDelay' are in topology.mongodb{} vs. a configuration-group: when a configuration-group is changed, an event is immediately triggered to update any attached trove instances. Therefore, if you have a heterogeneous mixture of configuration-groups in a replica-set, there is no way to coordinate a consolidated rs.reconfig().
    "status": "BUILD",
+
* Downside: if topology.mongodb{} may have fields returned on a GET that you cannot change in a PATCH/PUT; re-worded, the granularity of what is permissible to change in a PATCH becomes complicated to check and validate.
    "id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",
+
* TBD on what should be returned in the mongodb{} on a GET /instance/:id and GET /instance/:id/topology. It's a question of whether we should persist anything beyond the 'type' and 'replSet'. If say the 'priority' is stored, you introduce the possibility of drift from the truth, but can easily return it on a GET; if it's not stored, do we prompt MongoDB for the truth on a GET, or is that too computationally expensive?
    ...
+
<br>
  }
+
<br>
}
+
'''Option #2: POST /instances/:id/topology/action'''
</pre>
 
 
<br>
 
<br>
==== Show Instance ====
 
 
<br>
 
<br>
 
Request:
 
Request:
  
 
<pre>
 
<pre>
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998
+
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/action
</pre>
 
 
 
Response:
 
  
<pre>
 
 
{
 
{
   "instance": {
+
   "mongodb": {
    ...
+
     "update_members": {
     "topology": {
+
       "members": [
       "mongodb": {
+
        {
         "type": "member",
+
          "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
         "replica_set": "products"
+
          "priority": 2
       }
+
         },
     },
+
        {
    ...
+
          "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
 +
          "priority": 1
 +
         },
 +
        {
 +
          "id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",
 +
          "priority": 0.5
 +
        }
 +
       ]
 +
     }
 
   }
 
   }
 
}
 
}
 
</pre>
 
</pre>
 
<br>
 
<br>
==== Show Topology ====
+
Notes:
 +
* 'update_members.members[]' elements will only permit  'priority', 'hidden', and 'slaveDelay' (possibly 'votes' and 'hostname' as mentioned earlier).
 +
* Due to the limited field-set, this approach is much more fine-grained than the PATCH approach in Option #1.
 +
<br>
 +
<br>
 +
Decision: Option #2 is more fine-grained, easier to reason about, and less error-prone. As you'll see in later operations (like Remove a Member), the action vs. PUT/PATCH approach is also more appropriate.
 +
<br>
 +
<br>
 +
 
 +
==== Remove a Member ====
 +
<br>
 +
Removing a member is not the same as deleting one, therefore DELETE /instances/:id is not appropriate.
 +
<br>
 +
<br>
 +
'''Option #1: PUT /instances/:id/topology'''
 +
<br>
 
<br>
 
<br>
 
Request:
 
Request:
  
 
<pre>
 
<pre>
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology
+
PUT /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology
</pre>
 
  
Response:
 
 
<pre>
 
 
{
 
{
 
   "instance": {
 
   "instance": {
    ...
 
 
     "topology": {
 
     "topology": {
 
       "members": [
 
       "members": [
 
         {
 
         {
 
           "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
 
           "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
          "name": "product-a",
 
          ...
 
          "mongodb": {
 
    "type": "member",
 
    "replica_set": "products"
 
  }
 
 
         },
 
         },
 
         {
 
         {
 
           "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
 
           "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
          "name": "product-b",
 
          ...
 
          "mongodb": {
 
    "type": "member",
 
    "replica_set": "products"
 
  }
 
        },
 
        {
 
          "id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",
 
          "name": "product-c",
 
          ...
 
          "mongodb": {
 
    "type": "member",
 
    "replica_set": "products"
 
  }
 
 
         }
 
         }
 
       ]
 
       ]
 
     }
 
     }
    ...
 
 
   }
 
   }
}
 
 
</pre>
 
</pre>
 
<br>
 
<br>
==== Add Arbiter ====
+
Notes:
 +
* By omitting a member{} for id=3a72ee87-cf3e-40f1-a1e1-fe8c7263a782 in a PUT operation, this indicates the member should be removed from the replica-set.
 +
* It's possible that one might want to modify the 'priority', 'hidden', 'votes', etc. fields of the remaining members while dropping a member. So although the example above does not show it, mongodb{} can be included in a member to indicate other changes, *BUT*, since it's a PUT the expectation of what happens to omitted fields in mongodb{} becomes unclear.
 +
<br>
 +
Summary: Not very clean, mildly confusing, and very error-prone (nowhere is a "remove" action ever explicitly implied).
 +
<br>
 +
<br>
 +
'''Option #2: POST /instances/:id/topology/action'''
 +
<br>
 
<br>
 
<br>
 
Request:
 
Request:
  
 
<pre>
 
<pre>
POST /instances
+
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/action
 +
 
 
{
 
{
   "instance": {
+
   "mongodb": {
     "name": "product-arbiter",
+
     "remove_member": {
    ...
+
       "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"
    "topology": {
+
     }
       "mongodb": {
+
   }
        "type": "arbiter",
+
}
        "replica_set": "products",
 
        "join": true
 
      }
 
     },
 
    ...
 
   }
 
}
 
 
</pre>
 
</pre>
 +
<br>
 +
Notes:
 +
* mongodb{} wrapper isn't necessary, but provides the benefit of schema validation + declaration of intention/understanding.
 +
* The 'remove_member' action is explicit here, vs. implicit as seen in the PUT option.
 +
* 'remove_member' has a strict set of fields that are supported, so there is no question as to what can be provided and what will be honored (as compared to the PUT).
 +
<br>
 +
Summary: Fairly clean, with no real drawbacks.
 +
<br>
 +
<br>
 +
'''Option #3: POST /instances/:id/topology/remove'''
 +
<br>
 +
<br>
 +
<pre>
 +
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/remove
  
Response:
 
 
<pre>
 
 
{
 
{
   "instance": {
+
   "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"
    "status": "BUILD",
 
    "id": "a1b62aaa-7863-4384-8250-59024141c1f8",
 
    ...
 
  }
 
 
}
 
}
 
</pre>
 
</pre>
 
<br>
 
<br>
==== Add a Delayed Member ====
+
Notes:
 +
* Differs from Option #2 in that the action is in the URI vs. the payload.
 +
* One drawback of this approach is that not every action will be supported across all datastores. So for example, a POST /instances/:id/topology/changeoplogsize (http://docs.mongodb.org/manual/tutorial/change-oplog-size/) makes absolutely no sense to any datastore other than MongoDB.
 +
<br>
 +
Summary: At first glance is cleaner than Option #2 from a payload-perspective, but the URI discoverability and expansion is awful.
 +
<br>
 +
<br>
 +
'''Option #4: POST /instances/:id/action'''
 +
<br>
 
<br>
 
<br>
Request:
+
<pre>
 +
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/action
  
<pre>
 
POST /instances
 
 
{
 
{
   "instance": {
+
   "join": false
    "name": "product-delayed",
+
}
    ...
+
</pre>
    "topology": {
+
<br>
      "mongodb": {
+
Notes:
        "type": "member",
+
* Executed against the instance you wish to remove itself from the cluster, so providing the 'id' in the payload is unnecessary.
        "replica_set": "products",
+
* Drawback: There are actions that are replica-set-wide (or against a subset of the replica-set), meaning Option #1 or #2 or #3 would have to co-exist with this option anyway.
        "join": true,
+
* Drawback: Increase the number of ways to accomplish the same thing (could unjoin against /instances/:id/action, or against /instances/:id/topology)
        "priority": 0,
+
<br>
        "hidden": true,
+
Summary: For this very specific example it looks great, but isn't expressive enough for other actions.
        "slaveDelay": 3600
+
<br>
      }
+
<br>
    },
+
'''Option #5: POST /instances/:id/topology/:id/action'''
    ...
+
<br>
  }
+
<br>
}
+
<pre>
</pre>
+
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/dfbbd9ca-b5e1-4028-adb7-f78643e17998/action
  
Response:
 
 
<pre>
 
 
{
 
{
   "instance": {
+
   "mongodb": {
     "status": "BUILD",
+
     "remove": {}
    "id": "7d8eb019-931b-4b2a-88d2-4c9f0ca1b29e",
 
    ...
 
 
   }
 
   }
 
}
 
}
Line 724: Line 704:
 
<br>
 
<br>
 
Notes:
 
Notes:
* 'type', 'replica_set', 'join', 'priority', 'hidden', and 'slaveDelay' are the only fields supported in topology.mongodb{}. All other configuration values must be set via a configuration-group. After more thought, consider supporting 'hostname' and 'votes' as well.
+
* The /instances/:id is an arbitrary member in the replica-set, it doesn't matter which one; the topology/:id is then a member of said replica-set that this action will be applied to.
* Why isn't 'priority', 'hidden' and 'slaveDelay' in a configuration-group you ask? This is explained in "Modifying a Replica-Set" below.
+
* Executed against the instance you wish to remove itself from the cluster, so providing the 'id' in the payload is unnecessary.
 +
* Needs More Thought: Could conceivably allow only specific operations here (like remove/unjoin), but not others that could be accomplished in a PATCH against /instances/:id/topology (like 'priority', 'hidden', etc.)
 
<br>
 
<br>
==== Modifying a Replica-Set ====
+
Summary: Fairly clean with no real drawbacks.
 
<br>
 
<br>
Thus far we've been able to model building a replica-set, adding an arbiter, adding a delayed secondary member, etc. Let's continue with how to modify a replica-set.
 
 
<br>
 
<br>
<br>
+
Decision: Option #2; it's extremely fine-grained, easy to reason about, and matches the approach taken in Remove A Member.
Example:
+
 
 +
 
 +
==== MongoDB TokuMX ====
 +
 
 +
* TokuMUX will require a new datastore-version and *possibly* a new manager class (same reasoning as why Tungsten/Galera will have their own datastore-version for MySQL)
  
<pre>
+
== Cassandra ==
# from http://docs.mongodb.org/manual/tutorial/configure-secondary-only-replica-set-member/#example
 
cfg = rs.conf()
 
cfg.members[0].priority = 2
 
cfg.members[1].priority = 1
 
cfg.members[2].priority = 0.5
 
cfg.members[3].priority = 0
 
rs.reconfig(cfg)
 
</pre>
 
 
 
Executing these priority changes one at a time can have catastrophic results, so it must be done as a transaction (with rs.reconfig() commiting). However, without the ability to address the cluster (i.e. multiple members at once), this becomes impossible. The only backdoor solution would be to guarantee that the MongoDB user(s) presented to the cloud tenant all have the clusterAdmin role, as this would allow them to connect to the primary and execute such transactions themselves via the native client. Obviously however, granting clusterAdmin to every DBaaS user in MongoDB is unacceptable in most deployments.
 
<br>
 
<br>
 
'''Option #1: PATCH /instances/:id/topology'''
 
 
<br>
 
<br>
 +
==== Create Cluster ====
 
<br>
 
<br>
 
Request:
 
Request:
  
 
<pre>
 
<pre>
PATCH /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology
+
POST /instances
 
 
 
{
 
{
 
   "instance": {
 
   "instance": {
 +
    "name": "product-a",
 +
    ...
 +
    "datastore": {
 +
      "type": "cassandra",
 +
      "version": "cassandra-2.0.5"
 +
    },
 
     "topology": {
 
     "topology": {
       "members": [
+
       "cassandra": {
         {
+
         "cluster_name": "products",
          "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
+
        "num_tokens": 256,
          "mongodb": {
+
         "is_seed": true,
    "priority": 2
+
        "endpoint_snitch": "RackInferringSnitch",
  }
+
         "join": false
        },
+
      }
         {
+
    },
          "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
+
     ...
          "mongodb": {
 
    "priority": 1
 
  }
 
        },
 
         {
 
          "id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",
 
          "mongodb": {
 
    "priority": 0.5
 
  }
 
        }
 
      ]
 
     }
 
 
   }
 
   }
 +
}
 
</pre>
 
</pre>
  
Line 787: Line 753:
 
{
 
{
 
   "instance": {
 
   "instance": {
 +
    "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
 
     ...
 
     ...
    "topology": {
+
   }
      "members": [
+
}
        {
+
</pre>
          "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
 
          "name": "product-a",
 
          ...
 
          "mongodb": {
 
    "type": "member",
 
    "replica_set": "products"
 
  }
 
        },
 
        {
 
          "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
 
          "name": "product-b",
 
          ...
 
          "mongodb": {
 
    "type": "member",
 
    "replica_set": "products"
 
  }
 
        },
 
        {
 
          "id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",
 
          "name": "product-c",
 
          ...
 
          "mongodb": {
 
    "type": "member",
 
    "replica_set": "products"
 
  }
 
        }
 
      ]
 
    }
 
    ...
 
   }
 
}
 
</pre>
 
 
<br>
 
<br>
 
Notes:
 
Notes:
* An HTTP PATCH vs. PUT because the omission of a field or structure should not be an indication to drop/delete it.
+
* Unlike in MongoDB, the 'type' field is not required (because all members of the cluster are of the same type)
* All modified fields in a request will be changed transactionally in a single rs.reconfig().
+
* 'cluster_name', 'num_tokens', and 'is_seed' are always required, with 'endpoint_snitch' being required if 'join' is false (if 'join' is true, the endpoint_snitch is inherited) and 'auto_bootstrap' required if 'join' is true.
* It should now be clear why 'priority', 'hidden' and 'slaveDelay' are in topology.mongodb{} vs. a configuration-group: when a configuration-group is changed, an event is immediately triggered to update any attached trove instances. Therefore, if you have a heterogeneous mixture of configuration-groups in a replica-set, there is no way to coordinate a consolidated rs.reconfig().
+
* 'seed_provider' can optionally be provided, but conveniently defaults to 'org.apache.cassandra.locator.SimpleSeedProvider'
* Downside: if topology.mongodb{} may have fields returned on a GET that you cannot change in a PATCH/PUT; re-worded, the granularity of what is permissible to change in a PATCH becomes complicated to check and validate.
+
* 'join' indicates whether you're joining an existing cluster, or creating a new one. If 'join' is false, and an active cluster by that name for the tenant already exists, the request will be failed. 'join' by default will be false, but was included above for illustrative purposes.
* TBD on what should be returned in the mongodb{} on a GET /instance/:id and GET /instance/:id/topology. It's a question of whether we should persist anything beyond the 'type' and 'replica_set'. If say the 'priority' is stored, you introduce the possibility of drift from the truth, but can easily return it on a GET; if it's not stored, do we prompt MongoDB for the truth on a GET, or is that too computationally expensive?
+
* Likely the Keystone region and the availability-zone inherent to the trove request can be used for the data-center and rack (cassandra-topology.properties), but if it turns out that the naming schemes are incompatible, 'data_center' and 'rack' can be introduced.
<br>
+
* 'is_seed' is used vs. a seed list of ip-addresses because (1) the ip-address is not yet known and (2) when additional seeds are added, each node in the cluster must be notified and updated. More on this later.
<br>
+
 
'''Option #2: POST /instances/:id/topology/action'''
 
 
<br>
 
<br>
 +
==== Add Node to Cluster ====
 
<br>
 
<br>
 
Request:
 
Request:
  
 
<pre>
 
<pre>
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/action
+
POST /instances
 
 
 
{
 
{
   "mongodb": {
+
   "instance": {
     "update_members": {
+
     "name": "product-b",
       "members": [
+
    ...
         {
+
    "topology": {
          "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
+
       "cassandra": {
          "priority": 2
+
         "cluster_name": "products",
         },
+
        "num_tokens": 256,
         {
+
         "is_seed": false,
          "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
+
         "auto_bootstrap": false,
          "priority": 1
+
        "join": true
        },
+
      }
        {
+
    },
          "id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",
+
    ...
          "priority": 0.5
+
  }
        }
+
}
      ]
+
</pre>
    }
+
<br>
 +
Response:
 +
<pre>
 +
{
 +
  "instance": {
 +
    "status": "BUILD",
 +
    "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
 +
    ...
 
   }
 
   }
 
}
 
}
Line 863: Line 804:
 
<br>
 
<br>
 
Notes:
 
Notes:
* 'update_members.members[]' elements will only permit  'priority', 'hidden', and 'slaveDelay' (possibly 'votes' and 'hostname' as mentioned earlier).
+
* If 'join' is true, and there is no existing cluster for the tenant matching the 'cluster_name' value, the request will be failed.
* Due to the limited field-set, this approach is much more fine-grained than the PATCH approach in Option #1.
+
* If 'endpoint_snitch' is provided, and the value does not match that of the existing node(s) in the cluster, the request will be failed.
<br>
 
<br>
 
Decision: Option #2 is more fine-grained, easier to reason about, and less error-prone. As you'll see in later operations (like Remove a Member), the action vs. PUT/PATCH approach is also more appropriate.
 
<br>
 
<br>
 
 
 
==== Remove a Member ====
 
<br>
 
Removing a member is not the same as deleting one, therefore DELETE /instances/:id is not appropriate.
 
<br>
 
<br>
 
'''Option #1: PUT /instances/:id/topology'''
 
 
<br>
 
<br>
 +
==== Add Another Node to Cluster ====
 
<br>
 
<br>
 
Request:
 
Request:
  
 
<pre>
 
<pre>
PUT /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology
+
POST /instances
 
 
 
{
 
{
 
   "instance": {
 
   "instance": {
 +
    "name": "product-c",
 +
    ...
 
     "topology": {
 
     "topology": {
       "members": [
+
       "cassandra": {
         {
+
         "cluster_name": "products",
          "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
+
         "num_tokens": 256,
         },
+
         "is_seed": false,
         {
+
        "auto_bootstrap": false,
          "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
+
         "join": true
         }
+
       }
       ]
+
     },
     }
+
    ...
 
   }
 
   }
 +
}
 
</pre>
 
</pre>
 
<br>
 
<br>
Notes:
+
Response:
* By omitting a member{} for id=3a72ee87-cf3e-40f1-a1e1-fe8c7263a782 in a PUT operation, this indicates the member should be removed from the replica-set.
+
<pre>
* It's possible that one might want to modify the 'priority', 'hidden', 'votes', etc. fields of the remaining members while dropping a member. So although the example above does not show it, mongodb{} can be included in a member to indicate other changes, *BUT*, since it's a PUT the expectation of what happens to omitted fields in mongodb{} becomes unclear.
+
{
<br>
+
  "instance": {
Summary: Not very clean, mildly confusing, and very error-prone (nowhere is a "remove" action ever explicitly implied).
+
    "status": "BUILD",
<br>
+
    "id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",
<br>
+
    ...
'''Option #2: POST /instances/:id/topology/action'''
+
  }
 +
}
 +
</pre>
 
<br>
 
<br>
 +
==== Show Instance ====
 
<br>
 
<br>
 
Request:
 
Request:
  
 
<pre>
 
<pre>
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/action
+
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998
 +
</pre>
 +
 
 +
Response:
  
 +
<pre>
 
{
 
{
   "mongodb": {
+
   "instance": {
     "remove_member": {
+
    ...
       "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"
+
     "topology": {
     }
+
       "cassandra": {
 +
        "cluster_name": "products",
 +
        "num_tokens": 256,
 +
        "is_seed": true
 +
      }
 +
     },
 +
    ...
 
   }
 
   }
 
}
 
}
 
</pre>
 
</pre>
 
<br>
 
<br>
Notes:
+
==== Show Topology ====
* mongodb{} wrapper isn't necessary, but provides the benefit of schema validation + declaration of intention/understanding.
+
<br>
* The 'remove_member' action is explicit here, vs. implicit as seen in the PUT option.
+
Request:
* 'remove_member' has a strict set of fields that are supported, so there is no question as to what can be provided and what will be honored (as compared to the PUT).
+
 
<br>
 
Summary: Fairly clean, with no real drawbacks.
 
<br>
 
<br>
 
'''Option #3: POST /instances/:id/topology/remove'''
 
<br>
 
<br>
 
 
<pre>
 
<pre>
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/remove
+
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology
 +
</pre>
 +
 
 +
Response:
  
 +
<pre>
 
{
 
{
   "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"
+
   "topology": {
}
+
    "members": [
</pre>
+
      {
<br>
+
        "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
Notes:
+
        "name": "product-a",
* Differs from Option #2 in that the action is in the URI vs. the payload.
+
        ...
* One drawback of this approach is that not every action will be supported across all datastores. So for example, a POST /instances/:id/topology/changeoplogsize (http://docs.mongodb.org/manual/tutorial/change-oplog-size/) makes absolutely no sense to any datastore other than MongoDB.
+
        "cassandra": {
 +
          "cluster_name": "products",
 +
          "num_tokens": 256,
 +
          "is_seed": true
 +
        }
 +
      },
 +
      {
 +
        "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
 +
        "name": "product-b",
 +
        ...
 +
        "cassandra": {
 +
          "cluster_name": "products",
 +
          "num_tokens": 256,
 +
          "is_seed": false
 +
        }
 +
      },
 +
      {
 +
        "id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",
 +
        "name": "product-c",
 +
        ...
 +
        "cassandra": {
 +
          "cluster_name": "products",
 +
          "num_tokens": 256,
 +
          "is_seed": false
 +
        }
 +
      }
 +
    ]
 +
  }
 +
}
 +
</pre>
 
<br>
 
<br>
Summary: At first glance is cleaner than Option #2 from a payload-perspective, but the URI discoverability and expansion is awful.
+
==== Modifying a Cluster ====
<br>
 
<br>
 
'''Option #4: POST /instances/:id/action'''
 
 
<br>
 
<br>
 +
Example: Drain (http://www.datastax.com/documentation/cassandra/2.0/cassandra/tools/toolsDrain.html)
 +
 +
<pre>
 +
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/action
 +
 +
{
 +
  "cassandra": {
 +
    "drain_node": {
 +
      "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"
 +
    }
 +
  }
 +
}
 +
</pre>
 +
 +
 +
==== Remove a Member ====
 
<br>
 
<br>
 +
Request:
 +
 
<pre>
 
<pre>
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/action
+
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/action
  
 
{
 
{
   "join": false
+
   "cassandra": {
 +
    "remove_node": {
 +
      "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"
 +
    }
 +
  }
 
}
 
}
 
</pre>
 
</pre>
 
<br>
 
<br>
Notes:
+
 
* Executed against the instance you wish to remove itself from the cluster, so providing the 'id' in the payload is unnecessary.
+
== Couchbase ==
* Drawback: There are actions that are replica-set-wide (or against a subset of the replica-set), meaning Option #1 or #2 or #3 would have to co-exist with this option anyway.
 
* Drawback: Increase the number of ways to accomplish the same thing (could unjoin against /instances/:id/action, or against /instances/:id/topology)
 
 
<br>
 
<br>
Summary: For this very specific example it looks great, but isn't expressive enough for other actions.
+
==== Create Cluster ====
 
<br>
 
<br>
<br>
+
'''Create Initial Cluster'''
'''Option #5: POST /instances/:id/topology/:id/action'''
 
 
<br>
 
<br>
 
<br>
 
<br>
 +
Request:
 +
 
<pre>
 
<pre>
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/dfbbd9ca-b5e1-4028-adb7-f78643e17998/action
+
POST /instances
 
 
 
{
 
{
   "mongodb": {
+
   "instance": {
     "remove": {}
+
     "name": "product-a",
  }
+
    ...
}
+
    "datastore": {
</pre>
+
      "type": "couchbase",
<br>
+
      "version": "couchbase-2.2"
Notes:
+
    },
* The /instances/:id is an arbitrary member in the replica-set, it doesn't matter which one; the topology/:id is then a member of said replica-set that this action will be applied to.
+
    "topology": {
* Executed against the instance you wish to remove itself from the cluster, so providing the 'id' in the payload is unnecessary.
+
      "couchbase": {
* Needs More Thought: Could conceivably allow only specific operations here (like remove/unjoin), but not others that could be accomplished in a PATCH against /instances/:id/topology (like 'priority', 'hidden', etc.)
+
        "cluster_name": "products",
<br>
+
        "join": false
Summary: Fairly clean with no real drawbacks.
+
      }
<br>
+
    },
<br>
+
    ...
Decision: Option #2; it's extremely fine-grained, easy to reason about, and matches the approach taken in Remove A Member.
+
  }
 +
}
 +
</pre>
  
 +
Response:
  
==== MongoDB TokuMX ====
+
<pre>
 
+
{
* TokuMUX will require a new datastore-version and *possibly* a new manager class (same reasoning as why Tungsten/Galera will have their own datastore-version for MySQL)
+
  "instance": {
 
+
    "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
== Cassandra ==
+
    ...
 +
  }
 +
}
 +
</pre>
 
<br>
 
<br>
==== Create Cluster ====
+
Notes:
<br>
+
* Couchbase does not support a "cluster name", but does require a xdcr-cluster-name when using XDCR.
<br>
+
* Tip: Summary of operations at http://docs.couchbase.com/couchbase-manual-2.2/#xdcr-replicate-options
'''Create Initial Cluster'''
 
 
<br>
 
<br>
 +
==== Add Node to Cluster ====
 
<br>
 
<br>
 
Request:
 
Request:
Line 1,010: Line 1,004:
 
{
 
{
 
   "instance": {
 
   "instance": {
     "name": "product-a",
+
     "name": "product-b",
 
     ...
 
     ...
    "datastore": {
 
      "type": "cassandra",
 
      "version": "cassandra-2.0.5"
 
    },
 
 
     "topology": {
 
     "topology": {
       "cassandra": {
+
       "couchbase": {
 
         "cluster_name": "products",
 
         "cluster_name": "products",
         "num_tokens": 256,
+
         "join": true
        "is_seed": true,
 
        "endpoint_snitch": "RackInferringSnitch",
 
        "join": false
 
 
       }
 
       }
 
     },
 
     },
Line 1,029: Line 1,016:
 
}
 
}
 
</pre>
 
</pre>
 
+
<br>
 
Response:
 
Response:
 
 
<pre>
 
<pre>
 
{
 
{
 
   "instance": {
 
   "instance": {
     "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
+
    "status": "BUILD",
 +
     "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
 
     ...
 
     ...
 
   }
 
   }
Line 1,042: Line 1,029:
 
<br>
 
<br>
 
Notes:
 
Notes:
* Unlike in MongoDB, the 'type' field is not required (because all members of the cluster are of the same type)
+
* If 'join' is true, and there is no existing cluster for the tenant matching the 'cluster_name' value, the request will be failed.
* 'cluster_name', 'num_tokens', and 'is_seed' are always required, with 'endpoint_snitch' being required if 'join' is false (if 'join' is true, the endpoint_snitch is inherited) and 'auto_bootstrap' required if 'join' is true.
+
* In Couchbase, a new node can join the cluster by referencing any existing member, which is discoverable by trove via the cluster_name (unique per tenant)
* 'seed_provider' can optionally be provided, but conveniently defaults to 'org.apache.cassandra.locator.SimpleSeedProvider'
+
<br>
* 'join' indicates whether you're joining an existing cluster, or creating a new one. If 'join' is false, and an active cluster by that name for the tenant already exists, the request will be failed. 'join' by default will be false, but was included above for illustrative purposes.
+
==== Add Another Node to Cluster ====
* Likely the Keystone region and the availability-zone inherent to the trove request can be used for the data-center and rack (cassandra-topology.properties), but if it turns out that the naming schemes are incompatible, 'data_center' and 'rack' can be introduced.
+
<br>
* 'is_seed' is used vs. a seed list of ip-addresses because (1) the ip-address is not yet known and (2) when additional seeds are added, each node in the cluster must be notified and updated. More on this later.
+
Request/Response omitted due to a lack of any special considerations required
 
 
 
<br>
 
<br>
==== Add Node to Cluster ====
 
 
<br>
 
<br>
 +
==== Show Instance ====
 
<br>
 
<br>
Request:
+
Request/Response omitted due to a lack of any special considerations required
 
+
<br>
 +
<br>
 +
==== Show Topology ====
 +
<br>
 +
Request/Response omitted due to a lack of any special considerations required
 +
<br>
 +
<br>
 +
==== Modifying a Cluster ====
 +
<br>
 +
'''POST /instances/:id/topology/action'''
 +
<br>
 +
<br>
 +
Example: Create Bucket
 +
 
 
<pre>
 
<pre>
POST /instances
+
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/action
 +
 
 
{
 
{
   "instance": {
+
   "couchbase": {
     "name": "product-b",
+
     "create_bucket": {
    ...
+
       "bucket": "test_bucket",
    "topology": {
+
      "bucket_type": "couchbase",
       "cassandra": {
+
      "bucket_port": 11222,
        "cluster_name": "products",
+
      "bucket_ramsize": 200,
        "num_tokens": 256,
+
      "bucket_replica": 1
        "is_seed": false,
+
     }
        "auto_bootstrap": false,
 
        "join": true
 
      }
 
     },
 
    ...
 
 
   }
 
   }
 
}
 
}
 
</pre>
 
</pre>
 +
 +
 +
==== Remove a Member ====
 
<br>
 
<br>
Response:
+
'''POST /instances/:id/topology/action'''
 +
<br>
 +
<br>
 +
Request:
 +
 
 
<pre>
 
<pre>
 +
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/action
 +
 
{
 
{
   "instance": {
+
   "couchbase": {
     "status": "BUILD",
+
     "rebalance": {
    "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
+
      "server_remove": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"
     ...
+
     }
 
   }
 
   }
 
}
 
}
Line 1,087: Line 1,091:
 
<br>
 
<br>
 
Notes:
 
Notes:
* If 'join' is true, and there is no existing cluster for the tenant matching the 'cluster_name' value, the request will be failed.
+
* Note how removing a node is done in the context of a rebalance for Couchbase. This is an example of how homogenizing actions across trove, like "remove node", might not be appropriate.
* If 'endpoint_snitch' is provided, and the value does not match that of the existing node(s) in the cluster, the request will be failed.
+
 
<br>
+
== Redis ==
==== Add Another Node to Cluster ====
 
 
<br>
 
<br>
 +
==== Create Master ====
 
<br>
 
<br>
 
Request:
 
Request:
Line 1,099: Line 1,103:
 
{
 
{
 
   "instance": {
 
   "instance": {
     "name": "product-c",
+
     "name": "product-a",
 
     ...
 
     ...
 +
    "datastore": {
 +
      "type": "redis",
 +
      "version": "redis-2.8.6"
 +
    },
 
     "topology": {
 
     "topology": {
       "cassandra": {
+
       "redis": {}
        "cluster_name": "products",
+
     },
        "num_tokens": 256,
+
     ...
        "is_seed": false,
 
        "auto_bootstrap": false,
 
        "join": true
 
      }
 
     },
 
     ...
 
 
   }
 
   }
 
}
 
}
 
</pre>
 
</pre>
<br>
+
 
 
Response:
 
Response:
 +
 
<pre>
 
<pre>
 
{
 
{
 
   "instance": {
 
   "instance": {
    "status": "BUILD",
+
     "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
     "id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",
 
 
     ...
 
     ...
 
   }
 
   }
Line 1,126: Line 1,128:
 
</pre>
 
</pre>
 
<br>
 
<br>
==== Show Instance ====
+
Notes:
 +
* 'join' field is not required for redis. also notice the lack of a 'cluster_name' of sorts (see 'Add Slave' for reasoning)
 +
 
 
<br>
 
<br>
 +
==== Add Slave ====
 
<br>
 
<br>
 
Request:
 
Request:
  
 
<pre>
 
<pre>
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998
+
POST /instances
</pre>
 
 
 
Response:
 
 
 
<pre>
 
 
{
 
{
 
   "instance": {
 
   "instance": {
 +
    "name": "product-b",
 
     ...
 
     ...
 +
    "datastore": {
 +
      "type": "redis",
 +
      "version": "redis-2.8.6"
 +
    },
 
     "topology": {
 
     "topology": {
       "cassandra": {
+
       "redis": {
         "cluster_name": "products",
+
         "slave_of": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"
        "num_tokens": 256,
 
        "is_seed": true
 
 
       }
 
       }
 
     },
 
     },
Line 1,153: Line 1,156:
 
</pre>
 
</pre>
 
<br>
 
<br>
==== Show Topology ====
 
<br>
 
<br>
 
Request:
 
 
<pre>
 
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology
 
</pre>
 
 
 
Response:
 
Response:
 
 
<pre>
 
<pre>
 
{
 
{
 
   "instance": {
 
   "instance": {
 +
    "status": "BUILD",
 +
    "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
 
     ...
 
     ...
    "topology": {
+
  }
      "members": [
+
}
        {
+
</pre>
          "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
+
<br>
          "name": "product-a",
+
Notes:
          ...
+
* redis supports daisy-chaining slaves, therefore the 'slave_of' value needs to be a specific trove instance uuid vs. a manufactured 'cluster_name' of sorts.
          "cassandra": {
+
* whether it is a master or slave can be inferred from the presence (or lack thereof) of 'slave_of'.
            "cluster_name": "products",
 
            "num_tokens": 256,
 
            "is_seed": true
 
          }
 
        },
 
        {
 
          "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
 
          "name": "product-b",
 
          ...
 
          "cassandra": {
 
            "cluster_name": "products",
 
            "num_tokens": 256,
 
            "is_seed": false
 
          }
 
        },
 
        {
 
          "id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",
 
          "name": "product-c",
 
          ...
 
          "cassandra": {
 
            "cluster_name": "products",
 
            "num_tokens": 256,
 
            "is_seed": false
 
          }
 
        }
 
      ]
 
    }
 
    ...
 
  }
 
}
 
</pre>
 
 
<br>
 
<br>
==== Modifying a Cluster ====
+
==== Show Instance ====
 
<br>
 
<br>
<br>
+
Request:
'''POST /instances/:id/topology/action'''
+
 
<br>
+
<pre>
<br>
+
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998
Example: Drain (http://www.datastax.com/documentation/cassandra/2.0/cassandra/tools/toolsDrain.html)
+
</pre>
 +
 
 +
Response:
  
 
<pre>
 
<pre>
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/action
 
 
 
{
 
{
   "cassandra": {
+
   "instance": {
     "drain_node": {
+
    ...
       "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"
+
     "topology": {
     }
+
       "redis": {}
 +
     },
 +
    ...
 
   }
 
   }
 
}
 
}
 
</pre>
 
</pre>
 
 
==== Remove a Member ====
 
<br>
 
<br>
 
'''POST /instances/:id/topology/action'''
 
 
<br>
 
<br>
 +
==== Show Topology ====
 
<br>
 
<br>
 
Request:
 
Request:
  
 
<pre>
 
<pre>
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/action
+
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology
 +
</pre>
 +
 
 +
Response:
  
 +
<pre>
 
{
 
{
   "cassandra": {
+
   "topology": {
     "remove_node": {
+
     "members": [
       "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"
+
      {
     }
+
        "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
 +
        "name": "product-a",
 +
        ...
 +
        "redis": {}
 +
      },
 +
       {
 +
        "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
 +
        "name": "product-b",
 +
        ...
 +
        "redis": {
 +
          "slave_of": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"
 +
        }
 +
      }
 +
     ]
 
   }
 
   }
 
}
 
}
 
</pre>
 
</pre>
 
<br>
 
<br>
 
+
==== Promote/Disconnect Slave ====
== Couchbase ==
 
 
<br>
 
<br>
==== Create Cluster ====
+
'''POST /instances/:id/topology/action'''
 +
<br>
 +
<br>
 +
Request:
 +
 
 +
<pre>
 +
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/action
 +
 
 +
{
 +
  "redis": {
 +
    "slaveof_no_one": {
 +
      "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"
 +
    }
 +
  }
 +
}
 +
</pre>
 
<br>
 
<br>
'''Create Initial Cluster'''
+
 
 +
== Redis Cluster ==
 
<br>
 
<br>
 +
==== Create Cluster ====
 
<br>
 
<br>
 
Request:
 
Request:
Line 1,265: Line 1,259:
 
     ...
 
     ...
 
     "datastore": {
 
     "datastore": {
       "type": "couchbase",
+
       "type": "redis",
       "version": "couchbase-2.2"
+
       "version": "redis-3.0.0-beta1"
 
     },
 
     },
 
     "topology": {
 
     "topology": {
       "couchbase": {
+
       "redis_cluster": {
 
         "cluster_name": "products",
 
         "cluster_name": "products",
 +
        "cluster_timeout": 5000,
 
         "join": false
 
         "join": false
 
       }
 
       }
Line 1,291: Line 1,286:
 
<br>
 
<br>
 
Notes:
 
Notes:
* Couchbase does not support a "cluster name", but does require a xdcr-cluster-name when using XDCR.
+
* Will require datastore-version to topology.<namespace> validation (e.g. if redis 3+, permit topology.redis_cluster{} and topology.redis{}, but if redis 2.x only permit topology.redis{})
* Tip: Summary of operations at http://docs.couchbase.com/couchbase-manual-2.2/#xdcr-replicate-options
+
* 'cluster_name' and 'cluster_timeout' are unique to topology.redis_cluster{} (not supported in topology.redis{})
 +
* As seen in other datastore examples, uses 'join' and 'cluster_name'
 
<br>
 
<br>
==== Add Node to Cluster ====
+
==== Add Another Master to Cluster ====
 
<br>
 
<br>
 
Request:
 
Request:
Line 1,304: Line 1,300:
 
     "name": "product-b",
 
     "name": "product-b",
 
     ...
 
     ...
 +
    "datastore": {
 +
      "type": "redis",
 +
      "version": "redis-3.0.0-beta1"
 +
    },
 
     "topology": {
 
     "topology": {
       "couchbase": {
+
       "redis_cluster": {
 
         "cluster_name": "products",
 
         "cluster_name": "products",
 
         "join": true
 
         "join": true
Line 1,326: Line 1,326:
 
</pre>
 
</pre>
 
<br>
 
<br>
Notes:
+
==== Add Slave to Cluster ====
* If 'join' is true, and there is no existing cluster for the tenant matching the 'cluster_name' value, the request will be failed.
 
* In Couchbase, a new node can join the cluster by referencing any existing member, which is discoverable by trove via the cluster_name (unique per tenant)
 
<br>
 
==== Add Another Node to Cluster ====
 
<br>
 
Request/Response omitted due to a lack of any special considerations required
 
 
<br>
 
<br>
 +
Request:
 +
 +
<pre>
 +
POST /instances
 +
{
 +
  "instance": {
 +
    "name": "product-c",
 +
    ...
 +
    "datastore": {
 +
      "type": "redis",
 +
      "version": "redis-3.0.0-beta1"
 +
    },
 +
    "topology": {
 +
      "redis_cluster": {
 +
        "cluster_name": "products",
 +
        "slave_of": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"
 +
      }
 +
    },
 +
    ...
 +
  }
 +
}
 +
</pre>
 
<br>
 
<br>
==== Show Instance ====
+
Response:
<br>
+
<pre>
Request/Response omitted due to a lack of any special considerations required
+
{
<br>
+
  "instance": {
<br>
+
    "status": "BUILD",
==== Show Topology ====
+
    "id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",
<br>
+
    ...
Request/Response omitted due to a lack of any special considerations required
+
  }
 +
}
 +
</pre>
 
<br>
 
<br>
 +
Notes:
 +
* Despite redis clusters supporting the ability to add a slave without designating the master, we will require it to avoid unoptimal geographical relationships.
 
<br>
 
<br>
==== Modifying a Cluster ====
+
==== Show Instance ====
 
<br>
 
<br>
'''POST /instances/:id/topology/action'''
+
Request:
<br>
 
<br>
 
Example: Create Bucket
 
  
 
<pre>
 
<pre>
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/action
+
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998
 +
</pre>
 +
 
 +
Response:
  
 +
<pre>
 
{
 
{
   "couchbase": {
+
   "instance": {
     "create_bucket": {
+
    ...
       "bucket": "test_bucket",
+
     "topology": {
      "bucket_type": "couchbase",
+
       "redis_cluster": {
      "bucket_port": 11222,
+
        "cluster_name": "products",
       "bucket_ramsize": 200,
+
        "cluster_timeout": 5000
      "bucket_replica": 1
+
       }
     }
+
    },
 +
     ...
 
   }
 
   }
 
}
 
}
 
</pre>
 
</pre>
 
 
==== Remove a Member ====
 
<br>
 
'''POST /instances/:id/topology/action'''
 
 
<br>
 
<br>
 +
==== Show Topology ====
 
<br>
 
<br>
 
Request:
 
Request:
  
 
<pre>
 
<pre>
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/action
+
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology
 +
</pre>
  
 +
Response:
 +
 +
<pre>
 
{
 
{
   "couchbase": {
+
   "topology": {
     "rebalance": {
+
     "members": [
       "server_remove": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"
+
      {
     }
+
        "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
 +
        "name": "product-a",
 +
        ...
 +
        "redis_cluster": {
 +
          "cluster_name": "products",
 +
          "cluster_timeout": 5000
 +
        }
 +
      },
 +
       {
 +
        "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
 +
        "name": "product-b",
 +
        ...
 +
        "redis_cluster": {
 +
          "cluster_name": "products",
 +
          "cluster_timeout": 5000
 +
        }
 +
      },
 +
      {
 +
        "id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",
 +
        "name": "product-c",
 +
        ...
 +
        "redis_cluster": {
 +
          "cluster_name": "products",
 +
          "cluster_timeout": 5000,
 +
          "slave_of": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"
 +
        }
 +
      }
 +
     ]
 
   }
 
   }
 
}
 
}
 
</pre>
 
</pre>
 
<br>
 
<br>
Notes:
+
==== Promote/Disconnect Slave ====
* Note how removing a node is done in the context of a rebalance for Couchbase. This is an example of how homogenizing actions across trove, like "remove node", might not be appropriate.
+
<br>
 +
Work in Progress
  
== Redis ==
+
== Feedback ==
<br>
 
==== Create Master ====
 
<br>
 
Request:
 
 
 
<pre>
 
POST /instances
 
{
 
  "instance": {
 
    "name": "product-a",
 
    ...
 
    "datastore": {
 
      "type": "redis",
 
      "version": "redis-2.8.6"
 
    },
 
    "topology": {
 
      "redis": {}
 
    },
 
    ...
 
  }
 
}
 
</pre>
 
  
Response:
+
==== Masters/Slaves ====
 
+
*drewford: Can one make a master/slave out of an existing instance?
<pre>
+
*drewford: Type and version should follow the same schema as listing datastore types
{
 
  "instance": {
 
    "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
 
    ...
 
  }
 
}
 
</pre>
 
<br>
 
Notes:
 
* 'join' field is not required for redis. also notice the lack of a 'cluster_name' of sorts (see 'Add Slave' for reasoning)
 
 
 
<br>
 
==== Add Slave ====
 
<br>
 
Request:
 
 
 
<pre>
 
POST /instances
 
{
 
  "instance": {
 
    "name": "product-b",
 
    ...
 
    "datastore": {
 
      "type": "redis",
 
      "version": "redis-2.8.6"
 
    },
 
    "topology": {
 
      "redis": {
 
        "slave_of": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"
 
      }
 
    },
 
    ...
 
  }
 
}
 
</pre>
 
<br>
 
Response:
 
<pre>
 
{
 
  "instance": {
 
    "status": "BUILD",
 
    "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
 
    ...
 
  }
 
}
 
</pre>
 
<br>
 
Notes:
 
* redis supports daisy-chaining slaves, therefore the 'slave_of' value needs to be a specific trove instance uuid vs. a manufactured 'cluster_name' of sorts.
 
* whether it is a master or slave can be inferred from the presence (or lack thereof) of 'slave_of'.
 
<br>
 
==== Show Instance ====
 
<br>
 
Request:
 
 
 
<pre>
 
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998
 
</pre>
 
 
 
Response:
 
 
 
<pre>
 
{
 
  "instance": {
 
    ...
 
    "topology": {
 
      "redis": {}
 
    },
 
    ...
 
  }
 
}
 
</pre>
 
<br>
 
==== Show Topology ====
 
<br>
 
Request:
 
 
 
<pre>
 
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology
 
</pre>
 
 
 
Response:
 
 
 
<pre>
 
{
 
  "instance": {
 
    ...
 
    "topology": {
 
      "members": [
 
        {
 
          "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
 
          "name": "product-a",
 
          ...
 
          "redis": {}
 
        },
 
        {
 
          "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
 
          "name": "product-b",
 
          ...
 
          "redis": {
 
            "slave_of": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"
 
          }
 
        }
 
      ]
 
    }
 
    ...
 
  }
 
}
 
</pre>
 
<br>
 
==== Promote/Disconnect Slave ====
 
<br>
 
'''POST /instances/:id/topology/action'''
 
<br>
 
<br>
 
Request:
 
 
 
<pre>
 
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/action
 
 
 
{
 
  "redis": {
 
    "slaveof_no_one": {
 
      "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"
 
    }
 
  }
 
}
 
</pre>
 
<br>
 
 
 
== Redis Cluster ==
 
<br>
 
==== Create Cluster ====
 
<br>
 
Request:
 
 
 
<pre>
 
POST /instances
 
{
 
  "instance": {
 
    "name": "product-a",
 
    ...
 
    "datastore": {
 
      "type": "redis",
 
      "version": "redis-3.0.0-beta1"
 
    },
 
    "topology": {
 
      "redis_cluster": {
 
        "cluster_name": "products",
 
        "cluster_timeout": 5000,
 
        "join": false
 
      }
 
    },
 
    ...
 
  }
 
}
 
</pre>
 
 
 
Response:
 
 
 
<pre>
 
{
 
  "instance": {
 
    "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
 
    ...
 
  }
 
}
 
</pre>
 
<br>
 
Notes:
 
* Will require datastore-version to topology.<namespace> validation (e.g. if redis 3+, permit topology.redis_cluster{} and topology.redis{}, but if redis 2.x only permit topology.redis{})
 
* 'cluster_name' and 'cluster_timeout' are unique to topology.redis_cluster{} (not supported in topology.redis{})
 
* As seen in other datastore examples, uses 'join' and 'cluster_name'
 
<br>
 
==== Add Another Master to Cluster ====
 
<br>
 
Request:
 
 
 
<pre>
 
POST /instances
 
{
 
  "instance": {
 
    "name": "product-b",
 
    ...
 
    "datastore": {
 
      "type": "redis",
 
      "version": "redis-3.0.0-beta1"
 
    },
 
    "topology": {
 
      "redis_cluster": {
 
        "cluster_name": "products",
 
        "join": true
 
      }
 
    },
 
    ...
 
  }
 
}
 
</pre>
 
<br>
 
Response:
 
<pre>
 
{
 
  "instance": {
 
    "status": "BUILD",
 
    "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
 
    ...
 
  }
 
}
 
</pre>
 
<br>
 
==== Add Slave to Cluster ====
 
<br>
 
Request:
 
 
 
<pre>
 
POST /instances
 
{
 
  "instance": {
 
    "name": "product-c",
 
    ...
 
    "datastore": {
 
      "type": "redis",
 
      "version": "redis-3.0.0-beta1"
 
    },
 
    "topology": {
 
      "redis_cluster": {
 
        "cluster_name": "products",
 
        "slave_of": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"
 
      }
 
    },
 
    ...
 
  }
 
}
 
</pre>
 
<br>
 
Response:
 
<pre>
 
{
 
  "instance": {
 
    "status": "BUILD",
 
    "id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",
 
    ...
 
  }
 
}
 
</pre>
 
<br>
 
Notes:
 
* Despite redis clusters supporting the ability to add a slave without designating the master, we will require it to avoid unoptimal geographical relationships.
 
<br>
 
==== Show Instance ====
 
<br>
 
Request:
 
  
<pre>
+
==== Arbiters ====
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998
+
*drewford: Could/should arbiters be managed by the system automatically?  Is it worth putting that responsibility on the API user, or can it be automated by the system for an easier, less technical experience?  From reading the article noted in the link, it seems like it's a good enough idea to automate it to reduce the need for users to research and decide for themselves.
</pre>
 
  
Response:
+
==== Delayed Members ====
 +
*drewford: Could/should delayed members be managed by the system automatically? Can it just be fully automated for a better, less technical experience?
  
<pre>
+
==== Topology ====
{
+
*drewford: Members of the topology in the mongo example all have the same "mongodb" object. Unless this could also be "mysql" or "redis" in the same topology, it might make sense to move up "type" and "replSet" attributes as siblings to "name".
  "instance": {
+
*drewford: The "type" attribute also seems redundant to the name of the array. Are there attributes other than "members" on a "topology"?  If not, can "topology" be the array with each object having a "type"?
    ...
 
    "topology": {
 
      "redis_cluster": {
 
        "cluster_name": "products",
 
        "cluster_timeout": 5000
 
      }
 
    },
 
    ...
 
  }
 
}
 
</pre>
 
<br>
 
==== Show Topology ====
 
<br>
 
Request:
 
  
<pre>
+
==== Cassandra Create Cluster ====
GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology
+
*drewford: can "num_tokens", "endpoint_snitch" or "auto_bootstrap" be automated in any way so they receive a valid default value even if the user does not specify one?  Looking for ways in which we can take the decision workload off the user, and put it on the system in a healthy way.
</pre>
 
  
Response:
+
==== Promote/Disconnect Redis Slave ====
 +
*drewford: Seems like the promotion/disconnection json should be structured the same across all datastores if possible - with the backend handling any special manipulation to get the job done.  Standardize in favor of the user.
  
<pre>
+
==== Redis Cluster - Create Cluster ====
{
+
*drewford: "redis_cluster" here is labelled differently than the same object in Cassandra and Couchbase - they don't use "_cluster".
  "instance": {
 
    ...
 
    "topology": {
 
      "members": [
 
        {
 
          "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
 
          "name": "product-a",
 
          ...
 
          "redis_cluster": {
 
            "cluster_name": "products",
 
            "cluster_timeout": 5000
 
          }
 
        },
 
        {
 
          "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
 
          "name": "product-b",
 
          ...
 
          "redis_cluster": {
 
            "cluster_name": "products",
 
            "cluster_timeout": 5000
 
          }
 
        },
 
        {
 
          "id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",
 
          "name": "product-c",
 
          ...
 
          "redis_cluster": {
 
            "cluster_name": "products",
 
            "cluster_timeout": 5000,
 
            "slave_of": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"
 
          }
 
        }
 
      ]
 
    }
 
    ...
 
  }
 
}
 
</pre>
 
<br>
 
==== Promote/Disconnect Slave ====
 
<br>
 
Work in Progress
 

Latest revision as of 19:16, 12 May 2014

MySQL Master/Slave


Create Master


Request:

POST /instances
{
  "instance": {
    "availability_zone": "us-west-1",
    "name": "product-a",
    "datastore": {
      "type": "mysql",
      "version": "mysql-5.5"
    },
    "configuration": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6",
    "flavorRef": "7",
    "volume": {
      "size": 1
    }
  }
}

Response:

{
  "instance": {
    "status": "BUILD",
    "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
    "name": "product-a",
    "configuration": {
      "id": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6",
      "name": "config-a",
      "links": [{...}]
    },
    ...
  }
}


Create Slave


Request:

POST /instances
{
  "instance": {
    "availability_zone": "us-west-2",
    "name": "product-b",
    "datastore": {
      "type": "mysql",
      "version": "mysql-5.5"
    },
    "topology": {
      "mysql": {
        "slave_of": [{"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"}]
        "read_only": true
      }
    },
    "configuration": "fc318e00-3a6f-4f93-af99-146b44912188",
    "flavorRef": "7",,
    "volume": {
      "size": 1
    }
  }
}

Response:

{
  "instance": {
    "status": "BUILD",
    "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
    "name": "product-b",
    "configuration": {
      "id": "fc318e00-3a6f-4f93-af99-146b44912188",
      "name": "config-b",
      "links": [{...}]
    },
    ...
  }
}

Notes:

  • For master/slave wirings, the 'server_id' must differ between master and slave, and optionally the slave can specify whether it is read-only or not to avoid accidental writes.
  • Update: Agreed on 03/14/14 that the user should not have to specify 'server_id'. Instead, trove will be responsible for setting it and ensuring that a slave does not have the same server_id as its master, or any sibling slaves. As a part of this agreement, this requires removing 'server_id' from configuration-groups overrides (to avoid the user meddling with our bookkeeping).
  • 'read_only' was removed from configuration-groups and moved to topology.mysql{} because forcing the user to create a configuration-group for every MySQL slave is arduous and a poor user experience.
  • 'slave_of' and 'read_only' will be the only supported input fields for topology.mysql{}
  • 'read_only' is only permitted if 'slave_of' is set, otherwise the request will be failed.
  • Opinion: 'read_only' should default to true
  • 'slave_of' is an array to properly represent multi-source replication in the future (coming in MySQL 5.7)
  • For now, 'slave_of' requires a vanilla trove instance uuid, but will inevitably need to be prefixed with namespacing to support multiple dcs and sources (e.g. trove:us-west:tenant_id:instance:dfbbd9ca-b5e1-4028-adb7-f78643e17998)
  • Why topology.<datastore>{} vs. topology.metadata{}? This should be more evident as you walk through the rest of the datatores, but in short: it provides strong(er) typing, allows for apischema validation, easier to document, simpler to maintain, etc.


Show Instance


Request:

GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998

Response:

{
  "instance": {
    "status": "ACTIVE",
    "updated": "2014-02-16T03:38:49"
    "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
    "name": "product-a",
    "datastore": {
      "version": "mysql-5.5",
      "type": "mysql",
    },
    "flavor": {
      "id": "7",
      "links": [{...}]
    },
    "configuration": {
      "id": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6",
      "name": "config-a",
      "links": [{...}]
    }
  }
}


Show Topology


Request:

GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology

Response:

{
  "topology": {
    "members": [
      {
        "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
        "name": "product-a"
      },
      {
        "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
        "name": "product-b",
        "mysql": {
          "slave_of": [{"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"}],
          "read_only": true
        }
      }
    ]
  }
}


Remove Replication (aka "Promote" to Standalone)


Request:

POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/action

{
  "mysql": {
    "promote": {
      "id": "061aaf4c-3a57-411e-9df9-2d0f813db859"
    }
  }
}

Response:

TBD


MongoDB


Create Replica-Set


Request:

POST /instances
{
  "instance": {
    "name": "product-a",
    ...
    "datastore": {
      "type": "mongodb",
      "version": "mongodb-2.0.4"
    },
    "topology": {
      "mongodb": {
        "type": "member",
        "replSet": "products",
        "join": false
      }
    },
    ...
  }
}

Response:

{
  "instance": {
    "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
    ...
  }
}


Notes:

  • Enforce 'replSet' field to be provided for MongoDB, even in the case of a standalone/single instance. See http://www.mongodb.com/blog/post/dont-let-your-standalone-mongodb-server-stand-alone for reasoning.
  • 'join' indicates whether you're joining an existing replica-set, or creating a new one. If 'join' is false, and an active replica-set by that name for the tenant already exists, the request will be failed. 'join' by default will be false, but was included above for illustrative purposes.
  • 'type' is 'member' vs. 'primary' because in a replica-set, the primary is dynamic and can change in an election.


Add Member to Replica-Set


Request:

POST /instances
{
  "instance": {
    "name": "product-b",
    ...
    "topology": {
      "mongodb": {
        "type": "member",
        "replSet": "products",
        "join": true
      }
    },
    ...
  }
}


Response:

{
  "instance": {
    "status": "BUILD",
    "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
    ...
  }
}


Notes:

  • If 'join' is true, and there is no existing replica-set for the tenant matching the 'replSet' value, the request will be failed.
  • Will have to use 'db.isMaster()' to determine the current primary to execute replica-set commands against (since it can be dynamic due to elections)
  • Will use http://docs.mongodb.org/manual/tutorial/expand-replica-set/#configure-and-add-a-member
  • Should protect against adding more than 12 members to a replica-set
  • Should protect against adding more than 7 voting members to a replica-set
  • Should return warning when number of voting members is even and there is no arbiter


Add Another Member to Replica-Set


Request:

POST /instances
{
  "instance": {
    "name": "product-c",
    ...
    "topology": {
      "mongodb": {
        "type": "member",
        "replSet": "products",
        "join": true
      }
    },
    ...
  }
}


Response:

{
  "instance": {
    "status": "BUILD",
    "id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",
    ...
  }
}


Show Instance


Request:

GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998

Response:

{
  "instance": {
    ...
    "topology": {
      "mongodb": {
        "type": "member",
        "replSet": "products"
      }
    },
    ...
  }
}


Show Topology


Request:

GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology

Response:

{
  "topology": {
    "members": [
      {
        "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
        "name": "product-a",
        ...
        "mongodb": {
          "type": "member",
          "replSet": "products"
        }
      },
      {
        "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
        "name": "product-b",
        ...
        "mongodb": {
          "type": "member",
          "replSet": "products"
        }
      },
      {
        "id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",
        "name": "product-c",
        ...
        "mongodb": {
          "type": "member",
          "replSet": "products"
        }
      }
    ]
  }
}


Add Arbiter


Request:

POST /instances
{
  "instance": {
    "name": "product-arbiter",
    ...
    "topology": {
      "mongodb": {
        "type": "arbiter",
        "replSet": "products",
        "join": true
      }
    },
    ...
  }
}

Response:

{
  "instance": {
    "status": "BUILD",
    "id": "a1b62aaa-7863-4384-8250-59024141c1f8",
    ...
  }
}


Add a Delayed Member


Request:

POST /instances
{
  "instance": {
    "name": "product-delayed",
    ...
    "topology": {
      "mongodb": {
        "type": "member",
        "replSet": "products",
        "join": true,
        "priority": 0,
        "hidden": true,
        "slaveDelay": 3600
      }
    },
    ...
  }
}

Response:

{
  "instance": {
    "status": "BUILD",
    "id": "7d8eb019-931b-4b2a-88d2-4c9f0ca1b29e",
    ...
  }
}


Notes:

  • 'type', 'replSet', 'join', 'priority', 'hidden', and 'slaveDelay' are the only fields supported in topology.mongodb{}. All other configuration values must be set via a configuration-group. After more thought, consider supporting 'hostname' and 'votes' as well.
  • Why isn't 'priority', 'hidden' and 'slaveDelay' in a configuration-group you ask? This is explained in "Modifying a Replica-Set" below.


Modifying a Replica-Set


Thus far we've been able to model building a replica-set, adding an arbiter, adding a delayed secondary member, etc. Let's continue with how to modify a replica-set.

Example:

# from http://docs.mongodb.org/manual/tutorial/configure-secondary-only-replica-set-member/#example
cfg = rs.conf()
cfg.members[0].priority = 2
cfg.members[1].priority = 1
cfg.members[2].priority = 0.5
cfg.members[3].priority = 0
rs.reconfig(cfg)

Executing these priority changes one at a time can have catastrophic results, so it must be done as a transaction (with rs.reconfig() commiting). However, without the ability to address the cluster (i.e. multiple members at once), this becomes impossible. The only backdoor solution would be to guarantee that the MongoDB user(s) presented to the cloud tenant all have the clusterAdmin role, as this would allow them to connect to the primary and execute such transactions themselves via the native client. Obviously however, granting clusterAdmin to every DBaaS user in MongoDB is unacceptable in most deployments.

Option #1: PATCH /instances/:id/topology

Request:

PATCH /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology

{
  "topology": {
    "members": [
      {
        "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
        "mongodb": {
          "priority": 2
        }
      },
      {
        "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
        "mongodb": {
          "priority": 1
        }
      },
      {
        "id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",
        "mongodb": {
          "priority": 0.5
        }
      }
    ]
  }
}

Response:

{
  "topology": {
    "members": [
      {
        "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
        "name": "product-a",
        ...
        "mongodb": {
          "type": "member",
         "replSet": "products"
        }
      },
      {
        "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
        "name": "product-b",
        ...
        "mongodb": {
          "type": "member",
          "replSet": "products"
        }
      },
      {
        "id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",
        "name": "product-c",
        ...
        "mongodb": {
          "type": "member",
          "replSet": "products"
        }
      }
    ]
  }
}


Notes:

  • An HTTP PATCH vs. PUT because the omission of a field or structure should not be an indication to drop/delete it.
  • All modified fields in a request will be changed transactionally in a single rs.reconfig().
  • It should now be clear why 'priority', 'hidden' and 'slaveDelay' are in topology.mongodb{} vs. a configuration-group: when a configuration-group is changed, an event is immediately triggered to update any attached trove instances. Therefore, if you have a heterogeneous mixture of configuration-groups in a replica-set, there is no way to coordinate a consolidated rs.reconfig().
  • Downside: if topology.mongodb{} may have fields returned on a GET that you cannot change in a PATCH/PUT; re-worded, the granularity of what is permissible to change in a PATCH becomes complicated to check and validate.
  • TBD on what should be returned in the mongodb{} on a GET /instance/:id and GET /instance/:id/topology. It's a question of whether we should persist anything beyond the 'type' and 'replSet'. If say the 'priority' is stored, you introduce the possibility of drift from the truth, but can easily return it on a GET; if it's not stored, do we prompt MongoDB for the truth on a GET, or is that too computationally expensive?



Option #2: POST /instances/:id/topology/action

Request:

POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/action

{
  "mongodb": {
    "update_members": {
      "members": [
        {
          "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
          "priority": 2
        },
        {
          "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
          "priority": 1
        },
        {
          "id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",
          "priority": 0.5
        }
      ]
    }
  }
}


Notes:

  • 'update_members.members[]' elements will only permit 'priority', 'hidden', and 'slaveDelay' (possibly 'votes' and 'hostname' as mentioned earlier).
  • Due to the limited field-set, this approach is much more fine-grained than the PATCH approach in Option #1.



Decision: Option #2 is more fine-grained, easier to reason about, and less error-prone. As you'll see in later operations (like Remove a Member), the action vs. PUT/PATCH approach is also more appropriate.

Remove a Member


Removing a member is not the same as deleting one, therefore DELETE /instances/:id is not appropriate.

Option #1: PUT /instances/:id/topology

Request:

PUT /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology

{
  "instance": {
    "topology": {
      "members": [
        {
          "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
        },
        {
          "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
        }
      ]
    }
  }


Notes:

  • By omitting a member{} for id=3a72ee87-cf3e-40f1-a1e1-fe8c7263a782 in a PUT operation, this indicates the member should be removed from the replica-set.
  • It's possible that one might want to modify the 'priority', 'hidden', 'votes', etc. fields of the remaining members while dropping a member. So although the example above does not show it, mongodb{} can be included in a member to indicate other changes, *BUT*, since it's a PUT the expectation of what happens to omitted fields in mongodb{} becomes unclear.


Summary: Not very clean, mildly confusing, and very error-prone (nowhere is a "remove" action ever explicitly implied).

Option #2: POST /instances/:id/topology/action

Request:

POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/action

{
  "mongodb": {
    "remove_member": {
      "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"
    }
  }
}


Notes:

  • mongodb{} wrapper isn't necessary, but provides the benefit of schema validation + declaration of intention/understanding.
  • The 'remove_member' action is explicit here, vs. implicit as seen in the PUT option.
  • 'remove_member' has a strict set of fields that are supported, so there is no question as to what can be provided and what will be honored (as compared to the PUT).


Summary: Fairly clean, with no real drawbacks.

Option #3: POST /instances/:id/topology/remove

POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/remove

{
  "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"
}


Notes:

  • Differs from Option #2 in that the action is in the URI vs. the payload.
  • One drawback of this approach is that not every action will be supported across all datastores. So for example, a POST /instances/:id/topology/changeoplogsize (http://docs.mongodb.org/manual/tutorial/change-oplog-size/) makes absolutely no sense to any datastore other than MongoDB.


Summary: At first glance is cleaner than Option #2 from a payload-perspective, but the URI discoverability and expansion is awful.

Option #4: POST /instances/:id/action

POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/action

{
  "join": false
}


Notes:

  • Executed against the instance you wish to remove itself from the cluster, so providing the 'id' in the payload is unnecessary.
  • Drawback: There are actions that are replica-set-wide (or against a subset of the replica-set), meaning Option #1 or #2 or #3 would have to co-exist with this option anyway.
  • Drawback: Increase the number of ways to accomplish the same thing (could unjoin against /instances/:id/action, or against /instances/:id/topology)


Summary: For this very specific example it looks great, but isn't expressive enough for other actions.

Option #5: POST /instances/:id/topology/:id/action

POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/dfbbd9ca-b5e1-4028-adb7-f78643e17998/action

{
  "mongodb": {
    "remove": {}
  }
}


Notes:

  • The /instances/:id is an arbitrary member in the replica-set, it doesn't matter which one; the topology/:id is then a member of said replica-set that this action will be applied to.
  • Executed against the instance you wish to remove itself from the cluster, so providing the 'id' in the payload is unnecessary.
  • Needs More Thought: Could conceivably allow only specific operations here (like remove/unjoin), but not others that could be accomplished in a PATCH against /instances/:id/topology (like 'priority', 'hidden', etc.)


Summary: Fairly clean with no real drawbacks.

Decision: Option #2; it's extremely fine-grained, easy to reason about, and matches the approach taken in Remove A Member.


MongoDB TokuMX

  • TokuMUX will require a new datastore-version and *possibly* a new manager class (same reasoning as why Tungsten/Galera will have their own datastore-version for MySQL)

Cassandra


Create Cluster


Request:

POST /instances
{
  "instance": {
    "name": "product-a",
    ...
    "datastore": {
      "type": "cassandra",
      "version": "cassandra-2.0.5"
    },
    "topology": {
      "cassandra": {
        "cluster_name": "products",
        "num_tokens": 256,
        "is_seed": true,
        "endpoint_snitch": "RackInferringSnitch",
        "join": false
      }
    },
    ...
  }
}

Response:

{
  "instance": {
    "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
    ...
  }
}


Notes:

  • Unlike in MongoDB, the 'type' field is not required (because all members of the cluster are of the same type)
  • 'cluster_name', 'num_tokens', and 'is_seed' are always required, with 'endpoint_snitch' being required if 'join' is false (if 'join' is true, the endpoint_snitch is inherited) and 'auto_bootstrap' required if 'join' is true.
  • 'seed_provider' can optionally be provided, but conveniently defaults to 'org.apache.cassandra.locator.SimpleSeedProvider'
  • 'join' indicates whether you're joining an existing cluster, or creating a new one. If 'join' is false, and an active cluster by that name for the tenant already exists, the request will be failed. 'join' by default will be false, but was included above for illustrative purposes.
  • Likely the Keystone region and the availability-zone inherent to the trove request can be used for the data-center and rack (cassandra-topology.properties), but if it turns out that the naming schemes are incompatible, 'data_center' and 'rack' can be introduced.
  • 'is_seed' is used vs. a seed list of ip-addresses because (1) the ip-address is not yet known and (2) when additional seeds are added, each node in the cluster must be notified and updated. More on this later.


Add Node to Cluster


Request:

POST /instances
{
  "instance": {
    "name": "product-b",
    ...
    "topology": {
      "cassandra": {
        "cluster_name": "products",
        "num_tokens": 256,
        "is_seed": false,
        "auto_bootstrap": false,
        "join": true
      }
    },
    ...
  }
}


Response:

{
  "instance": {
    "status": "BUILD",
    "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
    ...
  }
}


Notes:

  • If 'join' is true, and there is no existing cluster for the tenant matching the 'cluster_name' value, the request will be failed.
  • If 'endpoint_snitch' is provided, and the value does not match that of the existing node(s) in the cluster, the request will be failed.


Add Another Node to Cluster


Request:

POST /instances
{
  "instance": {
    "name": "product-c",
    ...
    "topology": {
      "cassandra": {
        "cluster_name": "products",
        "num_tokens": 256,
        "is_seed": false,
        "auto_bootstrap": false,
        "join": true
      }
    },
    ...
  }
}


Response:

{
  "instance": {
    "status": "BUILD",
    "id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",
    ...
  }
}


Show Instance


Request:

GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998

Response:

{
  "instance": {
    ...
    "topology": {
      "cassandra": {
        "cluster_name": "products",
        "num_tokens": 256,
        "is_seed": true
      }
    },
    ...
  }
}


Show Topology


Request:

GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology

Response:

{
  "topology": {
    "members": [
      {
        "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
        "name": "product-a",
        ...
        "cassandra": {
          "cluster_name": "products",
          "num_tokens": 256,
          "is_seed": true
        }
      },
      {
        "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
        "name": "product-b",
        ...
        "cassandra": {
          "cluster_name": "products",
          "num_tokens": 256,
          "is_seed": false
        }
      },
      {
        "id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",
        "name": "product-c",
        ...
        "cassandra": {
          "cluster_name": "products",
          "num_tokens": 256,
          "is_seed": false
        }
      }
    ]
  }
}


Modifying a Cluster


Example: Drain (http://www.datastax.com/documentation/cassandra/2.0/cassandra/tools/toolsDrain.html)

POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/action

{
  "cassandra": {
    "drain_node": {
      "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"
    }
  }
}


Remove a Member


Request:

POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/action

{
  "cassandra": {
    "remove_node": {
      "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"
    }
  }
}


Couchbase


Create Cluster


Create Initial Cluster

Request:

POST /instances
{
  "instance": {
    "name": "product-a",
    ...
    "datastore": {
      "type": "couchbase",
      "version": "couchbase-2.2"
    },
    "topology": {
      "couchbase": {
        "cluster_name": "products",
        "join": false
      }
    },
    ...
  }
}

Response:

{
  "instance": {
    "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
    ...
  }
}


Notes:


Add Node to Cluster


Request:

POST /instances
{
  "instance": {
    "name": "product-b",
    ...
    "topology": {
      "couchbase": {
        "cluster_name": "products",
        "join": true
      }
    },
    ...
  }
}


Response:

{
  "instance": {
    "status": "BUILD",
    "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
    ...
  }
}


Notes:

  • If 'join' is true, and there is no existing cluster for the tenant matching the 'cluster_name' value, the request will be failed.
  • In Couchbase, a new node can join the cluster by referencing any existing member, which is discoverable by trove via the cluster_name (unique per tenant)


Add Another Node to Cluster


Request/Response omitted due to a lack of any special considerations required

Show Instance


Request/Response omitted due to a lack of any special considerations required

Show Topology


Request/Response omitted due to a lack of any special considerations required

Modifying a Cluster


POST /instances/:id/topology/action

Example: Create Bucket

POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/action

{
  "couchbase": {
    "create_bucket": {
      "bucket": "test_bucket",
      "bucket_type": "couchbase",
      "bucket_port": 11222,
      "bucket_ramsize": 200,
      "bucket_replica": 1
    }
  }
}


Remove a Member


POST /instances/:id/topology/action

Request:

POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/action

{
  "couchbase": {
    "rebalance": {
      "server_remove": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"
    }
  }
}


Notes:

  • Note how removing a node is done in the context of a rebalance for Couchbase. This is an example of how homogenizing actions across trove, like "remove node", might not be appropriate.

Redis


Create Master


Request:

POST /instances
{
  "instance": {
    "name": "product-a",
    ...
    "datastore": {
      "type": "redis",
      "version": "redis-2.8.6"
    },
    "topology": {
      "redis": {}
    },
    ...
  }
}

Response:

{
  "instance": {
    "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
    ...
  }
}


Notes:

  • 'join' field is not required for redis. also notice the lack of a 'cluster_name' of sorts (see 'Add Slave' for reasoning)


Add Slave


Request:

POST /instances
{
  "instance": {
    "name": "product-b",
    ...
    "datastore": {
      "type": "redis",
      "version": "redis-2.8.6"
    },
    "topology": {
      "redis": {
        "slave_of": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"
      }
    },
    ...
  }
}


Response:

{
  "instance": {
    "status": "BUILD",
    "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
    ...
  }
}


Notes:

  • redis supports daisy-chaining slaves, therefore the 'slave_of' value needs to be a specific trove instance uuid vs. a manufactured 'cluster_name' of sorts.
  • whether it is a master or slave can be inferred from the presence (or lack thereof) of 'slave_of'.


Show Instance


Request:

GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998

Response:

{
  "instance": {
    ...
    "topology": {
      "redis": {}
    },
    ...
  }
}


Show Topology


Request:

GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology

Response:

{
  "topology": {
    "members": [
      {
        "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
        "name": "product-a",
        ...
        "redis": {}
      },
      {
        "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
        "name": "product-b",
        ...
        "redis": {
          "slave_of": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"
        }
      }
    ]
  }
}


Promote/Disconnect Slave


POST /instances/:id/topology/action

Request:

POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology/action

{
  "redis": {
    "slaveof_no_one": {
      "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"
    }
  }
}


Redis Cluster


Create Cluster


Request:

POST /instances
{
  "instance": {
    "name": "product-a",
    ...
    "datastore": {
      "type": "redis",
      "version": "redis-3.0.0-beta1"
    },
    "topology": {
      "redis_cluster": {
        "cluster_name": "products",
        "cluster_timeout": 5000,
        "join": false
      }
    },
    ...
  }
}

Response:

{
  "instance": {
    "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
    ...
  }
}


Notes:

  • Will require datastore-version to topology.<namespace> validation (e.g. if redis 3+, permit topology.redis_cluster{} and topology.redis{}, but if redis 2.x only permit topology.redis{})
  • 'cluster_name' and 'cluster_timeout' are unique to topology.redis_cluster{} (not supported in topology.redis{})
  • As seen in other datastore examples, uses 'join' and 'cluster_name'


Add Another Master to Cluster


Request:

POST /instances
{
  "instance": {
    "name": "product-b",
    ...
    "datastore": {
      "type": "redis",
      "version": "redis-3.0.0-beta1"
    },
    "topology": {
      "redis_cluster": {
        "cluster_name": "products",
        "join": true
      }
    },
    ...
  }
}


Response:

{
  "instance": {
    "status": "BUILD",
    "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
    ...
  }
}


Add Slave to Cluster


Request:

POST /instances
{
  "instance": {
    "name": "product-c",
    ...
    "datastore": {
      "type": "redis",
      "version": "redis-3.0.0-beta1"
    },
    "topology": {
      "redis_cluster": {
        "cluster_name": "products",
        "slave_of": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"
      }
    },
    ...
  }
}


Response:

{
  "instance": {
    "status": "BUILD",
    "id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",
    ...
  }
}


Notes:

  • Despite redis clusters supporting the ability to add a slave without designating the master, we will require it to avoid unoptimal geographical relationships.


Show Instance


Request:

GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998

Response:

{
  "instance": {
    ...
    "topology": {
      "redis_cluster": {
        "cluster_name": "products",
        "cluster_timeout": 5000
      }
    },
    ...
  }
}


Show Topology


Request:

GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology

Response:

{
  "topology": {
    "members": [
      {
        "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
        "name": "product-a",
        ...
        "redis_cluster": {
          "cluster_name": "products",
          "cluster_timeout": 5000
        }
      },
      {
        "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
        "name": "product-b",
        ...
        "redis_cluster": {
          "cluster_name": "products",
          "cluster_timeout": 5000
        }
      },
      {
        "id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",
        "name": "product-c",
        ...
        "redis_cluster": {
          "cluster_name": "products",
          "cluster_timeout": 5000,
          "slave_of": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"
        }
      }
    ]
  }
}


Promote/Disconnect Slave


Work in Progress

Feedback

Masters/Slaves

  • drewford: Can one make a master/slave out of an existing instance?
  • drewford: Type and version should follow the same schema as listing datastore types

Arbiters

  • drewford: Could/should arbiters be managed by the system automatically? Is it worth putting that responsibility on the API user, or can it be automated by the system for an easier, less technical experience? From reading the article noted in the link, it seems like it's a good enough idea to automate it to reduce the need for users to research and decide for themselves.

Delayed Members

  • drewford: Could/should delayed members be managed by the system automatically? Can it just be fully automated for a better, less technical experience?

Topology

  • drewford: Members of the topology in the mongo example all have the same "mongodb" object. Unless this could also be "mysql" or "redis" in the same topology, it might make sense to move up "type" and "replSet" attributes as siblings to "name".
  • drewford: The "type" attribute also seems redundant to the name of the array. Are there attributes other than "members" on a "topology"? If not, can "topology" be the array with each object having a "type"?

Cassandra Create Cluster

  • drewford: can "num_tokens", "endpoint_snitch" or "auto_bootstrap" be automated in any way so they receive a valid default value even if the user does not specify one? Looking for ways in which we can take the decision workload off the user, and put it on the system in a healthy way.

Promote/Disconnect Redis Slave

  • drewford: Seems like the promotion/disconnection json should be structured the same across all datastores if possible - with the backend handling any special manipulation to get the job done. Standardize in favor of the user.

Redis Cluster - Create Cluster

  • drewford: "redis_cluster" here is labelled differently than the same object in Cassandra and Couchbase - they don't use "_cluster".