Jump to: navigation, search

Difference between revisions of "Trove/Replication-And-Clustering-With-Nodes-2"

(Example: MySQL)
 
(8 intermediate revisions by the same user not shown)
Line 1: Line 1:
 +
=This Wiki Is Outdated=
 +
 +
11/2014: This proposal has been superseded and is kept around for historical purposes. To see the true Clustering specification, visit https://blueprints.launchpad.net/trove/+spec/clustering . To see the true Replication specification, see https://blueprints.launchpad.net/trove/+spec/replication-v1
 +
 
== Summary ==
 
== Summary ==
 
<br>
 
<br>
Line 6: Line 10:
 
<br>
 
<br>
 
==== Summary of New Fields for Instance-Show ====
 
==== Summary of New Fields for Instance-Show ====
* TBF
+
* TBF.
 
<br>
 
<br>
  
Line 336: Line 340:
 
Notes:
 
Notes:
 
* For Phase One:
 
* For Phase One:
** cluster.allocations[].region would not be supported.
+
** cluster.allocations would not be supported.
** cluster.size for mysql would have to be 1.
+
** cluster.size for mysql will be limited to the value of 1 for now (pending discussion on multi-master vs. multiple masters)
 +
* For Phase Two (& Phase One technically):
 +
** if cluster.allocations is omitted, the region is assumed to be the current.
 
<br>
 
<br>
  
Line 371: Line 377:
 
<br>
 
<br>
 
Notes:
 
Notes:
 +
* All slaves will default to having read-only set in their configuration-group (how configuration groups will be auto-spawned will be discussed later)
 
* For Phase One:
 
* For Phase One:
 
** add_slaves.allocations[].region would not be supported.
 
** add_slaves.allocations[].region would not be supported.
Line 422: Line 429:
 
<pre>
 
<pre>
 
{
 
{
   "instance": {
+
   "node": {
 
     "status": "ACTIVE",
 
     "status": "ACTIVE",
 
     "id": "416b0b16-ba55-4302-bbd3-ff566032e1c1",
 
     "id": "416b0b16-ba55-4302-bbd3-ff566032e1c1",
Line 460: Line 467:
 
<pre>
 
<pre>
 
{
 
{
   "instance": {
+
   "node": {
 
     "status": "ACTIVE",
 
     "status": "ACTIVE",
 
     "id": "7f52e4f9-3fa6-4238-ac08-1ce15197329a",
 
     "id": "7f52e4f9-3fa6-4238-ac08-1ce15197329a",
Line 663: Line 670:
  
 
<pre>
 
<pre>
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998
+
POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/action
  
 
{
 
{
   "resize_arbiters": {
+
   "add_arbiters": {
     "size": 2,
+
     "num": 2,
 
     "allocations": [
 
     "allocations": [
       {"region": "lvs", "size": 2}
+
       {"region": "lvs", "num": 2}
 
     ]
 
     ]
 
   }
 
   }
Line 681: Line 688:
 
</pre>
 
</pre>
 
<br>
 
<br>
 +
 
==== Show Cluster (After Arbiters) ====
 
==== Show Cluster (After Arbiters) ====
 
<br>
 
<br>

Latest revision as of 23:23, 17 November 2014

This Wiki Is Outdated

11/2014: This proposal has been superseded and is kept around for historical purposes. To see the true Clustering specification, visit https://blueprints.launchpad.net/trove/+spec/clustering . To see the true Replication specification, see https://blueprints.launchpad.net/trove/+spec/replication-v1

Summary


Summary of New Fields for Instance-Create

  • TBF



Summary of New Fields for Instance-Show

  • TBF.


Summary of Route Changes


New Route:

GET /instances/<id>/node/<node-id>
{
  "node": {
    "status": "<status>",
    "id": "<node-id>",
    "name": "<name>",
    "created": "<timestamp>",
    "updated": "<timestamp>",
    "links": [{...}],
    "ip": ["<ip>"], //or hostname, depending on conf
    "configuration": "<configuration-id>",
    "flavorRef": "<flavor-id>",
    "volume": {
      "size": <size>,
      "used": <used>
    }
  }
}

New Route:

POST /instances/<id>/node/<node-id>/action
{
  "<action>": {
    <action-specific fields>
  }
}


Summary

  • TBF


Features for First Iteration


Notes:

  • TBF


Example: Cassandra


To illustrate the approach, Cassandra is used in the examples below. The eccentricities of each Datastore will be explained in their own sections.

Create Cluster


Request:

POST /instances
{
  "instance": {
    "name": "products",
    "datastore": {
      "type": "cassandra",
      "version": "2.0.6"
    },
    "configuration": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6",
    "flavorRef": "7",
    "volume": {
      "size": 1
    },
    "cluster": {
      "size": 3,
      "allocations": [
        {"region": "phx", "size": 1},
        {"region": "slc", "size": 1},
        {"region": "lvs", "size": 1}
      ]
    }
  }
}

Response:

{
  "instance": {
    "status": "BUILD",
    "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
    "name": "products",
    "created": "2014-04-25T20:19:23",
    "updated": "2014-04-25T20:19:23",
    "links": [{...}],
    "datastore": {
      "type": "cassandra",
      "version": "2.0.6"
    },
    "cluster": {
      "size": 3,
      "nodes": [
        {"id": "416b0b16-ba55-4302-bbd3-ff566032e1c1", "region": "phx"},
        {"id": "7f52e4f9-3fa6-4238-ac08-1ce15197329a", "region": "slc"},
        {"id": "ff9d680c-fde3-49c6-a84e-76173b6df39d", "region": "lvs"}
      ]
    }
  }
}


Notes:

  • For Phase One:
    • cluster.allocations{} will not be supported.
    • cluster.nodes[].region will not be returned.
  • For Phase Two:
    • if cluster.allocations{} is not provided, the current region is assumed.
    • cluster.nodes[].region will always be returned.
  • Cassandra-specific fields that are required to construct the initial cluster (num_tokens, endpoint_snitch, seed ip-list, etc.) are to be determined/calculated based on configuration file values and common-sense.


Show Cluster


Request:

GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998

Response:

{
  "instance": {
    "status": "ACTIVE",
    "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
    "name": "products",
    "created": "2014-04-25T20:19:23",
    "updated": "2014-04-25T20:19:23",
    "links": [{...}],
    "datastore": {
      "type": "cassandra",
      "version": "2.0.6"
    },
    "cluster": {
      "size": 3,
      "nodes": [
        {"id": "416b0b16-ba55-4302-bbd3-ff566032e1c1", "region": "phx"},
        {"id": "7f52e4f9-3fa6-4238-ac08-1ce15197329a", "region": "slc"},
        {"id": "ff9d680c-fde3-49c6-a84e-76173b6df39d", "region": "lvs"}
      ]
    }
  }
}


Notes:

  • In Phase One:
    • cluster.nodes[].region will not be returned.
  • Change: instance.volume.used, instance.ip[], and instance.hostname will never be returned
    • It's possible that instance.ip[] can remain if it only returns the seed ips.
    • It's possible that instance.hostname can remain if it's converted to an array and only contains the seed hostnames.


Show Node


Request:

GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/node/416b0b16-ba55-4302-bbd3-ff566032e1c1

Response:

{
  "node": {
    "status": "ACTIVE",
    "id": "416b0b16-ba55-4302-bbd3-ff566032e1c1",
    "name": "products-1",
    "created": "2014-04-25T20:19:23",
    "updated": "2014-04-25T20:19:23",
    "links": [{...}],
    "ip": ["10.0.0.1"],
    "configuration": {
      "id": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6",
      "links": [{...}],
    },
    "flavor": {
      "id": "7",
      "links": [{...}],
    },
    "volume": {
      "size": 2,
      "used": 0.17
    }
  }
}


Add Node(s)


Request:

POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/action

{
  # phase one
  "add_nodes": {
    "num": 3
  }

  # phase two
  "add_nodes": {
    "num": 3,
    "allocations": [
      {"region": "phx", "num": 1},
      {"region": "slc", "num": 1},
      {"region": "lvs", "num": 1}
    ]
  }
}


Response:

HTTP 202 (Empty Body)


Notes:

  • For Phase One:
    • add_nodes.num will be the only supported field (add_nodes.allocations will not be)
  • For Phase Two:
    • if add_nodes.allocations[] is not provided, the region of every node must match, otherwise the request is failed.


Replace Node


Request:

POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/nodes/7f52e4f9-3fa6-4238-ac08-1ce15197329a/action

{
  "replace_node": {}
}


Response:

HTTP 202 (Empty Body)


Notes:


Remove Node


Request:

DELETE /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/nodes/7f52e4f9-3fa6-4238-ac08-1ce15197329a


Response:

HTTP 202 (Empty Body)


Notes:


Example: MySQL


Create Cluster (w/ One Master)


Request:

POST /instances
{
  "instance": {
    "name": "products",
    "datastore": {
      "type": "mysql",
      "version": "5.5"
    },
    "configuration": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6",
    "flavorRef": "7",
    "volume": {
      "size": 1
    },
    "cluster": {
      "size": 1,
      "allocations": [
        {"region": "phx", "size": 1},
      ]
    }
  }
}

Response:

{
  "instance": {
    "status": "BUILD",
    "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
    "name": "products",
    "created": "2014-04-25T20:19:23",
    "updated": "2014-04-25T20:19:23",
    "links": [{...}],
    "datastore": {
      "type": "mysql",
      "version": "5.5"
    },
    "cluster": {
      "size": 1,
      "nodes": [
        {"id": "416b0b16-ba55-4302-bbd3-ff566032e1c1", "region": "phx"}
      ]
    }
  }
}


Notes:

  • For Phase One:
    • cluster.allocations would not be supported.
    • cluster.size for mysql will be limited to the value of 1 for now (pending discussion on multi-master vs. multiple masters)
  • For Phase Two (& Phase One technically):
    • if cluster.allocations is omitted, the region is assumed to be the current.


Create Slave(s)


Request:

POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/nodes/416b0b16-ba55-4302-bbd3-ff566032e1c1/action

{
  # phase one
  "add_slaves": {
    "num": 2
  }

  # phase two
  "add_slaves": {
    "num": 2,
    "allocations": [
      {"region": "phx", "num": 1},
      {"region": "slc", "num": 1},
    ]
  }
}

Response:

HTTP 202 (Empty Body)


Notes:

  • All slaves will default to having read-only set in their configuration-group (how configuration groups will be auto-spawned will be discussed later)
  • For Phase One:
    • add_slaves.allocations[].region would not be supported.


Show Cluster


Request:

GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998

Response:

{
  "instance": {
    "status": "ACTIVE",
    "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
    "name": "products",
    "created": "2014-04-25T20:19:23",
    "updated": "2014-04-25T20:19:23",
    "links": [{...}],
    "datastore": {
      "type": "mysql",
      "version": "5.5"
    },
    "cluster": {
      "size": 3,
      "nodes": [
        {"id": "416b0b16-ba55-4302-bbd3-ff566032e1c1", "region": "phx"},
        {"id": "7f52e4f9-3fa6-4238-ac08-1ce15197329a", "region": "slc", "type": "slave"}
        {"id": "c592183d-0aec-42f1-8a3b-199e6817c372", "region": "phx", "type": "slave"}
      ]
    }
  }
}


Show Master


Request:

GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/nodes/416b0b16-ba55-4302-bbd3-ff566032e1c1

Response:

{
  "node": {
    "status": "ACTIVE",
    "id": "416b0b16-ba55-4302-bbd3-ff566032e1c1",
    "name": "products-1",
    "created": "2014-04-25T20:19:23",
    "updated": "2014-04-25T20:19:23",
    "links": [{...}],
    "datastore": {
      "type": "mysql",
      "version": "5.5"
    },
    "configuration": {
      "id": "fc318e00-3a6f-4f93-af99-146b44912188",
      "links": [{...}],
    },
    "flavor": {
      "id": "7",
      "links": [{...}],
    },
    "volume": {
      "size": 1
    }
  }
}


Show Slave


Request:

GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/nodes/7f52e4f9-3fa6-4238-ac08-1ce15197329a

Response:

{
  "node": {
    "status": "ACTIVE",
    "id": "7f52e4f9-3fa6-4238-ac08-1ce15197329a",
    "name": "products-2",
    "created": "2014-04-25T20:19:23",
    "updated": "2014-04-25T20:19:23",
    "links": [{...}],
    "datastore": {
      "type": "mysql",
      "version": "5.5"
    },
    "configuration": {
      "id": "fc318e00-3a6f-4f93-af99-146b44912188",
      "links": [{...}],
    },
    "flavor": {
      "id": "7",
      "links": [{...}],
    },
    "volume": {
      "size": 1
    }
  }
}


Promote (Detach) Slave


Request:

POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/nodes/7f52e4f9-3fa6-4238-ac08-1ce15197329a/action

{
  "promote": {}
}

Response:

HTTP 202 (Empty Body)


Delete Slave


Request:

DELETE /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/nodes/7f52e4f9-3fa6-4238-ac08-1ce15197329a


Response:

HTTP 202 (Empty Body)


Example: MongoDB


Create Cluster


Request:

POST /instances
{
  "instance": {
    "name": "products",
    "datastore": {
      "type": "mongodb",
      "version": "2.4.10"
    },
    "configuration": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6",
    "flavorRef": "7",
    "volume": {
      "size": 1
    },
    "cluster": {
      "size": 5,
      "allocations": [
        {"region": "phx", "size": 3},
        {"region": "slc", "size": 2},
      ]
    }
  }
}

Response:

{
  "instance": {
    "status": "BUILD",
    "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
    "name": "products",
    "created": "2014-04-25T20:19:23",
    "updated": "2014-04-25T20:19:23",
    "links": [{...}],
    "datastore": {
      "type": "mongodb",
      "version": "2.4.10"
    },
    "cluster": {
      "size": 5,
      "nodes": [
        {"id": "416b0b16-ba55-4302-bbd3-ff566032e1c1", "region": "phx"},
        {"id": "965ef811-7c1d-47fc-89f2-a89dfdd23ef2", "region": "phx"},
        {"id": "3642f41c-e8ad-4164-a089-3891bf7f2d2b", "region": "phx"},
        {"id": "7f52e4f9-3fa6-4238-ac08-1ce15197329a", "region": "slc"},
        {"id": "ff9d680c-fde3-49c6-a84e-76173b6df39d", "region": "slc"}
      ]
    }
  }
}


Notes:

  • Same as explained in Cassandra.


Show Cluster


Request:

GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998

Response:

{
  "instance": {
    "status": "ACTIVE",
    "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
    "name": "products",
    "created": "2014-04-25T20:19:23",
    "updated": "2014-04-25T20:19:23",
    "links": [{...}],
    "datastore": {
      "type": "mongodb",
      "version": "2.4.10"
    },
    "cluster": {
      "size": 5,
      "nodes": [
        {"id": "416b0b16-ba55-4302-bbd3-ff566032e1c1", "region": "phx"},
        {"id": "965ef811-7c1d-47fc-89f2-a89dfdd23ef2", "region": "phx"},
        {"id": "3642f41c-e8ad-4164-a089-3891bf7f2d2b", "region": "phx"},
        {"id": "7f52e4f9-3fa6-4238-ac08-1ce15197329a", "region": "slc"},
        {"id": "ff9d680c-fde3-49c6-a84e-76173b6df39d", "region": "slc"}
      ]
    }
  }
}


Notes:

  • None.


Show Node


Request:

GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/node/416b0b16-ba55-4302-bbd3-ff566032e1c1

Response:

{
  "node": {
    "status": "ACTIVE",
    "id": "416b0b16-ba55-4302-bbd3-ff566032e1c1",
    "name": "products-1",
    "created": "2014-04-25T20:19:23",
    "updated": "2014-04-25T20:19:23",
    "links": [{...}],
    "ip": ["10.0.0.1"],
    "configuration": {
      "id": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6",
      "links": [{...}],
    },
    "flavor": {
      "id": "7",
      "links": [{...}],
    },
    "volume": {
      "size": 2,
      "used": 0.17
    }
  }
}


Create Arbiter(s)


Request:

POST /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/action

{
  "add_arbiters": {
    "num": 2,
    "allocations": [
      {"region": "lvs", "num": 2}
    ]
  }
}

Response:

HTTP 202 (Empty Body)


Show Cluster (After Arbiters)


Request:

GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998

Response:

{
  "instance": {
    "status": "ACTIVE",
    "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
    "name": "products",
    "created": "2014-04-25T20:19:23",
    "updated": "2014-04-25T20:19:23",
    "links": [{...}],
    "datastore": {
      "type": "mongodb",
      "version": "2.4.10"
    },
    "cluster": {
      "size": 7,
      "nodes": [
        {"id": "416b0b16-ba55-4302-bbd3-ff566032e1c1", "region": "phx"},
        {"id": "965ef811-7c1d-47fc-89f2-a89dfdd23ef2", "region": "phx"},
        {"id": "3642f41c-e8ad-4164-a089-3891bf7f2d2b", "region": "phx"},
        {"id": "7f52e4f9-3fa6-4238-ac08-1ce15197329a", "region": "slc"},
        {"id": "ff9d680c-fde3-49c6-a84e-76173b6df39d", "region": "slc"}
        {"id": "77032c55-4496-4e35-8c0d-6cd1c18e1a9c", "region": "lvs", "type": "arbiter"}
        {"id": "1fd054ed-221f-4c99-8d17-570bcff4c1d2", "region": "lvs", "type": "arbiter"}
      ]
    }
  }
}


Notes:

  • None.