Jump to: navigation, search

Difference between revisions of "Trove-Replication-And-Clustering-API"

Line 20: Line 20:
 
     "instance": {
 
     "instance": {
 
         "created": "2013-05-08T22:43:34",
 
         "created": "2013-05-08T22:43:34",
−
         "hostname": "e09ad9a3f73309469cf1f43d11e79549caf9acf2.rackspaceclouddb.com",
+
         "hostname": "mysweetdb.example.com",
 
         "id": "dcc5c518-73c7-4471-83e1-15fae67a98eb",
 
         "id": "dcc5c518-73c7-4471-83e1-15fae67a98eb",
 
         "name": "json_rack_instance",
 
         "name": "json_rack_instance",
Line 555: Line 555:
 
"size": 50,
 
"size": 50,
 
},
 
},
−
"nodes": 5,
 
 
"metadata": {
 
"metadata": {
 +
"nodes": 5,
 
"valuable_cluster_info": {}
 
"valuable_cluster_info": {}
 
}
 
}
Line 594: Line 594:
  
 
====Initialize cluster:====
 
====Initialize cluster:====
 +
Clusters can get mucked up sometimes, you may want to keep a node in the cluster but wipe
 +
it's data and let the cluster technology repopulate it's data.
 
'''POST''' /instance/{id}/initialize
 
'''POST''' /instance/{id}/initialize
 
<pre>
 
<pre>
 
{
 
{
−
"pertinent data should go here..."
+
"nodes": [
 +
{"id": "32e3c3b8-8d71-4b76-b2be-27fec62bc302"},
 +
{"id": "3c6176fc-bb33-429f-a926-e8307c115752"},
 +
]
 
}
 
}
 
</pre>
 
</pre>
  
 
====Rebalance/restripe cluster:====
 
====Rebalance/restripe cluster:====
 +
Changing the hashing algorithm on some cluster technologies may require a rebalance or
 +
restriping of the data.  This is a maintenance method to support that.
 
'''POST''' /instance/{id}/rebalance
 
'''POST''' /instance/{id}/rebalance
 
<pre>
 
<pre>
 
{
 
{
−
"pertinent data goes here"
+
"nodes": [
 +
{"id": "32e3c3b8-8d71-4b76-b2be-27fec62bc302"},
 +
{"id": "3c6176fc-bb33-429f-a926-e8307c115752"},
 +
]
 
}
 
}
 
</pre>
 
</pre>
Line 619: Line 629:
  
 
====Initialize a node:====
 
====Initialize a node:====
 +
Clusters can get mucked up sometimes, you may want to keep a node in the cluster but wipe
 +
it's data and let the cluster technology repopulate it's data.
 
'''POST''' /instance/{id}/node/{id}/initialize
 
'''POST''' /instance/{id}/node/{id}/initialize
 +
<pre>
 +
{
 +
"empty body?"
 +
}
 +
</pre>
 +
 +
====Rebalance/restripe cluster:====
 +
Changing the hashing algorithm on some cluster technologies may require a rebalance or
 +
restriping of the data.  This is a maintenance method to support that.
 +
'''POST''' /instance/{id}/node/{id}/rebalance
 
<pre>
 
<pre>
 
{
 
{
Line 635: Line 657:
  
 
What you cannot do:
 
What you cannot do:
−
- create a master and slave in one call
+
* create a master and slave in one call
−
- one call per action means simplified workflow
+
* one call per action means simplified workflow
  
  
 
Replication workflow:
 
Replication workflow:
−
1.  Create an instance
+
# Create an instance
−
2. Create an Nth instance with
+
# Create an Nth instance with
−
- metadata {"master_instance": "id from 1", "replication_type": "typeid from replication_types api"}
+
* Caveat: adding a replicated slave will alter data on the master instance and will use the agent to create the replication.
−
- Caveat: adding a replicated slave will alter data on the master instance and will use the agent
+
<pre>
−
to create the replication.
+
{
 +
"metadata": {
 +
"replicates_from": [
 +
"{master_instance_id}"
 +
],
 +
// The writeable flag should actually not be required but would populate based on
 +
// the underlying service type's replication setup.  It is an informative flag only
 +
// and shouldn't block users.
 +
"writeable": false
 +
}
 +
}
 +
</pre>
  
 
===Replication Operations===
 
===Replication Operations===
Line 654: Line 687:
 
{
 
{
 
"name": "foobar",
 
"name": "foobar",
−
"flavor": "flavor_id",
+
"flavor": "{flavor_id}",
 
"service_type": "mysql-5.1",
 
"service_type": "mysql-5.1",
 
"size": 50,
 
"size": 50,
 
"metadata": {
 
"metadata": {
−
"replication_type": "replication_type_id",
+
"replication_contract": {
−
"replication_data": {
+
"replicates_from": [
−
"master_instance": "instance_uuid"
+
"32e3c3b8-8d71-4b76-b2be-27fec62bc302",
 +
]
 
}
 
}
 
}
 
}
 
}
 
}
 
</pre>
 
</pre>

Revision as of 01:23, 10 October 2013

Instance Object Changes

Part of this proposal includes adding new properties to the json structure of an instance. The first would be to add a "nodes" property which would be list of refs back to the nova instance(s) that make up the trove instance.

The second would be a metadata storage that is of an open format to be used for storing critical data that is pertinent to the trove instance.

INSTANCES

Nodes

Because trove instances are a service type with an underlying nova resource(s) we can better display the nova resource(s) that make up the instance of a service type. When looking at a singular trove instance of something like redis or mysql the knowledge of the instance resource isn't all that important. Currently trove knows about the instance from a nova perspective but masks most of this information. This would look something like this:

GET /instances/{id} 

{
    "instance": {
        "created": "2013-05-08T22:43:34",
        "hostname": "mysweetdb.example.com",
        "id": "dcc5c518-73c7-4471-83e1-15fae67a98eb",
        "name": "json_rack_instance",
        "status": "ACTIVE",
        "updated": "2013-05-08T22:43:34",
        "service_type": "mysql-5.1",
        "nodes": [
        	{
        		"flavor": {
            		"id": "1", 
            		"links": [
                		{
                    		"href": "https://service/v1.0/1234/flavors/1", 
                    		"rel": "self"
                		},
                		{
                    		"href": "https://service/flavors/1", 
                    		"rel": "bookmark"
                		}
            		]
        		},
        		"links": [
            		{
                		"href": "https://service/v1.0/1234/instances/dcc5c518-73c7-4471-83e1-15fae67a98eb", 
                		"rel": "self"
            		}, 
            		{
                		"href": "https://service/instances/dcc5c518-73c7-4471-83e1-15fae67a98eb", 
                		"rel": "bookmark"
            		}
        		],
        		"volume": {
            		"size": 2,
            		"used": 0.16368598397821188
        		},
        	},
        ],
    }
}

Metadata

The metadata looks much like our previous proposal. It contains critical information to the configuration or topology of the cluster or replication agreement. This should look something like this:

This would be an example of a 3 node mult-master mysql replication contract GET /instances/{id} 

{
    "instance": {
        "created": "2013-05-08T22:43:34",
        "id": "cfeccbf4-ac5b-494a-99b7-61593a6a71b0",
        "name": "multi-master-hotness",
        "status": "ACTIVE",
        "updated": "2013-05-08T22:43:34",
        "service_type": "mysql-5.5",
        "nodes": [
        	{
        		"id": "dcc5c518-73c7-4471-83e1-15fae67a98eb",
        		"hostname": "master-n01.example.com",
        		"flavor": {
            		"id": "1", 
            		"links": [
                		{
                    		"href": "https://service/v1.0/1234/flavors/1", 
                    		"rel": "self"
                		},
                		{
                    		"href": "https://service/flavors/1", 
                    		"rel": "bookmark"
                		}
            		]
        		},
        		"links": [
            		{
                		"href": "https://service/v1.0/1234/instances/dcc5c518-73c7-4471-83e1-15fae67a98eb", 
                		"rel": "self"
            		}, 
            		{
                		"href": "https://service/instances/dcc5c518-73c7-4471-83e1-15fae67a98eb", 
                		"rel": "bookmark"
            		}
        		],
        		"volume": {
            		"size": 2,
            		"used": 0.16368598397821188
        		},
        	},
        	{
        		"id": "4530736b-c1ff-4c07-9a9f-fa9bca418157",
        		"hostname": "master-n02.example.com",
        		"flavor": {
            		"id": "1", 
            		"links": [
                		{
                    		"href": "https://service/v1.0/1234/flavors/1", 
                    		"rel": "self"
                		},
                		{
                    		"href": "https://service/flavors/1", 
                    		"rel": "bookmark"
                		}
            		]
        		},
        		"links": [
            		{
                		"href": "https://service/v1.0/1234/instances/4530736b-c1ff-4c07-9a9f-fa9bca418157", 
                		"rel": "self"
            		}, 
            		{
                		"href": "https://service/instances/4530736b-c1ff-4c07-9a9f-fa9bca418157", 
                		"rel": "bookmark"
            		}
        		],
        		"volume": {
            		"size": 2,
            		"used": 0.16368598397821188
        		},
        	},
        	{
        		"id": "736b9820-7e91-4775-84b0-78e71d60ce4c",
        		"hostname": "master-n03.example.com",
        		"flavor": {
            		"id": "1", 
            		"links": [
                		{
                    		"href": "https://service/v1.0/1234/flavors/1", 
                    		"rel": "self"
                		},
                		{
                    		"href": "https://service/flavors/1", 
                    		"rel": "bookmark"
                		}
            		]
        		},
        		"links": [
            		{
                		"href": "https://service/v1.0/1234/instances/736b9820-7e91-4775-84b0-78e71d60ce4c", 
                		"rel": "self"
            		}, 
            		{
                		"href": "https://service/instances/736b9820-7e91-4775-84b0-78e71d60ce4c", 
                		"rel": "bookmark"
            		}
        		],
        		"volume": {
            		"size": 2,
            		"used": 0.16368598397821188
        		},
        	},
        ],
        "metadata": {
        	"replication_contract": {
        		// We should use something more clear than just nova instance
        		// ids here but lets just use them for now for example. Maybe
        		// links? names? not sure what works best yet.
        		"dcc5c518-73c7-4471-83e1-15fae67a98eb": {
        			"replicates_to": [
        				"4530736b-c1ff-4c07-9a9f-fa9bca418157",
        			],
        			"replicates_from": [
        				"736b9820-7e91-4775-84b0-78e71d60ce4c",
        			],
        			"writeable": true,
        		},
        		"4530736b-c1ff-4c07-9a9f-fa9bca418157": {
        			"replicates_to": [
        				"736b9820-7e91-4775-84b0-78e71d60ce4c",
        			],
        			"replicates_from": [
        				"dcc5c518-73c7-4471-83e1-15fae67a98eb",
        			],
        			"writeable": true,
        		},
        		"736b9820-7e91-4775-84b0-78e71d60ce4c": {
        			"replicates_to": [
        				"dcc5c518-73c7-4471-83e1-15fae67a98eb",
        			],
        			"replicates_from": [
        				"4530736b-c1ff-4c07-9a9f-fa9bca418157",
        			],
        			"writeable": true
        		}
        	}
        }
    }
}

Other Examples

3 Node multi-master without nesting instances

Instance #1 (Master) 

{
    "instance": {
        "created": "2013-05-08T22:43:34",
        "id": "cfeccbf4-ac5b-494a-99b7-61593a6a71b0",
        "name": "multi-master-hotness-n01",
        "status": "ACTIVE",
        "updated": "2013-05-08T22:43:34",
        "service_type": "mysql-5.5",
        "nodes": [
        	{
        		"id": "dcc5c518-73c7-4471-83e1-15fae67a98eb",
        		"hostname": "master-n01.example.com",
        		"flavor": {
            		"id": "1", 
            		"links": [
                		{
                    		"href": "https://service/v1.0/1234/flavors/1", 
                    		"rel": "self"
                		},
                		{
                    		"href": "https://service/flavors/1", 
                    		"rel": "bookmark"
                		}
            		]
        		},
        		"links": [
            		{
                		"href": "https://service/v1.0/1234/instances/dcc5c518-73c7-4471-83e1-15fae67a98eb", 
                		"rel": "self"
            		}, 
            		{
                		"href": "https://service/instances/dcc5c518-73c7-4471-83e1-15fae67a98eb", 
                		"rel": "bookmark"
            		}
        		],
        		"volume": {
            		"size": 2,
            		"used": 0.16368598397821188
        		},
        	},
        ],
        "metadata": {
        	"replication_contract": {
        		// We should use something more clear than just nova instance
        		// ids here but lets just use them for now for example. Maybe
        		// links? names? not sure what works best yet.
        		"dcc5c518-73c7-4471-83e1-15fae67a98eb": {
        			"replicates_to": [
        				"4530736b-c1ff-4c07-9a9f-fa9bca418157",
        			],
        			"replicates_from": [
        				"736b9820-7e91-4775-84b0-78e71d60ce4c",
        			],
        			"writeable": true,
        		},
        	}
        }
    }
}

Instance #2 (Master) 

{
    "instance": {
        "created": "2013-05-08T22:43:34",
        "id": "cfeccbf4-ac5b-494a-99b7-61593a6a71b0",
        "name": "multi-master-hotness-n02",
        "status": "ACTIVE",
        "updated": "2013-05-08T22:43:34",
        "service_type": "mysql-5.5",
        "nodes": [
        	{
        		"id": "4530736b-c1ff-4c07-9a9f-fa9bca418157",
        		"hostname": "master-n02.example.com",
        		"flavor": {
            		"id": "1", 
            		"links": [
                		{
                    		"href": "https://service/v1.0/1234/flavors/1", 
                    		"rel": "self"
                		},
                		{
                    		"href": "https://service/flavors/1", 
                    		"rel": "bookmark"
                		}
            		]
        		},
        		"links": [
            		{
                		"href": "https://service/v1.0/1234/instances/4530736b-c1ff-4c07-9a9f-fa9bca418157", 
                		"rel": "self"
            		}, 
            		{
                		"href": "https://service/instances/4530736b-c1ff-4c07-9a9f-fa9bca418157", 
                		"rel": "bookmark"
            		}
        		],
        		"volume": {
            		"size": 2,
            		"used": 0.16368598397821188
        		},
        	},
        ],
        "metadata": {
        	"replication_contract": {
        		// We should use something more clear than just nova instance
        		// ids here but lets just use them for now for example. Maybe
        		// links? names? not sure what works best yet.
        		"4530736b-c1ff-4c07-9a9f-fa9bca418157": {
        			"replicates_to": [
        				"736b9820-7e91-4775-84b0-78e71d60ce4c",
        			],
        			"replicates_from": [
        				"dcc5c518-73c7-4471-83e1-15fae67a98eb",
        			],
        			"writeable": true,
        		},
        	}
        }
    }
}

Instance #3 (Master) 

{
    "instance": {
        "created": "2013-05-08T22:43:34",
        "id": "cfeccbf4-ac5b-494a-99b7-61593a6a71b0",
        "name": "multi-master-hotness-n03",
        "status": "ACTIVE",
        "updated": "2013-05-08T22:43:34",
        "service_type": "mysql-5.5",
        "nodes": [
        	{
        		"id": "736b9820-7e91-4775-84b0-78e71d60ce4c",
        		"hostname": "master-n03.example.com",
        		"flavor": {
            		"id": "1", 
            		"links": [
                		{
                    		"href": "https://service/v1.0/1234/flavors/1", 
                    		"rel": "self"
                		},
                		{
                    		"href": "https://service/flavors/1", 
                    		"rel": "bookmark"
                		}
            		]
        		},
        		"links": [
            		{
                		"href": "https://service/v1.0/1234/instances/736b9820-7e91-4775-84b0-78e71d60ce4c", 
                		"rel": "self"
            		}, 
            		{
                		"href": "https://service/instances/736b9820-7e91-4775-84b0-78e71d60ce4c", 
                		"rel": "bookmark"
            		}
        		],
        		"volume": {
            		"size": 2,
            		"used": 0.16368598397821188
        		},
        	},
        ],
        "metadata": {
        	"replication_contract": {
        		// We should use something more clear than just nova instance
        		// ids here but lets just use them for now for example. Maybe
        		// links? names? not sure what works best yet.
        		"736b9820-7e91-4775-84b0-78e71d60ce4c": {
        			"replicates_to": [
        				"dcc5c518-73c7-4471-83e1-15fae67a98eb",
        			],
        			"replicates_from": [
        				"4530736b-c1ff-4c07-9a9f-fa9bca418157",
        			],
        			"writeable": true
        		}
        	}
        }
    }
}

Replication without nesting slave instances with their masters

Instance #1 (Master) 

{
    "instance": {
        "created": "2013-05-08T22:43:34",
        "id": "cfeccbf4-ac5b-494a-99b7-61593a6a71b0",
        "name": "multi-master-hotness",
        "status": "ACTIVE",
        "updated": "2013-05-08T22:43:34",
        "service_type": "mysql-5.5",
        "nodes": [
        	{
        		"id": "dcc5c518-73c7-4471-83e1-15fae67a98eb",
        		"hostname": "master-n01.example.com",
        		"flavor": {
            		"id": "1", 
            		"links": [
                		{
                    		"href": "https://service/v1.0/1234/flavors/1", 
                    		"rel": "self"
                		},
                		{
                    		"href": "https://service/flavors/1", 
                    		"rel": "bookmark"
                		}
            		]
        		},
        		"links": [
            		{
                		"href": "https://service/v1.0/1234/instances/dcc5c518-73c7-4471-83e1-15fae67a98eb", 
                		"rel": "self"
            		}, 
            		{
                		"href": "https://service/instances/dcc5c518-73c7-4471-83e1-15fae67a98eb", 
                		"rel": "bookmark"
            		}
        		],
        		"volume": {
            		"size": 2,
            		"used": 0.16368598397821188
        		},
        	},
        ],
        "metadata": {
        	"replication_contract": {
        		// We should use something more clear than just nova instance
        		// ids here but lets just use them for now for example. Maybe
        		// links? names? not sure what works best yet.
        		"dcc5c518-73c7-4471-83e1-15fae67a98eb": {
        			"replicates_to": [
        				"4530736b-c1ff-4c07-9a9f-fa9bca418157",
        			],
        			"writeable": true,
        		},
        	}
        }
    }
}

Instance #2 (Read-only Slave) 

{
    "instance": {
        "created": "2013-05-08T22:43:34",
        "id": "cfeccbf4-ac5b-494a-99b7-61593a6a71b0",
        "name": "multi-master-hotness",
        "status": "ACTIVE",
        "updated": "2013-05-08T22:43:34",
        "service_type": "mysql-5.5",
        "nodes": [
        	{
        		"id": "4530736b-c1ff-4c07-9a9f-fa9bca418157",
        		"hostname": "master-n02.example.com",
        		"flavor": {
            		"id": "1", 
            		"links": [
                		{
                    		"href": "https://service/v1.0/1234/flavors/1", 
                    		"rel": "self"
                		},
                		{
                    		"href": "https://service/flavors/1", 
                    		"rel": "bookmark"
                		}
            		]
        		},
        		"links": [
            		{
                		"href": "https://service/v1.0/1234/instances/4530736b-c1ff-4c07-9a9f-fa9bca418157", 
                		"rel": "self"
            		}, 
            		{
                		"href": "https://service/instances/4530736b-c1ff-4c07-9a9f-fa9bca418157", 
                		"rel": "bookmark"
            		}
        		],
        		"volume": {
            		"size": 2,
            		"used": 0.16368598397821188
        		},
        	},
        ],
        "metadata": {
        	"replication_contract": {
        		// We should use something more clear than just nova instance
        		// ids here but lets just use them for now for example. Maybe
        		// links? names? not sure what works best yet.
        		"4530736b-c1ff-4c07-9a9f-fa9bca418157": {
        			"replicates_from": [
        				"dcc5c518-73c7-4471-83e1-15fae67a98eb",
        			],
        			"writeable": false,
        		},
        	}
        }
    }
}

Clustering and Replication

A cluster is an independent service type.

  • This simplifies the configurations (ties to service_type)
  • The guest impl can be different for each service_type
  • We don't need cluster type and service_type when defining a cluster


Think: A instance is an instance is a cluster is an instance is NOT replication

CLUSTERING

Definition of cluster for purposes of Trove:
A cluster must provide

  • High Availablility
  • Fault Tolerance


A cluster object is represented as an instance with a metadata attribute containing node information. A node is a sub-resource of a cluster and has a limited number of actions that can be performed on them. A cluster is made of of equal sized nodes using all the same flavor/disk size.

Cluster Operations

Create cluster:

Create a cluster with flavor X and size Y. All nodes are of equal size.
POST /instance 

{
	"name": "foobar",
	"flavor": "{flavor_id}",
	"service_type": "{service_type_id}",
	"volume": {
		"size": 50,
	},
	"metadata": {
		"nodes": 5,
		"valuable_cluster_info": {}
	}
}

Delete cluster:

  • Deletes all nodes in the cluster and the cluster itself.

DELETE /instance/{id}

Downsize a cluster (Delete nodes): PATCH /instance/{id} 

{
	"nodes": 3,
}

Restart a cluster:

POST /instance/{id}/restart 

{
	"empty body?"
}

Resize a cluster:

POST /instance/{id}/resize 

{
	"flavor": "flavor_id",
	"volume": {
		"size": 100
	}
}

Initialize cluster:

Clusters can get mucked up sometimes, you may want to keep a node in the cluster but wipe it's data and let the cluster technology repopulate it's data. POST /instance/{id}/initialize 

{
	"nodes": [
		{"id": "32e3c3b8-8d71-4b76-b2be-27fec62bc302"},
		{"id": "3c6176fc-bb33-429f-a926-e8307c115752"},
	]
}

Rebalance/restripe cluster:

Changing the hashing algorithm on some cluster technologies may require a rebalance or restriping of the data. This is a maintenance method to support that. POST /instance/{id}/rebalance 

{
	"nodes": [
		{"id": "32e3c3b8-8d71-4b76-b2be-27fec62bc302"},
		{"id": "3c6176fc-bb33-429f-a926-e8307c115752"},
	]
}

Node Operations:

Restart a node:

POST /instance/{id}/node/{id}/restart 

{
	"empty body?"
}

Initialize a node:

Clusters can get mucked up sometimes, you may want to keep a node in the cluster but wipe it's data and let the cluster technology repopulate it's data. POST /instance/{id}/node/{id}/initialize 

{
	"empty body?"
}

Rebalance/restripe cluster:

Changing the hashing algorithm on some cluster technologies may require a rebalance or restriping of the data. This is a maintenance method to support that. POST /instance/{id}/node/{id}/rebalance 

{
	"empty body?"
}


REPLICATION

We think of replication as a capability of some service types. Not every service type will have this capability and it should be configurable via the service types capabilities feature.

What you cannot do:

  • create a master and slave in one call

* one call per action means simplified workflow


Replication workflow:

  1. Create an instance
  2. Create an Nth instance with
  • Caveat: adding a replicated slave will alter data on the master instance and will use the agent to create the replication.
{
	"metadata": {
		"replicates_from": [
			"{master_instance_id}"
		],
		// The writeable flag should actually not be required but would populate based on
		// the underlying service type's replication setup.  It is an informative flag only
		// and shouldn't block users.
		"writeable": false 
	}
}

Replication Operations

Create Replication:

POST /instance 

{
	"name": "foobar",
	"flavor": "{flavor_id}",
	"service_type": "mysql-5.1",
	"size": 50,
	"metadata": {
		"replication_contract": {
			"replicates_from": [
				"32e3c3b8-8d71-4b76-b2be-27fec62bc302",
			]
		}
	}
}
Retrieved from "https://wiki.openstack.org/w/index.php?title=Trove-Replication-And-Clustering-API&oldid=32013"