Difference between revisions of "Trove-Replication-And-Clustering-API-Single"

Revision as of 03:07, 28 February 2014

MySQL

MySQL Master/Slave

For Master/Slave, the server_id must differ, and optionally the slave can specify whether it is read_only or not to avoid accidental writes.

Create Configuration-Group for Master

Request:

POST /configurations
{
  "configuration": {
    "name": "config-a",
    "datastore": {
      "type": "mysql",
      "version": "mysql-5.5"
    },
    "values": {
      "server_id": 1
    }
  }
}

Response:

{
  "configuration": {
    "id": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6",
    "name": "config-a",
    "description": null,
    "datastore_version_id": "c5ad9638-b7a1-464b-8cef-721d8c29dbf9",
    "values": {
      "server_id": 1
    }
  }
}

Create Configuration-Group for Slave

Request:

POST /configurations
{
  "configuration": {
    "name": "config-b",
    "datastore": {
      "type": "mysql",
      "version": "mysql-5.5"
    },
    "values": {
      "server_id": 2,
      "read_only": true
    }
  }
}

Response:

{
  "configuration": {
    "id": "fc318e00-3a6f-4f93-af99-146b44912188",
    "name": "config-b",
    "description": null,
    "datastore_version_id": "c5ad9638-b7a1-464b-8cef-721d8c29dbf9",
    "values": {
      "server_id": 2,
      "read_only": true
    }
  }
}

Notes:

• read_only here is used for illustrative purposes; it can be omitted or set to false on the slave/read-replica as desired.

Create Master

Request:

POST /instances
{
  "instance": {
    "availability_zone": "us-west-1",
    "name": "product-a",
    "datastore": {
      "type": "mysql",
      "version": "mysql-5.5"
    },
    "topology": {
      "mysql": {
        "type": "master",
      }
    },
    "configuration": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6",
    "flavorRef": "7",
    "volume": {
      "size": 1
    }
  }
}

Response:

{
  "instance": {
    "status": "BUILD",
    "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
    "name": "product-a",
    "configuration": {
      "id": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6",
      "name": "config-a",
      "links": [{...}]
    },
    ...
  }
}

Create Slave

Request:

POST /instances
{
  "instance": {
    "availability_zone": "us-west-2",
    "name": "product-b",
    "datastore": {
      "type": "mysql",
      "version": "mysql-5.5"
    },
    "topology": {
      "mysql": {
        "type": "slave",
        "replicates_from": [{"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"}]
      }
    },
    "configuration": "fc318e00-3a6f-4f93-af99-146b44912188",
    "flavorRef": "7",,
    "volume": {
      "size": 1
    }
  }
}

Response:

{
  "instance": {
    "status": "BUILD",
    "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
    "name": "product-b",
    "configuration": {
      "id": "fc318e00-3a6f-4f93-af99-146b44912188",
      "name": "config-b",
      "links": [{...}]
    },
    ...
  }
}

Notes:

• type and replicates_from will be the only supported input fields, despite replicates_to being returned in a GET /instances/<id> response
• replicates_from and replicates_to should be arrays to properly represent star + fan-in/multi-source + all-masters replication patterns (Tungsten, MySQL 5.7)
• For now, replicates_from takes a trove instance uuid, but inevitably will need to be prefixed with resource-names to support multiple dcs and sources
• Format of trove:dc:tenant:source:id/name, where source is instance, or backup or etc.
• *or* additional fields like datacenter, tenant, etc. will need to be added to each dict in replicates_from

Show Instance

Request:

GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998

Response:

{
  "instance": {
    "status": "ACTIVE",
    "updated": "2014-02-16T03:38:49"
    "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
    "name": "product-a",
    "datastore": {
      "version": "mysql-5.5",
      "type": "mysql",
    },
    "topology": {
      "mysql": {
        "type": "master",
        "replicates_to": [{"id": "061aaf4c-3a57-411e-9df9-2d0f813db859"}]
      }
    },
    "flavor": {
      "id": "7",
      "links": [{...}]
    },
    "configuration": {
      "id": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6",
      "name": "config-a",
      "links": [{...}]
    }
  }
}

Show Topology

Request:

GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology

Response:

{
  "instance": {
    "status": "ACTIVE",
    "updated": "2014-02-16T03:38:49"
    "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
    "name": "product-a",
    "datastore": {
      "version": "mysql-5.5",
      "type": "mysql",
    },
    "topology": {
      "members": [
        {
          "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
          "name": "product-a",
          "mysql": {
            "type": "master",
            "replicates_to": [{"id": "061aaf4c-3a57-411e-9df9-2d0f813db859"}]
          }
        },
        {
          "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
          "name": "product-b",
          "mysql": {
            "type": "slave",
            "replicates_from": [{"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"}]
          }
        }
      ]
    }
    "flavor": {
      "id": "7",
      "links": [{...}]
    },
    "configuration": {
      "id": "b9c8a3f8-7ace-4aea-9908-7b555586d7b6",
      "name": "config-a",
      "links": [{...}]
    }
  }
}

Remove Replication (aka "Promote" to Standalone)

Request:

POST /instances/061aaf4c-3a57-411e-9df9-2d0f813db859/topology
{
  "promote": {}
}

Response:

TBD

Notes:

• The PUT /instances/<id> approach requires bloat to prevent multiple unrelated fields from changing in-tandem.
• configuration-groups uses PUT, yet things like resize use /action; no ubiquitous approach.

MySQL Master/Master

For Master/Master, the server_id must differ and the increments be offset so as to avoid collisions.

Create Configuration-Group for Master A

Request:

POST /configurations
{
  "configuration": {
    ...
    "values": {
      "server_id": 1,
      "auto_increment_increment": 2,
      "auto_increment_offset": 1
    },
    ...
  }
}

Response:

{
  "configuration": {
    ...
    "values": {
      "server_id": 1,
      "auto_increment_increment": 2,
      "auto_increment_offset": 1
    },
    ...
  }
}

Create Configuration-Group for Master B

Request:

POST /configurations
{
  "configuration": {
    ...
    "values": {
      "server_id": 2,
      "auto_increment_increment": 2,
      "auto_increment_offset": 2
    },
    ...
  }
}

Response:

{
  "configuration": {
    ...
    "values": {
      "server_id": 2,
      "auto_increment_increment": 2,
      "auto_increment_offset": 2
    },
    ...
  }
}

Create Master A

Request:

• Same as seen in Master/Slave scenario

Response:

• Same as seen in Master/Slave scenario

Create Master B

Request:

POST /instances
{
  "instance": {
    ...
    "topology": {
      "mysql": {
        "type": "master",
        "join": [{"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"}]
      }
    },
    ...
  }
}

Response:

{
  "instance": {
    "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
    ...
  }
}

Show Instance

Request:

• Same as in Master/Slave scenario

Response:

{
  "instance": {
    ...
    "topology": {
      "mysql": {
        "type": "master",
        "join": [{"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"}]
      }
    },
    ...
  }
}

Show Topology

Request:

• Same as in Master/Slave scenario

Response:

{
  "topology": {
    "id": "377d54bb-9e89-4ac3-bf29-f78c2fd4faca",
    ...
    "members": [
      {
        "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
        ...
        "topology": {
          "mysql": {
            "type": "master",
            "join": [{"id": "061aaf4c-3a57-411e-9df9-2d0f813db859"}]
          }
        }
      },
      {
        "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
        ...
        "topology": {
          "mysql": {
            "type": "master",
            "join": [{"id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998"}]
          }
        }
      }
    ]
  }
}

Remove Replication (aka "Promote")

Request:

POST /instances/061aaf4c-3a57-411e-9df9-2d0f813db859/action
{
  "update_topology": {
    "topology": {
      "mysql": {
        "type": "master",
        "join": []
      }
    }
  }
}

*or*

PUT /instances/061aaf4c-3a57-411e-9df9-2d0f813db859
{
  "instance": {
    "topology": {
      "mysql": {
        "type": "master",
        "join": []
      }
    }
  }
}

*or*

POST /instances/061aaf4c-3a57-411e-9df9-2d0f813db859/topology
{
  "promote": {}
}

Response:

TBD

MongoDB

3 Member Replica-Set

Create Initial Replica-Set

Request:

POST /instances
{
  "instance": {
    "name": "product-a",
    ...
    "datastore": {
      "type": "mongodb",
      "version": "mongodb-2.0.4"
    },
    "topology": {
      "mongodb": {
        "type": "member",
        "replica_set": "products"
      }
    },
    ...
  }
}

Response:

{
  "instance": {
    "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
    ...
  }
}

Notes:

• Enforce replica_set field to be provided for MongoDB, even in the case of a standalone/single instance. See http://www.mongodb.com/blog/post/dont-let-your-standalone-mongodb-server-stand-alone for reasoning.
• The lack of a join field indicates that the intention is to create a new replica-set. If an existing replica-set by that name is found, the request will be failed.

Add 2nd Member to Replica-Set

Request:

POST /instances
{
  "instance": {
    "name": "product-b",
    ...
    "topology": {
      "mongodb": {
        "type": "member",
        "join": "products"
      }
    },
    ...
  }
}

Response:

{
  "instance": {
    "status": "BUILD",
    "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
    ...
  }
}

Notes:

• If the replica_set field is included in addition to the join field, the request will be failed.
• If there is no existing replica-set for the tenant by the join value, the request will be failed.
• Will have to use db.isMaster() to determine the current master to execute replica-set commands against.
• Will use http://docs.mongodb.org/manual/tutorial/expand-replica-set/#configure-and-add-a-member
• Should protect against adding more than 12 members to a replica-set
• Should protect against adding more than 7 voting members to a replica-set
• Should return warning when number of voting members is even and there is no arbiter

Add 3rd Member to Replica-Set

Request:

POST /instances
{
  "instance": {
    "name": "product-c",
    ...
    "topology": {
      "mongodb": {
        "type": "member",
        "join": "products"
      }
    },
    ...
  }
}

Response:

{
  "instance": {
    "status": "BUILD",
    "id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",
    ...
  }
}

Show Instance

Request:

GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998

Response:

{
  "instance": {
    ...
    "topology": {
      "mongodb": {
        "type": "member",
        "replica_set": "products"
      }
    },
    ...
  }
}

Show Topology

Request:

GET /instances/dfbbd9ca-b5e1-4028-adb7-f78643e17998/topology

Response:

{
  "instance": {
    ...
    "topology": {
      "members": [
        {
          "id": "dfbbd9ca-b5e1-4028-adb7-f78643e17998",
          "name": "product-a",
          ...
          "mongodb": {
	    "type": "member",
	    "replica_set": "products"
	  }
        },
        {
          "id": "061aaf4c-3a57-411e-9df9-2d0f813db859",
          "name": "product-b",
          ...
          "mongodb": {
	    "type": "member",
	    "replica_set": "products"
	  },
        {
          "id": "3a72ee87-cf3e-40f1-a1e1-fe8c7263a782",
          "name": "product-c",
          ...
          "mongodb": {
	    "type": "member",
	    "replica_set": "products"
	  }
        }
      ]
    }
    ...
  }
}

MongoDB TokuMX

• TokuMUX will require a new datastore-version and *possibly* a new manager class (same reasoning as why Tungsten/Galera will have their own datastore-version for MySQL)