Trove-Instance-Metadata

= Introduction = This wiki page describes the design for adding user managed metadata to instances.

Blueprint
https://blueprints.launchpad.net/trove/+spec/trove-metadata

= Goals =
 * 1) Give trove users the ability to store key/value pairs that have significance to them but is not essential to trove's functionality
 * 2) Create a user accessible rest API to manage instance metadata
 * 3) Mimic the design/interface used by Nova for metadata
 * 4) Implement the model so that it acts like a dictionary to make hooking in metadata into the instance very easy and small

= Description = Instance metadata is a feature that has been requested frequently by our users. They need a way to store critical information for their instances and have that be associated with the instance so that it is displayed whenever that instance is listed via the API. This also becomes very usable from a testing perspective when doing integration/ci. We can utilize the metadata to store things like what process created the instance, what the instance is being used for, etc... The design for this feature is modeled heavily on the Nova metadata API with a few tweaks in how it works internally.

With the likelihood of an Openstack official metadata service on the horizon I felt like we needed to keep the entry point of integration very small so that when that service becomes available there will be little code in the instance model to change to make that integration happen. To achieve this I made the model work using the dictionary interface and kept all the heavy lifting and fetching/saving operations are internal to the model and happen automatically when keys are modified.

All keys must be strings. Acceptable value types are:
 * Lists
 * Dictionaries
 * Strings
 * Integers
 * Floats

= Database Schema =

Table name
instance_metadata

Constraints
There is a unique constraint on the fields instance_uuid and key together to prevent duplicate keys per instance. There is code in the model and controller to prevent this also but the unique constraint is there as an added layer of security.

= API = The following section describes the API interface both what needs to be sent to the api and what the responses look like.

API Path and Method
POST /instances/{id}/metadata/{key}

Body
{   "metadata": { "value": {"one": {"two": [3, 4, 5]}} } }

Response Codes

 * Positive: 200
 * Negative: 400

Response Body
{   "metadata": { "newKey": {"one": {"two": [3, 4, 5]}} } }

Update
Substitute a key and value in the database with a new key and value.

API Path and Method
PUT /instances/{id}/metadata/{key}

Body
{   "metadata": { "key": "newKey2", "value": {"one": {"two": [3, 4, 5]}} } }

Response Codes

 * Positive: 200
 * Negative: 404

Response Body
None

API Path and Method
PATCH /instances/{id}/metadata/{key}

Body
{   "metadata": { "value": {"one": {"two": [3, 4, 5], "key2": "the value of this key is a string"}} } }

Response Codes

 * Positive: 200
 * Negative: 404

Response Body
None

API Path and Method
DELETE /instances/{id}/metadata/{key}

Body
None

Response Codes

 * Positive 200
 * Negative 404

Response Body
None

API Path and Method
GET /instances/{id}/metadata/{key}

Body
None

Response Codes

 * Positive 200
 * Negative 404

Response Body
{   "metadata": { "newKey": {"one": {"two": [3, 4, 5]}} } }

API Path and Method
GET /instances/{id}/metadata

Body
None

Response Codes

 * Positive: 200
 * No negative code because this returns an empty dictionary when no metadata exists for the instance

Response Body
{   "metadata": { "myKey": {"one": {"two": [3, 4, 5]}}, "keyTwo": "I wanted to store this sentence for this key" } }

= TroveClient =

Create
Create a new metadata entry for a given instance and key.

Arguments

 * instance_uuid: All metadata must be associated with an instance at this point so the instance uuid is necessary to form the API path
 * key: The key to store in the database.
 * value: The value to store in the database and associate with key above.

CLI
There is code in the cli library that attempts to deserialize any strings passed into the library to allow users to pass in complex data structures in JSON format via the CLI. A failed json.loads is ignored and the library then assumes that it is just a simple string and inserts that string into the database.

Example
trove metadata-create d241b3ce-e382-4dcc-ace7-769380438741 myKey '{"one": {"two": [3, 4, 5]}}'

Example
self.client.metadata.create('d241b3ce-e382-4dcc-ace7-769380438741', 'myKey', '{"one": {"two": [3, 4, 5]}}')

Update
Substitute a newkey for an orginal key and a new value for the old key's value.

Arguments

 * instance_id: All metadata must be associated with an instance at this point so the instance uuid is necessary to form the API path
 * key: The key to be updated.
 * newkey: The key to replace key above.
 * value: The value to be assigned to newkey above.

Example
trove metadata-update d241b3ce-e382-4dcc-ace7-769380438741 myKey instanceProvisionedBy 'Jenkins CI 4/1/14'

Example
self.client.metadata.update('d241b3ce-e382-4dcc-ace7-769380438741', 'myKey', 'instanceProvisionedBy', 'Jenkins CI')

Replace
Change the value of a given key.

Arguments

 * instance_id: All metadata must be associated with an instance at this point so the instance uuid is necessary to form the API path
 * key: The key to operate on
 * value: The new value to assign to key

Example
trove metadata-replace d241b3ce-e382-4dcc-ace7-769380438741 instanceProvisionedBy 'Jenkins CI 4/1/14'

Example
self.client.metadata.replace('d241b3ce-e382-4dcc-ace7-769380438741', 'instanceProvisionedBy', 'Jenkins CI')

Delete
Delete a key and it's associated value from the database

Arguments

 * instance_id: All metadata must be associated with an instance at this point so the instance uuid is necessary to form the API path
 * key: The key to be deleted

Example
trove metadata-delete d241b3ce-e382-4dcc-ace7-769380438741 instanceProvisionedBy

Example
self.client.metadata.delete('d241b3ce-e382-4dcc-ace7-769380438741', 'instanceProvisionedBy')

Show
Show the value of a key

Arguments

 * instance_id: All metadata must be associated with an instance at this point so the instance uuid is necessary to form the API path
 * key: The key to be deleted

Example
trove metadata-show d241b3ce-e382-4dcc-ace7-769380438741 newKey

Returned Body
{   "metadata": { "newKey": {"one": {"two": [3, 4, 5]}} } }

Example
self.client.metadata.show('d241b3ce-e382-4dcc-ace7-769380438741', 'newKey')

Returned Body
{   "metadata": { "newKey": {"one": {"two": [3, 4, 5]}} } }

Arguments

 * instance_id: All metadata must be associated with an instance at this point so the instance uuid is necessary to form the API path

Example
trove metadata-list d241b3ce-e382-4dcc-ace7-769380438741

Returned Body
{   "metadata": { "myKey": {"one": {"two": [3, 4, 5]}}, "keyTwo": "I wanted to store this sentence for this key" } }

Example
self.client.metadata.list('d241b3ce-e382-4dcc-ace7-769380438741')

Returned Body
{   "metadata": { "myKey": {"one": {"two": [3, 4, 5]}}, "keyTwo": "I wanted to store this sentence for this key" } }

= Integration with Instance = First pass I will not allow metadata to be passed in on instance create however this should be added soon after. You will however be able to see metadata in the instance object when you do a SHOW on the instance. Also when deleting an instance all metadata is deleted to keep things neat and tidy.

= Discussion =