Difference between revisions of "Trove-Instance-Metadata"
Imsplitbit (talk | contribs) (→Create) |
Imsplitbit (talk | contribs) (→API Path and Method) |
||
(28 intermediate revisions by the same user not shown) | |||
Line 15: | Line 15: | ||
With the likelihood of an Openstack official metadata service on the horizon I felt like we needed to keep the entry point of integration very small so that when that service becomes available there will be little code in the instance model to change to make that integration happen. To achieve this I made the model work using the dictionary interface and kept all the heavy lifting and fetching/saving operations are internal to the model and happen automatically when keys are modified. | With the likelihood of an Openstack official metadata service on the horizon I felt like we needed to keep the entry point of integration very small so that when that service becomes available there will be little code in the instance model to change to make that integration happen. To achieve this I made the model work using the dictionary interface and kept all the heavy lifting and fetching/saving operations are internal to the model and happen automatically when keys are modified. | ||
+ | |||
+ | All keys '''must''' be strings. Acceptable value types are: | ||
+ | * Lists | ||
+ | * Dictionaries | ||
+ | * Strings | ||
+ | * Integers | ||
+ | * Floats | ||
+ | |||
+ | = Database Schema = | ||
+ | == Table name == | ||
+ | instance_metadata | ||
+ | == Field Names and Types == | ||
+ | {| class="wikitable" | ||
+ | |- | ||
+ | ! Name !! Type | ||
+ | |- | ||
+ | | id || String(36) | ||
+ | |- | ||
+ | | instance_uuid || String(36) | ||
+ | |- | ||
+ | | key || String(255) | ||
+ | |- | ||
+ | | value || Text | ||
+ | |- | ||
+ | | created || DateTime | ||
+ | |- | ||
+ | | updated || DateTime | ||
+ | |- | ||
+ | | deleted || Boolean | ||
+ | |- | ||
+ | | deleted_at || DateTime | ||
+ | |} | ||
+ | |||
+ | == Constraints == | ||
+ | There is a unique constraint on the fields instance_uuid and key together to prevent duplicate keys per instance. There is code in the model and controller to prevent this also but the unique constraint is there as an added layer of security. | ||
= API = | = API = | ||
The following section describes the API interface both what needs to be sent to the api and what the responses look like. | The following section describes the API interface both what needs to be sent to the api and what the responses look like. | ||
== Create == | == Create == | ||
− | '''POST''' /instances/{id}/metadata | + | === API Call === |
+ | ==== API Path and Method ==== | ||
+ | '''POST''' /instances/{id}/metadata/{key} | ||
− | + | ==== Body ==== | |
<pre> | <pre> | ||
{ | { | ||
Line 30: | Line 67: | ||
</pre> | </pre> | ||
− | Response | + | === API Response === |
− | Codes | + | ==== Response Codes ==== |
* Positive: '''200''' | * Positive: '''200''' | ||
* Negative: '''400''' | * Negative: '''400''' | ||
+ | ==== Response Body ==== | ||
<pre> | <pre> | ||
{ | { | ||
Line 45: | Line 83: | ||
== Update == | == Update == | ||
Substitute a key and value in the database with a new key and value. | Substitute a key and value in the database with a new key and value. | ||
− | + | === API Call === | |
+ | ==== API Path and Method ==== | ||
'''PUT''' /instances/{id}/metadata/{key} | '''PUT''' /instances/{id}/metadata/{key} | ||
− | Body | + | ==== Body ==== |
<pre> | <pre> | ||
{ | { | ||
Line 58: | Line 97: | ||
</pre> | </pre> | ||
− | Response | + | === API Response === |
+ | |||
+ | ==== Response Codes ==== | ||
+ | * Positive: '''200''' | ||
+ | * Negative: '''404''' | ||
+ | ==== Response Body ==== | ||
+ | '''None''' | ||
+ | |||
+ | == Replace == | ||
+ | === API Call === | ||
+ | ==== API Path and Method ==== | ||
+ | '''PATCH''' /instances/{id}/metadata/{key} | ||
+ | ==== Body ==== | ||
+ | <pre> | ||
+ | { | ||
+ | "metadata": { | ||
+ | "value": {"one": {"two": [3, 4, 5], "key2": "the value of this key is a string"}} | ||
+ | } | ||
+ | } | ||
+ | </pre> | ||
− | + | === API Response === | |
− | + | ==== Response Codes ==== | |
+ | * Positive: '''200''' | ||
+ | * Negative: '''404''' | ||
+ | ==== Response Body ==== | ||
+ | None | ||
− | |||
== Delete == | == Delete == | ||
+ | === API Call === | ||
+ | ==== API Path and Method ==== | ||
+ | '''DELETE''' /instances/{id}/metadata/{key} | ||
+ | ==== Body ==== | ||
+ | None | ||
+ | === API Response === | ||
+ | ==== Response Codes ==== | ||
+ | * Positive '''200''' | ||
+ | * Negative '''404''' | ||
+ | ==== Response Body ==== | ||
+ | None | ||
+ | |||
== Show == | == Show == | ||
+ | === API Call === | ||
+ | ==== API Path and Method ==== | ||
+ | '''GET''' /instances/{id}/metadata/{key} | ||
+ | ==== Body ==== | ||
+ | None | ||
+ | === API Response === | ||
+ | ==== Response Codes ==== | ||
+ | * Positive '''200''' | ||
+ | * Negative '''404''' | ||
+ | ==== Response Body ==== | ||
+ | <pre> | ||
+ | { | ||
+ | "metadata": { | ||
+ | "newKey": {"one": {"two": [3, 4, 5]}} | ||
+ | } | ||
+ | } | ||
+ | </pre> | ||
+ | |||
== List == | == List == | ||
+ | === API Call === | ||
+ | ==== API Path and Method ==== | ||
+ | '''GET''' /instances/{id}/metadata | ||
+ | ==== Body ==== | ||
+ | None | ||
+ | === API Response === | ||
+ | ==== Response Codes ==== | ||
+ | * Positive: '''200''' | ||
+ | * No negative code because this returns an empty dictionary when no metadata exists for the instance | ||
+ | ==== Response Body ==== | ||
+ | <pre> | ||
+ | { | ||
+ | "metadata": { | ||
+ | "myKey": {"one": {"two": [3, 4, 5]}}, | ||
+ | "keyTwo": "I wanted to store this sentence for this key" | ||
+ | } | ||
+ | } | ||
+ | </pre> | ||
+ | |||
= TroveClient = | = TroveClient = | ||
== Create == | == Create == | ||
+ | Create a new metadata entry for a given instance and key. | ||
+ | === Arguments === | ||
+ | * '''instance_uuid:''' All metadata '''must''' be associated with an instance at this point so the instance uuid is necessary to form the API path | ||
+ | * '''key:''' The key to store in the database. | ||
+ | * '''value:''' The value to store in the database and associate with key above. | ||
+ | === CLI === | ||
+ | There is code in the cli library that attempts to deserialize any strings passed into the library to allow users to pass in complex data structures in JSON format via the CLI. A failed json.loads() is ignored and the library then assumes that it is just a simple string and inserts that string into the database. | ||
+ | ==== Example ==== | ||
+ | <pre> | ||
+ | trove metadata-create d241b3ce-e382-4dcc-ace7-769380438741 myKey '{"one": {"two": [3, 4, 5]}}' | ||
+ | </pre> | ||
+ | === Python Library === | ||
+ | ==== Example ==== | ||
+ | <pre> | ||
+ | self.client.metadata.create('d241b3ce-e382-4dcc-ace7-769380438741', 'myKey', '{"one": {"two": [3, 4, 5]}}') | ||
+ | </pre> | ||
== Update == | == Update == | ||
+ | Substitute a newkey for an orginal key and a new value for the old key's value. | ||
+ | === Arguments === | ||
+ | * '''instance_id:''' All metadata '''must''' be associated with an instance at this point so the instance uuid is necessary to form the API path | ||
+ | * '''key:''' The key to be updated. | ||
+ | * '''newkey:''' The key to replace key above. | ||
+ | * '''value:''' The value to be assigned to newkey above. | ||
+ | === CLI === | ||
+ | ==== Example ==== | ||
+ | <pre> | ||
+ | trove metadata-update d241b3ce-e382-4dcc-ace7-769380438741 myKey instanceProvisionedBy 'Jenkins CI 4/1/14' | ||
+ | </pre> | ||
+ | === Python Library === | ||
+ | ==== Example ==== | ||
+ | <pre> | ||
+ | self.client.metadata.update('d241b3ce-e382-4dcc-ace7-769380438741', 'myKey', 'instanceProvisionedBy', 'Jenkins CI') | ||
+ | </pre> | ||
+ | |||
== Replace == | == Replace == | ||
+ | Change the value of a given key. | ||
+ | === Arguments === | ||
+ | * '''instance_id:''' All metadata '''must''' be associated with an instance at this point so the instance uuid is necessary to form the API path | ||
+ | * '''key:''' The key to operate on | ||
+ | * '''value:''' The new value to assign to key | ||
+ | === CLI === | ||
+ | ==== Example ==== | ||
+ | <pre> | ||
+ | trove metadata-replace d241b3ce-e382-4dcc-ace7-769380438741 instanceProvisionedBy 'Jenkins CI 4/1/14' | ||
+ | </pre> | ||
+ | === Python Library === | ||
+ | ==== Example ==== | ||
+ | <pre> | ||
+ | self.client.metadata.replace('d241b3ce-e382-4dcc-ace7-769380438741', 'instanceProvisionedBy', 'Jenkins CI') | ||
+ | </pre> | ||
== Delete == | == Delete == | ||
+ | Delete a key and it's associated value from the database | ||
+ | === Arguments === | ||
+ | * '''instance_id:''' All metadata '''must''' be associated with an instance at this point so the instance uuid is necessary to form the API path | ||
+ | * '''key:''' The key to be deleted | ||
+ | === CLI === | ||
+ | ==== Example ==== | ||
+ | <pre> | ||
+ | trove metadata-delete d241b3ce-e382-4dcc-ace7-769380438741 instanceProvisionedBy | ||
+ | </pre> | ||
+ | === Python Library === | ||
+ | ==== Example ==== | ||
+ | <pre> | ||
+ | self.client.metadata.delete('d241b3ce-e382-4dcc-ace7-769380438741', 'instanceProvisionedBy') | ||
+ | </pre> | ||
== Show == | == Show == | ||
+ | Show the value of a key | ||
+ | === Arguments === | ||
+ | * '''instance_id:''' All metadata '''must''' be associated with an instance at this point so the instance uuid is necessary to form the API path | ||
+ | * '''key:''' The key to be deleted | ||
+ | === CLI === | ||
+ | ==== Example ==== | ||
+ | <pre> | ||
+ | trove metadata-show d241b3ce-e382-4dcc-ace7-769380438741 newKey | ||
+ | </pre> | ||
+ | ===== Returned Body ===== | ||
+ | <pre> | ||
+ | { | ||
+ | "metadata": { | ||
+ | "newKey": {"one": {"two": [3, 4, 5]}} | ||
+ | } | ||
+ | } | ||
+ | </pre> | ||
+ | === Python Library === | ||
+ | ==== Example ==== | ||
+ | <pre> | ||
+ | self.client.metadata.show('d241b3ce-e382-4dcc-ace7-769380438741', 'newKey') | ||
+ | </pre> | ||
+ | ===== Returned Body ===== | ||
+ | <pre> | ||
+ | { | ||
+ | "metadata": { | ||
+ | "newKey": {"one": {"two": [3, 4, 5]}} | ||
+ | } | ||
+ | } | ||
+ | </pre> | ||
== List == | == List == | ||
+ | === Arguments === | ||
+ | * '''instance_id:''' All metadata '''must''' be associated with an instance at this point so the instance uuid is necessary to form the API path | ||
+ | === CLI === | ||
+ | ==== Example ==== | ||
+ | <pre> | ||
+ | trove metadata-list d241b3ce-e382-4dcc-ace7-769380438741 | ||
+ | </pre> | ||
+ | ===== Returned Body ===== | ||
+ | <pre> | ||
+ | { | ||
+ | "metadata": { | ||
+ | "myKey": {"one": {"two": [3, 4, 5]}}, | ||
+ | "keyTwo": "I wanted to store this sentence for this key" | ||
+ | } | ||
+ | } | ||
+ | </pre> | ||
+ | === Python Library === | ||
+ | ==== Example ==== | ||
+ | <pre> | ||
+ | self.client.metadata.list('d241b3ce-e382-4dcc-ace7-769380438741') | ||
+ | </pre> | ||
+ | ===== Returned Body ===== | ||
+ | <pre> | ||
+ | { | ||
+ | "metadata": { | ||
+ | "myKey": {"one": {"two": [3, 4, 5]}}, | ||
+ | "keyTwo": "I wanted to store this sentence for this key" | ||
+ | } | ||
+ | } | ||
+ | </pre> | ||
+ | |||
= Integration with Instance = | = Integration with Instance = | ||
+ | First pass I will not allow metadata to be passed in on instance create however this should be added soon after. You will however be able to see metadata in the instance object when you do a SHOW on the instance. Also when deleting an instance all metadata is deleted to keep things neat and tidy. | ||
+ | |||
= Discussion = | = Discussion = |
Latest revision as of 20:07, 31 March 2014
Contents
Introduction
This wiki page describes the design for adding user managed metadata to instances.
Blueprint
https://blueprints.launchpad.net/trove/+spec/trove-metadata
Goals
- Give trove users the ability to store key/value pairs that have significance to them but is not essential to trove's functionality
- Create a user accessible rest API to manage instance metadata
- Mimic the design/interface used by Nova for metadata
- Implement the model so that it acts like a dictionary to make hooking in metadata into the instance very easy and small
Description
Instance metadata is a feature that has been requested frequently by our users. They need a way to store critical information for their instances and have that be associated with the instance so that it is displayed whenever that instance is listed via the API. This also becomes very usable from a testing perspective when doing integration/ci. We can utilize the metadata to store things like what process created the instance, what the instance is being used for, etc... The design for this feature is modeled heavily on the Nova metadata API with a few tweaks in how it works internally.
With the likelihood of an Openstack official metadata service on the horizon I felt like we needed to keep the entry point of integration very small so that when that service becomes available there will be little code in the instance model to change to make that integration happen. To achieve this I made the model work using the dictionary interface and kept all the heavy lifting and fetching/saving operations are internal to the model and happen automatically when keys are modified.
All keys must be strings. Acceptable value types are:
- Lists
- Dictionaries
- Strings
- Integers
- Floats
Database Schema
Table name
instance_metadata
Field Names and Types
Name | Type |
---|---|
id | String(36) |
instance_uuid | String(36) |
key | String(255) |
value | Text |
created | DateTime |
updated | DateTime |
deleted | Boolean |
deleted_at | DateTime |
Constraints
There is a unique constraint on the fields instance_uuid and key together to prevent duplicate keys per instance. There is code in the model and controller to prevent this also but the unique constraint is there as an added layer of security.
API
The following section describes the API interface both what needs to be sent to the api and what the responses look like.
Create
API Call
API Path and Method
POST /instances/{id}/metadata/{key}
Body
{ "metadata": { "value": {"one": {"two": [3, 4, 5]}} } }
API Response
Response Codes
- Positive: 200
- Negative: 400
Response Body
{ "metadata": { "newKey": {"one": {"two": [3, 4, 5]}} } }
Update
Substitute a key and value in the database with a new key and value.
API Call
API Path and Method
PUT /instances/{id}/metadata/{key}
Body
{ "metadata": { "key": "newKey2", "value": {"one": {"two": [3, 4, 5]}} } }
API Response
Response Codes
- Positive: 200
- Negative: 404
Response Body
None
Replace
API Call
API Path and Method
PATCH /instances/{id}/metadata/{key}
Body
{ "metadata": { "value": {"one": {"two": [3, 4, 5], "key2": "the value of this key is a string"}} } }
API Response
Response Codes
- Positive: 200
- Negative: 404
Response Body
None
Delete
API Call
API Path and Method
DELETE /instances/{id}/metadata/{key}
Body
None
API Response
Response Codes
- Positive 200
- Negative 404
Response Body
None
Show
API Call
API Path and Method
GET /instances/{id}/metadata/{key}
Body
None
API Response
Response Codes
- Positive 200
- Negative 404
Response Body
{ "metadata": { "newKey": {"one": {"two": [3, 4, 5]}} } }
List
API Call
API Path and Method
GET /instances/{id}/metadata
Body
None
API Response
Response Codes
- Positive: 200
- No negative code because this returns an empty dictionary when no metadata exists for the instance
Response Body
{ "metadata": { "myKey": {"one": {"two": [3, 4, 5]}}, "keyTwo": "I wanted to store this sentence for this key" } }
TroveClient
Create
Create a new metadata entry for a given instance and key.
Arguments
- instance_uuid: All metadata must be associated with an instance at this point so the instance uuid is necessary to form the API path
- key: The key to store in the database.
- value: The value to store in the database and associate with key above.
CLI
There is code in the cli library that attempts to deserialize any strings passed into the library to allow users to pass in complex data structures in JSON format via the CLI. A failed json.loads() is ignored and the library then assumes that it is just a simple string and inserts that string into the database.
Example
trove metadata-create d241b3ce-e382-4dcc-ace7-769380438741 myKey '{"one": {"two": [3, 4, 5]}}'
Python Library
Example
self.client.metadata.create('d241b3ce-e382-4dcc-ace7-769380438741', 'myKey', '{"one": {"two": [3, 4, 5]}}')
Update
Substitute a newkey for an orginal key and a new value for the old key's value.
Arguments
- instance_id: All metadata must be associated with an instance at this point so the instance uuid is necessary to form the API path
- key: The key to be updated.
- newkey: The key to replace key above.
- value: The value to be assigned to newkey above.
CLI
Example
trove metadata-update d241b3ce-e382-4dcc-ace7-769380438741 myKey instanceProvisionedBy 'Jenkins CI 4/1/14'
Python Library
Example
self.client.metadata.update('d241b3ce-e382-4dcc-ace7-769380438741', 'myKey', 'instanceProvisionedBy', 'Jenkins CI')
Replace
Change the value of a given key.
Arguments
- instance_id: All metadata must be associated with an instance at this point so the instance uuid is necessary to form the API path
- key: The key to operate on
- value: The new value to assign to key
CLI
Example
trove metadata-replace d241b3ce-e382-4dcc-ace7-769380438741 instanceProvisionedBy 'Jenkins CI 4/1/14'
Python Library
Example
self.client.metadata.replace('d241b3ce-e382-4dcc-ace7-769380438741', 'instanceProvisionedBy', 'Jenkins CI')
Delete
Delete a key and it's associated value from the database
Arguments
- instance_id: All metadata must be associated with an instance at this point so the instance uuid is necessary to form the API path
- key: The key to be deleted
CLI
Example
trove metadata-delete d241b3ce-e382-4dcc-ace7-769380438741 instanceProvisionedBy
Python Library
Example
self.client.metadata.delete('d241b3ce-e382-4dcc-ace7-769380438741', 'instanceProvisionedBy')
Show
Show the value of a key
Arguments
- instance_id: All metadata must be associated with an instance at this point so the instance uuid is necessary to form the API path
- key: The key to be deleted
CLI
Example
trove metadata-show d241b3ce-e382-4dcc-ace7-769380438741 newKey
Returned Body
{ "metadata": { "newKey": {"one": {"two": [3, 4, 5]}} } }
Python Library
Example
self.client.metadata.show('d241b3ce-e382-4dcc-ace7-769380438741', 'newKey')
Returned Body
{ "metadata": { "newKey": {"one": {"two": [3, 4, 5]}} } }
List
Arguments
- instance_id: All metadata must be associated with an instance at this point so the instance uuid is necessary to form the API path
CLI
Example
trove metadata-list d241b3ce-e382-4dcc-ace7-769380438741
Returned Body
{ "metadata": { "myKey": {"one": {"two": [3, 4, 5]}}, "keyTwo": "I wanted to store this sentence for this key" } }
Python Library
Example
self.client.metadata.list('d241b3ce-e382-4dcc-ace7-769380438741')
Returned Body
{ "metadata": { "myKey": {"one": {"two": [3, 4, 5]}}, "keyTwo": "I wanted to store this sentence for this key" } }
Integration with Instance
First pass I will not allow metadata to be passed in on instance create however this should be added soon after. You will however be able to see metadata in the instance object when you do a SHOW on the instance. Also when deleting an instance all metadata is deleted to keep things neat and tidy.