CloudInventoryManager


 * Launchpad Entry: Cloud Inventory Manager
 * Created: 02 April 2012
 * Contributors: Yehia Beyh, Lyle Wilkinson, Divakar Nandavar

Summary
Provide an extensible service to Nova to include Cloud inventory for OpenStack cloud resources and infrastructure. Cloud Inventory Manager provides insights into cloud resource system information and takes advantage of underlying virtualization manager (e.g. libvirt) resource hierarchy such as Cluster, Resource pool, Storage, Network and Server pools.

End users and system administrators can use the resource inventory to build a catalog of available Clusters, Resource Pools, Storage pools, Network pools and Server pools.


 * Resource details provide resources discovered via virtualization managers, e.g. libvirt.
 * Resource information is persisted in the database. Details as needed are obtained via dynamic REST API query
 * Discovers and monitors resource changes (add/update/delete) caused by OpenStack or underlying resource managers
 * Explicitly models resources that can be allocated to elements of a Service, these Resources are then organized into Resource pools

Further, this inventory manager lays the groundwork for an improved scheduler and resource allocation implementation that will enable multi-tenant public and private cloud resource management.

Release Note
Cloud Inventory Manager service provides APIs based on the OpenStack extension APIs model. Hence, existing users should not be impacted. Advanced users and cloud providers can take advantage of the resource inventory data for analysis, planning and provisioning decisions.

See API descriptions for additional details.

Rationale
Note that the concept of Aggregates was introduced into Nova via the following blueprint:
 * https://blueprints.launchpad.net/nova/+spec/host-aggregates

The Cloud Inventory Manager service extends this concept from Host Aggregation to general other cloud resources (e.g. Cluster, Resource pool, Host, VM, Storage, Network, Subnets,  VirtualSwitch, PortGroups). See Resource Model UML diagram below.

User stories

 * As a Cloud Administrator, I need support to build a catalog of available Cloud resources.
 * As a Cloud Administrator, I need to support Cloud Resource pool construction and maintenance based on capacities, quotas, and resource groups.
 * As a Cloud Administrator, I need to support viewing of resource inventory, resource consumption, utilization history and trends.
 * As a Cloud Administrator, I need to support importing of existing servers not yet managed by OpenStack.
 * As a Cloud User or Administrator, I need cloud resource metadata to support provisioning, optimization, and ongoing management of infrastructure services.
 * As a Cloud Administrator, I need to support analysis, manual modification, or autonomic modification of infrastructure services to improve their performance, resource usage, resource consumption, etc.

Design
Cloud Inventory Manager, part of the health and monitor service architecture, supports a plug-in architecture similar to other OpenStack component implementations. The design supports multiple implementations to be plugged using the driver model that is being followed in OpenStack through configuration settings. It is extendable to support multiple Hypervisor types (example: ESX, Hyper-V, KVM, etc.).



Cloud Inventory Manager provides:


 * Driver based plug-in model to support multiple implementations
 * plugged in using the driver model that is being followed in OpenStack through configuration settings.
 * Support for plugging in implementations for multiple Hypervisor types (example: ESX, Hyperv, KVM etc.,)
 * REST APIs for programmatic access as an Nova extension API.
 * Support Controller and Resource Extensions.
 * Inventory Data (Resource details) based on resource model (see Resource Model UML diagram below. Note that the Healthnmon XSD is the diagram above is derived from this Resource Model).



Implementation
Note: For discussion purpose the module name "Healthnmon" is used which implements the required functionality of Cloud Inventory details.

Architecture



Configure nova to use healthnmon extensions
Add the following to nova.conf file(/etc/nova/nova.conf) and restart nova-api server.

--osapi_compute_extension=healthnmon.api.healthnmon.Healthnmon

Sample Curl Usage
The following shows a sample usage to get list of vmhosts in xml format using the auth token created

curl -i -k -H "X-Auth-Token:novaadmin:admin"  -H "Accept:application/xml" http://10.10.120.158:8774/v2/admin/vmhosts

JSON Response
{ "extension" : { "updated" : "2012-01-22T13:25:27-06:00", "name" : "healthnmon", "links" : [], "namespace" : "http://docs.openstack.org/ext/healthnmon/api/v2.0", "alias" : "HP-HNM", "description" : " Health and monitoring API as nova compute extension API's\n   " } }

XML Response
 Health and monitoring API as nova compute extension API's

Normal Response Code(s): 200, 203 Error Response Code(s): computeFault (400), serviceUnavailable (503), unauthorized (401), forbidden (403), badRequest (400), badMethod (405), overLimit (413)

Get VmHost List
Normal Response Code: 200 Error(s) Response Code: Compute Fault(400,500), Bad Request(400)

JSON Response
{ "vmhosts" : [ { "id" : "34333832-3230-434e-3738-313630323650", "links" : [ {         "href" : "http://10.10.120.158:8774/v2/admin/vmhosts/34333832-3230-434e-3738-313630323650", "rel" : "self" },       {          "href" : "http://10.10.120.158:8774/admin/vmhosts/34333832-3230-434e-3738-313630323650", "rel" : "bookmark" } ],   "name" : "ubuntukvm.vmm.hp.com" } ] }

XML Response
   

Get VmHost details
Normal Response Code: 200 Error(s) Response Code: HTTP Not Found(404) -- if id is invalid or non-existent, Compute Fault(400,500), Bad Request(400)

XML Response
 34333832-3230-434e-3738-313630323650 ubuntukvm.vmm.hp.com 34333832-3230-434e-3738-313630323650  2054144 2 2 <processorArchitecture>X86_64</processorArchitecture> <serialNumber>CN7816026P     </serialNumber> <memoryConsumed>2054144</memoryConsumed> <os> <osType>KVM</osType> <osDescription>KVM</osDescription> <osName>QEMU</osName> <osVersion>14001</osVersion> </os> ProLiant BL465c G1 	 34333832-3230-434e-3738-313630323650 <virtualizationType>KVM</virtualizationType> <storagevolume xmlns:atom="http://www.w3.org/2005/Atom" id="78e3d6c0-698b-8a90-88e2-8043d6a82476"> <atom:link href="http://10.10.120.158:8774/v2/admin/storagevolumes/78e3d6c0-698b-8a90-88e2-8043d6a82476" rel="self" /> <atom:link href="http://10.10.120.158:8774/admin/storagevolumes/78e3d6c0-698b-8a90-88e2-8043d6a82476" rel="bookmark" /> <storagevolume xmlns:atom="http://www.w3.org/2005/Atom" id="915212e9-f8cc-2901-11dd-713107879e6d"> <atom:link href="http://10.10.120.158:8774/v2/admin/storagevolumes/915212e9-f8cc-2901-11dd-713107879e6d" rel="self" /> <atom:link href="http://10.10.120.158:8774/admin/storagevolumes/915212e9-f8cc-2901-11dd-713107879e6d" rel="bookmark" /> <storagevolume xmlns:atom="http://www.w3.org/2005/Atom" id="e52c1b6e-2128-48ce-65d7-38dec73a26fc"> <atom:link href="http://10.10.120.158:8774/v2/admin/storagevolumes/e52c1b6e-2128-48ce-65d7-38dec73a26fc" rel="self" /> <atom:link href="http://10.10.120.158:8774/admin/storagevolumes/e52c1b6e-2128-48ce-65d7-38dec73a26fc" rel="bookmark" /> <storagevolume xmlns:atom="http://www.w3.org/2005/Atom" id="d57ea5e3-c935-8c32-e883-4af4f4b66714"> <atom:link href="http://10.10.120.158:8774/v2/admin/storagevolumes/d57ea5e3-c935-8c32-e883-4af4f4b66714" rel="self" /> <atom:link href="http://10.10.120.158:8774/admin/storagevolumes/d57ea5e3-c935-8c32-e883-4af4f4b66714" rel="bookmark" /> <virtualmachine xmlns:atom="http://www.w3.org/2005/Atom" id="854f5cf4-233b-145f-cbc9-b558a7a65660"> <atom:link href="http://10.10.120.158:8774/v2/admin/virtualmachines/854f5cf4-233b-145f-cbc9-b558a7a65660" rel="self" /> <atom:link href="http://10.10.120.158:8774/admin/virtualmachines/854f5cf4-233b-145f-cbc9-b558a7a65660" rel="bookmark" /> <virtualmachine xmlns:atom="http://www.w3.org/2005/Atom" id="93295e51-3dcc-eacf-5ce0-67982c2e8997"> <atom:link href="http://10.10.120.158:8774/v2/admin/virtualmachines/93295e51-3dcc-eacf-5ce0-67982c2e8997" rel="self" /> <atom:link href="http://10.10.120.158:8774/admin/virtualmachines/93295e51-3dcc-eacf-5ce0-67982c2e8997" rel="bookmark" /> </VmHost>

JSON Response
{ "VmHost" : { "virtualizationType" : "KVM", "resourceManagerId" : "34333832-3230-434e-3738-313630323650", "storagevolume" : { "id" : "78e3d6c0-698b-8a90-88e2-8043d6a82476", "links" : [ {           "href" : "http://10.10.120.158:8774/v2/admin/storagevolumes/78e3d6c0-698b-8a90-88e2-8043d6a82476", "rel" : "self" },         {            "href" : "http://10.10.120.158:8774/admin/storagevolumes/78e3d6c0-698b-8a90-88e2-8043d6a82476", "rel" : "bookmark" } ]   },    "name" : "ubuntukvm.vmm.hp.com", "virtualmachine" : { "1" : {       "id" : "93295e51-3dcc-eacf-5ce0-67982c2e8997", "links" : [ {             "href" : "http://10.10.120.158:8774/v2/admin/virtualmachines/93295e51-3dcc-eacf-5ce0-67982c2e8997", "rel" : "self" },           {              "href" : "http://10.10.120.158:8774/admin/virtualmachines/93295e51-3dcc-eacf-5ce0-67982c2e8997", "rel" : "bookmark" } ]     },      "id" : "854f5cf4-233b-145f-cbc9-b558a7a65660", "links" : [ {           "href" : "http://10.10.120.158:8774/v2/admin/virtualmachines/854f5cf4-233b-145f-cbc9-b558a7a65660", "rel" : "self" },         {            "href" : "http://10.10.120.158:8774/admin/virtualmachines/854f5cf4-233b-145f-cbc9-b558a7a65660", "rel" : "bookmark" } ]   },    "memoryConsumed" : "2054144", "serialNumber" : "CN7816026P     ", "memorySize" : "2054144", "processorCoresCount" : "2", "processorCount" : "2", "processorArchitecture" : "X86_64", "model" : "ProLiant BL465c G1 ", "os" : { "osType" : "KVM", "osDescription" : "KVM", "osVersion" : "14001", "osName" : "QEMU" },   "id" : "34333832-3230-434e-3738-313630323650", "uuid" : "34333832-3230-434e-3738-313630323650" } }

Storage extension api

 * /v1.1/storagevolumes - Returns list of all available storageVolumes.
 * /v1.1/storagevolumes/{id} - Returns storagevolume details of the specified 'id'.

Get Storage Volume List
Normal Response Code: 200 Error(s) Response Code: Compute Fault(400,500), Bad Request(400)

XML Response
<storagevolumes xmlns="http://docs.openstack.org/ext/healthnmon/api/v2.0" xmlns:atom="http://www.w3.org/2005/Atom"> <storagevolume id="d57ea5e3-c935-8c32-e883-4af4f4b66714" name="iscsipool"> <atom:link href="http://10.10.120.158:8774/v2/admin/storagevolumes/d57ea5e3-c935-8c32-e883-4af4f4b66714" rel="self" /> <atom:link href="http://10.10.120.158:8774/admin/storagevolumes/d57ea5e3-c935-8c32-e883-4af4f4b66714" rel="bookmark" /> <storagevolume id="78e3d6c0-698b-8a90-88e2-8043d6a82476" name="default"> <atom:link href="http://10.10.120.158:8774/v2/admin/storagevolumes/78e3d6c0-698b-8a90-88e2-8043d6a82476" rel="self" /> <atom:link href="http://10.10.120.158:8774/admin/storagevolumes/78e3d6c0-698b-8a90-88e2-8043d6a82476" rel="bookmark" /> <storagevolume id="e52c1b6e-2128-48ce-65d7-38dec73a26fc" name="isopool"> <atom:link href="http://10.10.120.158:8774/v2/admin/storagevolumes/e52c1b6e-2128-48ce-65d7-38dec73a26fc" rel="self" /> <atom:link href="http://10.10.120.158:8774/admin/storagevolumes/e52c1b6e-2128-48ce-65d7-38dec73a26fc" rel="bookmark" /> <storagevolume id="915212e9-f8cc-2901-11dd-713107879e6d" name="nfspool"> <atom:link href="http://10.10.120.158:8774/v2/admin/storagevolumes/915212e9-f8cc-2901-11dd-713107879e6d" rel="self" /> <atom:link href="http://10.10.120.158:8774/admin/storagevolumes/915212e9-f8cc-2901-11dd-713107879e6d" rel="bookmark" />

JSON Response
{ "storagevolumes" : [ {   "id" : "d57ea5e3-c935-8c32-e883-4af4f4b66714", "links" : [ {      "href" : "http://10.10.120.158:8774/v2/admin/storagevolumes/d57ea5e3-c935-8c32-e883-4af4f4b66714", "rel" : "self" },     {       "href" : "http://10.10.120.158:8774/admin/storagevolumes/d57ea5e3-c935-8c32-e883-4af4f4b66714", "rel" : "bookmark" } ],   "name" : "iscsipool" },  {    "id" : "78e3d6c0-698b-8a90-88e2-8043d6a82476", "links" : [ {      "href" : "http://10.10.120.158:8774/v2/admin/storagevolumes/78e3d6c0-698b-8a90-88e2-8043d6a82476", "rel" : "self" },     {       "href" : "http://10.10.120.158:8774/admin/storagevolumes/78e3d6c0-698b-8a90-88e2-8043d6a82476", "rel" : "bookmark" } ],   "name" : "default" },  {    "id" : "e52c1b6e-2128-48ce-65d7-38dec73a26fc", "links" : [ {      "href" : "http://10.10.120.158:8774/v2/admin/storagevolumes/e52c1b6e-2128-48ce-65d7-38dec73a26fc", "rel" : "self" },     {       "href" : "http://10.10.120.158:8774/admin/storagevolumes/e52c1b6e-2128-48ce-65d7-38dec73a26fc", "rel" : "bookmark" } ],   "name" : "isopool" },  {    "id" : "915212e9-f8cc-2901-11dd-713107879e6d", "links" : [ {      "href" : "http://10.10.120.158:8774/v2/admin/storagevolumes/915212e9-f8cc-2901-11dd-713107879e6d", "rel" : "self" },     {       "href" : "http://10.10.120.158:8774/admin/storagevolumes/915212e9-f8cc-2901-11dd-713107879e6d", "rel" : "bookmark" } ],   "name" : "nfspool" } ] }

UI Changes
Should cover changes required to the UI, or specific UI that is required to implement this

Code Changes
Code changes should include an overview of what needs to change, and in some cases even the specific details.

Migration
n/a

Test/Demo Plan
This need not be added or completed until the specification is nearing beta.

Unresolved issues
This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.

BoF agenda and discussion
Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarizing what was discussed and note any options that were rejected.

http://etherpad.openstack.org/Cloud-Inventory-Manager