Jump to: navigation, search

Difference between revisions of "CloudInventoryManager"

Line 3: Line 3:
 
* '''Created''':  02 April 2012
 
* '''Created''':  02 April 2012
 
* '''Contributors''': Yehia Beyh, Lyle Wilkinson, Divakar Nandavar
 
* '''Contributors''': Yehia Beyh, Lyle Wilkinson, Divakar Nandavar
 +
 +
<<[[TableOfContents]]()>>
  
 
== Summary ==
 
== Summary ==
  
−
Provide an extensible service to allow [[OpenStack]] components (swift, glance, nova, quantum, etc.) to include Cloud inventory for [[OpenStack]] cloud resources and infrastructure. Cloud Inventory Manager provides insights into cloud resource system information and take advantage of underlying resource manager (vCenter, SCVMM, RHEVM etc.,) resource hierarchy such as Cluster, Resource pool, Storage, Network and Server pools.
+
Provide an extensible service to allow [[OpenStack]] components (swift, glance, nova, quantum, etc.) to include Cloud inventory for [[OpenStack]] cloud resources and infrastructure. Cloud Inventory Manager provides insights into cloud resource system information and takes advantage of underlying resource manager (vCenter, SCVMM, RHEVM etc.,) resource hierarchy such as Cluster, Resource pool, Storage, Network and Server pools.
  
 
End users and system administrators can use the resource inventory to build a catalog of available Clusters, Resource Pools, Storage pools, Network pools and Server pools.
 
End users and system administrators can use the resource inventory to build a catalog of available Clusters, Resource Pools, Storage pools, Network pools and Server pools.
  
 
* Resource details provide resources discovered via resource managers like vCenter, SCVMM, RHEVM etc.,
 
* Resource details provide resources discovered via resource managers like vCenter, SCVMM, RHEVM etc.,
−
* Sparse information about resource persisted in the database. Details as needed obtained via dynamic query via REST API extensions
+
* Resource information is persisted in the database. Details as needed are obtained via dynamic REST API query
−
* Discovers, monitors resources changes (add/update/delete) via [[OpenStack]] & underlying Resource managers
+
* Discovers and monitors resource changes (add/update/delete) caused by [[OpenStack]] or underlying resource managers
−
* Explicitly model resources that can be allocated to elements of a Service, these Resources are then organized into Resource pools
+
* Explicitly models resources that can be allocated to elements of a Service, these Resources are then organized into Resource pools
 +
 
 +
Further, this inventory manager lays the groundwork for an improved scheduler and resource allocation implementation that will enable multi-tenant public and private cloud resource management.
  
 
== Release Note ==
 
== Release Note ==
  
−
Cloud Inventory Manager service provides APIs based on Openstack extension APIs model. Hence, existing users should not be impacted.  Advanced users and cloud providers can take advantage of the resource inventory data for analysis, planning and provisioning decisions.
+
Cloud Inventory Manager service provides APIs based on the [[OpenStack]] extension APIs model. Hence, existing users should not be impacted.  Advanced users and cloud providers can take advantage of the resource inventory data for analysis, planning and provisioning decisions.
  
 
See API descriptions for additional details.
 
See API descriptions for additional details.
Line 23: Line 27:
 
== User stories ==
 
== User stories ==
  
−
* As a Nova administrator, I need support to build a catalog of available Cloud resources and infrastructure.
+
* As a Nova administrator, I need support to build a catalog of available Cloud resources.
−
* As a Nova administrator / tenant administrator, I need to support Cloud Resource pool construction and maintenance based on capacities, quotas, resource groups.
+
* As a Nova administrator (or tenant administrator), I need to support Cloud Resource pool construction and maintenance based on capacities, quotas, and resource groups.
−
* As a Nova administrator / tenant administrator, I need to support View resource inventory, resource consumption, utilization history and trends.
+
* As a Nova administrator (or tenant administrator), I need to support viewing of resource inventory, resource consumption, utilization history and trends.
−
* As a Nova administrator, I need to Support Import existing, running infrastructure as a service
+
* As a Nova administrator, I need to support importing of existing servers not yet managed by [[OpenStack]].
 
* As a Nova administrator, I need cloud resource metadata to support provisioning, optimization, and ongoing management of infrastructure services.
 
* As a Nova administrator, I need cloud resource metadata to support provisioning, optimization, and ongoing management of infrastructure services.
−
* As a Nova administrator, I need to support to Analyze, manually modify, or autonomically modify infrastructure services to improve their performance, resource usage/consumption, etc.
+
* As a Nova administrator, I need to support analysis, manual modification, or autonomic modification of infrastructure services to improve their performance, resource usage, resource consumption, etc.
  
 
== Assumptions ==
 
== Assumptions ==

Revision as of 20:03, 9 April 2012

  • Launchpad Entry: Cloud Inventory Manager
  • Created: 02 April 2012
  • Contributors: Yehia Beyh, Lyle Wilkinson, Divakar Nandavar

<<TableOfContents()>>

Summary

Provide an extensible service to allow OpenStack components (swift, glance, nova, quantum, etc.) to include Cloud inventory for OpenStack cloud resources and infrastructure. Cloud Inventory Manager provides insights into cloud resource system information and takes advantage of underlying resource manager (vCenter, SCVMM, RHEVM etc.,) resource hierarchy such as Cluster, Resource pool, Storage, Network and Server pools.

End users and system administrators can use the resource inventory to build a catalog of available Clusters, Resource Pools, Storage pools, Network pools and Server pools.

  • Resource details provide resources discovered via resource managers like vCenter, SCVMM, RHEVM etc.,
  • Resource information is persisted in the database. Details as needed are obtained via dynamic REST API query
  • Discovers and monitors resource changes (add/update/delete) caused by OpenStack or underlying resource managers
  • Explicitly models resources that can be allocated to elements of a Service, these Resources are then organized into Resource pools

Further, this inventory manager lays the groundwork for an improved scheduler and resource allocation implementation that will enable multi-tenant public and private cloud resource management.

Release Note

Cloud Inventory Manager service provides APIs based on the OpenStack extension APIs model. Hence, existing users should not be impacted. Advanced users and cloud providers can take advantage of the resource inventory data for analysis, planning and provisioning decisions.

See API descriptions for additional details.

User stories

  • As a Nova administrator, I need support to build a catalog of available Cloud resources.
  • As a Nova administrator (or tenant administrator), I need to support Cloud Resource pool construction and maintenance based on capacities, quotas, and resource groups.
  • As a Nova administrator (or tenant administrator), I need to support viewing of resource inventory, resource consumption, utilization history and trends.
  • As a Nova administrator, I need to support importing of existing servers not yet managed by OpenStack.
  • As a Nova administrator, I need cloud resource metadata to support provisioning, optimization, and ongoing management of infrastructure services.
  • As a Nova administrator, I need to support analysis, manual modification, or autonomic modification of infrastructure services to improve their performance, resource usage, resource consumption, etc.

Assumptions

Design

Cloud Inventory Manager service provides

  • Driver based implementation of the Cloud Inventory service to support multiple implemenations / to be plugged using the driver model that is being followed in openstack through configuration settings.
  • Support for plugging in implementations for multiple Hypervisor types (example: ESX, Hyperv, KVM etc.,)
  • REST APIs for programmatic access as an Nova extension API.
  • Inventory Data (Resource details) based on the attachment:healthnmonResourceModel.xsd "Healthnmon Resource model" definition.

Implementation

Note: For discussion purpose the module name "Healthnmon" is used which implements the required functionality of Cloud Inventory details.

Architecture

File:CloudInventoryManager$cloudinventorymanager-bp-arch.gif

Configure nova to use healthnmon extensions

Add the following to nova.conf file(/etc/nova/nova.conf) and restart nova-api server. 

--osapi_compute_extension=healthnmon.api.healthnmon.Healthnmon


Sample Curl Usage

The following shows a sample usage to get list of vmhosts in xml format using the auth token created 

curl -i -k  -H "X-Auth-Token:novaadmin:admin"  -H "Accept:application/xml" http://10.10.120.158:8774/v2/admin/vmhosts


Extension Metadata

Verb URI
GET v2/{project_id}/extensions/healthnmon

JSON Response

{
  "extension" : {
    "updated" : "2012-01-22T13:25:27-06:00",
    "name" : "healthnmon",
    "links" : [],
    "namespace" : "http://docs.openstack.org/ext/healthnmon/api/v2.0",
    "alias" : "healthnmon",
    "description" : " Health and monitoring API as nova compute extension API's\n    "
  }
}


XML Response

<extension xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns="http://docs.openstack.org/compute/api/v1.1" alias="healthnmon"
	updated="2012-01-22T13:25:27-06:00" namespace="http://docs.openstack.org/ext/healthnmon/api/v2.0"
	name="healthnmon">
	<description> Health and monitoring API as nova compute extension API's
	</description>
</extension>


Verb URI
GET /clusters?name=clusterName&marker=markerID&limit=int&changes-since=dateTime
GET /clusters/detail?name=clusterName&marker=markerID&limit=int&changes-since=dateTime
GET /clusters/id
Verb URI
GET /resourcepools?name=resourcepoolName&marker=markerID&limit=int&changes-since=dateTime
GET /resourcepools/detail?name=clusterName&marker=markerID&limit=int&changes-since=dateTime
GET /resourcepools/id
Verb URI
GET /hosts?type=hostType&name=hostName&status=hostStatus&marker=markerID&limit=int&changes-since=dateTime
GET /hosts/detail?type=hostType&name=hostName&status=hostStatus&marker=markerID&limit=int&changes-since=dateTime
GET /hosts/id
Verb URI
GET /subnets
GET /subnets/{id}
GET /virtualswitches
GET /virtualswitches/{id}
Verb URI
GET /storagevolumes
GET /storagevolumes/id
Verb URI
GET /virtualmachines
GET /virtualmachines/id

Normal Response Code(s): 200, 203 Error Response Code(s): computeFault (400), serviceUnavailable (503), unauthorized (401), forbidden (403), badRequest (400), badMethod (405), overLimit (413)

Get VmHost List

Verb URI
GET /vmhosts

Normal Response Code: 200
Error(s) Response Code: Compute Fault(400,500), Bad Request(400)

JSON Response

{
  "vmhosts" : [ {
    "id" : "34333832-3230-434e-3738-313630323650",
    "links" : [
        {
          "href" : "http://10.10.120.158:8774/v2/admin/vmhosts/34333832-3230-434e-3738-313630323650",
          "rel" : "self"
        },
        {
          "href" : "http://10.10.120.158:8774/admin/vmhosts/34333832-3230-434e-3738-313630323650",
          "rel" : "bookmark"
        } ],
    "name" : "ubuntukvm.vmm.hp.com"
  } ]
}

XML Response

<vmhosts xmlns="http://docs.openstack.org/ext/healthnmon/api/v2.0"
	xmlns:atom="http://www.w3.org/2005/Atom">
	<vmhost id="34333832-3230-434e-3738-313630323650" name="ubuntukvm.vmm.hp.com">
		<atom:link
			href="http://10.10.120.158:8774/v2/admin/vmhosts/34333832-3230-434e-3738-313630323650"
			rel="self" />
		<atom:link
			href="http://10.10.120.158:8774/admin/vmhosts/34333832-3230-434e-3738-313630323650"
			rel="bookmark" />
	</vmhost>
</vmhosts>


Get VmHost details

Verb URI
GET /vmhosts/id

Normal Response Code: 200
Error(s) Response Code: HTTP Not Found(404) -- if id is invalid or non-existent, Compute Fault(400,500), Bad Request(400)

XML Response

<VmHost>
	<id>34333832-3230-434e-3738-313630323650</id>
	<name>ubuntukvm.vmm.hp.com</name>
	<resourceManagerId>34333832-3230-434e-3738-313630323650
	</resourceManagerId>
	<memorySize>2054144</memorySize>
	<processorCount>2</processorCount>
	<processorCoresCount>2</processorCoresCount>
	<processorArchitecture>X86_64</processorArchitecture>
	<serialNumber>CN7816026P      </serialNumber>
	<memoryConsumed>2054144</memoryConsumed>
	<os>
		<osType>KVM</osType>
		<osDescription>KVM</osDescription>
		<osName>QEMU</osName>
		<osVersion>14001</osVersion>
	</os>
	<model>ProLiant BL465c G1  </model>
	<uuid>34333832-3230-434e-3738-313630323650</uuid>
	<virtualizationType>KVM</virtualizationType>
	<storagevolume xmlns:atom="http://www.w3.org/2005/Atom"
		id="78e3d6c0-698b-8a90-88e2-8043d6a82476">
		<atom:link
			href="http://10.10.120.158:8774/v2/admin/storagevolumes/78e3d6c0-698b-8a90-88e2-8043d6a82476"
			rel="self" />
		<atom:link
			href="http://10.10.120.158:8774/admin/storagevolumes/78e3d6c0-698b-8a90-88e2-8043d6a82476"
			rel="bookmark" />
	</storagevolume>
	<storagevolume xmlns:atom="http://www.w3.org/2005/Atom"
		id="915212e9-f8cc-2901-11dd-713107879e6d">
		<atom:link
			href="http://10.10.120.158:8774/v2/admin/storagevolumes/915212e9-f8cc-2901-11dd-713107879e6d"
			rel="self" />
		<atom:link
			href="http://10.10.120.158:8774/admin/storagevolumes/915212e9-f8cc-2901-11dd-713107879e6d"
			rel="bookmark" />
	</storagevolume>
	<storagevolume xmlns:atom="http://www.w3.org/2005/Atom"
		id="e52c1b6e-2128-48ce-65d7-38dec73a26fc">
		<atom:link
			href="http://10.10.120.158:8774/v2/admin/storagevolumes/e52c1b6e-2128-48ce-65d7-38dec73a26fc"
			rel="self" />
		<atom:link
			href="http://10.10.120.158:8774/admin/storagevolumes/e52c1b6e-2128-48ce-65d7-38dec73a26fc"
			rel="bookmark" />
	</storagevolume>
	<storagevolume xmlns:atom="http://www.w3.org/2005/Atom"
		id="d57ea5e3-c935-8c32-e883-4af4f4b66714">
		<atom:link
			href="http://10.10.120.158:8774/v2/admin/storagevolumes/d57ea5e3-c935-8c32-e883-4af4f4b66714"
			rel="self" />
		<atom:link
			href="http://10.10.120.158:8774/admin/storagevolumes/d57ea5e3-c935-8c32-e883-4af4f4b66714"
			rel="bookmark" />
	</storagevolume>
	<virtualmachine xmlns:atom="http://www.w3.org/2005/Atom"
		id="854f5cf4-233b-145f-cbc9-b558a7a65660">
		<atom:link
			href="http://10.10.120.158:8774/v2/admin/virtualmachines/854f5cf4-233b-145f-cbc9-b558a7a65660"
			rel="self" />
		<atom:link
			href="http://10.10.120.158:8774/admin/virtualmachines/854f5cf4-233b-145f-cbc9-b558a7a65660"
			rel="bookmark" />
	</virtualmachine>
	<virtualmachine xmlns:atom="http://www.w3.org/2005/Atom"
		id="93295e51-3dcc-eacf-5ce0-67982c2e8997">
		<atom:link
			href="http://10.10.120.158:8774/v2/admin/virtualmachines/93295e51-3dcc-eacf-5ce0-67982c2e8997"
			rel="self" />
		<atom:link
			href="http://10.10.120.158:8774/admin/virtualmachines/93295e51-3dcc-eacf-5ce0-67982c2e8997"
			rel="bookmark" />
	</virtualmachine>
</VmHost>

JSON Response

{
  "VmHost" : {
    "virtualizationType" : "KVM",
    "resourceManagerId" : "34333832-3230-434e-3738-313630323650",
    "storagevolume" : {
      "id" : "78e3d6c0-698b-8a90-88e2-8043d6a82476",
      "links" : [
          {
            "href" : "http://10.10.120.158:8774/v2/admin/storagevolumes/78e3d6c0-698b-8a90-88e2-8043d6a82476",
            "rel" : "self"
          },
          {
            "href" : "http://10.10.120.158:8774/admin/storagevolumes/78e3d6c0-698b-8a90-88e2-8043d6a82476",
            "rel" : "bookmark"
          } ]
    },
    "name" : "ubuntukvm.vmm.hp.com",
    "virtualmachine" : {
      "1" : {
        "id" : "93295e51-3dcc-eacf-5ce0-67982c2e8997",
        "links" : [
            {
              "href" : "http://10.10.120.158:8774/v2/admin/virtualmachines/93295e51-3dcc-eacf-5ce0-67982c2e8997",
              "rel" : "self"
            },
            {
              "href" : "http://10.10.120.158:8774/admin/virtualmachines/93295e51-3dcc-eacf-5ce0-67982c2e8997",
              "rel" : "bookmark"
            } ]
      },
      "id" : "854f5cf4-233b-145f-cbc9-b558a7a65660",
      "links" : [
          {
            "href" : "http://10.10.120.158:8774/v2/admin/virtualmachines/854f5cf4-233b-145f-cbc9-b558a7a65660",
            "rel" : "self"
          },
          {
            "href" : "http://10.10.120.158:8774/admin/virtualmachines/854f5cf4-233b-145f-cbc9-b558a7a65660",
            "rel" : "bookmark"
          } ]
    },
    "memoryConsumed" : "2054144",
    "serialNumber" : "CN7816026P      ",
    "memorySize" : "2054144",
    "processorCoresCount" : "2",
    "processorCount" : "2",
    "processorArchitecture" : "X86_64",
    "model" : "ProLiant BL465c G1  ",
    "os" : {
      "osType" : "KVM",
      "osDescription" : "KVM",
      "osVersion" : "14001",
      "osName" : "QEMU"
    },
    "id" : "34333832-3230-434e-3738-313630323650",
    "uuid" : "34333832-3230-434e-3738-313630323650"
  }
}


Storage extension api

  • /v1.1/storagevolumes - Returns list of all available storageVolumes.
  • /v1.1/storagevolumes/{id} - Returns storagevolume details of the specified 'id'.

Get Storage Volume List

Verb URI
GET /storagevolumes

Normal Response Code: 200
Error(s) Response Code: Compute Fault(400,500), Bad Request(400)

XML Response

<storagevolumes xmlns="http://docs.openstack.org/ext/healthnmon/api/v2.0"
	xmlns:atom="http://www.w3.org/2005/Atom">
	<storagevolume id="d57ea5e3-c935-8c32-e883-4af4f4b66714"
		name="iscsipool">
		<atom:link
			href="http://10.10.120.158:8774/v2/admin/storagevolumes/d57ea5e3-c935-8c32-e883-4af4f4b66714"
			rel="self" />
		<atom:link
			href="http://10.10.120.158:8774/admin/storagevolumes/d57ea5e3-c935-8c32-e883-4af4f4b66714"
			rel="bookmark" />
	</storagevolume>
	<storagevolume id="78e3d6c0-698b-8a90-88e2-8043d6a82476"
		name="default">
		<atom:link
			href="http://10.10.120.158:8774/v2/admin/storagevolumes/78e3d6c0-698b-8a90-88e2-8043d6a82476"
			rel="self" />
		<atom:link
			href="http://10.10.120.158:8774/admin/storagevolumes/78e3d6c0-698b-8a90-88e2-8043d6a82476"
			rel="bookmark" />
	</storagevolume>
	<storagevolume id="e52c1b6e-2128-48ce-65d7-38dec73a26fc"
		name="isopool">
		<atom:link
			href="http://10.10.120.158:8774/v2/admin/storagevolumes/e52c1b6e-2128-48ce-65d7-38dec73a26fc"
			rel="self" />
		<atom:link
			href="http://10.10.120.158:8774/admin/storagevolumes/e52c1b6e-2128-48ce-65d7-38dec73a26fc"
			rel="bookmark" />
	</storagevolume>
	<storagevolume id="915212e9-f8cc-2901-11dd-713107879e6d"
		name="nfspool">
		<atom:link
			href="http://10.10.120.158:8774/v2/admin/storagevolumes/915212e9-f8cc-2901-11dd-713107879e6d"
			rel="self" />
		<atom:link
			href="http://10.10.120.158:8774/admin/storagevolumes/915212e9-f8cc-2901-11dd-713107879e6d"
			rel="bookmark" />
	</storagevolume>
</storagevolumes>


JSON Response

{
 "storagevolumes" : [
   {
    "id" : "d57ea5e3-c935-8c32-e883-4af4f4b66714",
    "links" : [
      {
       "href" : "http://10.10.120.158:8774/v2/admin/storagevolumes/d57ea5e3-c935-8c32-e883-4af4f4b66714",
       "rel" : "self"
      },
      {
       "href" : "http://10.10.120.158:8774/admin/storagevolumes/d57ea5e3-c935-8c32-e883-4af4f4b66714",
       "rel" : "bookmark"
      } ],
    "name" : "iscsipool"
   },
   {
    "id" : "78e3d6c0-698b-8a90-88e2-8043d6a82476",
    "links" : [
      {
       "href" : "http://10.10.120.158:8774/v2/admin/storagevolumes/78e3d6c0-698b-8a90-88e2-8043d6a82476",
       "rel" : "self"
      },
      {
       "href" : "http://10.10.120.158:8774/admin/storagevolumes/78e3d6c0-698b-8a90-88e2-8043d6a82476",
       "rel" : "bookmark"
      } ],
    "name" : "default"
   },
   {
    "id" : "e52c1b6e-2128-48ce-65d7-38dec73a26fc",
    "links" : [
      {
       "href" : "http://10.10.120.158:8774/v2/admin/storagevolumes/e52c1b6e-2128-48ce-65d7-38dec73a26fc",
       "rel" : "self"
      },
      {
       "href" : "http://10.10.120.158:8774/admin/storagevolumes/e52c1b6e-2128-48ce-65d7-38dec73a26fc",
       "rel" : "bookmark"
      } ],
    "name" : "isopool"
   },
   {
    "id" : "915212e9-f8cc-2901-11dd-713107879e6d",
    "links" : [
      {
       "href" : "http://10.10.120.158:8774/v2/admin/storagevolumes/915212e9-f8cc-2901-11dd-713107879e6d",
       "rel" : "self"
      },
      {
       "href" : "http://10.10.120.158:8774/admin/storagevolumes/915212e9-f8cc-2901-11dd-713107879e6d",
       "rel" : "bookmark"
      } ],
    "name" : "nfspool"
   } ]
}


UI Changes

Should cover changes required to the UI, or specific UI that is required to implement this

Code Changes

Code changes should include an overview of what needs to change, and in some cases even the specific details.

Migration

n/a

Test/Demo Plan

This need not be added or completed until the specification is nearing beta.

Unresolved issues

This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.

BoF agenda and discussion

Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarizing what was discussed and note any options that were rejected.

Retrieved from "https://wiki.openstack.org/w/index.php?title=CloudInventoryManager&oldid=7547"