Jump to: navigation, search

Difference between revisions of "CloudInventoryManager"

 
(One intermediate revision by one other user not shown)
Line 1: Line 1:
__NOTOC__
+
{{ImplementedFeature}}
 +
 
 
* '''Launchpad Entry''': [https://blueprints.launchpad.net/openstack-devops/+spec/cloud-inventory-manager  Cloud Inventory Manager]
 
* '''Launchpad Entry''': [https://blueprints.launchpad.net/openstack-devops/+spec/cloud-inventory-manager  Cloud Inventory Manager]
 
* '''Created''':  02 April 2012
 
* '''Created''':  02 April 2012
 
* '''Contributors''': [https://launchpad.net/~yehia-beyh Yehia Beyh], [https://launchpad.net/~lyle-wilkinson Lyle Wilkinson], [https://launchpad.net/~divakar-padiyar-nandavar Divakar Nandavar]
 
* '''Contributors''': [https://launchpad.net/~yehia-beyh Yehia Beyh], [https://launchpad.net/~lyle-wilkinson Lyle Wilkinson], [https://launchpad.net/~divakar-padiyar-nandavar Divakar Nandavar]
 
<<[[TableOfContents]]()>>
 
  
 
== Summary ==
 
== Summary ==
Line 45: Line 44:
 
== Design ==
 
== Design ==
  
Cloud Inventory Manager, part of the health and monitor service architecture, supports a plug-in architecture similar to other [[OpenStack]] component implementations. The design supports multiple implementations to be plugged using the driver model that is being followed in [[OpenStack]] through configuration settings. It is extendable to support multiple Hypervisor types (example: ESX, Hyper-V, KVM, etc.).  
+
Cloud Inventory Manager, part of the health and monitor service architecture, supports a plug-in architecture similar to other OpenStack component implementations. The design supports multiple implementations to be plugged using the driver model that is being followed in [[OpenStack]] through configuration settings. It is extendable to support multiple Hypervisor types (example: ESX, Hyper-V, KVM, etc.).  
  
[[Image:CloudInventoryManager$HealthnmonArch.gif]]
+
[[Image:HealthnmonArch.gif]]
  
 
Cloud Inventory Manager provides:
 
Cloud Inventory Manager provides:
Line 58: Line 57:
 
* Inventory Data (Resource details) based on resource model (see Resource Model UML diagram below. Note that the Healthnmon XSD is the diagram above is derived from this Resource Model).
 
* Inventory Data (Resource details) based on resource model (see Resource Model UML diagram below. Note that the Healthnmon XSD is the diagram above is derived from this Resource Model).
  
[[Image:CloudInventoryManager$HnmResourceModel.gif]]
+
[[Image:HnmResourceModel.gif]]
  
 
== Implementation ==
 
== Implementation ==
Line 66: Line 65:
 
'''Architecture'''
 
'''Architecture'''
  
[[Image:CloudInventoryManager$cloudinventorymanager-bp-arch.gif]]
+
[[Image:cloudinventorymanager-bp-arch.gif]]
  
 
=== Configure nova to use healthnmon extensions ===
 
=== Configure nova to use healthnmon extensions ===

Latest revision as of 06:50, 12 December 2013

Warning.svg Old Design Page

This page was used to help design a feature that has been implemented. As a result, this page is unlikely to be updated and could contain outdated information. It was last updated on 2013-12-12

Summary

Provide an extensible service to Nova to include Cloud inventory for OpenStack cloud resources and infrastructure. Cloud Inventory Manager provides insights into cloud resource system information and takes advantage of underlying virtualization manager (e.g. libvirt) resource hierarchy such as Cluster, Resource pool, Storage, Network and Server pools.

End users and system administrators can use the resource inventory to build a catalog of available Clusters, Resource Pools, Storage pools, Network pools and Server pools.

  • Resource details provide resources discovered via virtualization managers, e.g. libvirt.
  • Resource information is persisted in the database. Details as needed are obtained via dynamic REST API query
  • Discovers and monitors resource changes (add/update/delete) caused by OpenStack or underlying resource managers
  • Explicitly models resources that can be allocated to elements of a Service, these Resources are then organized into Resource pools

Further, this inventory manager lays the groundwork for an improved scheduler and resource allocation implementation that will enable multi-tenant public and private cloud resource management.

Release Note

Cloud Inventory Manager service provides APIs based on the OpenStack extension APIs model. Hence, existing users should not be impacted. Advanced users and cloud providers can take advantage of the resource inventory data for analysis, planning and provisioning decisions.

See API descriptions for additional details.

Rationale

Note that the concept of Aggregates was introduced into Nova via the following blueprint:

The Cloud Inventory Manager service extends this concept from Host Aggregation to general other cloud resources (e.g. Cluster, Resource pool, Host, VM, Storage, Network, Subnets, VirtualSwitch, PortGroups). See Resource Model UML diagram below.

User stories

  • As a Cloud Administrator, I need support to build a catalog of available Cloud resources.
  • As a Cloud Administrator, I need to support Cloud Resource pool construction and maintenance based on capacities, quotas, and resource groups.
  • As a Cloud Administrator, I need to support viewing of resource inventory, resource consumption, utilization history and trends.
  • As a Cloud Administrator, I need to support importing of existing servers not yet managed by OpenStack.
  • As a Cloud User or Administrator, I need cloud resource metadata to support provisioning, optimization, and ongoing management of infrastructure services.
  • As a Cloud Administrator, I need to support analysis, manual modification, or autonomic modification of infrastructure services to improve their performance, resource usage, resource consumption, etc.

Assumptions

Design

Cloud Inventory Manager, part of the health and monitor service architecture, supports a plug-in architecture similar to other OpenStack component implementations. The design supports multiple implementations to be plugged using the driver model that is being followed in OpenStack through configuration settings. It is extendable to support multiple Hypervisor types (example: ESX, Hyper-V, KVM, etc.).

HealthnmonArch.gif

Cloud Inventory Manager provides:

  • Driver based plug-in model to support multiple implementations
    • plugged in using the driver model that is being followed in OpenStack through configuration settings.
  • Support for plugging in implementations for multiple Hypervisor types (example: ESX, Hyperv, KVM etc.,)
  • REST APIs for programmatic access as an Nova extension API.
    • Support Controller and Resource Extensions.
  • Inventory Data (Resource details) based on resource model (see Resource Model UML diagram below. Note that the Healthnmon XSD is the diagram above is derived from this Resource Model).

HnmResourceModel.gif

Implementation

Note: For discussion purpose the module name "Healthnmon" is used which implements the required functionality of Cloud Inventory details.

Architecture

Cloudinventorymanager-bp-arch.gif

Configure nova to use healthnmon extensions

Add the following to nova.conf file(/etc/nova/nova.conf) and restart nova-api server.

--osapi_compute_extension=healthnmon.api.healthnmon.Healthnmon


Sample Curl Usage

The following shows a sample usage to get list of vmhosts in xml format using the auth token created

curl -i -k  -H "X-Auth-Token:novaadmin:admin"  -H "Accept:application/xml" http://10.10.120.158:8774/v2/admin/vmhosts


Extension Metadata

Verb URI
GET v2/{project_id}/extensions/healthnmon

JSON Response

{
  "extension" : {
    "updated" : "2012-01-22T13:25:27-06:00",
    "name" : "healthnmon",
    "links" : [],
    "namespace" : "http://docs.openstack.org/ext/healthnmon/api/v2.0",
    "alias" : "HP-HNM",
    "description" : " Health and monitoring API as nova compute extension API's\n    "
  }
}


XML Response

<extension xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns="http://docs.openstack.org/compute/api/v1.1" alias="HP-HNM"
	updated="2012-01-22T13:25:27-06:00" namespace="http://docs.openstack.org/ext/healthnmon/api/v2.0"
	name="healthnmon">
	<description> Health and monitoring API as nova compute extension API's
	</description>
</extension>


Verb URI
GET /clusters?name=clusterName&marker=markerID&limit=int&changes-since=dateTime
GET /clusters/detail?name=clusterName&marker=markerID&limit=int&changes-since=dateTime
GET /clusters/id
Verb URI
GET /resourcepools?name=resourcepoolName&marker=markerID&limit=int&changes-since=dateTime
GET /resourcepools/detail?name=clusterName&marker=markerID&limit=int&changes-since=dateTime
GET /resourcepools/id
Verb URI
GET /hosts?type=hostType&name=hostName&status=hostStatus&marker=markerID&limit=int&changes-since=dateTime
GET /hosts/detail?type=hostType&name=hostName&status=hostStatus&marker=markerID&limit=int&changes-since=dateTime
GET /hosts/id
Verb URI
GET /subnets
GET /subnets/{id}
GET /virtualswitches
GET /virtualswitches/{id}
Verb URI
GET /storagevolumes
GET /storagevolumes/id
Verb URI
GET /virtualmachines
GET /virtualmachines/id

Normal Response Code(s): 200, 203 Error Response Code(s): computeFault (400), serviceUnavailable (503), unauthorized (401), forbidden (403), badRequest (400), badMethod (405), overLimit (413)

Get VmHost List

Verb URI
GET /vmhosts

Normal Response Code: 200
Error(s) Response Code: Compute Fault(400,500), Bad Request(400)

JSON Response

{
  "vmhosts" : [ {
    "id" : "34333832-3230-434e-3738-313630323650",
    "links" : [
        {
          "href" : "http://10.10.120.158:8774/v2/admin/vmhosts/34333832-3230-434e-3738-313630323650",
          "rel" : "self"
        },
        {
          "href" : "http://10.10.120.158:8774/admin/vmhosts/34333832-3230-434e-3738-313630323650",
          "rel" : "bookmark"
        } ],
    "name" : "ubuntukvm.vmm.hp.com"
  } ]
}

XML Response

<vmhosts xmlns="http://docs.openstack.org/ext/healthnmon/api/v2.0"
	xmlns:atom="http://www.w3.org/2005/Atom">
	<vmhost id="34333832-3230-434e-3738-313630323650" name="ubuntukvm.vmm.hp.com">
		<atom:link
			href="http://10.10.120.158:8774/v2/admin/vmhosts/34333832-3230-434e-3738-313630323650"
			rel="self" />
		<atom:link
			href="http://10.10.120.158:8774/admin/vmhosts/34333832-3230-434e-3738-313630323650"
			rel="bookmark" />
	</vmhost>
</vmhosts>


Get VmHost details

Verb URI
GET /vmhosts/id

Normal Response Code: 200
Error(s) Response Code: HTTP Not Found(404) -- if id is invalid or non-existent, Compute Fault(400,500), Bad Request(400)

XML Response

<VmHost>
	<id>34333832-3230-434e-3738-313630323650</id>
	<name>ubuntukvm.vmm.hp.com</name>
	<resourceManagerId>34333832-3230-434e-3738-313630323650
	</resourceManagerId>
	<memorySize>2054144</memorySize>
	<processorCount>2</processorCount>
	<processorCoresCount>2</processorCoresCount>
	<processorArchitecture>X86_64</processorArchitecture>
	<serialNumber>CN7816026P      </serialNumber>
	<memoryConsumed>2054144</memoryConsumed>
	<os>
		<osType>KVM</osType>
		<osDescription>KVM</osDescription>
		<osName>QEMU</osName>
		<osVersion>14001</osVersion>
	</os>
	<model>ProLiant BL465c G1  </model>
	<uuid>34333832-3230-434e-3738-313630323650</uuid>
	<virtualizationType>KVM</virtualizationType>
	<storagevolume xmlns:atom="http://www.w3.org/2005/Atom"
		id="78e3d6c0-698b-8a90-88e2-8043d6a82476">
		<atom:link
			href="http://10.10.120.158:8774/v2/admin/storagevolumes/78e3d6c0-698b-8a90-88e2-8043d6a82476"
			rel="self" />
		<atom:link
			href="http://10.10.120.158:8774/admin/storagevolumes/78e3d6c0-698b-8a90-88e2-8043d6a82476"
			rel="bookmark" />
	</storagevolume>
	<storagevolume xmlns:atom="http://www.w3.org/2005/Atom"
		id="915212e9-f8cc-2901-11dd-713107879e6d">
		<atom:link
			href="http://10.10.120.158:8774/v2/admin/storagevolumes/915212e9-f8cc-2901-11dd-713107879e6d"
			rel="self" />
		<atom:link
			href="http://10.10.120.158:8774/admin/storagevolumes/915212e9-f8cc-2901-11dd-713107879e6d"
			rel="bookmark" />
	</storagevolume>
	<storagevolume xmlns:atom="http://www.w3.org/2005/Atom"
		id="e52c1b6e-2128-48ce-65d7-38dec73a26fc">
		<atom:link
			href="http://10.10.120.158:8774/v2/admin/storagevolumes/e52c1b6e-2128-48ce-65d7-38dec73a26fc"
			rel="self" />
		<atom:link
			href="http://10.10.120.158:8774/admin/storagevolumes/e52c1b6e-2128-48ce-65d7-38dec73a26fc"
			rel="bookmark" />
	</storagevolume>
	<storagevolume xmlns:atom="http://www.w3.org/2005/Atom"
		id="d57ea5e3-c935-8c32-e883-4af4f4b66714">
		<atom:link
			href="http://10.10.120.158:8774/v2/admin/storagevolumes/d57ea5e3-c935-8c32-e883-4af4f4b66714"
			rel="self" />
		<atom:link
			href="http://10.10.120.158:8774/admin/storagevolumes/d57ea5e3-c935-8c32-e883-4af4f4b66714"
			rel="bookmark" />
	</storagevolume>
	<virtualmachine xmlns:atom="http://www.w3.org/2005/Atom"
		id="854f5cf4-233b-145f-cbc9-b558a7a65660">
		<atom:link
			href="http://10.10.120.158:8774/v2/admin/virtualmachines/854f5cf4-233b-145f-cbc9-b558a7a65660"
			rel="self" />
		<atom:link
			href="http://10.10.120.158:8774/admin/virtualmachines/854f5cf4-233b-145f-cbc9-b558a7a65660"
			rel="bookmark" />
	</virtualmachine>
	<virtualmachine xmlns:atom="http://www.w3.org/2005/Atom"
		id="93295e51-3dcc-eacf-5ce0-67982c2e8997">
		<atom:link
			href="http://10.10.120.158:8774/v2/admin/virtualmachines/93295e51-3dcc-eacf-5ce0-67982c2e8997"
			rel="self" />
		<atom:link
			href="http://10.10.120.158:8774/admin/virtualmachines/93295e51-3dcc-eacf-5ce0-67982c2e8997"
			rel="bookmark" />
	</virtualmachine>
</VmHost>

JSON Response

{
  "VmHost" : {
    "virtualizationType" : "KVM",
    "resourceManagerId" : "34333832-3230-434e-3738-313630323650",
    "storagevolume" : {
      "id" : "78e3d6c0-698b-8a90-88e2-8043d6a82476",
      "links" : [
          {
            "href" : "http://10.10.120.158:8774/v2/admin/storagevolumes/78e3d6c0-698b-8a90-88e2-8043d6a82476",
            "rel" : "self"
          },
          {
            "href" : "http://10.10.120.158:8774/admin/storagevolumes/78e3d6c0-698b-8a90-88e2-8043d6a82476",
            "rel" : "bookmark"
          } ]
    },
    "name" : "ubuntukvm.vmm.hp.com",
    "virtualmachine" : {
      "1" : {
        "id" : "93295e51-3dcc-eacf-5ce0-67982c2e8997",
        "links" : [
            {
              "href" : "http://10.10.120.158:8774/v2/admin/virtualmachines/93295e51-3dcc-eacf-5ce0-67982c2e8997",
              "rel" : "self"
            },
            {
              "href" : "http://10.10.120.158:8774/admin/virtualmachines/93295e51-3dcc-eacf-5ce0-67982c2e8997",
              "rel" : "bookmark"
            } ]
      },
      "id" : "854f5cf4-233b-145f-cbc9-b558a7a65660",
      "links" : [
          {
            "href" : "http://10.10.120.158:8774/v2/admin/virtualmachines/854f5cf4-233b-145f-cbc9-b558a7a65660",
            "rel" : "self"
          },
          {
            "href" : "http://10.10.120.158:8774/admin/virtualmachines/854f5cf4-233b-145f-cbc9-b558a7a65660",
            "rel" : "bookmark"
          } ]
    },
    "memoryConsumed" : "2054144",
    "serialNumber" : "CN7816026P      ",
    "memorySize" : "2054144",
    "processorCoresCount" : "2",
    "processorCount" : "2",
    "processorArchitecture" : "X86_64",
    "model" : "ProLiant BL465c G1  ",
    "os" : {
      "osType" : "KVM",
      "osDescription" : "KVM",
      "osVersion" : "14001",
      "osName" : "QEMU"
    },
    "id" : "34333832-3230-434e-3738-313630323650",
    "uuid" : "34333832-3230-434e-3738-313630323650"
  }
}


Storage extension api

  • /v1.1/storagevolumes - Returns list of all available storageVolumes.
  • /v1.1/storagevolumes/{id} - Returns storagevolume details of the specified 'id'.

Get Storage Volume List

Verb URI
GET /storagevolumes

Normal Response Code: 200
Error(s) Response Code: Compute Fault(400,500), Bad Request(400)

XML Response

<storagevolumes xmlns="http://docs.openstack.org/ext/healthnmon/api/v2.0"
	xmlns:atom="http://www.w3.org/2005/Atom">
	<storagevolume id="d57ea5e3-c935-8c32-e883-4af4f4b66714"
		name="iscsipool">
		<atom:link
			href="http://10.10.120.158:8774/v2/admin/storagevolumes/d57ea5e3-c935-8c32-e883-4af4f4b66714"
			rel="self" />
		<atom:link
			href="http://10.10.120.158:8774/admin/storagevolumes/d57ea5e3-c935-8c32-e883-4af4f4b66714"
			rel="bookmark" />
	</storagevolume>
	<storagevolume id="78e3d6c0-698b-8a90-88e2-8043d6a82476"
		name="default">
		<atom:link
			href="http://10.10.120.158:8774/v2/admin/storagevolumes/78e3d6c0-698b-8a90-88e2-8043d6a82476"
			rel="self" />
		<atom:link
			href="http://10.10.120.158:8774/admin/storagevolumes/78e3d6c0-698b-8a90-88e2-8043d6a82476"
			rel="bookmark" />
	</storagevolume>
	<storagevolume id="e52c1b6e-2128-48ce-65d7-38dec73a26fc"
		name="isopool">
		<atom:link
			href="http://10.10.120.158:8774/v2/admin/storagevolumes/e52c1b6e-2128-48ce-65d7-38dec73a26fc"
			rel="self" />
		<atom:link
			href="http://10.10.120.158:8774/admin/storagevolumes/e52c1b6e-2128-48ce-65d7-38dec73a26fc"
			rel="bookmark" />
	</storagevolume>
	<storagevolume id="915212e9-f8cc-2901-11dd-713107879e6d"
		name="nfspool">
		<atom:link
			href="http://10.10.120.158:8774/v2/admin/storagevolumes/915212e9-f8cc-2901-11dd-713107879e6d"
			rel="self" />
		<atom:link
			href="http://10.10.120.158:8774/admin/storagevolumes/915212e9-f8cc-2901-11dd-713107879e6d"
			rel="bookmark" />
	</storagevolume>
</storagevolumes>


JSON Response

{
 "storagevolumes" : [
   {
    "id" : "d57ea5e3-c935-8c32-e883-4af4f4b66714",
    "links" : [
      {
       "href" : "http://10.10.120.158:8774/v2/admin/storagevolumes/d57ea5e3-c935-8c32-e883-4af4f4b66714",
       "rel" : "self"
      },
      {
       "href" : "http://10.10.120.158:8774/admin/storagevolumes/d57ea5e3-c935-8c32-e883-4af4f4b66714",
       "rel" : "bookmark"
      } ],
    "name" : "iscsipool"
   },
   {
    "id" : "78e3d6c0-698b-8a90-88e2-8043d6a82476",
    "links" : [
      {
       "href" : "http://10.10.120.158:8774/v2/admin/storagevolumes/78e3d6c0-698b-8a90-88e2-8043d6a82476",
       "rel" : "self"
      },
      {
       "href" : "http://10.10.120.158:8774/admin/storagevolumes/78e3d6c0-698b-8a90-88e2-8043d6a82476",
       "rel" : "bookmark"
      } ],
    "name" : "default"
   },
   {
    "id" : "e52c1b6e-2128-48ce-65d7-38dec73a26fc",
    "links" : [
      {
       "href" : "http://10.10.120.158:8774/v2/admin/storagevolumes/e52c1b6e-2128-48ce-65d7-38dec73a26fc",
       "rel" : "self"
      },
      {
       "href" : "http://10.10.120.158:8774/admin/storagevolumes/e52c1b6e-2128-48ce-65d7-38dec73a26fc",
       "rel" : "bookmark"
      } ],
    "name" : "isopool"
   },
   {
    "id" : "915212e9-f8cc-2901-11dd-713107879e6d",
    "links" : [
      {
       "href" : "http://10.10.120.158:8774/v2/admin/storagevolumes/915212e9-f8cc-2901-11dd-713107879e6d",
       "rel" : "self"
      },
      {
       "href" : "http://10.10.120.158:8774/admin/storagevolumes/915212e9-f8cc-2901-11dd-713107879e6d",
       "rel" : "bookmark"
      } ],
    "name" : "nfspool"
   } ]
}


UI Changes

Should cover changes required to the UI, or specific UI that is required to implement this

Code Changes

Code changes should include an overview of what needs to change, and in some cases even the specific details.

Migration

n/a

Test/Demo Plan

This need not be added or completed until the specification is nearing beta.

Unresolved issues

This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.

BoF agenda and discussion

Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarizing what was discussed and note any options that were rejected.

http://etherpad.openstack.org/Cloud-Inventory-Manager