Jump to: navigation, search

Difference between revisions of "WholeHostAllocation"

(→‎Summary)
(→‎Summary)
Line 14: Line 14:
  
  
−
=== Supported Operations ===
+
=== Supported Operations ===
  
 
The Cloud Administrator can:
 
The Cloud Administrator can:
Line 30: Line 30:
 
* See the list of hosts and instances running in their Pcloud
 
* See the list of hosts and instances running in their Pcloud
  
−
A tenant that has been authorized by a Pcloud can:
+
A tenant that has been authorized by a Pcloud can:
 
* Schedule an instance to run in the Pcloud.  Scheduling is controlled by an additional filter, so all other filters (Availability Zones, affinity, etc) are still valid   
 
* Schedule an instance to run in the Pcloud.  Scheduling is controlled by an additional filter, so all other filters (Availability Zones, affinity, etc) are still valid   
  
  
 +
=== Host Flavors ===
  
 +
Host flavors define the types of hosts that users can allocate to their Pclouds (c.f. flavors for instances).  Because physical host have fixed characteristics a host flavor is defined by just three properties:
 +
* An unique Identifier
 +
* A description (which can include details of the number of cpus memory, etc)
 +
* A Units value which is used to control the quota cost of a host
 +
 +
Host Flavors are implemented as aggregates with the name "Pcloud:host_flavor:<id>", and have the following metadata values:
 +
 +
{| class="wikitable"
 +
|-
 +
! Key !! Value !! Notes
 +
|-
 +
| pcloud:type || 'host_flavor_pool' || Identifies this as a host flavor aggregate
 +
|-
 +
| pcloud:flavor_id || ''flavor_id'' ||
 +
|-
 +
| pcloud:description || ''description'' || A description of the host type 
 +
|-
 +
| pcloud:units || ''units'' || The number of quota units consumed by a host of this type
 +
|}
 +
 +
A host must be empty and its compute service disabled before it can be added to a host flavor pool.  Adding it to the host flavor pool will enable the compute-service (the Pcloud scheduler filter will stop anything from being schedule to it until it is allocated to a Pcloud).  Hosts remain in this aggregate even when they are allocated to a Pcloud - membership of the host flavor aggregate defines the the host flavor of the host - so there is no need to keep a separate mapping of hosts to host types.   
 +
 +
 +
=== Pclouds ===
 +
 +
A Pcloud is in effect a user defined and managed aggregate.  However the Pclouds API provides an abstraction layer to control ownership and provide authorization schematics, and encapsulate system system properties such as host names.  On creation a Pcloud is assigned a uuid, which forms part of the aggregate name and provides the Pcloud ID for users to use in the API.
 +
 +
Pclouds are implemented as aggregates with the name "Pcloud:<uuid>", and have the following metadata values on creation:
 +
 +
{| class="wikitable"
 +
|-
 +
! Key !! Value !! Notes
 +
|-
 +
| pcloud:type || 'pcloud' || Identifies this as a pcloud aggregate
 +
|-
 +
| pcloud:id || ''uuid'' || The generated uuid for this Pcloud
 +
|-
 +
| pcloud:name || ''name'' || The user assigned name for this Pcloud 
 +
|-
 +
| pcloud:owner || ''tenant_id'' || The tenant_id that created the Pcloud.  Used for authorisation and billing 
 +
|}
 +
 +
 +
==== Host Operations ==== 
 +
 
 +
A user adds a host by specifying a name (which must be unique within the pcloud) and the required host flavor type and optionally an availability zone.    The Plcoud API finds a free host in the appropriate host flavor aggregate and adds it to the Pcloud aggregate.  Note that because this is a simple allocation mechanism there is no need to call to the scheduler, there is either a free host in the host flavor aggregate or there isn't.
 +
 +
In adding the host to the Pcloud aggregate the following additional metadata values are also added   
 +
 +
{| class="wikitable"
 +
|-
 +
! Key !! Value !! Notes
 +
|-
 +
| pcloud:host:<name> || ''hypervisor_hostname'' || Provides the mapping from the users name to the real hostname
 +
|-
 +
| pcloud:host_state:<hostname> || ''disabled'' || Controls whether a host can be used for scheduling.  Hosts are always disabled when added
 +
|}
 +
 +
 +
Because the allocation occurs in the API sever, there is a risk of a race condition when two allocation requests are processed at the same time.  The Pcloud API provide a protection  against this by:
 +
* Host are selected randomly from the list of available hosts.
 +
* After the host has been added to the Pcloud aggregate a further check is made to see how many Pcloud aggregates the host is a member of.  If it is a member of more than one Pcloud aggregate then it is removed from the Pcloud and the allocation re-tried (up to a configurable number of attempts).  In this way if more that one Pcloud allocates the same host one or all of them will release it and retry.   
 +
 +
In order to remove a host from a Pcloud it must be empty (so that it can be used by another Pcloud) and in order to empty it the Pcloud owner must be able to prevent further instances from being scheduled to it.  Pclouds therefore provide a simple enable / disable mechanism for hosts, which is recorded in the metadata and implemented by the Pcloud filter.  The underlying service enable / disable is not used so that this remains available to the System administrators.
 +
 +
A host must be both empty and disabled before it can be removed fro a Pcloud. 
 +
 +
The Pcloud Owner can see a list of hosts, their states, and the uuid's of instances on those hosts
 +
 +
 +
==== Tenant Operations ====
 +
 +
The owner of a Pcloud can authorize other tenants to be able to schedule to the Pcloud.  Authorizing a tenant adds the following record to the aggregate metadata:
 +
 +
{| class="wikitable"
 +
|-
 +
! Key !! Value !! Notes
 +
|-
 +
| pcloud:<tenant_id> || 'pcloud-tenant' || Indicates that <tenant_id> can schedule to this Pcloud
 +
|}
 +
 +
Note that to avoid being limited by the metadata record size a separate metadata value is created for each authorized tenant
 +
 +
 +
==== Scheduler Configuration ====
 +
 +
Where a scheduler filter can be configured by aggregate properties its possible to provide that configuration on a per Pcloud basis.
 +
Currently the Core Filter and Ram Filter support per Aggregate configuration, and so the Pcloud API provides the capability to set the appropriate values on the underlying aggregate
 +
   
  
 
The proposal splits into three parts:
 
The proposal splits into three parts:

Revision as of 23:35, 25 October 2013

  • Launchpad Entry: NovaSpec:whole-host-allocation
  • Created: 6th May 2013
  • Contributors: Phil Day, HP Cloud Services

Summary

Pclouds (PrivateClouds) allow a tenant to allocate all of the capacity of a host for their exclusive use. The host remains part of the Nova configuration, i.e this is different from bare metal provisioning in that the tenant is not getting access to the Host OS - just a dedicated pool of compute capacity. This gives the tenant guaranteed isolation for their instances, at the premium of paying for a whole host.

The Pcloud mechanism provides an abstraction between the user and the underlying physical servers. A user can uniquely name and control servers in the Pcloud, but then never get to know the actual host names or perform admin commands on those servers.

The initial implementation provides all of the operations need create and manage a pool of servers, and to schedule instances to them. Extending this further in the future could allow additional operations such as specific scheduling policies, control over which flavors and images can be used, etc.

Pclouds are implemented as an abstraction layer on top of Host Aggregates. Each host flavor pool and each Pcloud is an aggregate. Properties of the Host Flavors and Pclouds are stored as aggregate metadata. Hosts in a host flavor pool are not available for general scheduling. Host are allocated to Pclouds by adding them into the Pcloud aggregate. (they may also be in other types of aggregates as well, such as an AZ). All of the manipulation of aggregates including host allocation is performed by the Pcloud API layer - users never have direct access to the aggregates, aggregate metadata, or hosts. (Admins of course can still see and manipulate the aggregates)


Supported Operations

The Cloud Administrator can:

  • Define the types of server that can be allocated (host-flavors)
  • Add or remove physical servers from a host-flavor pool
  • See details of a host flavor pool

A User can:

  • Create a Pcloud
  • See the list of available host flavors
  • Allocate hosts of a particular host-flavor and availability zone to their Pcloud
  • Enable or disable hosts in their Pcloud
  • Authorize other tenants to be able to schedule to their Pcloud
  • Set the ram and cpu allocation ratios used by the scheduler within their Pcloud
  • See the list of hosts and instances running in their Pcloud

A tenant that has been authorized by a Pcloud can:

  • Schedule an instance to run in the Pcloud. Scheduling is controlled by an additional filter, so all other filters (Availability Zones, affinity, etc) are still valid


Host Flavors

Host flavors define the types of hosts that users can allocate to their Pclouds (c.f. flavors for instances). Because physical host have fixed characteristics a host flavor is defined by just three properties:

  • An unique Identifier
  • A description (which can include details of the number of cpus memory, etc)
  • A Units value which is used to control the quota cost of a host

Host Flavors are implemented as aggregates with the name "Pcloud:host_flavor:<id>", and have the following metadata values:

Key Value Notes
pcloud:type 'host_flavor_pool' Identifies this as a host flavor aggregate
pcloud:flavor_id flavor_id
pcloud:description description A description of the host type
pcloud:units units The number of quota units consumed by a host of this type

A host must be empty and its compute service disabled before it can be added to a host flavor pool. Adding it to the host flavor pool will enable the compute-service (the Pcloud scheduler filter will stop anything from being schedule to it until it is allocated to a Pcloud). Hosts remain in this aggregate even when they are allocated to a Pcloud - membership of the host flavor aggregate defines the the host flavor of the host - so there is no need to keep a separate mapping of hosts to host types.


Pclouds

A Pcloud is in effect a user defined and managed aggregate. However the Pclouds API provides an abstraction layer to control ownership and provide authorization schematics, and encapsulate system system properties such as host names. On creation a Pcloud is assigned a uuid, which forms part of the aggregate name and provides the Pcloud ID for users to use in the API.

Pclouds are implemented as aggregates with the name "Pcloud:<uuid>", and have the following metadata values on creation:

Key Value Notes
pcloud:type 'pcloud' Identifies this as a pcloud aggregate
pcloud:id uuid The generated uuid for this Pcloud
pcloud:name name The user assigned name for this Pcloud
pcloud:owner tenant_id The tenant_id that created the Pcloud. Used for authorisation and billing


Host Operations

A user adds a host by specifying a name (which must be unique within the pcloud) and the required host flavor type and optionally an availability zone. The Plcoud API finds a free host in the appropriate host flavor aggregate and adds it to the Pcloud aggregate. Note that because this is a simple allocation mechanism there is no need to call to the scheduler, there is either a free host in the host flavor aggregate or there isn't.

In adding the host to the Pcloud aggregate the following additional metadata values are also added

Key Value Notes
pcloud:host:<name> hypervisor_hostname Provides the mapping from the users name to the real hostname
pcloud:host_state:<hostname> disabled Controls whether a host can be used for scheduling. Hosts are always disabled when added


Because the allocation occurs in the API sever, there is a risk of a race condition when two allocation requests are processed at the same time. The Pcloud API provide a protection against this by:

  • Host are selected randomly from the list of available hosts.
  • After the host has been added to the Pcloud aggregate a further check is made to see how many Pcloud aggregates the host is a member of. If it is a member of more than one Pcloud aggregate then it is removed from the Pcloud and the allocation re-tried (up to a configurable number of attempts). In this way if more that one Pcloud allocates the same host one or all of them will release it and retry.

In order to remove a host from a Pcloud it must be empty (so that it can be used by another Pcloud) and in order to empty it the Pcloud owner must be able to prevent further instances from being scheduled to it. Pclouds therefore provide a simple enable / disable mechanism for hosts, which is recorded in the metadata and implemented by the Pcloud filter. The underlying service enable / disable is not used so that this remains available to the System administrators.

A host must be both empty and disabled before it can be removed fro a Pcloud.

The Pcloud Owner can see a list of hosts, their states, and the uuid's of instances on those hosts


Tenant Operations

The owner of a Pcloud can authorize other tenants to be able to schedule to the Pcloud. Authorizing a tenant adds the following record to the aggregate metadata:

Key Value Notes
pcloud:<tenant_id> 'pcloud-tenant' Indicates that <tenant_id> can schedule to this Pcloud

Note that to avoid being limited by the metadata record size a separate metadata value is created for each authorized tenant


Scheduler Configuration

Where a scheduler filter can be configured by aggregate properties its possible to provide that configuration on a per Pcloud basis. Currently the Core Filter and Ram Filter support per Aggregate configuration, and so the Pcloud API provides the capability to set the appropriate values on the underlying aggregate


The proposal splits into three parts:

  1. Allowing the user to define and manage a pool of servers
  2. Adding / Removing servers to that pool, and generating the associated notification messages for billing
  3. Allowing the user to schedule instances to a specific pool that they own or have rights to use

Defining and managing pools of servers

The basic mechanism for grouping servers into pools with associated metadata already exists in the form of Host aggregates, and there are already various parts of the code that rely on specific metadata values for things like scheduling schematics. However host aggregates were never designed or intended to be directly use facing:

  • There is no concept of an aggregate owner
  • They have a simple ID (rather than a uuid)
  • Metadata key values are effectively internal details of Nova
  • There are no quotas associated with aggregates

So instead of changing aggregates to be user facing we will provide a wrapper layer that creates aggregrates in a controlled way with a limited set of pre-defined metadata. We call these "wrapped" aggregates Pclouds (Private Clouds)

A user can perform the following opertaions on a Pcloud:

Create a Pcloud: User specifies a descriptive name of the Pcloud. The system will validate the request against quotas and create a host aggregate with the following properties:

  • The aggregate name is set to a generated uuid (This is what the user will use to identify the Pcloud in subsequent calls)
  • The user supplied name is stored as a metadata value of the aggregate
  • The project_id of the owner is stored as a metadata value of the aggregate


Add a project to a Pcloud: Pclouds are owned by a single project for quota, billing, and mgmt purposes. But the owner may want to allow other projects to use the resources within a Pcloud (a common use case in enterprises). So the owner of a Pcloud is allowed to add project_Ids as tenants of the Pcloud, which are then recorded in the aggregate metadata. Because metadata keys have to be unique, and metadata values are limited in length, project access rights are recorded as a metadata item per project using the project ID as the key. Note that owner of a Pcloud is automatically added as a tenant and creation time, but they can remove themselves if required.


Remove a project from a Pcloud: Simply deletes the metadata key for that project

See here for examples using nova client Pcloud Examples

Change the properties of a Pcloud:

(Future): Provides a controlled mechanism to update other aggregate metadata values that affect scheduling - for example providing a different value of the CPU or Memory over commit


List Pclouds: List all Pclouds that are available to a project. This includes Pclouds that the project has access rights to (so Projects can "discover" which Plcouds they can use without having to ask the Pcloud owner). Both the UUID and Name are shown in the list, along with the role (Owner or Tenant). If the project is the Owner of a Pcloud then they can also see the number of hosts in the Pcloud


Show Pcloud Details: Only available to the owner of the Pcloud, show details such as the list of projects allowed access, list of hosts, and instances running on those hosts. Hosts will be presented as a hash of hostname and project_id (as in the server details)


Delete a Pcloud: Remove a Pcloud. Can only be performed when there are no hosts associated with the Pcloud.

Adding / Removing Servers

There are a number of different ways that this could be specified and implemented, but to start with we propose to keep to a very simple definition and mechanism:


  • There will be a pool of prebuild but disabled servers in a specific aggregate (the Pcloud_free_pool aggregate) that are used exclusively for whole host allocation. (Don't want the complexity at this stage of interacting with the scheduler to find a suitable host with no VMs on it)
  • The types of hosts available (#cpus, memory. disk, capabilities)will automatically derived from this pool (this avoids having to create a "hosts flavour conceprt at this time, and keeps the code Pcloud code as a layer on top of Nova). Expect this to change in the future when we have an agreed model for deeper integration - for example we might want to introuce a "flavour type" attribute into the flavours table to allow both host and instance flavours).
  • A user will be able to request that a host of a particular host flavour and from a particular AZ is added to a Pcloud. The system will look for a suitable host from the pcloud_free_pool move it to the aggregate behind the Pcloud, and enable it.
  • A user will be able to request that a specific host (identified by its hash) is removed from a Pcloud. Providing there are no instances on the host, the system will disable the service, and move the host back to the pcloud_free_pool


In addition to notification messages that are generated on allocation and deallocation of a host, there also needs to be the equivalent of a instance.exists message (this is typically needed by billing systems for example to ensure consistency). This could be generated by the periodic task in the compute manager that currently generates the instance.exists messages. It may also be necessary to extend the instance.exists messages to include the uuid of Pcloud the host is part of.

Schedule instances to a specific pool

Users should be able to specify whether they want to schedule to a specific Pcloud or the general resource pool (I.e they are not constrained to one or the other). Neither of the Current Scheduler isolation filters provide this. The user can specify teh uuid of a Pcloud to use via scheduler hints

The basic flow of the Pcloud filter is as follows: 

   If a pcloud is specified in scheduler hints
        if the host is not part of the specified Pcloud:
             Fail
        else if the user does not has access to Pcloud:
            Fail
        else
            Pass
   else if the host is part of a Pcloud
       Fail
   else
       Pass
Retrieved from "https://wiki.openstack.org/w/index.php?title=WholeHostAllocation&oldid=34067"