WholeHostAllocation

• Launchpad Entry: NovaSpec:whole-host-allocation
• Created: 6th May 2013
• Contributors: Phil Day, HP Cloud Services

Summary

Allow a tenant to allocate all of the capacity of a host for their exclusive use. The host remains part of the Nova configuration, i.e this is different from bare metal provisioning in that the tenant is not getting access to the Host OS - just a dedicated pool of compute capacity. This gives the tenant guaranteed isolation for their instances, at the premium of paying for a whole host.

Extending this further in the future could form the basis of hosted private clouds – i.e. schematics of having a private could without the operational overhead.

Use Cases

The required features are explored by stepping through the main use cases:

1. Tenant allocates a host: 1.1 Make the request to the API.

Q) What do they need to be able to specify ? Required size characteristics. Flavors would seem to be the natural thing to use here. Maybe we extend the flavor definition to include a flavor type (instance, host, bare metal). Note that Bare Metal also uses flavors in this way, adding a type would allow combined BM and non BM systems.

Q) Should this have to be an exact match (Mem,Disk,CPU), a minimum, or should we allow the cloud provider to tag the hosts in some way to match the flavors ?

Pool name: The tenant might want to have several pools of servers. Behind the scenes the system would create a host aggregate (project_id-pool_name). Having a default pool ensures at least one aggregate, and allow us to use aggregates as a way to manage the hosts allocated to a tenant. This saves us having to expose the aggregate API directly to tenants (Don’t want to do this as aggregates work on host names, don’t’ have an owner, etc)

Availability Zone: Tenant should be able to build pools of hosts that span AZs.

Min/max count: Tenant should be able to request multiple hosts in a single call

Don’t need image, Security Group, key name, metadata, nics, config_drive as these only apply to an instance.

Q) Should this be a new API call, or overloaded onto “server create” as with Bare Metal Provisioning ? Don’t really want to return an instance, and the parameter overlap with server create is pretty limited, so think it should be a new API

1.2) System validates request against the Quotas: Need a new set of quotas to cover hosts: #hosts, #cpus, memory(GB), disk(GB)

1.3) System allocates a free host, and adds it to the aggregate corresponding to the specified pool. Q) Do we need a scheduler for this, or can we do it in the API ? There is no need to interact with the compute manager, and no complex sequence of state changes (the host is already in the required state), so no real need for a separate scheduler.

Q) Do we allocate from the general pool of hosts, or does there need to be a way to create a reserved pool ? Can see a case for either, depending on how the cloud provider wants to manage their capacity. So provide configuration option to identify the host aggregate that hosts are allocated from. Add an aggregate filter to the main scheduler so that it can also be limited to allocating from specific aggregates. These aggregates used for instance creation and whole host allocation can then overlap or be disjoint.

Q) How does the system record that the host allocated ? Just transferring it into the tenant’s aggregate is enough – providing we make this an atomic operation when moving it from the "allocation aggregate". Otherwise we would need to do something like adding a project_id to the compute_hosts table.

Q) What do we return to the tenant ? Don’t really want to return host names, as these should be private to the cloud provider. Can't use the same hash of hostname and tenant that are used in server details, as we need to be able to extract the hostname for deallocation (see 3). So maybe we return id of the compute_nodes entry. Would be better if this was a uuid rather than an id.

1.4) System generates a notification event for the billing system Need a new class of even to represent hosts. Also need periodic “exists” event (billing systems like regular confirmation for self correction in case they miss an event)

Q) What else does the billing system need to be able to charge differently for instances on hosts allocated to tenants (the tenant is already paying for the host)? Add a list of aggregates the host belongs to in the instance notification events ?

2) Tenant creates an instance on one of their allocated hosts Since we’re using host aggregates to manage the hosts, this just requires a scheduler hint that can be used by a filter to identify the pool and translate into the aggregate name. Tenants can still also create instances on the general pool of hosts by not giving the scheduler hint.

3) Tenant deallocates a host System moves the host out of the project specific aggregate and back into the aggregate that it is using for whole host scheduling If there are instances running on the host then the API should refuse to de-allocate it.

Extended use cases (maybe future additional Blue Prints ?)

4) Tenant defines their own flavor(s) to be used within their pool(s) One of the use cases for having dedicated hosts is that the tenant wants to be able to use the full capacity of a host, or to create “off size” flavours that that Cloud Provider doesn’t want to have generally available (as they become hard to build pricing plans around).

There is a flavoraccess API extension, but looks like this just allows an admin to define access policies on flavours, not give control of specific flavours at the tenant level. Also not clear if the access polict is actually validated during instance creation

Maybe need a new TenantFlavor extension which creates the flavors as private to the calling tenant and adds the required extra_data to limit them to the aggregates owned by this tenant ?

5) Tenant can define scheduling policy for their hosts For example they may want to be able to specify a different rate of over provisioning, or maybe even a different set of filters.