Nova/SchedulingBasedOnInstanceResourceQuota

= Scheduling based on Instance Resource Quota = draft

User Storie
An operator wants to create tiny instances that consume only 1/5 of physical CPU frequency.

Overview
Nova provides the ability to set CPU time consumption percent for instances: https://wiki.openstack.org/wiki/InstanceResourceQuota.

For instance, setting Quota/Period with a ratio of 1/5 will result in instances using 1/5 of CPU time. We need to find a way for the scheduler to take this into account during scheduling.

Isolate hosts for specific flavor
A first solution could be to reserve some specific hosts for a particular flavor and then configuring cpu allocation with the desired ratio.

How to use it
Context: For a flavor with a resource limited to 1/5 cpu
 * 1) Configure the scheduler with filters 'AggregateCoreFilter' and 'AggregateInstanceExtraSpecFilter'
 * 2) Create a flavor and set the 'aggregate_instance_extra_specs:flavor_type' metadata to 'tiny_flavor' (or another value)
 * 3) Create an aggregate to isolate a resource limited flavor and set the 'cpu_allocation_ratio' metadata of this aggregate to 5.
 * 4) Set the 'flavor_type' metadata of this aggregate to 'tiny_flavor'

Problems

 * Dedicated hosts for flavor: host provisioning issue.
 * Impact of important number of VMs per host: workload density (to be studied with Neutron/Nova).

New metric `cpu_used`
We propose to add the ability to Nova scheduler to take Instance CPU Resource Quota into account. The proposal consists of adding a new metric into the nova database that counts used CPU time instead of number of used VCPU as current. As well the filter CoreFilter need to be modify to take care of this new metric or a new filter CpuUsageFilter which counts on this metric could be presented

cpu_used = cpu_used + nvcpus * cpu_ratio, with cpu_ratio equal to Quota/Period.

How it works
For instance, let suppose that the host is free of VM, then the `cpu_used` is 0.

0 + 1 * 1/5 = 0.2   0.2  + 2 * 1/2 = 1.2
 * 1) An user creates an instance of 1 VCPU with Quota/Period ratio of 1/5 (equivalent to 20% of physical CPU requested), the `cpu_used` is set to:
 * 1) an user creates an instance of 2 VCPU with Quota/Period ratio of 1/2 (equivalent to 2* 50% =100 of physical CPU time requested), the `cpu_used` is set to:

Scheduling
The filter that takes care of this metric will count on the number of available CPU time to determine of the host can accept the request or not.  =  * cpu_used_ratio - cpu_used

in which `cpu_used_ratio` is the oversubscription ratio. cpu_used_ratio can be set in metadata of the aggregate or in nova.conf for default value. The host accepts the request if and only if the available CPU time is more than requested.

Important impacts

 * New field in the host_state model.
 * Update this cpu_used metric whenever a VM is created, deleted or migrated