Scheduling based on Instance Resource Quota

draft

User Storie

An operator wants to create tiny instances that consume only 1/5 of physical CPU frequency.

Overview

Nova provides the ability to set CPU time consumption percent for instances: https://wiki.openstack.org/wiki/InstanceResourceQuota.

For instance, setting Quota/Period with a ratio of 1/5 will result in instances using 1/5 of CPU time. We need to find a way for the scheduler to take this into account during scheduling.

Drivers	Supported	Future
Libvirt	YES	YES
XenApi	NO	N/A
VmWare	NO	N/A
HyperV	NO	N/A

Currently Supported

Isolate hosts for specific flavor

A first solution could be to reserve some specific hosts for a particular flavor and then configuring cpu allocation with the desired ratio.

How to use it

Context: For a flavor with a resource limited to 1/5 cpu

Configure the scheduler with filters 'AggregateCoreFilter' and 'AggregateInstanceExtraSpecFilter'
Create a flavor and set the 'aggregate_instance_extra_specs:flavor_type' metadata to 'tiny_flavor' (or another value)
Create an aggregate to isolate a resource limited flavor and set the 'cpu_allocation_ratio' metadata of this aggregate to 5.
Set the 'flavor_type' metadata of this aggregate to 'tiny_flavor'

Problems

Dedicated hosts for flavor: host provisioning issue.
Impact of important number of VMs per host: workload density (to be studied with Neutron/Nova).

Proposal

New metric `cpu_used`

We propose to add the ability to Nova scheduler to take Instance CPU Resource Quota into account. The proposal consists of adding a new metric into the nova database that counts used CPU time instead of number of used VCPU as current. As well the filter CoreFilter need to be modify to take care of this new metric or a new filter CpuUsageFilter which counts on this metric could be presented

   cpu_used = cpu_used + nvcpus * cpu_ratio, with cpu_ratio equal to Quota/Period.

How it works

For instance, let suppose that the host is free of VM, then the `cpu_used` is 0.

An user creates an instance of 1 VCPU with Quota/Period ratio of 1/5 (equivalent to 20% of physical CPU requested), the `cpu_used` is set to:

   0 + 1 * 1/5 = 0.2

an user creates an instance of 2 VCPU with Quota/Period ratio of 1/2 (equivalent to 2* 50% =100 of physical CPU time requested), the `cpu_used` is set to:

   0.2  + 2 * 1/2 = 1.2

Scheduling

The filter that takes care of this metric will count on the number of available CPU time to determine of the host can accept the request or not.

     <Available CPU time> = <Numer of cores> * cpu_used_ratio - cpu_used

in which `cpu_used_ratio` is the oversubscription ratio. cpu_used_ratio can be set in metadata of the aggregate or in nova.conf for default value. The host accepts the request if and only if the available CPU time is more than requested.

Important impacts

New field in the host_state model.
Update this cpu_used metric whenever a VM is created, deleted or migrated

Nova/SchedulingBasedOnInstanceResourceQuota

Contents