Difference between revisions of "CPUEntitlement"

Revision as of 06:52, 18 September 2013

• Launchpad Entry: NovaSpec:cpu-entitlement
• Created: 8 Jan 2013
• Contributors: Oshrit Feder

Summary

Extending Openstack Nova resource model to support differentiated CPU reservation

(entitlement) for vCPUs of different VM instance types

Release Note

Administrators enablement to guarantee cpu reservation for VMs

Rationale

Cloud users are provided with the flexibility to order instances from various instance types offering based on their computing needs and are charged accordingly. The vcpu resource indicates the amount of compute resources the VM is assigned with, but as the hardware and its physical CPU resource are shared between all the VMs, a VM is not guaranteed any minimal CPU reservation. Actually, the amount of time a VM is scheduled on a CPU vary over time and is depended on the underneath hypervisor scheduling algorithm implementation, other VMs CPU consumption demand, overcommit ratio, host's CPU strength, etc. Some hypervisors expose CPU reservation features (IBM PowerVM, ESX, Xen), while in others it can be enforced using other tools (e.g. cgroups in KVM). We propose a unified API to support differentiated CPU reservation (entitlement) for vCPUs of different VM instance types.

Related issue when using commodity hardware is that the different compute nodes that host VMs may be of heterogeneous compute performance. In such a case, the CPU capacity a VM receives (or reserved with) depends on the compute node it was scheduled on, and may be different in case of a migration to a different compute node. We propose a way for an administrator to define the compute capability for some host, as a way to weight a compute node relatively to the other compute nodes and to ensure a consistent amount of CPU capacity no matter what the actual underlying hardware is.

Another feature to consider with CPU reservation is quota. It may be desired for administrators to control user's "CPU entitlement quota" – just as with # of instances quota and # of vcpus quota, cpu entitlement quota puts an upper limit to the user's amount of pcpu consumed by his instances.

User stories

• Administrator wants to guarantee cpu reservation for VMs

Assumptions

Design

As for the day this blueprint was written, KVM is the common hypervisor used with Openstack. Administrators can control the VM's CPU resource in a limited way, by assigning it vcpus, setting overall CPU overcommit level and use core_filter to enforce it. The hypervisor's scheduler schedules all the VMs on the pcpu according to the schduler's algorithm. When fully over-committed and all provisioned VMs request CPU, a fair scheduler will provide each VM with 1/overcommit_level of CPU. To assign each VM with CPU reservation, we suggest expending Instances types with compute_units_per_vcpu that determines the cpu_entitlement level and defines how much of a core should be reserved. An Entitlement_filter will deprecate the existing Core_filter and enforce the sum of compute_units_per_vcpu on a single compute_node to be less than the total calculated node's compute_units_per_core. In Nova Compute each platform will translate the compute_unit into specific hypervisor's internals; e.g. shares in KVM.

Inspired from EC2, adding a compute_unit_per_core to each of Nova compute will provide administrators with a notion to differentiate between nodes' CPU capacity. Node's compute_unit_per_core specifies the node's CPU capacity relatively to a default CPU capacity.

Implementation

We suggest exposing this capability with simple, yet flexible approach:

• In Nova.conf administrators will be allowed to defined the various VM's compute_unit_per_vcpu, in the form compute_unit_per_vcpu = high:1.0, normal:0.5, low:0.25, default:0.25. In the above example, high indicates 1 core (100% of pcpu) cpu entitlement, for each of the VM's vcpu. In case the VM has 2 vcpu, it's total cpu entitlement = 2 cores (200%). Default indicates the default VM cpu entitlement in case the instance type wasn’t specified with any.
• In the extra_spec field in Instance_Type, administrators will specify one of the predefined compute_unit_per_vcpu, to set per VM cpu entitlement.
• To enforce cpu_entitlement, scheduler_default_filters should include entitlement_filter
• To differentiate between the various compute node's cpu capacity, administrators can set each compute node with compute_unit_per_core (0.3, 0.5, etc) which indicates their relative capacity in relation to a default cpu capacity.

Code Changes

Affected components: Nova.conf, Instance_type, Nova_Compute: host capacity, drivers, Scheduler: entitlement_filter

Related entries

https://blueprints.launchpad.net/nova/+spec/quota-instance-resource

Quota-instance-resource entry deals with extending Nova's InstaceType with "CPU control params". This blueprint is complementary to it, as it uses the same mechanism to expose VM CPU reservation.

Test/Demo Plan

This need not be added or completed until the specification is nearing beta.

Unresolved issues

This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.

BoF agenda and discussion

Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarising what was discussed and note any options that were rejected.