VirtDriverGuestCPUTopology
Virtualization Driver Guest CPU Topology
Background information
Each virtualization driver in OpenStack has its own approach to defining the CPU topology seen by guest virtual machines. The libvirt driver will expose all vCPUs as individual sockets, with 1 core and no hyper-threads.
UNIX operating systems will happily use any CPU topology that is exposed to them within a upper bound on the total number of logical CPUs. That said there can be performance implications from choosing different topologies. For example, 2 hyper-threads are usually not equivalent in performance to 2 cores or sockets, and as such operating system schedulers have special logic to deal with task placement. So if a host has a CPU with 2 cores with 2 threads, and two tasks to run, it will try to place them on different cores, rather than in different threads within a core. It follows that if a guest is shown 4 sockets, the operating system will not be making optimal scheduler placement decisions to avoid competing for constrained thread resources.
Windows operating systems meanwhile are more restrictive in the CPU topology they are willing to use. In particular some versions will have restrictions on the number of sockets they are prepared to use. So if a OS is limited to using 4 sockets and a 8 vCPU guest is desired, then the hypervisor must ensure it exposes a topology with at least 2 cores per socket. Failure to do this will result in the guest refusing to use some of the vCPUs it is assigned.
High level requirements / design
Since the restrictions on CPU topology vary according to the guest operating system being run, placing constraints at either the host level or flavour level is insufficiently flexible. Either the restrictions need to be in a lookaside database, such as that provided by the libosinfo project, or more immediately be expressed using metadata attributes on images. Flavours record the total number of vCPUs to exposed to the guest machine. It is desirable that a single image be capable of booting on arbitrary flavours. It follows from this that images should not express a literal number of sockets/core/threads, but rather express the upper limits for each of these.
Thus a Microsoft Windows Server 2008 R2 Standard Edition image may be tagged with max_sockets=4. If booted with a flavour saying vcpus=4 then any of the following topologies would be valid
- sockets=4, cores=1, threads=1
- sockets=2, cores=2, threads=1
- sockets=2, cores=1, threads=2
- sockets=1, cores=2, threads=2
- sockets=1, cores=4, threads=1
- sockets=1, cores=1, threads=4
If, however, the flavour said vcpus=8 then valid topologies would be
- sockets=4, cores=2, threads=1
- sockets=4, cores=1, threads=2
- sockets=2, cores=4, threads=1
- sockets=2, cores=4, threads=4
- sockets=1, cores=4, threads=2
- sockets=1, cores=2, threads=4
- sockets=1, cores=8, threads=1
- sockets=1, cores=1, threads=8
A compute node may choose between cores and threads based on what topology the host pCPUs have. eg if the host has 2 cores + 2 threads, then it would likely want to use only guest configs with threads==2, and then do strict placement of host<->guest CPUs to match topologies.
Conversely though the image owner may not want to have any threads at all, in which case they might tag the image with max_sockets=4,max_threads=1, in which case the compute node would only have control over the sockets & cores.
If the guest OS had an upper bound on all three levels, they could fully specify max_sockets=4,max_cores=12,max_threads=1. In such a case, the maximum vCPU that could be supported by the image is 48 (4x12). If an attempt was made to boot the image using a flavour which had vcpu=64, then this should result in failure to boot, since the topology constraints could not be satisfied while still providing the requisite number of vCPUs.
