HeterogeneousInstanceTypes

Launchpad Entry: NovaSpec:heterogeneous-instance-types
Created: Brian Schott
Contributors: USC Information Sciences Institute

Summary

Nova should have support for cpu architectures, accelerator architectures, and network interfaces as part of the definition of an instance type (or flavor using RackSpace API parlance).

An etherpad for discussion of this blueprint is available at http://etherpad.openstack.org/heterogeneousinstancetypes

Release Note

Nova has been extended to allow deployments to advertise and users to request specific processor, accelerator, and network interface options using instance_types (or flavors).

The nova-manage instance_types command supports additional fields:

cpu_arch - processor architecture. Ex: "x86_64", "i386", "P7", etc. (default x86_64)
cpu_info - json-formatted extended processor information
xcpu_arch - accelerator architecture Ex: "fermi" (default "")
xcpu_info - json-formatted extended accelerator information
xcpus - Number of accelerators or accelerator processors
net_arch - primary network interface. Ex: "ethernet", "infiniband", "myrinet"
net_info - json-formatted extended network information
net_mbps - allocated network bandwidth (megabits per second)

Example:

22 GB of memory 33.5 EC2 Compute Units (2 x Intel Xeon X5570, quad-core “Nehalem” architecture) 2 x NVIDIA Tesla “Fermi” M2050 GPUs 1690 GB of instance storage 64-bit platform I/O Performance: Very High (10 Gigabit Ethernet) API name: cg1.4xlarge

cg1.4xlarge:
 * memory_mb= 22000
 * vcpus = 8
 * local_gb = 1690
 * cpu_arch = "x86_64"
 * cpu_info = '{"model":"Nehalem", "features":["tdtscp", "xtpr"]}' 
 * xcpu_arch = "gpu"
 * xcpus = 2
 * xcpu_info ='{"gpu_arch":"fermi", "model":"Tesla 2050", "gcores":"448"}'
 * net_arch = "ethernet"
 * net_info = '{"encap":"Ethernet", "MTU":"8000"}'
 * net_mbps = 10000

Rationale

Currently AWS supports two different CPU architecture types, "i386" and "x86_64". In addition, AWS describes other instance type attributes by reference, such as: I/O Performance: (Moderate/High/Very High 10Gigabit Ethernet), extended CPU information (Intel Xeon X5570, quad-core “Nehalem” architecture), and now GPU accelerators (2 x NVIDIA Tesla “Fermi” M2050 GPUs). In order to implement this functionality in nova, we need to capture this in a way that is accessible to advanced schedulers.

There are several related blueprints:

[1]
[2]

User stories

Mary manages a cloud datacenter. In addition to her x86 blades, she wants to advertise her power7 high performance computing cloud with 40Gbit QDR Infiniband support to customers. Mary uses nova-manage instance_types create to define "p7.sippycup", "p7.tall", "p7.grande", and "p7.venti" with cpu_arch="power7" and an increasing number of default memory, storage, cores, and reserved bandwidth. Mary also has a small number of GPU-accelerated systems, so she defines "p7f.grande" and "p7f.venti" options with xpu_arch="gpu", xpu_info = '{"gpu_arch":"fermi"}', and xcpus = 1 for grande and xcpus = 2 for venti.

Fred wants to run an 8 core machine with 1 fermi-based GPU accelerator. He looks on Mary's web site for text description, then wants the p7f.grande virtual machine. He runs:

euca-run-instances -t p7f.grande -k fred-keypair emi-12345678

Assumptions

This assumes there is a scheduler that is smart enough select a host that matches the requested architecture and confirms that the virtual machine image is compatible. This also assumes that there is a capability to pass thru accelerators to the virtual machine instance. The USC-ISI team is working on all of these capabilities. We will link in related blueprints, but this functionality will stand alone.

Design

TBD

Implementation

This extended capability information should be included in the instance_type table recently included in the Cactus target trunk: [3]

This section should describe a plan of action (the "how") to implement the changes discussed. Could include subsections like:

UI Changes

Should cover changes required to the UI, or specific UI that is required to implement this

TBD:

Code Changes

Code changes should include an overview of what needs to change, and in some cases even the specific details.

TBD:

Migration

Include:

data migration, if any
redirects from old URLs to new ones, if any
how users will be pointed to the new way of doing things, if necessary.

Test/Demo Plan

This need not be added or completed until the specification is nearing beta.

Unresolved issues

This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.

BoF agenda and discussion

Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarising what was discussed and note any options that were rejected.