Meetings/Passthrough

Agenda on Jan. 8th, 2014

Let's go over the key concepts and use cases. In the use cases, neutron or neutron plugin specific configurations are not mentioned.

Key Concepts

• PCI Groups

1. A PCI group is a collection of PCI devices that share the same functions or belong to the same subsystem in a cloud.
2. it's worth mentioning that using an API to define PCI groups make them owned by the tenant who creates them.

• Pre-defined PCI Groups

   For each PCI device class that openstack supports, a PCI group is defined and associated with the PCI devices belonging to that device class. For example, for the PCI device class net, there is a predefined PCI group named net

• User-defined PCI Groups

   User can define PCI groups using a Nova API.

• PCI Passthrough List (whitelist)

   Specified on a compute node to define all the PCI passthrough devices and their associated PCI groups that are available on the node.
   blacklist may be added later if deemed necessary.

• vnic_type:

   # virtio: a virtual port that is attached to a virtual switch
   # direct: SRIOV without macvtap
   # macvtap: SRIOV with macvtap

• nova boot: new parameters in --nic option

   # vnic-type=“vnic” | “direct” | “macvtap”
   # pci-group=pci-group-name
   # port-profile=port-profile-name 

• neutron port-create: new arguments

   # --vnic-type “vnic” | “direct” | “macvtap”
   # --pci-group pci-group-name
   # port-profile port-profile-name

Nova SRIOV Configuration

  # vnic_type = <vnic-type>: specified in controller node to indicate the default vnic-type that VMs will be booted with. default value is "vnic"
  # sriov_auto_all = <on | off>: specified in compute nodes to indicate that all sriov capable ports are added into the ‘net’ PCI group.
  # sriov_only = <on | off>: specified in compute nodes to indicate that nova can only place VMs with sriov vnics onto these nodes. Default value is on for nodes with SRIOV ports. 
  # sriov_pci_group = <pci-group-name>: specified in compute nodes in which all of its SRIOV ports belong to a single pci group.

Use Cases

• SRIOV based cloud

  # All the compute nodes are identical and all the NICs are SRIOV based
  # All the NICs are connected to the same physical network

In this cloud, the admin only needs to specify vnic_type=direct on the controller and sriov_auto_all=on on the compute nodes in the nova configuration file. In addition, the new arguments introduced in the nova boot command are not required.

• A cloud with mixed Vnics

  # On compute nodes with sriov ports only, set sriov_auto_all = on
  # On compute nodes without sriov ports, no change is required

In such a cloud, when booting a VM with sriov vnic, the nova boot command would look like:

   nova boot --flavor m1.large --image <image_id>
                          --nic net-id=<net-id>,vnic-type=direct <vm-name>

This will require some minimum change in the existing applications.

• A Cloud that requires multiple SRIOV PCI groups

  # create all the pci-groups in the cloud by invoking a Nova API
  # on compute nodes that support a single pci group and all of its SRIOV devices belong to this group, set sriov_auto_all=on, sriov_pci_group=<group_name>
  # on compute nodes that support multiple pci groups, define the pci-passthrough-list

In such a cloud, when booting a VM with sriov macvtap, the nova boot command would look like:

    nova boot --flavor m1.large --image <image_id> 
                   --nic net-id=<net-id>,vnic-type=macvtap,pci-group=<group-name> <vm-name>

• Introducing new compute nodes with SRIOV into an existing cloud

Depending on the cloud and the compute node being introduced:

   # it could be as simple as adding sriov_auto_all=on into the nova config file
   # it could be setting sriov_auto_all=on and pci_group=<gourp_name>
   # it could be defining the pci-passthrough-list.

• NIC hot plug

Evolving Design Doc

https://docs.google.com/document/d/1EMwDg9J8zOxzvTnQJ9HwZdiotaVstFWKIuKrPse6JOs/edit?usp=sharing

Previous Meetings

http://eavesdrop.openstack.org/meetings/pci_passthrough/