Difference between revisions of "Meetings/Passthrough"

Revision as of 00:14, 8 January 2014

Agenda on Jan. 8th, 2014

Let's go over the key concepts and use cases. In the use cases, neutron or neutron plugin specific configurations are not mentioned.

Key Concepts

• PCI Groups

1. A PCI group is a collection of PCI devices that share the same functions or belong to the same subsystem in a cloud.

In fact, two proposals exist for PCI group definition - via API, with the implication that they're stored centrally in the database, and via config, specifically a (match expression -> PCI group name) in the compute node configuration. A competing proposal is PCI aliases, which work on the current assumption that all PCI device data is returned to the database and PCI devices can be selected by doing matching at schedule time and thus a name -> match expression mapping is all that need be saved. Thus the internal question of "should all device information be returned to the controller" drives some of the design options.

1. it's worth mentioning that using an API to define PCI groups make them owned by the tenant who creates them.

• Pre-defined PCI Groups

For each PCI device class that openstack supports, a PCI group is defined and associated with the PCI devices belonging to that device class. For example, for the PCI device class net, there is a predefined PCI group named net

• User-defined PCI Groups

User can define PCI groups using a Nova API.

• PCI Passthrough List (whitelist)

1. Specified on a compute node to define all the PCI passthrough devices and their associated PCI groups that are available on the node.
2. blacklist (exclude list) may be added later if deemed necessary.

• vnic_type:

1. virtio: a virtual port that is attached to a virtual switch
2. direct: SRIOV without macvtap
3. macvtap: SRIOV with macvtap

This configuration item is not essential to PCI passthrough. It's also a Neutron configuration item.

• nova boot: new parameters in --nic option

1. vnic-type=“vnic” | “direct” | “macvtap”
2. pci-group=pci-group-name
3. port-profile=port-profile-name This property is not related directly to use of PCI passthrough for networks. It is a requirement of 802.1BR-based systems.

• neutron port-create: new arguments

1. --vnic-type “vnic” | “direct” | “macvtap”
2. --pci-group pci-group-name
3. port-profile port-profile-name

• Nova SRIOV Configuration

1. vnic_type = <vnic-type>: specified in controller node to indicate the default vnic-type that VMs will be booted with. default value is "vnic"
2. sriov_auto_all = <on | off>: specified in compute nodes to indicate that all sriov capable ports are added into the ‘net’ PCI group.
3. sriov_only = <on | off>: specified in compute nodes to indicate that nova can only place VMs with sriov vnics onto these nodes. Default value is on for nodes with SRIOV ports.
4. sriov_pci_group = <pci-group-name>: specified in compute nodes in which all of its SRIOV ports belong to a single pci group.

The SRIOV cofiguration items are enhancements to the base proposal that make it much easier to configure compute hosts where it is known that all VFs will be available to cloud users.

Use Cases

These use cases do not include non-network passthrough cases.

• SRIOV based cloud

1. All the compute nodes are identical and all the NICs are SRIOV based
2. All the NICs are connected to the same physical network

In this cloud, the admin only needs to specify vnic_type=direct on the controller and sriov_auto_all=on on the compute nodes in the nova configuration file. In addition, the new arguments introduced in the nova boot command are not required.

• A cloud with mixed Vnics

1. On compute nodes with sriov ports only, set sriov_auto_all = on
2. On compute nodes without sriov ports, no change is required

In such a cloud, when booting a VM with sriov vnic, the nova boot command would look like:

   nova boot --flavor m1.large --image <image_id>
                          --nic net-id=<net-id>,vnic-type=direct <vm-name>

This will require some minimum change in the existing applications.

• A Cloud that requires multiple SRIOV PCI groups

1. create all the pci-groups in the cloud by invoking a Nova API
2. on compute nodes that support a single pci group and in which all of the SRIOV ports belong to this group, set sriov_auto_all=on, sriov_pci_group=<group_name>
3. on compute nodes that support multiple pci groups, define the pci-passthrough-list

In such a cloud, when booting a VM with sriov macvtap, the nova boot command would look like:

    nova boot --flavor m1.large --image <image_id> 
                   --nic net-id=<net-id>,vnic-type=macvtap,pci-group=<group-name> <vm-name>

• Introducing new compute nodes with SRIOV into an existing cloud

Depending on the cloud and the compute node being introduced:

1. it could be as simple as adding sriov_auto_all=on into the nova config file
2. it could be setting sriov_auto_all=on and pci_group=<group_name>
3. it could be defining the pci-passthrough-list.

• NIC hot plug

Evolving Design Doc

https://docs.google.com/document/d/1EMwDg9J8zOxzvTnQJ9HwZdiotaVstFWKIuKrPse6JOs/edit?usp=sharing

Previous Meetings

http://eavesdrop.openstack.org/meetings/pci_passthrough/