Meetings/Passthrough
Contents
Agenda on Jan. 8th, 2014
- Let's go over the key concepts and use cases. In the use cases, neutron or neutron plugin specific configurations are not mentioned.
Key Concepts
- PCI Groups
- A PCI group is a collection of PCI devices that share the same functions or belong to the same subsystem in a cloud.
- it's worth mentioning that using an API to define PCI groups make them owned by the tenant who creates them.
- Pre-defined PCI Groups
- For each PCI device class that openstack supports, a PCI group is defined and associated with the PCI devices belonging to that device class. For example, for the PCI device class net, there is a predefined PCI group named net
- User-defined PCI Groups
- User can define PCI groups using a Nova API.
- PCI Passthrough List (whitelist)
- Specified on a compute node to define all the PCI passthrough devices and their associated PCI groups that are available on the node.
- blacklist (exclude list) may be added later if deemed necessary.
- vnic_type:
- virtio: a virtual port that is attached to a virtual switch
- direct: SRIOV without macvtap
- macvtap: SRIOV with macvtap
- nova boot: new parameters in --nic option
- vnic-type=“vnic” | “direct” | “macvtap”
- pci-group=pci-group-name
- port-profile=port-profile-name
- neutron port-create: new arguments
- --vnic-type “vnic” | “direct” | “macvtap”
- --pci-group pci-group-name
- port-profile port-profile-name
- Nova SRIOV Configuration
- vnic_type = <vnic-type>: specified in controller node to indicate the default vnic-type that VMs will be booted with. default value is "vnic"
- sriov_auto_all = <on | off>: specified in compute nodes to indicate that all sriov capable ports are added into the ‘net’ PCI group.
- sriov_only = <on | off>: specified in compute nodes to indicate that nova can only place VMs with sriov vnics onto these nodes. Default value is on for nodes with SRIOV ports.
- sriov_pci_group = <pci-group-name>: specified in compute nodes in which all of its SRIOV ports belong to a single pci group.
Use Cases
- SRIOV based cloud
- All the compute nodes are identical and all the NICs are SRIOV based
- All the NICs are connected to the same physical network
- In this cloud, the admin only needs to specify vnic_type=direct on the controller and sriov_auto_all=on on the compute nodes in the nova configuration file. In addition, the new arguments introduced in the nova boot command are not required.
- A cloud with mixed Vnics
- On compute nodes with sriov ports only, set sriov_auto_all = on
- On compute nodes without sriov ports, no change is required
- In such a cloud, when booting a VM with sriov vnic, the nova boot command would look like:
nova boot --flavor m1.large --image <image_id> --nic net-id=<net-id>,vnic-type=direct <vm-name>
- This will require some minimum change in the existing applications.
- A Cloud that requires multiple SRIOV PCI groups
- create all the pci-groups in the cloud by invoking a Nova API
- on compute nodes that support a single pci group and in which all of the SRIOV ports belong to this group, set sriov_auto_all=on, sriov_pci_group=<group_name>
- on compute nodes that support multiple pci groups, define the pci-passthrough-list
- In such a cloud, when booting a VM with sriov macvtap, the nova boot command would look like:
nova boot --flavor m1.large --image <image_id> --nic net-id=<net-id>,vnic-type=macvtap,pci-group=<group-name> <vm-name>
- Introducing new compute nodes with SRIOV into an existing cloud
- Depending on the cloud and the compute node being introduced:
- it could be as simple as adding sriov_auto_all=on into the nova config file
- it could be setting sriov_auto_all=on and pci_group=<gourp_name>
- it could be defining the pci-passthrough-list.
- NIC hot plug
Evolving Design Doc
https://docs.google.com/document/d/1EMwDg9J8zOxzvTnQJ9HwZdiotaVstFWKIuKrPse6JOs/edit?usp=sharing