|
|
| (One intermediate revision by the same user not shown) |
| Line 1: |
Line 1: |
| | =SR-IOV Networking in OpenStack Juno= | | =SR-IOV Networking in OpenStack Juno= |
| − | OpenStack Juno adds inbox support to request VM access to virtual network via SR-IOV NIC. With the introduction of SR-IOV based NICs, the traditional virtual bridge is no longer required. Each SR-IOV port is associated with a virtual function (VF). SR-IOV ports may be provided by Hardware-based Virtual Ethernet Bridging (HW VEB); or they may be extended to an upstream physical switch (IEEE 802.1br).
| |
| − | There are two ways that SR-IOV port may be connected:
| |
| − | * directly connected to its VF
| |
| − | * connected with a macvtap device that resides on the host, which is then connected to the corresponding VF
| |
| | | | |
| − | ==Nova==
| + | [[SR-IOV-Passthrough-For-Networking|SR-IOV configuration]] |
| − | Nova support for SR-IOV enables scheduling an instance with SR-IOV ports based on their network connectivity. The neutron ports' associated physical networks have to be considered in making the scheduling decision.
| |
| − | PCI Whitelist has been enchanced to allow tags to be associated with PCI devices. PCI devices available for SR-IOV networking should be tagged with physical_network label.
| |
| − | | |
| − | For SR-IOV networking, a pre-defined tag "physical_network" is used to define the physical network to which the devices are attached. A whitelist entry is defined as:
| |
| − | ["vendor_id": "<id>",] ["product_id": "<id>",]
| |
| − | ["address": "[[[[<domain>]:]<bus>]:][<slot>][.[<function>]]" |
| |
| − | "devname": "Ethernet Interface Name",]
| |
| − | "physical_network":"name string of the physical network"
| |
| − | | |
| − | <id> can be an asterisk (*) or a valid vendor/product ID as displayed by the Linux utility lspci. The address uses the same syntax as in lspci. The devname can be a valid PCI device name. The only device names that are supported are those displayed by the Linux utility ifconfig -a and correspond to either a PF or a VF on a vNIC.
| |
| − | | |
| − | If the device defined by the address or devname corresponds to a SR-IOV PF, all VFs under the PF will match the entry.
| |
| − | | |
| − | Multiple whitelist entries per host are supported.
| |
| − | | |
| − | ==Neutron==
| |
| − | Neutron support for SR-IOV requires ML2 Plugin with SR-IOV supporting mechanism driver.
| |
| − | Currently there is ML2 Mechanism Driver for SR-IOV capable NIC based switching (HW VEB).
| |
| − | There are network adapters from different vendors that vary by supporting various functionality.
| |
| − | If VF link state update is supported by vendor network adapter, the SR-IOV NIC L2 agent should be deployed to leverage this functionality .
| |
| − | | |
| − | ==VM creation flow with SR-IOV vNIC==
| |
| − | * Create one or more neutron ports. Run:
| |
| − | neutron port-create <net-id> --binding:vnic-type <direct | macvtap | normal>
| |
| − | | |
| − | * Boot VM with one or more neutron ports. Run:
| |
| − | nova boot --flavor m1.large --image <image>
| |
| − | --nic port-id=<port1> --nic port-id=<port2> <vm name>
| |
| − |
| |
| − | Note that in the nova boot API, users can specify either a port-ID or a net-ID. If a net-ID is specified, it is assumed that the user is requesting a normal virtual port (which is not an SR-IOV port).
| |
| − | | |
| − | =SR-IOV Configuration=
| |
| − | | |
| − | | |
| − | ===Neutron Server===
| |
| − | Using ML2 Neutron plugin modify /etc/neutron/plugins/ml2/ml2_conf.ini:
| |
| − | | |
| − | [ml2]
| |
| − | tenant_network_types = vlan
| |
| − | type_drivers = vlan
| |
| − | mechanism_drivers = openvswitch,sriovnicswitch
| |
| − | [ml2_type_vlan]
| |
| − | network_vlan_ranges = physnet1:2:100
| |
| − | | |
| − | Add supported PCI vendor VF devices, defined by vendor_id:product_id according to the PCI ID Repository in the /etc/neutron/plugins/ml2/ml2_conf_sriov.ini:
| |
| − | | |
| − | [ml2_sriov]
| |
| − | supported_pci_vendor_devs = vendor_id:product_id
| |
| − | | |
| − | Example for Intel NIC that supports SR-IOV:
| |
| − | supported_pci_vendor_devs = 8086:10ca
| |
| − | | |
| − | If SRIOV network adapters support VF link state setting and admin state management is desired, make sure to add /etc/neutron/plugins/ml2/ml2_conf_sriov.ini [ml2_sriov] section
| |
| − | the following setting:
| |
| − | | |
| − | agent_required = True
| |
| − | | |
| − | Neutron server should be run with the two configuration files /etc/neutron/plugins/ml2/ml2_conf.in and /etc/neutron/plugins/ml2/ml2_conf_sriov.ini
| |
| − | neutron-server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2_conf_sriov.ini
| |
| − | | |
| − | ==Compute==
| |
| − | ===nova-compute===
| |
| − | On each compute node you have to associate the VFs available to each physical network.
| |
| − | That is performed by configuring pci_passthrough_whitelist in /etc/nova/nova.conf. So, for example:
| |
| − | pci_passthrough_whitelist = {"address":"*:0a:00.*","physical_network":"physnet1"}
| |
| − | This associates any VF with address that includes ':0a:00.' in its address to the physical network physnet1.
| |
| − | | |
| − | After configuring the whitelist you have to restart nova-compute service.
| |
| − | | |
| − | When using devstack pci_passthrough_whitelist can be configured in local.conf file, for example:
| |
| − | <pre>
| |
| − | [[post-config|$NOVA_CONF]]
| |
| − | [DEFAULT]
| |
| − | pci_passthrough_whitelist = {"'"address"'":"'"*:02:00.*"'","'"physical_network"'":"'"default"'"}
| |
| − | </pre>
| |
| − | | |
| − | ===SR-IOV neutron agent===
| |
| − | If the hardware supports it and you want to enable changing the port admin_state, you have to run the Neutron SR-IOV agent.<br />
| |
| − | | |
| − | '''Note:'''If you configured agent_required=True on the Neutron server, you must run the Agent on each compute node.
| |
| − | | |
| − | In /etc/neutron/plugins/ml2/ml2_conf.ini make sure you have the following:
| |
| − | [securitygroup]
| |
| − | firewall_driver = neutron.agent.firewall.NoopFirewallDriver
| |
| − | | |
| − | Modify /etc/neutron/plugins/ml2/ml2_conf_sriov.ini as follows:
| |
| − | | |
| − | [sriov_nic]
| |
| − | physical_device_mappings = physnet1:eth1
| |
| − | exclude_devices =
| |
| − | | |
| − | Where:
| |
| − | * physnet1 is the physical network
| |
| − | * eth1 is the physical function (PF)
| |
| − | * exclude_devices is empty so all the VFs associated with eth1 may be configured by the agent
| |
| − | | |
| − | After modifying the configuration file, start the Neutron SR-IOV agent. Run:
| |
| − | neutron-sriov-nic-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2_conf_sriov.ini
| |
| − | | |
| − | ====Exclude VFs====
| |
| − | If you want to exclude some of the VFs so the agent does not configure them, you need to list them in the sriov_nic section:<br />
| |
| − | | |
| − | '''Example:''' exclude_devices = eth1:0000:07:00.2; 0000:07:00.3, eth2:0000:05:00.1; 0000:05:00.2
| |
| − | | |
| − | =References=
| |
| − | | |
| − | [http://community.mellanox.com/docs/DOC-1484 Openstack ML2 SR-IOV driver support]
| |
