Difference between revisions of "SR-IOV-Passthrough-For-Networking-Mitaka-Ethernet"
(Created page with "=SR-IOV Networking in OpenStack Mitaka Ethernet= OpenStack Juno adds inbox support to request VM access to virtual network via SR-IOV NIC. With the introduction of SR-IOV bas...") |
(→Neutron Server) |
||
(4 intermediate revisions by the same user not shown) | |||
Line 47: | Line 47: | ||
===Neutron Server=== | ===Neutron Server=== | ||
− | + | Edit '''/etc/neutron/plugins/ml2/ml2_conf_sriov.ini''' supported PCI vendor VF devices, defined by vendor_id:product_id according to the PCI ID Repository in the: | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
[ml2_sriov] | [ml2_sriov] | ||
Line 63: | Line 54: | ||
# | # | ||
# supported_pci_vendor_devs = 15b3:1004, 8086:10ca | # supported_pci_vendor_devs = 15b3:1004, 8086:10ca | ||
− | # Example: | + | # Example: Mellanox ConnectX-3Pro and ConnectX-4 |
− | supported_pci_vendor_devs = | + | supported_pci_vendor_devs='15b3:1004','15b3:1014' |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | '' | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
Restart neutron-server | Restart neutron-server | ||
systemctl restart neutron-server.service | systemctl restart neutron-server.service | ||
Line 88: | Line 66: | ||
* [http://www.intel.com/content/www/us/en/embedded/products/networking/xl710-sr-iov-config-guide-gbe-linux-brief.html Intel SR-IOV driver support] | * [http://www.intel.com/content/www/us/en/embedded/products/networking/xl710-sr-iov-config-guide-gbe-linux-brief.html Intel SR-IOV driver support] | ||
− | + | ===SR-IOV neutron agent=== | |
− | + | Edit '''/etc/neutron/plugins/ml2/sriov_agent.ini''': | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | ===SR-IOV neutron agent | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
Add/modify the following lines at the end of the file. | Add/modify the following lines at the end of the file. | ||
Line 144: | Line 76: | ||
other firewall_driver for other Agents by updating their conf with the requested firewall driver. | other firewall_driver for other Agents by updating their conf with the requested firewall driver. | ||
− | + | Restart Neutron SR-IOV agent. Run: | |
− | systemctl | + | systemctl restart neutron-sriov-nic-agent.service |
− | |||
− | |||
====Exclude VFs==== | ====Exclude VFs==== |
Latest revision as of 13:29, 14 December 2015
Contents
SR-IOV Networking in OpenStack Mitaka Ethernet
OpenStack Juno adds inbox support to request VM access to virtual network via SR-IOV NIC. With the introduction of SR-IOV based NICs, the traditional virtual bridge is no longer required. Each SR-IOV port is associated with a virtual function (VF). SR-IOV ports may be provided by Hardware-based Virtual Ethernet Bridging (HW VEB); or they may be extended to an upstream physical switch (IEEE 802.1br). There are two ways that SR-IOV port may be connected:
- directly connected to its VF
- connected with a macvtap device that resides on the host, which is then connected to the corresponding VF
Nova
Nova support for SR-IOV enables scheduling an instance with SR-IOV ports based on their network connectivity. The neutron ports' associated physical networks have to be considered in making the scheduling decision. PCI Whitelist has been enhanced to allow tags to be associated with PCI devices. PCI devices available for SR-IOV networking should be tagged with physical_network label.
For SR-IOV networking, a pre-defined tag "physical_network" is used to define the physical network to which the devices are attached. A whitelist entry is defined as:
["vendor_id": "<id>",] ["product_id": "<id>",] ["address": "[[[[<domain>]:]<bus>]:][<slot>][.[<function>]]" | "devname": "Ethernet Interface Name",] "physical_network":"name string of the physical network"
<id> can be an asterisk (*) or a valid vendor/product ID as displayed by the Linux utility lspci. The address uses the same syntax as in lspci. The devname can be a valid PCI device name. The only device names that are supported are those displayed by the Linux utility ifconfig -a and correspond to either a PF or a VF on a vNIC.
If the device defined by the address or devname corresponds to a SR-IOV PF, all VFs under the PF will match the entry.
Multiple whitelist entries per host are supported.
Neutron
Neutron support for SR-IOV requires ML2 Plugin with SR-IOV supporting mechanism driver. Currently there is ML2 Mechanism Driver for SR-IOV capable NIC based switching (HW VEB). There are network adapters from different vendors that vary by supporting various functionality. If VF link state update is supported by vendor network adapter, the SR-IOV NIC L2 agent should be deployed to leverage this functionality .
VM creation flow with SR-IOV vNIC
- Create one or more neutron ports. Run:
neutron port-create <net-id> --binding:vnic-type <direct | macvtap | normal>
- Boot VM with one or more neutron ports. Run:
nova boot --flavor m1.large --image <image> --nic port-id=<port1> --nic port-id=<port2> <vm name>
Note that in the nova boot API, users can specify either a port-ID or a net-ID. If a net-ID is specified, it is assumed that the user is requesting a normal virtual port (which is not an SR-IOV port).
SR-IOV Configuration
Controller Node
Nova Scheduler
Enabling PciPassthroughFilter modify /etc/nova/nova.conf
scheduler_available_filters = nova.scheduler.filters.all_filters scheduler_default_filters = RetryFilter, AvailabilityZoneFilter, RamFilter, ComputeFilter, ComputeCapabilitiesFilter, ImagePropertiesFilter, PciPassthroughFilter
Neutron Server
Edit /etc/neutron/plugins/ml2/ml2_conf_sriov.ini supported PCI vendor VF devices, defined by vendor_id:product_id according to the PCI ID Repository in the:
[ml2_sriov] # (ListOpt) Comma-separated list of # supported Vendor PCI Devices, in format vendor_id:product_id # # supported_pci_vendor_devs = 15b3:1004, 8086:10ca # Example: Mellanox ConnectX-3Pro and ConnectX-4 supported_pci_vendor_devs='15b3:1004','15b3:1014'
Restart neutron-server
systemctl restart neutron-server.service
Compute Node
enable SR-IOV on network adapter
By default SRIOV is disabled, you will have to enable SRIOV and create the VFs on each compute host that should support SRIOV functionality. Currently specific Intel and Mellanox cards are known to support SRIOV. Below resources contain information on how to enable and create the VFs:
SR-IOV neutron agent
Edit /etc/neutron/plugins/ml2/sriov_agent.ini:
Add/modify the following lines at the end of the file.
[securitygroup] firewall_driver = neutron.agent.firewall.NoopFirewallDriver
Note:SR-IOV agent only work with NoopFirewallDriver when Security Groups are enabled, but you can still use other firewall_driver for other Agents by updating their conf with the requested firewall driver.
Restart Neutron SR-IOV agent. Run:
systemctl restart neutron-sriov-nic-agent.service
Exclude VFs
If you want to exclude some of the VFs so the agent does not configure them, you need to list them in the sriov_nic section:
Example: exclude_devices = eth1:0000:07:00.2; 0000:07:00.3, eth2:0000:05:00.1; 0000:05:00.2