Mellanox-Neutron-Icehouse-Ubuntu-Ethernet

= Overview =

This wiki page describes how to tune Mellanox OpenStack on Ubuntu 12.04 servers

= Prerequisites =


 * A running OpenStack environment installed with the ML2 plugin on top of OVS.
 * All nodes equipped with Mellanox ConnectX®-3 Network Adapter (http://www.mellanox.com/page/products_dyn?product_family=119)
 * Mellanox OFED 2.2 or greater installed on all nodes. Please refer to Mellanox website for the latest OFED: http://www.mellanox.com/page/products_dyn?product_family=26&mtag=linux_sw_drivers
 * SR-IOV enabled on all compute nodes. For more information, please refer to Mellanox Community click here.
 * The software package iproute2 - (http://www.linuxfoundation.org/collaborate/workgroups/networking/iproute2 ) installed on all Compute nodes
 * VLANs configured on the ports in the switch.

= Ethernet Network =

Controller Node
In /etc/nova/nova.conf, ensure you have the following line: security_group_api=nova

If the value is changed, restart the Nova services: # for i in $(initctl list | grep '^nova' | awk '{print $1}'); do service $i restart; done

Neutron Server Node
NOTE: The Neutron Server can sometimes be part of the Controller node.

To configure the Neutron-Server node:

1. Edit /etc/neutron/neutron.conf, and make sure Neutron is using ML2 plugin: [DEFAULT] core_plugin = neutron.plugins.ml2.plugin.Ml2Plugin 2. Modify the ML2 configuration file (/etc/neutron/plugins/ml2/ml2_conf.ini) as follows: [ml2] type_drivers = vlan,flat tenant_network_types = vlan mechanism_drivers = openvswitch,mlnx [ml2_type_vlan] network_vlan_ranges=:: //(i.e. default:2:10) [securitygroup] enable_security_group = True [eswitch] vnic_type = hostdev apply_profile_patch = True 3. Install the required Deb for the Mellanox Neutron plugin (from Ubuntu’s repository): apt-get install neutron-plugin-mlnx-agent 4. Edit /etc/neutron/plugins/mlnx/mlnx_conf.ini to match your environment: [mlnx] network_vlan_ranges = : //(i.e. default:2:10) tenant_network_type = vlan [eswitch] physical_interface_mappings = : //(i.e. default:eth0) vnic_type = hostdev [agent] support_old_agents = True

5. Restart neutron-server: /etc/init.d/neutron-server start

Compute Nodes
To configure the Compute node:

1. Download mlnxvif and eSwitch Daemon (eSwitchd) from the following link: http://www.mellanox.com/downloads/solutions/openstack/icehouse/debian/ 2. Install the Mellanox plugin services mlnxvif and eSwitchd: dpkg –i python-mlnxvif_2014.1.1-1_all.deb dpkg –i python-eswitchd_0.10-1_all.deb 3. Edit /etc/eswitchd/eswitchd.conf as follows: [DAEMON] fabrics=: //(i.e. default:eth0) 4. Edit /etc/nova/nova.conf as follows: [DEFAULT] compute_driver = libvirt.LibvirtDriver libvirt_vif_driver = mlnxvif.vif.MlxEthVIFDriver security_group_api = nova 5. Check if “ebrctl” appears in one of Nova’s rootwrap files: grep 'ebrctl' /etc/nova/rootwrap.d/* If “ebrctl” does not appear in one of Nova’s rootwrap files, edit the file /etc/nova/rootwrap.d/compute.filters as follows: [Filters] ebrctl: CommandFilter, ebrctl, root 6. Install the Neutron Mellanox agent (from Ubuntu repository): apt-get install neutron-plugin-mlnx-agent 7. Copy /etc/neutron/plugins/mlnx/mlnx_conf.ini from the Neutron Server node to the same path on each Compute node. 8. Restart the following services: /etc/init.d/nova-compute restart /etc/init.d/eswitchd start /etc/init.d/neutron-plugin-mlnx-agent start

Network Node
To configure the Network node:

1. Configure eIPoIB (for Linux bridge plugin), and edit the file /etc/infiniband/openib.conf as follows: E_IPOIB_LOAD=yes 2. Edit /etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini as follows: physical_interface_mappings = : //(i.e. default:eth2) 3. Restart Open vSwitch agent: service neutron-openvswitch-agent restart 4. Copy /etc/neutron/neutron.conf from the Compute node to the same path on the Network node.

5. Edit /etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini as follows: physical_interface_mappings = : //(i.e. default:eth2) 6. Edit /etc/neutron/dhcp_agent.ini and /etc/neutron/l3_agent.ini as follows: [DEFAULT] interface_driver = neutron.agent.linux.interface.OVSInterfaceDriver 7. Restart the following services: service neutron-plugin-linuxbridge-agent restart service neutron-dhcp-agent restart service neutron-l3-agent restart

Troubleshooting

 * If you're seeing request timeout errors in the plugin's logs, make sure that the eswitchd service is started and try to increase the request_timeout value in the plugin configuration file.