Jump to: navigation, search

Difference between revisions of "Mellanox-Neutron-Icehouse-Ubuntu-Ethernet"

(Created page with "test")
 
(→‎Network Node)
 
(20 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
test
+
= Overview =
 +
 
 +
This wiki page describes how to tune Mellanox OpenStack on Ubuntu 12.04 servers
 +
 
 +
= Prerequisites =
 +
 
 +
* A running OpenStack environment  installed with the ML2 plugin on top of OVS.
 +
* All nodes equipped with Mellanox ConnectX®-3 Network Adapter (http://www.mellanox.com/page/products_dyn?product_family=119)
 +
* Mellanox OFED 2.2 or greater installed on all nodes. Please refer to Mellanox website for the latest OFED: http://www.mellanox.com/page/products_dyn?product_family=26&mtag=linux_sw_drivers
 +
* SR-IOV enabled on all compute nodes. For more information, please refer to Mellanox Community [http://community.mellanox.com/docs/DOC-1317 click here].
 +
* The software package iproute2 - (http://www.linuxfoundation.org/collaborate/workgroups/networking/iproute2 ) installed on all Compute nodes
 +
* VLANs configured on the ports in the switch.
 +
 
 +
= Ethernet Network =
 +
 
 +
== Controller Node ==
 +
 
 +
In /etc/nova/nova.conf, ensure you have the following line:
 +
  security_group_api=nova
 +
 
 +
If the value is changed, restart the Nova services:
 +
  # for i in $(initctl list | grep '^nova' | awk '{print $1}'); do service $i restart; done
 +
 
 +
=== Neutron Server Node ===
 +
NOTE: The Neutron Server can sometimes be part of the Controller node.
 +
 
 +
To configure the Neutron-Server node:
 +
 
 +
1. Edit /etc/neutron/neutron.conf, and make sure Neutron is using ML2 plugin:
 +
[DEFAULT]
 +
core_plugin = neutron.plugins.ml2.plugin.Ml2Plugin
 +
2. Modify the ML2 configuration file (/etc/neutron/plugins/ml2/ml2_conf.ini) as follows:
 +
[ml2]
 +
type_drivers = vlan,flat
 +
tenant_network_types = vlan
 +
mechanism_drivers = openvswitch,mlnx 
 +
[ml2_type_vlan]
 +
network_vlan_ranges=<physical_network>:<min_vlan>:<max_vlan> //(i.e. default:2:10) 
 +
[securitygroup]
 +
enable_security_group = True 
 +
[eswitch]
 +
vnic_type = hostdev
 +
apply_profile_patch = True
 +
3. Install the required Deb for the Mellanox Neutron plugin (from Ubuntu’s repository):
 +
apt-get install neutron-plugin-mlnx-agent
 +
4. Edit /etc/neutron/plugins/mlnx/mlnx_conf.ini to match your environment:
 +
[mlnx]
 +
network_vlan_ranges = <physical_network>:<min_vlan:<max_vlan> //(i.e. default:2:10)
 +
tenant_network_type = vlan
 +
[eswitch]
 +
physical_interface_mappings = <physical_network>:<eth interface> //(i.e. default:eth0)
 +
vnic_type = hostdev
 +
[agent]
 +
support_old_agents = True
 +
 
 +
5. Restart neutron-server:
 +
/etc/init.d/neutron-server start
 +
 
 +
== Compute Nodes ==
 +
 
 +
To configure the Compute node:
 +
 
 +
1. Download mlnxvif and eSwitch Daemon (eSwitchd) from the following link:
 +
http://www.mellanox.com/downloads/solutions/openstack/icehouse/debian/
 +
2. Install the Mellanox plugin services mlnxvif and eSwitchd:
 +
dpkg –i python-mlnxvif_2014.1.1-1_all.deb
 +
dpkg –i python-eswitchd_0.10-1_all.deb
 +
3. Edit /etc/eswitchd/eswitchd.conf as follows:
 +
[DAEMON]
 +
fabrics=<physical_network>:<eth-interface> //(i.e. default:eth0)
 +
4. Edit /etc/nova/nova.conf as follows:
 +
[DEFAULT]
 +
compute_driver = libvirt.LibvirtDriver
 +
libvirt_vif_driver = mlnxvif.vif.MlxEthVIFDriver
 +
security_group_api = nova
 +
5. Check if “ebrctl” appears in one of Nova’s rootwrap files:
 +
grep 'ebrctl' /etc/nova/rootwrap.d/*
 +
If “ebrctl” does not appear in one of Nova’s rootwrap files, edit the file /etc/nova/rootwrap.d/compute.filters as follows:
 +
[Filters]
 +
ebrctl: CommandFilter, ebrctl, root
 +
6. Install the Neutron Mellanox agent (from Ubuntu repository):
 +
apt-get install neutron-plugin-mlnx-agent
 +
7. Copy /etc/neutron/plugins/mlnx/mlnx_conf.ini from the Neutron Server node to the same path on each Compute node.
 +
8. Restart the following services:
 +
/etc/init.d/nova-compute restart
 +
/etc/init.d/eswitchd start
 +
/etc/init.d/neutron-plugin-mlnx-agent start
 +
 
 +
== Network Node ==
 +
 
 +
To configure the Network node:
 +
 
 +
1. Configure eIPoIB (for Linux bridge plugin), and edit the file /etc/infiniband/openib.conf as follows:
 +
E_IPOIB_LOAD=yes
 +
2. Edit /etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini as follows:
 +
physical_interface_mappings = <physical_network>:<eth-interface>
 +
//(i.e. default:eth2)
 +
3. Restart Open vSwitch agent:
 +
service neutron-openvswitch-agent restart
 +
4. Copy /etc/neutron/neutron.conf from the Compute node to the same path on the Network node.
 +
 
 +
5. Edit /etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini as follows:
 +
physical_interface_mappings = <physical_network>:<eth interface>
 +
//(i.e. default:eth2)
 +
6. Edit /etc/neutron/dhcp_agent.ini and /etc/neutron/l3_agent.ini as follows:
 +
[DEFAULT]
 +
interface_driver = neutron.agent.linux.interface.OVSInterfaceDriver
 +
7. Restart the following services:
 +
service neutron-plugin-linuxbridge-agent restart
 +
service neutron-dhcp-agent restart
 +
service neutron-l3-agent restart
 +
 
 +
== Troubleshooting ==
 +
 
 +
* If you're seeing request timeout errors in the plugin's logs, make sure that the eswitchd service is started and try to increase the request_timeout value in the plugin configuration file.

Latest revision as of 16:30, 26 November 2014

Overview

This wiki page describes how to tune Mellanox OpenStack on Ubuntu 12.04 servers

Prerequisites

Ethernet Network

Controller Node

In /etc/nova/nova.conf, ensure you have the following line: 

  security_group_api=nova

If the value is changed, restart the Nova services: 

  # for i in $(initctl list | grep '^nova' | awk '{print $1}'); do service $i restart; done

Neutron Server Node

NOTE: The Neutron Server can sometimes be part of the Controller node.

To configure the Neutron-Server node:

1. Edit /etc/neutron/neutron.conf, and make sure Neutron is using ML2 plugin: 

[DEFAULT]
core_plugin = neutron.plugins.ml2.plugin.Ml2Plugin

2. Modify the ML2 configuration file (/etc/neutron/plugins/ml2/ml2_conf.ini) as follows: 

[ml2]
type_drivers = vlan,flat
tenant_network_types = vlan
mechanism_drivers = openvswitch,mlnx  
[ml2_type_vlan]
network_vlan_ranges=<physical_network>:<min_vlan>:<max_vlan> //(i.e. default:2:10)   
[securitygroup]
enable_security_group = True   
[eswitch]
vnic_type = hostdev
apply_profile_patch = True

3. Install the required Deb for the Mellanox Neutron plugin (from Ubuntu’s repository): 

apt-get install neutron-plugin-mlnx-agent

4. Edit /etc/neutron/plugins/mlnx/mlnx_conf.ini to match your environment: 

[mlnx]
network_vlan_ranges = <physical_network>:<min_vlan:<max_vlan> //(i.e. default:2:10)
tenant_network_type = vlan 
[eswitch]
physical_interface_mappings = <physical_network>:<eth interface> //(i.e. default:eth0)
vnic_type = hostdev
[agent]
support_old_agents = True

5. Restart neutron-server: 

/etc/init.d/neutron-server start

Compute Nodes

To configure the Compute node:

1. Download mlnxvif and eSwitch Daemon (eSwitchd) from the following link: 

http://www.mellanox.com/downloads/solutions/openstack/icehouse/debian/

2. Install the Mellanox plugin services mlnxvif and eSwitchd: 

dpkg –i python-mlnxvif_2014.1.1-1_all.deb
dpkg –i python-eswitchd_0.10-1_all.deb

3. Edit /etc/eswitchd/eswitchd.conf as follows: 

[DAEMON]
fabrics=<physical_network>:<eth-interface> //(i.e. default:eth0)

4. Edit /etc/nova/nova.conf as follows: 

[DEFAULT]
compute_driver = libvirt.LibvirtDriver
libvirt_vif_driver = mlnxvif.vif.MlxEthVIFDriver
security_group_api = nova

5. Check if “ebrctl” appears in one of Nova’s rootwrap files: 

grep 'ebrctl' /etc/nova/rootwrap.d/*

If “ebrctl” does not appear in one of Nova’s rootwrap files, edit the file /etc/nova/rootwrap.d/compute.filters as follows: 

[Filters]
ebrctl: CommandFilter, ebrctl, root

6. Install the Neutron Mellanox agent (from Ubuntu repository): 

apt-get install neutron-plugin-mlnx-agent

7. Copy /etc/neutron/plugins/mlnx/mlnx_conf.ini from the Neutron Server node to the same path on each Compute node. 8. Restart the following services: 

/etc/init.d/nova-compute restart
/etc/init.d/eswitchd start
/etc/init.d/neutron-plugin-mlnx-agent start

Network Node

To configure the Network node:

1. Configure eIPoIB (for Linux bridge plugin), and edit the file /etc/infiniband/openib.conf as follows: 

E_IPOIB_LOAD=yes

2. Edit /etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini as follows: 

physical_interface_mappings = <physical_network>:<eth-interface> 
//(i.e. default:eth2)

3. Restart Open vSwitch agent: 

service neutron-openvswitch-agent restart

4. Copy /etc/neutron/neutron.conf from the Compute node to the same path on the Network node.

5. Edit /etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini as follows: 

physical_interface_mappings = <physical_network>:<eth interface> 
//(i.e. default:eth2)

6. Edit /etc/neutron/dhcp_agent.ini and /etc/neutron/l3_agent.ini as follows: 

[DEFAULT] 
interface_driver = neutron.agent.linux.interface.OVSInterfaceDriver

7. Restart the following services: 

service neutron-plugin-linuxbridge-agent restart 
service neutron-dhcp-agent restart
service neutron-l3-agent restart

Troubleshooting

  • If you're seeing request timeout errors in the plugin's logs, make sure that the eswitchd service is started and try to increase the request_timeout value in the plugin configuration file.
Retrieved from "https://wiki.openstack.org/w/index.php?title=Mellanox-Neutron-Icehouse-Ubuntu-Ethernet&oldid=69066"