OpenStack - User contributions [en]

Mellanox-Neutron-Icehouse-Ubuntu

2014-09-21T07:31:31Z

Gil Meir: /* Configuration */

= Page is under construction =

Note: The page is not ready yet.

Once it will be ready, the note will be removed.

= Overview =

This wiki page describes how to tune Mellanox OpenStack on Ubuntu 12.04 servers

= Prerequisites =

You need at least 3 nodes:
* Controller node running the neutron-server service.
* Compute node running the eswitchd, nova-compute & neutron-plugin-mlnx-agent services.
* Network node running the neutron-plugin-linuxbridge-agent, neutron-dhcp-agent and neutron-l3-agent services.

= Ethernet Network =

== Controller Node ==

In /etc/nova/nova.conf ensure you have the following line:
security_group_api=nova

If you change, restart the nova services
# for i in $(initctl list | grep '^nova' | awk '{print $1}'); do service $i restart; done

== Neutron Server Node ==
=== Installation ===

1. Neutron server uses MySQL database. Make sure you have running MySQL database for neutron.

If Neutron server is already running, you should stop it.

#/etc/init.d/neutron-server stop

If you want to use a database that was created you have to drop it and create another one. For example:

mysql -u root -ppassword <<EOF
drop database if exists neutron;
create database neutron;
EOF

If you want to create a new one:

mysql -u root -ppassword <<EOF
drop database if exists neutron;
create database neutron;
GRANT ALL PRIVILEGES ON neutron.* TO 'neutron'@'localhost' IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES ON neutron.* TO 'neutron' IDENTIFIED BY 'password';
FLUSH PRIVILEGES;
EOF

2. Install the required Deb for the Mellanox Neutron plugin:

apt-get install -y neutron-plugin-mlnx-agent

3. Modify the /etc/neutron/plugins/mlnx/mlnx_conf.ini file to reflect your environment.

4. Change /etc/default/neutron-server :

NEUTRON_PLUGIN_CONFIG="/etc/neutron/plugins/mlnx/mlnx_conf.ini"

=== Configuration ===

1. Make the Mellanox plugin the current Neutron plugin by editing neutron.conf and changing core_plugin.

core_plugin = neutron.plugins.ml2.plugin.Ml2Plugin

2. Plugin configuration: Edit the configuration file: /etc/neutron/plugins/mlnx/mlnx_conf.ini

[MLNX]
tenant_network_type - must be set on of supported tenant network types.
Possible values: 'vlan' for Ethernet or 'ib' for Infiniband.'vlan' is the default.
network_vlan_ranges - must be configured to specify the names of the physical

networks managed by the mellanox plugin,
along with the ranges of VLAN IDs available on each physical network for allocation to virtual networks.
Possible range is vlan range is 1-4093.
The default is "default:1:100"
<fabric name >:<vlan range start>:<vlan end range>

[agent]
rpc_support_old_agents = True
Note: rpc_support_old_agents should be set to 'True' (non default).

=== Start Services ===

Start (or restart) the Neutron server
#/etc/init.d/neutron-server start

== Compute Nodes ==

=== Installation ===

1. If you didn't download Mellanox OpenStack debian files download it :

http://www.mellanox.com/downloads/solutions/openstack/

2. Install the eswitchd debian files:
dpkg -i eswitchd_0.8-1_amd64.deb
dpkg -i python-mlnxvif_0.6-1_all.deb
apt-get install -f # fix dependencies
apt-get install -y neutron-plugin-mlnx-agent

3. In case you would like to use Ethernet in para-virtualized mode the VIF driver is already included in Nova package. Otherwise, Install Mellanox VIF driver (Make sure nova is installed on your server)

=== Configuration ===

1. Configure /etc/eswitchd/eswitchd.conf if needed

Please Refer to Mellanox Community for the eSwitchd installation notes ( click [http://community.mellanox.com/docs/DOC-1126 here])

2. Modify /etc/nova/nova.conf

compute_driver=nova.virt.libvirt.driver.LibvirtDriver
libvirt_vif_driver=mlnxvif.vif.MlxEthVIFDriver
security_group_api=nova
connection_type=libvirt

In case you didn't install the Mellanox VIF driver, and you plan to use Ethernet only in paravirtualized ,mode: change the following:

libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtGenericVIFDriver

3. Modify the /etc/neutron/plugins/mlnx/mlnx_conf.ini file to reflect your environment.
[AGENT]
rpc_support_old_agents = True
[ESWITCH]
physical_interface_mapping - the network_interface_mappings maps each physical network name to the physical interface (on top of Mellanox Adapter) connecting the node to that physical network. The format of this paramter is: <fabric name>:< PF name> (Only releavant on Compute node). PF Name can either be the PF (Physical Function) Name or 'autoeth' for automatic Ethernet configuration,'autoib' for automatic Infiniband configuration.The default is "default:autoeth".
vnic_type - type of VM network interface: 'mlnx_direct' or 'hostdev' according to libvirt terminology.
hostdev: this is the traditional method of assigning any generic PCI device to a guest (SR-IOV).
mlnx_direct: this is a method to provide macvtap device on top of the PCI device (default).
bridge: - When using Linux Bridge Plugin on top of eIPoIB device
daemon_endpoint - 'tcp://127.0.0.1:60001'

For a plugin configuration file example (Havana), please refer to Mellanox config *ini file (click [https://github.com/openstack/neutron/blob/stable/havana/etc/neutron/plugins/mlnx/mlnx_conf.ini here])

4. (Optional) Modify /etc/init/neutron-plugin-mlnx-agent.conf as described in LP bug #[https://bugs.launchpad.net/ubuntu/+source/neutron/+bug/1257732 1257732]

5. Check that ebrctl is listed somewhere in /etc/nova/rootwrap.d/*, otherwise add this to /etc/nova/rootwrap.d/compute.filters file in the [Filters] section.
ebrctl: CommandFilter, ebrctl, root

=== Start Services ===

1. Restart Nova.

#/etc/init.d/nova-compute restart

2. Start eswitch Daemon

#/etc/init.d/eswitchd start

3. Start the Neutron agent

#/etc/init.d/neutron-plugin-mlnx-agent start

Note: eswitch Daemon should be running before the Neutron Agent is started.

== Network Node ==

1. Install the Neutron Linux bridge plugin, the Neutron DHCP agent and the Neutron L3 agent:

# apt-get install neutron-plugin-linuxbridge-agent neutron-dhcp-agent neutron-l3-agent

2. Copy /etc/neutron/neutron.conf from the Compute node.

3. Change the following configuration of the ini file (/etc/neutron/plugins/linuxbridge/linuxbridge_conf.ini). The "default" in the example is the name of the physical network as configured in /etc/neutron/plugins/mlnx/mlnx_conf.ini .
[linux_bridge]
physical_interface_mappings = default:eth2

4. Configure the DHCP agent according to the following guidelines

Update the following file: /etc/neutron/dhcp_agent.ini with:
[DEFAULT]
interface_driver = neutron.agent.linux.interface.BridgeInterfaceDriver

For additional information refer to the [http://docs.openstack.org/admin-guide-cloud/content/ch_networking.html#section_adv_cfg_dhcp_agent OpenStack documentation]

5. Configure the L3 agent according to the following guidelines

Update the following file: /etc/neutron/l3_agent.ini with:
[DEFAULT]
interface_driver = neutron.agent.linux.interface.BridgeInterfaceDriver

For additional information refer to the [http://docs.openstack.org/admin-guide-cloud/content/ch_networking.html#section_adv_cfg_l3_agent OpenStack documentation]

6. Configure the Metadata agent according to the [http://docs.openstack.org/admin-guide-cloud/content/ch_networking.html#nova_config_metadata OpenStack documentation]

7. Start the Neutron Linux Bridge, DHCP and L3 agents
# restart neutron-plugin-linuxbridge-agent
# restart neutron-dhcp-agent
# restart neutron-l3-agent
# restart neutron-metadata-agent

== Troubleshooting ==

* If you're seeing request timeout errors in the plugin's logs, make sure that the eswitchd service is started and try to increase the request_timeout value in the plugin configuration file.

Mellanox-Neutron-Icehouse-Ubuntu

2014-09-16T14:01:21Z

Gil Meir: /* Configuration */

= Page is under construction =

Note: The page is not ready yet.

Once it will be ready, the note will be removed.

= Overview =

This wiki page describes how to tune Mellanox OpenStack on Ubuntu 12.04 servers

= Prerequisites =

You need at least 3 nodes:
* Controller node running the neutron-server service.
* Compute node running the eswitchd, nova-compute & neutron-plugin-mlnx-agent services.
* Network node running the neutron-plugin-linuxbridge-agent, neutron-dhcp-agent and neutron-l3-agent services.

= Ethernet Network =

== Controller Node ==

In /etc/nova/nova.conf ensure you have the following line:
security_group_api=nova

If you change, restart the nova services
# for i in $(initctl list | grep '^nova' | awk '{print $1}'); do service $i restart; done

== Neutron Server Node ==
=== Installation ===

1. Neutron server uses MySQL database. Make sure you have running MySQL database for neutron.

If Neutron server is already running, you should stop it.

#/etc/init.d/neutron-server stop

If you want to use a database that was created you have to drop it and create another one. For example:

mysql -u root -ppassword <<EOF
drop database if exists neutron;
create database neutron;
EOF

If you want to create a new one:

mysql -u root -ppassword <<EOF
drop database if exists neutron;
create database neutron;
GRANT ALL PRIVILEGES ON neutron.* TO 'neutron'@'localhost' IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES ON neutron.* TO 'neutron' IDENTIFIED BY 'password';
FLUSH PRIVILEGES;
EOF

2. Install the required Deb for the Mellanox Neutron plugin:

apt-get install -y neutron-plugin-mlnx-agent

3. Modify the /etc/neutron/plugins/mlnx/mlnx_conf.ini file to reflect your environment.

4. Change /etc/default/neutron-server :

NEUTRON_PLUGIN_CONFIG="/etc/neutron/plugins/mlnx/mlnx_conf.ini"

=== Configuration ===

1. Make the Mellanox plugin the current Neutron plugin by editing neutron.conf and changing core_plugin.

core_plugin = neutron.plugins.ml2.plugin.Ml2Plugin

2. Plugin configuration: Edit the configuration file: /etc/neutron/plugins/mlnx/mlnx_conf.ini

[MLNX]
tenant_network_type - must be set on of supported tenant network types.
Possible values: 'vlan' for Ethernet or 'ib' for Infiniband.'vlan' is the default.
network_vlan_ranges - must be configured to specify the names of the physical

networks managed by the mellanox plugin,
along with the ranges of VLAN IDs available on each physical network for allocation to virtual networks.
Possible range is vlan range is 1-4093.
The default is "default:1:100"
<fabric name >:<vlan range start>:<vlan end range>

[agent]
rpc_support_old_agents = True
Note: rpc_support_old_agents should be set to 'True' (non default).

=== Start Services ===

Start (or restart) the Neutron server
#/etc/init.d/neutron-server start

== Compute Nodes ==

=== Installation ===

1. If you didn't download Mellanox OpenStack debian files download it :

http://www.mellanox.com/downloads/solutions/openstack/

2. Install the eswitchd debian files:
dpkg -i eswitchd_0.8-1_amd64.deb
dpkg -i python-mlnxvif_0.6-1_all.deb
apt-get install -f # fix dependencies
apt-get install -y neutron-plugin-mlnx-agent

3. In case you would like to use Ethernet in para-virtualized mode the VIF driver is already included in Nova package. Otherwise, Install Mellanox VIF driver (Make sure nova is installed on your server)

=== Configuration ===

1. Configure /etc/eswitchd/eswitchd.conf if needed

Please Refer to Mellanox Community for the eSwitchd installation notes ( click [http://community.mellanox.com/docs/DOC-1126 here])

2. Modify /etc/nova/nova.conf

compute_driver=nova.virt.libvirt.driver.LibvirtDriver
libvirt_vif_driver=mlnxvif.vif.MlxEthVIFDriver
security_group_api=nova
connection_type=libvirt

In case you didn't install the Mellanox VIF driver, and you plan to use Ethernet only in paravirtualized ,mode: change the following:

libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtGenericVIFDriver

3. Modify the /etc/neutron/plugins/mlnx/mlnx_conf.ini file to reflect your environment.
[AGENT]
rpc_support_old_agents = True
[ESWITCH]
physical_interface_mapping - the network_interface_mappings maps each physical network name to the physical interface (on top of Mellanox Adapter) connecting the node to that physical network. The format of this paramter is: <fabric name>:< PF name> (Only releavant on Compute node). PF Name can either be the PF (Physical Function) Name or 'autoeth' for automatic Ethernet configuration,'autoib' for automatic Infiniband configuration.The default is "default:autoeth".
vnic_type - type of VM network interface: 'mlnx_direct' or 'hostdev' according to libvirt terminology.
hostdev: this is the traditional method of assigning any generic PCI device to a guest (SR-IOV).
mlnx_direct: this is a method to provide macvtap device on top of the PCI device (default).
bridge: - When using Linux Bridge Plugin on top of eIPoIB device
daemon_endpoint - 'tcp://127.0.0.1:60001'

For a plugin configuration file example (Havana), please refer to Mellanox config *ini file (click [https://github.com/openstack/neutron/blob/stable/havana/etc/neutron/plugins/mlnx/mlnx_conf.ini here])

4. (Optional) Modify /etc/init/neutron-plugin-mlnx-agent.conf as described in LP bug #[https://bugs.launchpad.net/ubuntu/+source/neutron/+bug/1257732 1257732]

5. (Optional) Check that ebrctl is listed somewhere in /etc/nova/rootwrap.d/*, otherwise add this to /etc/nova/rootwrap.d/compute.filters file in the [Filters] section.
ebrctl: CommandFilter, ebrctl, root

=== Start Services ===

1. Restart Nova.

#/etc/init.d/nova-compute restart

2. Start eswitch Daemon

#/etc/init.d/eswitchd start

3. Start the Neutron agent

#/etc/init.d/neutron-plugin-mlnx-agent start

Note: eswitch Daemon should be running before the Neutron Agent is started.

== Network Node ==

1. Install the Neutron Linux bridge plugin, the Neutron DHCP agent and the Neutron L3 agent:

# apt-get install neutron-plugin-linuxbridge-agent neutron-dhcp-agent neutron-l3-agent

2. Copy /etc/neutron/neutron.conf from the Compute node.

3. Change the following configuration of the ini file (/etc/neutron/plugins/linuxbridge/linuxbridge_conf.ini). The "default" in the example is the name of the physical network as configured in /etc/neutron/plugins/mlnx/mlnx_conf.ini .
[linux_bridge]
physical_interface_mappings = default:eth2

4. Configure the DHCP agent according to the following guidelines

Update the following file: /etc/neutron/dhcp_agent.ini with:
[DEFAULT]
interface_driver = neutron.agent.linux.interface.BridgeInterfaceDriver

For additional information refer to the [http://docs.openstack.org/admin-guide-cloud/content/ch_networking.html#section_adv_cfg_dhcp_agent OpenStack documentation]

5. Configure the L3 agent according to the following guidelines

Update the following file: /etc/neutron/l3_agent.ini with:
[DEFAULT]
interface_driver = neutron.agent.linux.interface.BridgeInterfaceDriver

For additional information refer to the [http://docs.openstack.org/admin-guide-cloud/content/ch_networking.html#section_adv_cfg_l3_agent OpenStack documentation]

6. Configure the Metadata agent according to the [http://docs.openstack.org/admin-guide-cloud/content/ch_networking.html#nova_config_metadata OpenStack documentation]

7. Start the Neutron Linux Bridge, DHCP and L3 agents
# restart neutron-plugin-linuxbridge-agent
# restart neutron-dhcp-agent
# restart neutron-l3-agent
# restart neutron-metadata-agent

== Troubleshooting ==

* If you're seeing request timeout errors in the plugin's logs, make sure that the eswitchd service is started and try to increase the request_timeout value in the plugin configuration file.