Jump to: navigation, search

Difference between revisions of "Mellanox-Neutron"

(→‎Mellanox Quantum Plugin Installation (for Havana))
(→‎Mellanox Quantum Plugin Installation (for Havana))
Line 1: Line 1:
−
= Mellanox Quantum Plugin Installation (for Havana) =
+
= Overview =
−
== On the Quantum Server Node ==
+
== Mellanox Quantum Plugin ==
 +
The Openstack Mellanox Quantum plugin supports Mellanox embedded switch functionality as part of the VPI (Ethernet/InfiniBand) HCA.
 +
Mellanox Quantum Plugin allows hardware vNICs (based on SR-IOV virtual functions) per each Virtual Machine vNIC to have its unique
 +
connectivity, security, and QoS attributes. Hardware vNICs can be mapped to the guest VMs through para-virtualization (using a Tap device), or directly as a Virtual PCI device to the guest, allowing higher performance and advanced features such as RDMA (remote direct memory access).
  
−
1. Select Mellanox eSwitch plugin as core plugin for Quantum server.
+
Hardware-based switching, provides better performance, functionality, and security/isolation for virtual cloud environments.
−
+
Future versions of the plug-in will include OpenFlow API to control and monitor the embedded switch and vNICs functionality
−
2. Modify the /etc/quantum/plugins/mlnx/mlnx_conf.ini file to reflect your environment.
 
  
−
3. If you run the Quantum server using init script - change the quantum server configuration to point to Mellanox Quantum Plugin
+
This plugin is implemented according to Plugin-Agent pattern.
  
−
For example, In Ubuntu change /etc/default/quantum-server
 
−
QUANTUM_PLUGIN_CONFIG="/etc/quantum/plugins/mlnx/mlnx_conf.ini"
 
  
−
4. Run the server.
+
            +-----------------+                      +--------------+
−
   #quantum-server --config-file /etc/quantum/quantum.conf --config-file /etc/quantum/plugins/mlnx/mlnx_conf.ini
+
                  | Controller node |                      | Compute node |
−
or
+
        +-----------------------------------+    +-----------------------------------+
−
  #/etc/init.d/quantum-server start
+
        |  +-----------+      +----------+  |    |  +----------+      +----------+  |
 +
        |  |          |      |          |  |    |  |          |  zmq  |          |  |
 +
        |  | Openstack | v2.0 | Mellanox |  | RPC |  | Mellanox |REQ/REP| Mellanox |  |
 +
        |  | Quantum  +------+ Quantum  +-----------+ Quantum  +-------+ Embedded |  |
 +
        |  |          |      | Plugin  |  |    |  | Agent    |      | Switch  |  |
 +
        |  |          |      |          |  |    |  |          |      | (NIC)   |  |
 +
        |  +-----------+      +----------+  |    |  +----------+      +----------+  |
 +
        +-----------------------------------+    +-----------------------------------+
 +
 +
* Openstack Mellanox Quantum Plugin implements the Quantum v2.0 API.
 +
* Mellanox Quantum Plugin processes the Quantum API calls and manages network segmentation ID allocation.
 +
* The plugin uses databases to store configuration and allocation mapping.
 +
* The plugin maintains compatibility to Linux Bridge Plugin, supports DHCP and L3 Agents by running L2 Linux Bridge Agent on Network Node.
 +
* Mellanox Openstack Quantum Agent (L2 Agent) runs on each compute node.
 +
* Agent should apply VIF connectivity based on mapping between a VIF (VM vNIC) and Embedded Switch port.
  
−
==  On Compute Nodes ==
+
==  Mellanox Nova VIF Driver ==
 +
The Mellanox Nova VIF driver should be used when running Mellanox Quantum Plugin. This driver supports the VIF plugin by binding vNIC (para-virtualized or SR-IOV  with optional RDMA guest access) to the embedded switch port.
  
−
=== Nova-compute ===
+
== Prerequisites ==
 +
'''The following are the Mellanox Quantum Plugin prerequisites:'''
  
−
1. Copy the Nova Mellanox VIF driver.
+
1. The software package python-zmq ([https://github.com/zeromq/pyzmq github]) must be installed. EPEL repository can be used as well.
−
    #cp -a mellanox-quantum-plugin/nova/nova/virt/libvirt/mlnx /usr/lib/python2.6/site-packages/nova/virt/libvirt
 
  
−
2. Modify nova.conf.
+
2. python-setuptools (use "yum install python-setuptools")
−
    compute_driver=nova.virt.libvirt.driver.LibvirtDriver
 
−
    libvirt_vif_driver=nova.virt.libvirt.mlnx.vif.MlxEthVIFDriver
 
−
    fabric=default - specifies physical network for vNICs (currently support one fabric per node)
 
  
−
3. Restart Nova.
+
3. python-pip (use "yum install python-pip")
  
−
=== The eswitchd Daemon ===
+
'''Prerequisites only for Compute Servers :'''
−
The eswitchd source files are located in the mellanox-eswitchd repository [https://github.com/mellanox-openstack/mellanox-eswitchd eSwitchd source repository]
 
−
1. Clone and install the eSwitchd
 
−
#git clone https://github.com/mellanox-openstack/mellanox-eswitchd
 
−
#cd mellanox-eswitchd
 
−
#python setup.py install
 
  
−
2. Copy the configuration and init files of the eSwitchd.  
+
1. RH 6.3 or above.
−
#mkdir /etc/eswitchd
 
−
#cp etc/eswitchd/eswitchd.conf /etc/eswitchd
 
−
#cp etc/init.d/eswitchd /etc/init.d
 
  
−
3. Modify the configuration file according to your  environment
+
2. Compute nodes should be equiped with Mellanox ConnectX®-3 Network Adapter ([http://www.mellanox.com/page/infiniband_cards_overview link])
−
# cat /etc/eswitchd/eswitchd.conf
 
−
[DEFAULT]
 
−
default_log_levels="eswitchd=DEBUG"
 
−
verbose=False
 
−
debug=False
 
−
log_file=/var/log/eswitchd.log
 
−
[DAEMON]
 
−
'''fabrics='mlx1:eth4'''' --> Important line  '<fabric_name>:<PF Name>'.
 
−
                    --> The fabric name should match the name in quantum agent and/or openFlow agent.
 
−
default_timeout=4000
 
  
−
3. Run the daemon:
+
3. Mellanox OFED 2.0.3 is installed. Contact [mailto:openstack@mellanox.com?subject=MLNX_OFED2.0  openstack@mellanox.com] to retreive this version.
−
# /etc/init.d/eswitchd start
+
Refer to Mellanox website for the latest OFED documentation ([http://www.mellanox.com/page/products_dyn?product_family=26 link])
  
−
=== Quantum Agent ===
+
4. Enable SR-IOV on ConnectX-3 card. Refer to [http://community.mellanox.com/docs/DOC-1317 Mellanox Community]
  
−
1. Modify the Quantum Agent configuration at /etc/quantum/plugins/mlnx/mlnx_conf.ini.
+
5. The software package iproute2 - ([https://www.kernel.org/pub/linux/utils/net/iproute2/ Code] [http://www.policyrouting.org/iproute2.doc.html Documentation]) must be installed. Reuired only to be installed on compute nodes
  
−
2. Run the agent.
+
6. The software package ethtool ([http://www.kernel.org/pub/software/network/ethtool/ Code]) must be installed (version 3.8 or higher). Required only to be installed on compute nodes.
−
    #python /usr/lib/python2.6/site-packages/quantum/plugins/mlnx/agent/eswitch_quantum_agent.py --config-file /etc/quantum/quantum.conf  --config-file /etc/quantum/plugins/mlnx/mlnx_conf.ini
 
  
−
For a plugin configuration file example (master), please refer to [https://github.com/mellanox-openstack/mellanox-quantum-plugin/blob/master/quantum/etc/quantum/plugins/mlnx/mlnx_conf.ini Mellanox config *ini file].
+
7. oslo.config (use "pip-python install oslo.config"). Reuired only to be installed on compute nodes.
 +
 
 +
8. python-ethtool and python-zmq packages
 +
 
 +
'''Installation with RHOS'''
 +
 
 +
For each section refer to 'Installation via RPMs'. No additional Prerequisites except for those required by RHOS.<br />
 +
 
 +
Please refer to http://www.redhat.com/products/enterprise-linux/openstack-platform For additional information regarding RHOS.
 +
 
 +
== Code Structure ==
 +
 
 +
Mellanox Quantum Plugin and the Nova VIF driver are located at [http://community.mellanox.com/docs/DOC-1187 Mellanox OpenStack RPMs]
 +
 
 +
1. Quantum Plugin package structure:
 +
  quantum/etc/quantum/plugins/mlnx -plugin configuration
 +
  mlnx_conf.ini - sample plugin configuration
 +
 
 +
  quantum/quantum/plugins/mlnx - plugin code
 +
  /agent - Agent code
 +
  /common - common  code
 +
  /db - plugin persistency model and wrapping methods
 +
  mlnx_plugin.py - Mellanox Openstack Plugin
 +
  rpc_callbacks.py - RPC handler for received messages
 +
  agent_notify_api.py - Agent RPC notify methods
 +
 
 +
Mellanox Quantum Plugin is located under quantum/quantum/plugins/.
 +
 
 +
2. Nova VIF driver package structure is:
 +
  nova/nova/mlnx - nova vif driver code
 +
 
 +
Mellanox Nova VIF driver is located under nova/virt/libvirt/mlnx.

Revision as of 11:14, 3 September 2013

Overview

Mellanox Quantum Plugin

The Openstack Mellanox Quantum plugin supports Mellanox embedded switch functionality as part of the VPI (Ethernet/InfiniBand) HCA. Mellanox Quantum Plugin allows hardware vNICs (based on SR-IOV virtual functions) per each Virtual Machine vNIC to have its unique connectivity, security, and QoS attributes. Hardware vNICs can be mapped to the guest VMs through para-virtualization (using a Tap device), or directly as a Virtual PCI device to the guest, allowing higher performance and advanced features such as RDMA (remote direct memory access).

Hardware-based switching, provides better performance, functionality, and security/isolation for virtual cloud environments. Future versions of the plug-in will include OpenFlow API to control and monitor the embedded switch and vNICs functionality

This plugin is implemented according to Plugin-Agent pattern. 


   	         +-----------------+                       +--------------+
                 | Controller node |                       | Compute node |
       +-----------------------------------+     +-----------------------------------+
       |  +-----------+      +----------+  |     |  +----------+       +----------+  |
       |  |           |      |          |  |     |  |          |  zmq  |          |  |
       |  | Openstack | v2.0 | Mellanox |  | RPC |  | Mellanox |REQ/REP| Mellanox |  |
       |  | Quantum   +------+ Quantum  +-----------+ Quantum  +-------+ Embedded |  |
       |  |           |      | Plugin   |  |     |  | Agent    |       | Switch   |  |
       |  |           |      |          |  |     |  |          |       | (NIC)    |  |
       |  +-----------+      +----------+  |     |  +----------+       +----------+  |
       +-----------------------------------+     +-----------------------------------+
  • Openstack Mellanox Quantum Plugin implements the Quantum v2.0 API.
  • Mellanox Quantum Plugin processes the Quantum API calls and manages network segmentation ID allocation.
  • The plugin uses databases to store configuration and allocation mapping.
  • The plugin maintains compatibility to Linux Bridge Plugin, supports DHCP and L3 Agents by running L2 Linux Bridge Agent on Network Node.
  • Mellanox Openstack Quantum Agent (L2 Agent) runs on each compute node.
  • Agent should apply VIF connectivity based on mapping between a VIF (VM vNIC) and Embedded Switch port.

Mellanox Nova VIF Driver

The Mellanox Nova VIF driver should be used when running Mellanox Quantum Plugin. This driver supports the VIF plugin by binding vNIC (para-virtualized or SR-IOV with optional RDMA guest access) to the embedded switch port.

Prerequisites

The following are the Mellanox Quantum Plugin prerequisites:

1. The software package python-zmq (github) must be installed. EPEL repository can be used as well.

2. python-setuptools (use "yum install python-setuptools")

3. python-pip (use "yum install python-pip")

Prerequisites only for Compute Servers :

1. RH 6.3 or above.

2. Compute nodes should be equiped with Mellanox ConnectX®-3 Network Adapter (link)

3. Mellanox OFED 2.0.3 is installed. Contact openstack@mellanox.com to retreive this version. Refer to Mellanox website for the latest OFED documentation (link)

4. Enable SR-IOV on ConnectX-3 card. Refer to Mellanox Community

5. The software package iproute2 - (Code Documentation) must be installed. Reuired only to be installed on compute nodes

6. The software package ethtool (Code) must be installed (version 3.8 or higher). Required only to be installed on compute nodes.

7. oslo.config (use "pip-python install oslo.config"). Reuired only to be installed on compute nodes.

8. python-ethtool and python-zmq packages

Installation with RHOS

For each section refer to 'Installation via RPMs'. No additional Prerequisites except for those required by RHOS.

Please refer to http://www.redhat.com/products/enterprise-linux/openstack-platform For additional information regarding RHOS.

Code Structure

Mellanox Quantum Plugin and the Nova VIF driver are located at Mellanox OpenStack RPMs

1. Quantum Plugin package structure: 

  quantum/etc/quantum/plugins/mlnx -plugin configuration
  mlnx_conf.ini - sample plugin configuration

  quantum/quantum/plugins/mlnx -  plugin code
  /agent - Agent code
  /common - common  code
  /db - plugin persistency model and wrapping methods
  mlnx_plugin.py - Mellanox Openstack Plugin
  rpc_callbacks.py - RPC handler for received messages
  agent_notify_api.py - Agent RPC notify methods

Mellanox Quantum Plugin is located under quantum/quantum/plugins/.

2. Nova VIF driver package structure is: 

  nova/nova/mlnx - nova vif driver code

Mellanox Nova VIF driver is located under nova/virt/libvirt/mlnx.

Retrieved from "https://wiki.openstack.org/w/index.php?title=Mellanox-Neutron&oldid=29269"