Jump to: navigation, search

Difference between revisions of "QuantumDevstackOvsXcp"

(Install OpenStack)
 
(18 intermediate revisions by 4 users not shown)
Line 1: Line 1:
__NOTOC__
+
 
 
== Install XCP ==
 
== Install XCP ==
  
Line 12: Line 12:
  
 
NOTE - [[OpenStack]] requires that virtual machine storage be ext. To specify this, select "Enable thin provisioning (Optimised storage for [[XenDesktop]])".
 
NOTE - [[OpenStack]] requires that virtual machine storage be ext. To specify this, select "Enable thin provisioning (Optimised storage for [[XenDesktop]])".
 +
 +
NOTE - The following setup assumes that the XCP machine is connected to a network with a DHCP server, on its eth0. This network will be used as a Management network for the OpenStack installation. Also that network will be used for netinstalling the ubuntu VM. Please note, that if you ran previously a devstack, you might run into issues, as some of your network configuration is hard-coded to saved templates. To avoid using your old templates, make sure, that CLEAN_TEMPLATES is set to True. This will make the installation longer, because a brand new Ubuntu installation will take place. This is an issue, that will be addressed, as part of [https://blueprints.launchpad.net/devstack/+spec/xenapi-devstack-cleanup xenapi devstack cleanup]
 +
 +
Your public and your VM networks will not be connected to any interfaces of your XCP box.
  
 
== Configure a Storage Repository ==
 
== Configure a Storage Repository ==
  
 
To configure a storage repository, login as root to the dom0 host and execute the following:
 
To configure a storage repository, login as root to the dom0 host and execute the following:
 
  
 
<pre><nowiki>
 
<pre><nowiki>
Line 36: Line 39:
 
</nowiki></pre>
 
</nowiki></pre>
  
 +
== Install OpenStack ==
  
== Install git in dom0 (optional) ==
+
Execute the following script on your workstation. Please note, that the script below is a hardcopy of this one: [https://github.com/citrix-openstack/qa/blob/master/xenserver-neutron-trunk.sh CI test script] (automated test runs removed). [http://goo.gl/BuAdg This picture might help to understand what components are installed].
  
Installing software in dom0 is generally frowned upon, but will make it easier to ensure a devstack working tree in dom0.  Copying the devstack tree from another host is also an option.
+
<source lang=bash>
 +
#!/bin/bash
  
To install git, execute the following commands in dom0:
+
set -eu
  
 +
function print_usage_and_die
 +
{
 +
cat >&2 << EOF
 +
usage: $0 XENSERVER_IP XENSERVER_PASS
  
<pre><nowiki>
+
A simple script to test a XenServer devstack with Neutron
## Configure epel repo
+
 
$ rpm --upgrade --replacepkgs http://dl.fedoraproject.org/pub/epel/5/i386/epel-release-5-4.noarch.rpm
+
positional arguments:
 +
XENSERVER_IP The IP address of the XenServer
 +
XENSERVER_PASS The root password for the XenServer
  
## Disable the epel repos, just to be safe
+
An example run:
$ sed -i 's/enabled=1/enabled=0/' /etc/yum.repos.d/epel.repo
 
  
## Install git
+
$0 10.219.10.25 mypassword
$ yum --enablerepo=epel -y install git
+
EOF
</nowiki></pre>
+
exit 1
 +
}
  
 +
XENSERVER_IP="${1-$(print_usage_and_die)}"
 +
XENSERVER_PASS="${2-$(print_usage_and_die)}"
  
== Prepare dom0 devstack ==
+
set -eux
  
Whether on dom0 or a local workstation, execute the following git commands to prepare a devstack tree that supports OVS+XCP:  
+
ssh -q \
 +
    -o Batchmode=yes \
 +
    -o StrictHostKeyChecking=no \
 +
    -o UserKnownHostsFile=/dev/null \
 +
    "root@$XENSERVER_IP" bash -s -- << END_OF_XENSERVER_COMMANDS
 +
set -exu
 +
rm -rf "devstack-master"
 +
wget -qO - https://github.com/openstack-dev/devstack/archive/master.tar.gz |
 +
tar -xzf -
 +
cd "devstack-master"
  
 +
cat << LOCALRC_CONTENT_ENDS_HERE > localrc
 +
# Passwords
 +
MYSQL_PASSWORD=citrix
 +
SERVICE_TOKEN=citrix
 +
ADMIN_PASSWORD=citrix
 +
SERVICE_PASSWORD=citrix
 +
RABBIT_PASSWORD=citrix
 +
GUEST_PASSWORD=citrix
 +
XENAPI_PASSWORD="$XENSERVER_PASS"
 +
SWIFT_HASH="66a3d6b56c1f479c8b4e70ab5c2000f5"
  
<pre><nowiki>
+
# Use xvdb for backing cinder volumes
$ git clone https://github.com/openstack-dev/devstack.git
+
XEN_XVDB_SIZE_GB=10
</nowiki></pre>
+
VOLUME_BACKING_DEVICE=/dev/xvdb
  
 +
# Tempest
 +
DEFAULT_INSTANCE_TYPE="m1.tiny"
  
A suggested localrc for OVS+XCP is as follows:
+
# Compute settings
 +
EXTRA_OPTS=("xenapi_disable_agent=True")
 +
API_RATE_LIMIT=False
 +
VIRT_DRIVER=xenserver
  
 +
# Use a XenServer Image
 +
IMAGE_URLS="https://github.com/downloads/citrix-openstack/warehouse/cirros-0.3.0-x86_64-disk.vhd.tgz"
 +
DEFAULT_IMAGE_NAME="cirros-0.3.0-x86_64-disk"
  
<pre><nowiki>
+
# OpenStack VM settings
# The OVS+XCP-supporting quantum branch, currently in review
+
OSDOMU_MEM_MB=4096
+
UBUNTU_INST_RELEASE=precise
QUANTUM_REPO=https://review.openstack.org/openstack/quantum
+
UBUNTU_INST_IFACE="eth2"
QUANTUM_BRANCH=refs/changes/23/15023/6
+
OSDOMU_VDI_GB=40
  
# 12.04 is only supported with XCP 1.6                 
+
# Exercise settings
UBUNTU_INST_RELEASE="precise"
+
ACTIVE_TIMEOUT=500
UBUNTU_INST_TEMPLATE_NAME="Ubuntu 12.04 (64-bit) for DevStack"
+
TERMINATE_TIMEOUT=500
  
# Devstack's XCP support requires this setting for 12.04 
+
# Increase boot timeout for quantum tests:
PUB_IP=dhcp
+
BOOT_TIMEOUT=500
  
# Configure Q_PLUGIN with openvswitch when running in XS/XCP.  This is
+
# DevStack settings
# the default in stack.sh, but not in the xen installer, so it needs
+
LOGFILE=/tmp/devstack/log/stack.log
# to be set explicitly in localrc
+
SCREEN_LOGDIR=/tmp/devstack/log/
Q_PLUGIN=openvswitch
+
VERBOSE=False
  
## Configure the vm network to support the Quantum OVS DHCP agent
+
# XenAPI specific
# Trunk all VLANS                                         
+
XENAPI_CONNECTION_URL="http://$XENSERVER_IP"
VM_VLAN=-1
+
VNCSERVER_PROXYCLIENT_ADDRESS="$XENSERVER_IP"
# Avoid associating the VM network with a device           
 
VM_DEV=invalid
 
  
# This is the password for your guest (for both stack and root users)
 
GUEST_PASSWORD="<CHANGEME>"
 
# IMPORTANT: The following must be set to your dom0 root password!
 
XENAPI_PASSWORD=${GUEST_PASSWORD}
 
# Explicitly set virt driver here                       
 
VIRT_DRIVER=xenserver
 
# Explicitly set multi-host                                   
 
 
MULTI_HOST=False
 
MULTI_HOST=False
# Give extra time for boot                               
 
ACTIVE_TIMEOUT=360
 
# Interface on which you would like to access services       
 
HOST_IP_IFACE=eth3
 
NETINSTALLIP="dhcp"
 
NAMESERVERS=""
 
NETMASK=""
 
GATEWAY=""
 
  
# Service overrides                                           
+
# Skip boot from volume exercise
# Add quantum, remove nova network                         
+
SKIP_EXERCISES=boot_from_volume,client-env
ENABLED_SERVICES+=,quantum,q-svc,q-agt,q-dhcp,q-l3,-n-net
+
 
# Enable tempest                                           
+
# Quantum specific
ENABLED_SERVICES+=,tempest
+
Q_PLUGIN=openvswitch
 +
ENABLED_SERVICES+=,tempest,quantum,q-svc,q-agt,q-dhcp,q-l3,q-meta,q-domua,-n-net
  
# Disable rate limit to support tempest                     
+
# Disable security groups
API_RATE_LIMIT=False
+
Q_USE_SECGROUP=False
</nowiki></pre>
 
  
 +
# With XenServer single box install, VLANs need to be enabled
 +
ENABLE_TENANT_VLANS="True"
 +
OVS_VLAN_RANGES="physnet1:1000:1024"
  
== Execute devstack ==
+
# CLEAN_TEMPLATES=true
  
To invoke devstack on XCP, execute the following in dom0:
+
Q_USE_DEBUG_COMMAND=True
  
 +
LOCALRC_CONTENT_ENDS_HERE
  
<pre><nowiki>
+
cd tools/xen
$ cd devstack/tools/xen
+
./install_os_domU.sh
$ ./install_os_domU.sh
+
END_OF_XENSERVER_COMMANDS
</nowiki></pre>
+
</source>
  
  
Line 139: Line 166:
  
 
After a successful devstack run using the provided localrc, tempest can be used to verify that Quantum+OVS+XCP can correctly provision networking to [[OpenStack]] VM's.  To run the quantum smoke tests, execute the following in domU:
 
After a successful devstack run using the provided localrc, tempest can be used to verify that Quantum+OVS+XCP can correctly provision networking to [[OpenStack]] VM's.  To run the quantum smoke tests, execute the following in domU:
 
  
 
<pre><nowiki>
 
<pre><nowiki>
 
$ cd /opt/stack/tempest  
 
$ cd /opt/stack/tempest  
$ nosetests tempest/tests/network/test_network_basic_ops.py
+
$ nosetests tempest/scenario/test_network_basic_ops.py
 
</nowiki></pre>
 
</nowiki></pre>
 +
 +
=== Results ===
 +
<source lang=text>
 +
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_001_create_keypairs[gate,smoke] ... ok
 +
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_002_create_security_groups[gate,smoke] ... ok
 +
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_003_create_networks[gate,smoke] ... ok
 +
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_004_check_networks[gate,smoke] ... ok
 +
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_005_create_servers[gate,smoke] ... ok
 +
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_006_check_tenant_network_connectivity[gate,smoke] ... SKIP: Tenant networks not configured to be reachable.
 +
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_007_assign_floating_ips[gate,smoke] ... ok
 +
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_008_check_public_network_connectivity[gate,smoke] ... ok
 +
 +
----------------------------------------------------------------------
 +
Ran 8 tests in 74.313s
 +
 +
OK (SKIP=1)
 +
</source>
 +
 +
== Verify with exercise ==
 +
To verify that the setup is working, you can run quantum-adv-test. I choose to move away all the other files, so that quantum is the only one that runs:
 +
<source lang=bash>
 +
cd /opt/stack/devstack/
 +
mkdir disabled
 +
mv exercises/* disabled/
 +
mv disabled/quantum-adv-test.sh exercises/
 +
./exercise.sh 2>&1 | tee quantumlog
 +
</source>
 +
 +
 +
Please note, that I had to increase the BOOT_TIMEOUT to get these tests passing. At the moment, it is set to an extreme high value, 500 (see the script above)
 +
 +
=== Results ===
 +
<source lang=bash>
 +
tail quantumlog
 +
</source>
 +
<source lang=text>
 +
++ NOVA_VERSION=1.1
 +
++ export COMPUTE_API_VERSION=1.1
 +
++ COMPUTE_API_VERSION=1.1
 +
+ set +o xtrace
 +
*********************************************************************
 +
SUCCESS: End DevStack Exercise: /opt/stack/devstack/exercises/quantum-adv-test.sh
 +
*********************************************************************
 +
=====================================================================
 +
PASS quantum-adv-test
 +
=====================================================================
 +
</source>

Latest revision as of 20:26, 30 June 2013

Install XCP

Installation has been tested with XCP 1.6, which can be downloaded here (http://www.xen.org/download/xcp/index_1.6.0.html).

Presuming installation in a virtual machine:

  • ensure that nested virtualization is configured
  • ensure that 2 hard disks of type scsi are available
    • 1 for xcp installation (10gb dynamic allocation suggested)
    • 1 for vm storage (40gb dynamic allocation suggested)

NOTE - OpenStack requires that virtual machine storage be ext. To specify this, select "Enable thin provisioning (Optimised storage for XenDesktop)".

NOTE - The following setup assumes that the XCP machine is connected to a network with a DHCP server, on its eth0. This network will be used as a Management network for the OpenStack installation. Also that network will be used for netinstalling the ubuntu VM. Please note, that if you ran previously a devstack, you might run into issues, as some of your network configuration is hard-coded to saved templates. To avoid using your old templates, make sure, that CLEAN_TEMPLATES is set to True. This will make the installation longer, because a brand new Ubuntu installation will take place. This is an issue, that will be addressed, as part of xenapi devstack cleanup

Your public and your VM networks will not be connected to any interfaces of your XCP box.

Configure a Storage Repository

To configure a storage repository, login as root to the dom0 host and execute the following:

## List the machine’s UUID
$ xe host-list

## Create an EXT storage repository (SR) with thin provisioning support and direct access to the VHD files.  
# This example uses /dev/sdb, but this may differ for your host.
$ xe sr-create host-uuid=<hit TAB should match UUID above> content-type=user name-label=”Local EXT3 SR” shared=false device-config:device=/dev/sdb type=ext

## Set the SR to use thin provisioning. Otherwise, nova won’t be able to find the SR when it needs to write to disk.
# Look for 'Local EXT3 SR', making note of the associated UUID.
$ xe sr-list
$ xe sr-param-set uuid=<SR uuid> other-config:i18n-key=local-storage

## Set the configured SR as the default.
$ xe pool-list
$ xe pool-param-set uuid=<hit TAB> default-SR=<SR uuid>

Install OpenStack

Execute the following script on your workstation. Please note, that the script below is a hardcopy of this one: CI test script (automated test runs removed). This picture might help to understand what components are installed.

#!/bin/bash

set -eu

function print_usage_and_die
{
cat >&2 << EOF
usage: $0 XENSERVER_IP XENSERVER_PASS

A simple script to test a XenServer devstack with Neutron

positional arguments:
XENSERVER_IP The IP address of the XenServer
XENSERVER_PASS The root password for the XenServer

An example run:

$0 10.219.10.25 mypassword
EOF
exit 1
}

XENSERVER_IP="${1-$(print_usage_and_die)}"
XENSERVER_PASS="${2-$(print_usage_and_die)}"

set -eux

ssh -q \
    -o Batchmode=yes \
    -o StrictHostKeyChecking=no \
    -o UserKnownHostsFile=/dev/null \
    "root@$XENSERVER_IP" bash -s -- << END_OF_XENSERVER_COMMANDS
set -exu
rm -rf "devstack-master"
wget -qO - https://github.com/openstack-dev/devstack/archive/master.tar.gz |
tar -xzf -
cd "devstack-master"

cat << LOCALRC_CONTENT_ENDS_HERE > localrc
# Passwords
MYSQL_PASSWORD=citrix
SERVICE_TOKEN=citrix
ADMIN_PASSWORD=citrix
SERVICE_PASSWORD=citrix
RABBIT_PASSWORD=citrix
GUEST_PASSWORD=citrix
XENAPI_PASSWORD="$XENSERVER_PASS"
SWIFT_HASH="66a3d6b56c1f479c8b4e70ab5c2000f5"

# Use xvdb for backing cinder volumes
XEN_XVDB_SIZE_GB=10
VOLUME_BACKING_DEVICE=/dev/xvdb

# Tempest
DEFAULT_INSTANCE_TYPE="m1.tiny"

# Compute settings
EXTRA_OPTS=("xenapi_disable_agent=True")
API_RATE_LIMIT=False
VIRT_DRIVER=xenserver

# Use a XenServer Image
IMAGE_URLS="https://github.com/downloads/citrix-openstack/warehouse/cirros-0.3.0-x86_64-disk.vhd.tgz"
DEFAULT_IMAGE_NAME="cirros-0.3.0-x86_64-disk"

# OpenStack VM settings
OSDOMU_MEM_MB=4096
UBUNTU_INST_RELEASE=precise
UBUNTU_INST_IFACE="eth2"
OSDOMU_VDI_GB=40

# Exercise settings
ACTIVE_TIMEOUT=500
TERMINATE_TIMEOUT=500

# Increase boot timeout for quantum tests:
BOOT_TIMEOUT=500

# DevStack settings
LOGFILE=/tmp/devstack/log/stack.log
SCREEN_LOGDIR=/tmp/devstack/log/
VERBOSE=False

# XenAPI specific
XENAPI_CONNECTION_URL="http://$XENSERVER_IP"
VNCSERVER_PROXYCLIENT_ADDRESS="$XENSERVER_IP"

MULTI_HOST=False

# Skip boot from volume exercise
SKIP_EXERCISES=boot_from_volume,client-env

# Quantum specific
Q_PLUGIN=openvswitch
ENABLED_SERVICES+=,tempest,quantum,q-svc,q-agt,q-dhcp,q-l3,q-meta,q-domua,-n-net

# Disable security groups
Q_USE_SECGROUP=False

# With XenServer single box install, VLANs need to be enabled
ENABLE_TENANT_VLANS="True"
OVS_VLAN_RANGES="physnet1:1000:1024"

# CLEAN_TEMPLATES=true

Q_USE_DEBUG_COMMAND=True

LOCALRC_CONTENT_ENDS_HERE

cd tools/xen
./install_os_domU.sh
END_OF_XENSERVER_COMMANDS


The initial run of devstack may take a long time (depending on the available internet connection), since devstack has to create a new VM (domU) and install ubuntu on it before installing OpenStack. Devstack saves the domU VM as a template after the first install, so subsequent runs will be much faster. If issues occur during installation (an unfortunately common occurrence when XCP is running in a VM), the XenCenter tool (unfortunately Windows-only) will aid in troubleshooting the problem.

Devstack should provide instructions once installation has completed as to how to log in to the domU instance, which will be the equivalent of a regular devstack'd host.

Verify with tempest

After a successful devstack run using the provided localrc, tempest can be used to verify that Quantum+OVS+XCP can correctly provision networking to OpenStack VM's. To run the quantum smoke tests, execute the following in domU:

$ cd /opt/stack/tempest 
$ nosetests tempest/scenario/test_network_basic_ops.py

Results

tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_001_create_keypairs[gate,smoke] ... ok
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_002_create_security_groups[gate,smoke] ... ok
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_003_create_networks[gate,smoke] ... ok
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_004_check_networks[gate,smoke] ... ok
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_005_create_servers[gate,smoke] ... ok
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_006_check_tenant_network_connectivity[gate,smoke] ... SKIP: Tenant networks not configured to be reachable.
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_007_assign_floating_ips[gate,smoke] ... ok
tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_008_check_public_network_connectivity[gate,smoke] ... ok

----------------------------------------------------------------------
Ran 8 tests in 74.313s

OK (SKIP=1)

Verify with exercise

To verify that the setup is working, you can run quantum-adv-test. I choose to move away all the other files, so that quantum is the only one that runs:

cd /opt/stack/devstack/
mkdir disabled
mv exercises/* disabled/
mv disabled/quantum-adv-test.sh exercises/
./exercise.sh 2>&1 | tee quantumlog


Please note, that I had to increase the BOOT_TIMEOUT to get these tests passing. At the moment, it is set to an extreme high value, 500 (see the script above)

Results

tail quantumlog
++ NOVA_VERSION=1.1
++ export COMPUTE_API_VERSION=1.1
++ COMPUTE_API_VERSION=1.1
+ set +o xtrace
*********************************************************************
SUCCESS: End DevStack Exercise: /opt/stack/devstack/exercises/quantum-adv-test.sh
*********************************************************************
=====================================================================
PASS quantum-adv-test
=====================================================================