Jump to: navigation, search

Difference between revisions of "QuantumDevstackOvsXcp"

(Install XCP)
(Prepare dom0 devstack)
Line 54: Line 54:
 
</nowiki></pre>
 
</nowiki></pre>
  
== Prepare dom0 devstack ==
+
== Install OpenStack ==
  
Whether on dom0 or a local workstation, execute the following commands to prepare a devstack tree that supports OVS+XCP:  
+
Execute the following script on your workstation. The script will cherry-pick the required changes, and push a branch to that user's repo. The github user must have a fork of [https://github.com/openstack-dev/devstack openstack-dev/devstack] and [https://github.com/openstack/quantum openstack/quantum]. Please note, that the script below is a hardcopy of this one (infrastructure - proxy specific details removed): [https://github.com/citrix-openstack/qa/blob/master/xenserver-quantum-devstack.sh CI test script].
  
<pre><nowiki>
+
<source lang=bash>
$ git clone https://github.com/openstack-dev/devstack.git
+
#!/bin/bash
$ cd devstack
 
$ git fetch https://review.openstack.org/openstack-dev/devstack refs/changes/82/27982/1 && git checkout FETCH_HEAD
 
</nowiki></pre>
 
  
A suggested localrc for OVS+XCP is as follows:
+
set -eu
  
<pre><nowiki>
+
function print_usage_and_die
# The OVS+XCP-supporting quantum branch, currently in review
+
{
+
cat >&2 << EOF
QUANTUM_REPO=https://review.openstack.org/openstack/quantum
+
usage: $0 XENSERVER_IP XENSERVER_PASS
QUANTUM_BRANCH=refs/changes/23/15023/10
 
  
# 12.04 is only supported with XCP 1.6                 
+
A simple script to setup a XenServer installation with Quantum.
UBUNTU_INST_RELEASE="precise"
 
UBUNTU_INST_TEMPLATE_NAME="Ubuntu 12.04 (64-bit) for DevStack"
 
  
# Devstack's XCP support requires this setting for 12.04 
+
positional arguments:
PUB_IP=dhcp
+
XENSERVER_IP    The IP address of the XenServer
 +
XENSERVER_PASS  The root password for the XenServer
 +
GITHUB_USER      The github user to use for temporary branches
  
# Configure Q_PLUGIN with openvswitch when running in XS/XCP. This is
+
An example run:
# the default in stack.sh, but not in the xen installer, so it needs
+
 
# to be set explicitly in localrc
+
$0 10.219.10.25 mypassword citrix-openstack
Q_PLUGIN=openvswitch
+
EOF
 +
exit 1
 +
}
 +
 
 +
XENSERVER_IP="${1-$(print_usage_and_die)}"
 +
XENSERVER_PASS="${2-$(print_usage_and_die)}"
 +
GITHUB_USER="${3-$(print_usage_and_die)}"
 +
 
 +
set -eux
 +
 
 +
function create_branch() {
 +
    local source_repo
 +
    local target_repo
 +
    local branchname
 +
 
 +
    source_repo="$1"
 +
    target_repo="$2"
 +
    branchname="$3"
 +
 
 +
    local tmpdir
 +
 
 +
    branchname=$(date +%s)
 +
 
 +
    tmpdir=$(mktemp -d)
 +
    (
 +
        cd $tmpdir
 +
        git clone "$source_repo" repo
 +
        cd repo
 +
        git checkout -b "$branchname"
 +
        git remote add target_repo "$target_repo"
 +
 
 +
        ( echo "set -exu"; cat ) | bash -s --
 +
        git push target_repo "$branchname"
 +
    )
 +
    rm -rf "$tmpdir"
 +
}
 +
 
 +
# Create custom devstack branch
 +
devstack_branch=$(date +%s)
 +
create_branch \
 +
    "https://github.com/openstack-dev/devstack.git" \
 +
    "git@github.com:$GITHUB_USER/devstack.git" \
 +
    "$devstack_branch" << EOF
 +
git fetch https://review.openstack.org/openstack-dev/devstack refs/changes/16/30416/3 && git cherry-pick FETCH_HEAD
 +
git fetch https://review.openstack.org/openstack-dev/devstack refs/changes/03/30703/1 && git cherry-pick FETCH_HEAD
 +
git fetch https://review.openstack.org/openstack-dev/devstack refs/changes/35/31035/3 && git cherry-pick FETCH_HEAD
 +
EOF
 +
 
 +
# Create custom quantum branch
 +
quantum_branch=$(date +%s)
 +
create_branch \
 +
    "https://github.com/openstack/quantum.git" \
 +
    "git@github.com:$GITHUB_USER/quantum.git" \
 +
    "$quantum_branch" << EOF
 +
git fetch https://review.openstack.org/openstack/quantum refs/changes/77/31077/1 && git cherry-pick FETCH_HEAD
 +
EOF
 +
 
 +
ssh -q \
 +
    -o Batchmode=yes \
 +
    -o StrictHostKeyChecking=no \
 +
    -o UserKnownHostsFile=/dev/null \
 +
    "root@$XENSERVER_IP" bash -s -- << EOF
 +
set -exu
 +
rm -rf "devstack-$devstack_branch"
 +
wget -qO - https://github.com/$GITHUB_USER/devstack/archive/$devstack_branch.tar.gz |
 +
    tar -xzf -
 +
cd "devstack-$devstack_branch"
 +
 
 +
cat << LOCALRC_CONTENT_ENDS_HERE > localrc
 +
# Passwords
 +
MYSQL_PASSWORD=citrix
 +
SERVICE_TOKEN=citrix
 +
ADMIN_PASSWORD=citrix
 +
SERVICE_PASSWORD=citrix
 +
RABBIT_PASSWORD=citrix
 +
GUEST_PASSWORD=citrix
 +
XENAPI_PASSWORD="$XENSERVER_PASS"
 +
SWIFT_HASH="66a3d6b56c1f479c8b4e70ab5c2000f5"
  
## Configure the vm network to support the Quantum OVS DHCP agent
+
# Tempest
# Trunk all VLANS                                         
+
DEFAULT_INSTANCE_TYPE="m1.small"
VM_VLAN=-1
 
# Avoid associating the VM network with a device           
 
VM_DEV=invalid
 
  
# This is the password for your guest (for both stack and root users)
+
# Compute settings
GUEST_PASSWORD="<CHANGEME>"
+
EXTRA_OPTS=("xenapi_disable_agent=True")
# IMPORTANT: The following must be set to your dom0 root password!
+
API_RATE_LIMIT=False
XENAPI_PASSWORD=${GUEST_PASSWORD}
 
# Explicitly set virt driver here                       
 
 
VIRT_DRIVER=xenserver
 
VIRT_DRIVER=xenserver
# Explicitly set multi-host                                   
+
 
 +
# Cinder settings
 +
VOLUME_BACKING_FILE_SIZE=10000M
 +
 
 +
# Networking
 +
MGT_IP="dhcp"
 +
 
 +
PUB_IP=172.24.4.10
 +
PUB_NETMASK=255.255.255.0
 +
 
 +
# Expose OpenStack services on management interface
 +
HOST_IP_IFACE=eth2
 +
 
 +
# OpenStack VM settings
 +
OSDOMU_MEM_MB=4096
 +
UBUNTU_INST_RELEASE=precise
 +
UBUNTU_INST_IFACE="eth2"
 +
OSDOMU_VDI_GB=40
 +
 
 +
# Exercise settings
 +
ACTIVE_TIMEOUT=500
 +
TERMINATE_TIMEOUT=500
 +
 
 +
# DevStack settings
 +
LOGFILE=/tmp/devstack/log/stack.log
 +
SCREEN_LOGDIR=/tmp/devstack/log/
 +
VERBOSE=False
 +
 
 +
# XenAPI specific
 +
XENAPI_CONNECTION_URL="http://$XENSERVER_IP"
 +
VNCSERVER_PROXYCLIENT_ADDRESS="$XENSERVER_IP"
 +
 
 +
# Custom branches
 +
QUANTUM_REPO=https://github.com/$GITHUB_USER/quantum.git
 +
QUANTUM_BRANCH=$quantum_branch
 +
Q_PLUGIN=openvswitch
 
MULTI_HOST=False
 
MULTI_HOST=False
# Give extra time for boot                               
+
ENABLED_SERVICES+=,tempest,quantum,q-svc,q-agt,q-dhcp,q-l3,q-meta,q-domua,-n-net
ACTIVE_TIMEOUT=360
 
# Interface on which you would like to access services       
 
HOST_IP_IFACE=eth3
 
NETINSTALLIP="dhcp"
 
NAMESERVERS=""
 
NETMASK=""
 
GATEWAY=""
 
  
# Service overrides                                           
+
# Disable security groups
# Add quantum, remove nova network                         
+
Q_USE_SECGROUP=False
ENABLED_SERVICES+=,quantum,q-svc,q-agt,q-dhcp,q-l3,-n-net
 
# Enable tempest                                           
 
ENABLED_SERVICES+=,tempest
 
  
# Disable rate limit to support tempest                     
+
# With XenServer single box install, VLANs need to be enabled
API_RATE_LIMIT=False
+
ENABLE_TENANT_VLANS="True"
</nowiki></pre>
+
OVS_VLAN_RANGES="physnet1:1000:1024"
 +
 
 +
CLEAN_TEMPLATES=true
 +
 
 +
Q_USE_DEBUG_COMMAND=True
 +
LOCALRC_CONTENT_ENDS_HERE
 +
 
 +
cd tools/xen
 +
./install_os_domU.sh
 +
EOF
 +
</source>
  
 
== Execute devstack ==
 
== Execute devstack ==

Revision as of 10:44, 31 May 2013

Install XCP

Installation has been tested with XCP 1.6, which can be downloaded here (http://www.xen.org/download/xcp/index_1.6.0.html).

Presuming installation in a virtual machine:

  • ensure that nested virtualization is configured
  • ensure that 2 hard disks of type scsi are available
    • 1 for xcp installation (10gb dynamic allocation suggested)
    • 1 for vm storage (40gb dynamic allocation suggested)

NOTE - OpenStack requires that virtual machine storage be ext. To specify this, select "Enable thin provisioning (Optimised storage for XenDesktop)".

NOTE - The following setup assumes that the XCP machine is connected to a network with a DHCP server, on its eth0. This network will be used as a Management network for the OpenStack installation

Configure a Storage Repository

To configure a storage repository, login as root to the dom0 host and execute the following:

## List the machine’s UUID
$ xe host-list

## Create an EXT storage repository (SR) with thin provisioning support and direct access to the VHD files.  
# This example uses /dev/sdb, but this may differ for your host.
$ xe sr-create host-uuid=<hit TAB should match UUID above> content-type=user name-label=”Local EXT3 SR” shared=false device-config:device=/dev/sdb type=ext

## Set the SR to use thin provisioning. Otherwise, nova won’t be able to find the SR when it needs to write to disk.
# Look for 'Local EXT3 SR', making note of the associated UUID.
$ xe sr-list
$ xe sr-param-set uuid=<SR uuid> other-config:i18n-key=local-storage

## Set the configured SR as the default.
$ xe pool-list
$ xe pool-param-set uuid=<hit TAB> default-SR=<SR uuid>

Install git in dom0 (optional)

Installing software in dom0 is generally frowned upon, but will make it easier to ensure a devstack working tree in dom0. Copying the devstack tree from another host is also an option.

To install git, execute the following commands in dom0:

## Configure epel repo 
$ rpm --upgrade --replacepkgs http://dl.fedoraproject.org/pub/epel/5/i386/epel-release-5-4.noarch.rpm

## Disable the epel repos, just to be safe
$ sed -i 's/enabled=1/enabled=0/' /etc/yum.repos.d/epel.repo

## Install git
$ yum --enablerepo=epel -y install git

Install OpenStack

Execute the following script on your workstation. The script will cherry-pick the required changes, and push a branch to that user's repo. The github user must have a fork of openstack-dev/devstack and openstack/quantum. Please note, that the script below is a hardcopy of this one (infrastructure - proxy specific details removed): CI test script.

#!/bin/bash

set -eu

function print_usage_and_die
{
cat >&2 << EOF
usage: $0 XENSERVER_IP XENSERVER_PASS

A simple script to setup a XenServer installation with Quantum.

positional arguments:
 XENSERVER_IP     The IP address of the XenServer
 XENSERVER_PASS   The root password for the XenServer
 GITHUB_USER      The github user to use for temporary branches

An example run:

$0 10.219.10.25 mypassword citrix-openstack
EOF
exit 1
}

XENSERVER_IP="${1-$(print_usage_and_die)}"
XENSERVER_PASS="${2-$(print_usage_and_die)}"
GITHUB_USER="${3-$(print_usage_and_die)}"

set -eux

function create_branch() {
    local source_repo
    local target_repo
    local branchname

    source_repo="$1"
    target_repo="$2"
    branchname="$3"

    local tmpdir

    branchname=$(date +%s)

    tmpdir=$(mktemp -d)
    (
        cd $tmpdir
        git clone "$source_repo" repo
        cd repo
        git checkout -b "$branchname"
        git remote add target_repo "$target_repo"

        ( echo "set -exu"; cat ) | bash -s --
        git push target_repo "$branchname"
    )
    rm -rf "$tmpdir"
}

# Create custom devstack branch
devstack_branch=$(date +%s)
create_branch \
    "https://github.com/openstack-dev/devstack.git" \
    "git@github.com:$GITHUB_USER/devstack.git" \
    "$devstack_branch" << EOF
git fetch https://review.openstack.org/openstack-dev/devstack refs/changes/16/30416/3 && git cherry-pick FETCH_HEAD
git fetch https://review.openstack.org/openstack-dev/devstack refs/changes/03/30703/1 && git cherry-pick FETCH_HEAD
git fetch https://review.openstack.org/openstack-dev/devstack refs/changes/35/31035/3 && git cherry-pick FETCH_HEAD
EOF

# Create custom quantum branch
quantum_branch=$(date +%s)
create_branch \
    "https://github.com/openstack/quantum.git" \
    "git@github.com:$GITHUB_USER/quantum.git" \
    "$quantum_branch" << EOF
git fetch https://review.openstack.org/openstack/quantum refs/changes/77/31077/1 && git cherry-pick FETCH_HEAD
EOF

ssh -q \
    -o Batchmode=yes \
    -o StrictHostKeyChecking=no \
    -o UserKnownHostsFile=/dev/null \
    "root@$XENSERVER_IP" bash -s -- << EOF
set -exu
rm -rf "devstack-$devstack_branch"
wget -qO - https://github.com/$GITHUB_USER/devstack/archive/$devstack_branch.tar.gz |
    tar -xzf -
cd "devstack-$devstack_branch"

cat << LOCALRC_CONTENT_ENDS_HERE > localrc
# Passwords
MYSQL_PASSWORD=citrix
SERVICE_TOKEN=citrix
ADMIN_PASSWORD=citrix
SERVICE_PASSWORD=citrix
RABBIT_PASSWORD=citrix
GUEST_PASSWORD=citrix
XENAPI_PASSWORD="$XENSERVER_PASS"
SWIFT_HASH="66a3d6b56c1f479c8b4e70ab5c2000f5"

# Tempest
DEFAULT_INSTANCE_TYPE="m1.small"

# Compute settings
EXTRA_OPTS=("xenapi_disable_agent=True")
API_RATE_LIMIT=False
VIRT_DRIVER=xenserver

# Cinder settings
VOLUME_BACKING_FILE_SIZE=10000M

# Networking
MGT_IP="dhcp"

PUB_IP=172.24.4.10
PUB_NETMASK=255.255.255.0

# Expose OpenStack services on management interface
HOST_IP_IFACE=eth2

# OpenStack VM settings
OSDOMU_MEM_MB=4096
UBUNTU_INST_RELEASE=precise
UBUNTU_INST_IFACE="eth2"
OSDOMU_VDI_GB=40

# Exercise settings
ACTIVE_TIMEOUT=500
TERMINATE_TIMEOUT=500

# DevStack settings
LOGFILE=/tmp/devstack/log/stack.log
SCREEN_LOGDIR=/tmp/devstack/log/
VERBOSE=False

# XenAPI specific
XENAPI_CONNECTION_URL="http://$XENSERVER_IP"
VNCSERVER_PROXYCLIENT_ADDRESS="$XENSERVER_IP"

# Custom branches
QUANTUM_REPO=https://github.com/$GITHUB_USER/quantum.git
QUANTUM_BRANCH=$quantum_branch
Q_PLUGIN=openvswitch
MULTI_HOST=False
ENABLED_SERVICES+=,tempest,quantum,q-svc,q-agt,q-dhcp,q-l3,q-meta,q-domua,-n-net

# Disable security groups
Q_USE_SECGROUP=False

# With XenServer single box install, VLANs need to be enabled
ENABLE_TENANT_VLANS="True"
OVS_VLAN_RANGES="physnet1:1000:1024"

CLEAN_TEMPLATES=true

Q_USE_DEBUG_COMMAND=True
LOCALRC_CONTENT_ENDS_HERE

cd tools/xen
./install_os_domU.sh
EOF

Execute devstack

To invoke devstack on XCP, execute the following in dom0:

$ cd devstack/tools/xen
$ ./install_os_domU.sh

The initial run of devstack may take a long time (depending on the available internet connection), since devstack has to create a new VM (domU) and install ubuntu on it before installing OpenStack. Devstack saves the domU VM as a template after the first install, so subsequent runs will be much faster. If issues occur during installation (an unfortunately common occurrence when XCP is running in a VM), the XenCenter tool (unfortunately Windows-only) will aid in troubleshooting the problem.

Devstack should provide instructions once installation has completed as to how to log in to the domU instance, which will be the equivalent of a regular devstack'd host.

Verify with tempest

After a successful devstack run using the provided localrc, tempest can be used to verify that Quantum+OVS+XCP can correctly provision networking to OpenStack VM's. To run the quantum smoke tests, execute the following in domU:

$ cd /opt/stack/tempest 
$ nosetests tempest/tests/network/test_network_basic_ops.py