Jump to: navigation, search


XenServer Development

Getting Started Notes

#!wiki green/solid
To get started using XenServer, take a look at the [[XenServer/GettingStarted|XenServer Getting Started Guide]].

XenServer requires nova-compute runs in a para-virtualized (PV) Linux VM (domU not dom0) on the hypervisor you want to manage. This is required to inject images into the VHDs. It pulls jobs and applies them to the hypervisor it resides on. It's required to run on the hypervisor because it utilizes /sys/hypervisor, and needs to attach virtual disks, so nova can populate them with the required data.

Instance Management

Prepare Images

mkdir -p ~/openstack/images && cd ~/openstack/images
mkdir glance

AMI Images

mkdir lucid_ami && cd lucid_ami

wget -q -O - | tar xSv

glance add name=ramdisk disk_format=ari container_format=ari is_public=True < initrd.img-2.6.32-23-server
glance add name=kernel disk_format=aki container_format=aki is_public=True < vmlinuz-2.6.32-23-server
glance add name=lucid_ami disk_format=ami container_format=ami is_public=True ramdisk_id=<ramdisk ID> kernel_id=<kernel ID> < ubuntu-lucid.img

OVF Images

mkdir linux_ovf && cd linux_ovf

glance add name=lucid_ovf disk_format=vhd container_format=ovf is_public=True < lucid.ova

mkdir windows && cd windows

glance add name=windows disk_format=vhd container_format=ovf type=raw os_type=windows is_public=True < windows.ova

Run Instance

ssh-keygen -f nova_key && chmod 600 nova_key
euca-add-keypair nova_key > nova_key.priv

nova flavor-list
nova image-list
nova boot test --flavor <flavor ID> --image <image ID>
nova list

Connect to instance console


ssh root@<XenServer IP> netstat -tuanp | grep vncterm
ssh -L 590N:localhost:590N root@<XenServer IP>


ssh root@<XenServer IP>
ps ax | grep vnc | grep <domain ID>
kill -9 <VNC PIDs>
/usr/lib/xen/bin/xenconsole <domain ID>

Install XenTools

Make xs-tools.iso available to the instance in XenCenter, OpenXenManager or the CLI (not documented here) before proceeding.

ssh root@<instance IP>
mount /dev/xvdb /mnt
cd /mnt/Linux
dpkg -i ./xe-guest-utilities_5.5.0-464_amd64.deb
update-rc.d -f xe-linux-distribution remove
update-rc.d xe-linux-distribution defaults

Install OpenXenManager on OS X

#!wiki note
The rev48 "packaged" tarball [[http://sourceforge.net/projects/openxenmanager/forums/forum/1159068/topic/4529278|may be broken]] -- SVN trunk worked for me, with the Homebrew dependency installation method I've documented below. ''--ChesMartin''

  1. Download: http://xensemaking.com/openxenmanager/
  2. Compile pygtk:

sudo port install py26-gtk

If you're a Homebrew user, you're unfortunately in for a more work due in part to some dependencies not being available as formula yet. You can try this procedure for building PyGTK and dependencies.

1.#3 Open a X11 shell and go to the openxenmanager directory
  1. Run OpenXenManager:

python window.py

The project README also mentions python-gtk-vnc and rrdtool for graphs -- neither path here takes those into account so you'll need to look into them if you wish to have those features working.

Legacy way to Prepare XenServer

Install Nova plugins

cd ~/openstack/nova/plugins/xenserver/xenapi 
scp -r etc/ root@<XenServer IP>:/

ssh root@<XenServer IP>

chmod a+x /etc/xapi.d/plugins/*
sed -i -e "s/enabled=0/enabled=1/" /etc/yum.repos.d/CentOS-Base.repo
yum install parted
mkdir /boot/guest
sed -i -e "s/enabled=1/enabled=0/" /etc/yum.repos.d/CentOS-Base.repo

Configure SR storage

ssh root@<XenServer IP> xe sr-list  # look for the 'Local storage' SR
ssh root@<XenServer IP> xe sr-param-set uuid=<SR uuid> other-config:i18n-key=local-storage

<Provisional message: placeholder update when Nova E4 releases> From Nova Essex 4, a new flag called 'sr_matching_filter' has been introduced (which defaults to 'other-config:i18n-key=local-storage'). This filter is used for finding the SR on which to install guest instances. The default value is the Local Storage in default XenServer/XCP installations as detailed above. However, if you want to select an SR with different matching criteria, this flag can be set to 'other-config:my_favorite_sr=true'. This means that XenAPI will look for an SR whose other-config contains the key-value pair (my_favorite_sr, true). On the other hand, to fall back on the Default SR, as displayed by XenCenter and as returned by xenapi.pool.get_default_SR, this flag can be set to to 'default-sr:true'.

Legacy way to Prepare Nova

This is how you can manually install nova. It is probably better to try DevStack.

Install dependencies

sudo apt-get update

sudo apt-get install git-core mysql-server build-essential rabbitmq-server unzip swig screen parted curl euca2ools python-pip python-dev python-mysqldb libxml2 libxslt-dev libmysqlclient-dev libpq-dev bash-completion

sudo pip install xenapi

Setup development branch

mkdir ~/openstack && cd ~/openstack

git clone https://git.openstack.org/openstack/nova.git
sudo pip install -r nova/requirements.txt

Install Glance

Note: The glance-scrubber.conf file may be missing from some packages, refer to https://bugs.launchpad.net/ubuntu/+source/glance/+bug/816972.

cd glance
sudo cp etc/glance-*.conf ~
# set filesystem_store_datadir to somewhere writable (/home/name/openstack/images/glance perhaps) in glance-api.conf
# set sql_connection to MySQL (mysql://root:password@ in glance-registry.conf and glance-scrubber.conf
sudo python setup.py install

Setup MySQL

mysql -u root -p -e "create database nova; create database glance;"

Configure Nova flags

cd nova

# xenapi_inject_image == false if using an ovf type glance file.  otherwise set to true to inject the bits into the image

cat > ../nova.conf << EOF
xenapi_connection_url=https://<XenServer IP>

# Add the following to your flagfile if you're using Ubuntu Maverick
cat >> ../nova.conf << EOF

sudo mkdir /etc/nova
sudo ln -s <PATH>/nova.conf /etc/nova/

Symlink api-paste.ini into /etc/nova Symlink policy.json into /etc/nova

Ensure tests pass

./run_tests.sh -x -N

Setup Nova

cd nova/nova/CA && ./genrootca.sh && cd ../..


./bin/nova-manage db sync
./bin/nova-manage user admin $NAME
./bin/nova-manage project create openstack $NAME
./bin/nova-manage network create private 1 64 [0 0 <IPv6 Prefix>::/<Mask> <IPv6 Gateway>]
./bin/nova-manage project zip openstack $NAME

unzip nova.zip
. novarc

Enable Legacy Auth

vim etc/nova/api-paste.ini
# set [pipeline:openstack_compute_api_v2]
# pipeline = faultwrap noauth ratelimit osapi_compute_app_v2
# to
# [pipeline:openstack_compute_api_v2]
# pipeline = faultwrap auth ratelimit osapi_compute_app_v2

Run Nova

screen  # each process gets its own window


Configure Glance

Make sure glance-registry.conf has


Setup Glance

sudo mkdir /var/log/glance
sudo glance-manage db_sync

Run Glance

sudo glance-control all start

Configure Novaclient

Create a novarc file and source it with follows

export OS_USERNAME=admin
export OS_PASSWORD=admin
export OS_TENANT_NAME=openstack
export OS_AUTH_URL=

Test with Novaclient

$ nova list
| ID | Name | Status | Task State | Power State | Networks |