XenServer/Development

= XenServer Development =

Getting Started Notes
#!wiki green/solid To get started using XenServer, take a look at the XenServer Getting Started Guide.

XenServer requires nova-compute runs in a para-virtualized (PV) Linux VM (domU not dom0) on the hypervisor you want to manage. This is required to inject images into the VHDs. It pulls jobs and applies them to the hypervisor it resides on. It's required to run on the hypervisor because it utilizes /sys/hypervisor, and needs to attach virtual disks, so nova can populate them with the required data.

Prepare Images
mkdir -p ~/openstack/images && cd ~/openstack/images mkdir glance

AMI Images
mkdir lucid_ami && cd lucid_ami

wget -q -O - http://173.203.107.207/ubuntu-lucid.tar | tar xSv

glance add name=ramdisk disk_format=ari container_format=ari is_public=True < initrd.img-2.6.32-23-server glance add name=kernel disk_format=aki container_format=aki is_public=True < vmlinuz-2.6.32-23-server glance add name=lucid_ami disk_format=ami container_format=ami is_public=True ramdisk_id= kernel_id= < ubuntu-lucid.img

Linux
mkdir linux_ovf && cd linux_ovf

glance add name=lucid_ovf disk_format=vhd container_format=ovf is_public=True < lucid.ova

Windows
mkdir windows && cd windows

glance add name=windows disk_format=vhd container_format=ovf type=raw os_type=windows is_public=True < windows.ova

Run Instance
ssh-keygen -f nova_key && chmod 600 nova_key euca-add-keypair nova_key > nova_key.priv

nova flavor-list nova image-list nova boot test --flavor  --image  nova list

VNC
ssh root@ netstat -tuanp | grep vncterm ssh -L 590N:localhost:590N root@ vnc://localhost:590N

XenConsole
ssh root@ list_domains ps ax | grep vnc | grep  kill -9  /usr/lib/xen/bin/xenconsole 

Install XenTools
Make xs-tools.iso available to the instance in XenCenter, OpenXenManager or the CLI (not documented here) before proceeding.

ssh root@ mount /dev/xvdb /mnt cd /mnt/Linux dpkg -i ./xe-guest-utilities_5.5.0-464_amd64.deb update-rc.d -f xe-linux-distribution remove update-rc.d xe-linux-distribution defaults reboot

Install OpenXenManager on OS X
#!wiki note The rev48 "packaged" tarball [|may be broken] -- SVN trunk worked for me, with the Homebrew dependency installation method I've documented below. --ChesMartin


 * 1) Download: http://xensemaking.com/openxenmanager/
 * 2) Compile pygtk:

sudo port install py26-gtk

If you're a Homebrew user, you're unfortunately in for a more work due in part to some dependencies not being available as formula yet. You can try this procedure for building PyGTK and dependencies.

1.#3 Open a X11 shell and go to the openxenmanager directory
 * 1) Run OpenXenManager:

python window.py

The project  also mentions   and   for graphs -- neither path here takes those into account so you'll need to look into them if you wish to have those features working.

Install Nova plugins
cd ~/openstack/nova/plugins/xenserver/xenapi scp -r etc/ root@:/

ssh root@

chmod a+x /etc/xapi.d/plugins/* sed -i -e "s/enabled=0/enabled=1/" /etc/yum.repos.d/CentOS-Base.repo yum install parted mkdir /boot/guest sed -i -e "s/enabled=1/enabled=0/" /etc/yum.repos.d/CentOS-Base.repo

Configure SR storage
ssh root@ xe sr-list # look for the 'Local storage' SR ssh root@ xe sr-param-set uuid= other-config:i18n-key=local-storage

<Provisional message: placeholder update when Nova E4 releases> From Nova Essex 4, a new flag called 'sr_matching_filter' has been introduced (which defaults to 'other-config:i18n-key=local-storage'). This filter is used for finding the SR on which to install guest instances. The default value is the Local Storage in default XenServer/XCP installations as detailed above. However, if you want to select an SR with different matching criteria, this flag can be set to 'other-config:my_favorite_sr=true'. This means that XenAPI will look for an SR whose other-config contains the key-value pair (my_favorite_sr, true). On the other hand, to fall back on the Default SR, as displayed by XenCenter and as returned by xenapi.pool.get_default_SR, this flag can be set to to 'default-sr:true'.

Legacy way to Prepare Nova
This is how you can manually install nova. It is probably better to try DevStack.

Install dependencies
sudo apt-get update

sudo apt-get install git-core mysql-server build-essential rabbitmq-server unzip swig screen parted curl euca2ools python-pip python-dev python-mysqldb libxml2 libxslt-dev libmysqlclient-dev libpq-dev bash-completion

sudo pip install xenapi

Setup development branch
mkdir ~/openstack && cd ~/openstack

git clone https://git.openstack.org/openstack/nova.git sudo pip install -r nova/requirements.txt

Install Glance
Note: The glance-scrubber.conf file may be missing from some packages, refer to https://bugs.launchpad.net/ubuntu/+source/glance/+bug/816972.

https://github.com/git.openstack.org/glance.git cd glance sudo cp etc/glance-*.conf ~ sudo python setup.py install
 * 1) set filesystem_store_datadir to somewhere writable (/home/name/openstack/images/glance perhaps) in glance-api.conf
 * 2) set sql_connection to MySQL (mysql://root:password@127.0.0.1/glance) in glance-registry.conf and glance-scrubber.conf

Setup MySQL
mysql -u root -p -e "create database nova; create database glance;"

Configure Nova flags
cd nova


 * 1) xenapi_inject_image == false if using an ovf type glance file.  otherwise set to true to inject the bits into the image

cat > ../nova.conf << EOF [DEFAULT] api_paste_config=/etc/nova/api-paste.ini debug=True verbose=True sql_connection=mysql://root: @127.0.0.1/nova network_manager=nova.network.manager.FlatManager flat_network_bridge=xenbr0 xenapi_connection_url=https://<XenServer IP> xenapi_connection_username=root xenapi_connection_password=password reboot_timeout=600 rescue_timeout=86400 resize_confirm_window=86400 enabled_apis=osapi_compute allow_resize_to_same_host=True firewall_driver=nova.virt.xenapi.firewall.Dom0IptablesFirewallDriver glance_api_servers=http://<glance_ip>:9292 EOF

cat >> ../nova.conf << EOF xenapi_remap_vbd_dev=True EOF
 * 1) Add the following to your flagfile if you're using Ubuntu Maverick

sudo mkdir /etc/nova sudo ln -s <PATH>/nova.conf /etc/nova/

Symlink api-paste.ini into /etc/nova Symlink policy.json into /etc/nova

Ensure tests pass
./run_tests.sh -x -N

Setup Nova
cd nova/nova/CA && ./genrootca.sh && cd ../..

NAME=

./bin/nova-manage db sync ./bin/nova-manage user admin $NAME ./bin/nova-manage project create openstack $NAME ./bin/nova-manage network create private 10.0.0.0/8 1 64 [0 0 <IPv6 Prefix>::/<Mask> <IPv6 Gateway>] ./bin/nova-manage project zip openstack $NAME

unzip nova.zip . novarc

Enable Legacy Auth
vim etc/nova/api-paste.ini
 * 1) set [pipeline:openstack_compute_api_v2]
 * 2) pipeline = faultwrap noauth ratelimit osapi_compute_app_v2
 * to
 * 1) [pipeline:openstack_compute_api_v2]
 * 2) pipeline = faultwrap auth ratelimit osapi_compute_app_v2

Run Nova
screen # each process gets its own window

./bin/nova-api ./bin/nova-compute ./bin/nova-conductor ./bin/nova-network ./bin/nova-scheduler

Configure Glance
Make sure glance-registry.conf has sql_connection=mysql://root@127.0.0.1/glance

Setup Glance
sudo mkdir /var/log/glance sudo glance-manage db_sync

Run Glance
sudo glance-control all start

Configure Novaclient
Create a novarc file and source it with follows

export OS_USERNAME=admin export OS_PASSWORD=admin export OS_TENANT_NAME=openstack export OS_AUTH_URL=http://127.0.0.1:8774/v2

Test with Novaclient
$ nova list ++--+++-+--+ ++--+++-+--+ ++--+++-+--+
 * ID | Name | Status | Task State | Power State | Networks |