XenServer/Development
XenServer Development
#!wiki red/solid Note: you can now use DevStack to get started with XenServer. For more details see: [[XenXCPAndXenServer]]
<<TableOfContents()>>
Getting Started Notes
XenServer requires a nova-compute domU to run on the hypervisor in order to inject images into the VHDs. Nova-compute is the worker for the Hypervisor. It pulls jobs and applies them to the hypervisor it resides on. It's required to run on the hypervisor because it utilizes /sys/hypervisor.
Prepare Nova
Install dependencies
sudo apt-get install python-software-properties sudo add-apt-repository ppa:nova-core/trunk sudo apt-get update sudo apt-get install git-core mysql-server build-essential rabbitmq-server unzip swig screen parted curl euca2ools python-pip python-dev python-mysqldb libxml2 libxslt-dev sudo pip install xenapi
Setup development branch
mkdir ~/openstack && cd ~/openstack git clone https://github.com/openstack/nova.git sudo pip install -r nova/tools/pip-requires
You may run into issues when when running the sudo pip install -r nova/tools/pip-requires
command with an error similar to this on Ubuntu 10.04.2 LTS:
<backtrace> VersionConflict: (M2Crypto 0.20.1 (/usr/lib/pymodules/python2.6), Requirement.parse('M2Crypto==0.20.2')) Storing complete log in ./pip-log.txt
This is because euca2ools and cloud-utils install an older version of M2Crypto. You can solve this by installing the required version:
wget http://repo.vexxhost.com/openstack/m2crypto/python-m2crypto_0.20.2-1_amd64.deb sudo dpkg -i python-m2crypto_0.20.2-1_amd64.deb
You can rerun the sudo pip install -r nova/tools/pip-requires
command and it will install with no problems afterwards
Install Glance
Note: The glance-scrubber.conf file may be missing from some packages, refer to https://bugs.launchpad.net/ubuntu/+source/glance/+bug/816972.
https://github.com/openstack/glance.git cd glance sudo cp etc/glance-*.conf ~ # set filesystem_store_datadir to somewhere writable (/home/name/openstack/images/glance perhaps) in glance-api.conf # set sql_connection to MySQL (mysql://root:password@127.0.0.1/glance) in glance-registry.conf and glance-scrubber.conf sudo python setup.py install
Setup MySQL
mysql -u root -p -e "create database nova; create database glance;"
Configure Nova flags
cd nova # xenapi_inject_image == false if using an ovf type glance file. otherwise set to true to inject the bits into the image cat > ../nova.conf << EOF --verbose --nodaemon --sql_connection=mysql://root:<password>@127.0.0.1/nova --network_manager=nova.network.manager.FlatManager --image_service=nova.image.glance.GlanceImageService --flat_network_bridge=xenbr0 --connection_type=xenapi --xenapi_connection_url=https://<XenServer IP> --xenapi_connection_username=root --xenapi_connection_password=password --reboot_timeout=600 --rescue_timeout=86400 --resize_confirm_window=86400 --allow_admin_api --enabled_apis=osapi_compute --allow_resize_to_same_host --xenapi_generate_swap --firewall_driver=nova.virt.xenapi.firewall.Dom0IptablesFirewallDriver EOF # Add the following to your flagfile if you're using Ubuntu Maverick cat >> ../nova.conf << EOF --xenapi_remap_vbd_dev EOF sudo mkdir /etc/nova sudo ln -s ~/openstack/nova.conf /etc/nova/
Ensure tests pass
./run_tests.sh -x -N
Test XenAPI
./bin/nova-manage shell python import XenAPI import nova.virt.xenapi_conn nova.virt.xenapi_conn.XenAPI = XenAPI x = nova.virt.xenapi_conn.XenAPIConnection("https://<XenServer IP>", "root", "password") x.list_instances()
Setup Nova
cd nova/nova/CA && ./genrootca.sh && cd ../.. NAME=<name> ./bin/nova-manage db sync ./bin/nova-manage user admin $NAME ./bin/nova-manage project create openstack $NAME ./bin/nova-manage network create private 10.0.0.0/8 1 64 [0 0 <IPv6 Prefix>::/<Mask> <IPv6 Gateway>] ./bin/nova-manage project zip openstack $NAME unzip nova.zip . novarc
Enable Legacy Auth
vim etc/nova/api-paste.ini # set [pipeline:openstack_compute_api_v2] # pipeline = faultwrap noauth ratelimit osapi_compute_app_v2 # to # [pipeline:openstack_compute_api_v2] # pipeline = faultwrap auth ratelimit osapi_compute_app_v2
Run Nova
screen # each process gets its own window ./bin/nova-api ./bin/nova-compute ./bin/nova-network ./bin/nova-scheduler
Setup Glance
sudo mkdir /var/log/glance sudo glance-manage db_sync
Run Glance
sudo glance-control all start
Prepare XenServer
Install Nova plugins
cd ~/openstack/nova/plugins/xenserver/xenapi scp -r etc/ root@<XenServer IP>:/ ssh root@<XenServer IP> chmod a+x /etc/xapi.d/plugins/* sed -i -e "s/enabled=0/enabled=1/" /etc/yum.repos.d/CentOS-Base.repo yum install parted mkdir /boot/guest sed -i -e "s/enabled=1/enabled=0/" /etc/yum.repos.d/CentOS-Base.repo
Configure SR storage
ssh root@<XenServer IP> xe sr-list # look for the 'Local storage' SR ssh root@<XenServer IP> xe sr-param-set uuid=<SR uuid> other-config:i18n-key=local-storage
<Provisional message: placeholder update when Nova E4 releases> From Nova Essex 4, a new flag called 'sr_matching_filter' has been introduced (which defaults to 'other-config:i18n-key=local-storage'). This filter is used for finding the SR on which to install guest instances. The default value is the Local Storage in default XenServer/XCP installations as detailed above. However, if you want to select an SR with different matching criteria, this flag can be set to 'other-config:my_favorite_sr=true'. This means that XenAPI will look for an SR whose other-config contains the key-value pair (my_favorite_sr, true). On the other hand, to fall back on the Default SR, as displayed by XenCenter and as returned by xenapi.pool.get_default_SR, this flag can be set to to 'default-sr:true'.
Instance Management
Prepare Images
mkdir -p ~/openstack/images && cd ~/openstack/images mkdir glance
AMI Images
mkdir lucid_ami && cd lucid_ami wget -q -O - http://173.203.107.207/ubuntu-lucid.tar | tar xSv glance add name=ramdisk disk_format=ari container_format=ari is_public=True < initrd.img-2.6.32-23-server glance add name=kernel disk_format=aki container_format=aki is_public=True < vmlinuz-2.6.32-23-server glance add name=lucid_ami disk_format=ami container_format=ami is_public=True ramdisk_id=<ramdisk ID> kernel_id=<kernel ID> < ubuntu-lucid.img
OVF Images
Linux
mkdir linux_ovf && cd linux_ovf glance add name=lucid_ovf disk_format=vhd container_format=ovf is_public=True < lucid.ova
Windows
mkdir windows && cd windows glance add name=windows disk_format=vhd container_format=ovf type=raw os_type=windows is_public=True < windows.ova
Run Instance
ssh-keygen -f nova_key && chmod 600 nova_key euca-add-keypair nova_key > nova_key.priv nova flavor-list nova image-list nova boot test --flavor <flavor ID> --image <image ID> nova list
Connect to instance console
VNC
ssh root@<XenServer IP> netstat -tuanp | grep vncterm ssh -L 590N:localhost:590N root@<XenServer IP> vnc://localhost:590N
XenConsole
ssh root@<XenServer IP> list_domains ps ax | grep vnc | grep <domain ID> kill -9 <VNC PIDs> /usr/lib/xen/bin/xenconsole <domain ID>
Install XenTools
Make xs-tools.iso available to the instance in XenCenter, OpenXenManager or the CLI (not documented here) before proceeding.
ssh root@<instance IP> mount /dev/xvdb /mnt cd /mnt/Linux dpkg -i ./xe-guest-utilities_5.5.0-464_amd64.deb update-rc.d -f xe-linux-distribution remove update-rc.d xe-linux-distribution defaults reboot
Install OpenXenManager on OS X
#!wiki note The rev48 "packaged" tarball [[http://sourceforge.net/projects/openxenmanager/forums/forum/1159068/topic/4529278|may be broken]] -- SVN trunk worked for me, with the Homebrew dependency installation method I've documented below. ''--ChesMartin''
- Download: http://xensemaking.com/openxenmanager/
- Compile pygtk:
sudo port install py26-gtk
If you're a Homebrew user, you're unfortunately in for a more work due in part to some dependencies not being available as formula yet. You can try this procedure for building PyGTK and dependencies.
1.#3 Open a X11 shell and go to the openxenmanager directory
- Run OpenXenManager:
python window.py
The project README
also mentions python-gtk-vnc
and rrdtool
for graphs -- neither path here takes those into account so you'll need to look into them if you wish to have those features working.