Jump to: navigation, search

Obsolete:NovaInstall/CentOSNotes

CentOS 6/ RHEL 6

See NovaInstall/RHEL6Notes

CentOS 5/ RHEL 5 / Oracle Enterprise Linux 5

Preface

The principle bottleneck for running nova on CentOS 5 is Python 2.6. Nova is written in Python 2.6 and CentOS 5 comes with python 2.4. We can not update python system wide as some core utilities (like yum) is dependent on python 2.4. Also very few python 2.6 modules are available in centos/epel repos.

You will also need to get libvirt version 0.8+.

Installation

Step 1: Get pre-reqs Add euca2ools and EPEL repo first.

EPEL Repositiory

cat >/etc/yum.repos.d/euca2ools.repo << EUCA_REPO_CONF_EOF
[eucalyptus]
name=euca2ools
baseurl=http://www.eucalyptussoftware.com/downloads/repo/euca2ools/1.3.1/yum/centos/
enabled=1
gpgcheck=0

EUCA_REPO_CONF_EOF

rpm -Uvh 'http://download.fedora.redhat.com/pub/epel/5/x86_64/epel-release-5-4.noarch.rpm'

Step 2: Now install python2.6, kvm and few other libraries through yum

Dependencies & Core Packages

yum -y  install dnsmasq  vblade kpartx kvm gawk iptables ebtables  bzr screen euca2ools  curl rabbitmq-server gcc gcc-c++ autoconf automake swig  openldap openldap-servers python26 python26-devel python26-distribute git openssl-devel  python26-tools mysql-server qemu kmod-kvm libxml2 libxslt libxslt-devel mysql-devel libvirt
rpm -ivh ftp://ftp.pbone.net/mirror/ftp.freshrpms.net/pub/freshrpms/pub/dag/redhat/el5/en/x86_64/RPMS.dag/swig-1.3.25-1.el5.rf.x86_64.rpm

Step 3: Then download the latest aoetools and then build (and install) it You also need to check for the latest version on sourceforge because the exact url will change if there's a new release.

AOE Tools

    wget -c http://sourceforge.net/projects/aoetools/files/aoetools/32/aoetools-32.tar.gz/download
    tar -zxvf aoetools-32.tar.gz
    cd aoetools-32
    make
    make install

Step 4: Add the udev rules for aoetools

udev Rules
cat > /etc/udev/rules.d/60-aoe.rules << AOE_RULES_EOF
SUBSYSTEM=="aoe", KERNEL=="discover",    NAME="etherd/%k", GROUP="disk", MODE="0220"
SUBSYSTEM=="aoe", KERNEL=="err",    NAME="etherd/%k", GROUP="disk", MODE="0440"
SUBSYSTEM=="aoe", KERNEL=="interfaces",    NAME="etherd/%k", GROUP="disk", MODE="0220"
SUBSYSTEM=="aoe", KERNEL=="revalidate",    NAME="etherd/%k", GROUP="disk", MODE="0220"
# aoe block devices
KERNEL=="etherd*",       NAME="%k", GROUP="disk"
AOE_RULES_EOF

Step 5: Load the kernel modules and python modules

Kernel Modules

    modprobe aoe
    modprobe kvm
    modprobe nbd

Step 6: Fixing qemu-img

qemu

Stock Nova uses a newer version of qemu-img which uses different command line options. To avoid having to rebuild the whole qemu-kvm binary we need to add following wrapper file for qemu-img


#!/bin/sh
mv /usr/bin/qemu-img /usr/bin/qemu-img.bin
cat > /usr/bin/qemu-img <<EOF

ARGS="\$*"

CHANGED_ARGS=`echo \$* | sed "s/-o cluster_size=2M,backing_file=/-b /g"`

/usr/bin/qemu-img.bin  \$CHANGED_ARGS
EOF

Now, install the python modules using easy_install-2.6, this ensures the installation are done against python 2.6


easy_install-2.6 twisted sqlalchemy mox greenlet carrot python-daemon eventlet tornado \
  IPy routes lxml MySQL-python sphinx webob netaddr paste pastedeploy sqlalchemy-migrate \
  glance Cheetah python-novaclient
easy_install-2.6 python-daemon==1.5.5
easy_install-2.6 lockfile==0.8
easy_install-2.6 boto==1.9b
easy_install-2.6 python-gflags==1.4

Ensure all libraries installs correct, for some reasons some libs may not be. Do the same for python2.6-libxml2 module, notice the --with-python and --prefix flags. --with-python ensures we are building it against python2.6 (otherwise it will build against python2.4, which is default).


 wget -c "ftp://xmlsoft.org/libxml2/libxml2-2.7.3.tar.gz"
    tar -zxvf libxml2-2.7.3.tar.gz
    cd libxml2-2.7.3
    ./configure --with-python=/usr/bin/python26 --prefix=/usr
    make all
    make install
    cd python
    python2.6 setup.py install
    cd ..

You will also need M2Crypto. Unfortunately you will need to make a change to opensslconf.h


sed -i  's_opensslconf-\(.*\)_/usr/include/openssl/opensslconf-\1_'  /usr/include/openssl/opensslconf.h
easy_install-2.6 M2Crypto==0.20.2

Last but not least you will need truncate which for some reason is not included with Centos 5 coreutils. To install execute following


wget http://ftp.gnu.org/gnu/coreutils/coreutils-8.9.tar.gz
tar -zxvf coreutils-8.9.tar.gz
cd coreutils-8.9
./configure
make
sudo cp src/truncate /usr/bin/

Now you should have a system that is ready to install Nova from a BZR branch (you can use yum to install the bzr package: su -c 'yum install bzr').

Branch Checkout

Once you are done installing bzr you can use it to check out the latest branch. I qwill check it out to /opt/nova directory e.g.


bzr branch lp:nova /opt/nova

Once you do the cd into /opt/nova directory and type


python26 setup.py install

Configuration

Once that is done you will need to copy the stock nova-api.conf file from /opt/nova/etc/nova-api.conf to /etc/nova ie.


cp /opt/nova/etc/nova-api.conf /etc/nova

You also need to copy contents of /opt/nova/CA to /var/lib/nova/CA/


mkdir -p /var/lib/nova/CA
rsync -av /opt/nova/nova/CA/ /var/lib/nova/CA/

MySQL

Make sure you supply flagfile argument as otherwise nova-manage looks in the Nova bin directory for nova.conf


nova-manage --flagfile /etc/nova/nova.conf db sync

Start services

To start up services I use following script (borrowed from nova.sh)


export PATH=/usr/bin:/bin:/sbin:/usr/sbin

function screen_it {
    screen -S nova -X screen -t $1
    screen -S nova -p $1 -X stuff "$2$NL"
}
   screen -d -m -S nova -t nova
    sleep 1

# Opens up screen with following commands. You will have to go screen by screen
# and press RETURN to start them
screen_it api "/usr/bin/nova-api --nodaemon --flagfile /etc/nova/nova.conf"
screen_it objectstore "/usr/bin/nova-objectstore --nodaemon --flagfile /etc/nova/nova.conf"
screen_it compute "/usr/bin/nova-compute --nodaemon --flagfile /etc/nova/nova.conf"
screen_it network "/usr/bin/nova-network --nodaemon --flagfile /etc/nova/nova.conf"
screen_it scheduler "/usr/bin/nova-scheduler --nodaemon --flagfile /etc/nova/nova.conf"
#screen_it volume "/usr/bin/nova-volume --nodaemon --flagfile /etc/nova/nova.conf"
#screen_it test ". /etc/nova/novarc"
screen -S nova -x

I recommend adding following line to /etc/screenrc


hardstatus alwayslastline "%-Lw%{= BW}%50>%n%f* %t%{-}%+Lw%< %= %H"

It will show the name of all windows at the bottom of the screen

Troubleshooting

If you get an error similar to this


No such file or directory: '/usr/local/lib/python2.6/dist-packages/nova-2011.1-py2.6.egg/nova/..//CA'

Add


--ca_path=/var/lib/nova/CA/

in /etc/nova/nova.conf and make sure you followed the step for rsyncing contents of CA directory.

Network Setup

Sample VLAN Network Setup