Difference between revisions of "Obsolete:NovaInstall/CentOSNotes"
m (Fifieldt moved page NovaInstall/CentOSNotes to Obsolete:NovaInstall/CentOSNotes: page is veerrrrry outdated) |
|||
(26 intermediate revisions by 6 users not shown) | |||
Line 1: | Line 1: | ||
− | |||
− | |||
− | |||
− | The principle bottleneck for running nova on CentOS is Python 2.6. Nova is written in Python 2.6 and CentOS 5 | + | __TOC__ |
+ | |||
+ | == CentOS 6/ RHEL 6 == | ||
+ | |||
+ | See [[NovaInstall/RHEL6Notes]] | ||
+ | |||
+ | == CentOS 5/ RHEL 5 / Oracle Enterprise Linux 5 == | ||
+ | |||
+ | === Preface === | ||
+ | The principle bottleneck for running nova on CentOS 5 is Python 2.6. Nova is written in Python 2.6 and CentOS 5 comes with python 2.4. We can not update python system wide as some core utilities (like yum) is dependent on python 2.4. Also very few python 2.6 modules are available in centos/epel repos. | ||
+ | |||
+ | You will also need to get libvirt version 0.8+. | ||
+ | |||
+ | === Installation === | ||
'''Step 1: Get pre-reqs''' Add euca2ools and EPEL repo first. | '''Step 1: Get pre-reqs''' Add euca2ools and EPEL repo first. | ||
− | + | ==== EPEL Repositiory ==== | |
<pre><nowiki> | <pre><nowiki> | ||
Line 17: | Line 27: | ||
EUCA_REPO_CONF_EOF | EUCA_REPO_CONF_EOF | ||
− | |||
− | |||
− | + | rpm -Uvh 'http://download.fedora.redhat.com/pub/epel/5/x86_64/epel-release-5-4.noarch.rpm' | |
− | rpm -Uvh 'http://download.fedora.redhat.com/pub/epel/5/ | ||
</nowiki></pre> | </nowiki></pre> | ||
'''Step 2: Now install python2.6, kvm and few other libraries through yum''' | '''Step 2: Now install python2.6, kvm and few other libraries through yum''' | ||
− | + | ==== Dependencies & Core Packages ==== | |
<pre><nowiki> | <pre><nowiki> | ||
− | yum -y install dnsmasq vblade kpartx kvm gawk iptables ebtables bzr screen euca2ools curl rabbitmq-server gcc gcc-c++ autoconf automake swig openldap openldap-servers | + | yum -y install dnsmasq vblade kpartx kvm gawk iptables ebtables bzr screen euca2ools curl rabbitmq-server gcc gcc-c++ autoconf automake swig openldap openldap-servers python26 python26-devel python26-distribute git openssl-devel python26-tools mysql-server qemu kmod-kvm libxml2 libxslt libxslt-devel mysql-devel libvirt |
+ | rpm -ivh ftp://ftp.pbone.net/mirror/ftp.freshrpms.net/pub/freshrpms/pub/dag/redhat/el5/en/x86_64/RPMS.dag/swig-1.3.25-1.el5.rf.x86_64.rpm | ||
</nowiki></pre> | </nowiki></pre> | ||
'''Step 3: Then download the latest aoetools and then build (and install) it''' You also need to check for the latest version on sourceforge because the exact url will change if there's a new release. | '''Step 3: Then download the latest aoetools and then build (and install) it''' You also need to check for the latest version on sourceforge because the exact url will change if there's a new release. | ||
− | + | ==== AOE Tools ==== | |
<pre><nowiki> | <pre><nowiki> | ||
Line 43: | Line 51: | ||
'''Step 4: Add the udev rules for aoetools''' | '''Step 4: Add the udev rules for aoetools''' | ||
− | + | ===== udev Rules ===== | |
<pre><nowiki> | <pre><nowiki> | ||
Line 57: | Line 65: | ||
'''Step 5: Load the kernel modules and python modules''' | '''Step 5: Load the kernel modules and python modules''' | ||
− | + | ==== Kernel Modules ==== | |
<pre><nowiki> | <pre><nowiki> | ||
modprobe aoe | modprobe aoe | ||
+ | modprobe kvm | ||
+ | modprobe nbd | ||
</nowiki></pre> | </nowiki></pre> | ||
+ | |||
+ | '''Step 6: Fixing qemu-img''' | ||
+ | ==== qemu ==== | ||
+ | Stock Nova uses a newer version of qemu-img which uses different command line options. To avoid having to rebuild the whole qemu-kvm binary we need to add following wrapper file for qemu-img | ||
<pre><nowiki> | <pre><nowiki> | ||
− | + | #!/bin/sh | |
+ | mv /usr/bin/qemu-img /usr/bin/qemu-img.bin | ||
+ | cat > /usr/bin/qemu-img <<EOF | ||
+ | |||
+ | ARGS="\$*" | ||
+ | |||
+ | CHANGED_ARGS=`echo \$* | sed "s/-o cluster_size=2M,backing_file=/-b /g"` | ||
+ | |||
+ | /usr/bin/qemu-img.bin \$CHANGED_ARGS | ||
+ | EOF | ||
</nowiki></pre> | </nowiki></pre> | ||
Line 72: | Line 95: | ||
<pre><nowiki> | <pre><nowiki> | ||
− | easy_install-2.6 twisted sqlalchemy mox greenlet carrot python-daemon eventlet tornado IPy routes lxml MySQL-python sphinx | + | easy_install-2.6 twisted sqlalchemy mox greenlet carrot python-daemon eventlet tornado \ |
+ | IPy routes lxml MySQL-python sphinx webob netaddr paste pastedeploy sqlalchemy-migrate \ | ||
+ | glance Cheetah python-novaclient | ||
easy_install-2.6 python-daemon==1.5.5 | easy_install-2.6 python-daemon==1.5.5 | ||
easy_install-2.6 lockfile==0.8 | easy_install-2.6 lockfile==0.8 | ||
− | easy_install-2.6 | + | easy_install-2.6 boto==1.9b |
+ | easy_install-2.6 python-gflags==1.4 | ||
</nowiki></pre> | </nowiki></pre> | ||
− | + | Ensure all libraries installs correct, for some reasons some libs may not be. | |
Do the same for python2.6-libxml2 module, notice the --with-python and --prefix flags. --with-python ensures we are building it against python2.6 (otherwise it will build against python2.4, which is default). | Do the same for python2.6-libxml2 module, notice the --with-python and --prefix flags. --with-python ensures we are building it against python2.6 (otherwise it will build against python2.4, which is default). | ||
Line 94: | Line 120: | ||
</nowiki></pre> | </nowiki></pre> | ||
− | You will also need [[M2Crypto]]. Unfortunately you will need to make a | + | You will also need [[M2Crypto]]. Unfortunately you will need to make a change to opensslconf.h |
<pre><nowiki> | <pre><nowiki> | ||
− | + | sed -i 's_opensslconf-\(.*\)_/usr/include/openssl/opensslconf-\1_' /usr/include/openssl/opensslconf.h | |
− | easy_install-2.6 | + | easy_install-2.6 M2Crypto==0.20.2 |
</nowiki></pre> | </nowiki></pre> | ||
+ | Last but not least you will need truncate which for some reason is not included with Centos 5 coreutils. To install execute following | ||
+ | |||
+ | |||
+ | <pre><nowiki> | ||
+ | wget http://ftp.gnu.org/gnu/coreutils/coreutils-8.9.tar.gz | ||
+ | tar -zxvf coreutils-8.9.tar.gz | ||
+ | cd coreutils-8.9 | ||
+ | ./configure | ||
+ | make | ||
+ | sudo cp src/truncate /usr/bin/ | ||
+ | </nowiki></pre> | ||
Now you should have a system that is ready to install Nova from a BZR branch (you can use yum to install the bzr package: su -c 'yum install bzr'). | Now you should have a system that is ready to install Nova from a BZR branch (you can use yum to install the bzr package: su -c 'yum install bzr'). | ||
− | Once you are done installing bzr you can use it to check out the latest branch. I | + | ==== Branch Checkout ==== |
+ | |||
+ | Once you are done installing bzr you can use it to check out the latest branch. I qwill check it out to /opt/nova directory e.g. | ||
Line 111: | Line 150: | ||
bzr branch lp:nova /opt/nova | bzr branch lp:nova /opt/nova | ||
</nowiki></pre> | </nowiki></pre> | ||
− | |||
Once you do the cd into /opt/nova directory and type | Once you do the cd into /opt/nova directory and type | ||
Line 120: | Line 158: | ||
</nowiki></pre> | </nowiki></pre> | ||
− | + | === Configuration === | |
Once that is done you will need to copy the stock nova-api.conf file from /opt/nova/etc/nova-api.conf to /etc/nova ie. | Once that is done you will need to copy the stock nova-api.conf file from /opt/nova/etc/nova-api.conf to /etc/nova ie. | ||
Line 127: | Line 165: | ||
cp /opt/nova/etc/nova-api.conf /etc/nova | cp /opt/nova/etc/nova-api.conf /etc/nova | ||
</nowiki></pre> | </nowiki></pre> | ||
+ | |||
+ | You also need to copy contents of /opt/nova/CA to /var/lib/nova/CA/ | ||
+ | |||
+ | |||
+ | <pre><nowiki> | ||
+ | mkdir -p /var/lib/nova/CA | ||
+ | rsync -av /opt/nova/nova/CA/ /var/lib/nova/CA/ | ||
+ | </nowiki></pre> | ||
+ | |||
+ | ==== MySQL ==== | ||
+ | Make sure you supply flagfile argument as otherwise nova-manage looks in the Nova bin directory for nova.conf | ||
+ | |||
+ | |||
+ | <pre><nowiki> | ||
+ | nova-manage --flagfile /etc/nova/nova.conf db sync | ||
+ | </nowiki></pre> | ||
+ | |||
+ | ==== Start services ==== | ||
+ | To start up services I use following script (borrowed from nova.sh) | ||
+ | |||
+ | |||
+ | <pre><nowiki> | ||
+ | export PATH=/usr/bin:/bin:/sbin:/usr/sbin | ||
+ | |||
+ | function screen_it { | ||
+ | screen -S nova -X screen -t $1 | ||
+ | screen -S nova -p $1 -X stuff "$2$NL" | ||
+ | } | ||
+ | screen -d -m -S nova -t nova | ||
+ | sleep 1 | ||
+ | |||
+ | # Opens up screen with following commands. You will have to go screen by screen | ||
+ | # and press RETURN to start them | ||
+ | screen_it api "/usr/bin/nova-api --nodaemon --flagfile /etc/nova/nova.conf" | ||
+ | screen_it objectstore "/usr/bin/nova-objectstore --nodaemon --flagfile /etc/nova/nova.conf" | ||
+ | screen_it compute "/usr/bin/nova-compute --nodaemon --flagfile /etc/nova/nova.conf" | ||
+ | screen_it network "/usr/bin/nova-network --nodaemon --flagfile /etc/nova/nova.conf" | ||
+ | screen_it scheduler "/usr/bin/nova-scheduler --nodaemon --flagfile /etc/nova/nova.conf" | ||
+ | #screen_it volume "/usr/bin/nova-volume --nodaemon --flagfile /etc/nova/nova.conf" | ||
+ | #screen_it test ". /etc/nova/novarc" | ||
+ | screen -S nova -x | ||
+ | </nowiki></pre> | ||
+ | |||
+ | I recommend adding following line to /etc/screenrc | ||
+ | |||
+ | |||
+ | <pre><nowiki> | ||
+ | hardstatus alwayslastline "%-Lw%{= BW}%50>%n%f* %t%{-}%+Lw%< %= %H" | ||
+ | </nowiki></pre> | ||
+ | |||
+ | It will show the name of all windows at the bottom of the screen | ||
+ | |||
+ | == Troubleshooting == | ||
+ | If you get an error similar to this | ||
+ | |||
+ | |||
+ | <pre><nowiki> | ||
+ | No such file or directory: '/usr/local/lib/python2.6/dist-packages/nova-2011.1-py2.6.egg/nova/..//CA' | ||
+ | </nowiki></pre> | ||
+ | |||
+ | Add | ||
+ | |||
+ | |||
+ | <pre><nowiki> | ||
+ | --ca_path=/var/lib/nova/CA/ | ||
+ | </nowiki></pre> | ||
+ | |||
+ | in /etc/nova/nova.conf and make sure you followed the step for rsyncing contents of CA directory. | ||
+ | |||
+ | == Network Setup == | ||
+ | Sample [[VlanNetworkSetup|VLAN Network Setup]] |
Latest revision as of 04:05, 18 October 2013
Contents
CentOS 6/ RHEL 6
CentOS 5/ RHEL 5 / Oracle Enterprise Linux 5
Preface
The principle bottleneck for running nova on CentOS 5 is Python 2.6. Nova is written in Python 2.6 and CentOS 5 comes with python 2.4. We can not update python system wide as some core utilities (like yum) is dependent on python 2.4. Also very few python 2.6 modules are available in centos/epel repos.
You will also need to get libvirt version 0.8+.
Installation
Step 1: Get pre-reqs Add euca2ools and EPEL repo first.
EPEL Repositiory
cat >/etc/yum.repos.d/euca2ools.repo << EUCA_REPO_CONF_EOF [eucalyptus] name=euca2ools baseurl=http://www.eucalyptussoftware.com/downloads/repo/euca2ools/1.3.1/yum/centos/ enabled=1 gpgcheck=0 EUCA_REPO_CONF_EOF rpm -Uvh 'http://download.fedora.redhat.com/pub/epel/5/x86_64/epel-release-5-4.noarch.rpm'
Step 2: Now install python2.6, kvm and few other libraries through yum
Dependencies & Core Packages
yum -y install dnsmasq vblade kpartx kvm gawk iptables ebtables bzr screen euca2ools curl rabbitmq-server gcc gcc-c++ autoconf automake swig openldap openldap-servers python26 python26-devel python26-distribute git openssl-devel python26-tools mysql-server qemu kmod-kvm libxml2 libxslt libxslt-devel mysql-devel libvirt rpm -ivh ftp://ftp.pbone.net/mirror/ftp.freshrpms.net/pub/freshrpms/pub/dag/redhat/el5/en/x86_64/RPMS.dag/swig-1.3.25-1.el5.rf.x86_64.rpm
Step 3: Then download the latest aoetools and then build (and install) it You also need to check for the latest version on sourceforge because the exact url will change if there's a new release.
AOE Tools
wget -c http://sourceforge.net/projects/aoetools/files/aoetools/32/aoetools-32.tar.gz/download tar -zxvf aoetools-32.tar.gz cd aoetools-32 make make install
Step 4: Add the udev rules for aoetools
udev Rules
cat > /etc/udev/rules.d/60-aoe.rules << AOE_RULES_EOF SUBSYSTEM=="aoe", KERNEL=="discover", NAME="etherd/%k", GROUP="disk", MODE="0220" SUBSYSTEM=="aoe", KERNEL=="err", NAME="etherd/%k", GROUP="disk", MODE="0440" SUBSYSTEM=="aoe", KERNEL=="interfaces", NAME="etherd/%k", GROUP="disk", MODE="0220" SUBSYSTEM=="aoe", KERNEL=="revalidate", NAME="etherd/%k", GROUP="disk", MODE="0220" # aoe block devices KERNEL=="etherd*", NAME="%k", GROUP="disk" AOE_RULES_EOF
Step 5: Load the kernel modules and python modules
Kernel Modules
modprobe aoe modprobe kvm modprobe nbd
Step 6: Fixing qemu-img
qemu
Stock Nova uses a newer version of qemu-img which uses different command line options. To avoid having to rebuild the whole qemu-kvm binary we need to add following wrapper file for qemu-img
#!/bin/sh mv /usr/bin/qemu-img /usr/bin/qemu-img.bin cat > /usr/bin/qemu-img <<EOF ARGS="\$*" CHANGED_ARGS=`echo \$* | sed "s/-o cluster_size=2M,backing_file=/-b /g"` /usr/bin/qemu-img.bin \$CHANGED_ARGS EOF
Now, install the python modules using easy_install-2.6, this ensures the installation are done against python 2.6
easy_install-2.6 twisted sqlalchemy mox greenlet carrot python-daemon eventlet tornado \ IPy routes lxml MySQL-python sphinx webob netaddr paste pastedeploy sqlalchemy-migrate \ glance Cheetah python-novaclient easy_install-2.6 python-daemon==1.5.5 easy_install-2.6 lockfile==0.8 easy_install-2.6 boto==1.9b easy_install-2.6 python-gflags==1.4
Ensure all libraries installs correct, for some reasons some libs may not be. Do the same for python2.6-libxml2 module, notice the --with-python and --prefix flags. --with-python ensures we are building it against python2.6 (otherwise it will build against python2.4, which is default).
wget -c "ftp://xmlsoft.org/libxml2/libxml2-2.7.3.tar.gz" tar -zxvf libxml2-2.7.3.tar.gz cd libxml2-2.7.3 ./configure --with-python=/usr/bin/python26 --prefix=/usr make all make install cd python python2.6 setup.py install cd ..
You will also need M2Crypto. Unfortunately you will need to make a change to opensslconf.h
sed -i 's_opensslconf-\(.*\)_/usr/include/openssl/opensslconf-\1_' /usr/include/openssl/opensslconf.h easy_install-2.6 M2Crypto==0.20.2
Last but not least you will need truncate which for some reason is not included with Centos 5 coreutils. To install execute following
wget http://ftp.gnu.org/gnu/coreutils/coreutils-8.9.tar.gz tar -zxvf coreutils-8.9.tar.gz cd coreutils-8.9 ./configure make sudo cp src/truncate /usr/bin/
Now you should have a system that is ready to install Nova from a BZR branch (you can use yum to install the bzr package: su -c 'yum install bzr').
Branch Checkout
Once you are done installing bzr you can use it to check out the latest branch. I qwill check it out to /opt/nova directory e.g.
bzr branch lp:nova /opt/nova
Once you do the cd into /opt/nova directory and type
python26 setup.py install
Configuration
Once that is done you will need to copy the stock nova-api.conf file from /opt/nova/etc/nova-api.conf to /etc/nova ie.
cp /opt/nova/etc/nova-api.conf /etc/nova
You also need to copy contents of /opt/nova/CA to /var/lib/nova/CA/
mkdir -p /var/lib/nova/CA rsync -av /opt/nova/nova/CA/ /var/lib/nova/CA/
MySQL
Make sure you supply flagfile argument as otherwise nova-manage looks in the Nova bin directory for nova.conf
nova-manage --flagfile /etc/nova/nova.conf db sync
Start services
To start up services I use following script (borrowed from nova.sh)
export PATH=/usr/bin:/bin:/sbin:/usr/sbin function screen_it { screen -S nova -X screen -t $1 screen -S nova -p $1 -X stuff "$2$NL" } screen -d -m -S nova -t nova sleep 1 # Opens up screen with following commands. You will have to go screen by screen # and press RETURN to start them screen_it api "/usr/bin/nova-api --nodaemon --flagfile /etc/nova/nova.conf" screen_it objectstore "/usr/bin/nova-objectstore --nodaemon --flagfile /etc/nova/nova.conf" screen_it compute "/usr/bin/nova-compute --nodaemon --flagfile /etc/nova/nova.conf" screen_it network "/usr/bin/nova-network --nodaemon --flagfile /etc/nova/nova.conf" screen_it scheduler "/usr/bin/nova-scheduler --nodaemon --flagfile /etc/nova/nova.conf" #screen_it volume "/usr/bin/nova-volume --nodaemon --flagfile /etc/nova/nova.conf" #screen_it test ". /etc/nova/novarc" screen -S nova -x
I recommend adding following line to /etc/screenrc
hardstatus alwayslastline "%-Lw%{= BW}%50>%n%f* %t%{-}%+Lw%< %= %H"
It will show the name of all windows at the bottom of the screen
Troubleshooting
If you get an error similar to this
No such file or directory: '/usr/local/lib/python2.6/dist-packages/nova-2011.1-py2.6.egg/nova/..//CA'
Add
--ca_path=/var/lib/nova/CA/
in /etc/nova/nova.conf and make sure you followed the step for rsyncing contents of CA directory.
Network Setup
Sample VLAN Network Setup