Jump to: navigation, search


Running OpenShift on Heat


OpenShift is a Platform as a Service (PaaS) application platform.

The guide below can be used to demonstrate launching a single-instance OpenShift implementation (a broker and node on one instance), via a heat template.

Note, initially we had a template which allowed us to build the broker and node from scratch (see Openshift.template), but this takes a very long time (nearly 1 hour) on typical development machines, and recent migration of openshift code repositories on github makes it hard to build from source (crankcase->origin-server move, Rakefile etc are now gone so we need a new build procedure and the [openshift wiki build-from-source](https://openshift.redhat.com/community/wiki/build-openshift-origin-from-source) page is outdated/non-working)

So the alternative approach described below is to build a manually-created JEOS image, based on the live instructions on the OpenShift Wiki

This approach is clearly suboptimal, but at least allows for a simple demo capability - in time we can work on getting the build-from-source and install-from-rpm approaches working (when the OpenShift repository move stabilizes and procedure for installing from the RPM repository becomes clearer/works)

Getting Started guide to running the OpenShift example template

Create a jeos image containing the core OpenShift installation

Note the procedure below is manual, we could probably automate this by bootstrapping the livecd into liveinst mode then customizing via oz, ie use the oz "advanced" mode. Not sure exactly how to do this at present though, so the manual steps are documented below:

1 - Install live image as per OpenShift Wiki

# Boot the livecd as an install image for a VM with a disk image
sudo su -
# reboot, create a test user
sudo sed -i -e '/gpgkey=/aexclude=ruby ruby-devel ruby-irb ruby-libs ruby-rdoc' /etc/yum.repos.d/fedora-updates.repo
sudo chkconfig httpd on
sudo service httpd start
sudo chkconfig stickshift-broker on
sudo service stickshift-broker start
sudo chkconfig --del livesys
sudo chkconfig --del livesys-late
sudo chkconfig --del livesys-late-openshift
# reboot (or the rhc app create will fail)
#Test via rhc domain create, rhc app create

2 - Allow access to named on the OpenShift guest

So you can use the nameserver on the OpenShift broker it is necessary to open up UDP port 53 on the guest:

iptables -A INPUT -m state --state NEW -m udp -p udp --dport 53 -j ACCEPT

To make this persistent, add this line to /etc/sysconfig/iptables, so it looks like this:

# Manually hacked iptables file..
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT
-A INPUT -m state --state NEW -m udp -p udp --dport 53 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dport 443 -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -j REJECT --reject-with icmp-host-prohibited

Now you can add the IP of the OpenShift guest to the host's /etc/resolv.conf so you can resolve application-domain names on the host. Note you'll need to remove it when the guest is powered off to avoid slowing down your other DNS queries.

3 - Copy the heat-jeos cfntools onto the guest

ssh onto the guest, create the directory-tree and scp the files on from the host (assumes root allowed via ssh to the guest, is the IP of the VM):

sh root@
mkdir -p /opt/aws/bin

On the host, scp the cfntools onto the guest:

cd /home/shardy/git/heat-jeos/heat_jeos/cfntools/
scp cfn* root@
scp boto.cfg root@

4 - Do similar customization to heat-jeos (ref the F16 tdl)

/usr/sbin/useradd ec2-user
echo -e 'ec2-user\tALL=(ALL)\tNOPASSWD: ALL' >> /etc/sudoers
yum -y install yum-plugin-fastestmirror cloud-init python-psutil python-boto
sed --in-place -e s/Type=oneshot/"Type=oneshot\nTimeoutSec=0"/ /lib/systemd/system/cloud-final.service
chmod +x /opt/aws/bin/cfn-*

Note I avoid doing the yum update here, as I was worried it might break the openshift install

5 - Remove some not needed packages to reduce the image size (could probably prune a lot more here, but this is the list specified on the openshift wiki..)

yum remove aisleriot brasero cheese deja-dup duplicity empathy eog festival gnome-backgrounds gnome-contacts gnome-games icedtea-web libpurple orca shotwell simple-scan tracker transmission-common vinagre vino·

6 - Remove domain/app (if created)

So that we can create the domain/app based on heat template parameters, remove the example domain/app if created ref the openshift wiki:

rhc app stop -l admin -p admin -a knockknock
rhc app destroy -l admin -p admin -a knockknock
rhc domain destroy -n minipaas -l admin -p admin

7 - Remove the "shifter" user (if created), copy the .openshift config dir to /root

cp -r ~shifter/.openstack /root
userdel shifter

8 - Set the system to boot to runlevel 3 (or whatever systemd calls it nowadays..;)

ls -l /etc/systemd/system/default.target
rm /etc/systemd/system/default.target
ln -s /lib/systemd/system/multi-user.target /etc/systemd/system/default.target

9 - halt & poweroff the guest

10 - Make a sparse copy to be imported into glance

(Also this means if necessary you can power up and modify the original "gold image" we've just created independent of the version imported into glance as it is left unmodified)

export TMPDIR=/var/tmp
virt-sparsify --compress openshift_F16_live2disk.qcow2 openshift_F16_live2disk-cfntools-sparse.qcow2

This saves a lot of space! (and will make the instance launch faster as less disk I/O)

# du -h openshift_F16_live2disk*
1.8G    openshift_F16_live2disk-cfntools-sparse.qcow2
7.0G    openshift_F16_live2disk.qcow2

11 - Import the copy into glance

glance add name=openshift_F16_live2disk-cfntools is_public=true disk_format=qcow2 container_format=bare < /var/lib/libvirt/images/openshift_F16_live2disk-cfntools-sparse.qcow2

Create the OpenShift stack

Note you specify the application and domain as template parameters.

Also note the template below is not currently in the heat git repository - I created this gist : https://gist.github.com/4024749

heat -d create openshift2 --template-file=templates/Openshift_Single_Instance.template --parameters="InstanceType=m1.xlarge;KeyName=${USER}_key;DomainName=cat;ApplicationName=mouse"

Establish the IP address of the OpenShift broker (which we need for DNS) and website URL

heat describe openshift
            <Description>Broker IP (for DNS)</Description>
            <Description>Website URL</Description>

Modify your DNS configuration to use the openshift broker for name resolution

Temporarily hack the host resolv.conf:

cp /etc/resolv.conf /etc/resolv.conf.bak
echo "nameserver" > /etc/resolv.conf

Check that the name is resolving correctly:

# host  mouse-cat.example.com
mouse-cat.example.com is an alias for broker.example.com.example.com.
broker.example.com.example.com has address

View the example OpenShift application in a browser


TODO : Screenshot, you can see my demo screencast: http://people.redhat.com/shardy/heat/london_developerday/openshift_heat_demo.ogv

Modify the application

Now you can pull a git repo containing the application, as the user with the KeyName SSH key parameter specified at stack create time (without needing any openshift credentials, heat has registered the key as part of the stack launch)

Note heat has inserted the GITURL into the index page after creating the app, so as in the screenshot above you can cut/paste this into your terminal to do the clone:

git clone ssh://0d57d86357894c6e9df3f37216ce2d03@mouse-cat.example.com/~/git/mouse.git/
cd mouse/
vim php/index.php 
git commit -a -m "this is a test"
git push

Then reload your browser, and observe the page update with whatever modification you made to index.php!