Heat/Running-openshift

= Running OpenShift on Heat =

Overview
OpenShift is a Platform as a Service (PaaS) application platform.

The guide below can be used to demonstrate launching a single-instance OpenShift implementation (a broker and node on one instance), via a heat template.

Note, initially we had a template which allowed us to build the broker and node from scratch (see Openshift.template), but this takes a very long time (nearly 1 hour) on typical development machines, and recent migration of openshift code repositories on github makes it hard to build from source (crankcase->origin-server move, Rakefile etc are now gone so we need a new build procedure and the [openshift wiki build-from-source](https://openshift.redhat.com/community/wiki/build-openshift-origin-from-source) page is outdated/non-working)

So the alternative approach described below is to build a manually-created JEOS image, based on the live instructions on the OpenShift Wiki

This approach is clearly suboptimal, but at least allows for a simple demo capability - in time we can work on getting the build-from-source and install-from-rpm approaches working (when the OpenShift repository move stabilizes and procedure for installing from the RPM repository becomes clearer/works)

Create a jeos image containing the core OpenShift installation
Note the procedure below is manual, we could probably automate this by bootstrapping the livecd into liveinst mode then customizing via oz, ie use the oz "advanced" mode. Not sure exactly how to do this at present though, so the manual steps are documented below:

1 - Install live image as per OpenShift Wiki
xhost+ sudo su - liveinst sudo sed -i -e '/gpgkey=/aexclude=ruby ruby-devel ruby-irb ruby-libs ruby-rdoc' /etc/yum.repos.d/fedora-updates.repo sudo chkconfig httpd on sudo service httpd start sudo chkconfig stickshift-broker on sudo service stickshift-broker start sudo chkconfig --del livesys sudo chkconfig --del livesys-late sudo chkconfig --del livesys-late-openshift
 * 1) Boot the livecd as an install image for a VM with a disk image
 * 1) reboot, create a test user
 * 1) reboot (or the rhc app create will fail)
 * 2) Test via rhc domain create, rhc app create

2 - Allow access to named on the OpenShift guest
So you can use the nameserver on the OpenShift broker it is necessary to open up UDP port 53 on the guest:

iptables -A INPUT -m state --state NEW -m udp -p udp --dport 53 -j ACCEPT

To make this persistent, add this line to /etc/sysconfig/iptables, so it looks like this:


 * 1) Manually hacked iptables file..
 * filter
 * INPUT ACCEPT [0:0]
 * FORWARD ACCEPT [0:0]
 * OUTPUT ACCEPT [0:0]

-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT -A INPUT -p icmp -j ACCEPT -A INPUT -i lo -j ACCEPT -A INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT -A INPUT -m state --state NEW -m udp -p udp --dport 53 -j ACCEPT -A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT -A INPUT -m state --state NEW -m tcp -p tcp --dport 443 -j ACCEPT -A INPUT -j REJECT --reject-with icmp-host-prohibited -A FORWARD -j REJECT --reject-with icmp-host-prohibited COMMIT

Now you can add the IP of the OpenShift guest to the host's /etc/resolv.conf so you can resolve application-domain names on the host. Note you'll need to remove it when the guest is powered off to avoid slowing down your other DNS queries.

3 - Copy the heat-jeos cfntools onto the guest
ssh onto the guest, create the directory-tree and scp the files on from the host (assumes root allowed via ssh to the guest, 192.168.122.214 is the IP of the VM):

sh root@192.168.122.214 mkdir -p /opt/aws/bin

On the host, scp the cfntools onto the guest:

cd /home/shardy/git/heat-jeos/heat_jeos/cfntools/ scp cfn* root@192.168.122.214:/opt/aws/bin/ scp boto.cfg root@192.168.122.214:/etc

4 - Do similar customization to heat-jeos (ref the F16 tdl)
/usr/sbin/useradd ec2-user echo -e 'ec2-user\tALL=(ALL)\tNOPASSWD: ALL' >> /etc/sudoers yum -y install yum-plugin-fastestmirror cloud-init python-psutil python-boto sed --in-place -e s/Type=oneshot/"Type=oneshot\nTimeoutSec=0"/ /lib/systemd/system/cloud-final.service chmod +x /opt/aws/bin/cfn-*

Note I avoid doing the yum update here, as I was worried it might break the openshift install

5 - Remove some not needed packages to reduce the image size (could probably prune a lot more here, but this is the list specified on the openshift wiki..)
yum remove aisleriot brasero cheese deja-dup duplicity empathy eog festival gnome-backgrounds gnome-contacts gnome-games icedtea-web libpurple orca shotwell simple-scan tracker transmission-common vinagre vino·

6 - Remove domain/app (if created)
So that we can create the domain/app based on heat template parameters, remove the example domain/app if created ref the openshift wiki:

rhc app stop -l admin -p admin -a knockknock rhc app destroy -l admin -p admin -a knockknock rhc domain destroy -n minipaas -l admin -p admin

7 - Remove the "shifter" user (if created), copy the .openshift config dir to /root
cp -r ~shifter/.openstack /root userdel shifter

8 - Set the system to boot to runlevel 3 (or whatever systemd calls it nowadays..;)
ls -l /etc/systemd/system/default.target rm /etc/systemd/system/default.target ln -s /lib/systemd/system/multi-user.target /etc/systemd/system/default.target

10 - Make a sparse copy to be imported into glance
(Also this means if necessary you can power up and modify the original "gold image" we've just created independent of the version imported into glance as it is left unmodified)

export TMPDIR=/var/tmp virt-sparsify --compress openshift_F16_live2disk.qcow2 openshift_F16_live2disk-cfntools-sparse.qcow2

This saves a lot of space! (and will make the instance launch faster as less disk I/O)

1.8G   openshift_F16_live2disk-cfntools-sparse.qcow2 7.0G   openshift_F16_live2disk.qcow2
 * 1) du -h openshift_F16_live2disk*

11 - Import the copy into glance
glance add name=openshift_F16_live2disk-cfntools is_public=true disk_format=qcow2 container_format=bare < /var/lib/libvirt/images/openshift_F16_live2disk-cfntools-sparse.qcow2

Create the OpenShift stack
Note you specify the application and domain as template parameters.

Also note the template below is not currently in the heat git repository - I created this gist : https://gist.github.com/4024749

heat -d create openshift2 --template-file=templates/Openshift_Single_Instance.template --parameters="InstanceType=m1.xlarge;KeyName=${USER}_key;DomainName=cat;ApplicationName=mouse"

Establish the IP address of the OpenShift broker (which we need for DNS) and website URL
heat describe openshift ...      BrokerIP 192.168.250.3 Broker IP (for DNS) WebsiteURL http://mouse-cat.example.com Website URL 

Modify your DNS configuration to use the openshift broker for name resolution
Temporarily hack the host resolv.conf:

cp /etc/resolv.conf /etc/resolv.conf.bak echo "nameserver 192.168.250.3" > /etc/resolv.conf

Check that the name is resolving correctly:

mouse-cat.example.com is an alias for broker.example.com.example.com. broker.example.com.example.com has address 192.168.250.3
 * 1) host  mouse-cat.example.com

View the example OpenShift application in a browser
http://mouse-cat.example.com/

TODO : Screenshot, you can see my demo screencast: http://people.redhat.com/shardy/heat/london_developerday/openshift_heat_demo.ogv

Modify the application
Now you can pull a git repo containing the application, as the user with the KeyName SSH key parameter specified at stack create time (without needing any openshift credentials, heat has registered the key as part of the stack launch)

Note heat has inserted the GITURL into the index page after creating the app, so as in the screenshot above you can cut/paste this into your terminal to do the clone:

git clone ssh://0d57d86357894c6e9df3f37216ce2d03@mouse-cat.example.com/~/git/mouse.git/ cd mouse/ vim php/index.php git commit -a -m "this is a test" git push

Then reload your browser, and observe the page update with whatever modification you made to index.php!

Resources
https://openshift.redhat.com/community/wiki/build-your-own-paas-from-the-openshift-origin-livecd-using-liveinst

https://openshift.redhat.com/community/wiki/build-multi-node-paas-from-scratch

http://www.krishnaraman.net/openshift-origin-on-openstack/

http://fedoraproject.org/wiki/Features/OpenShift_Origin

http://www.slideshare.net/fallenpegasus/openshift-openstack-fedora-awesome