Revision as of 14:53, 25 April 2012

Real deployments

This page documents details of real OpenStack deployments.

Wikimedia

Contact: Ryan Lane

Documentation

• http://www.mediawiki.org/wiki/Wikimedia_Labs
• http://wikitech.wikimedia.org/view/OpenStack
• http://www.mediawiki.org/wiki/Extension:OpenStackManager
• http://ryandlane.com/blog/wp-content/uploads/2012/02/Infrastructure-as-an-Open-Source-Project-FOSDEM-publish.odp

The ODP file of the FOSDEM talk has full notes if you switch to notes view.

Deployment scripts

Puppet repository, which has OpenStack manifests (for swift and nova) and some scripts used for managing gluster, nfs and ganglia in a per-project way:

• https://gerrit.wikimedia.org/r/gitweb?p=operations/puppet.git;a=summary
• https://labsconsole.wikimedia.org/wiki/Git#Restrictions_and_Anonymous_access

Blog posts

Ryan Lane's blog about how certain things are handled:

• http://ryandlane.com/blog/2012/04/24/per-project-sudo-policies-using-sudo-ldap-and-puppet/
• http://ryandlane.com/blog/2011/11/01/sharing-home-directories-to-instances-within-a-project-using-puppet-ldap-autofs-and-nova/
• http://ryandlane.com/blog/2011/11/02/a-process-for-puppetization-of-a-service-using-nova/
• http://ryandlane.com/blog/2011/01/24/announcing-openstackmanager-extension-for-mediawiki/
• http://ryandlane.com/blog/2011/01/02/building-a-test-and-development-infrastructure-using-openstack/

Wikimedia blog about design decisions:

• http://blog.wikimedia.org/2012/04/16/introduction-to-wikimedia-labs/

Argonne National Labs (DOE Magellan)

Current Diablo environment

• ubuntu 10.11 oneiric
• Bcfg2 configuration management
• openstack Diablo via managed IT PPA
• nova network 10GigE, VLAN manager
• nova volume serivce using iscsi over ipoib
• nginx load balancer / HA (frontend for all client API connections)
• 2 x nova api servers, each with 4 instances
• glance on gluster (over native ib to compute nodes)
• keystone
• dashboard
• euca2ools via EC2 api
• 500 compute nodes
• IBM iDataplex
• 2 x 2.6 intel nehalem
• 24GB memory
• 1GigE NIC
• QDR infiniband (only used for storage atm)
• ~100 users spread across ~15 tenants

Planned Essex environment

• TBD