Jump to: navigation, search


Revision as of 20:08, 19 February 2015 by StackScribe (talk | contribs) (Keystone Identity services)

(Moved original to the bottom of the page for reference.)

Proposed Revision

Strategy and assumptions:

  1. Audience is people who have some experience installing OpenStack, not first time users
  2. Focus on installation of OpenStack core services
  3. Structure the guide sequentially -- the steps to take in a reasonable order
  4. Avoid redundancy with the Install Guide; for steps that are identical for HA and non-HA installations, link to appropriate sections in the Install Guide
  5. One guide for all Linux distros/platforms
  6. Emphasize a reasonable, standard deployment based on open source components. We can provide some notes about alternatives as appropriate (for example, using a commercial load-balancer might be a better alternative than relying on HAProxy) and perhaps a link to the OpenStack Marketplace.


HA Intro and Concepts

  1. Redundancy and failover
  2. Stateless/stateful, active/passive, active/active
  3. Quorums; many services should use an odd number of nodes equal to or greater than 3
  4. Single-controller HA mode and scaling up to 3 or more

Hardware setup

  1. Minimal Architecture Example -- Network Layout, styled as in http://docs.openstack.org/juno/install-guide/install/apt/content/ch_basic_environment.html#basics-prerequisites for easy comparison


  1. Link to Install Guide: Install O/S on each node
  2. Install pacemaker, crmsh, corosync, cluster-glue, fence-agents (Fedora only), resource-agents.
  3. Set up and start Corosync and Pacemaker. Stick with 'crm' tool for Ubuntu/Debian and 'pcs' for RHEL/Fedora
  4. Set basic cluster properties
  5. Configure fencing for Pacemaker cluster (Links to http://clusterlabs.org/doc/)
  6. Configure the VIP
  7. API services
  8. Schedulers
  9. Memcached service

Configure networking on each node

  1. Link to Networking Guide
  2. (Neutron agents should be described for active/active; deprecate single agent's instances case)
  3. For Kilo and beyond, focus on L3HA and DVR

Install and Configure MySQL

  1. Two nodes plus GARBD.
  2. MySQL with Galera
  3. Pacemaker multistate clone resource for Galera cluster
  4. Pacemaker resource agent for Galera cluster management
  5. Deprecate MySQL DRBD configuration because of split-brain issues

RabbitMQ Message broker

  1. Oslo messaging for active/active
  2. No need for active/passive AMQP; Two-node active/active cluster with mirrored queues instead
  3. Pacemaker multistate clone resource for RabbitMQ cluster
  4. Pacemaker resource agent for RabbitMQ cluster management
  5. Deprecate DRBD for RabbitMQ


  1. Link to Install Guide

Keystone Identity services

  1. Install Guide: basic installation, register each service to Keystone
  2. Configure Keystone for HA MySQL and HA RabbitMQ
  3. Add Keystone resource to Pacemaker
  4. Change bind parameters in keystone.conf
  5. Configure OpenStack services to use HA Keystone

Cinder Block Storage Service

Swift Object Storage

Heat Orchestration

  1. Link to Install Guide for basics
  2. How to set up so that VMs on a failed compute node are quickly migrated to other compute nodes

Ceilometer Telemetry and MongoDB

Database Service (Trove)


  1. Configure Pacemaker service group to ensure that the VIP is linked to the API services resource
  2. Systemd alternative to OCF scripts for Pacemaker RA
  3. MariaDB with Galera alternative to MySQL
  4. Install and configure HAProxy for API services and MySQL with Galera cluster load balancing
  5. Mention value of redundant hardware load balancers for stateless services such as REST APIs
  6. Describe scaling single node to 3 nodes HA
  7. Ceph?
  8. Murano?

Original for reference

NOTE: This is the original for us to depart from.

I. Introduction to OpenStack High Availability

  1. Stateless vs. Stateful services
  2. Active/Passive
  3. Active/Active

II. HA Using Active/Passive

1. The Pacemaker Cluster Stack

  1. Installing Packages
  2. Setting up Corosync
  3. Starting Corosync
  4. Starting Pacemaker
  5. Setting basic cluster properties

2. Cloud Controller Cluster Stack

  1. Highly available MySQL
  2. Highly available RabbitMQ

3. API Node Cluster Stack

  1. Configure the VIP
  2. Highly available OpenStack Identity
  3. Highly available OpenStack Image API
  4. Highly available Cinder API
  5. Highly available OpenStack Networking Server
  6. Highly available Ceilometer Central Agent
  7. Configure Pacemaker Group

4. Network Controller Cluster Stack

  1. Highly available Neutron L3 Agent
  2. Highly available Neutron DHCP Agent
  3. Highly available Neutron Metadata Agent
  4. Manage network resources

III. HA Using Active/Active

5. Database

  1. MySQL with Galera
  2. Galera Monitoring Scripts
  3. Other ways to provide a Highly Available database

6. RabbitMQ

  1. Install RabbitMQ
  2. Configure RabbitMQ
  3. Configure OpenStack Services to use RabbitMQ

7. HAproxy Nodes 8. OpenStack Controller Nodes

  1. Running OpenStack API & schedulers
  2. Memcached

9. OpenStack Network Nodes

  1. Running Neutron DHCP Agent
  2. Running Neutron L3 Agent
  3. Running Neutron Metadata Agent