Heat Bootstrap Config

--steve-stevebaker (talk) 04:36, 12 September 2013 (UTC)

NOTE: This specification is still under discussion and has not been endorsed by the heat project.

Background

The purpose of bootstrapping an orchestrated server is to provide just enough package installation and configuration to be able to hand off to a full Configuration Management tool. Traditionally this has been achieved in heat templates with cfn-init from the heat-cfntools package however this has the following issues:

It needs to be invoked explicitly from the template UserData
It is limited to providing cfn compatible functionality
It acts on a single block of metadata which is associated with the instance resource, which makes it difficult to write templates that are composable (ie, templates which split the application configuration from the architecture the application is being orchestrated into).

Heat's current underlying implementation relies on cloud-init to deliver the Metadata and the UserData script to the instance on boot.

This blueprint proposes that the functionality of cloud-init be directly exposed in heat templates to fulfill bootstrap config tasks. This blueprint also proposes that this be used as a foundation to expose the features of other configuration management tools.

Relationship to HOT blueprints

Blueprint hot-software-config is flagged as depending on this blueprint. However in reality the blueprints are interdependent and need to evolve cooperatively.

For the purposes of this specification it is assumed that:

blueprint native-tools-bootstrap-config specifies the internal implementation of how HOT components are consumed by server/instance resources, and the interaction between heat and orchestrated servers
blueprint hot-software-config specifies the syntax and semantics of HOT components

However, this spec states the opinion that HOT config components should transparently expose whatever underlying bootstrap/configuration-management is being used. This has the following benefits:

There is no need to create and maintain yet another configuration management abstraction
Heat can explicitly claim that it is not itself a configuration management tool by documenting the tools it natively supports
Configuration management tools which use YAML as their native format have potential for tighter integration with Heat templates

HOT examples in this spec are meant for illustration purposes only, and the specifics may change in the final implementation.

CM tools to consider

Implementing components for some combination of the the following tools would be enough to mark this blueprint as completed:

cloud-init
heat-cfntools cfn-init
os-refresh-config / os-apply-config

Other configuration management tools could be considered for implementing a HOT component in their own blueprint. They are included in this spec to demonstrate how far the scope of this cloud-init approach could be extended. Such tools include:

Puppet
Ansible
Chef
SaltStack

OS::Heat::CloudInit

It is proposed that each subclass of Component can provide its hosted_on resources with some metadata, along with a collection of cloud-init user data that will perform the desired configuration. The CloudInit component will do this by directly exposing appropriate cloud-init user data types. This may be limited to #cloud-config files and user data scripts.

cloud-init version compatibility

Heat has been hindered in the past in attempts to use newer cloud-init features due to the older version of cloud-init used on the launched OS. It is hoped that this will be mitigated by transparently exposing cloud-init configuration into the template, so that it is up to the user to only use the features supported by the cloud-init of the server images they launch.

Stack updates

cloud-init is only for first-boot configuration. This means that updating a stack with different OS::Heat::CloudInit data cannot be applied to a running server. In light of this situation, the options are:

do not implement OS::Heat::CloudInit; only implement components for configuration management tools which can be run repeatedly for the lifecycle of the server
mark the stack update is failed if an attempt is made to change existing OS::Heat::CloudInit config data
make the server UpdateReplace if an attempt is made to change existing OS::Heat::CloudInit config data

Examples

cloud-init examples could be represented in HOT templates in the following way:

Writing out arbitrary files

components:
  the_component:
    type: OS::Heat::CloudInit
    relationships:
      - hosted_on: the_server
    config:
      cloud_config:
        write_files:
        - encoding: b64
          # Note here that the HOT get_param function is being called
          # to directly populate #cloud-config yaml
          content:
            get_param: the_file_contents
          owner: root:root
          path: /foo/bar
          permissions: '0644'

Adding a yum repository

components:
  the_component:
    type: OS::Heat::CloudInit
    relationships:
      - hosted_on: the_server
    config:
      cloud_config:
        yum_repos:
          epel-testing:
            baseurl: http://download.fedoraproject.org/pub/epel/testing/5/$basearch
            enabled: false
            failovermethod: priority
            gpgcheck: true
            gpgkey: file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL
            name: Extra Packages for Enterprise Linux 5 - Testing

Install arbitrary packages

components:
  the_component:
    type: OS::Heat::CloudInit
    relationships:
      - hosted_on: the_server
    config:
      cloud_config:
        packages:
        - pwgen
        - pastebinit
        - [libpython2.7, 2.7.3-0ubuntu3.1]

User-Data Script

components:
  the_component:
    type: OS::Heat::CloudInit
    relationships:
      - hosted_on: the_server
    config:
      user_data: |
        #!/bin/sh
        echo "Hello World.  The time is now $(date -R)!" | tee /root/output.txt

Components for other Configuration Management tools

Other components can be implemented to expose the features of a particular Configuration Management tool. The implementation of each component will deliver the metadata that contains the configuration data. It will also generate the appropriate cloud-init user data to perform the following actions on the orchestrated server:

install the packages for the configuration management tool (optional, so no install is attempted when a golden image is known to have the required packages)
invoke the tool to perform configuration on the supplied configuration metadata

For these components, the actual cloud-init user data is derived from the abstracted component data. This would risk the component only working on particular versions of cloud-init. In this case, the component author could choose which versions of cloud-init to support and clearly document that.

os-collect-config

Whatever configuration management tool is used, the tool needs configuration data to act on, and the tool needs to be invoked whenever that data changes.

Traditionally heat has delivered the initial metadata via cloud-init and cfn-hup could be used to poll for changes and trigger any actions based on those changes. os-collect-config was written for the TripleO project to be a daemon which polls for metadata and runs the configured CM tool when the metadata changes. For rest of the components described in this spec, it is assumed that os-collect-config is the tool which fetches metadata and invokes the CM tool when that metadata changes.

OS::Heat::CfnConfig (previously OS::Heat::SoftwareConfig)

Example HOT templates have been assuming cfn-init AWS::CloudFormation::Init semantics for demonstration purposes, and for lack of a more appropriate configuration format.

We should consider not implementing this component at all, so that bootstrap config is handled by OS::Heat::CloudInit and any further configuration is handled by the chosen configuration management component. In this case, the use of cfn-init would be limited to AWS compatible templates.

An example usage:

components:
  the_component:
    type: OS::Heat::CfnConfig
    relationships:
      - hosted_on: the_server
    parameters:
      install: 'pip_venv' # valid values are 'package', 'pip', 'pip_venv'
      run_on_update: true # configures os-collect-config to re-run cnf-init if scripts changes
    config:
      packages:
        yum:
          mysql: []
          mysql-server: []
          httpd: []
          wordpress: []
      services:
        systemd:
          mysqld: {enabled: 'true', ensureRunning: 'true'}
          httpd: {enabled: 'true', ensureRunning: 'true'}

This would provide the_server with the following:

#cloud-config or user-data script to install heat-cfntools (only if install is specified)
the metadata which contains the cfn-init AWS::CloudFormation::Init that specifies the configuration tasks
a user-data script which actually runs /opt/aws/bin/cfn-init
if run_on_update is specified, #cloud-config which configures os-collect-config to run cfn-init again on metadata change

OS::Heat::Config

os-refresh-config and os-apply-config are the lightweight configuration management tools used by the TripleO project.

TripleO have a strong preference for using golden images, where the following will already be on the image that the server is launched with:

os-refresh-config, os-apply-config and os-collect-config
all other packages required to be on the server
os-refresh-config run-parts scripts which are executed in order on metadata changes
os-apply-config templates which transform metadata into application configuration files

In this case, the only purpose of a OS::Heat::Config component would be to deliver the metadata without any modification or cloud-init user data, for example:

components:
  the_component:
    type: OS::Heat::Config
    relationships:
      - hosted_on: the_server
    config:
      mysql:
        create-users:
          - database: keystone
            username: keystone
            password: {get_param: KeystoneDBPassword}
          - database: heat
            username: heat
            password: {get_param: HeatDBPassword}

For the cases where the user can't or won't use golden images, it should be possible for OS::Heat::Config to work on pristine images. To make OS::Heat::Config consistent with the other configuration management components, it should be responsible for generating the cloud-init to install os-refresh-config and os-apply-config. However this still leaves the pristine image without installed packages, os-refresh-config scripts or os-apply-config templates. These could be installed with a OS::Heat::CloudInit component. Consider the following example:

components:
  mysql_install:
    type: OS::Heat::CloudInit
    relationships:
      - hosted_on: the_server
    config:
      cloud_config:
        packages:
        - mysql-server-5.5
        - mysql-client-5.5
        write_files:
        - path: /opt/stack/os-apply-config/templates/etc/mysql/dbusers.json
          content: Template:Mysql.create-users
        - path: /opt/stack/os-config-refresh/post-configure.d/50-mysql-users
          permissions: '0755'
          content: |
            #!/usr/bin/python
            # Assert users that came from metadata config
            # ...

  mysql_configure:
    type: OS::Heat::Config
    relationships:
      - hosted_on: the_server
      - depends_on: mysql_install
    install: pip
    config:
      mysql:
        create-users:
          - database: keystone
            username: keystone
            password: {get_param: KeystoneDBPassword}
          - database: heat
            username: heat
            password: {get_param: HeatDBPassword}

OS::Heat::Puppet

A component which allows Puppet to be used for configuration of the server, either by specifying a puppet master or by allowing a manifest to be specified and applied on boot.

Puppet with master server

components:
  the_component:
  type: OS::Heat::Puppet
  relationships:
    - hosted_on: the_server
  parameters:
    server: "puppetmaster.example.org"
    certname: "%i.%f"
    ca_cert:
      get_param: puppet_cert

This will provide the_server with puppet #cloud-config configuration

Puppet serverless

components:
  the_component:
  type: OS::Heat::Puppet
  relationships:
    - hosted_on: the_server
  config:
    manifest: |
      file {'testfile':
        path    => '/tmp/testfile',
        ensure  => present,
        mode    => 0640,
        content => "I'm a test file.",
      }

This will provide the_server with a #cloud-config that writes out the puppet manifest file, and a cloud-init user-data script which runs puppet apply.

Other parameters which would be useful to implement include one to specify what module repositories to download required modules from.

OS::Heat::Ansible

Ansible is generally considered to be an agent-less configuration tool which configures servers by SSHing into them from where the playbook is invoked. However Ansible also supports pull-mode playbooks where each server repeatedly polls for a playbook, and executes the playbook in local mode if it has changed.

This mode of operation maps well to Heat, where the playbook can be delivered on boot via cloud-init and subsequent playbook changes can be picked up by os-collect-config to re-run ansible-playbook.

Given that Ansible is written in python and the playbook format is YAML, it has the potential to integrate well with Heat.

components:
  the_component:
  type: OS::Heat::Ansible
  relationships:
    - hosted_on: the_server
  config:
    vars:
      http_port:
        get_param: apache_port
      max_clients: 
        get_param: max_clients
    user: root
    tasks:
    - name: ensure apache is at the latest version
      yum: pkg=httpd state=latest
    - name: write the apache config file
      template: src=/srv/httpd.j2 dest=/etc/httpd.conf
      notify:
      - restart apache
    - name: ensure apache is running
      service: name=httpd state=started
    handlers:
      - name: restart apache
        service: name=httpd state=restarted

Heat/Blueprints/native-tools-bootstrap-config

Contents