Heat/Blueprints/native-tools-bootstrap-config
Contents
Heat Bootstrap Config
--steve-stevebaker (talk) 04:36, 12 September 2013 (UTC)
NOTE: This specification is still under discussion and has not been endorsed by the heat project.
Background
The purpose of bootstrapping an orchestrated server is to provide just enough package installation and configuration to be able to hand off to a full Configuration Management tool. Traditionally this has been achieved in heat templates with cfn-init from the heat-cfntools package however this has the following issues:
- It needs to be invoked explicitly from the template UserData
- It is limited to providing cfn compatible functionality
- It acts on a single block of metadata which is associated with the instance resource, which makes it difficult to write templates that are composable (ie, templates which split the application configuration from the architecture the application is being orchestrated into).
Heat's current underlying implementation relies on cloud-init to deliver the Metadata and the UserData script to the instance on boot.
This blueprint proposes that the functionality of cloud-init be directly exposed in heat templates to fulfill bootstrap config tasks. This blueprint also proposes that this be used as a foundation to expose the features of other configuration management tools.
Relationship to HOT blueprints
Blueprint hot-software-config is flagged as depending on this blueprint. However in reality the blueprints are interdependent and need to evolve cooperatively.
For the purposes of this specification it is assumed that:
- blueprint native-tools-bootstrap-config specifies the internal implementation of how HOT components are consumed by server/instance resources, and the interaction between heat and orchestrated servers
- blueprint hot-software-config specifies the syntax and semantics of HOT components
However, this spec states the opinion that HOT config components should transparently expose whatever underlying bootstrap/configuration-management is being used. This has the following benefits:
- There is no need to create and maintain yet another configuration management abstraction
- Heat can explicitly claim that it is not itself a configuration management tool by documenting the tools it natively supports
- Configuration management tools which use YAML as their native format have potential for tighter integration with Heat templates
HOT examples in this spec are meant for illustration purposes only, and the specifics may change in the final implementation.
OS::Heat::CloudInit
It is proposed that each subclass of Component can provide its hosted_on resources with some metadata, along with a collection of cloud-init user data that will perform the desired configuration. The CloudInit component will do this by directly exposing appropriate cloud-init user data types. This may be limited to #cloud-config files and user data scripts.
cloud-init version compatibility
Heat has been hindered in the past in attempts to use newer cloud-init features due to the older version of cloud-init used on the launched OS. It is hoped that this will be mitigated by transparently exposing cloud-init configuration into the template, so that it is up to the user to only use the features supported by the cloud-init of the server images they launch.
Examples
cloud-init examples could be represented in HOT templates in the following way:
Writing out arbitrary files
components: the_component: type: OS::Heat::CloudInit relationships: - hosted_on: the_server config: cloud_config: write_files: - encoding: b64 # Note here that the HOT get_param function is being called # to directly populate #cloud-config yaml content: get_param: the_file_contents owner: root:root path: /foo/bar permissions: '0644'
Adding a yum repository
components: the_component: type: OS::Heat::CloudInit relationships: - hosted_on: the_server config: cloud_config: yum_repos: epel-testing: baseurl: http://download.fedoraproject.org/pub/epel/testing/5/$basearch enabled: false failovermethod: priority gpgcheck: true gpgkey: file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL name: Extra Packages for Enterprise Linux 5 - Testing
Install arbitrary packages
components: the_component: type: OS::Heat::CloudInit relationships: - hosted_on: the_server config: cloud_config: packages: - pwgen - pastebinit - [libpython2.7, 2.7.3-0ubuntu3.1]
User-Data Script
components: the_component: type: OS::Heat::CloudInit relationships: - hosted_on: the_server config: user_data: | #!/bin/sh echo "Hello World. The time is now $(date -R)!" | tee /root/output.txt
Implementing OS::Heat::CloudInit would be enough to mark this blueprint as completed. Subsequent components in this spec can be raised as their own blueprints if they are to be implemented, and are included here to demonstrate how far the scope of this cloud-init approach could be extended.
Components for other Configuration Management tools
Other components can be implemented to expose the features of a particular Configuration Management tool. The implementation of each component will deliver the metadata that contains the configuration data. It will also generate the appropriate cloud-init user data to perform the following actions on the orchestrated server:
- install the packages for the configuration management tool (optional, so no install is attempted when a golden image is known to have the required packages)
- invoke the tool to perform configuration on the supplied configuration metadata
For these components, the actual cloud-init user data is derived from the abstracted component data. This would risk the component only working on particular versions of cloud-init. In this case, the component author could choose which versions of cloud-init to support and clearly document that.
os-collect-config
Whatever configuration management tool is used, the tool needs configuration data to act on, and the tool needs to be invoked whenever that data changes.
Traditionally heat has delivered the initial metadata via cloud-init and cfn-hup could be used to poll for changes and trigger any actions based on those changes. os-collect-config was written for the TripleO project to be a daemon which polls for metadata and runs the configured CM tool when the metadata changes. For rest of the components described in this spec, it is assumed that os-collect-config is the tool which fetches metadata and invokes the CM tool when that metadata changes.
CM tools to consider
Many configuration management tools could be considered for implementing a HOT component, including:
- heat-cfntools cfn-init
- os-refresh-config / os-apply-config
- Puppet
- Ansible
- Chef
- SaltStack
Some of these will be considered below.
OS::Heat::CfnConfig (previously OS::Heat::SoftwareConfig)
Example HOT templates have been assuming cfn-init AWS::CloudFormation::Init semantics for demonstration purposes, and for lack of a more appropriate configuration format.
We should consider not implementing this component at all, so that bootstrap config is handled by OS::Heat::CloudInit and any further configuration is handled by the chosen configuration management component. In this case, the use of cfn-init would be limited to AWS compatible templates.
An example usage:
components: the_component: type: OS::Heat::CfnConfig relationships: - hosted_on: the_server parameters: install: 'pip_venv' # valid values are 'package', 'pip', 'pip_venv' run_on_update: true # configures os-collect-config to re-run cnf-init if scripts changes config: packages: yum: mysql: [] mysql-server: [] httpd: [] wordpress: [] services: systemd: mysqld: {enabled: 'true', ensureRunning: 'true'} httpd: {enabled: 'true', ensureRunning: 'true'}
This would provide the_server with the following:
- #cloud-config or user-data script to install heat-cfntools (only if install is specified)
- the metadata which contains the cfn-init AWS::CloudFormation::Init that specifies the configuration tasks
- a user-data script which actually runs /opt/aws/bin/cfn-init
- if run_on_update is specified, #cloud-config which configures os-collect-config to run cfn-init again on metadata change
OS::Heat::Config
os-refresh-config and os-apply-config are the lightweight configuration management tools used by the TripleO project.
TripleO have a strong preference for using golden images, where the following will already be on the image that the server is launched with:
- os-refresh-config, os-apply-config and os-collect-config
- all other packages required to be on the server
- os-refresh-config run-parts scripts which are executed in order on metadata changes
- os-apply-config templates which transform metadata into application configuration files
In this case, the only purpose of a OS::Heat::Config component would be to deliver the metadata without any modification or cloud-init user data, for example:
components: the_component: type: OS::Heat::Config relationships: - hosted_on: the_server config: mysql: create-users: - database: keystone username: keystone password: {get_param: KeystoneDBPassword} - database: heat username: heat password: {get_param: HeatDBPassword}
For the cases where the user can't or won't use golden images, it should be possible for OS::Heat::Config to work on pristine images. To make OS::Heat::Config consistent with the other configuration management components, it should be responsible for generating the cloud-init to install os-refresh-config and os-apply-config. However this still leaves the pristine image without installed packages, os-refresh-config scripts or os-apply-config templates. These could be installed with a OS::Heat::CloudInit component. Consider the following example:
components: mysql_install: type: OS::Heat::CloudInit relationships: - hosted_on: the_server config: cloud_config: packages: - mysql-server-5.5 - mysql-client-5.5 write_files: - path: /opt/stack/os-apply-config/templates/etc/mysql/dbusers.json content: Template:Mysql.create-users - path: /opt/stack/os-config-refresh/post-configure.d/50-mysql-users permissions: '0755' content: | #!/usr/bin/python # Assert users that came from metadata config # ...
mysql_configure: type: OS::Heat::Config relationships: - hosted_on: the_server - depends_on: mysql_install install: pip config: mysql: create-users: - database: keystone username: keystone password: {get_param: KeystoneDBPassword} - database: heat username: heat password: {get_param: HeatDBPassword}
OS::Heat::Puppet
A component which allows Puppet to be used for configuration of the server, either by specifying a puppet master or by allowing a manifest to be specified and applied on boot.
Puppet with master server
components: the_component: type: OS::Heat::Puppet relationships: - hosted_on: the_server parameters: server: "puppetmaster.example.org" certname: "%i.%f" ca_cert: get_param: puppet_cert
This will provide the_server with puppet #cloud-config configuration
Puppet serverless
components: the_component: type: OS::Heat::Puppet relationships: - hosted_on: the_server config: manifest: | file {'testfile': path => '/tmp/testfile', ensure => present, mode => 0640, content => "I'm a test file.", }
This will provide the_server with a #cloud-config that writes out the puppet manifest file, and a cloud-init user-data script which runs puppet apply.
Other parameters which would be useful to implement include one to specify what module repositories to download required modules from.
OS::Heat::Ansible
Ansible is generally considered to be an agent-less configuration tool which configures servers by SSHing into them from where the playbook is invoked. However Ansible also supports pull-mode playbooks where each server repeatedly polls for a playbook, and executes the playbook in local mode if it has changed.
This mode of operation maps well to Heat, where the playbook can be delivered on boot via cloud-init and subsequent playbook changes can be picked up by os-collect-config to re-run ansible-playbook.
Given that Ansible is written in python and the playbook format is YAML, it has the potential to integrate well with Heat.
components: the_component: type: OS::Heat::Ansible relationships: - hosted_on: the_server config: vars: http_port: get_param: apache_port max_clients: get_param: max_clients user: root tasks: - name: ensure apache is at the latest version yum: pkg=httpd state=latest - name: write the apache config file template: src=/srv/httpd.j2 dest=/etc/httpd.conf notify: - restart apache - name: ensure apache is running service: name=httpd state=started handlers: - name: restart apache service: name=httpd state=restarted