Heat/Blueprints/hot-software-config-spec

= Background = This page presents a work-in-progress design for the HOT software configuration feature. It should be seen as an evolution of a previous hot-software-config proposal, but factoring in results of design discussions at the recent OpenStack summit in Hong Kong. This refined proposal is captured as a new wiki page for readability reasons. Once the design is finalized, we will consolidate the various wiki pages into a single one.

Discussions page: https://wiki.openstack.org/wiki/Talk:Heat/hot-software-config-spec

= Requirements = A number of requirements have been stated during design discussions, and they are also captures in the design summit etherpad. The most important ones to be addressed by this design proposal are summarized below again.


 * Composability and re-use: It must be possible to define software components once and compose and re-use them in different contexts without duplicating definitions.
 * Separation of component definitions and deployment: It must be possible to define multiple deployments of a software component, e.g. a software component defined once must be able to be deployed on different servers in a template.
 * Software components as stateful entities: It must be possible to track state of software components, i.e. whether a software deployment is in progress, has completed, or has failed.
 * Reference to software component outputs: It must be possible to retrieve outputs (attributes) of software components.
 * Ability to express dependencies between software components: It must be possible to define dependencies between software component, e.g. that input for one component is obtained from output of another component.

= High level overview = Re-usable software components are modeled as  resources. Those  definitions point to the actual automation to later perform software configuration (e.g. Chef, Puppet, scripts, ...) and they provide the metadata that is necessary for a generic   resource to deploy a given software on a server. The  resource represents one incarnation, i.e. one concrete use, of a software component in a template. It provides specific input parameters for that deployment, it will provide outputs produced by that deployment, and most importantly it maps the deployment to a specific target server.

It is assumed that there will be different pluggable implementations for  and   - one per backend configuration tool like Chef, Puppet, scripting etc. - since each configuration tool will have specific requirements on metadata and runtime implementation. Yet, it is assumed that a lot of common behavior can be covered by generic base classes for SoftwareConfig and SoftwareDeployment so only very thin plug-ins will be needed for the various CM tools (this is ongoing design work).

The following figure shows those concepts based on a Wordpress example. It is assumed that there are Chef cookbooks for configuring the Wordpress application and MySQL. Those are referenced by two  resources, along with the respective metadata for each automation. The automation plus the corresponding  definitions are the re-usable entities; they are not mapped to any concrete deployment target, and they do not define specific input values. There are two  resources, one for MySQL and one for Wordpress, each mapping the deployment to a separate server. A data dependency exists between the Wordpress deployment and the MySQL deployment (see also snippets later on this page) to get endpoint information about the MySQL database for configuring the Worpress application.



For simple templates, it is possible to define all elements (SoftwareConfig, SoftwareDeployment and other base resources) in one template file. For more complex scenarios, and to increase composability, a subset of resources can be split into separate provider templates that can be bound via environments. This is explained in more detail and with example snippets in section Packaging for re-use as provider templates.

= Software Configs = resources contain definitions of metadata for automation like Chef cookbooks, scripts, etc. along with a reference to the actual automation. Once defined, they can be mapped to one or more deployment targets (servers) by means of  resources (see Software Deployments). SoftwareConfig definitions are specific to the used software configuration tool, since they provide tool specific metadata. The following example shows a snippet for a Chef Software Config resource (the complete example is given in section Wordpress all-in-one Example):

resources: wordpress_sw_config: type: OS::Heat::SoftwareConfig::Chef properties: cookbook: http://www.example.com/hot/chef/wordpress.zip role: wordpress # input parameters that the chef role(s) need inputs: wp_admin_user: type: string mapping: wordpress/admin_user wp_admin_pw: type: string mapping: wordpress/admin_password db_endpoint_url: type: string mapping: wordpress/db_url # more input parameters ... # output data that the chef automation produces outputs: wp_url: type: string mapping: wordpress/url

The resource type  indicates that this is a Chef-specific Software Config definition. The  property points to the used Chef cookbook, and the   property points to the role to be set up via this Software Config. The  section contains the definition of input parameters that have to be passed to Chef for configuring the role. Input parameters are defined in terms of name and type. In addition, a  specifies to which role attribute the respective input parameters needs to be assigned (i.e. Chef-specific metadata).

The  section defines attributes that can be retrieved once the software deployment has completed at runtime. Those values will be available as attributes of the corresponding  resource at runtime (see also Software Deployments).

= Software Deployments = A  resource represents one concrete use of a piece of software (defined via a   resource) in a template. It points to the SoftwareConfig that shall be applied to a deployment target, and it points to the actual deployment target (server). As with SoftwareConfig, it is assumed that SoftwareDeployment implementations will be specific to the used software configuration tools, since tool specific steps will have to be performed at runtime.

The following example shows a SoftwareDeployment definition for the Wordpress component defined earlier using Chef. For brevity, definitions of overall template parameters, outputs or other resources have been left out - please refer to section Wordpress all-in-one Example for the complete example:

resources: wordpress_sw_config: type: OS::Heat::SoftwareConfig::Chef properties: # ...  wordpress_deployment: type: OS::Heat::SoftwareDeployment::Chef properties: apply_config: { get_resource: wordpress_sw_config } server: { get_resource: wp_server } input_values: wp_admin_user: { get_param: wp_admin_user } wp_admin_pw: { get_param: wp_admin_pw } # more input parameters ... wp_server: type: OS::Nova::Server properties: # ...

The  resource points to the   SoftwareConfig resource and specifies that one incarnation of it shall be deployed on (applied to) server. In the  section of the SoftwareDeployment properties, input for the configurable parameters of the Wordpress deployment is provided, for example, by getting global template parameters specified by the user at deployment time. Those parameters map to those defined in the  resource shown earlier.

The output parameters defined under  in the   resource can be observed as attributes of the   resource via the   instrinsic function. For example, the following snippet in a HOT template would pass the URL of the deployed Wordpress application to the user as an output value:

outputs: wordpress_url: description: URL to access deployed Wordpress application value: { get_attr: [ wordpress_deployment, wp_url ]}

Responsibilities of the Software Deployment Resource
There are several aspects to be covered by implementations of Software Deployment resources. First of all, the resource code is responsible for injecting metadata into the referenced deployed target (server) for doing bootstrapping of the respective software config tool (Chef etc.). Furthermore, the resource is responsible for triggering deployment of the respective software (by invoking the underlying software config tool) when all dependencies are met. Upon completion of software deployment, the resource has to update its state to CREATE_COMPLETE so overall orchestration through Heat can progress (or failures have to be indicated via the appropriate failure state). Finally, attributes specified in the associated SoftwareConfig resource have to be obtained from the underlying software config tool so that uses of the  can be resolved.

= Dependencies between Software Deployments = Software Deployments in many cases depend on other Software Deployments. For example, the Wordpress application requires a MySQL database to be set up for storing content. There are two ways for declaring dependencies between Software Deployments: data flow based and explicit definition.

Data flow based
A data flow based dependency between two Software Deployments exists, when a property (input) of one Software Deployment is obtained from an attribute (output) of another Software Deployment. A dependency between the two SoftwareDeployment resources is enforced by the Heat engine implicitly. For example

resources: wordpress_deployment: type: OS::Heat::SoftwareDeployment::Chef properties: apply_config: { get_resource: wordpress_sw_config } server: { get_resource: wp_server } input_values: wp_admin_user: { get_param: wp_admin_user } wp_admin_pw: { get_param: wp_admin_pw } db_endpoint_url: { get_attr: [ mysql_deployment, db_url ] } # more input parameters ... mysql_deployment: type: OS::Heat::SoftwareDeployment::Chef properties: apply_config: { get_resource: mysql_sw_config } server: { get_resource: db_server } input_values: # input parameters for MySQL deployment ...

would introduce a dependency from  to   since one of the properties of   is set using the   function refering to an attribute of the   resource. As a result, resource  must be in state CREATE_COMPLETE before processing of resource   starts. The complete example is shown in section Wordpress all-in-one Example.

Explicit dependency
If no data dependency exists, but there is still a timing dependency (e.g. a process must be up before a client can connect to it), a mechanism for declaring an explicit dependency is required. This can be solved by explicitly defining dependencies of a software deployment in a  clause which is a list of resource IDs of other resources that a Software Deployment depends on, for example:

resources: client: type: OS::Heat::SoftwareDeployment::Chef properties: apply_config: { get_resource: client_sw_config } server: { get_resource: my_server } input_values: # params ... depends_on: - get_resource: server_process1 - get_resource: server_process2 server_process1: type: OS::Heat::SoftwareDeployment::Chef properties: apply_config: { get_resource: server_process1_sw_config } server: { get_resource: my_server } input_values: # params ... server_process2: type: OS::Heat::SoftwareDeployment::Chef properties: apply_config: { get_resource: server_process2_sw_config } server: { get_resource: my_server } input_values: # params ... my_server: type: OS::Nova::Server properties: # ...

In the example above,  would depend on both   and   to be completed.

= Packaging for re-use as provider templates = In descriptions given so far it was outlined how  and   can be used in a HOT template to address software orchestration in Heat. For simple examples or for getting started, it is possible to put all definitions into a single template file. However, to make definitions more re-usable or when it comes to bigger, more complex scenarios, it makes sense to split definitions of software components into separate files. For example, instead of having the Software Config definition of the Wordpress application ( in examples above) with all details (pointer to Chef cookbook, definition of inputs and outputs etc.) duplicated in each template that uses Wordpress, it is better to put those definitions into one file that can be leveraged by other templates as a provider template.

One possible option is to define both the  resource and the   resource for the Wordpress application in a separate template file and provide all input parameters as well as the pointer to the target server as input when using the provider template. The corresponding provider template would look like the following snippet (complete example given in section Wordpress example with re-usable provider templates):

heat_template_version: 2013-05-23 parameters: wp_admin_user: description: Username of the Wordpress admin user. type: string wp_admin_pw: description: Password of the Wordpress admin user. type: string db_endpoint_url: description: Endpoint URL of the database to be used. type: string # more software related parameters ... server: description: Reference to server onto which to install Wordpress. type: string resources: wordpress_sw_config: type: OS::Heat::SoftwareConfig::Chef properties: cookbook: http://www.example.com/hot/chef/wordpress.zip role: wordpress # input parameters that the chef role(s) need inputs: # ...      # output data that the chef automation produces outputs: # ...  wordpress_deployment: type: OS::Heat::SoftwareDeployment::Chef properties: apply_config: { get_resource: wordpress_sw_config } server: { get_param: server } input_values: wp_admin_user: { get_param: wp_admin_user } wp_admin_pw: { get_param: wp_admin_pw } db_endpoint_url: { get_param: db_endpoint_url } # ... outputs: wordpress_url: description: URL to access deployed Wordpress application value: { get_attr: [ wordpress_deployment, wp_url ]}

Note that even though the  resource is contained within the provider template, the definition is still re-usable since no mapping to any server is defined. A reference to the server on which the Software Deployment shall be run is provided as input parameter in the place where the provider template is used. This is achieved by having the  parameter defined in the   section of the template, and by having the   property of the   resource be initialized with the user-provided parameter.

Note that in addition to having better re-usability of Software Config definitions in contrast to having them included in the actual template file, the approach of defining them in provider templates also offers the possibility to do richer input parameter validation, since the  section of a HOT template features more schema-like expressiveness through   that can be expressed for each input parameter.

With a re-usable definition for MySQL in a similar file, a HOT template for an actual deployment can then be composed as outlined in the following snippet:

heat_template_version: 2013-05-23 parameters: # parameter definitions omitted for brevity resources: wordpress: type: Software::Wordpress properties: server: { get_resource: wp_server } wp_admin_user: { get_param: wp_admin_user } wp_admin_pw: { get_param: wp_admin_pw } db_endpoint_url: { get_attr: [ mysql, db_endpoint_url ] } db_user: { get_param: db_user } db_pw: { get_param: db_pw } wp_server: type: OS::Nova::Server properties: image: { get_param: image } flavor: { get_param: flavor } mysql: type: Software::MySQL properties: server: { get_resource: db_server } db_user: { get_param: db_user } db_pw: { get_param: db_pw } db_name: { get_param: db_name } db_server: type: OS::Nova::Server properties: image: { get_param: image } flavor: { get_param: flavor } outputs: wordpress_url: description: URL to access deployed Wordpress application value: { get_attr: [ wordpress_deployment, wp_url ]}

The template above does not contain the detailed definitions of the Wordpress and MySQL Software Configs any more, but just defines two concrete uses of those software components - the  resource and the   resource. The target server for each use, as well as specific input parameters are provided as properties to those resources.

The resource type names used for both resources,  or , are bound to the respective provider templates by means of environment definitions. This also provides for more flexibility, since different provider templates may be used in different environments without having to change the template. For example, there may be provider templates for software that work well on Fedora but others that work on Ubuntu.

Assuming that the definition of the Wordpress component is contained in a provider templates with name  and the definition for MySQL is contained in a file   - both available on a server - a corresponding environment definition would look like the following:

resource_registry: "Software::Wordpress": http://www.example.com/hot/software/wordpress_component.yaml "Software::MySQL": http://www.example.com/hot/software/mysql_component.yaml

= Implementation considerations = This section is still very much in progress, but more or less a collection of some thoughts for now.

Bootstrapping of software configuration tools could be done using cloud-init. The software configuration resource implementation will have to inject the respective metadata into the server resource definition on which it is hosted.

Implementations like  etc. could be used for collecting software config metadata etc.

For synchronization purposes (e.g. in case of an explicit dependency), existing mechanisms (e.g. WaitCondition signaling) could be used under the covers, however, without surfacing them in templates.

= Wordpress all-in-one Example = The following listing shows the Wordpress example referenced earlier as a complete HOT template. The template includes all definitions of Software Configs, Software Deployments, as well as server resources (thus all-in-one example). Note that the example is not meant to be 100% correct so it could be used in Heat, but it is meant to give a complete end-to-end draft of the concepts described on this wiki page.

heat_template_version: 2013-05-23 description: > This is an all-in-one template for deployment of Wordpress and MySQL on two servers. The definition of software configs for Wordpress and MySQL as well as definition for deployment onto servers is contained in this single template file. parameters: wp_admin_user: description: Username of the Wordpress admin user. type: string wp_admin_pw: description: Password of the Wordpress admin user. type: string db_user: description: Username of the database admin user. type: string db_pw: description: Password of the database admin user. type: string db_name: description: Database name for the Wordpress database. type: string image: description: Image to be used for servers. type: string flavor: description: Flavor to be used for servers. type: string resources: wordpress_sw_config: type: OS::Heat::SoftwareConfig::Chef properties: cookbook: http://www.example.com/hot/chef/wordpress.zip role: wordpress # input parameters that the chef role(s) need inputs: wp_admin_user: type: string mapping: wordpress/admin_user wp_admin_pw: type: string mapping: wordpress/admin_password db_endpoint_url: type: string mapping: wordpress/db_url db_user: type: string mapping: wordpress/db_user db_pw: type: string mapping: wordpress/db_password # output data that the chef automation produces outputs: wp_url: type: string mapping: wordpress/url wordpress_deployment: type: OS::Heat::SoftwareDeployment::Chef properties: apply_config: { get_resource: wordpress_sw_config } server: { get_resource: wp_server } input_values: wp_admin_user: { get_param: wp_admin_user } wp_admin_pw: { get_param: wp_admin_pw } db_endpoint_url: { get_attr: [ mysql_deployment, db_url ] } db_user: { get_param: db_user } db_pw: { get_param: db_pw } wp_server: type: OS::Nova::Server properties: image: { get_param: image } flavor: { get_param: flavor } mysql_sw_config: type: OS::Heat::SoftwareConfig::Chef properties: cookbook: http://www.example.com/hot/chef/mysql.zip role: mysql-server # input parameters that the chef role(s) need inputs: db_name: type: string mapping: mysql-server/db_name db_user: type: string mapping: mysql-server/db_user db_pw: type: string mapping: mysql-server/db_password # output data that the chef automation produces outputs: db_endpoint_url: type: string mapping: mysql-server/db_url mysql_deployment: type: OS::Heat::SoftwareDeployment::Chef properties: apply_config: { get_resource: mysql_sw_config } server: { get_resource: db_server } input_values: db_name: { get_param: db_name } db_user: { get_param: db_user } db_pw: { get_param: db_pw } db_server: type: OS::Nova::Server properties: image: { get_param: image } flavor: { get_param: flavor } outputs: wordpress_url: description: URL to access deployed Wordpress application value: { get_attr: [ wordpress_deployment, wp_url ]}

= Wordpress example with re-usable provider templates = In section Wordpress all-in-one Example a HOT template was shown that included all definitions in a single file. This section lists the complete definitions of the same example, but with parts split into separate HOT templates that can be used as provider templates for better re-use.

Provider template for Wordpress
wordpress_component.yaml:

heat_template_version: 2013-05-23 description: > This template contains the definition of a Wordpress SoftwareConfig and corresponding SoftwareDeployment for re-use in other templates as a HOT provider template. parameters: wp_admin_user: description: Username of the Wordpress admin user. type: string wp_admin_pw: description: Password of the Wordpress admin user. type: string db_endpoint_url: description: Endpoint URL of the database to be used. type: string db_user: description: Username to connect to the database. type: string db_pw: description: Password to connect to the database. type: string server: description: Reference to server onto which to install Wordpress. type: string resources: wordpress_sw_config: type: OS::Heat::SoftwareConfig::Chef properties: cookbook: http://www.example.com/hot/chef/wordpress.zip role: wordpress # input parameters that the chef role(s) need inputs: wp_admin_user: type: string mapping: wordpress/admin_user wp_admin_pw: type: string mapping: wordpress/admin_password db_endpoint_url: type: string mapping: wordpress/db_url db_user: type: string mapping: wordpress/db_user db_pw: type: string mapping: wordpress/db_password # output data that the chef automation produces outputs: wp_url: type: string mapping: wordpress/url wordpress_deployment: type: OS::Heat::SoftwareDeployment::Chef properties: apply_config: { get_resource: wordpress_sw_config } server: { get_param: server } input_values: wp_admin_user: { get_param: wp_admin_user } wp_admin_pw: { get_param: wp_admin_pw } db_endpoint_url: { get_param: db_endpoint_url } db_user: { get_param: db_user } db_pw: { get_param: db_pw } outputs: wordpress_url: description: URL to access deployed Wordpress application value: { get_attr: [ wordpress_deployment, wp_url ]}

Provider template for MySQL
mysql_component.yaml

heat_template_version: 2013-05-23 description: > This template contains the definition of a MySQL SoftwareConfig and corresponding SoftwareDeployment for re-use in other templates as a HOT provider template. parameters: db_user: description: Username of the database admin user. type: string db_pw: description: Password of the database admin user. type: string db_name: description: Database name for the Wordpress database. type: string server: description: Reference to server onto which to install MySQL. type: string resources: mysql_sw_config: type: OS::Heat::SoftwareConfig::Chef properties: cookbook: http://www.example.com/hot/chef/mysql.zip role: mysql-server # input parameters that the chef role(s) need inputs: db_name: type: string mapping: mysql-server/db_name db_user: type: string mapping: mysql-server/db_user db_pw: type: string mapping: mysql-server/db_password # output data that the chef automation produces outputs: db_endpoint_url: type: string mapping: mysql-server/db_url mysql_deployment: type: OS::Heat::SoftwareDeployment::Chef properties: apply_config: { get_resource: mysql_sw_config } server: { get_param: server } input_values: db_name: { get_param: db_name } db_user: { get_param: db_user } db_pw: { get_param: db_pw } outputs: db_endpoint_url: description: Endpoint URL of the MySQL database. value: { get_attr: [ mysql_deployment, db_endpoint_url ]}

HOT template to deploy Wordpress and MySQL on two servers
wp-with-provider-templates.yaml:

heat_template_version: 2013-05-23 description: > This is a template for deployment of Wordpress and MySQL on two servers. The definition of the software configs for Wordpress and MySQL are contained in  separate files that are bound as provider templates via the environment. parameters: wp_admin_user: description: Username of the Wordpress admin user. type: string wp_admin_pw: description: Password of the Wordpress admin user. type: string db_user: description: Username of the database admin user. type: string db_pw: description: Password of the database admin user. type: string db_name: description: Database name for the Wordpress database. type: string image: description: Image to be used for servers. type: string flavor: description: Flavor to be used for servers. type: string resources: wordpress: type: Software::Wordpress properties: server: { get_resource: wp_server } wp_admin_user: { get_param: wp_admin_user } wp_admin_pw: { get_param: wp_admin_pw } db_endpoint_url: { get_attr: [ mysql, db_endpoint_url ] } db_user: { get_param: db_user } db_pw: { get_param: db_pw } wp_server: type: OS::Nova::Server properties: image: { get_param: image } flavor: { get_param: flavor } mysql: type: Software::MySQL properties: server: { get_resource: db_server } db_user: { get_param: db_user } db_pw: { get_param: db_pw } db_name: { get_param: db_name } db_server: type: OS::Nova::Server properties: image: { get_param: image } flavor: { get_param: flavor } outputs: wordpress_url: description: URL to access deployed Wordpress application value: { get_attr: [ wordpress_deployment, wp_url ]}

Environment file
wordpress-and-mysql-environment.yaml:

resource_registry: "Software::Wordpress": http://www.example.com/hot/software/wordpress_component.yaml "Software::MySQL": http://www.example.com/hot/software/mysql_component.yaml