Jump to: navigation, search

Difference between revisions of "StarlingX/Config/StarlingX Puppet Usage"

m
m
 
(2 intermediate revisions by the same user not shown)
Line 8: Line 8:
 
* '''storage.pp''' - storage host manifest
 
* '''storage.pp''' - storage host manifest
  
* '''bootstrap.pp''' - special purpose manifest to bootstrap the first controller (currently runs as part of config_controller and operates only on static.yaml and packaged yaml files)
+
* '''bootstrap.pp''' - special purpose manifest to bootstrap the first controller (currently runs as part of ansible bootstrap playbook and operates only on static.yaml and packaged yaml files)
 
* '''runtime.pp''' - special purpose manifest to apply runtime configuration (dynamic puppet class list)
 
* '''runtime.pp''' - special purpose manifest to apply runtime configuration (dynamic puppet class list)
 
* '''upgrade.pp''' - special purpose manifest to perform a software upgrade of the first controller
 
* '''upgrade.pp''' - special purpose manifest to perform a software upgrade of the first controller
  
  
The manifests are setup to leverage [https://docs.puppet.com/puppet/5.1/hiera_automatic.html#hiera:-looking-up-data:-automatic-class-parameter-lookup automatic class parameter lookup]. Classes are declared using the [https://docs.puppet.com/puppet/5.1/lang_classes.html#include-like-vs-resource-like include-like] syntax to avoid explicitly overriding class variable values during the declaration. This allows the parameter resolution to walk the data hierachy, choosing the most appropriate value. Class declarations that use the resource-like syntax should be used sparingly, and are only applicable when the attribute value should never be overwritten or is derived from other manifest parameters.
+
The manifests are setup to leverage [https://docs.puppet.com/puppet/5.1/hiera_automatic.html#hiera:-looking-up-data:-automatic-class-parameter-lookup automatic class parameter lookup]. Classes are declared using the [https://docs.puppet.com/puppet/5.1/lang_classes.html#include-like-vs-resource-like include-like] syntax to avoid explicitly overriding class variable values during the declaration. This allows the parameter resolution to walk the data hierarchy, choosing the most appropriate value. Class declarations that use the resource-like syntax should be used sparingly, and are only applicable when the attribute value should never be overwritten or is derived from other manifest parameters.
  
The following Hiera data hierachy is configured to define the order for parameter resolution. The yaml files are managed by System Inventory where indicated, otherwise they are static.
+
The following Hiera data hierarchy is configured to define the order for parameter resolution. The yaml files are managed by System Inventory where indicated, otherwise they are static.
 
# '''runtime.yaml''' - dynamically created by the sysinv-agent during manifest apply (optional - only present during runtime.pp manifest apply)
 
# '''runtime.yaml''' - dynamically created by the sysinv-agent during manifest apply (optional - only present during runtime.pp manifest apply)
 
# '''host.yaml''' - generated file by sysinv-conductor during host configuration changes (contains host specific parameter values - stored by host IP address)
 
# '''host.yaml''' - generated file by sysinv-conductor during host configuration changes (contains host specific parameter values - stored by host IP address)
Line 34: Line 34:
 
* modules  
 
* modules  
 
** platform - platform puppet module structure containing service manifests and templates
 
** platform - platform puppet module structure containing service manifests and templates
** openstack - openstack puppet module structure containing service manifests and templates
+
** openstack - openstack puppet module structure containing service manifests and templates (used only for openstack services in the platform like keystone and horizon)
  
 
=== Design Best Practices ===
 
=== Design Best Practices ===
  
 
* Use [https://docs.puppet.com/puppet/5.1/lang_classes.html#include-like-vs-resource-like include-like] class declarations with no explicit parameters  
 
* Use [https://docs.puppet.com/puppet/5.1/lang_classes.html#include-like-vs-resource-like include-like] class declarations with no explicit parameters  
** Ensures class parameters resolution will traverse the data hierachy for parameter lookup
+
** Ensures class parameters resolution will traverse the data hierarchy for parameter lookup
 
* Define constant values or defaults in static yaml files  
 
* Define constant values or defaults in static yaml files  
 
** Fixed values must be defined in a static yaml file (e.g. controller.yaml, worker.yaml or storage.yaml)
 
** Fixed values must be defined in a static yaml file (e.g. controller.yaml, worker.yaml or storage.yaml)

Latest revision as of 17:23, 21 August 2019

Architecture Overview

StarlingX system configuration files and system setup is managed by Puppet. Hosts are configured according to the Puppet Hiera yaml data files which are either static defaults or populated by System Inventory from the system configuration data stored in the database.

Manifest apply operations to the system are performed by one of the top level static manifests which are unique to each host personality, or the specific operation that is being performed.

  • controller.pp - controller host manifest
  • worker.pp - worker host manifest
  • storage.pp - storage host manifest
  • bootstrap.pp - special purpose manifest to bootstrap the first controller (currently runs as part of ansible bootstrap playbook and operates only on static.yaml and packaged yaml files)
  • runtime.pp - special purpose manifest to apply runtime configuration (dynamic puppet class list)
  • upgrade.pp - special purpose manifest to perform a software upgrade of the first controller


The manifests are setup to leverage automatic class parameter lookup. Classes are declared using the include-like syntax to avoid explicitly overriding class variable values during the declaration. This allows the parameter resolution to walk the data hierarchy, choosing the most appropriate value. Class declarations that use the resource-like syntax should be used sparingly, and are only applicable when the attribute value should never be overwritten or is derived from other manifest parameters.

The following Hiera data hierarchy is configured to define the order for parameter resolution. The yaml files are managed by System Inventory where indicated, otherwise they are static.

  1. runtime.yaml - dynamically created by the sysinv-agent during manifest apply (optional - only present during runtime.pp manifest apply)
  2. host.yaml - generated file by sysinv-conductor during host configuration changes (contains host specific parameter values - stored by host IP address)
  3. secure_system.yaml - generated file by sysinv-conductor during system configuration changes (contains secure system wide parameter values, such as passwords, excluded by collect)
  4. system.yaml - generated file by sysinv-conductor during system configuration changes (contains system wide parameter values)
  5. secure_static.yaml - generated file by sysinv-puppet during the initial controller bootstrap (contains secure site generated values, such as passwords, excluded by collect - one-time generation and does not depend on system configuration)
  6. static.yaml - generated file by sysinv-puppet during the initial controller bootstrap (contains site generated values - one-time generation and does not depend on system configuration)
  7. personality.yaml - static file which is sourced from one of the per-personality packaged hieradata files (controller.yaml, worker.yaml, storage.yaml)
  8. global.yaml - static file which contains defaults that apply to all personalities

Puppet Manifests

The puppet manifest are defined in package puppet-manifests which includes the top level static manifests listed above, as well as service level manifests and templates for each service.

The puppet-manifests package has the following structure:

  • hieradata - static hiera data files
  • manifests - static top level manifests
  • modules
    • platform - platform puppet module structure containing service manifests and templates
    • openstack - openstack puppet module structure containing service manifests and templates (used only for openstack services in the platform like keystone and horizon)

Design Best Practices

  • Use include-like class declarations with no explicit parameters
    • Ensures class parameters resolution will traverse the data hierarchy for parameter lookup
  • Define constant values or defaults in static yaml files
    • Fixed values must be defined in a static yaml file (e.g. controller.yaml, worker.yaml or storage.yaml)
    • Do not pass fixed hard coded values to class parameters if a resource-like declaration is used
    • This ensures values can be overriden if necessary via configuration without changing the default value
  • Store passwords and other secret/protected information in the secure yaml files
    • The secure_system.yaml and secure_static.yaml files are excluded by collect in order to protect sensitive customer data
  • Per-Domain Parameter Classes
    • Promotes re-use within other domains when configured data needs to be shared among different subsystems
    • Simplifies hiera data population by system inventory since parameters can be populated once
    • Usage by multiple subsystems is well defined since parameter class include is explicit
    • Example:
      • class namespace::subsystem::params($myparam = 'default') {}
  • Use Full Top Scope Variable and Class references (empty top scope prefix ::)
    • Prevents accidental name scope issues
    • Example include or inherits:
      • include ::namespace::class
      • inherits ::namespace::class::params
    • Example variable references:
      • $::namespace::class::variable
      • "${::namespace::class::variable}"
  • Use Embedded Ruby templates for configuration files
    • File contents that are not specifically managed by a ruby class should use an ERB template
    • This ensures the entire contents are managed by puppet and does not rely on File_Line operations
    • NEVER use a sed exec to do what puppet can do for you

Dynamic Resources

Since manifests are now static, there are scenarios where the contents need to be generated based on a set of conditions (data) generated for the system or host. To accomplish that, a well defined puppet resource (see example) should be used which can be instantiated multiple times with the required parameter data. This ensures the logic is coded in the puppet resource in the manifest, but the instance data is generated by system inventory.

A good example of this is used by the puppet host resource used to populate the /etc/hosts file. The puppet manifest takes the $hosts as a parameter and instantiates the resources using the puppet create_resources function. The hosts is a hash (dictionary) of the resources by title and parameters required. You can define new resources with it's own set of parameters if you need to do something similar that is not part of an existing puppet module.

class platform::config::hosts
  inherits ::platform::config::params {

  create_resources('host', $hosts, {})
}

# system.yaml
platform::config::params::hosts:
  controller:
    ip: 192.168.204.2
  controller-0:
    ip: 192.168.204.3
  controller-1:
    ip: 192.168.204.4

System Inventory

System Inventory is responsible for managing the Puppet hiera yaml files and will regenerate them during configuration operations. The management of the hiera parameters is organized by service and driven by a top level PuppetOperator class. The entry points to this class are the following and correspond to the different dynamically generated Hiera data files in the hierarchy:

  • create_static_config - gathers parameters to be populated in the static.yaml (one-time)
  • create_system_config - gathers parameters to be populated in the system.yaml
  • create_host_config - gathers parameter to be populated in the host.yaml


The top level entry-points of the PuppetOperator (in sysinv/puppet/puppet.py) will invoke the service level operators to define each of the required parameters for that service. The service puppet operators return the configuration dictionary for each of the puppet parameters that are unique to that particular service which gets combined into the complete configuration file for the corresponding file in the data hierarchy.

The resulting files are stored in the following path:

/opt/platform/puppet/<software-version>/hieradata/static.yaml
/opt/platform/puppet/<software-version>/hieradata/secure_static.yaml
/opt/platform/puppet/<software-version>/hieradata/system.yaml
/opt/platform/puppet/<software-version>/hieradata/secure_system.yaml
/opt/platform/puppet/<software-version>/hieradata/<ip-address>.yaml

In order to improve the development cycle, the hiera data files can be regenerated via command line. This allows for rapid development since code changes can be quickly tested to ensure the resulting yaml files are properly populated before the data is consumed by the manifests (e.g. host unlock). In addition, if you don't want to overwrite the real system files, you can alternatively specify an alternate output path.

sysinv-puppet create-static-config /tmp
sysinv-puppet create-system-config
sysinv-puppet create-host-config