OpenStack - User contributions [en]

Heat/Blueprints/StackLifecycleSchedulerHint

2014-10-21T21:36:22Z

Barnold:

'''Stack lifecycle scheduler hint blueprint'''

'''Scope:''' [Short overview and high level description of what the blueprint is trying to achieve.] 

A heat provider to may have a need for custom code to examine stack requests prior to performing the operations to create or update a stack. After the custom code completes, the provider may want to provide hints to the nova scheduler with stack related identifiers, for custom scheduler plug-in processing within Nova. The blueprint describes a mechanism
whereby when heat processes a stack, the stack id, root stack id, stack resource id, stack resource name and the path in the stack (as a list of tuples, (stackresourcename, stackname)) can be passed to nova by heat as scheduler hints, to the configured schedulers for nova. 

'''Use Cases:''' [Short overview and high level description of what the blueprint is trying to achieve.] 
Enabling holistic scheduling in Heat to provide heat stack/resource identifiers as a hint for scheduler processing within nova, such that a resource scheduler can identify scheduling decisions previously made for resources in a heat stack. 

Implementation Overview: [Provide an overview of the implementation and any algorithms that will be used] 

Modifications to instance.py and server.py classes to support providing stack hints to the nova scheduler if a new heat configuration option stack_ids_as_scheduler_hints is set to true. Within both these two classes the handle_create() makes the call to nova create to create the individual servers. Before this call is made additional scheduler hints are added with values of the stack id, root stack id, stack resource id, stack resource name and the path in the stack (as a list of tuples, (stackresourcename, stackname)). heat_root_stack_id will be set to the id of the root stack of the resource, heat_stack_id will be set to the id of the resource's parent stack, # heat_stack_name will be set to the id of the resource's parent stack, heat_path_in_stack will be set to a list of tuples, (stackresourcename, stackname) with list[0] being (None, rootstackname), and heat_resource_name will be set to the resource's name. . The implementer of a Nova plugin scheduler can use these hints to identify holistic scheduler decisions previously made, e.g. in a callout within heat via an extension made using the stack lifecycle plugpoint. 

'''Data Model Changes:''' [Are you introducing new model classes, or extending existing ones?] 

[N/A] 

'''Configuration variables:''' [List and explanation of the new configuration variables (if they exist)] 

[N/A] 

'''API's:''' [List and explanation of the new API's (if they exist)] 

[N/A] 

'''Plugin Interface:''' [Does this feature introduce any change?] 

[N/A] 

'''Required Plugin support:''' [What should the plugins do to support this new feature? (If applicable)]
Plugins that support holistic scheduling should use this hint to identify scheduling decisions previously made in Heat.
Dependencies: [List of python packages and/or OpenStack components? (If applicable)]
[N/A] 

'''CLI Requirements:''' [List of CLI requirements (If applicable)] 

[N/A] 

'''Horizon Requirements:''' [List of Horizon requirements (If applicable)] 

[N/A] 

'''Usage Example:''' [How to run/use/interface with the new feature. (If applicable)] 

Samples would be provided 

'''Test Cases:''' [Description of various test cases. (If applicable)] 

Submit heat template to the heat server. Develop a nova scheduler plugin stub to read heat_stack_id in hint and validate new id.

Heat/Blueprints/StackLifecycleSchedulerHint

2014-05-30T18:17:44Z

Barnold:

'''Stack lifecycle scheduler hint blueprint'''

'''Scope:''' [Short overview and high level description of what the blueprint is trying to achieve.] 

A heat provider to may have a need for custom code to examine stack requests prior to performing the operations to create or update a stack. After the custom code completes, the provider may want to provide hints to the nova scheduler with stack related identifiers, for custom scheduler plug-in processing within Nova. The blueprint describes a mechanism whereby when heat processes a stack resource, the stack id/stack resource id can be passed by heat as a hint, to the configured schedulers for nova. 

'''Use Cases:''' [Short overview and high level description of what the blueprint is trying to achieve.] 
Enabling holistic scheduling in Heat to provide heat stack/resource identifiers as a hint for scheduler processing within nova, such that a resource scheduler can identify scheduling decisions previously made for resources in a heat stack. 

Implementation Overview: [Provide an overview of the implementation and any algorithms that will be used] 

Modifications to instance.py and server.py classes to support providing stack hints to the nova scheduler if a new heat configuration option stack_ids_as_scheduler_hints is set to true. Within both these two classes the handle_create() makes the call to nova create to create the individual servers. Before this call is made an additional scheduler hint for the stack id plus stack resource name will be added. The key for this key/value pair will be ‘heat_stack_id’ the stack id of the stack being processed will be set as the value. The implementer of a Nova plugin scheduler can use this id to identify holistic scheduler decisions previously made, e.g. in a callout within heat via an extension made using the stack lifecycle plugpoint. 

'''Data Model Changes:''' [Are you introducing new model classes, or extending existing ones?] 

[N/A] 

'''Configuration variables:''' [List and explanation of the new configuration variables (if they exist)] 

[N/A] 

'''API's:''' [List and explanation of the new API's (if they exist)] 

[N/A] 

'''Plugin Interface:''' [Does this feature introduce any change?] 

[N/A] 

'''Required Plugin support:''' [What should the plugins do to support this new feature? (If applicable)]
Plugins that support holistic scheduling should use this hint to identify scheduling decisions previously made in Heat.
Dependencies: [List of python packages and/or OpenStack components? (If applicable)]
[N/A] 

'''CLI Requirements:''' [List of CLI requirements (If applicable)] 

[N/A] 

'''Horizon Requirements:''' [List of Horizon requirements (If applicable)] 

[N/A] 

'''Usage Example:''' [How to run/use/interface with the new feature. (If applicable)] 

Samples would be provided 

'''Test Cases:''' [Description of various test cases. (If applicable)] 

Submit heat template to the heat server. Develop a nova scheduler plugin stub to read heat_stack_id in hint and validate new id.

Heat/Blueprints/StackLifecycleSchedulerHint

2014-05-30T17:57:35Z

Barnold:

'''Stack lifecycle scheduler hint blueprint'''

'''Scope:''' [Short overview and high level description of what the blueprint is trying to achieve.] 

A heat provider to may have a need for custom code to examine stack requests prior to performing the operations to create or update a stack. After the custom code completes, the provider may want to provide hints to the nova scheduler with stack related identifiers of custom code processing for scheduler plug-in processing within Nova. The blueprint describes a mechanism whereby the stack id/stack resource id can be passed to the nova scheduler as a hint when processing the stack resource. 

'''Use Cases:''' [Short overview and high level description of what the blueprint is trying to achieve.] 
Enabling holistic scheduling in Heat to provide heat stack/resource identifiers as a hint for scheduler processing within nova, such that a resource scheduler can identify scheduling decisions previously made for resources in a heat stack. 

Implementation Overview: [Provide an overview of the implementation and any algorithms that will be used] 

Modifications to instance.py and server.py classes to support providing stack hints to the nova scheduler if a new heat configuration option stack_ids_as_scheduler_hints is set to true. Within both these two classes the handle_create() makes the call to nova create to create the individual servers. Before this call is made an additional scheduler hint for the stack id plus stack resource name will be added. The key for this key/value pair will be ‘heat_stack_id’ the stack id of the stack being processed will be set as the value. The implementer of a Nova plugin scheduler can use this id to identify holistic scheduler decisions previously made, e.g. in a callout within heat via an extension made using the stack lifecycle plugpoint. 

'''Data Model Changes:''' [Are you introducing new model classes, or extending existing ones?] 

[N/A] 

'''Configuration variables:''' [List and explanation of the new configuration variables (if they exist)] 

[N/A] 

'''API's:''' [List and explanation of the new API's (if they exist)] 

[N/A] 

'''Plugin Interface:''' [Does this feature introduce any change?] 

[N/A] 

'''Required Plugin support:''' [What should the plugins do to support this new feature? (If applicable)]
Plugins that support holistic scheduling should use this hint to identify scheduling decisions previously made in Heat.
Dependencies: [List of python packages and/or OpenStack components? (If applicable)]
[N/A] 

'''CLI Requirements:''' [List of CLI requirements (If applicable)] 

[N/A] 

'''Horizon Requirements:''' [List of Horizon requirements (If applicable)] 

[N/A] 

'''Usage Example:''' [How to run/use/interface with the new feature. (If applicable)] 

Samples would be provided 

'''Test Cases:''' [Description of various test cases. (If applicable)] 

Submit heat template to the heat server. Develop a nova scheduler plugin stub to read heat_stack_id in hint and validate new id.

Heat/Blueprints/StackLifecycleSchedulerHint

2014-05-21T18:28:33Z

Barnold: Stack lifecycle scheduler hint

'''Stack lifecycle scheduler hint blueprint'''

'''Scope:''' [Short overview and high level description of what the blueprint is trying to achieve.] 

A heat provider to may have a need for custom code to examine stack requests prior to performing the operations to create or update a stack. After the custom code completes, the provider may want to provide hints to the nova scheduler with stack related identifiers of custom code processing for scheduler plug-in processing within Nova. The blueprint describes a mechanism whereby the stack id/stack resource id can be passed to the nova scheduler as a hint when processing the stack resource. 

'''Use Cases:''' [Short overview and high level description of what the blueprint is trying to achieve.] 
Enabling holistic scheduling in Heat to provide heat stack/resource identifiers as a hint for scheduler processing within nova, such that a resource scheduler can identify scheduling decisions previously made for resources in a heat stack. 

Implementation Overview: [Provide an overview of the implementation and any algorithms that will be used] 

Modifications to instance.py and server.py classes to support providing stack hints to the nova scheduler. Within both these two classes the handle_create() makes the call to nova create to create the individual servers. Before this call is made an additional scheduler hint for the stack id plus stack resource name will be added. The key for this key/value pair will be ‘heat_stack_id’ the stack id of the stack being processed will be set as the value. The implementer of a Nova plugin scheduler can use this id to identify holistic scheduler decisions previously made in the callout within heat. 

'''Data Model Changes:''' [Are you introducing new model classes, or extending existing ones?] 

[N/A] 

'''Configuration variables:''' [List and explanation of the new configuration variables (if they exist)] 

[N/A] 

'''API's:''' [List and explanation of the new API's (if they exist)] 

[N/A] 

'''Plugin Interface:''' [Does this feature introduce any change?] 

[N/A] 

'''Required Plugin support:''' [What should the plugins do to support this new feature? (If applicable)]
Plugins that support holistic scheduling should use this hint to identify scheduling decisions previously made in Heat.
Dependencies: [List of python packages and/or OpenStack components? (If applicable)]
[N/A] 

'''CLI Requirements:''' [List of CLI requirements (If applicable)] 

[N/A] 

'''Horizon Requirements:''' [List of Horizon requirements (If applicable)] 

[N/A] 

'''Usage Example:''' [How to run/use/interface with the new feature. (If applicable)] 

Samples would be provided 

'''Test Cases:''' [Description of various test cases. (If applicable)] 

Submit heat template to the heat server. Develop a nova scheduler plugin stub to read heat_stack_id in hint and validate new id.

Heat/Blueprints/StackLifecyclePlugpoint

2014-05-20T20:43:37Z

Barnold:

'''Stack lifecycle plugpoint blueprint'''

'''Scope:''' [Short overview and high level description of what the blueprint is trying to achieve.]
 
A heat provider may have a need for custom code to examine stack requests prior to performing the operations to create or update a stack. Some applications may also need code to run after operations on a stack complete. The blueprint describes a mechanism whereby providers may easily add pre-operation calls from heat to their own code, which is called prior to performing the stack work, and post-operation calls which are made after a stack operation completes or fails.

'''Use Cases:''' [Short overview and high level description of what the blueprint is trying to achieve.]
 
There are at least two primary use cases. 
(1) Enabling holistic (whole-pattern) scheduling of the virtual resources in a template instance (stack) prior to creating or deleting them. This would usually include making decisions about where to host virtual resources in the physical infrastructure to satisfy policy requirements. It would also cover failing a stack create or update if the policies included with the stack create or update were not satisfiable, or other cloud provider policies being checked were not satisfiable. 
As an example, an application owner requires that VMs and volumes attached to them are deployed on the same rack. As another example, a cloud provider may want to enforce consultation with a license server before deploying an application. As another example, an application owner may require that their VMs be spread across a given number of availability zones. 
(2) Enabling checking of policies not related to virtual resource scheduling, with stack create or update failure if the policies would not be satisfied. 
As an example, a cloud provider may want to verify that compute resources for certain types of applications are deployed with certain security groups. As another example, a cloud provider may want to be warned when patterns with > 100 VMs are deployed. 

'''Implementation Overview:''' [Provide an overview of the implementation and any algorithms that will be used] 
An ordered registry of python classes which implement pre-operation and/or post-operation methods is required. This could be done through the existing heat plugin loading mechanism that uses plugin_manager.py, with some addition to force a full (or partial) ordering on the classes, and eventually (2) through stevedore when/if heat switches to stevedore.
Pre and post operation methods should not modify the parameter stack(s). Any modifications would be considered to be a bug. A possible exception would be to allow status changes to the stack, to facilitate error handling. [The no-modifications rule could be enforced by passing deep copies to the plug point implementations but this might be costly.]
Both pre-operation and post-operation methods can both indicate failure, which would be treated like any other stack failure. On failure, when more than one plug point implementation is registered, the post-op methods would be called for all the classes already processed, to indicate to the plug point implementation that any decisions that it made with respect to the stack can be safely undone.

All stack actions would need calls to either pre or post operations, or both. This includes at least create, update, delete, abandon, and adopt. In a basic design, modifications to the Stack class in parser.py are sufficient for adding the call to the pre-operation and post-operation methods found via the lifecycle plugin registry. The post-operation calls would need to be called in both the normal paths and all error paths.

'''Data Model Changes:''' [Are you introducing new model classes, or extending existing ones?] 
[N/A]

'''Configuration variables:''' [List and explanation of the new configuration variables (if they exist)] 
Depends on implementation details. Design proposed uses no new config variables

'''API's:''' [List and explanation of the new API's (if they exist)] 
[TODO]

'''Plugin Interface''': [Does this feature introduce any change?] 
[N/A]

'''Required Plugin support:''' [What should the plugins do to support this new feature? (If applicable)] 
Resource plugins would not be changed. Changes would be limited to the plugin loading process, and to the Stack class, and additions would include a lifecycle plugin executor, base class for lifecycle plugins, and samples, tests

'''Dependencies:''' [List of python packages and/or OpenStack components? (If applicable)] 
[N/A], unless the cloud provider has a lifecycle plug-point implementation with additional requirements

'''CLI Requirements:''' [List of CLI requirements (If applicable)] 
[N/A]

'''Horizon Requirements:''' [List of Horizon requirements (If applicable)]
[N/A]

'''Usage Example:''' [How to run/use/interface with the new feature. (If applicable)] 
Samples would be provided

'''Test Cases:''' [Description of various test cases. (If applicable)] 
The existing plugin manager tests are generic and would serve to test lifecycle plugin loading for an implementation that used the plugin manager.
[TODO] other tests are needed, and perhaps tempest tests

Heat/Blueprints/StackLifecyclePlugpoint

2014-05-20T20:42:56Z

Barnold:

'''Stack lifecycle plugpoint blueprint'''

'''Scope:''' [Short overview and high level description of what the blueprint is trying to achieve.]
 
A heat provider may have a need for custom code to examine stack requests prior to performing the operations to create or update a stack. Some applications may also need code to run after operations on a stack complete. The blueprint describes a mechanism whereby providers may easily add pre-operation calls from heat to their own code, which is called prior to performing the stack work, and post-operation calls which are made after a stack operation completes or fails.

'''Use Cases:''' [Short overview and high level description of what the blueprint is trying to achieve.]
 
There are at least two primary use cases. 
(1) Enabling holistic (whole-pattern) scheduling of the virtual resources in a template instance (stack) prior to creating or deleting them. This would usually include making decisions about where to host virtual resources in the physical infrastructure to satisfy policy requirements. It would also cover failing a stack create or update if the policies included with the stack create or update were not satisfiable, or other cloud provider policies being checked were not satisfiable. 
As an example, an application owner requires that VMs and volumes attached to them are deployed on the same rack. As another example, a cloud provider may want to enforce consultation with a license server before deploying an application. As another example, an application owner may require that their VMs be spread across a given number of availability zones.
(2) Enabling checking of policies not related to virtual resource scheduling, with stack create or update failure if the policies would not be satisfied. 
As an example, a cloud provider may want to verify that compute resources for certain types of applications are deployed with certain security groups. As another example, a cloud provider may want to be warned when patterns with > 100 VMs are deployed. 

'''Implementation Overview:''' [Provide an overview of the implementation and any algorithms that will be used] 
An ordered registry of python classes which implement pre-operation and/or post-operation methods is required. This could be done through the existing heat plugin loading mechanism that uses plugin_manager.py, with some addition to force a full (or partial) ordering on the classes, and eventually (2) through stevedore when/if heat switches to stevedore.
Pre and post operation methods should not modify the parameter stack(s). Any modifications would be considered to be a bug. A possible exception would be to allow status changes to the stack, to facilitate error handling. [The no-modifications rule could be enforced by passing deep copies to the plug point implementations but this might be costly.]
Both pre-operation and post-operation methods can both indicate failure, which would be treated like any other stack failure. On failure, when more than one plug point implementation is registered, the post-op methods would be called for all the classes already processed, to indicate to the plug point implementation that any decisions that it made with respect to the stack can be safely undone.

All stack actions would need calls to either pre or post operations, or both. This includes at least create, update, delete, abandon, and adopt. In a basic design, modifications to the Stack class in parser.py are sufficient for adding the call to the pre-operation and post-operation methods found via the lifecycle plugin registry. The post-operation calls would need to be called in both the normal paths and all error paths.

'''Data Model Changes:''' [Are you introducing new model classes, or extending existing ones?] 
[N/A]

'''Configuration variables:''' [List and explanation of the new configuration variables (if they exist)] 
Depends on implementation details. Design proposed uses no new config variables

'''API's:''' [List and explanation of the new API's (if they exist)] 
[TODO]

'''Plugin Interface''': [Does this feature introduce any change?] 
[N/A]

'''Required Plugin support:''' [What should the plugins do to support this new feature? (If applicable)] 
Resource plugins would not be changed. Changes would be limited to the plugin loading process, and to the Stack class, and additions would include a lifecycle plugin executor, base class for lifecycle plugins, and samples, tests

'''Dependencies:''' [List of python packages and/or OpenStack components? (If applicable)] 
[N/A], unless the cloud provider has a lifecycle plug-point implementation with additional requirements

'''CLI Requirements:''' [List of CLI requirements (If applicable)] 
[N/A]

'''Horizon Requirements:''' [List of Horizon requirements (If applicable)]
[N/A]

'''Usage Example:''' [How to run/use/interface with the new feature. (If applicable)] 
Samples would be provided

'''Test Cases:''' [Description of various test cases. (If applicable)] 
The existing plugin manager tests are generic and would serve to test lifecycle plugin loading for an implementation that used the plugin manager.
[TODO] other tests are needed, and perhaps tempest tests

Heat/Blueprints/StackLifecyclePlugpoint

2014-04-17T21:28:23Z

Barnold: Stack lifecycle plugpoint blueprint

'''Stack lifecycle plugpoint blueprint'''

'''Scope:''' [Short overview and high level description of what the blueprint is trying to achieve.]
 
A heat provider may have a need for custom code to examine stack requests prior to performing the operations to create or update a stack. Some applications may also need code to run after operations on a stack complete. The blueprint describes a mechanism whereby providers may easily add pre-operation calls from heat to their own code, which is called prior to performing the stack work, and post-operation calls which are made after a stack operation completes or fails.

'''Use Cases:''' [Short overview and high level description of what the blueprint is trying to achieve.]
 
There are at least two primary use cases. 
(1) Enabling holistic scheduling of the virtual resources in a template instance (stack) prior to creating or deleting them. This would usually include making decisions about where to host virtual resources in the physical infrastructure to satisfy policy requirements. It would also cover failing a stack create or update if the policies included with the stack create or update were not satisfiable. 
(2) Enabling checking of policies not related to virtual resource scheduling, with stack create or update failure if the policies would not be satisfied. 

'''Implementation Overview:''' [Provide an overview of the implementation and any algorithms that will be used] 
An ordered registry of python classes which implement pre-operation and/or post-operation methods is required. This could be done through the existing heat plugin loading mechanism that uses plugin_manager.py, with some addition to force a full (or partial) ordering on the classes, and eventually (2) through stevedore when/if heat switches to stevedore.
Pre and post operation methods should not modify the parameter stack(s). Any modifications would be considered to be a bug. A possible exception would be to allow status changes to the stack, to facilitate error handling. [The no-modifications rule could be enforced by passing deep copies to the plug point implementations but this might be costly.]
Both pre-operation and post-operation methods can both indicate failure, which would be treated like any other stack failure. On failure, when more than one plug point implementation is registered, the post-op methods would be called for all the classes already processed, to indicate to the plug point implementation that any decisions that it made with respect to the stack can be safely undone.

All stack actions would need calls to either pre or post operations, or both. This includes at least create, update, delete, abandon, and adopt. In a basic design, modifications to the Stack class in parser.py are sufficient for adding the call to the pre-operation and post-operation methods found via the lifecycle plugin registry. The post-operation calls would need to be called in both the normal paths and all error paths.

'''Data Model Changes:''' [Are you introducing new model classes, or extending existing ones?] 
[N/A]

'''Configuration variables:''' [List and explanation of the new configuration variables (if they exist)] 
Depends on implementation details. Design proposed uses no new config variables

'''API's:''' [List and explanation of the new API's (if they exist)] 
[TODO]

'''Plugin Interface''': [Does this feature introduce any change?] 
[N/A]

'''Required Plugin support:''' [What should the plugins do to support this new feature? (If applicable)] 
Resource plugins would not be changed. Changes would be limited to the plugin loading process, and to the Stack class, and additions would include a lifecycle plugin executor, base class for lifecycle plugins, and samples, tests

'''Dependencies:''' [List of python packages and/or OpenStack components? (If applicable)] 
[N/A], unless the cloud provider has a lifecycle plug-point implementation with additional requirements

'''CLI Requirements:''' [List of CLI requirements (If applicable)] 
[N/A]

'''Horizon Requirements:''' [List of Horizon requirements (If applicable)]
[N/A]

'''Usage Example:''' [How to run/use/interface with the new feature. (If applicable)] 
Samples would be provided

'''Test Cases:''' [Description of various test cases. (If applicable)] 
The existing plugin manager tests are generic and would serve to test lifecycle plugin loading for an implementation that used the plugin manager.
[TODO] other tests are needed, and perhaps tempest tests