Difference between revisions of "Mistral/Blueprints/ActionsDesign"
(→Test) |
|||
(15 intermediate revisions by 2 users not shown) | |||
Line 2: | Line 2: | ||
NOT FINISHED | NOT FINISHED | ||
This BP describes the main requirements to how actions should be designed in Mistral in order to address flexibility and clarity of their usage. | This BP describes the main requirements to how actions should be designed in Mistral in order to address flexibility and clarity of their usage. | ||
+ | === Big Picture === | ||
+ | Note: in the Big Picture section the names are intentionally not final. | ||
+ | [[File:Mistral-ActionRunner-Flow.png |center]] | ||
+ | # On start(workflow), engine creates a new workflow execution, computes the first batch of tasks, sends them to ActionRunner [1]. | ||
+ | # ActionRunner creates an action and calls action.run(input) | ||
+ | # Action does the work (e.g. compute factorial of 10), produces the results, and return the results to ActionRunner. If it returns, status=SUCCESS. If it fails it throws exception, status=ERROR. | ||
+ | # ActionRunner notifies Engine that the task is complete engine.task_done(execution, task, status, result)[2] | ||
+ | # Engine computes the next task(s) ready to trigger, according to control flow and data flow, and sends them to ActionRunner. | ||
+ | # Like step 2: ActionRunner calls the action's run(input) | ||
+ | # A delegate action doesn't produce results: it calls out the 3rd party system, which is expected to make a callback to a workflow service with the results. It returns to ActionRunner without results, "immediately" after delegating. | ||
+ | # ActionRunner marks status=RUNNING [?] | ||
+ | # 3rd party system takes 'long time' == longer then any system component can be assumed to stay alive. | ||
+ | # Like step 4: 3rd party component calls Mistral WebHook which resolves to engine.task_done(execution, task, status, result) | ||
+ | |||
+ | === Detailed Spec === | ||
From implementation perspective we'll be referring to python class Action that represents Mistral action: | From implementation perspective we'll be referring to python class Action that represents Mistral action: | ||
class Action(object): | class Action(object): | ||
Line 41: | Line 56: | ||
... | ... | ||
− | ==== | + | ==== Test mode ==== |
− | Actions may optionally implement | + | Actions may optionally implement method test() so that a user can test their workflows in 'test' mode, the mode in which every action doesn't perform a real work but instead just returns a representable result. |
− | def | + | def test(): |
− | LOG.info("Running action in | + | LOG.info("Running action in test mode [parameters=%s]" % self.parameters) |
return 'my_result' | return 'my_result' | ||
Line 52: | Line 67: | ||
Action may optionally have parameters needed to alter its behaviour. Each action has different set of parameters depending on the underlying protocol/library/technology it uses. For example, in case of HTTP action parameters will be: | Action may optionally have parameters needed to alter its behaviour. Each action has different set of parameters depending on the underlying protocol/library/technology it uses. For example, in case of HTTP action parameters will be: | ||
* url | * url | ||
− | * query string | + | * query string (params) |
* method | * method | ||
* headers | * headers | ||
Line 59: | Line 74: | ||
When Action instance is created it gets initialized with required parameters: | When Action instance is created it gets initialized with required parameters: | ||
− | class | + | class HTTPAction(object): |
− | def __init__(self, | + | def __init__(self, url, params, method, headers, body): |
− | self. | + | self.url = url |
+ | self.params = params | ||
+ | self.method = method | ||
+ | self.headers = headers | ||
+ | self.body = body | ||
+ | |||
+ | So that signature of action initializer defines all the parameters it needs. Respectively, when an action is declared in DSL it is expected to have the same set of parameters. | ||
+ | |||
+ | ==== Plugin Architecture ==== | ||
+ | Mistral should provide actions extensibility so that anyone could write their own actions and plug them into the system. The essential idea is that actions form a class hierarchy with the root class Action described above. The main requirements for action plugin system are: | ||
+ | * Ability to specify action namespace. | ||
+ | * Ability to specify action shortcut. | ||
+ | * Once an action is registered in Mistral it can be accessed in DSL using form 'namespace.action_shortcut'. | ||
+ | |||
+ | ==== DSL ==== | ||
+ | ===== Namespaces and Invocation ===== | ||
+ | Actions are grouped into namespaces. Out of the box Mistral has a number of standard actions. The standard actions can be referred via "std" namespace. For example, "std.http". With extensibility, actions can be referred with appropriate namespace, e.g. "jira.create_ticket". | ||
+ | In Mistral DSL actions can be used like shown below: | ||
+ | |||
+ | tasks: | ||
+ | task1: | ||
+ | action: std.http | ||
+ | parameters: | ||
+ | url: http://myhost.org/ping_me | ||
+ | method: GET | ||
+ | |||
+ | When an action is referenced this way a set of parameters exactly matches action class initializer signature as shown above. | ||
+ | |||
+ | ===== In-place Declarations ===== | ||
+ | New namespaces and actions can be defined in DSL, based on existing actions. | ||
+ | |||
+ | Namespaces: | ||
+ | nova: | ||
+ | actions: | ||
+ | create_vm: | ||
+ | class: std.http | ||
+ | base-parameters: | ||
+ | url: ${nova_url}/servers | ||
+ | method: POST | ||
+ | body: | ||
+ | server: | ||
+ | name: my_server | ||
+ | image_ref: ${image_ref} | ||
+ | flavor_ref: http://openstack.example.com/openstack/flavors/1 | ||
+ | parameters: | ||
+ | - nova_url | ||
+ | - image_ref | ||
+ | output: | ||
+ | vm_id: response.body.server.id | ||
+ | |||
+ | This snippet introduces new namespace "nova" and action "create_vm" so that it's possible to refer to it using "nova.create_vm". | ||
+ | Note that this DSL snippet basically creates a new action class that extends HTTPAction and works as an adapter to it transforming one set of parameters to another. It also transforms action output from raw HTTP response to a single value returned under key "vm_id". "base-parameters" is a keyword for specifying parameters of the base action (HTTPAction) where parameter values may contain placeholders of the form "${some_value}" where "some_value" refers to one of the new action parameters declared under "parameters". | ||
− | + | So a task declaration that uses this new action "nova.create_vm" can now look as follows: | |
− | + | tasks: | |
+ | create_vm: | ||
+ | action: nova.create_vm | ||
+ | parameters: | ||
+ | nova_url: 'http://localhost:999' | ||
+ | image_ref: 'http://openstack.example.com/openstack/images/70a599e0-31e7-49b7-b260-868f441e862b' |
Latest revision as of 09:34, 4 April 2014
Contents
Mistral Actions
NOT FINISHED
This BP describes the main requirements to how actions should be designed in Mistral in order to address flexibility and clarity of their usage.
Big Picture
Note: in the Big Picture section the names are intentionally not final.
- On start(workflow), engine creates a new workflow execution, computes the first batch of tasks, sends them to ActionRunner [1].
- ActionRunner creates an action and calls action.run(input)
- Action does the work (e.g. compute factorial of 10), produces the results, and return the results to ActionRunner. If it returns, status=SUCCESS. If it fails it throws exception, status=ERROR.
- ActionRunner notifies Engine that the task is complete engine.task_done(execution, task, status, result)[2]
- Engine computes the next task(s) ready to trigger, according to control flow and data flow, and sends them to ActionRunner.
- Like step 2: ActionRunner calls the action's run(input)
- A delegate action doesn't produce results: it calls out the 3rd party system, which is expected to make a callback to a workflow service with the results. It returns to ActionRunner without results, "immediately" after delegating.
- ActionRunner marks status=RUNNING [?]
- 3rd party system takes 'long time' == longer then any system component can be assumed to stay alive.
- Like step 4: 3rd party component calls Mistral WebHook which resolves to engine.task_done(execution, task, status, result)
Detailed Spec
From implementation perspective we'll be referring to python class Action that represents Mistral action:
class Action(object): @abc.abstractmethod def run(): pass
Method run() here implements action logic (issuing SSH command, sending a message over MQ etc.). All specific action classes must extend class Action and implement abstract method run().
Synchronous/Asynchronous
Mistral action can be synchronous and asynchronous. Synchronous actions act as a regular Python method: when it's called it returns a result upon completion so a caller can immediately use this result. Asynchronous actions don't return a result immediately but rather "launch" an independent activity (e.g. sending a request to an external system to do the heavy job) and when it completes a result becomes know to Mistral.
class Action(object):
@abc.abstractmethod def run(): pass def is_sync(): return False
Depending on action implementation and its properties method is_sync() returns True or False. This part of Action contract is required because a caller subsystem must know how to handle action result.
Result
The notion of 'result' only makes sense for synchronous type of actions since in case of asynchronous action a result gets delivered (conveyed directly via Mistral public API).
Asynchronous Actions
Method run() always returns None.
Synchronous Actions
Method run() returns action result. In cases when action is based on using some protocols like HTTP action implementation itself is responsible for converting protocol specific result to a form meaningful for a particular workflow. So, for example, HTTP response that action receives during its work typically can't be treated as action result since a user may be interested only in part of information residing in the response.
Errors
During action work various errors may occur. To notify action callers action must throw ActionException. In this case corresponding task state will be set to ERROR.
def run(): ... raise ActionException("Failed to send an SSH command. %s" % cause) ...
Test mode
Actions may optionally implement method test() so that a user can test their workflows in 'test' mode, the mode in which every action doesn't perform a real work but instead just returns a representable result.
def test(): LOG.info("Running action in test mode [parameters=%s]" % self.parameters) return 'my_result'
Input
Action may optionally have parameters needed to alter its behaviour. Each action has different set of parameters depending on the underlying protocol/library/technology it uses. For example, in case of HTTP action parameters will be:
- url
- query string (params)
- method
- headers
- body
When Action instance is created it gets initialized with required parameters:
class HTTPAction(object): def __init__(self, url, params, method, headers, body): self.url = url self.params = params self.method = method self.headers = headers self.body = body
So that signature of action initializer defines all the parameters it needs. Respectively, when an action is declared in DSL it is expected to have the same set of parameters.
Plugin Architecture
Mistral should provide actions extensibility so that anyone could write their own actions and plug them into the system. The essential idea is that actions form a class hierarchy with the root class Action described above. The main requirements for action plugin system are:
- Ability to specify action namespace.
- Ability to specify action shortcut.
- Once an action is registered in Mistral it can be accessed in DSL using form 'namespace.action_shortcut'.
DSL
Namespaces and Invocation
Actions are grouped into namespaces. Out of the box Mistral has a number of standard actions. The standard actions can be referred via "std" namespace. For example, "std.http". With extensibility, actions can be referred with appropriate namespace, e.g. "jira.create_ticket". In Mistral DSL actions can be used like shown below:
tasks: task1: action: std.http parameters: url: http://myhost.org/ping_me method: GET
When an action is referenced this way a set of parameters exactly matches action class initializer signature as shown above.
In-place Declarations
New namespaces and actions can be defined in DSL, based on existing actions.
Namespaces: nova: actions: create_vm: class: std.http base-parameters: url: ${nova_url}/servers method: POST body: server: name: my_server image_ref: ${image_ref} flavor_ref: http://openstack.example.com/openstack/flavors/1 parameters: - nova_url - image_ref output: vm_id: response.body.server.id
This snippet introduces new namespace "nova" and action "create_vm" so that it's possible to refer to it using "nova.create_vm". Note that this DSL snippet basically creates a new action class that extends HTTPAction and works as an adapter to it transforming one set of parameters to another. It also transforms action output from raw HTTP response to a single value returned under key "vm_id". "base-parameters" is a keyword for specifying parameters of the base action (HTTPAction) where parameter values may contain placeholders of the form "${some_value}" where "some_value" refers to one of the new action parameters declared under "parameters".
So a task declaration that uses this new action "nova.create_vm" can now look as follows:
tasks: create_vm: action: nova.create_vm parameters: nova_url: 'http://localhost:999' image_ref: 'http://openstack.example.com/openstack/images/70a599e0-31e7-49b7-b260-868f441e862b'