Difference between revisions of "TaskFlow/Task Arguments and Results"

Revision as of 12:54, 21 October 2013

Overview

In TaskFlow, all flow state should go to storage. That includes all the information that task needs when it is executed (task arguments), and all the information task produces (task results). Developer who implements task or flow can specify what arguments task accepts and what result it returns in several ways.

Set of names of task arguments is available as requires property of the task instance. When task is about to be executed values with this names are retrieved from storage and passed to execute method of the task as keyword arguments.

Set of names of task results (what task provides) is available as provides property of task instance. After task finishes successfully, it's result(s) (what task execute method returns) are available by these names from storage (there will be examples below).

Arguments Specification

There are different way to specify task argument set.

Arguments Inference

Task arguments can be inferred from arguments of execute method of the task. For example:

   >>> class MyTask(task.Task):
   ...     def execute(self, spam, eggs):
   ...         return spam + eggs
   ... 
   >>> MyTask().requires
   set(['eggs', 'spam'])

Inference from signature is simplest way to specify task arguments. Optional arguments (with default values), and special arguments like self, *args and **kwargs are ignored on iferrence:

   >>> class MyTask(task.Task):
   ...     def execute(self, spam, eggs=()):
   ...         return spam + eggs
   ... 
   >>> MyTask().requires
   set(['spam'])
   >>>
   >>> class UniTask(task.Task):
   ...     def execute(self,  *args, **kwargs):
   ...         pass
   ... 
   >>> UniTask().requires
   set([])

@@ Line 1: / Line 1: @@
-== Step 0: rename __call__ to execute ==
+== Overview ==
-Just do that, __call__ looks too magical.
+In TaskFlow, all flow state should go to storage. That includes
+all the information that task needs when it is executed (task arguments), and
+all the information task produces (task results). Developer who implements task
+or flow can specify what arguments task accepts and what result it returns
+in several ways.
-== Required task arguments ==
+Set of names of task arguments is available as <code>requires</code> property
+of the task instance. When task is about to be executed values with this names
+are retrieved from storage and passed to <code>execute</code> method of the task
+as keyword arguments.
-Required arguments MUST be passed to task when it is executed, as
+Set of names of task results (what task provides) is available as
-<code>execute</code> method arguments.
+<code>provides</code> property of task instance. After task finishes
+successfully, it's result(s) (what task <code>execute</code> method returns) are
+available by these names from storage (there will be examples below).
-Required arguments list should be inferred from <code>execute</code> method
+== Arguments Specification ==
-signature, like we already do for <code>@task</code> decorator.
-We also need provide a way to override it, to make adaptors like FunctorTask
+There are different way to specify task argument set.
-possible.
-== Task results ==
+=== Arguments Inference ===
-Task result is what task returns from <code>execute</code> method. It is saved
+Task arguments can be inferred from arguments of <code>execute</code> method of
-in persistence layer with task details and is passed to <code>revert</code> method when/if
+the task. For example:
-task is reverted.
-To map result of already run task to task arguments, we must name task results
+    >>> class MyTask(task.Task):
-This names are used as a key to storage ('memory' in TaskMachine terms) access
+    ...     def execute(self, spam, eggs):
+    ...         return spam + eggs
+    ...
+    >>> MyTask().requires
+    set(['eggs', 'spam'])
-But we can't infer name from task signature, result naming should be explicit.
+Inference from signature is simplest way to specify task arguments.
-To make solution more symmetric to inferring names from signature, we propose
+Optional arguments (with default values), and special arguments like
-to use decorator on execute method for that.
+<code>self</code>, <code>*args</code> and <code>**kwargs</code> are
+ignored on iferrence:
-For example:
+    >>> class MyTask(task.Task):
+    ...     def execute(self, spam, eggs=()):
+    ...         return spam + eggs
+    ...
+    >>> MyTask().requires
+    set(['spam'])
+    >>>
+    >>> class UniTask(task.Task):
+    ...     def execute(self,  *args, **kwargs):
+    ...         pass
+    ...
+    >>> UniTask().requires
+    set([])
-    class MyTask(task.Task):
+=== Manually Specifying Requirements ===
-        @decorators.returns('foo', 'bar', 'baz'):
-        def execute(self, context, spam, eggs):
-            return 4, 8, 15  # 23 42
-This required arguments are <code>context</code>, <code>spam</code> and
+=== Rebind ===
-<code>eggs</code>; it returns three values: <code>foo</code>, <code>bar</code>
-and <code>baz</code>. From it's implementation it is easy to see that
-<code>foo</code> will be equal to 4, <code>bar</code> to 8 and <code>baz</code>
-to 15.
-Again, we need provide a way to override it, to make adaptors like FunctorTask
-possible.
-== Note ==
+== Results Specification ==
-Farther examples specify flows using blocks/patterns from [[TaskFlow/Patterns and Engines]].
-== Saving result with different name ==
+=== Returning One Value ===
-This is sometimes convenient to save task result with different name while
+=== Returning Tuple ===
-defining flow. For that purpose, we propose adding <code>save_as</code>
-non-required parameter for task block.
-For example, to spawn a VM, you might implement a simple task
+=== Returning Dictionary ===
-    class SpawnVMTask(taskTask):
+=== Default Provides ===
-        @decorator.returns('vm_id')
-        def execute(self, context, vm_name, vm_params):
-            servers = get_server_manager(context)
-            return servers.create(name=vm_name, **vm_params).id
-Then, to create two servers with same parameters in parallel, you can use
-parallel flow; to save VM ids with different names, e.g. <code>'vm_id_one'</code> and <code>'vm_id_two'</code>,
-add <code>save_as</code> parameters like this:
-    blocks.ParallelFlow().add(
-        blocks.Task(SpawnVMTask, save_as='vm_id_one')
-        blocks.Task(SpawnVMTask, save_as='vm_id_two')
-    )
-== Rebinding arguments ==
-There are cases when value you want to pass to task is stored with name other
-then corresponding task argument. For that it is proposed to add <code>rebind_args</code>
-task block parameter.
-There are two possible way of using it. First is to pass dictionary that maps
-task argument name to name of saved value.  For example, if you saved vm name
-with 'name' key, you can spawn vm with such name like this:
-    blocks.Task(SpawnVMTask, rebind_args={'vm_name': 'name'})
-Second, you can pass a tuple or list of argument names, and values with that
-names are passed to task. The length of tuple or list should not be less then
-number of task required parameters. For example, you can achieve same effect as
-the previous example with
-    blocks.Task(SpawnVMTask, rebind_args=('context', 'name', 'vm_params'))
-which is equivalent to more elaborate
-    blocks.Task(SpawnVMTask,
-                rebind_args=dict(context='context',
-                                 vm_name='name',
-                                 vm_params='vm_params'))
-== Optional task arguments ==
-Task gets optional arguments as keyword arguments to <code>execute</code>.
-Task authores may use <code>**kwargs</code> syntax or add optional parameters
-as positional parameters with default values.
-There is no need to specify what optional arguments task accepts, also it
-definitely worth to document them. Which optional arguments are passed to task
-is specified in flow definition, in the same way as it is done
-for argument rebinding: add extra arguments to list or dict and these arguments
-will be fetched from storage and passed to <code>execute</code> as keyword
-arguments.
-TODO(imelnikov): example here.

Difference between revisions of "TaskFlow/Task Arguments and Results"

Revision as of 12:54, 21 October 2013

Contents

Overview

Arguments Specification

Arguments Inference

Manually Specifying Requirements

Rebind

Results Specification

Returning One Value

Returning Tuple

Returning Dictionary

Default Provides