Difference between revisions of "TaskFlow/States of Task and Flow"

Revision as of 05:35, 7 February 2014

Revised on: 2/7/2014 by Harlowja

Flow States

A Flow starts its life in the PENDING state. Then, when it is run, it transitions to the RUNNING state, and stays in that state until it has finished; once all tasks have finished successfully the flow transitions to the SUCCESS state or, in case of failure, to the FAILURE state.

From the FAILURE state the flow begins the process of reversion; when this happens, the flow first transitions to the REVERTING state, then reversion happens, then the flow transitions to the REVERTED state, if reversion completed successfully, or to the FAILURE state if some tasks failed to revert.

In the RUNNING state and the REVERTING state the flow can be suspended. When this happens, flow transitions to SUSPENDING state immediately. In that state the engine running the flow waits for running tasks to finish (as the engine can not preempt tasks that are active). Then, when no tasks are running and all results received so far are saved, the flow leaves the SUSPENDING state. It may go to the SUCCESS state if all tasks were in fact ran, or to the REVERTED state if the flow was reverting and all tasks were reverted while the engine was waiting for running tasks to finish, or to the FAILURE state if tasks were run or reverted and some of them failed, but in most common case flow transitions to the SUSPENDED state after the SUSPENDING state. From the SUSPENDED state the flow can resume running (aka, transition back to the RUNNING state) or perform reversion (aka, transition to the REVERTING state).

When the flow is interrupted 'in a hard way' (e.g. server crashes), it can be loaded from storage in any state. If the state is not PENDING (aka, the flow was never run) or SUCCESS, FAILURE or REVERTED (in which case the flow is already finished), the flow gets set to the RESUMING state for the short time period when it is loaded from backend storage [a database, a filesystem...] (this transition is not shown on the diagram). When the flow is finally loaded, it goes to into the SUSPENDED state.

From the SUCCESS, FAILURE or REVERTED states the flow can be ran again (and thus it goes back into the RUNNING state). One of the possible use cases for this transition is to allow for alteration of a flow or flow details associated with a previously ran flow after the flow has finished, and client code wants to ensure that each task from this new (potentially updated) flow has its chance to run.

Changed in 0.1.1: When flow in REVERTED state is run, it goes through PENDING state. On REVERTED -> PENDING transition tasks are reset to PENDING and all task failure information (if any) is removed from storage.

Note: The current code also contains strong checks during each flow state transition using the model described above and raises an InvalidState exception if a invalid transition is attempted. This exception being triggered usually means there is some kind of bug in the engine code or some type of misuse/state violation is occurring, and should be reported as such.

Task States

When a task is added to a flow, it starts in the PENDING state, which means it can be executed immediately or waits for all of task it depends on to complete. Then, when the task is executed, it transitions to the RUNNING state, and stays in this state until its execute() method returns. When task is finished, it transitions to the SUCCESS state if it was finished successfully, or to the FAILURE state if not.

When the flow containing this task is being reverted, all tasks it's tasks are walked in particular order. Tasks in PENDING and REVERTED states are left as is; tasks from all the other states transition to the REVERTING state, and their revert() method is called. When that method returns, the task transitions to the REVERTED state. If this method failes (raises exception), task goes to FAILURE.

Changed in 0.1.1: Before TaskFlow 0.1.1 task was transitioned from the REVERTED state to the PENDING state happened immediately, and the main purpose of this REVERTED state was notifications. Since TaskFlow 0.1.1, task stay in REVERTED state until flow is run again (if ever). This makes task failure information available for reverted flows. When flow is run again, it's tasks are reset to PENDING state before any task is executed, and all failure information is removed.

Notable mention: Transition from the RUNNING state to the REVERTING state happens when a task was interrupted while running, and then reversion occurs. This may take place e.g. when the daemon running is killed (ex, kill -9) while running the task. This will happen even if a task does not finish its RUNNING state (aka transitions to the [SUCCESS, FAILURE] states).

3rd party states

Tasks that are running vendor-specific code may have other states that implement some custom logic/notifications and change task's metadata. However transitions between such vendor-specific states are enclosed in one of the common states which are shown in the above diagrams.

For example a task's state transitions can be:

PENDING --> RUNNING --> (*any vendor specific states/transitions*) --> RUNNING --> SUCCESS

Note: there should likely be a way to make any vendor specific states/transitions configurable in that the vendor specific states/transitions may not need to be persisted to permanent storage. Perhaps a task property maybe appropriate?