Difference between revisions of "TaskFlow/Engines"
(→Supported Types) |
|||
| (29 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
| + | '''Revised on:''' {{REVISIONMONTH1}}/{{REVISIONDAY}}/{{REVISIONYEAR}} by {{REVISIONUSER}} | ||
| + | |||
== Engine == | == Engine == | ||
| − | + | Engines are what ''really'' runs your <code>tasks</code> and <code>flows</code>. | |
| − | + | [[File:4StrokeEngine_Ortho_3D_Small_Mini.gif|frame]] | |
| − | + | An engine takes a <code>flow</code> structure (described by patterns) and uses it to decide which <code>task</code> to run and when. | |
| − | There may be different implementation of engines. Some may be easier to use ( | + | There may be different implementation of engines. Some may be easier to use (ie, require no additional infrastructure setup) and understand, others might require more complicated setup but provide better scalability. The idea and ''ideal'' is that deployers/developers of a service that uses taskflow can select an engine that suites their setup best without modifying the code of said service. This allows for that deployer/developer to start off using a simpler implementation and scaling out the service that is powered by taskflow as the service grows. In concept, all engines should implement the same interface to make it easy to replace one engine with another, and provide the same guarantees on how patterns are interpreted -- for example, if an engine runs a linear flow, the tasks should be run one after another in order no matter what type of engine is actually running that linear flow. |
| − | '''Note:''' | + | '''Note:''' engines might have different capabilities/configuration but overall the interface '''will''' remain the same and should be transparent to developers and users using taskflow. |
| − | + | === Supported Types === | |
| − | |||
| − | === | ||
==== Distributed ==== | ==== Distributed ==== | ||
| Line 19: | Line 19: | ||
When you want your applications <code>tasks</code> and <code>flows</code> to be performed in a system that is highly available & resilient to individual failure. | When you want your applications <code>tasks</code> and <code>flows</code> to be performed in a system that is highly available & resilient to individual failure. | ||
| − | + | [[Distributed_Task_Management_With_RPC|Distributed via RPC]] | |
| − | + | Supports the following: | |
| + | |||
| + | * Remote workers that connect over [http://kombu.readthedocs.org/ kombu] supported transports. | ||
| + | * Combined with jobboards, provides a high-available engine ''orchestrator'' and worker combination. | ||
| + | * ''And more...'' | ||
==== Traditional ==== | ==== Traditional ==== | ||
| Line 29: | Line 33: | ||
Supports the following: | Supports the following: | ||
| − | * Threaded engine using thread | + | * Threaded engine using a thread based [http://docs.python.org/dev/library/concurrent.futures.html#executor-objects executor]. |
| + | * Threaded engine using a provided [http://eventlet.net/ eventlet] greenthread based [http://docs.python.org/dev/library/concurrent.futures.html#executor-objects executor]. | ||
* Single threaded engine using no threads. | * Single threaded engine using no threads. | ||
| − | + | * ''And more...'' | |
| − | |||
| − | |||
| − | '' | ||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
Latest revision as of 00:45, 15 March 2014
Revised on: 3/15/2014 by Harlowja
Engine
Engines are what really runs your tasks and flows.
An engine takes a flow structure (described by patterns) and uses it to decide which task to run and when.
There may be different implementation of engines. Some may be easier to use (ie, require no additional infrastructure setup) and understand, others might require more complicated setup but provide better scalability. The idea and ideal is that deployers/developers of a service that uses taskflow can select an engine that suites their setup best without modifying the code of said service. This allows for that deployer/developer to start off using a simpler implementation and scaling out the service that is powered by taskflow as the service grows. In concept, all engines should implement the same interface to make it easy to replace one engine with another, and provide the same guarantees on how patterns are interpreted -- for example, if an engine runs a linear flow, the tasks should be run one after another in order no matter what type of engine is actually running that linear flow.
Note: engines might have different capabilities/configuration but overall the interface will remain the same and should be transparent to developers and users using taskflow.
Supported Types
Distributed
When you want your applications tasks and flows to be performed in a system that is highly available & resilient to individual failure.
Supports the following:
- Remote workers that connect over kombu supported transports.
- Combined with jobboards, provides a high-available engine orchestrator and worker combination.
- And more...
Traditional
When you want your tasks and flows to just run inside your applications existing framework and still take advantage of the functionality offered.
Supports the following:
