Mistral/DSLv2

Mistral DSL version 2 specification

Main objects

• 'Actions
• Workflows
• 'Tasks
• Triggers
• Triggers

Action

TODO

Attributes

• name - action name (string without space, mandatory attribute).
• base - name of base action that this action is built on top of.
• base-input - dictionary whose structure is defined by action class. For example, for 'std.http' action it contains 'url', 'method', 'body' and 'headers' according to HTTP protocol specification.
• input - list containing parameter names which should or could be specified in task. This attribute is optional and used only for documenting purposes.
• output - any data structure defining how to transform the output of base action into the output of this action. I can optionally have YAQL expressions to access properties of base action output. (See more about YAQL at https://pypi.python.org/pypi/yaql/0.3)

YAML example:

TODO

Workflow

Attributes

• tasks - list of tasks in this workflow, each task represents a computational step in the workflow.

YAML example:

TODO

Task

Represents a step in workflow, for example 'TODO'

Attributes

• action - name of action to perform
• requires - list of tasks which should be execute before this tasks, or list of task names as a keys and condition as a value, this is optional parameter
• parameters - actual parameters for the task, each value can be either some number, string etc, or YAQL expression to retrieve value from task context
• on-success - task which will be scheduled on execution after current task has finished with state 'SUCCESS'
• on-error - task which will be scheduled on execution after current task has finished with state 'ERROR'
• on-finish - task which will be scheduled on execution after current task has finished

YAML example:

create-vm:
  action: Nova.create-vm
  parameters:
    image_id: $.image_id
    flavor_id: 42
  requires:
    task2: '$.value2 = 123'
    task4: '$.value4 = 122'
  on-success: task3

Triggers

Using triggers it is possible to run workflows according to specific rules: periodically setting a cron (http://en.wikipedia.org/wiki/Cron) pattern or on external events like ceilometer alarm.

Attributes

• type - can be PERIODIC, CEILOMETER_ALARM
• tasks - list of tasks which should be execute on trigger
• parameters - list of task parameters

YAML example:

triggers:
  backup-vm:
    type: periodic
    tasks: [create_backup, delete_old_backup] 
    parameters:
      cron-pattern: 1 0 * * *

Full YAML example:

This example requires the following properties provided in execution context:

1. - nova_url ## url to Nova service, e.g. http://0.0.0.0:8774/v3
2. - server_name ## Name you want to give to new instance
3. - image_id ## image id from Glance service
4. - flavor_id ## flavor id - type of instance hardware
5. - ssh_username ## username of your VM
6. - ssh_password ## password to your VM
7. - admin_email ## email address to send notifications to
8. - from_email ## email address to send notifications from
9. - smtp_server ## SMTP server to use for sending emails (e.g. smtp.gmail.com:587)
10. - smtp_password ## password to connect to SMTP server

Namespaces:
 Nova:
   # Nova actions for creating VM, retrieving IP and VM deleting.
   class: std.http
   actions:
     createVM:
       base-parameters:
         url: '{$.nova_url}/{$.project_id}/servers'
         method: POST
         headers:
           X-Auth-Token: $.auth_token
           Content-Type: application/json
         body:
           server:
             name: $.server_name
             imageRef: $.image_id
             flavorRef: $.flavor_id
       output:
         vm_id: $.content.server.id
 
     getIP:
       base-parameters:
         url: '{$.nova_url}/{$.project_id}/servers/{$.vm_id}'
         method: GET
         headers:
           X-Auth-Token: $.auth_token
       output:
         vm_ip: "$.content.server.addresses.novanetwork.where($.'OS-EXT-IPS:type' = 'floating')[0].addr"
 
     deleteVM:
       base-parameters:
         url: '{$.nova_url}/{$.project_id}/servers/{$.vm_id}'
         method: DELETE
         headers:
           X-Auth-Token: $.auth_token
       output:
         status: $.status
 
 Server:
   actions:
     # HTTP request to the server.
     calcSumm:
       class: std.http
       base-parameters:
         url: 'http://{$.vm_ip}:5000/summ'
         method: POST
         body:
           arguments: $.arguments
       output:
         summ_result: $.content.result
 
 Ssh:
   class: std.ssh
   base-parameters:
     host: $.vm_ip
     username: $.username
     password: $.password
   actions:
     # Simple SSH command.
     waitSSH:
       base-parameters:
         cmd: 'ls -l'
 
     # SSH command to run the server.
     runServer:
       base-parameters:
         cmd: 'nohup python ~/web_app.py > web_app.log &'
 
Workflow:
 tasks:
   # Create a VM (request to Nova).
   createVM:
     action: Nova.createVM
     parameters:
       server_name: $.server_name
       image_id: $.image_id
       flavor_id: $.flavor_id
       nova_url: $.nova_url
       project_id: $.project_id
       auth_token: $.auth_token
     publish:
       vm_id: vm_id
     on-success: waitForIP
     on-error: sendCreateVMError
 
   # Wait till the VM is assigned with IP address (request to Nova).
   waitForIP:
     action: Nova.getIP
     retry:
       count: 10
       delay: 10
     publish:
       vm_ip: vm_ip
     parameters:
       nova_url: $.nova_url
       project_id: $.project_id
       auth_token: $.auth_token
       vm_id: $.vm_id
     on-success: waitSSH
     on-error: sendCreateVMError
 
   # Wait till operating system on the VM is up (SSH command).
   waitSSH:
     action: Ssh.waitSSH
     retry:
       count: 10
       delay: 10
     parameters:
       username: $.ssh_username
       password: $.ssh_password
       vm_ip: $.vm_ip
     on-success: runServer
     on-error: sendCreateVMError
 
   # When SSH is up, we are able to run the server on VM (SSH command).
   runServer:
     action: Ssh.runServer
     parameters:
       vm_ip: $.vm_ip
       username: $.ssh_username
       password: $.ssh_password
     on-success: calcSumm
     on-error: sendCreateVMError
 
   # Send HTTP request on server and calc the result.
   calcSumm:
     action: Server.calcSumm
     retry:
       count: 10
       delay: 1
     parameters:
       arguments:
         - 32
         - 45
         - 23
       vm_ip: $.vm_ip
     publish:
       result: summ_result
     on-finish: sendResultEmail
 
   # In case of createVM error send e-mail with error message.
   sendResultEmail:
     action: std.email
     parameters:
       params:
         to: [$.admin_email]
         subject: Workflow result
         body: |
           Workflow result of execution {$.__execution.id} is {$.result}
 
           -- Thanks, Mistral Team.
       settings:
         smtp_server: $.smtp_server
         from: $.from_email
         password: $.smtp_password
     on-finish: deleteVM
 
   # In case of createVM error send e-mail with error message.
   sendCreateVMError:
     action: std.email
     parameters:
       params:
         to: [$.admin_email]
         subject: Workflow error
         body: |
           Failed to create a VM in execution {$.__execution.id}
 
           -- Thanks, Mistral Team.
       settings:
         smtp_server: $.smtp_server
         from: $.from_email
         password: $.smtp_password
     on-finish: deleteVM
 
   # Destroy the VM (request to Nova).
   deleteVM:
     action: Nova.deleteVM
     parameters:
       nova_url: $.nova_url
       project_id: $.project_id
       auth_token: $.auth_token
       vm_id: $.vm_id

Initial execution context

 {
   "nova_url": <nova_url>,
   "image_id": <image_id>,
   "flavor_id": <flavor_id>,
   "server_name": <server_name>,
   "ssh_username": <ssh_username>,
   "ssh_password": <ssh_password>,
   "admin_email": <admin_email>,
   "from_email": <from_email>,
   "smtp_server": <smtp_server>,
   "smtp_password": <smtp_password>,
 }

When a workflow starts Mistral also adds execution information into the context so the context looks like the following:

 {
   "nova_url": TBD,
   "image_id": TBD,
   "image_id": <image_id>,
   "flavor_id": <flavor_id>,
   "server_name": <server_name>,
   "ssh_username": <ssh_username>,
   "ssh_password": <ssh_password>,
   "admin_email": <admin_email>,
   "from_email": <from_email>,
   "smtp_server": <smtp_server>,
   "smtp_password": <smtp_password>,
   "__execution": {
     "id": "234234",
     "workbook_name" : "my_workbook",
     "project_id": "ghfgsdfasdfasdf"
     "auth_token": "sdfljsdfsdf-234234234",
     "trust_id": "oiretoilkjsdfglkjsdfglkjsdfg"
   }
 }