Mistral/Testing
Contents
Different types of tests
On Mistral project we have two separate test suites:
- Unit tests - executed by Jenkins CI job in OpenStack gerrit (python-style checks and execution of all unit tests)
- Integration tests - executed by Devstack Gate job in OpenStack gerrit (integration tests for Mistral after the OpenStack deployment with devstack)
Where we can find automated tests
Mistral:
- Unit tests can be found here: https://github.com/stackforge/mistral/tree/master/mistral/tests/unit
- Integration tests can be found here: https://github.com/stackforge/mistral/tree/master/mistral/tests/functional
Python-mistralclient:
- Unit tests can be found here: https://github.com/stackforge/python-mistralclient/tree/master/mistralclient/tests/unit
- Integration tests can be found here: https://github.com/stackforge/python-mistralclient/tree/master/mistralclient/tests/functional
How to execute tests manually
Almost all existing automated tests can be executed manually on the developer's desktop (except those which check OpenStack actions). To do this you should clone mistral repository (or python-mistralclient) and perform the following shell commands:
git clone https://github.com/stackforge/mistral cd mistral
Unit tests
To run unit tests suite
tox
To run unit tests suite against a specific python version
tox -e py27
To run tests from a specific test class (using a specific python version)
tox -e py27 -- 'DataFlowEngineTest'
Integration tests
There are several test suites of intergation tests in both repositories:
Mistral:
- mistral/tests/functional/api/v1/test_mistral_basic.py - contains tests which check Mistral API v1 (workbooks, executions, tasks endpoints)
- mistral/tests/functional/api/v1/test_workflow_execution.py - contains tests which check execution of standard scenarios, task results and dependencies in v1
- mistral/tests/functional/api/v2/test_mistral_basic.py - contains tests which check Mistral API v2 (workbooks, workflows, executions, tasks, actions endpoints)
- mistral/tests/functional/engine/actions - contains tests which check Mistral integration with OpenStack components (Nova, Glance, Keystone)
Python-mistralclient:
- mistralclient/tests/functional/cli/ - contains test suites for v1 and v1 which check interaction with Mistral using CLI
- mistralclient/tests/functional/client/ - contains test suites which check integration and interaction of Mistral client and API
To run integration test suite:
- in OpenStack mode (when auth in Mistral is enabled and Mistral integrates with OpenStack components)
pip install git+http://github.com/openstack/tempest.git nosetests <mistral or mistralclient>/tests/functional
- in Non-OpenStack mode:
set auth_enable=false in the mistral.conf restart Mistral server execute: ./run_functional_tests
Test Plans
Discussion of the general direction of testing can be found here: https://etherpad.openstack.org/p/MistralTests
More detailed test plan will be available later.