Testing OpenStack Projects
Projects generally have three categories of tests that commits are must pass before being ready for review:
- Unit Tests
- Self contained in each project
- Style Checks
- Integration Tests
- Part of tempest
See also oslo-incubator/TESTING.rst.
There are two ways to run the unit tests for a project. The recommended way is with tox; there is an obsolescent way, which is using the project's run_tests.sh script.
Unit Testing with Tox
[apt-get | yum] install python-pip
pip install tox
At the moment there is a bug in the latest tox; see https://bugs.launchpad.net/openstack-ci/+bug/1274135 for a workaround; when installing from scratch, install a suitable downlevel version.
Sample tox.ini file https://github.com/openstack/nova/blob/master/tox.ini
There will be a tox.ini file in the root directory of every project.
Within the tox.ini file you might see any of the following envlist options:
[tox] envlist = py26,py27,py33,pep8,pylint
In order to run tox against one of these environments, you run, for example:
tox -e py27
Most projects configure tox to use testr to run the tests; see Testr. In such cases you have the option to use testr to run the tests without a virtualenv, if your main environment is suitable.
Tox Run Time
Why does tox take so long to run? The reason tox takes a long time is two-fold: On the first run it has to create a virtual environment, which can take anywhere from 5 to 30+ minutes depending on the project and the system. The other reason is that it just takes a long time to run all of the test cases in some of the projects.
There is an obsolescent convention, as follows. Most projects have a shell script, named "run_tests.sh", that runs the unit tests of that project. To invoke it, simply "cd" to the project's directory and invoke the script. For example, to run the Nova unit tests in a default DevStack installation,
cd /opt/stack/nova ./run_tests.sh
Style checks can be run via tox with `tox -epep8` or manually using `flake8`.
Manual Integration Tests
DevStack produces a running system, with all the processes running under `screen`.
will re-attach to that screen session (if it is the only one). For manual system testing: change your code in place and recycle the relevant processes, in the right order. You have to understand the code in order to know which are the relevant processes and what is the right order.
Automated Integration Tests
Running Swift Tests
Swift has functional tests, functional nose tests, and unit tests available in the source in the test directory. At Rackspace we test several environments including a staging area with 36 storage nodes.