Obsolete:ProjectTestingInterface
Consistent Operational Interface
OpenStack has a lot of projects. For each project, the OpenStack Jenkins needs to be able to perform a lot of tasks. If each project has a slightly different way to accomplish those tasks, it makes the management of a consistent testing infrastructure very difficult to deal with. Additionally, because of the high volume of development changes and testing, the testing infrastructure has to be able to pre-cache artifacts that are normally fetched over the internet. To that end, each project should support a consistent interface for driving tests and other necessary tasks.
End results needed
Each python project must be able to do:
- Unit tests for python2.6
- Unit tests for python2.7
- pep8 checks
- Testing Coverage Report
- source tarball generation
- Translations import/export and merge
- Documentation generation
Specific commands
To drive the above tasks, the following commands should be supported in a clean tree:
- tox -ejenkins26
- tox -ejenkins27
- tox -epep8
- tox -ejenkinscover
- tox -ejenkinsvenv python setup.py sdist
- tox -ejenkinsvenv python setup.py extract_messages
- tox -ejenkinsvenv python setup.py update_catalog
- tox -ejenkinsvenv python setup.py build_sphinx
Requirements Listing
Each project should list its operations dependencies in tools/pip-requires and additional dependencies required for testing in tools/test-requires.
Additionally, there is code in openstack-common to support 'freezing' requirements into a requirements.txt file in the root directory based on what versions of dependencies were actually installed at the time of testing. If this is enabled, the requirements.txt file will only be generated if the tarball is created inside of a virtualenv, and if it's not there, it will be ignored.
Virtual Environment Management
To support sensible testing across multiple python versions, we've started adding tox config files to the projects with the hope that eventually we can remove having install_venv.py copied in to each of the projects. There are a couple of quirks that we're working to fix with upstream which should be noted, and a couple of behaviors that the tox configs want to support.
The first is for pip caching purposes, Jenkins pre-builds a bundle of all needed requirements so that PyPI does not need to be queried at test time. Jenkins injects this into the directory as .cache.bundle. At the moment, because of the way tox does dependencies, the jobs listed above call tox environments prefixed with the work jenkins. Each tox config should have non-jenkins-prefixed jobs that are for devs to run locally that directly work off of tools/pip-requires, and then the jenkins-prefixed ones that use .cache.bundle. We're working on a patch to tox to allow it to notice a .cache.bundle file automatically, use it if it's there, and otherwise use the configured depends.
