Difference between revisions of "Rally/RoadMap"
(→CI/CD) |
(→Unit & Functional testing) |
||
Line 38: | Line 38: | ||
= Unit & Functional testing = | = Unit & Functional testing = | ||
− | + | ||
+ | == Unit == | ||
+ | |||
+ | Unit testing is done just like in any other project: unit tests run by tox in own venv. Only thing we can do is improve coverage (currently 83% overall). | ||
+ | |||
+ | Code style is covered by tests too, so all Rally code is meets the requirements of current OpenStack Style Guidelines (http://docs.openstack.org/developer/hacking/) | ||
+ | |||
+ | == Functional == | ||
+ | |||
+ | Rally is covered by functional tests from two sides: command line interface and task configuration files. | ||
+ | |||
+ | Command line interface is covered by job gate-rally-dsvm-cli which runs various combinations of rally cli and checks for proper result. Currently we have poor coverage by this tests, but this direction is just started. | ||
+ | |||
+ | For tasks we have two jobs: gate-rally-dsvm-rally and gate-rally-dsvm-neutron-rally. Second job is just like the first, but using neutron instead of nova-network. This direction is covered near 100%, so we should just keep it so. |
Revision as of 09:17, 17 July 2014
Contents
Benchmarking
TBD
Context
TBD
Runners
TBD
Scenarios
TBD
Production Read Clean Up
TBD
Non Admin support
TBD
Pre Created Users
TBD
CLI
TBD
as-a-Service
TBD
Verification
TBD
CI/CD
Rally is integrated in OpenStack CI system just like any other project: on community servers we run unittest jobs (python-jobs) and devstack jobs (dsvm). Also other projects are using rally on gates to keep an eye on project's performance. Currently it is used by glance, keystone, cinder and neutron.
One of the main goals for today is use rally-gates in nova to investigate a famous nova quota performance issue.
Unit & Functional testing
Unit
Unit testing is done just like in any other project: unit tests run by tox in own venv. Only thing we can do is improve coverage (currently 83% overall).
Code style is covered by tests too, so all Rally code is meets the requirements of current OpenStack Style Guidelines (http://docs.openstack.org/developer/hacking/)
Functional
Rally is covered by functional tests from two sides: command line interface and task configuration files.
Command line interface is covered by job gate-rally-dsvm-cli which runs various combinations of rally cli and checks for proper result. Currently we have poor coverage by this tests, but this direction is just started.
For tasks we have two jobs: gate-rally-dsvm-rally and gate-rally-dsvm-neutron-rally. Second job is just like the first, but using neutron instead of nova-network. This direction is covered near 100%, so we should just keep it so.