Jump to: navigation, search

Difference between revisions of "Rally/RoadMap"

(CI/CD)
(Unit & Functional testing)
Line 38: Line 38:
  
 
=  Unit & Functional testing =
 
=  Unit & Functional testing =
TBD
+
 
 +
== Unit ==
 +
 
 +
Unit testing is done just like in any other project: unit tests run by tox in own venv. Only thing we can do is improve coverage (currently 83% overall).
 +
 
 +
Code style is covered by tests too, so all Rally code is meets the requirements of current OpenStack Style Guidelines (http://docs.openstack.org/developer/hacking/)
 +
 
 +
== Functional ==
 +
 
 +
Rally is covered by functional tests from two sides: command line interface and task configuration files.
 +
 
 +
Command line interface is covered by job gate-rally-dsvm-cli which runs various combinations of rally cli and checks for proper result. Currently we have poor coverage by this tests, but this direction is just started.
 +
 
 +
For tasks we have two jobs: gate-rally-dsvm-rally and gate-rally-dsvm-neutron-rally. Second job is just like the first, but using neutron instead of nova-network. This direction is covered near 100%, so we should just keep it so.

Revision as of 09:17, 17 July 2014

Benchmarking

TBD

Context

TBD

Runners

TBD

Scenarios

TBD

Production Read Clean Up

TBD

Non Admin support

TBD

Pre Created Users

TBD

CLI

TBD

as-a-Service

TBD

Verification

TBD

CI/CD

Rally is integrated in OpenStack CI system just like any other project: on community servers we run unittest jobs (python-jobs) and devstack jobs (dsvm). Also other projects are using rally on gates to keep an eye on project's performance. Currently it is used by glance, keystone, cinder and neutron.

One of the main goals for today is use rally-gates in nova to investigate a famous nova quota performance issue.

Unit & Functional testing

Unit

Unit testing is done just like in any other project: unit tests run by tox in own venv. Only thing we can do is improve coverage (currently 83% overall).

Code style is covered by tests too, so all Rally code is meets the requirements of current OpenStack Style Guidelines (http://docs.openstack.org/developer/hacking/)

Functional

Rally is covered by functional tests from two sides: command line interface and task configuration files.

Command line interface is covered by job gate-rally-dsvm-cli which runs various combinations of rally cli and checks for proper result. Currently we have poor coverage by this tests, but this direction is just started.

For tasks we have two jobs: gate-rally-dsvm-rally and gate-rally-dsvm-neutron-rally. Second job is just like the first, but using neutron instead of nova-network. This direction is covered near 100%, so we should just keep it so.