Jump to: navigation, search

Rally/RoadMap

< Rally
Revision as of 10:16, 17 July 2014 by Sergey Skripnick (talk | contribs) (CI/CD: add about SLA)

Benchmarking

TBD

Context

TBD

Runners

TBD

Scenarios

TBD

Production Read Clean Up

TBD

Non Admin support

TBD

Pre Created Users

TBD

CLI

TBD

as-a-Service

TBD

Verification

TBD

CI/CD

Rally is integrated in OpenStack CI system just like any other project: on community servers we run unittest jobs (python-jobs) and devstack jobs (dsvm). Also other projects are using rally on gates to keep an eye on project's performance. Currently it is used by glance, keystone, cinder and neutron.

One of the main goals for today is use rally-gates in nova to investigate a famous nova quota performance issue.

Other goal is use Rally SLA on gates for all projects. Rally SLA is used for check if various cloud metrics within the limits of declared. These metrics can be maximum API responce time, maximum error rate etc.

Unit & Functional testing

Unit

Unit testing is done just like in any other project: unit tests run by tox in own venv. Only thing we can do is improve coverage (currently 83% overall).

Code style is covered by tests too, so all Rally code is meets the requirements of current OpenStack Style Guidelines (http://docs.openstack.org/developer/hacking/)

Functional

Rally is covered by functional tests from two sides: command line interface and task configuration files.

Command line interface is covered by job gate-rally-dsvm-cli which runs various combinations of rally cli and checks for proper result. Currently we have poor coverage by this tests, but this direction is just started.

For tasks we have two jobs: gate-rally-dsvm-rally and gate-rally-dsvm-neutron-rally. Second job is just like the first, but using neutron instead of nova-network. This direction is covered near 100%, so we should just keep it so.