Rally/Updates
Contents
Weekly updates
August 4, 2014
Hello stackers,
below you will find the most recent updates in Rally:
- The periodic scenario runner has been refactored and renamed, as of now, to rpc (runs per second). NOte that this renaming also affects task configuration files that use that runner type. The runner itself now has been reimplemented with the multiprocessing.Process class (instead of multiprocessing.Pool), which potentially decreases memory usage on large iterations and reduces errors.
- A nice optimization to the chart generation code ensures that the Rally plots will be rendered fast in browsers even for tasks with a huge number of iterations completed.
- New benchmark scenarios include those testing the Nova server resize operation and a set of benchmarks for Sahara group node templates.
- We continue extending our gates with nice features. A new one are the SLA checks that add information about whether benchmark scenarios pass a set of success criteria (see an [http://logs.openstack.org/47/111347/5/gate/gate-rally-dsvm-rally/a14f11b/rally-plot/sla.txt example)
Many interesting improvements are on their path to being merged to Rally. Among them, let us mention the "rally info" command, which prints descriptions for different entities in Rally to the console, further work on the Rally documentation which gets cleaner and cleaner, and also a set of new context classes, including those for generating images for VM benchmarks, as well as for creating Neutron networks.
We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.
Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z
Stay tuned.
Regards,
The Rally team
July 17, 2014
Hello stackers,
here are some recent updates in Rally:
- Results of scenario runners are now stored asynchronously. This is precondition for such further features like piecemeal storing of results to DB (to reduce RAM usage), or progress displaying.
- Add check if required services are available before starting the scenario. This is realized by adding services validation to scenarios. Also this patch includes new validation mechanism.
- Ability to cancel task by keyboard interrupts is availiable now;
- Add base to provide interation testing;
- A lot of of patches to improve unit tests coverage.
This week, our work continues with such main novelties as adding support for Keystone API v3, adding context classes for avoiding vm creation if tenant has no network and prepare an image that will have installed the required programs, mechanism to provide rally as a service, code refactoring in context manager to make code more clear.
We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.
Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z
Stay tuned.
Regards,
The Rally team
July 07, 2014
Hello stackers,
here is the update for the last weeks. From all the work we've completed we would like to highlight the following:
- New benchmark scenarios:
- Benchmark to validate a keystone token N times at service endpoint that allows to check effect of caching related to tokens;
- Scenario for benchmark Neutron ports.
- Add nova floating ip management in VM scenario, before this patch the VM runcommand scenario used the fixed ip to ssh connect to the instance, this could only worked if fixed ip range was directly accessible with was the case in very limited deployments.
- Add image context class that allows adding images to each user for benchmarks, also this patch provides scenario which tests glance image-list command;
- Large update of documentation that simplifies its index structure and headings, extends the 'Benchmark' page with info from the "Main concepts" and adds some introductory text on Deploy & Verify parts;
- Add service-level agreement checking that allows to add section in task configuration that contains the criteria of success, e.g "less then 5% of failure rate", or "faster then 5 seconds for one iteration";
- Use oslo.db module and remove rally/openstack/common/db.
Current work includes such interesting novelties as context semantic validation, periodic runner refactoring, "rally info" command which prints descriptions for different entities in Rally, checking if required services are available before starting the scenario and refactoring of validation system, continue work on "stress" runner.
We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.
Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z
Stay tuned.
Regards,
The Rally team
June 23, 2014
Hello stackers,
here is a quick overview of the major contribution in Rally made over the past week:
- New benchmark scenarios:
- Neutron routers API;
- Ceilometer scenario fixes (e.g. the CeilometerBasic scenario group has now been renamed to CeilometerAlarms).
- A quite nice work on code cleanup according to the new hacking 0.9.x requirements has been done to be able to update Rally's global requirements;
- Rally gates have been prepared to be able to run VM scenarios (this work will be finished in the near future).
Current work includes such interesting novelties as the new "stress" scenario runner that generates a growing load until it breaks the cloud, new contexts: Image (boots VMs before benchmarking) and NeutronNetwork (adds pre-created networks) and many others. Also note that there will be soon a large update in the ReadTheDocs Rally page that will make it much more user-oriented.
We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.
Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z
Stay tuned.
Regards,
The Rally team
June 16, 2014
Hello stackers,
during the past week, our work has primarily been concentrated on minor bugfixes and enhancements in different parts of Rally. These include:
- Aggregated tables for atomic action runtimes are now available not only in the CLI, but also in the HTML output (generated by the $ rally task plot2html ... command). Let us remind that atomic actions are basic procedures that benchmark scenarios consist of (such as booting a server in Nova or deleting a user in Keystone).
- The docs section of Rally has been extended with the so-called "User stories", which describe use cases of Rally in the real world, some results obtained by Rally as well as their analysis. The first user story is about token creation under different load in Keystone.
- Bugfixes include:
- Fixed installation failure in case pip is not installed on the target system;
- Bugfixes in the quotas context introduce namings corrections and also catching possible errors in the cleanup procedure;
- The constant_for_duration scenario runner now stores results for all the benchmark iterations it has performed, not only for the last ones.
Current work includes such interesting novelties as the new "stress" scenario runner that generated a growing load until it breaks the cloud, the new Image context that boots VMs before benchmarking and many others.
We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.
Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z
Stay tuned.
Regards,
The Rally team
June 2, 2014
Hello stackers,
one of the most pleasant news for the past week is the appearance of a brand new Rally ReadTheDocs page, where all the docs from the Rally repository are available in a human-readable format.
As for recent patches merged to Rally, so let's pick up the most interesting ones:
- There is now the new Deploy Engine for FUEL (that, as other deploy engines, deploys an OpenStack installation using FUEL and returns endpoints to it).
- A nice refactoring patch has rewritten much CLI code so that it uses more Oslo code.
- Following our good tradition to extend the set of benchmark scenarios available in Rally each week, there are now scenarios for Ceilometer Statistics API.
Among new interesting patches being on their way to Rally, let's mention a filesystem benchmark (that will be launched inside created servers), new context classes for creating images and Neutron networks before launching becnhmarks, and many others.
We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.
Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z
Stay tuned.
Regards,
The Rally team
