Difference between revisions of "Rally/Updates"

Revision as of 08:13, 2 June 2014

Weekly updates

June 2, 2014

Hello stackers,

one of the most pleasant news for the past week is the appearance of a brand new Rally ReadTheDocs page, where all the docs from the Rally repository are available in a human-readable format.

As for recent patches merged to Rally, so let's pick up the most interesting ones:

• There is now the new Deploy Engine for FUEL (that, as other deploy engines, deploys an OpenStack installation using FUEL and returns endpoints to it).
• A nice refactoring patch has rewritten much CLI code so that it uses more Oslo code.
• Following our good tradition to extend the set of benchmark scenarios available in Rally each week, there are now scenarios for Ceilometer Statistics API.

Among new interesting patches being on their way to Rally, let's mention a filesystem benchmark (that will be launched inside created servers), new context classes for creating images and Neutron networks before launching becnhmarks, and many others.

We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.

Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z

Stay tuned.

Regards,
The Rally team

May 26, 2014

Hello stackers,

the interesting contributions to Rally during the past week comprise:

• Yet another set of new benchmark scenarios: this time, for Ceilometer Resources GET API;
• Documentation for the recently merged plugins functionality (let us remind you that this mechanism enables the developers to extend Rally with new scenarios in an easy way, without actually contributing to the Rally code);
• There is now a new syntax for marking blocks of benchmark scenarios code that should be timed by Rally as atomic actions (their runtime statistics is available in the detailed results output after benchmarking completes). This syntax is based on the with-clause in Python and enables the developers to write constructs like with AtomicAction(scenario, "name_of_action"): <do_smth>, which tends to be much simpler in many cases that marking entire methods with a special @atomic_action_timer decorator (the previous approach).

This week, our contribution to Rally is going to be continued with new benchmark scenarios (e.g. those for Celiometer Statistics), several refactoring patches (e.g. aimed at somewhat more frequent usage of the Oslo code) and many others.

We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.

Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z

Stay tuned.

Regards,
The Rally team

May 19, 2014

Hello stackers,

during the OpenStack Summit in Atlanta we still have accomplished several important contributions to Rally:

• Another set of new benchmark scenarios in Rally includes those for:

• Neutron subnets;
• Ceilometer meters;
• there is now also a "dummy" scenario which throws an Exception with the given probability, which is very useful for testing purposes.

• Important bugfixes have been issued for:

• OpenStack clients usage in cleanup;
• CLI behaviour in case of a reference to non-existing benchmark task.

• Finally, pretty much work has been done on code refactoring, such as:

• unifying the methods for random resource name generation;
• removing the usage of the deprecated "message" attribute of Exception objects.

Among the most interesting patches being either in progress of ready for review, let's mention more benchmark scenarios for Neutron, Heat and Ceilometer, further improvements in the CLI code, and also the work on the extended input config format which will allow the end-user to put resource names there (e.g. image names to boot servers from) instead of concrete IDs.

We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.

Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z

Stay tuned.

Regards,
The Rally team

May 12, 2014

Hello stackers,

here's what has happened in the project during the past week:

• The list of benchmark scenarios available in Rally "out-of-box" has been extended with new scenarios for Neutron, Ceilometer (Alarms API) and Tempest.
• There is now a new mechanism for dynamic scenario plugins loading, which enables the developers to extend Rally with new scenarios in an easy way. Note also the docs on how to use plugins in Rally.
• Overall code improvements include further work on getting rid of the undocumented ThreadPool class, improving logs and others.

Despite the ongoing OpenStack Summit in Atlanta, we still have ambitious plans for this week, which include adding more benchmark scenarios (Heat, Neutron, Ceilometer etc.), working on code runtime efficiency improvements, improving the code for drawing plots and so on.

We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.

Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z

Stay tuned.

Regards,
The Rally team

May 05, 2014

Hello stackers,

due to holidays there hasn't been very much contribution to Rally during the past week. The highlights are as follows:

• Improved CLI output after a benchmark task gets finished. The two tables for atomic actions / overall results now got merged into one very neatly looking table;
• Several important code improvements include getting rid of the undocumented ThreadPool class and also moving some useful functions for benchmarks data processing into a separate module;
• New functionality in Rally for this week is the support for the Ironic client (and potentially also for benchmark scenarios for Ironic) and the introduction of Dummy benchmark scenarios that tend to be very useful for developers while testing their changes.

There is going to be much work this week that includes adding new benchmark scenarios (e.g. for Tempest, Neutron and Ceilometer), further improvements in CLI and graphs and also several bugfixes.

We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.

Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z

Stay tuned.

Regards,
The Rally team

April 28, 2014

Hello stackers,

the most important changes during the past week in Rally have been concentrated on creating completely new benchmark scenario classes and adding support for new contexts. To be more precise:

• We've added benchmark scenarios that launch Tempest tests and check the performance of updating quotas for nova/cinder. Both are quite new to Rally and allow users to compose more interesting benchmark cases.
• There is one new context, the "roles" context that allows to change user roles before using those users in benchmark scenarios. Much work has also been done on refactoring the "cleanup" context and helper methods that it uses.
• We've integrated the new oslotest library, wich allowed us to remove much of the "common openstack" code from our repository.

This week, we'll continue our work on expanding the range projects supported by our benchmarks. Benchmark scenarios already pending for review are those for Heat, Ceilometer and also Tempest. The ongoing work also includes several improvents in the overall user experience, e.g. changes in the CLI output format.

We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.

Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z

Stay tuned.

Regards,
The Rally team

April 21, 2014

Hello stackers,

here is the report of our recent activities in Rally. They have been quite numerous this week due to the contributions of our constantly growing team and include a couple of changes that touch the end user:

• The scenario runner previously known as continuous has been split to 2 scenario runners named constant and constant_for_duration, thus becoming more "atomic" (previously, the continuous scenario runner was able to run scenarios both for the specified amount of times and for some duration. These two types of behaviour now belong to different runners). "Constant" means that both scenario runners generate a constant load on the cloud;
• Several changes in the input configuration files formats include:

• Renaming the "name" attribute of deployments to "type" in the deployment configuration file. The same change now also applies to server providers;
• Renaming the "active_users" attribute of the constant scenario runners to "concurrency" in the benchmark task configuration files.

• Several improvements in context classes:

• Fixing a couple of issues with the cleaner context: incorrect context initialization and buggy cleanup of Glance resources;
• Improving the user context by making it create the temporary users in concurrent mode by default.

• New benchmark scenarios include those for Keystone tenants and users listing;

This week, we continue our work on new benchmark scenarios for Tempest and quotas, new contexts as well as on many other improvements in Rally.

We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.

Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z

Stay tuned.

Regards,
The Rally team

April 14, 2014

Hello stackers,

let us list our main contributions to Rally during the past week:

• The Bechmark context manager class, which mostly finishes the work on contexts integration in Rally; this manager allows Rally to use only those contexts that are really needed for the current benchmark scenario, thus greatly reducing the runtime of the preparation step, manages the order of entering those contexts, adds logging for contexts and does a couple of other important things;
• Improvement of the CLI for the Tempest verifier: we've added the functionality of saving verification results into a database and also a range of new commands to handle these results, such as rally verify list, rally verify detailed and others;
• Validation for required parameters in benchmark scenarios has now been implemented through a new validator, added, as usual, as a decorator to the corresponding benchmark scenarios;

This week, our work continues with several new benchmark scenarios (including those for Tempest), new validators and also new functionality in the Rally core, e.g. the ability to use already existing tenants/users for benchmarking, implemented now through contexts.

We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.

Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=icehouse&metric=commits&project_type=all&module=rally
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z

Stay tuned.

Regards,
The Rally team

April 7, 2014

Hello stackers,

our contributions this week have been pretty diverse and touched different parts of Rally. Let us name a few that seem to be the most interesting ones:

• Nova and Cinder quotas management through a new context class. Another example of how useful and easy to use contexts are in Rally;
• Concurrent users creation and deletion in the user context. This change greatly reduces the runtime of this benchmarking step;
• New benchmark scenarios for Keystone and Glance;
• Nice enhancement of the auto-generated report html page through adding histograms for runtimes there.

There are also other parts of Rally which are in progress now and will hopefully have their patches merged the next week, e.g. the work on Tempest integration or further context classes implementation.

We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.

Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=icehouse&metric=commits&project_type=all&module=rally
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z

Stay tuned.

Regards,
The Rally team

Archives

• Weekly updates archives - March 2014
• Weekly updates archives - February 2014
• Weekly updates archives - January 2014
• Weekly updates archives - December 2013
• Weekly updates archives - November 2013