Jump to: navigation, search

Difference between revisions of "Rally/Updates"

(Weekly updates)
(Archived April & May 2014)
Line 146: Line 146:
 
Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally<br/>
 
Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally<br/>
 
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally <br/>
 
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally <br/>
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z
 
 
 
Stay tuned.
 
 
 
Regards,<br/>
 
The Rally team
 
 
 
 
 
==== May 26, 2014 ====
 
 
Hello stackers,
 
 
the interesting contributions to Rally during the past week comprise:
 
* '''Yet another set of new benchmark scenarios''': this time, for [https://review.openstack.org/90203 Ceilometer Resources GET API];
 
* [https://review.openstack.org/92364 Documentation] for the recently merged '''plugins functionality''' (let us remind you that this mechanism enables the developers to extend Rally with new scenarios in an easy way, without actually contributing to the Rally code);
 
* There is now a [https://review.openstack.org/#/c/92800/ new syntax] for marking blocks of benchmark scenarios code that should be timed by Rally as '''atomic actions''' (their runtime statistics is available in the detailed results output after benchmarking completes). This syntax is based on the '''with'''-clause in Python and enables the developers to write constructs like ''with AtomicAction(scenario, "name_of_action"): <do_smth>'', which tends to be much simpler in many cases that marking entire methods with a special '''@atomic_action_timer''' decorator (the previous approach).
 
 
 
This week, our contribution to Rally is going to be continued with new benchmark scenarios (e.g. those for [https://review.openstack.org/91320 Celiometer Statistics]), several refactoring patches (e.g. aimed at [https://review.openstack.org/94143 somewhat more frequent usage of the Oslo code]) and many others.
 
 
 
We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.
 
 
Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally<br/>
 
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally <br/>
 
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z
 
 
 
Stay tuned.
 
 
 
Regards,<br/>
 
The Rally team
 
 
 
 
 
==== May 19, 2014 ====
 
 
Hello stackers,
 
 
during the OpenStack Summit in Atlanta we still have accomplished several important contributions to Rally:
 
* Another set of '''new benchmark scenarios''' in Rally includes those for:
 
:* [https://review.openstack.org/91350 '''''Neutron''''' subnets];
 
:* [https://review.openstack.org/#/c/90331/ '''''Ceilometer''''' meters];
 
:* there is now also a [https://review.openstack.org/91679 "dummy" scenario] which throws an ''Exception'' with the given probability, which is very useful for testing purposes.
 
* Important '''bugfixes''' have been issued for:
 
:* [https://review.openstack.org/#/c/93413/ OpenStack clients usage] in cleanup;
 
:* [https://review.openstack.org/89998 CLI behaviour] in case of a reference to non-existing benchmark task.
 
* Finally, pretty much work has been done on '''code refactoring''', such as:
 
:* [https://review.openstack.org/93586 unifying the methods for random resource name generation];
 
:* [https://review.openstack.org/93575 removing the usage of the deprecated "message" attribute of Exception objects].
 
 
 
Among the most interesting patches being either in progress of ready for review, let's mention more benchmark scenarios for [https://review.openstack.org/92873 '''''Neutron'''''], [https://review.openstack.org/86431 '''''Heat'''''] and [https://review.openstack.org/90203 '''''Ceilometer'''''], [https://review.openstack.org/90005 further] [https://review.openstack.org/94143 improvements] in the CLI code, and also the work on the [https://review.openstack.org/86116 extended input config format] which will allow the end-user to put resource names there (e.g. image names to boot servers from) instead of concrete IDs.
 
 
 
We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.
 
 
Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally<br/>
 
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally <br/>
 
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z
 
 
 
Stay tuned.
 
 
 
Regards,<br/>
 
The Rally team
 
 
 
 
 
==== May 12, 2014 ====
 
 
Hello stackers,
 
 
here's what has happened in the project during the past week:
 
* The list of benchmark scenarios available in Rally "out-of-box" has been extended with new scenarios for [https://review.openstack.org/88537 '''''Neutron'''''], [https://review.openstack.org/88216 '''''Ceilometer''''' ''(Alarms API)''] and [https://review.openstack.org/86836 '''''Tempest'''''].
 
* There is now a new mechanism for [https://review.openstack.org/72679 dynamic '''''scenario plugins''''' loading], which enables the developers to extend Rally with new scenarios in an easy way. Note also the [https://review.openstack.org/#/c/92364/ docs] on how to use plugins in Rally.
 
* Overall code improvements include further work on [https://review.openstack.org/91398 getting rid of the undocumented ThreadPool class], [https://review.openstack.org/93276 improving logs] and others.
 
 
 
Despite the ongoing OpenStack Summit in Atlanta, we still have ambitious plans for this week, which include adding more benchmark scenarios ([https://review.openstack.org/86431 '''''Heat'''''], [https://review.openstack.org/91350 '''''Neutron'''''], [https://review.openstack.org/90331 '''''Ceilometer'''''] etc.), working on [https://review.openstack.org/93413 code runtime efficiency improvements], improving the [https://review.openstack.org/91680 code for drawing plots] and so on.
 
 
 
We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.
 
 
Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally<br/>
 
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally <br/>
 
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z
 
 
 
Stay tuned.
 
 
 
Regards,<br/>
 
The Rally team
 
 
 
 
 
==== May 05, 2014 ====
 
 
Hello stackers,
 
 
due to holidays there hasn't been very much contribution to Rally during the past week. The highlights are as follows:
 
* [https://review.openstack.org/89941 Improved '''CLI output'''] after a benchmark task gets finished. The two tables for atomic actions / overall results now got merged into [http://paste.openstack.org/show/77630/ one very neatly looking table];
 
* Several important code improvements include [https://review.openstack.org/89375 getting rid of the undocumented ThreadPool class] and also [https://review.openstack.org/89526 moving some useful functions for benchmarks data processing into a separate module];
 
* New functionality in Rally for this week is [https://review.openstack.org/87603 the support for the '''Ironic''' client] (and potentially also for benchmark scenarios for Ironic) and the introduction of [https://review.openstack.org/89219 '''Dummy''' benchmark scenarios] that tend to be very useful for developers while testing their changes.
 
 
 
There is going to be much work this week that includes adding new benchmark scenarios (e.g. for [https://review.openstack.org/92022 Tempest], [https://review.openstack.org/88537 Neutron] and [https://review.openstack.org/88216 Ceilometer]), further improvements in CLI and graphs and also several bugfixes.
 
 
 
We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.
 
 
Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally<br/>
 
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally <br/>
 
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z
 
 
 
Stay tuned.
 
 
 
Regards,<br/>
 
The Rally team
 
 
 
 
 
==== April 28, 2014 ====
 
 
Hello stackers,
 
 
the most important changes during the past week in Rally have been concentrated on creating completely new benchmark scenario classes and adding support for new contexts. To be more precise:
 
* We've added benchmark scenarios that [https://review.openstack.org/86337 launch '''Tempest''' tests] and check the performance of [https://review.openstack.org/#/c/87283/ updating '''quotas for nova/cinder''']. Both are quite new to Rally and allow users to compose more interesting benchmark cases.
 
* There is one new context, the [https://review.openstack.org/87984 '''''"roles"''''' context] that allows to change user roles before using those users in benchmark scenarios. Much work has also been done on [https://review.openstack.org/90579 refactoring the '''''"cleanup"''''' context] and helper methods that it uses.
 
* We've [https://review.openstack.org/89988 integrated the new '''''oslotest''''' library], wich allowed us to remove much of the "common openstack" code from our repository.
 
 
 
This week, we'll continue our work on expanding the range projects supported by our benchmarks. Benchmark scenarios already pending for review are those for [https://review.openstack.org/#/c/86431/ '''''Heat'''''], [https://review.openstack.org/#/c/88216/ '''''Ceilometer'''''] and also [https://review.openstack.org/86836 '''''Tempest''''']. The ongoing work also includes several improvents in the overall user experience, e.g. changes in the [https://review.openstack.org/89941 CLI output format].
 
 
 
We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.
 
 
Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally<br/>
 
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally <br/>
 
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z
 
 
 
Stay tuned.
 
 
 
Regards,<br/>
 
The Rally team
 
 
 
 
 
==== April 21, 2014 ====
 
 
Hello stackers,
 
 
here is the report of our recent activities in Rally. They have been quite numerous this week due to the contributions of our constantly growing team and include a couple of changes that touch the end user:
 
* The scenario runner previously known as '''''continuous''''' [https://review.openstack.org/#/c/83316/ has been split] to 2 scenario runners named '''''constant''''' and '''''constant_for_duration''''', thus becoming more "atomic" (previously, the continuous scenario runner was able to run scenarios both for the specified amount of times and for some duration. These two types of behaviour now belong to different runners). '''''"Constant"''''' means that both scenario runners generate a constant load on the cloud;
 
* Several changes in the input configuration files formats include:
 
:* [https://review.openstack.org/#/c/88493/ Renaming the '''''"name"''''' attribute of deployments to '''''"type"'''''] in the deployment configuration file. The same change now also applies to [https://review.openstack.org/#/c/88648/ server providers];
 
:* [https://review.openstack.org/#/c/89309/ Renaming the '''''"active_users"''''' attribute of the constant scenario runners to '''''"concurrency"'''''] in the benchmark task configuration files.
 
* Several '''''improvements in context classes''''':
 
:* Fixing a couple of issues with the  '''''cleaner context''''': [https://review.openstack.org/#/c/87761 incorrect context initialization] and [https://review.openstack.org/#/c/87764 buggy cleanup of Glance resources];
 
:* [https://review.openstack.org/#/c/89292/ Improving the '''''user context'''''] by making it create the temporary users in concurrent mode by default.
 
* '''''New benchmark scenarios''''' include those for [https://review.openstack.org/#/c/86267/ '''''Keystone tenants and users listing'''''];
 
 
 
This week, we continue our work on new benchmark scenarios for [https://review.openstack.org/#/c/86337/ '''''Tempest'''''] and [https://review.openstack.org/#/c/87283/ '''''quotas'''''], new [https://review.openstack.org/#/c/87984/ '''''contexts'''''] as well as on many other improvements in Rally.
 
 
 
We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.
 
 
Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally<br/>
 
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally <br/>
 
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z
 
 
 
Stay tuned.
 
 
 
Regards,<br/>
 
The Rally team
 
 
 
 
 
==== April 14, 2014 ====
 
 
Hello stackers,
 
 
let us list our main contributions to Rally during the past week:
 
* The [https://review.openstack.org/#/c/85732/ '''''Bechmark context manager class'''''], which mostly finishes the work on contexts integration in Rally; this manager allows Rally to use only those contexts that are really needed for the current benchmark scenario, thus greatly reducing the runtime of the preparation step, manages the order of entering those contexts, adds logging for contexts and does a couple of other important things;
 
* Improvement of the CLI for the '''''Tempest verifier''''': we've added [https://review.openstack.org/#/c/85584/ '''''the functionality of saving verification results into a database'''''] and also [https://review.openstack.org/#/c/85584/ '''''a range of new commands to handle these results'''''], such as ''rally verify list'', ''rally verify detailed'' and others;
 
* [https://review.openstack.org/#/c/84961/ '''''Validation for required parameters in benchmark scenarios'''''] has now been implemented through a new validator, added, as usual, as a decorator to the corresponding benchmark scenarios;
 
 
 
This week, our work continues with several new '''''benchmark scenarios''''' (including those for [https://review.openstack.org/#/c/86337/ '''''Tempest''''']), new [https://review.openstack.org/#/c/86655/ '''''validators'''''] and also new functionality in the Rally core, e.g. the ability to use already existing tenants/users for benchmarking, implemented now through contexts.
 
 
 
We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.
 
 
Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally<br/>
 
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=icehouse&metric=commits&project_type=all&module=rally <br/>
 
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z
 
 
 
Stay tuned.
 
 
 
Regards,<br/>
 
The Rally team
 
 
 
 
 
==== April 7, 2014 ====
 
 
Hello stackers,
 
 
our contributions this week have been pretty diverse and touched different parts of Rally. Let us name a few that seem to be the most interesting ones:
 
* [https://review.openstack.org/#/c/83835/ '''''Nova and Cinder quotas management'''''] through a new context class. Another example of how useful and easy to use contexts are in Rally;
 
* '''''Concurrent users [https://review.openstack.org/#/c/79968/ creation] and [https://review.openstack.org/#/c/84106/ deletion]''''' in the user context. This change greatly reduces the runtime of this benchmarking step;
 
* '''''New benchmark scenarios for [https://review.openstack.org/#/c/81781/ Keystone] and [https://review.openstack.org/#/c/82717/ Glance]''''';
 
* Nice enhancement of the auto-generated report html page through adding [https://review.openstack.org/#/c/82912/ '''''histograms for runtimes'''''] there.
 
 
 
There are also other parts of Rally which are in progress now and will hopefully have their patches merged the next week, e.g. the work on Tempest integration or further context classes implementation.
 
 
 
We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.
 
 
Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally<br/>
 
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=icehouse&metric=commits&project_type=all&module=rally <br/>
 
 
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z
 
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z
  
Line 403: Line 158:
  
 
==== Archives ====
 
==== Archives ====
 +
* [[Rally/UpdatesMay2014|Weekly updates archives - May 2014]]
 +
* [[Rally/UpdatesApril2014|Weekly updates archives - April 2014]]
 
* [[Rally/UpdatesMarch2014|Weekly updates archives - March 2014]]
 
* [[Rally/UpdatesMarch2014|Weekly updates archives - March 2014]]
 
* [[Rally/UpdatesFebruary2014|Weekly updates archives - February 2014]]
 
* [[Rally/UpdatesFebruary2014|Weekly updates archives - February 2014]]

Revision as of 14:49, 4 August 2014

Weekly updates

July 17, 2014

Hello stackers,

here are some recent updates in Rally:


This week, our work continues with such main novelties as adding support for Keystone API v3, adding context classes for avoiding vm creation if tenant has no network and prepare an image that will have installed the required programs, mechanism to provide rally as a service, code refactoring in context manager to make code more clear.


We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.

Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z


Stay tuned.


Regards,
The Rally team


July 07, 2014

Hello stackers,

here is the update for the last weeks. From all the work we've completed we would like to highlight the following:

  • New benchmark scenarios:


Current work includes such interesting novelties as context semantic validation, periodic runner refactoring, "rally info" command which prints descriptions for different entities in Rally, checking if required services are available before starting the scenario and refactoring of validation system, continue work on "stress" runner.


We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.

Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z


Stay tuned.


Regards,
The Rally team


June 23, 2014

Hello stackers,

here is a quick overview of the major contribution in Rally made over the past week:

  • New benchmark scenarios:


Current work includes such interesting novelties as the new "stress" scenario runner that generates a growing load until it breaks the cloud, new contexts: Image (boots VMs before benchmarking) and NeutronNetwork (adds pre-created networks) and many others. Also note that there will be soon a large update in the ReadTheDocs Rally page that will make it much more user-oriented.


We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.

Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z


Stay tuned.


Regards,
The Rally team


June 16, 2014

Hello stackers,

during the past week, our work has primarily been concentrated on minor bugfixes and enhancements in different parts of Rally. These include:

  • Aggregated tables for atomic action runtimes are now available not only in the CLI, but also in the HTML output (generated by the $ rally task plot2html ... command). Let us remind that atomic actions are basic procedures that benchmark scenarios consist of (such as booting a server in Nova or deleting a user in Keystone).
  • The docs section of Rally has been extended with the so-called "User stories", which describe use cases of Rally in the real world, some results obtained by Rally as well as their analysis. The first user story is about token creation under different load in Keystone.
  • Bugfixes include:


Current work includes such interesting novelties as the new "stress" scenario runner that generated a growing load until it breaks the cloud, the new Image context that boots VMs before benchmarking and many others.


We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.

Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z


Stay tuned.


Regards,
The Rally team


June 2, 2014

Hello stackers,

one of the most pleasant news for the past week is the appearance of a brand new Rally ReadTheDocs page, where all the docs from the Rally repository are available in a human-readable format.

As for recent patches merged to Rally, so let's pick up the most interesting ones:


Among new interesting patches being on their way to Rally, let's mention a filesystem benchmark (that will be launched inside created servers), new context classes for creating images and Neutron networks before launching becnhmarks, and many others.


We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.

Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z


Stay tuned.


Regards,
The Rally team


Archives