Jump to: navigation, search

Rally/Updates

< Rally
Revision as of 10:39, 27 October 2014 by Mikhail Dubov (talk | contribs)

Weekly updates

October 27, 2014

Hello stackers,

much time has passed since our last update and we are happy to announce that we are about to make our first official Rally release! Our active recent contribution to Rally has enabled us to make a significant progress. Here are the highlights of the novelties in Rally:

  • We have completely redesigned the auto-generated benchmark report page so that it looks now even nicer than before and is much more easy to navigate. Besides, further improvements of this HTML report page are on their way to being merged very soon.
  • Rally now has an extended support of plugins: in addition to writing custom scenarios, the plugin mechanism now enables to extend Rally with new context classes/scenario runners without actually contributing to the Rally master branch.
  • The work on extending the support of OpenStack projects in Rally has been conducted for Heat and Sahara.
  • 'CLI improvements: the command-line interface gets more and more user-friendly over time: recently, it has begun to support detailed informations about correct commands usage in case of a failure. The "rally info" command has also been improved so that it now supports misspellings handling. Finally, there has been some work on bash completion, which hasn't been completely finished yet.
  • Test code improvements: we have greatly proceeded in out continuous work on unit/functional test coverage improvement. We also have moved all the tests into a special tests/ directory so that the test code is now organized in a more neat way. We also have removed the ./run_tests.sh script for the sake of using the tox command to launch the test suite.


In the nearest future, several interesting refactoring patches are going to come to Rally. To be more specific, there will be a vast change of the benchmark engine and cleanup mechanisms.

We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better!

Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=kilo&metric=commits&project_type=all&module=rally
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z


Stay tuned.


Regards,
The Rally team


September 26, 2014

Hello stackers,

here is a brief overview of what has happened in Rally recently:

  • We have refactored the code responsible for atomic actions processing in Rally benchmarks. Let's remind you that each benchmark scenario in Rally consists of a series of atomic actions, whereas the running time of each atomic action is measured in the same way as that of the whole scenario. After refactoring, we now ensure that Rally doesn't skip atomic actions that failed and distinguishes different runs of two atomic actions with the same name (see an example of how a typical results table could look before and looks after refactoring).
  • Another direction of refactoring was the SLA code; it has been modified so that now SLA results are stored in the DB along with other benchmarking data.
  • Check out a nice user story about VMs boot performance in Nova with Rally. It shows how Rally can be used in practice to catch reals bugs and improve OpenStack performance.
  • We work hard on achieving 100% test coverage in Rally: last week, additional unit tests for contexts and scenarios have been merged.


Our current priorities are further refactoring steps, including those in critical parts of Rally (e.g. temporary user creation and cloud cleanup code); we also strive towards making Rally bug-free and continuously issue different bugfixing patches.

We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.

Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z


Stay tuned.


Regards,
The Rally team


September 15, 2014

Hello stackers,

there has been much diverse and useful contribution to Rally recently. Let us highlight some interesting updates in our project:

  • Rally is on its way to support of benchmarking OpenStack clouds using ordinary user accounts that already exist. Rally lacked such functionality (it only supported benchmarking either from an admin account or from a bunch of temporarily created users), which posed a problem since some deployments don't allow temporary users creation. There have been two patches that prepare the code for this new functionality. It is going to come very soon - stay tuned.
  • We constantly improve our gate jobs (jobs that run a test suite for every patch pending review in Rally). Recently, we have introduced a nice aggregated results page for that gate job (see an example). It now makes it very easy for developers to navigate through the results of tests agains their patches to Rally.
  • We have added a new "volumes" context which makes it possible to create cinder volumes in the benchmark environment and use them later in the actual benchmark scenarios.
  • Much work has been accomplished on the overall code quality improvement. We have both made several code refactoring patches and introduced several new test suites (say, functional tests for CLI).


Current work centers around introducing benchmarking with pre-created users as mentioned above; much work is currently devoted to new benchmark scenarios as well. Also please note that there will be soon an important update on Rally ReadTheDocs page, which will make it much easier to navigate and get through, especially for newbies.

We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.

Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z


Stay tuned.


Regards,
The Rally team


September 1, 2014

Hello stackers,

over the past week, Rally has been extended with the following features:

  • Rally now offers a pretty simple feature request mechanism. Everyone who is interested in adding new functionality to Rally now can share his ideas in a standartized format (see an example). All you need to do is to write down your feature request in a separate rst-file in the doc/feature_request folder and submit it as a patch to Rally (if you are unsure about how to do this, read our "How to contribute" tutorial).
  • We added a new benchmark scenario for Cinder quotas creation/deletion.
  • Designate support in Rally has been extended so that it is posisble now to set up Designate quotas.


There is pretty much work going on this week, including the introduction of benchmarking with existing users (in addition to those created temporarily, which was the only option before), and also improvements in the plugin mechanism, so that it will be possible now to write plugins for runners and contexts. We do lots of code refactoring and will do even more in the upcoming weeks.

We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.

Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z


Stay tuned.


Regards,
The Rally team


August 25, 2014

Hello stackers,

here is a quick overview of what has been added to Rally recently:

  • Rally now supports Designate, which is a DNaaS service for OpenStack (providing REST API for domain/record management). Several new bechmark scenarios for Designate have been added as well;
  • In the CLI, we have added a new command called rally info. It is a essentially a special search engine embedded into Rally, which, for a given search query, prints documentation for the corresponding benchmark scenario/deploy engine/... as fetched from the source code. Thus you can learn about different Rally entities without leaving the Command Line Interface. For usage samples, see this link;
  • We have extended the SLA output which indicates whether a benchmark has passed some set of predefined success criteria with customizable messages;
  • We have performed a range of nice optimizations:

The main work directions for the next week comprise further improvement in the contexts mechanism (both the algorithm of context creation and new context classes), further optimizations (e.g., multithreaded objects deletion) and also code refactoring (e.g., of that for tracking atomic actions in benchmark scenarios)

We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.

Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z


Stay tuned.


Regards,
The Rally team


August 4, 2014

Hello stackers,

below you will find the most recent updates in Rally:

  • The periodic scenario runner has been refactored and renamed, as of now, to rpc (runs per second). Note that this renaming also affects task configuration files that use that runner type. The runner itself now has been reimplemented with the multiprocessing.Process class (instead of multiprocessing.Pool), which potentially decreases memory usage on large iterations and reduces errors.
  • A nice optimization to the chart generation code ensures that the Rally plots will be rendered fast in browsers even for tasks with a huge number of iterations completed.
  • New benchmark scenarios include those testing the Nova server resize operation and a set of benchmarks for Sahara group node templates.
  • We continue extending our gates with nice features. A new one are the SLA checks that add information about whether benchmark scenarios pass a set of success criteria (see an example)


Many interesting improvements are on their path to being merged to Rally. Among them, let us mention the "rally info" command, which prints descriptions for different entities in Rally to the console, further work on the Rally documentation which gets cleaner and cleaner, and also a set of new context classes, including those for generating images for VM benchmarks, as well as for creating Neutron networks.

We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.

Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z


Stay tuned.


Regards,
The Rally team


July 17, 2014

Hello stackers,

here are some recent updates in Rally:


This week, our work continues with such main novelties as adding support for Keystone API v3, adding context classes for avoiding vm creation if tenant has no network and prepare an image that will have installed the required programs, mechanism to provide rally as a service, code refactoring in context manager to make code more clear.


We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.

Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z


Stay tuned.


Regards,
The Rally team


July 07, 2014

Hello stackers,

here is the update for the last weeks. From all the work we've completed we would like to highlight the following:

  • New benchmark scenarios:


Current work includes such interesting novelties as context semantic validation, periodic runner refactoring, "rally info" command which prints descriptions for different entities in Rally, checking if required services are available before starting the scenario and refactoring of validation system, continue work on "stress" runner.


We encourage you to take a look at new patches in Rally pending for review and to help us make Rally better.

Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=juno&metric=commits&project_type=all&module=rally
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z


Stay tuned.


Regards,
The Rally team


Archives