https://wiki.openstack.org/w/index.php?title=Rally/UpdatesFebruary2014&feed=atom&action=historyRally/UpdatesFebruary2014 - Revision history2024-03-29T09:07:09ZRevision history for this page on the wikiMediaWiki 1.28.2https://wiki.openstack.org/w/index.php?title=Rally/UpdatesFebruary2014&diff=48565&oldid=prevMikhail Dubov at 06:58, 14 April 20142014-04-14T06:58:38Z<p></p>
<table class="diff diff-contentalign-left" data-mw="interface">
<col class='diff-marker' />
<col class='diff-content' />
<col class='diff-marker' />
<col class='diff-content' />
<tr style='vertical-align: top;' lang='en'>
<td colspan='2' style="background-color: white; color:black; text-align: center;">← Older revision</td>
<td colspan='2' style="background-color: white; color:black; text-align: center;">Revision as of 06:58, 14 April 2014</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l108" >Line 108:</td>
<td colspan="2" class="diff-lineno">Line 108:</td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>==== February <del class="diffchange diffchange-inline">03</del>, 2014 ====</div></td><td class='diff-marker'>+</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>==== February <ins class="diffchange diffchange-inline">3</ins>, 2014 ====</div></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td></tr>
<tr><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>Hello stackers,</div></td><td class='diff-marker'> </td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"><div>Hello stackers,</div></td></tr>
</table>Mikhail Dubovhttps://wiki.openstack.org/w/index.php?title=Rally/UpdatesFebruary2014&diff=48564&oldid=prevMikhail Dubov: Created page with "== Weekly updates - February 2014 == ==== February 24, 2014 ==== Hello stackers, this week, several important contributions have been made to Rally, considering both the ov..."2014-04-14T06:58:05Z<p>Created page with "== Weekly updates - February 2014 == ==== February 24, 2014 ==== Hello stackers, this week, several important contributions have been made to Rally, considering both the ov..."</p>
<p><b>New page</b></p><div>== Weekly updates - February 2014 ==<br />
<br />
==== February 24, 2014 ====<br />
<br />
Hello stackers,<br />
<br />
this week, several important contributions have been made to Rally, considering both the overall system stability and the improvements of the user interface. To name a few:<br />
* '''''Vast refactoring of the ScenarioRunner class''''' has enabled to stop sharing OpenStack clients objects between processes in the core of the system, which occasionally caused bugs in Rally (https://review.openstack.org/#/c/74769/);<br />
* Another important refactoring step resulted in the '''''replacement of OpenStack endpoint dictionaries with special objects throughout the system''''', which has made the code more reliable and extendable (https://review.openstack.org/#/c/74425/);<br />
* Perhaps the prettiest patch of the week was '''''the introduction of a benchmark result visualization tool''''', implemented with the ''nvd3'' plugin to ''d3.js'' (so that the actual charts are drawn to a ''html'' file). The graphs look really nice and will be of great use for those who want to share their benchmarking results (https://review.openstack.org/#/c/72970/);<br />
* Several '''''nice improvements in the CLI''''' include the showing of 90- and 95- percentile results in the benchmark summary (https://review.openstack.org/#/c/73522/) and a new '''show''' command which allows the user to get the information on ''images/flavors/networks/etc.'' available in the current deployment in a very quick way (https://review.openstack.org/#/c/75699/).<br />
<br />
<br />
The ongoing work includes:<br />
* An extention of the '''use''' command which will be applicable soon not only to deployments but also to tasks (https://review.openstack.org/#/c/75936/);<br />
* Further refactoring of the core benchmark engine, including the work around input configuration parameters validation (for a detailed description of what's going to be done, see [https://docs.google.com/a/mirantis.com/document/d/1LYUAHkZQD8W7dtlj2I3PDA6x67TiD3AMnSWG6ljsups/edit#heading=h.ae5lk415py0q| this special document]);<br />
* After finishing some major refactoring procedues, we have also resumed the work around passing pre-created user endpoints to the DummyEngine (https://review.openstack.org/#/c/67720/) and generating the ''"stress"'' load on the cloud.<br />
<br />
<br />
We encourage you to take a look at new patches in Rally pending for review and to help us making Rally better.<br />
<br />
Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally<br /><br />
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=icehouse&metric=commits&project_type=all&module=rally <br/><br />
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z<br />
<br />
<br />
Stay tuned. <br />
<br />
<br />
Regards,<br /><br />
The Rally team<br />
<br />
<br />
<br />
==== February 17, 2014 ====<br />
<br />
Hello stackers,<br />
<br />
the first thing we would like to mention today is an extensive piece of work that has been done recently on our Wiki. We have updated both the [[Rally|main page]] and the basic tutorials: [[Rally/installation|how to install Rally]] and [[Rally/HowTo|how to use it]]. These tutorials have been simplified quite a lot and have been updated with new features that have been merged in Rally recently.<br />
<br />
As for the actual updates in the Rally code, the main ones are as follows:<br />
* '''''The refactoring of the ScenarioRunner class''''' has been successfully continued by reimplementing different benchmark execution strategies (''continuous''/''periodic'' executions), which resided previously inside the original class, via subclassing. The new '''''ContinuousScenarioRunner''''' and '''''PeriodicScenarioRunner''''' classes enable us to make the code much more readable (no more complicated ''if...else'' logic to choose the appropriate execution strategy is present in the code) and extendable, so that it is now very easy to add your own ScenarioRunner (https://review.openstack.org/#/c/70771/);<br />
* We've successfully started the work on '''''Tempest & Rally integration''''' in order for the latter to be able to perform OpenStack deployment verification procedures as well as to possibly add new Tempest-based benchmark scenarios (https://review.openstack.org/#/c/70131/);<br />
* On the way to adding a full support for '''''benchmarking with predefined OpenStack users''''' (instead of using the generated ones), we've refactored the '''''Endpoint''''' entity, making it able to distinguish between administrator/user permisisons. Besides, each deployment gets now stored in the database with a list of endpoints instead of only one endpoint, since we are going to enable the DummyEngine to take several endpoints as its input (https://review.openstack.org/#/c/67154/);<br />
* One of the future features of Rally will be '''''Heat-based benchmark scenarios''''' which will make it possible to test the VMs performance. This week, we have started contributing to this as well (https://review.openstack.org/#/c/72749/);<br />
* Among many other simplier refactoring patches finished this week, we'd like to mention the one that '''''improves the CLI code''''' by structuring it to submodules (https://review.openstack.org/#/c/73059/).<br />
<br />
<br />
The ongoing work includes:<br />
* Further changes needed for the DummyEngine in order to be able to '''''accept predefined user endpoints''''' (instead of a single admin endpoint) and for the ScenarioRunner to use them in benchmarks (https://review.openstack.org/#/c/67643/, https://review.openstack.org/#/c/67710/, https://review.openstack.org/#/c/67720/);<br />
* New '''''benchmark result visualization tool''''' based on ''nvd3'' plugin to ''d3.js'' (so that the actual charts are drawn to a ''html'' file). This also includes a new CLI command '''''rally task plot2html <task_uuid>''''' (https://review.openstack.org/#/c/72970/);<br />
* Adding '''''90- and 95-percentile results to the CLI output''''' for benchmark runtimes, i.e. printing the runtime ranges that encompass 90% and 95% of benchmarks respectively (https://review.openstack.org/#/c/73522/);<br />
* Continuing the work on '''''Rally & Tempest integration''''', new '''''deployment engine types''''' etc.<br />
<br />
<br />
We encourage you to take a look at new patches in Rally pending for review and to help us making Rally better.<br />
<br />
Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally<br /><br />
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=icehouse&metric=commits&project_type=all&module=rally <br/><br />
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z<br />
<br />
<br />
Stay tuned. <br />
<br />
<br />
Regards,<br /><br />
The Rally team<br />
<br />
<br />
<br />
==== February 10, 2014 ====<br />
<br />
Hello stackers,<br />
<br />
the past week has been extremely successful for us regarding the overall Rally code improvement, bugfixing, as well as new features implementation. Rally is actually about to become a totally ''easy-to-understand'' and ''easy-to-use'' piece of software which can be exploited by everyone interested in it.<br />
<br />
The most important contributions to Rally made during the past week are as follows:<br />
* The code refactoring stuff has been quite involved:<br />
:* We have issued '''''a drastic rearrangement of the ScenarioRunner class''''' (that is responsible for the actual benchmark method calls using a particular benchmarking strategy) by moving out some code from this class to new context classes. This change also enabled Rally to process all errors occuring on the cloud during benchmarking/cleanup correctly (https://review.openstack.org/#/c/69886/);<br />
:* Another significant contribution is the '''''sshutils module refactoring''''', which involves the API improvement as well as the new ability to process the stdin data (https://review.openstack.org/#/c/68063/);<br />
:* Finally, a very nice work has been done on the '''''benchmark scenarios refactoring''''' by moving the hardcoded timeout and cloud poll interval values to rally.conf (https://review.openstack.org/#/c/71272/).<br />
* Very important '''''bugfixes''''' addressing the improper '''''implementation of OpenStack resource deletion''''' (https://review.openstack.org/#/c/66856/) and '''''benchmark timeout handling''''' (https://review.openstack.org/#/c/72103/) have been merged this week as well;<br />
* Our set of available benchmark scenarios has been expanded with '''''benchmark scenarios for Glance''''': they include a scenario for ''adding and deleting an image'' and a scenario for ''booting several instances from a previously added image'' (https://review.openstack.org/#/c/60469/).<br />
<br />
<br />
<br />
The current work encompasses the following directions:<br />
* Further '''''scenario runners refactoring''''': we are now reimplementing the different benchmark execution strategies (''continuous''/''periodic'' executions) via subclasses of the base '''''ScenarioRunner''''' class thus making the code much more readable and extendable (https://review.openstack.org/#/c/70771/);<br />
* Reimplementing the patches for '''''DummyEngine refactoring''''' (making it able to work with a predefined set of users instead of the generated ones) based on the updated scenario runners (https://review.openstack.org/#/c/67154/, https://review.openstack.org/#/c/67643/, https://review.openstack.org/#/c/67710/, https://review.openstack.org/#/c/67720/);<br />
* '''''Rally & Tempest integration''''' is a new ambitious piece of work we are conducting now. Tempest is going to be used inside Rally as a base for the cloud verification functionality in Rally as well as for new benchmark scenarios that use Tempest (https://review.openstack.org/#/c/70131/).<br />
* We are putting now much effort in finishing the work on '''''new deploy engines''''': the '''''MultihostEngine''''' (https://review.openstack.org/#/c/57240/), the '''''LxcEngine''''' (https://review.openstack.org/#/c/56222/) and the '''''FuelEngine''''' (https://review.openstack.org/#/c/61963/).<br />
<br />
<br />
<br />
We encourage you to take a look at new patches in Rally pending for review and to help us making Rally better.<br />
<br />
Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally<br /><br />
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=all&metric=commits&project_type=All&module=rally <br/><br />
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z<br />
<br />
<br />
Stay tuned. <br />
<br />
<br />
Regards,<br /><br />
The Rally team<br />
<br />
<br />
<br />
==== February 03, 2014 ====<br />
<br />
Hello stackers,<br />
<br />
our efforts during the past week were heavily focused on code refactoring and bugfixing. Among the most significant contrubutions are:<br />
* A fix for certain inconsistencies in the code that checked the availability of resources of OpenStack, e.g. whether a particular resource got deleted or not (https://review.openstack.org/#/c/66856/);<br />
* The work on refactoring the scenario runner to make its code clean (https://review.openstack.org/#/c/69846/).<br />
<br />
<br />
Several novelties have been introduced to Rally:<br />
* After having developed the abstract '''''validators''''' mechanism, we have developed a couple of useful concrete validators as well: the one that checks that the image indicated in the config for, say, the ''NovaServers.boot_and_delete_server'' benchmark scenario really exists and can be used (https://review.openstack.org/#/c/68055/) and another validator that does the same for flavors (https://review.openstack.org/#/c/70082/). Both validators have been attached to benchmark scenarios where they are of great use.<br />
* We've implemented the mechanism of measuring the time taken by '''''atomic actions''''' in our benchmark scneario (https://review.openstack.org/#/c/69828/): e.g. now Rally outputs not only the information on how long it took the cloud to boot and delete a single server (in the ''NovaServers.boot_and_delete_server'' scenario), but also how much time it took to boot the server and to delete it.<br />
<br />
<br />
This week there is still a huge amount of work to be done around '''''refactoring the very fundamental code in Rally'''''. Among other things, we now rewrite the ''ScenarioRunner'' class which is the tool for launching benchmark scenarios (https://review.openstack.org/#/c/69886/) so that its functionality gets split into several context classes (responsible for temporary users management and resource cleanup after benchmarking), and also implementing different scenario launching strategies via inheritance (https://review.openstack.org/#/c/70771/).<br />
<br />
We continue implementing new features in Rally as well. One example is the ongoing work on '''''atomic actions runtime measurement''''': it is about to be supported by the CLI which will now display this detailed runtime information in a user-friendly way (https://review.openstack.org/#/c/70362/).<br />
<br />
<br />
We encourage you to take a look at new patches in Rally pending for review and to help us making Rally better.<br />
<br />
Source code for Rally is hosted at GitHub: https://github.com/stackforge/rally<br /><br />
You can track the overall progress in Rally via Stackalytics: http://stackalytics.com/?release=all&metric=commits&project_type=All&module=rally <br/><br />
Open reviews for Rally: https://review.openstack.org/#/q/status:open+rally,n,z<br />
<br />
<br />
Stay tuned. <br />
<br />
<br />
Regards,<br /><br />
The Rally team</div>Mikhail Dubov