Difference between revisions of "Rally"

Revision as of 22:55, 31 January 2014

Why Rally?

If you are here, you are probably familiar with OpenStack and know that it's really huge, very huge ecosystem of co-working services and when something went wrong, or work slow or doesn't scale it's really hard to answer on questions "why", "what" and "where"? Another reason why you could be here is that you would like to build OpenStack CI/CD system that will allow you to improve SLA, performance and stability of OpenStack continuously.

Currently OpenStack community is to busy around adding new features and working on CI/CD that ensures that new patches don't brake specific single node installation of OpenStack. Rally should help to answer on question "How OpenStack works at scale?". So it combines all existing tools together to make it possible to Deploy/Verify/Benchmark OpenStack from your sources in "1 click":

Use Cases

Before diving deep in Rally arch let's take a look at 3 major high level Rally Use Cases:

Use Cases

1. Investigate how different deployments affect OS performance:
   • Find the set of good OpenStack deployment architectures,
   • Create deployment specifications for different loads (amount of controllers, swift nodes, etc.).
2. Automate search for hardware best suited for particular OpenStack cloud.
3. Automate production cloud specification generation:
   • Determine terminal loads for basic cloud operations: VM start & stop, Block Device create/destroy & various OpenStack API methods.
   • Check performance of basic cloud operations in case of different loads.
4. Automate measuring & profiling focused on how new code changes affect OS performance.
5. Using Rally profiler to detect scaling & performance issues.

Architecture

Rally consists of 4 main components:

1. Deploy Engine, which is responsible for processing and deploying VM images (using DevStack or FUEL according to user’s preferences). The engine is capable of:
   • deploying OS on already existing VMs,
   • starting VMs from a VM image with pre-installed OS and OpenStack,
   • deploying multiple VMs each of which has running OpenStack compute node based on a VM image.
2. Server Provider, which provides servers (virtual servers) to deploy OpenStack.
3. Benchmarking Tool, which does benchmarking in several stages:
   • runs Tempest tests, reduced to 5-minute length (to save the usually expensive computing time),
   • runs a set of benchmark scenarios (using the Rally testing framework),
   • collects all test' results and processes them with Zipkin tracer,
   • puts together a benchmarking report and stores it on the machine Rally was lauched on.
4. Orchestrator, which is the central component of the system. It uses the Deploy Engine to run control and compute nodes and to launch an OpenStack distribution and, after that, calls the Benchmarking Tool to start the benchmarking process.

To dive deeper into Rally architecture, see Rally architecture for developers.

Rally in action

How amqp_rpc_single_reply_queue affects performance

To show Rally's capabilities and potential we used NovaServers.boot_and_destroy scenario to see how amqp_rpc_single_reply_queue option affects VM bootup time. Some time ago it was shown that cloud performance can be boosted by setting it on so naturally we decided to check this result. To make this test we issued requests for booting up and deleting VMs for different number of concurrent users ranging from one to 30 with and without this option set. For each group of users a total number of 200 requests was issued. Averaged time per request is shown below:

So apparently this option affects cloud performance, but not in the way it was thought before.

How To

Weekly updates

Each week we write up on a special weekly updates page what sort of things have been accomplished in Rally during the past week and what are our plans for the next one.

Join Rally team

Discussions & RoadMap

https://etherpad.openstack.org/p/Rally_Main

Open and assigned tasks

https://trello.com/b/DoD8aeZy/rally

To get account ping Boris in IRC (boris-42) or email me (boris(at)pavlovic.me)

IRC chat

server: freenode.net

chanel: #openstack-rally

Weekly Meetings

The Rally project team holds weekly meetings on Tuesdays at 1700 UTC in IRC, at the #openstack-meeting channel.

Source

https://github.com/stackforge/rally

Project space

http://launchpad.net/rally

Blueprints

active:    http://blueprints.launchpad.net/rally

v1 base: https://blueprints.launchpad.net/rally/+spec/init

Bugs

https://bugs.launchpad.net/rally

Pending Code Reviews

https://review.openstack.org/#/q/status:open+rally,n,z