Jump to: navigation, search

Rally

Revision as of 21:13, 1 February 2014 by Boris-42 (talk | contribs) (Architecture)

What is Rally?

If you are here, you are probably familiar with OpenStack and know that it's really huge, very huge ecosystem of co-working services and when something went wrong, or work slow or doesn't scale it's really hard to answer on questions "why", "what" and "where"? Another reason why you could be here is that you would like to build OpenStack CI/CD system that will allow you to improve SLA, performance and stability of OpenStack continuously.

OpenStack QA team mostly works on CI/CD that ensures that new patches don't brake specific single node installation of OpenStack. On the other hand it's clear that such CI/CD is only requirement and it is not sufficient condition.. (e.g. if cloud works well on single node installation it doesn't mean that it will work good as well on 1k servers installation under high load)/.. Rally aims to fix this and help us to get answer on question "How OpenStack works at scale". To make it possible we are going to automate and unify all steps that are required for benchmarking OpenStack at scale: multi node OS deployment, verification, benchmarking & profiling.

Rally-Actions.png

Use Cases

Before diving deep in Rally architecture let's take a look at 3 major high level Rally Use Cases:

Rally-UseCases.png


Typical cases where Rally aims to help are:

  1. Automate measuring & profiling focused on how new code changes affect OS performance.
  2. Using Rally profiler to detect scaling & performance issues.
  3. Investigate how different deployments affect OS performance:
    • Find the set of good OpenStack deployment architectures,
    • Create deployment specifications for different loads (amount of controllers, swift nodes, etc.).
  4. Automate search for hardware best suited for particular OpenStack cloud.
  5. Automate production cloud specification generation:
    • Determine terminal loads for basic cloud operations: VM start & stop, Block Device create/destroy & various OpenStack API methods.
    • Check performance of basic cloud operations in case of different loads.

Architecture

Rally consists of 4 main components:

  1. Server Provider - which provides servers (virtual servers), with ssh access, in one L3 network.
  2. Deploy Engine - that deploy OpenStack cloud on servers that are presented by Server Provider
  3. Verification - run tempest (or specific set of tests) against cloud, collect results & present in human readable form.
  4. Benchmark engine - allows to write parameterized benchmark scenarios & run them against cloud.

Rally in action

How amqp_rpc_single_reply_queue affects performance

To show Rally's capabilities and potential we used NovaServers.boot_and_destroy scenario to see how amqp_rpc_single_reply_queue option affects VM bootup time. Some time ago it was shown that cloud performance can be boosted by setting it on so naturally we decided to check this result. To make this test we issued requests for booting up and deleting VMs for different number of concurrent users ranging from one to 30 with and without this option set. For each group of users a total number of 200 requests was issued. Averaged time per request is shown below:

amqp_rpc_single_replya_queue

So apparently this option affects cloud performance, but not in the way it was thought before.


How To

  1. Rally installation
  2. How to use Rally
  3. Available Deploy engines
  4. Available Server providers
  5. Available Benchmark scenarios
  6. Extend Rally functionality
  7. Rally Road Map


Weekly updates

Each week we write up on a special weekly updates page what sort of things have been accomplished in Rally during the past week and what are our plans for the next one.


Join Rally team

Discussions & RoadMap

https://etherpad.openstack.org/p/Rally_Main

Open and assigned tasks

https://trello.com/b/DoD8aeZy/rally

To get account ping Boris in IRC (boris-42) or email me (boris(at)pavlovic.me)

IRC chat

server: freenode.net

chanel: #openstack-rally

Weekly Meetings

The Rally project team holds weekly meetings on Tuesdays at 1700 UTC in IRC, at the #openstack-meeting channel.

Source

https://github.com/stackforge/rally

Project space

http://launchpad.net/rally

Blueprints

active:    http://blueprints.launchpad.net/rally

v1 base: https://blueprints.launchpad.net/rally/+spec/init

Bugs

https://bugs.launchpad.net/rally

Pending Code Reviews

https://review.openstack.org/#/q/status:open+rally,n,z