Jump to: navigation, search

Difference between revisions of "Rally/HowTo"

(Step 2. Benchmarking)
(Step 2. Benchmarking)
Line 89: Line 89:
  
 
For more details run:
 
For more details run:
rally -vd task detailed 392c803b-37fd-4915-9732-3523f4252e9b</pre>
+
rally -vd task detailed 392c803b-37fd-4915-9732-3523f4252e9b
 
</pre>
 
</pre>
 
</small>
 
</small>

Revision as of 06:21, 20 March 2014

Usage demo

NOTE: Throughout this demo, we assume that you have a configured Rally installation and an already existing OpenStack deployment has keystone available at <KEYSTONE_AUTH_URL>.


Step 1. Deployment initialization

First, you have to provide Rally with an Openstack deployment it is going to benchmark. This is done through deployment configuration files. The actual deployment can be either created by Rally (see /doc/samples for configuration examples) or, as in our example, an already existing one. The configuration file (let's call it dummy_deployment.json) should contain the deployment strategy (in our case, the deployment will be performed by the so called "DummyEngine", since the deployment is ready to use) and some specific parameters (for the DummyEngine, an endpoint with administrator permissions):

{
    "name": "DummyEngine",
    "endpoint": {
        "auth_url": <KEYSTONE_AUTH_URL>,
        "username": <ADMIN_USER_NAME>,
        "password": <ADMIN_PASSWORD>,
        "tenant_name": <ADMIN_TENANT>
    }
}


To register this deployment in Rally, use the deployment create command:

$ rally deployment create --filename=dummy_deployment.json --name=dummy
+---------------------------+----------------------------+-------+------------------+
|            uuid           |         created_at         |  name |      status      |
+---------------------------+----------------------------+-------+------------------+
|     <Deployment UUID>     | 2014-02-15 22:00:28.270941 | dummy | deploy->finished |
+---------------------------+----------------------------+-------+------------------+
Using deployment : <Deployment UUID>


Note the last line in the output. It says that the just created deployment is now used by Rally; that means that all the benchmarking operations from now on are going to be performed on this deployment. In case you want to switch to another deployment, execute the use deployment command:

$ rally use deployment --deploy-id=<Another deployment UUID>
Using deployment : <Another deployment UUID>


Finally, the deployment check command enables you to verify that your current deployment is healthy and ready to be benchmarked:

$ rally deployment check
+----------+-----------+-----------+
| services |    type   |   status  |
+----------+-----------+-----------+
|   nova   |  compute  | Available |
| cinderv2 |  volumev2 | Available |
|  novav3  | computev3 | Available |
|    s3    |     s3    | Available |
|  glance  |   image   | Available |
|  cinder  |   volume  | Available |
|   ec2    |    ec2    | Available |
| keystone |  identity | Available |
+----------+-----------+-----------+


Step 2. Benchmarking

Now that we have a working and registered deployment, we can start benchmarking it. Again, the sequence of benchmark scenarios to be launched by Rally should be specified in a benchmark task configuration file. Note that there is already a set of nice benchmark tasks examples in doc/samples/tasks/ (assuming that you are in the Rally root directory). The natural thing would be just to try one of these sample benchmark tasks, say, the one that boots and deletes multiple servers (doc/samples/tasks/nova/boot-and-delete.json). To start a benchmark task, run the task start command:

ubuntu@tempeste-test:~$ rally -v task start rally/doc/samples/tasks/nova/boot-and-delete.json 
================================================================================
Task  392c803b-37fd-4915-9732-3523f4252e9b is started
--------------------------------------------------------------------------------
2014-03-20 06:17:39.994 27502 INFO rally.benchmark.engine [-] Task 392c803b-37fd-4915-9732-3523f4252e9b | Starting:  Check cloud.
2014-03-20 06:17:40.123 27502 INFO rally.benchmark.engine [-] Task 392c803b-37fd-4915-9732-3523f4252e9b | Completed: Check cloud.
2014-03-20 06:17:40.123 27502 INFO rally.benchmark.engine [-] Task 392c803b-37fd-4915-9732-3523f4252e9b | Starting:  Task validation.
2014-03-20 06:17:40.133 27502 INFO rally.benchmark.engine [-] Task 392c803b-37fd-4915-9732-3523f4252e9b | Starting:  Task validation of scenarios names.
2014-03-20 06:17:40.137 27502 INFO rally.benchmark.engine [-] Task 392c803b-37fd-4915-9732-3523f4252e9b | Completed: Task validation of scenarios names.
2014-03-20 06:17:40.138 27502 INFO rally.benchmark.engine [-] Task 392c803b-37fd-4915-9732-3523f4252e9b | Starting:  Task validation of syntax.
2014-03-20 06:17:40.140 27502 INFO rally.benchmark.engine [-] Task 392c803b-37fd-4915-9732-3523f4252e9b | Completed: Task validation of syntax.
2014-03-20 06:17:40.140 27502 INFO rally.benchmark.engine [-] Task 392c803b-37fd-4915-9732-3523f4252e9b | Starting:  Task validation of semantic.
2014-03-20 06:17:41.098 27502 ERROR glanceclient.common.http [-] Request returned failure status.

================================================================================
Task 392c803b-37fd-4915-9732-3523f4252e9b is failed.
--------------------------------------------------------------------------------
<class 'rally.exceptions.InvalidBenchmarkConfig'>
Task config is invalid.
	Benchmark NovaServers.boot_and_delete_server has wrong configuration of args at position 0: {'image_id': '73257560-c59b-4275-a1ec-ab140e5b9979', 'flavor_id': 1}
	Reason: Image with id '73257560-c59b-4275-a1ec-ab140e5b9979' not found

For more details run:
rally -vd task detailed 392c803b-37fd-4915-9732-3523f4252e9b

This attempt, however, will most likely fail because of an input arguments validation error (due to a non-existing image id). The thing is that the benchmark scenario that boots a server needs to do that using a concrete image available in the OpenStack deployment; these images have different ids. That's why you should first make a copy of the sample benchmark task:

cp doc/samples/tasks/nova/boot-and-delete.json my-task.json


and then edit it with the resource uuids from your OpenStack installation:

{
  "NovaServers.boot_and_delete_server": [
    {"args": {"flavor_id": <NOVA_FLAVOR_ID>, "image_id": <GLANCE_IMAGE_UUID>},
     "config": {"times": 2, "active_users": 1}},
    {"args": {"flavor_id": <NOVA_FLAVOR_ID>, "image_id": <GLANCE_IMAGE_UUID>},
     "config": {"times": 4, "active_users": 2}}
  ]
}


To obtain proper image_id and flavor_id, you can use the subcommand show of rally.

let's get a proper image uuid:

$ rally show images
+--------------------------------------+-----------------------+-----------+
|                 UUID                 |          Name         |  Size (B) |
+--------------------------------------+-----------------------+-----------+
| <UUID_THAT_YOU_NEED>                 |  CentOS 6.5 (x86_64)  | 344457216 |
| 2b8d119e-9461-48fc-885b-1477abe2edc5 | CirrOS 0.3.1 (x86_64) |  13147648 |
+--------------------------------------+-----------------------+-----------+


and a proper flavor id:

$ rally show flavors
+---------------------+-----------+-------+----------+-----------+-----------+
| ID                  |    Name   | vCPUs | RAM (MB) | Swap (MB) | Disk (GB) |
+---------------------+-----------+-------+----------+-----------+-----------+
| <ID_THAT_YOU_NEED>  |  m1.tiny  |   1   |   512    |           |     1     |
| 2                   |  m1.small |   1   |   2048   |           |     20    |
| 3                   | m1.medium |   2   |   4096   |           |     40    |
| 4                   |  m1.large |   4   |   8192   |           |     80    |
| 5                   | m1.xlarge |   8   |  16384   |           |    160    |
+---------------------+-----------+-------+----------+-----------+-----------+


After you've edited the my-task.json file, you can run this benchmark task again. This time, let's also use the --verbose parameter that will allow us to retrieve more logging from Rally while it performs benchmarking:

$ rally --verbose task start --task=my-task.json


Using another terminal (or ssh connection to the VM with Rally), you can watch the current task status by the task list command:

$ rally task list 
+------------------------------+----------------------------+-------------------------+--------+
|             uuid             |         created_at         |          status         | failed |
+------------------------------+----------------------------+-------------------------+--------+
|         <Task UUID>          | 2013-09-16 05:28:57.241456 | test_tool->benchmarking | False  |
+------------------------------+----------------------------+-------------------------+--------+


Once the benchmark task has been finished, you can get the detailed information about the results by the task detailed command. Note that Rally not only prints out the aggregated information (min/max/avg running time for each task as well as about the fraction of successfully finished tasks), but also presents the load created by separate atomic actions inside benchmark scenarios (in the case of the NovaServers.boot_and_delete_server scenario, this is the time taken by booting and deleting servers separately).

$ rally task detailed <Task UUID>

================================================================================
Task <Task UUID> is finished. Failed: False
--------------------------------------------------------------------------------

test scenario NovaServers.boot_and_delete_server
args position 0
args values:
{u'args': {u'flavor_id': <Flavor UUID>,
           u'image_id': u'<Image UUID>'},
 u'config': {u'active_users': 1, u'times': 2}}
+--------------------+---------------+---------------+---------------+
|       action       |   max (sec)   |   avg (sec)   |   min (sec)   |
+--------------------+---------------+---------------+---------------+
|  nova.boot_server  | 9.22798299789 | 8.90022659302 | 8.57247018814 |
| nova.delete_server | 4.24928498268 | 3.26377093792 | 2.27825689316 |
+--------------------+---------------+---------------+---------------+

+---------------+---------------+---------------+---------------+-------------+
|   max (sec)   |   avg (sec)   |   min (sec)   | success/total | total times |
+---------------+---------------+---------------+---------------+-------------+
| 13.4775559902 | 12.1641695499 | 10.8507831097 |      1.0      |      2      |
+---------------+---------------+---------------+---------------+-------------+
--------------------------------------------------------------------------------

test scenario NovaServers.boot_and_delete_server
args position 1
args values:
{u'args': {u'flavor_id': <Flavor UUID>,
           u'image_id': u'<Image UUID>'},
 u'config': {u'active_users': 2, u'times': 4}}
+--------------------+---------------+---------------+---------------+
|       action       |   max (sec)   |   avg (sec)   |   min (sec)   |
+--------------------+---------------+---------------+---------------+
|  nova.boot_server  | 9.64801907539 | 8.30236756802 | 6.95671606064 |
| nova.delete_server | 4.46917510033 | 4.45528066158 | 4.44138622284 |
+--------------------+---------------+---------------+---------------+

+---------------+---------------+--------------+---------------+-------------+
|   max (sec)   |   avg (sec)   |  min (sec)   | success/total | total times |
+---------------+---------------+--------------+---------------+-------------+
| 14.0895900726 | 12.7578320503 | 11.426074028 |      0.5      |      4      |
+---------------+---------------+--------------+---------------+-------------+

Available Rally facilities

To be able to run complex benchmark scenarios on somewhat more sophisticated OpenStack deployment types, you should familiarize yourself with more deploy engines, server providers and benchmark scenarios available in Rally.

List of available Deploy engines (including their description and usage examples): Deploy engines

List of available Server providers (including their description and usage examples): Server providers

List of available Benchmark scenarios (including their description and usage examples): Benchmark scenarios