GSoC2014/Rally/BenchmarksVirtualMachinesOpenStack

Introduction

The ability to benchmark a Virtual Machine is an important activity that more and more developers will need to perform as they host their SaaS applications in a cloud. The aim of this project is to integrate into the Rally project the ability to run easily and in an automated manner various benchmarks for measuring the performance of the deployed Virtual Machines of an OpenStack cloud.

Description

The project can be divided in two parts. The first part has to do with the development of an architecture (some kind of framework) that defines a standard and easy way of porting different benchmarks to Rally, and in the second part, with the use of this framework, port existing popular benchmarks that are used to measure the performance of different aspects of a computer system.

For the first part, a new benchmark context (benchmark_image) is developed that generates an image that has installed all the required programs to run the specified benchmark. The context takes an image, a flavor and some other necessary information from the task configuration file of the benchmark scenario and boots a virtual machine. Then, using the already created users and their keypairs and security groups, it gains access to the virtual machine with SSH and executes the setup script of the specified benchmark. The setup script is a Bash script that installs the benchmark (and its dependencies) in the virtual machine. Finally, the context takes a snapshot of that virtual machine and returns the name of the newly created benchmark-ready image. Now the benchmark scenario uses the image that the context returned, in order to boot the virtual machine. It gains access to the virtual machine with SSH and executes the run script of the specified benchmark. The run script is a Python script that executes the benchmark and returns the results of it in JSON format.

For the second part, because of the architecture that was defined in the first part, there is only need to develop the setup script that installs the benchmark, and the run script that executes the benchmark and returns the result in JSON format. This will be done for every possible benchmark that needs to be ported to Rally in order to be executed in a virtual machine of an OpenStack cloud.

Source Code

All the code that was written during the official GSoC period for the development of this project.

Under Review

• Add the benchmark 'Blogbench' for the Virtual Machines (https://review.openstack.org/#/c/97030/)
• Add the VM scenario 'boot_benchmark_delete' (https://review.openstack.org/#/c/98172/)
• Add the context 'benchmark_image' (https://review.openstack.org/#/c/104564/)

Merged

• Modify install_rally.sh to install under BSDs (https://review.openstack.org/#/c/95341/)
• Add error handling to install_rally.sh (https://review.openstack.org/#/c/98399/)
• Change the default value of scenario_output (https://review.openstack.org/#/c/104180/)
• Change the default value of use_public_urls (https://review.openstack.org/#/c/104924/)
• Modify config semantic validation of benchmark engine (https://review.openstack.org/#/c/112981/)
• Add required_contexts validator (https://review.openstack.org/#/c/111603/)
• Decrease jobs time in gates (https://review.openstack.org/#/c/114839/)
• Fix semantic validation of context images (https://review.openstack.org/#/c/113904/)

Links

• Blogbench, http://www.pureftpd.org/project/blogbench