Tempest API Tests for Neutron
The fundamental interface to the OpenStack services is a set of ReST API's. Other interfaces, such as CLI commands or the Horizon portal, are built on top of these ReST API's. As a consequence, it is of paramount importance to have tools to validate that an OpenStack deployment behaves according to the ReST API's specification.
Tempest is a set of functional integration tests intended to be run against actual OpenStack deployments. It is a functional testing framework that uses the testools Python library as a base. Test cases execute a series of API calls against OpenStack service endpoints (like Neutron or Nova) and validate the responses received from said endpoints.
The purpose of this page is to walk new OpenStack developers through the initial steps of writing Tempest API tests for Neutron.
If you are an absolute beginner: joining the community
Before writing any code for OpenStack or Tempest, you need to take some administrative steps to join the community. Here is a good overview of our workflow. Colin McNamara put together an excellent presentation on this topic. You can watch Colin delivering the presentation here.
Setting up a development system
Devstack is a documented shell script to install and run an OpenStack cloud on your laptop. Most developers don't run DevStack on their laptop's "bare metal". Instead, DevStack is frequently deployed to a VM created using virtualization software such as VirtualBox, VMware Fusion or Virtual Machine Manager. DevStack is ideal for potential users who want to see what the Dashboard looks like from an admin or user perspective, and OpenStack contributors wanting to test against a complete local environment.
Learning the Neutron API
This is the official Neutron ReST API documentation. It consists of a Core API that provide the 3 basic Neutron L2 abstractions: networks, subnets and ports. There are also a series of API extensions that provide abstractions such as routers, floating ip's, load balancers as a service, quotas, etc, that span L3 to L7 in the OSI model.
The goal of the Neutron and Tempest group of developers is to create a comprehensive set of tests that exercise all the documented positive paths and options of the APIs, as well as to provide them with invalid data to ensure they fail in expected and documented ways. Over the course of the OpenStack project Tempest has discovered many fundamental bugs by doing just this.
Getting to know the Tempest test framework
The code of the Tempest tests is stored in the OpenStack Git repo. Of special interest for Neutron / Tempest API test developer are two packages within the repo:
- The Tempest Neutron ReST client. Tempest implements its own ReST clients for the different OpenStack services. As noted in the documentation, tests should always use the Tempest implementation of the OpenStack API, as we want to ensure that bugs aren’t hidden by the official clients. Neutron / Tempest API developers must extend the ReST client in this package to increase the coverage of the Neutron API, as defined in its official documentation. This ReST client is in turn a subclass of the Tempest base client
- The Tempest Neutron API tests. Please take a look in the modules in this package, so you can familiarize yourself in the way the Neutron API is tested.
Deciding what to develop
We are keeping track of the tests that have to be developed for the Icehouse cycle in QA Neutron page. Go to the "API tests gap analysis" section and select the functionality you want to implement. Don't forget that it may entail extending the ReST client for Neutron besides writing the test itself, as explained above. Also, please write your IRC handle next to the functionality selected, so we don't duplicate efforts.
If you get stuck and need help, don't hesitate to ask for help in the followings IRC channels (chat.freenode.net):