NovaTestingHudson

Etherpad for Summit: http://etherpad.openstack.org/NovaHudsonTesting

• Launchpad Entry: NovaSpec:testing-hudson-integration
• Created: 2010-10-29
• Contributors: JayPipes

Summary

Provide developers with better testing infrastructure for Nova using Hudson and additional platforms.

Release Note

Additional testing infrastructure and integration with the Hudson build platform was implemented.

Rationale

Besides unit tests, many of which use "faked" services or "stubbed out" components, there is little smoke (end-to-end), functional, or integration testing done on Nova via the automated build process. Increased testing leads to more stable and predictable software.

User Stories

Vish has a branch that he believes to be ready for merging, but the branch makes significant changes to a part of Nova. Vish wants to have a set of testing clusters pull his branch and run a series of tests before he proposes the branch for merging.

Implementation

The following pieces need to be implemented:

• Establish test clusters for real testing/stage environments
• Create a way for a developer to have testing cluster test arbitrary code branches

Real Test Environments

Currently, Hudson runs a simple job that executes Nova's unit tests on a single machine, with most drivers and adapters "faked". This is fine for coarse-grained validation of new code, but is not considered to be exhaustive testing of the code base, nor does it test critical pieces of the virtualisation and messaging layers.

We need to create a number of clusters that should have both the unit tests and a series of smoke tests run against them for each commit to Nova's trunk. These test clusters should be set up with Nova's most common supported platforms.

Proposed Platforms for Testing

Here are the proposed specifications for the test clusters to build:

Allowing Developers to Test Arbitrary Branches in a Test Environment

Infrastructure should be built to allow developers to have Hudson run tests in the test clusters against an arbitrary branch. Monty Taylor solved this problem in Drizzle using the innovative Parameterized Build Farm. It would be great to do the same for Nova.