Nova/CoverageExtension

Note: The coverage extension was removed from Nova during the Icehouse release. This was because of issues with running coverage with eventlet causing the results to not be very useful.
= Nova Coverage Extension =

The nova coverage extension adds support for enabling Ned Batchelder's coverage module for nova from an external api call. This module currently already has a nose plugin which is what is used for the periodic nova-coverage runs performed by Jenkins. The intended goal is to enable an external integrated test suite, such as tempest, to generate coverage reports for it use of nova (see: https://blueprints.launchpad.net/nova/+spec/coverage-extension and https://blueprints.launchpad.net/tempest/+spec/tempest-coverage-reporting). This will enable tempest to easily identify which parts of nova are being exercised by it's tests and use that information to add more testing to ensure that more of nova is covered.

Expected usage scenario
Currently, the coverage extension will only work with if all the nova services are running on the same machine. This is due to a limitation in eventlet backdoor, which is used for intra-service control of the coverage module, binding to 127.0.0.1 by default. This works fine for devstack with tempest, as long as the system's hostname resolves to 127.0.0.1. To use the eventlet backdoor you will still need to set the backdoor port in nova.conf. You can do this with 1 conf file by using 0 for the port. This will randomly assign a port to each service.

Starting coverage
Starting coverage on all the openstack services is performed with the api action 'start'. Start takes one optional field 'combine', which enables combined reports. If combine  is set to True in the api request then when the coverage report is generate it will output a single file that contains the combined reports from each of the nova services. The alternative being a report file being generated for each service.

Generating reports
Generating reports after your tests have completed is accomplished by using the api action report. The api request has the mandatory field 'file' which is a filename that will be used for generating the reports. In the case where combine was set in start coverage the output file will have this filename, however if combine was not set then '. service_name' will be automatically appended to the filename for each service.

There is also an optional field for the report action, 'xml', which is used to specify the output type for the generated report. The coverage module allows for 3 types of      report output: human readable text, xml, and html. The coverage extension supports outputing either text or xml reports. If the 'xml' field is set to True then the generated   report (or reports) will be in the xml format, otherwise the output will be in the human readable text.

Stopping coverage
Stop coverage is only used if you you do not want to use the default report generation that is provided by the report action. Sending the stop action for coverage will stop    coverage report and save the generated data files to the temporary directory. This enables using coverage.py directly to generate your own reports in a different format. Just  as with report coverage must be running for this step to succeed.

If you are using the coverage extension api to generate the reports then there is no need to manually send the 'stop' action the 'report' action automatically runs this step.

Current Services that support using backdoor for coverage:

 * Compute
 * API
 * Network
 * Conductor
 * Cert
 * Console
 * Console Auth
 * Scheduler