Jump to: navigation, search



Now adding missing Tempest tests is going forward aggressively in a Tempest BP(https://blueprints.launchpad.net/tempest/+spec/missing-tempest-tests).

This BP promotes to find missing tests effectively by listing all Nova APIs with tested/untested marks.

Developers can start/stop API recording, list tested APIs, and find missing test by looking for untested APIs.


The main target is test developers. Developers start/stop recording and create a tested API report to find missing tests.

Components to Consider

At first, the main target is Nova. Extend other components after enough mature codes.



Implement a new middleware and apis to create Nova API list, analyze incoming APIs, and manage actions of start/report/reset recording.

ListTestedApis overview.jpg

New API specifications

New APIs are based on "nova coverage" APIs.

Recording tested APIs is started by API action "start" and stopped by action "report".

Action "report" also create a report that shows if each API is tested or not.

The record is reset by action "reset".

Action "report" has a mandatory request field 'file' that is a name of a report file.

  • Start API recording
 POST /v2/{project_id}/os-api-coverage/action
 {"start": {}}
  • Stop API recording and Output API record
 POST /v2/{project_id}/os-api-coverage/action
 {"report": {"file": "xxx"}}
  • Reset API record
 POST /v2/{project_id}/os-api-coverage/action
 {"reset": {}}

Sample Report of Prototype

Tested APIs are marked "Y" at "Tested" column.

Tested API: xxx/yyy
Tested No. Method URL                                              Action
       000 GET    /
       001 POST   /{project_id}/servers/:server_id/consoles
       002 GET    /{project_id}/servers/:server_id/consoles
       003 PUT    /{project_id}/servers/:server_id/consoles/:(id)
       004 DELETE /{project_id}/servers/:server_id/consoles/:(id)
       005 GET    /{project_id}/servers/:server_id/consoles/:(id)
     Y 006 POST   /{project_id}/servers
       007 GET    /{project_id}/servers/detail
     Y 008 GET    /{project_id}/servers
     Y 009 PUT    /{project_id}/servers/:(id)
       010 POST   /{project_id}/servers/:(id)/action               restore
       011 POST   /{project_id}/servers/:(id)/action               suspend
       012 POST   /{project_id}/servers/:(id)/action               migrate
       239 GET /{project_id}/os-snapshots/:(id)

Future Work

  • Add an option to Tempest
  • Analyze request body
  • Measure performance of each API