Jump to: navigation, search

Difference between revisions of "ListTestedApis"

(Summary)
(New API specifications)
Line 25: Line 25:
  
 
New APIs are based on "nova coverage" APIs.
 
New APIs are based on "nova coverage" APIs.
 +
 
Recording tested APIs is started by API action "start" and stopped by action "report".
 
Recording tested APIs is started by API action "start" and stopped by action "report".
 +
 
Action "report" also create a report that shows if each API is tested or not.
 
Action "report" also create a report that shows if each API is tested or not.
 +
 
The record is reset by action "reset".
 
The record is reset by action "reset".
 +
 
Action "report" has a mandatory request field 'file' that is a name of a report file.
 
Action "report" has a mandatory request field 'file' that is a name of a report file.
 +
 
* Start API recording
 
* Start API recording
 
<pre>
 
<pre>

Revision as of 07:13, 25 March 2013

Summary

Now adding missing Tempest tests is going forward aggressively in a Tempest BP(https://blueprints.launchpad.net/tempest/+spec/missing-tempest-tests).

This BP promotes to find missing tests effectively by listing all Nova APIs with tested/untested marks.

Developers can start/stop API recording, list tested APIs, and find missing test by looking for untested APIs.

Usage

The main target is test developers. Developers start/stop recording and create a tested API report to find missing tests.

Components to Consider

At first, the main target is Nova. Extend other components after enough mature codes.

Design

Overview

Implement a new middleware and apis to create Nova API list, analyze incoming APIs, and manage actions of start/report/reset recording.

New API specifications

New APIs are based on "nova coverage" APIs.

Recording tested APIs is started by API action "start" and stopped by action "report".

Action "report" also create a report that shows if each API is tested or not.

The record is reset by action "reset".

Action "report" has a mandatory request field 'file' that is a name of a report file.

  • Start API recording
 POST /v2/{project_id}/os-api-coverage/action
 {"start": {}}
  • Stop API recording and Output API record
 POST /v2/{project_id}/os-api-coverage/action
 {"report": {"file": "xxx"}}
  • Reset API record
 POST /v2/{project_id}/os-api-coverage/action
 {"reset": {}}

Sample Report of Prototype

Tested APIs are marked "Y" at "Tested" column.

Tested API: xxx/yyy
Tested No. Method URL                                              Action
       000 GET    /
       001 POST   /{project_id}/servers/:server_id/consoles
       002 GET    /{project_id}/servers/:server_id/consoles
       003 PUT    /{project_id}/servers/:server_id/consoles/:(id)
       004 DELETE /{project_id}/servers/:server_id/consoles/:(id)
       005 GET    /{project_id}/servers/:server_id/consoles/:(id)
     Y 006 POST   /{project_id}/servers
       007 GET    /{project_id}/servers/detail
     Y 008 GET    /{project_id}/servers
     Y 009 PUT    /{project_id}/servers/:(id)
       010 POST   /{project_id}/servers/:(id)/action               restore
       011 POST   /{project_id}/servers/:(id)/action               suspend
       012 POST   /{project_id}/servers/:(id)/action               migrate
...
       239 GET /{project_id}/os-snapshots/:(id)

Future Work

  • Add an option to Tempest
  • Analyze request body
  • Measure performance of each API