Jump to: navigation, search

NovaApiValidationFramework

Revision as of 05:03, 11 October 2013 by Ken1ohmichi (talk | contribs) (→‎Implementation)

Nova API Validation Framework

Discussion

https://etherpad.openstack.org/NovaApiValidationFramework

Introduction

Nova is a RESTful API based HTTP service.
These APIs allow for cloud managed server operations.
For example, we can create a virtual machine instance by sending the following POST request to Nova's URI: /servers.

{
   "server" : {
       "name" : "new-server-test",
       "imageRef" : "http://servers.api.openstack.org/1234/images/52415800-8b69-11e0-9b19-734f6f006e54",
       "flavorRef" : 1
   }
}

Nova contains many RESTful APIs to facilitate its many features.
Moreover, each RESTful API can be controlled flexibly through multiple parameters.
For example, the above "create a virtual machine instance" API has 19 parameters including "name", "imageRef", etc.

The HTTP service need to validate practically every API parameter in terms of acceptable type and minimum/maximum length and range.
Unnecessary workload can be avoided by validating API parameters before executing the API operation.
Moreover, API validation is an important feature from the viewpoint of HTTP service security.

Background

Figure 1 shows the current API validation overview in Nova.

  1. Select API method corresponding to the received API request.
  2. Execute API method.
  3. Possible to execute certain operations even if a parameter is invalid.
Figure 1

Figure 1

Not all API parameters are completely validated, so many API operations are executed without API parameter validation.
Furthermore, it is difficult to handle error messages on the API caller side because error message formats are not unified.

Objectives

As Nova is an HTTP service, it should be able to validate API parameters as follows:

  • Validate every API parameter.
  • Return an error response before API operation, if API parameter is invalid.
  • Unify the error message format of the response, if the cause is the same.
    (ex) ".. is too short.", ".. is too long.", ".. is not integer."

Migration plan

TBD

Future

I wish the following implementations after applying this framework to every Nova APIs.

  • Add the existence check of API parameter definition to Jenkins
    OpenStack utilizes a good development process through Jenkins which is used to verify the coding style and to ensure functionality, when a code change is proposed. We feel it is good to add the existence check of API parameter definition to this process. By doing so, we will be able to keep the APIs secure.
  • Apply API validation framework to other components
    By applying this framework to other components, we feel that the overall safety OpenStack APIs will be improved.
    If we can successfully apply this framework to Nova, which contains many APIs with complex API parameters, we will be able to apply it to other components as well.
    We hope to collaborate with many developers on this framework.

Abandon Proposal

We would like to propose an API validation framework to implement comprehensive validation.

Validation point

In this framework, the API validations are separated from API methods, and the API validations are executed before each API method.
Figure 2 shows the API validation of this framework.

  1. Select both the API schema definition and API method corresponding to the received API request.
  2. Validate API parameters based on the API schema definition.
  3. Execute API method if the API validation is successful.
Figure 2

Figure 2

Merits

The OpenStack community releases a new Nova every six months, and the Nova APIs are increasing with each version. Nova currently has hundreds of APIs.
With multiple parameters for each API, a lot of effort is required to review and implement the API schema definition.
However, we believe this framework is worth implementing because of the following merits.

  • Clarification of the API parameter definitions.
    The type and acceptable range and length of each parameter are clarified in the API schema definitions.
    By defining the schemas of every API, developers will be able to create application more easily using Nova APIs.
  • Cleaned up code
    The amount of Nova code can be reduced by merging validations and error response methods.
    For example, in the “create a virtual machine instance” API, the following 8 validation lines exist for the “min_count” parameter. 
       try:
           min_count = int(str(min_count))
       except ValueError:
           msg = _('min_count must be an integer value')
           raise exc.HTTPBadRequest(explanation=msg)
       if min_count < 1:
           msg = _('min_count must be > 0')
           raise exc.HTTPBadRequest(explanation=msg)
By using this proposed framework, we can merge these into the following single line.
       'min_count': {'type': 'integer', 'minimum': 1},

Appendix

Retrieved from "https://wiki.openstack.org/w/index.php?title=NovaApiValidationFramework&oldid=32090"