Difference between revisions of "NovaApiValidationFramework"
Ken1ohmichi (talk | contribs) (→Appendix) |
Ken1ohmichi (talk | contribs) (→Implementation) |
||
Line 43: | Line 43: | ||
* Return an error response before API operation, if API parameter is invalid. | * Return an error response before API operation, if API parameter is invalid. | ||
* Unify the error message format of the response, if the cause is the same.<br />(ex) ".. is too short.", ".. is too long.", ".. is not integer." | * Unify the error message format of the response, if the cause is the same.<br />(ex) ".. is too short.", ".. is too long.", ".. is not integer." | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
== Migration plan == | == Migration plan == |
Revision as of 05:03, 11 October 2013
Nova API Validation Framework
Contents
Discussion
https://etherpad.openstack.org/NovaApiValidationFramework
Introduction
Nova is a RESTful API based HTTP service.
These APIs allow for cloud managed server operations.
For example, we can create a virtual machine instance by sending the following POST request to Nova's URI: /servers.
{ "server" : { "name" : "new-server-test", "imageRef" : "http://servers.api.openstack.org/1234/images/52415800-8b69-11e0-9b19-734f6f006e54", "flavorRef" : 1 } }
Nova contains many RESTful APIs to facilitate its many features.
Moreover, each RESTful API can be controlled flexibly through multiple parameters.
For example, the above "create a virtual machine instance" API has 19 parameters including "name", "imageRef", etc.
The HTTP service need to validate practically every API parameter in terms of acceptable type and minimum/maximum length and range.
Unnecessary workload can be avoided by validating API parameters before executing the API operation.
Moreover, API validation is an important feature from the viewpoint of HTTP service security.
Background
Figure 1 shows the current API validation overview in Nova.
- Select API method corresponding to the received API request.
- Execute API method.
- Possible to execute certain operations even if a parameter is invalid.
Figure 1
Not all API parameters are completely validated, so many API operations are executed without API parameter validation.
Furthermore, it is difficult to handle error messages on the API caller side because error message formats are not unified.
Objectives
As Nova is an HTTP service, it should be able to validate API parameters as follows:
- Validate every API parameter.
- Return an error response before API operation, if API parameter is invalid.
- Unify the error message format of the response, if the cause is the same.
(ex) ".. is too short.", ".. is too long.", ".. is not integer."
Migration plan
TBD
Future
I wish the following implementations after applying this framework to every Nova APIs.
- Add the existence check of API parameter definition to Jenkins
OpenStack utilizes a good development process through Jenkins which is used to verify the coding style and to ensure functionality, when a code change is proposed. We feel it is good to add the existence check of API parameter definition to this process. By doing so, we will be able to keep the APIs secure.
- Create API specification document
We think it is possible to create the elements of an API specification document from the API schema definition.
OpenStack community has API specification documents of all OpenStack components[1] and ones for each component[2].
We can extract certain elements of all the API parameters from the API schema definition.
[1]: http://api.openstack.org/api-ref.html
[2]: Nova: http://docs.openstack.org/api/openstack-compute/2/content/
- Apply API validation framework to other components
By applying this framework to other components, we feel that the overall safety OpenStack APIs will be improved.
If we can successfully apply this framework to Nova, which contains many APIs with complex API parameters, we will be able to apply it to other components as well.
We hope to collaborate with many developers on this framework.
Abandon Proposal
We would like to propose an API validation framework to implement comprehensive validation.
Validation point
In this framework, the API validations are separated from API methods, and the API validations are executed before each API method.
Figure 2 shows the API validation of this framework.
- Select both the API schema definition and API method corresponding to the received API request.
- Validate API parameters based on the API schema definition.
- Execute API method if the API validation is successful.
Figure 2
Merits
The OpenStack community releases a new Nova every six months, and the Nova APIs are increasing with each version.
Nova currently has hundreds of APIs.
With multiple parameters for each API, a lot of effort is required to review and implement the API schema definition.
However, we believe this framework is worth implementing because of the following merits.
- Clarification of the API parameter definitions.
The type and acceptable range and length of each parameter are clarified in the API schema definitions.
By defining the schemas of every API, developers will be able to create application more easily using Nova APIs. - Cleaned up code
The amount of Nova code can be reduced by merging validations and error response methods.
For example, in the “create a virtual machine instance” API, the following 8 validation lines exist for the “min_count” parameter.
try: min_count = int(str(min_count)) except ValueError: msg = _('min_count must be an integer value') raise exc.HTTPBadRequest(explanation=msg) if min_count < 1: msg = _('min_count must be > 0') raise exc.HTTPBadRequest(explanation=msg)
- By using this proposed framework, we can merge these into the following single line.
'min_count': {'type': 'integer', 'minimum': 1},
Appendix
- [1]: http://lists.openstack.org/pipermail/openstack-dev/2013-June/009824.html
- [2]: http://lists.openstack.org/pipermail/openstack-dev/2013-June/010814.html
- [3] API validation framework for Nova[Reviewing]: https://review.openstack.org/#/c/25358/
- API validation framework for Oslo[Abondoned]: https://review.openstack.org/#/c/25884/
- Discussion: https://etherpad.openstack.org/NovaApiValidationFramework