Launchpad Entry: easy-api
Created: 2010-12-28
Contributors: termie
Summary
The current available public REST-like interfaces are rather different from each other, following very different philosophies, but are both rather large in their scope.
Both current systems go through great effort in transforming data and operations to map existing interface specifications to our underlying implementations. The current systems also are very hard-coded in what they contain and the only way to extend them is to modify sections of their (very differently styled) code.
Easy API attempts to back away from the rigidity and imperfect mappings of those existing systems and instead provide a simple, thin and easily extensible direct mapping of the underlying systems to allow the quickest path from idea to available feature.
Specific goals:
- Allow users to add additional services to Nova without changes to core Nova code.
- Provide a reflection service to list the available services and methods.
- Provide a command-line tool built upon the reflection service to automatically provide access to all available services and methods.
- Offer delegated auth, allowing user authentication to be performed anywhere up the stack (either via wsgi or any other http-rewriting tool such as a proxy).
- Be a thin shim between the services and the end users.
Specific non-goals:
- Not a full-featured plugin system, the extensibility interface should remain simple enough that it need not be considered a feature of its own.
- Make a large decision about how auth will be implemented.
Release Note
This release includes a new, extensible REST-like interface directly to all of the Nova services and a tool named stack to interact with them from the command-line.
As of this release, separate packages to extend the functionality of Nova will be possbile and will likely register themselves by placing a python file in /etc/nova/include.d
Rationale
new paradigm vs extending the existing code
The decision to provide a completely separate approach to the public API came largely from the complexity of extending the current APIs where a significant amount of boilerplate and filler code was necessary to connect functionality to actual end users, and even once that was in place work was still necessary to update tools to take advantage of the new feature.
The new approach allows for the public API to be built dynamically from the services that make it up and for a tool to be written that automatically has access to all new features meaning no new code needs to be written to provide access to new functionality.
delegated auth vs $SOME_AUTH_SYSTEM
Rather than attempt to make the decisions about which auth system should be used, the scope of this interface is explicitly limited to allowing other auth systems to be used.
By expecting a user to be authenticated by the time it gets to the method dispatch phase and providing an additional simple HTTP-based method of passing along the authenticated credentials this proposal avoids making any strong demands on the design of the auth system and provides an easy connection point for a large number of existing auth systems.
external code as services vs modifications to the Nova project
Allowing code external to the Nova project to register itself with the API provides developers with a faster path to adding new features than attempting to get their changes merged into or maintaining patches against Nova core.
Letting external code easily provide new functionality, or even override existing functionality, allows developers to package and release new features separate from the Nova release process.
User stories
Susan wants to add a new method to the Compute service
Susan realized that many users probably would like to get some statistics about the load on their instances, so she is going to add a get_instance_load method to the public API.
She goes into nova/compute/api.py and adds the method to the ComputeAPI class.
She now can run stack compute get_instance_load instance_id=my_id from the command-line to get the output of her new method.
Basti thinks Nova could really use a backup service
Basti runs a service that hosts secure backups and wants to try to sell it to people who use Nova.
Basti writes a python library that interacts with Nova to make backups from instances to his service. He then packages it in a debian package, nova-backuppro with a file backuppro.py that will be copied into /etc/nova/include.d and read by nova-easy-api on startup.
That file, backuppro.py will load his python library and call nova.api.easy.register_service('backuppro', BackupProInstance)
Assumptions
- The benefits of a faster iteration time and extensibility outweigh the pains of a new interface.
Design
Self-registration
Easy API should provide an in-process method for libraries that have been loaded to register their intent to be part of the API rather than having the available functionality be hard-coded.
Minimal data munging
Easy API should expect that request parameters provided map directly to a service, method and the keyword arguments for that method, the result of that method will be serialized as is and returned.
Reflection for a better tomorrow
Easy API should provide reflection methods so that tools can quickly discover and interact with all available services and methods.
No special public API classes
Easy API should not require anything beyond the inter-service API handle to be written in order to provide functionality to the public API.
Delegated auth
Easy API expects the user to have been authenticated already, a new authentication system is outside the scope of this proposal.
Serializable input and output
Easy API makes the requirement that inter-service API methods accept serializable input and return serializable output, they should not attempt to pass function or instance references.
Implementation
Related branch: http://code.launchpad.net/~termie/nova/easy_api
UI Changes
Provides an entirely new command-line tool to interact with the whole API.
Existing tools are unchanged.
Code Changes
As this code is almost entirely self-contained there are minimal changes to existing code, the largest changes being to ensure that inter-service APIs are accepting and returning serializable data.
The Delegated Auth Middleware
The DelegatedAuthMiddleware will read the values for 'user' and 'project' from the HTTP headers, defaulting to X-OpenStack-User and X-OpenStack-Project, allowing anything before it to set those headers to authenticate the user.
Obviously, when deployed whatever sits in front of this interface should prevent external users from sending those headers by stripping them from input.
The Reflection service
The Reflection service provides a list of available methods, their arguments and their documentation by introspecting the services that have been registered by nova.api.easy.register_service()
Self-Registration
nova.api.easy.register_service takes a path and python object instance causing requests that begin with that path to be routed to methods on that python object.
Migration
Code only change, no data migration.
While not required, it is possible (and straightforward) to migrate the existing REST interfaces ("ec2" and "openstack") to backend themselves on this interface. Towards that possibility a Proxy class is provided that acts as if it were the inter-service API handle but actually moves method calls through the Easy API.
Test/Demo Plan
Normal unit tests are expected to run.
To test the new code in realistic conditions, the Proxy class is also used to wrap inter-service API handles and monkeypatch them while running the tests for existing public API tests.
Unresolved issues
- We should probably provide one additional auth integration point besides the HTTP-based one, probably something like register_auth().
BoF agenda and discussion
Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarising what was discussed and note any options that were rejected.