Writing Request Extensions

This page is, at present, sketchy notes on how to create a new "request" extension (an extension applying to existing nova API operations, such as showing information about a server). The primary emphasis is using XML templates, which were created as a means of enabling the request extensions to actually exist in the first place.

Preliminaries

First, we'll discuss the creation of an extension in general. Extensions to be distributed with nova should live in the `nova/api/openstack/compute/contrib/` directory, and the class describing the extension (which we'll get to shortly) should have a name identical to that of the module, with the first letter capitalized; that is, if your module is "admin_actions.py", then your class should be named "Admin_actions" (yes, with the underscore; this is documentation of existing practice, not an approval of it). This naming convention is only necessary for extensions that should always be active; optional extensions should not live in this directory and can have any desired naming scheme. To load these extensions, specify the dotted path to the module as an argument to the `--osapi_extension` flag. (This flag may be given multiple times, to specify additional extensions.)

Now, on to the structure of an extension. An extension is simply a class; it is recommended that it extend `nova.api.openstack.extensions.ExtensionDescriptor`, but this is not required. What is required is that the class have a doc string, which will be used as the description of the extension sent to the user upon request. Additionally, the following four class attributes must be present:

name: The name of the extension. This need not match the class name.
alias: An alias for the extension. This is used as the namespace prefix on XML elements.
namespace: An XML namespace declaration, typically a URL to a document describing the extension.
updated: A complete ISO 8601-formatted date and time, with timezone, indicating the last update time of the extension. This is used for versioning. An example value would be "2011-10-27T15:00:00-0500", corresponding to 3 PM in US Central Daylight Time on October 27, 2011.

The extension must have an `init()` method taking a single argument; it should call the `register()` method of that argument, passing it `self`. This is provided by `ExtensionDescriptor`:

#!highlight python
def __init__(self, ext_mgr):
    ext_mgr.register(self)

The only other thing the extension requires is at least one of the following three methods:

get_resources(): Returns a list of `nova.api.openstack.extensions.ResourceExtension` objects describing new resources. A resource extension introduces a new resource (i.e., a new URL component). This document does not describe these further.
get_actions(): Returns a list of `nova.api.openstack.extensions.ActionExtension` objects describing new actions. An action extension introduces a new action defined on the `action` endpoint of an existing resource. This document does not describe these further.
get_request_extensions(): Returns a list of `nova.api.openstack.extensions.RequestExtension` objects describing extensions to existing resources.

Methods not needed to implement the extension are not required to be present. The `ExtensionDescriptor` class provides default implementations of these methods which simply return empty lists, but it is legal to omit them if you are not extending that class.

Request Extensions

As mentioned above, `get_request_extensions()` returns a list of `RequestExtension` instances. Creating a request extension is as simple as creating a function or other callable taking three arguments, and passing it—along with the HTTP method and the URL to extend—to the `RequestExtension` constructor, like so:

#!highlight python
def handle_request(req, res, body):
    pass
...
    def get_request_extensions(self):
        return [RequestExtension('GET', '/foo', handle_request)]

(For an example of this in practice, see `nova/tests/api/openstack/compute/extensions/foxinsocks.py`.)

The handler is passed a `Request` object, the `Response` object generated by the nova API, and `body`, which is the actual, unserialized object being returned to the caller. (This object is deserialized from the response's `body` attribute.) The handler should not touch the `body` attribute of the response; it should, instead, manipulate the object passed as the `body` argument. It is perfectly reasonable for a request extension to manipulate other parts of the response, for instance, setting a header.

Any elements that a request extension adds to the `body` object it is passed should be prefixed by the extension's alias value. For example, an extension with alias "EXA-EXT", adding the 'example' key to the `body`, would do something like the following:

#!highlight python
    body['EXA-EXT:example'] = "Example value"

XML Responses

The procedure indicated above is relatively straightforward, when the response body is requested in JSON format. However, nova also supports XML serialization, and by default, extra attributes such as the above will not be serialized. Serialization was recently rewritten to enable this capability through the use of XML templates. The XML templates support was written to look similar to the ElementTree interface, so the reader may wish to familiarize themselves with that system before proceeding.

Basic overview: An XML template is constructed using `nova.api.openstack.xmlutil.TemplateElement` instances, arranged into a tree (for which, the `nova.api.openstack.xmlutil.SubTemplateElement()` helper function may be useful). Each such template element corresponds to an XML element, and has attributes (settable using the `set()` method, or using keyword arguments to the constructor) and text (settable using the `text` property of the template element). Once a tree of elements has been constructed, it is used to construct a `nova.api.openstack.xmlutil.Template` (of which there are two usable subclasses, `nova.api.openstack.xmlutil.MasterTemplate` and `nova.api.openstack.xmlutil.SlaveTemplate`; more about these in a moment). The critical component is data selectors, which specify the source of the data to include in the text or attribute. A selector is simply a callable taking two arguments; the first is the `body` object (or some already-selected subcomponent of the `body` object), and the second is simply a `do_raise` boolean indicating whether the selector should return `None` if the data is not found (a `False` value of `do_raise`) or raise a `KeyError` (a `True` value of `do_raise`). There exists `nova.api.openstack.xmlutil.Selector` and `nova.api.openstack.xmlutil.ConstantSelector` classes for building these selectors, but the templates system normally constructs these automatically.

We go into further detail in the following sections.

TemplateElement

Each kind of element in the final XML tree corresponds to an instance of `TemplateElement`. A `TemplateElement` has a tag—which is normally a string, but may also be a selector to set the name dynamically—; a set of attributes; and text. (Note that `TemplateElement` does not have the `ElementTree` concept of "tail" text.) A `TemplateElement` also has an optional selector associated with it: the selector picks out the part of the object which it, and all its children, will operate on; this is the object which will be passed to the selectors used by text, attributes, and optionally the tag name. If not given, `TemplateElement` uses the "identity" selector, i.e., the object passed in to the template element is the object that will be used by attributes, text, and children. Additionally, for ease of use, the selector may be a string or integer, in which case it is converted to a selector which extracts the object indexed by that value. For instance, if the selector is given as `"foo"`, and the object is given by:

#!highlight python
    {'foo': [1, 2, 3],
     'bar': [4, 5, 6]
    }

Then the object the template element uses will be the list `[1, 2, 3]`. Note that it is recommended to always give the `selector` argument to the constructor as a keyword argument. The `TemplateElement` constructor also takes a dictionary, named `attrib`, as well as optional keyword arguments; these are combined together and used to set attributes (see Setting Attributes below).

Template elements may have child elements; these child elements can be appended to their parent element using the `append()` or `extend()` methods. Alternatively, the helper function `SubTemplateElement()` is provided, which takes as its first argument the parent element. (Note that, unlike with `ElementTree`'s `SubElement()` constructor, the parent argument may be `None`, in which case the resulting element has no parent.) This may be used to easily build full trees of `TemplateElement` instances, describing a final XML template capable of rendering a JSON object into XML.

One last note about template element selectors: when the selector returns a list, the final XML document will contain one corresponding element for each entry in the list.

As an example, let's assume we have a template element `"foo"`, and the object obtained by its selector is the list `[1, 2, 3]`. We will assume that the `text` attribute of the element has been set to `Selector()` (`None` causes no text to be rendered). We will further assume that the top-level element is named `"example"`, just to have valid XML output. When we render this template, we will obtain the following document:

#!highlight xml
<example>
    <foo>1</foo>
    <foo>2</foo>
    <foo>3</foo>
</example>

Note that, by default, if no object is selected by the template element selector, the element will be omitted from the final document. This may be overridden by extending `TemplateElement` and overriding the `will_render()` method. The `will_render()` method is passed the object selected (which may be `None` if the selector could not locate the data item), and must return `True` or `False` depending on whether the element should be rendered into an XML element or not.

Setting Attributes

Template elements may specify attributes to set on the result XML element, either via the constructor or through the use of the `set()` method. For the `set()` method, the first argument is the name of the attribute (which must be a fixed string; no selectors here, unlike the `tag` name of the template element). The second argument is a selector, which extracts the required data from the object; however, just as with element selectors, there are some reasonable defaults which make constructing selectors easy. If the value is a string or integer, then the corresponding element of the object is used, just as with template elements; however, the `set()` method also allows the value argument to be omitted, in which case the selector is constructed as if the second argument were a string identical to the first. For instance, to have an attribute on an element that has the same name as the corresponding JSON element, using `set("key")` is sufficient.

Note that, if a selected value does not exist on the object, the attribute is omitted.

Setting Text

Template elements have an optional text value associated with them, through the `text` property. It may be assigned an integer or string, which will be converted to a selector as for the element selector; or it may be directly assigned a selector; or it may be set to None, in which case no text will be rendered. Note that, because `text` is a property, `del elem.text` is equivalent to `elem.text = None`. Also note that, if the text selector cannot extract the value, the literal text `"None"` will be rendered.

Selectors

We have frequently referred to selectors, specially-constructed callables which extract subobjects from the object passed in to be rendered. Most of the time, the default behavior of converting strings and integers into selectors is sufficient; however, two classes have been provided to explicitly build selectors in the event that this is insufficient. The first class is the `ConstantSelector` class; when constructed, it takes a single argument which is a constant; when this selector is used, instead of extracting data from the object passed in, it returns the constant value it was constructed with.

The second class is the `Selector` class; its constructor takes the specified arguments—which must be strings, integers, or one-argument callables (more on this in a moment)—and saves them. When this selector is called on an object, it applies each index or callable in turn, then returns the result of the chain of keys and callables. For instance, if we construct `Selector("foo", 1)` and call it on our example object from above, we obtain the value `2`.

As mentioned, the `Selector` class may take callables as well as indexes. There are XML templates in nova which require that a JSON dictionary be broken down into keys and values. To accomplish this, the templates system provides `nova.api.openstack.xmlutil.get_items()`, which is a callable expecting a dictionary, upon which it calls the `items()` method. Consider a template element—again with a parent `"example"` element, to produce compliant XML—where the tag name is given as `Selector(0)` and the `text` property is given as `1`, and the template element selector is `Selector("foo", get_items)`. Given the following object:

#!highlight python
{"foo": {
  "a": 1,
  "b": 2,
  "c": 3
  }
}

When we render the described template, we will have the document:

#!highlight xml
<example>
    <a>1</a>
    <b>2</b>
    <c>3</c>
</example>

Templates

So far, we have described template elements; however, a single template element is not able to render itself into XML without some external support. This external support comes from the subclasses of the `Template` class. There are two subclasses of `Template` in particular: `MasterTemplate` and `SlaveTemplate`.

A `MasterTemplate` is an object combining a root `TemplateElement` with a simple version number and the XML namespace, as a dictionary. Within the nova API, each JSON object returned by the base API controllers has a corresponding `MasterTemplate`. The true power of the template approach is that `SlaveTemplate` instances may be attached to a `MasterTemplate`. When the object is rendered into XML, the attached `SlaveTemplate`s are also rendered. Further, each `SlaveTemplate` can include a range of `MasterTemplate` versions to which it applies, and the `attach()` method of the `MasterTemplate` automatically selects only those `SlaveTemplate`s which apply to it. (A `SlaveTemplate` is instantiated with the root `TemplateElement`, a minimum master template version, an optional maximum master template version, and an XML namespace, as a dictionary. When the final XML document is rendered, the namespace provided for all templates is merged, so the slave template need only specify the namespace elements it needs for itself.)

TemplateBuilder

The templates system provides one additional piece of functionality to improve efficiency: templates can take as much computational overhead to generate as could the desired XML document. In order to reduce this overhead, the `TemplateBuilder` class is provided. `TemplateBuilder` is an unusual class, in that users will never obtain instances of it from its constructor; it operates much more like a factory function, which implements a template cache. The first time a `TemplateBuilder` subclass is invoked, it calls its `construct()` method (which must be provided by the subclass). The `construct()` method must return an instance of `MasterTemplate` or `SlaveTemplate`, which will then be cached. Subsequent invocations return the cached value, or, in the case of `MasterTemplate`, shallow copies of the cached value. (These shallow copies mean that attaching a slave template to a master template does not affect other copies of the master template.)

Lazy Serialization

Now that we have described XML templates, we need to consider how an extension uses them. When a caller to the API requests an XML document, the `nova.template` variable will be set in the request environment. (This variable is accessible as `req.environ["nova.template"]`; its presence or absence should be tested using `"nova.template" in req.environ`.) If a request extension adds data to the `body` object, it should also test to see if an XML template is available in the environment. If one is, then it should build a slave template and attach it to the template in the environment.

Example

#!highlight python
from nova.api.openstack import extensions
from nova.api.openstack import xmlutil


# Describe our extension.
class ExampleExtension(extensions.ExtensionDescriptor):
    "An example request extension."

    name = "Example Extension"
    alias = "EXA-EXT"
    namespace = "http://example.org/extension/v1.0"
    updated = "2011-10-27T15:00:00-0500"

    # This method is called to get the list of request extensions.
    def get_request_extensions(self):
        return [extensions.RequestExtension('GET',
                "/v1.1/:(project_id)/servers/:(id)",
                self.example_handler)]

    # Here we've opted to have the implementation be part of the
    # extension descriptor.  We don't have to do things this way,
    # though; it could be another function or a method of another
    # class, or even an inner function.
    def example_handler(self, request, response, body):
        # Add a value to the body
        key = '%s:example' % self.alias
        body['server'][key] = "An example extension"

        # Do we need to attach a template?
        if 'nova.template' in request.environ:
            tmpl = request.environ['nova.template']
            tmpl.attach(ExampleTemplate())

        return response


# Describe our XML template.  Remember that TemplateBuilder
# acts more like a factory function than a class.
class ExampleTemplate(xmlutil.TemplateBuilder):
    def construct(self):
        # Our root element is a <server> element
        root = xmlutil.TemplateElement('server')

        # We're adding an attribute, but we could also do this
        # as another element.  The expression in the braces comes
        # from the way lxml works with namespaced attributes, while
        # the value here is the name of the additional data in our
        # JSON.
        root.set('{%s}example' % ExampleExtension.namespace,
                 '%s:example' % ExampleExtension.alias)

        # Construct and return the actual template.  Notice how
        # we constructed and specified the namespace map.
        return xmlutil.SlaveTemplate(root, 1, nsmap={
            ExampleExtension.alias: ExampleExtension.namespace
            })

WritingRequestExtensions

Contents