Writing Request Extensions

This page is, at present, sketchy notes on how to create a new "request" extension (an extension applying to existing nova API operations, such as showing information about a server). The primary emphasis is using XML templates, which were created as a means of enabling the request extensions to actually exist in the first place.

Preliminaries

First, we'll discuss the creation of an extension in general. Extensions to be distributed with nova should live in the nova/api/openstack/contrib directory, and the class describing the extension (which we'll get to shortly) should have a name identical to that of the module, with the first letter capitalized; that is, if your module is "admin_actions.py", then your class should be named "Admin_actions" (yes, with the underscore; this is documentation of existing practice, not an approval of it). This naming convention is only necessary for extensions that should always be active; optional extensions should not live in this directory and can have any desired naming scheme. To load these extensions, specify the dotted path to the module as an argument to the --osapi_extension flag. (This flag may be given multiple times, to specify additional extensions.)

Now, on to the structure of an extension. An extension is simply a class; it is recommended that it extend nova.api.openstack.extensions.ExtensionDescriptor, but this is not required. What is required is that the class have a doc string, which will be used as the description of the extension sent to the user upon request. Additionally, the following four class attributes must be present:

name: The name of the extension. This need not match the class name.
alias: An alias for the extension. This is used as the namespace prefix on XML elements.
namespace: An XML namespace declaration, typically a URL to a document describing the extension.
updated: A complete ISO 8601-formatted date and time, with timezone, indicating the last update time of the extension. This is used for versioning. An example value would be "2011-10-27T15:00:00-0500", corresponding to 3 PM in US Central Daylight Time on October 27, 2011.

The extension must have an __init__() method taking a single argument; it should call the register() method of that argument, passing it self. This is provided by ExtensionDescriptor:

def __init__(self, ext_mgr):
    ext_mgr.register(self)

The only other thing the extension requires is at least one of the following three methods:

get_resources(): Returns a list of nova.api.openstack.extensions.ResourceExtension objects describing new resources. A resource extension introduces a new resource (i.e., a new URL component). This document does not describe these further.
get_actions(): Returns a list of nova.api.openstack.extensions.ActionExtension objects describing new actions. An action extension introduces a new action defined on the action endpoint of an existing resource. This document does not describe these further.
get_request_extensions(): Returns a list of nova.api.openstack.extensions.RequestExtension objects describing extensions to existing resources.

Methods not needed to implement the extension are not required to be present. The ExtensionDescriptor class provides default implementations of these methods which simply return empty lists, but it is legal to omit them if you are not extending that class.

Request Extensions

As mentioned above, get_request_extensions() returns a list of RequestExtension instances. Creating a request extension is as simple as creating a function or other callable taking three arguments, and passing it—along with the HTTP method and the URL to extend—to the RequestExtension constructor, like so:

def handle_request(req, res, body):
    pass
...
    def get_request_extensions(self):
        return [RequestExtension('GET', '/foo', handle_request)]

(For an example of this in practice, see nova/tests/api/openstack/extensions/foxinsocks.py.)

The handler is passed a Request object, the Response object generated by the nova API, and body, which is the actual, unserialized object being returned to the caller. (This object is deserialized from the response's body attribute.) The handler should not touch the body attribute of the response; it should, instead, manipulate the object passed as the body argument. It is perfectly reasonable for a request extension to manipulate other parts of the response, for instance, setting a header.

Any elements that a request extension adds to the body object it is passed should be prefixed by the extension's alias value. For example, an extension with alias "EXA-EXT", adding the 'example' key to the body, would do something like the following:

    body['EXA-EXT:example'] = "Example value"

XML Responses

The procedure indicated above is relatively straightforward, when the response body is requested in JSON format. However, nova also supports XML serialization, and by default, extra attributes such as the above will not be serialized. Serialization was recently rewritten to enable this capability through the use of XML templates. The XML templates support was written to look similar to the ElementTree interface, so the reader may wish to familiarize themselves with that system before proceeding.

Basic overview: An XML template is constructed using nova.api.openstack.xmlutil.TemplateElement instances, arranged into a tree (for which, the nova.api.openstack.xmlutil.SubTemplateElement() helper function may be useful). Each such template element corresponds to an XML element, and has attributes (settable using the set() method, or using keyword arguments to the constructor) and text (settable using the text property of the template element). Once a tree of elements has been constructed, it is used to construct a nova.api.openstack.xmlutil.Template (of which there are two usable subclasses, nova.api.openstack.xmlutil.MasterTemplate and nova.api.openstack.xmlutil.SlaveTemplate; more about these in a moment). The critical component is data selectors, which specify the source of the data to include in the text or attribute. A selector is simply a callable taking two arguments; the first is the body object (or some already-selected subcomponent of the body object), and the second is simply a do_raise boolean indicating whether the selector should return None if the data is not found (a False value of do_raise) or raise a KeyError (a True value of do_raise). There exists nova.api.openstack.xmlutil.Selector and nova.api.openstack.xmlutil.ConstantSelector classes for building these selectors, but the templates system normally constructs these automatically.

We go into further detail in the following sections.

TemplateElement

Each kind of element in the final XML tree corresponds to an instance of TemplateElement. A TemplateElement has a tag—which is normally a string, but may also be a selector to set the name dynamically—; a set of attributes; and text. (Note that TemplateElement does not have the ElementTree concept of "tail" text.) A TemplateElement also has an optional selector associated with it: the selector picks out the part of the object which it, and all its children, will operate on; this is the object which will be passed to the selectors used by text, attributes, and optionally the tag name. If not given, TemplateElement uses the "identity" selector, i.e., the object passed in to the template element is the object that will be used by attributes, text, and children. Additionally, for ease of use, the selector may be a string or integer, in which case it is converted to a selector which extracts the object indexed by that value. For instance, if the selector is given as "foo", and the object is given by:

    {'foo': [1, 2, 3],
     'bar': [4, 5, 6]
    }

Then the object the template element uses will be the list [1, 2, 3]. Note that it is recommended to always give the selector argument to the constructor as a keyword argument. The TemplateElement constructor also takes a dictionary, named attrib, as well as optional keyword arguments; these are combined together and used to set attributes (see Setting Attributes below).

Template elements may have child elements; these child elements can be appended to their parent element using the append() or extend() methods. Alternatively, the helper function SubTemplateElement() is provided, which takes as its first argument the parent element. (Note that, unlike with ElementTree's SubElement() constructor, the parent argument may be None, in which case the resulting element has no parent.) This may be used to easily build full trees of TemplateElement instances, describing a final XML template capable of rendering a JSON object into XML.

One last note about template element selectors: when the selector returns a list, the final XML document will contain one corresponding element for each entry in the list.

As an example, let's assume we have a template element "foo", and the object obtained by its selector is the list [1, 2, 3]. We will assume that the text attribute of the element has been set to Selector() (None causes no text to be rendered). We will further assume that the top-level element is named "example", just to have valid XML output. When we render this template, we will obtain the following document:

<example>
    <foo>1</foo>
    <foo>2</foo>
    <foo>3</foo>
</example>

Note that, by default, if no object is selected by the template element selector, the element will be omitted from the final document. This may be overridden by extending TemplateElement and overriding the will_render() method. The will_render() method is passed the object selected (which may be None if the selector could not locate the data item), and must return True or False depending on whether the element should be rendered into an XML element or not.

Setting Attributes

Template elements may specify attributes to set on the result XML element, either via the constructor or through the use of the set() method. For the set() method, the first argument is the name of the attribute (which must be a fixed string; no selectors here, unlike the tag name of the template element). The second argument is a selector, which extracts the required data from the object; however, just as with element selectors, there are some reasonable defaults which make constructing selectors easy. If the value is a string or integer, then the corresponding element of the object is used, just as with template elements; however, the set() method also allows the value argument to be omitted, in which case the selector is constructed as if the second argument were a string identical to the first. For instance, to have an attribute on an element that has the same name as the corresponding JSON element, using set("key") is sufficient.

Note that, if a selected value does not exist on the object, the attribute is omitted.

Setting Text

Template elements have an optional text value associated with them, through the text property. It may be assigned an integer or string, which will be converted to a selector as for the element selector; or it may be directly assigned a selector; or it may be set to None, in which case no text will be rendered. Note that, because text is a property, del elem.text is equivalent to elem.text = None. Also note that, if the text selector cannot extract the value, the literal text "None" will be rendered.

Selectors

We have frequently referred to selectors, specially-constructed callables which extract subobjects from the object passed in to be rendered. Most of the time, the default behavior of converting strings and integers into selectors is sufficient; however, two classes have been provided to explicitly build selectors in the event that this is insufficient. The first class is the ConstantSelector class; when constructed, it takes a single argument which is a constant; when this selector is used, instead of extracting data from the object passed in, it returns the constant value it was constructed with.

The second class is the Selector class; its constructor takes the specified arguments—which must be strings, integers, or one-argument callables (more on this in a moment)—and saves them. When this selector is called on an object, it applies each index or callable in turn, then returns the result of the chain of keys and callables. For instance, if we construct Selector("foo", 1) and call it on our example object from above, we obtain the value 2.

As mentioned, the Selector class may take callables as well as indexes. There are XML templates in nova which require that a JSON dictionary be broken down into keys and values. To accomplish this, the templates system provides nova.api.openstack.xmlutil.get_items(), which is a callable expecting a dictionary, upon which it calls the items() method. Consider a template element—again with a parent "example" element, to produce compliant XML—where the tag name is given as Selector(0) and the text property is given as 1, and the template element selector is Selector("foo", get_items). Given the following object:

{"foo": {
  "a": 1,
  "b": 2,
  "c": 3
  }
}

When we render the described template, we will have the document:

<example>
    <a>1</a>
    <b>2</b>
    <c>3</c>
</example>

Templates

So far, we have described template elements; however, a single template element is not able to render itself into XML without some external support. This external support comes from the subclasses of the Template class. There are two subclasses of Template in particular: MasterTemplate and SlaveTemplate.

A MasterTemplate is an object combining a root TemplateElement with a simple version number and the XML namespace, as a dictionary. Within the nova API, each JSON object returned by the base API controllers has a corresponding MasterTemplate. The true power of the template approach is that SlaveTemplate instances may be attached to a MasterTemplate. When the object is rendered into XML, the attached SlaveTemplates are also rendered. Further, each SlaveTemplate can include a range of MasterTemplate versions to which it applies, and the attach() method of the MasterTemplate automatically selects only those SlaveTemplates which apply to it. (A SlaveTemplate is instantiated with the root TemplateElement, a minimum master template version, an optional maximum master template version, and an XML namespace, as a dictionary. When the final XML document is rendered, the namespace provided for all templates is merged, so the slave template need only specify the namespace elements it needs for itself.)

TemplateBuilder

The templates system provides one additional piece of functionality to improve efficiency: templates can take as much computational overhead to generate as could the desired XML document. In order to reduce this overhead, the TemplateBuilder class is provided. TemplateBuilder is an unusual class, in that users will never obtain instances of it from its constructor; it operates much more like a factory function, which implements a template cache. The first time a TemplateBuilder subclass is invoked, it calls its construct() method (which must be provided by the subclass). The construct() method must return an instance of MasterTemplate or SlaveTemplate, which will then be cached. Subsequent invocations return the cached value, or, in the case of MasterTemplate, shallow copies of the cached value. (These shallow copies mean that attaching a slave template to a master template does not affect other copies of the master template.)

Lazy Serialization

Now that we have described XML templates, we need to consider how an extension uses them. When a caller to the API requests an XML document, the nova.template variable will be set in the request environment. (This variable is accessible as req.environ["nova.template"]; its presence or absence should be tested using "nova.template" in req.environ.) If a request extension adds data to the body object, it should also test to see if an XML template is available in the environment. If one is, then it should build a slave template and attach it to the template in the environment.

Example

from nova.api.openstack import extensions
from nova.api.openstack import xmlutil


# Describe our extension.
class ExampleExtension(extensions.ExtensionDescriptor):
    "An example request extension."

    name = "Example Extension"
    alias = "EXA-EXT"
    namespace = "http://example.org/extension/v1.0"
    updated = "2011-10-27T15:00:00-0500"

    # This method is called to get the list of request extensions.
    def get_request_extensions(self):
        return [extensions.RequestExtension('GET',
                "/v1.1/:(project_id)/servers/:(id)",
                self.example_handler)]

    # Here we've opted to have the implementation be part of the
    # extension descriptor.  We don't have to do things this way,
    # though; it could be another function or a method of another
    # class, or even an inner function.
    def example_handler(self, request, response, body):
        # Add a value to the body
        key = '%s:example' % self.alias
        body['server'][key] = "An example extension"

        # Do we need to attach a template?
        if 'nova.template' in request.environ:
            tmpl = request.environ['nova.template']
            tmpl.attach(ExampleTemplate())

        return response


# Describe our XML template.  Remember that TemplateBuilder
# acts more like a factory function than a class.
class ExampleTemplate(xmlutil.TemplateBuilder):
    def construct(self):
        # Our root element is a <server> element
        root = xmlutil.TemplateElement('server')

        # We're adding an attribute, but we could also do this
        # as another element.  The expression in the braces comes
        # from the way lxml works with namespaced attributes, while
        # the value here is the name of the additional data in our
        # JSON.
        root.set('{%s}example' % ExampleExtension.namespace,
                 '%s:example' % ExampleExtension.alias)

        # Construct and return the actual template.  Notice how
        # we constructed and specified the namespace map.
        return xmlutil.SlaveTemplate(root, 1, nsmap={
            ExampleExtension.alias: ExampleExtension.namespace
            })

Wiki

Page

User