XMLTemplates

XMLTemplates
Drafter: Vek

Drafters Email:  <>

Status:  To be completed by POC

Currently, request extensions have to deserialize data, manipulate it, and reserialize it. This happens for each extension. This is generally not much of a problem for JSON data, but XML is expensive to deserialize and reserialize, and it is currently impossible to extend the XML without directly editing the serializer or writing an independent serializer.

This proposal is to establish new middleware which performs the XML serialization, and to also establish a means of creating XML templates, which can be extended by attaching slave templates to describe the extra data that an extension may wish to include.

Much of the code for this is already written, and many elements of nova-api converted over to use template-based serializers in preference to the hand-coded serializers currently utilized. This page is to document what has been done.

Lazy Serialization
The first and simplest part of the specification is lazy serialization. Basically, this involves the creation of a LazySerializationMiddleware which is added to the openstackapi pipelines, and some modifications to nova.api.openstack.wsgi—specifically, Response is modified to track a lazy_serialization attribute (defaulting to True), and ResponseSerializer's serialize_body method is modified to store the actual serializer and action (and template, if available) into the request if lazy_serialization is True. The data itself is serialized as a JSON object in order to be passed through the webob pipeline (webob is designed to pass text response bodies, and extending it to allow raw objects to be passed between components would be difficult).

Extensions
Request extensions are also slightly modified; the loop that calls all the handlers for a particular request deserializes the body and passes that as a third parameter to all the handlers. The handlers can modify the body in place, and additionally modify the template in the wsgi environment (if present; for data requested to be encoded in JSON format, no template will be present in the wsgi environment).

XML Templates
The most complicated part of the proposal is the introduction of an XML template. XML templates can be constructed the same way an XML tree is constructed using the ElementTree interface of lxml; there exists a TemplateElement class and a SubTemplateElement helper function that operate in a similar manner to ElementTree. The critical difference is the use of selectors; when an object is serialized against an XML template, the object is passed in, and a selector (basically, a simple callable) is used to select the data to operate on. There is a selector associated with each element, and selectors can also be attached as attribute values and as the text contents of the element; these selectors further refine the element's selector. Subelements will also be passed the object selected by their parent, and have the chance to further refine selection for their own purposes.

Once an element tree has been constructed, it can be wrapped up in a Template subclass. There are two subclasses available—MasterTemplate and SlaveTemplate. Both can serialize an object against the template directly, but MasterTemplate instances can additionally have SlaveTemplate instances attached which can further extend the serialization; the effect is as if the elements in the SlaveTemplate instances were merged with the elements in the MasterTemplate. Additionally, MasterTemplate instances have version numbers, and automatically exclude SlaveTemplate instances which do not apply to that MasterTemplate version (SlaveTemplate instances have a minimum version and an optional maximum version for this selection criteria). This makes it easy for extensions to modify a serializer to include their extended data.

The existing code also includes a TemplateBuilder class, which can be used to ensure that a given template only ever needs to be built once; and an XMLTemplateSerializer, which is similar to the existing XMLDictSerializer.

WSGI Environment
The lazy serialization potentially adds 3 new variables to the WSGI environment. (Note "potentially"—for JSON serialization, none of these variables are added.) The nova.serializer variable contains a reference to the body serializer to use (typically a subclass of XMLDictSerializer or, preferably, a subclass of XMLTemplateSerializer), and the nova.action variable contains the action to pass to the serializer's serialize method. If the serializer has a get_template method, the return value of that method will be assigned to the nova.template variable; extensions may attach their slave templates to this master template as necessary.

Blueprint: https://blueprints.launchpad.net/nova/+spec/xml-templates