State of Oslo notifications
Currently, Oslo provides code to send notification via the module openstack.common.notifier. Its submodule named openstack..common.notifier.api allows an application to use a Python API to send notifications. Notifications are sent by using a set of drivers included in openstack.common.notifier, such as log or rpc.
The prototype of the main notification function is openstack.common.notifier.api.notify(context, publisher_id, event_type, priority, payload).
Currently the notify() arguments include very few structured information. The argument payload can be anything, and is usually a Python dict containing a various amount of data whose fields cannot be known in advance.
Some of these fields are documented on a wiki page, but it has obviously a lot of drawbacks:
- the list is easily outdated;
- the list is not even complete;
- the list doesn't help any sort of automatic testing around content events.
In Ceilometer, the main consumer of events in OpenStack, we've been dealing for a long time with all of that. Our unit test tries to mimic what is sent by the other OpenStack components by including copies of the payload we've seen at some point. As soon as the payload format is changed in an OpenStack component, our tests will still pass, but the code will not be able to cope with the events in a real deployment.
Having non structured events also implies a lot of problems in storage area. Storing unstructured events relies on storing EAV based data. While this isn't a problem in some database model (NoSQL) this is a terrible issue for others (SQL) and can be a sensible one in both cases in term of performance (e.g. indexing).
The proposed solution is to build a new API call with a different prototype that would relies on Python objects rather than dict. This would solve all of the mentioned problems.
The function would something like openstack.common.notifier.notify(context, priority, event) where event would be a Python object. In such context event could be a simple object, that later could be translated to a simple dict before being sent to a driver that will publisher it over the wire.
# This is peudo code written in the wiki, I didn't run it class Event(object): user_id = str project_id = str
def __init__(self, **kwargs): fields = set(filter(lambda x: not x.startswith('_'), self.__dict__.keys())) if set(kwargs.keys()) != fields: raise Exception("Too much or missing fields given in this notification") # should also indicate what for attr in fields: setattr(self, getattr(self.__class__, attr)(kwargs[attr])) def as_dict(self): return self.__dict__ # + filter out private fields, or something like that @property def event(self): """Return the event type by converting the class name from CamelCase to dotted notation.""" return re.sub('([A-Z]+)', r'.\1',self.__class__.__name__).lower().strip('.') class ComputeInstanceCreateEvent(Event): instance_id = str instance_type_id = str display_name = str created_at = datetime.datetime launchad_at = datetime.datetime image_ref_url = str state = str memory_mb = int
def notify(context, priority, event): get_driver().notify(context, priority, event.type, event.as_dict())
It's also possible for an application to introspect easily the event list and their fields by introspecting the module. This could be leveraged to automatically construct SQL schema for example, or validation code.
Using Python classes will also make sure that if any part of the code (e.g. event definition) is updated in Oslo, updating Oslo in e.g. Nova will break the code in Nova and the unit testing will show that easily enough so it'll be fixed. The same will go for consumer like Ceilometer, that will have to adapt to deal with the new events.
Specyfing type for attributes is useful to:
- Validate that the data given are good
- Build correct storage schema through possible introspection
This is heavily inspired by the approach that WSME has taken to provides automatically generated REST API.