Difference between revisions of "NotificationsAndCADF"
SandyWalsh (talk | contribs) (Created page with "Notifications and CADF The Problem # No versioning of notifications # No schema definition around notifications The Options # Use an off-the-shelf schema grammar like json...") |
SandyWalsh (talk | contribs) m |
||
(5 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
− | |||
− | |||
The Problem | The Problem | ||
# No versioning of notifications | # No versioning of notifications | ||
# No schema definition around notifications | # No schema definition around notifications | ||
+ | |||
The Options | The Options | ||
Line 11: | Line 10: | ||
# Adopt a standard, like CADF and change the existing message format. | # Adopt a standard, like CADF and change the existing message format. | ||
# Hybrid approach. Off the shelf schema with "transformation rules" for converting to standards format. | # Hybrid approach. Off the shelf schema with "transformation rules" for converting to standards format. | ||
+ | |||
Advantages of using an off-the-shelf schema grammar | Advantages of using an off-the-shelf schema grammar | ||
Line 16: | Line 16: | ||
# Does not break existing notifications, it simply enforces what we have today. | # Does not break existing notifications, it simply enforces what we have today. | ||
# Easy to implement | # Easy to implement | ||
+ | |||
Disadvantages of using an off-the-shelf schema grammar | Disadvantages of using an off-the-shelf schema grammar | ||
Line 21: | Line 22: | ||
# Yet-another-ad-hoc-schema. No basis in standards or cross-domain compatibility. | # Yet-another-ad-hoc-schema. No basis in standards or cross-domain compatibility. | ||
# No inherent support for versioning or transformations between versions. We would have layer that on. | # No inherent support for versioning or transformations between versions. We would have layer that on. | ||
+ | |||
Advantages of adopting a standard (like CADF) | Advantages of adopting a standard (like CADF) | ||
Line 27: | Line 29: | ||
# Versioning built in | # Versioning built in | ||
# Domain-specific traits can be stored in "Attachments" to core spec. | # Domain-specific traits can be stored in "Attachments" to core spec. | ||
+ | # Someone else will likely write/maintain the transformation engine | ||
+ | |||
Disadvantages to adopting a standard | Disadvantages to adopting a standard | ||
Line 33: | Line 37: | ||
# Transformation between versions not supported, we would have to layer that on. | # Transformation between versions not supported, we would have to layer that on. | ||
# We would still need to define a schema for the "attachment" data ... all the same adv/dis as off-the-shelf apply. | # We would still need to define a schema for the "attachment" data ... all the same adv/dis as off-the-shelf apply. | ||
+ | |||
The Hybrid Model | The Hybrid Model | ||
Line 49: | Line 54: | ||
} | } | ||
} | } | ||
+ | |||
we can easily define a schema for this that states: | we can easily define a schema for this that states: | ||
Line 55: | Line 61: | ||
# the required and optional traits | # the required and optional traits | ||
# the public/private status of each trait (should the data be redacted for public consumption?) | # the public/private status of each trait (should the data be redacted for public consumption?) | ||
+ | |||
We will need to add a trait to identify the version. | We will need to add a trait to identify the version. | ||
Line 81: | Line 88: | ||
# Where to get the necessary values from the source notification (per version) | # Where to get the necessary values from the source notification (per version) | ||
# Any transformations necessary from the source to destination traits (something like stack-distiller can do this easily). | # Any transformations necessary from the source to destination traits (something like stack-distiller can do this easily). | ||
+ | |||
Ideally this same transformation grammar can be used to move from version to version: | Ideally this same transformation grammar can be used to move from version to version: | ||
Line 113: | Line 121: | ||
# the addition of the "why" field, giving acceptable defaults | # the addition of the "why" field, giving acceptable defaults | ||
# the removal of the "to_who" field | # the removal of the "to_who" field | ||
+ | |||
+ | |||
+ | This is much the same way db migrations occur today. Sadly, this is likely to be some sort of "xslt for json" thing. | ||
+ | |||
+ | http://stackoverflow.com/questions/1618038/xslt-equivalent-for-json |
Latest revision as of 14:00, 4 September 2014
The Problem
- No versioning of notifications
- No schema definition around notifications
The Options
- Use an off-the-shelf schema grammar like json-schema (or a python wrapper around such libraries, like voluptuous)
- Adopt a standard, like CADF and change the existing message format.
- Hybrid approach. Off the shelf schema with "transformation rules" for converting to standards format.
Advantages of using an off-the-shelf schema grammar
- Does not break existing notifications, it simply enforces what we have today.
- Easy to implement
Disadvantages of using an off-the-shelf schema grammar
- Yet-another-ad-hoc-schema. No basis in standards or cross-domain compatibility.
- No inherent support for versioning or transformations between versions. We would have layer that on.
Advantages of adopting a standard (like CADF)
- Cross-domain audit trail. Suitable for enterprise.
- Versioning built in
- Domain-specific traits can be stored in "Attachments" to core spec.
- Someone else will likely write/maintain the transformation engine
Disadvantages to adopting a standard
- Breaks existing notifications
- Transformation between versions not supported, we would have to layer that on.
- We would still need to define a schema for the "attachment" data ... all the same adv/dis as off-the-shelf apply.
The Hybrid Model
In the hybrid model, we would follow the off-the-shelf approach, but define a separate set of transformation rules for converting traits of an event from one version to another or from one schema to another. We will likely need these transformation rules anyway for version conversion, so they should be applicable to schema conversion as well.
For example, say we have an existing notification in this format:
{ 'timestamp': 'some iso1801 datetime stamp', 'event_type': 'compute.instance.foo', 'payload': { 'who': 123, 'where': 'this data center', 'to_where': 'that data center', 'to_who': 567, } }
we can easily define a schema for this that states:
- the data types for each trait
- the required and optional traits
- the public/private status of each trait (should the data be redacted for public consumption?)
We will need to add a trait to identify the version.
A CADF notification might have the following format
{ 'typeURI': 'http://schemas.dmtf.org/cloud/audit/1.0/event', 'id': 'openstack:a80dc5ee-be83-48ad-ad5e-6577f2217637‘ 'eventType': 'activity', 'action': 'read/list', 'outcome': 'success', 'eventTime': '2014-01-17T23:23:38.109989+0000', 'initiator': { 'id': 'openstack:95f12d248a234a969f456cd2c794f29a' 'typeURI': 'service/security/account/user', }, 'target': { 'id': 'openstack:0f126160203748a5b4923f2eb6e3b7db', 'typeURI': ‘service/compute/servers', }, 'observer': { 'id': 'target‘ }, }
We would need to create a template for this notification (perhaps a jinja2 template) and a transformation grammar which states:
- Where to get the necessary values from the source notification (per version)
- Any transformations necessary from the source to destination traits (something like stack-distiller can do this easily).
Ideally this same transformation grammar can be used to move from version to version:
Version 1
{ 'timestamp': 'some iso1801 datetime stamp', 'event_type': 'compute.instance.foo', 'version': 1, 'payload': { 'who': 123, 'where': 'this data center', 'to_where': 'that data center', 'to_who': 567, } }
Version 2
{ 'timestamp': 'some iso1801 datetime stamp', 'event_type': 'compute.instance.foo', 'version': 2, 'payload': { 'who': "string name of who", 'where': 'this data center', 'to_where': 'that data center', 'why': "need this" } }
The transformation rules would have a template for the new version 2 format and describe:
- the change in the "who" field (to string)
- the addition of the "why" field, giving acceptable defaults
- the removal of the "to_who" field
This is much the same way db migrations occur today. Sadly, this is likely to be some sort of "xslt for json" thing.
http://stackoverflow.com/questions/1618038/xslt-equivalent-for-json