Jump to: navigation, search

Difference between revisions of "NotificationsAndCADF"

m
 
(4 intermediate revisions by the same user not shown)
Line 1: Line 1:
Notifications and CADF
 
 
 
The Problem
 
The Problem
  
Line 31: Line 29:
 
# Versioning built in
 
# Versioning built in
 
# Domain-specific traits can be stored in "Attachments" to core spec.  
 
# Domain-specific traits can be stored in "Attachments" to core spec.  
 +
# Someone else will likely write/maintain the transformation engine
  
  
Line 55: Line 54:
 
           }
 
           }
 
     }
 
     }
 +
  
 
we can easily define a schema for this that states:
 
we can easily define a schema for this that states:
Line 61: Line 61:
 
# the required and optional traits
 
# the required and optional traits
 
# the public/private status of each trait (should the data be redacted for public consumption?)
 
# the public/private status of each trait (should the data be redacted for public consumption?)
 +
  
 
We will need to add a trait to identify the version.  
 
We will need to add a trait to identify the version.  
Line 87: Line 88:
 
# Where to get the necessary values from the source notification (per version)
 
# Where to get the necessary values from the source notification (per version)
 
# Any transformations necessary from the source to destination traits (something like stack-distiller can do this easily).
 
# Any transformations necessary from the source to destination traits (something like stack-distiller can do this easily).
 +
  
 
Ideally this same transformation grammar can be used to move from version to version:
 
Ideally this same transformation grammar can be used to move from version to version:
Line 119: Line 121:
 
# the addition of the "why" field, giving acceptable defaults
 
# the addition of the "why" field, giving acceptable defaults
 
# the removal of the "to_who" field
 
# the removal of the "to_who" field
 +
 +
 +
This is much the same way db migrations occur today. Sadly, this is likely to be some sort of "xslt for json" thing.
 +
 +
http://stackoverflow.com/questions/1618038/xslt-equivalent-for-json

Latest revision as of 14:00, 4 September 2014

The Problem

  1. No versioning of notifications
  2. No schema definition around notifications


The Options

  1. Use an off-the-shelf schema grammar like json-schema (or a python wrapper around such libraries, like voluptuous)
  2. Adopt a standard, like CADF and change the existing message format.
  3. Hybrid approach. Off the shelf schema with "transformation rules" for converting to standards format.


Advantages of using an off-the-shelf schema grammar

  1. Does not break existing notifications, it simply enforces what we have today.
  2. Easy to implement


Disadvantages of using an off-the-shelf schema grammar

  1. Yet-another-ad-hoc-schema. No basis in standards or cross-domain compatibility.
  2. No inherent support for versioning or transformations between versions. We would have layer that on.


Advantages of adopting a standard (like CADF)

  1. Cross-domain audit trail. Suitable for enterprise.
  2. Versioning built in
  3. Domain-specific traits can be stored in "Attachments" to core spec.
  4. Someone else will likely write/maintain the transformation engine


Disadvantages to adopting a standard

  1. Breaks existing notifications
  2. Transformation between versions not supported, we would have to layer that on.
  3. We would still need to define a schema for the "attachment" data ... all the same adv/dis as off-the-shelf apply.


The Hybrid Model

In the hybrid model, we would follow the off-the-shelf approach, but define a separate set of transformation rules for converting traits of an event from one version to another or from one schema to another. We will likely need these transformation rules anyway for version conversion, so they should be applicable to schema conversion as well.

For example, say we have an existing notification in this format:

   { 'timestamp': 'some iso1801 datetime stamp',
     'event_type': 'compute.instance.foo',
     'payload': {
               'who': 123,
               'where': 'this data center',
               'to_where': 'that data center',
               'to_who': 567,
          }
   }


we can easily define a schema for this that states:

  1. the data types for each trait
  2. the required and optional traits
  3. the public/private status of each trait (should the data be redacted for public consumption?)


We will need to add a trait to identify the version.

A CADF notification might have the following format

   {  'typeURI': 'http://schemas.dmtf.org/cloud/audit/1.0/event', 
       'id': 'openstack:a80dc5ee-be83-48ad-ad5e-6577f2217637‘ 
       'eventType': 'activity', 
       'action': 'read/list',    
       'outcome': 'success', 
       'eventTime': '2014-01-17T23:23:38.109989+0000', 
       'initiator': { 
             'id': 'openstack:95f12d248a234a969f456cd2c794f29a' 
             'typeURI': 'service/security/account/user', 
        }, 
       'target': { 
            'id': 'openstack:0f126160203748a5b4923f2eb6e3b7db', 
            'typeURI': ‘service/compute/servers', 
        }, 
       'observer': { 'id': 'target‘ }, 
   } 

We would need to create a template for this notification (perhaps a jinja2 template) and a transformation grammar which states:

  1. Where to get the necessary values from the source notification (per version)
  2. Any transformations necessary from the source to destination traits (something like stack-distiller can do this easily).


Ideally this same transformation grammar can be used to move from version to version:

Version 1

   { 'timestamp': 'some iso1801 datetime stamp',
     'event_type': 'compute.instance.foo',
     'version': 1,
     'payload': {
               'who': 123,
               'where': 'this data center',
               'to_where': 'that data center',
               'to_who': 567,
          }
   }

Version 2

   { 'timestamp': 'some iso1801 datetime stamp',
     'event_type': 'compute.instance.foo',
     'version': 2,
     'payload': {
               'who': "string name of who",
               'where': 'this data center',
               'to_where': 'that data center',
               'why': "need this"
          }
   }

The transformation rules would have a template for the new version 2 format and describe:

  1. the change in the "who" field (to string)
  2. the addition of the "why" field, giving acceptable defaults
  3. the removal of the "to_who" field


This is much the same way db migrations occur today. Sadly, this is likely to be some sort of "xslt for json" thing.

http://stackoverflow.com/questions/1618038/xslt-equivalent-for-json