MagnetoDB/specs/notification

= Summary = MagnetoDB administrators want data on user usage for monitoring, capacity planning, billing, and chargeback purposes. This specification defines the notifications that can be used to generate events and aggregate metrics data.

= User Stories = The use case this blueprint tries to address: MagnetoDB administrators want to see user usage data including table creation/delete, data item put/delete/update/retrieval, aggregated into a graph of the total amount of reads, writes, and storage used by all services broken down by domain and project, in order to determine service health, project capacity and billing/chargeback needs.

Initially we will focus on notifications of table/data item CRUD activities, and the notification mechanism. The monitoring tools and metering metrics will be deferred to later, possibly integrating with Ceilometer.

= Design and Implementation = MagnetoDB system usage data is emitted in the form of usage events from using Oslo Messaging's Notifier component. Notifier is an Oslo incubator project right now. We will use Oslo Notifier to emit events through logging, or send them to a series of AMQP queues (one queue per notification priority), such as table and data item CRUD events. Different types of usage events are distinguished via the notifications' event_type, which is a hierarchical dotted string such as magnetodb.table.create. This way user usage data can be easily grouped for aggregation.

Notifications can be delivered immediately or periodically: Event is created when a specific increment of usage occurs (such as creation of an table) and notification is sent to notifier immediately Notification is generated by a periodic task, like a cron job, covering usage data for a certain period of time.
 * immediate notification
 * periodic notification

Event Attributes
All MagnetoDB events contain the following attributes:
 * priority
 * timestamp
 * event_type
 * tenant_id

The tenant_id is the same as the tenant_id concept in Keystone.

Beside the standard MagnetoDB event attributes, usage events contain a payload of data that will vary depending on the event_type. This is presented as a json-formatted hash of key-value pairs. Some of the keys, such as tenant_id will always be present in any event, others will be data relevant to that event_type (For example, table related notifications will contain data describing the table.)

MagnetoDB events follow the format below:

{'message_id': str(uuid.uuid4), 'publisher_id': 'magnetodb.host1', 'timestamp': timeutils.utcnow, 'priority': 'INFO', 'event_type': 'magnetodb.table.create', 'tenant_id': user_default_tenant, 'payload': {'schema': ... }}.

Note that MagnetoDB itself should not be concerned with billing/charge-backs, but it needs to present a feed of usage data that an external application could use to aggregate the billing data for a requested time period.

Also note that user usage data is for the local MagnetoDB server only, and that data must be aggregated across all MagnetoDB servers to have a full view of the cluster wide usage data.

Event Types
For a simple and short activity, events are only generated when the activity completes or when error happens. For a lengthy or asynchronous activity, there are two events generated: an event when the activity starts, and when the activity ends. The events are in the format of: magnetodb.table. .start' and 'magnetodb.table. .end'. If error happens, instead of .end event, a .error event will be emitted.

Table Events
{'message_id': str(uuid.uuid4), 'publisher_id': 'magnetodb.host1', 'timestamp': timeutils.utcnow, 'priority': 'INFO', 'event_type': 'magnetodb.table.create', 'tenant_id': user_default_tenant, 'payload': {'schema': ... }}.
 * magnetodb.table.create.{start,error,end}:

Notification upon creation of a new user table. tenant_id: Tenant ID that owns the this user table (string) user_id: User ID that owns this user table (string) table_name: user table name (string) table_schema: user table schema (string) state: user table status. (string, such as 'active' or 'deleting', 'creating', 'not_exist') disk_gb: disk allocation for this user table (in gb) message: High-level message describing the notification. If the event type is "create.error", it will contain error details.


 * magnetodb.table.delete.start/.end:

Notification upon deletion of a user table. tenant_id: Tenant ID that owns the this user table (string) user_id: User ID that owns this user table (string) table_name: user table name (string) table_schema: user table schema (string) state: user table status. (string, such as 'active' or 'deleting', 'creating', 'not_exist') disk_gb: disk allocation for this user table (in gb) message: High-level message describing the notification. If the event type is "delete.error", it will contain error details.


 * magnetodb.table.usage:

There is no .start/.end event for this activity ... just the 'magnetodb.table.usage' event.

Periodic usage notification generated by the magnetodb-table-usage-audit cron job. These usages are generated for each user table (including secondary indexes) that was active during the specified audit period. tenant_id: Tenant ID that owns the the user table (string) user_id: User ID that owns the user table (string) table_name: user table name (string) table_schema: user table schema (string) state: user table status. (string, such as 'active' or 'deleting', 'creating', 'not_exist') disk_gb: disk allocation for the user table (in gb) message: High-level message describing the notification. If the event type is "usage.error", it will contain error details.

This notification is disabled by default.

Data Item Events
The data item events are generated for data item CRUD operations. They have all common attributes of standard MagnetoDB events. In addition, details of read/write operations, such as number of rows impacted, consistency level used, will be in the payload.


 * magnetodb.data.getitem
 * magnetodb.data.putitem
 * magnetodb.data.updateitem
 * magnetodb.data.deleteitem
 * magnetodb.data.query,start/.end
 * magnetodb.data.scan,start/.end

Notification Drivers
Different notification drivers can be plugged in to define different notifiers. By default, the Log notifier will be used. Some pre-defined notifiers include: Log notifier uses MagnetoDB's default logging system RPC notifier uses AMQP (rabbitmq) queues to send events. RPC notifier uses AMQP (rabbitmq) queues to send events. The only difference between the above one and this one is the message envelop. No-OP notifier doesn't do anything Test notifier stores notification events in memory. This is convenient for unit test purpose.
 * Log
 * AMQP queues/RPC
 * AMQP queues/RPC with message envelop
 * No-OP
 * Test