Jump to: navigation, search


< Ceilometer‎ | blueprints
Revision as of 16:16, 17 June 2014 by PhilNeal (talk | contribs) (Event Format)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)


Up until recently the focus of OpenStack has been on infrastructure level services (compute, storage, networking), but there are a large number of PaaS services that are currently in development and also a growing number of applications running on OpenStack systems. Many of these new applications will need to be metered for a variety of reasons: internal systems management, license management, end customer billing. Ceilometer currently supports collection of metering information for infrastructure level services. The general mechanism has been to either create a targeted agent or use the existing service APIs to meter these systems. While this approach has worked for the existing set of services, it will run into a several problems as the number of PaaS and SaaS services that need to be metered expand. The main issue is that it will not be viable to do custom integrations for the potentially hundreds (and eventually thousands) of services that may be hosted in OpenStack. This blueprint provides a proposal for standardizing the format of integrations so that Ceilometer can consume metering data from any conforming application without doing any custom integration or even a code change.

Example Use Cases

Database as a Service

Database as a Service (DBaaS), i.e. Trove, has an architecture where a service controller manages special Nova compute instances on behalf of the customer. From their perspective they are running instances of a database, not compute instances. Metering and billing will be based on hours that the database has been running, not necessarily how long the hosting instance has been run. This requires a unique set of metering records to be generated and stored to enable usage tracking and billing of the individual database instances.

DNS as a Service

DNS as a service runs on a similar service controller architecture to DBaaS. In this case, the meters that need to be measured are the existence of a DNS zone and the number of DNS queries that have been served. Once again these are application level meters that do not correlate directly to the host instances that are running. Queries could be served by a variety of hosts and a DNS zone's existence does not depend on a specific compute instance. Application level metrics must be tracked by the DNS service and reported out so that these systems can be tracked.

Load Balancing as a Service

A load balancer is a logical system that consists of some number of compute instances that host the load balancing software. Meters that must be measured would be things like the existence of a load balancer instance and the amount of data that is being transferred through the system. Once again, this does not directly correlate to the underlying infrastructure but must be reported at the application level.


The previously listed uses cases show that PaaS and other application level metering have their own metering requirements that are similar to but distinct from the infrastructure level components such as compute. Each application or service may have its own unique meters that must be tracked and processed. Since there are a large number of potential services/applications, it will be difficult (if not impossible) to support these without some kind of standard integration model. A full integration model includes event format, field and metadata standards and delivery mechanism(s) required. This blueprint only addresses the event format. The following are the requirements for an event format to be able to generally support the service/application level metering:

  1. Uses a standard format that can easily be generated and consumed using commonly available libraries.
  2. Consistent structure of usage records to facilitate easy parsing
  3. Standardized field types and names, allows consumers of information to understand what each field refers to and facilitates processing of data.
  4. Support for reporting of the full range of meter types along with supporting metadata required to process them.

Event Format

PaaS Events and Usage Records will share the same format. The following table provides details of the fields that will be contained in the records. The format was derived from service manager events that are already generated by Infrastructure level services such as Nova and Cinder, so the overall format and any fields that overlap with existing events are essentially the same. Some additional metadata has been added to help with the processing of the events. The goal is to make the events as self-describing as possible and ensure there is no ambiguity about the data or its source.

Field Sub-field Sub-field2 Type Description Required Notes
event_type enumeration for event type records, this describes the actual event that occurred yes for events depends on service, defaults to create, exists, deletetimestamp
timestamp DateTime timestamp for record yes
message_id String unique identifier for event yes
version String version of event format yes
audit_period_beginning DateTime timestamp of reporting period start yes
audit_period_ending DateTime timestamp of reporting period end yes
record_type enumeration [event,quantity] events describe some kind of state change in the service, quantity describes a usage metric value yes
tenant_id String identifies the owner of the service instance yes
user_id String identifies specific user no
service_id UUID uniquely identifies a service yes
instance_id UUID uniquely identifies an instance of the service yes assuming instance level reporting
display_name String text description of the service no used if customer names instances
instance_type_id enumeration used to describe variations of a service yes needed if variations of service have different prices or need to be broken out separately on the invoice
instance_type String text description of service variations no
availability zone String where the service is deployed no required if service is deployed at an AZ level
region String data center that the service is deployed in no required if service is deployed at a regional level
state enumeration status of the service at the time of record generation no required for existence events
state_description String text description of the state of service no
license_code enumeration value that describes a specific license model no
metrics array
metric_name String unique name for the metric that is represented in this record yes
metric_type enumeration gauge, cumulative, delta yes describes the behavior of the metric, from ceilometer
metric_value Float value of metric for quantity type records yes for quantities
metric_units enumeration describes the units for the quantity yes


The event format listed above can be used to deliver two basic types of events: quantities and state events

State Events

These events describe the state of the metered service. They are very similar to the existing state events generated by Infrastructure. Generally there would be at least three types of events: create, exists and delete. Examples of these events for a DNS service are listed below.


This event is sent after a zone has been created


"event_type":"dns.zone.create", "time_stamp":"2013-04-07 22:56:30.026191", "message_id": 52232791371,

"payload": { "instance_type":"type1", "availability_zone" :"az1", "instance_id" :"6accc078-81de-4567-894f-53af5653ac63", "audit_period_beginning":"2013-04-07 21:56:32.249876",  "state":"active",  "audit_period_ending":"2013-04-07 22:56:32.249712", "service_id":"1abbb078-81cd-4758-974e-35fa5653ac63",        "version":"1.0",  "tenant_id":"12345", "instance_type_id": 1, "display_name":"example100.com", "message_id": 52232791371, "user_id":"6789", "state_description":"happy DNS" }


This event is sent every hour for existing zones


"event_type":"dns.zone.exists", "time_stamp":"2013-04-07 22:56:37.782573", "message_id": 52232791372,

"payload": { "instance_type":"type1", "availability_zone" :"az1", "instance_id" :"6accc078-81de-4567-894f-53af5653ac63", "audit_period_beginning":"2013-04-07 21:56:37.783215", "state":"active", "audit_period_ending":"2013-04-07 22:56:37.783153", "service_id":"1abbb078-81cd-4758-974e-35fa5653ac63",  "version":"1.0",  "tenant_id":"12345", "instance_type_id": 1, "display_name":"example100.com", "message_id": 52232791371, "user_id":"6789",   "state_description":"happy DNS" }


This event is sent when a zone is deleted


"event_type":"dns.zone.delete", "time_stamp":"2013-04-07 22:56:37.787774", "message_id": 52232791373,

"payload": {"instance_type":"type1", "availability_zone" :"az1",  "instance_id" :"6accc078-81de-4567-894f-53af5653ac63",  "audit_period_beginning":"2013-04-07 21:56:37.788177",  "state":"active",        "audit_period_ending":"2013-04-07 22:56:37.788144", "service_id":"1abbb078-81cd-4758-974e-35fa5653ac63", "version":"1.0", "tenant_id":"12345", "instance_type_id": 1, "display_name":"example100.com",    "message_id": 52232791371, "user_id":"6789", "state_description":"happy DNS" }


Quantity Events

Quantity events have the same overall format, but additionally have a section called metrics which is an array of information about the meters that the event is reporting on. Each metric entry has a type, units, name and value. Since it is an array, multiple values can be reported in one event.


Hourly event sent with usage for each zone instance


"event_type":"dns.zone.usage", "time_stamp":"2013-04-08 10:05:31.618074", "message_id": 52232791371,

"payload": {"metrics": { "metric_type":"delta", "metric_value": 42,  "metric_units":"hits", "metric_name":"queries" }, "instance_type":"type1", "availability_zone" :"az1", "instance_id" :"6accc078-81de-4567-894f-53af5653ac63", "audit_period_beginning":"2013-04-08 09:05:31.618204", "state":"active", "audit_period_ending":"2013-04-08 10:05:31.618191",  "service_id":"1abbb078-81cd-4758-974e-35fa5653ac63",       "version":"1.0", "tenant_id":"12345", "instance_type_id": 1, "display_name":"example100.com", "message_id": 52232791371, "user_id":"6789", "state_description":"happy DNS"    }


© Copyright 2013 Hewlett-Packard Development Company, L.P.