Ceilometer/blueprints/support-standard-audit-formats
Contents
Provide support for auditing events in standardized formats
From the onset, Ceilometer’s goal was:
”to become the infrastructure to collect measurements within OpenStack” with its “primary targets [being] are monitoring and metering”. However it was noted that its framework “should be easily expandable to collect for other needs. To that effect, Ceilometer should be able to share collected data with a variety of consumers”.
It is clear that core project developers appreciate the strengths of Ceilometer in having a reliable, core centralized service with the ability to track usage information towards statistical usage analysis and billing. It seems that many of these same projects are seeing similar “auditing” requirements but now with the emphasis on tracking access to services (by users and other services) for the purposes of security auditing. Leveraging Ceilometer’s design to enable different types of event auditing standards (and to provide for different purposes and views on events) seems to make good sense.
Proposal Background
In the auditing and compliance space, standards are being developed for “Cloud Audit” that are designed to align with international and industry requirements yet address “clouds” unique deployment and service models. These standards recognize the challenge of auditing workloads in all types of cloud deployments while providing the data required for regulatory compliance (e.g. PCI-DSS, SoX, ISO 27017, etc.), as well as for proving adherence to localized customer security policies. One of these standards is the Distributed Mgmt. Task Force’s (DMTF) “Cloud Audit” standard which provides a standardized format (in JSON) and a RESTful query syntax.
DMTF Cloud Audit Standard Resources
The last public draft can be found here: http://dmtf.org/sites/default/files/standards/documents/DSP0262_1.0.0a_0.pdf. A “final” v1.0 working draft is due out May 2013 with hopes of being confirmed as committee spec. by end of June. The specification is designed to address a wide range of auditing use cases as published here: http://www.dmtf.org/sites/default/files/standards/documents/DSP2028_1.0.0a.pdf
Proposal
This proposal seeks to leverage the design (agents/meters, etc.) and logging facilities already in Ceilometer and enable users to configure additional meters to allow event data to be produced in one or more (standard) formats. The configuration may also provide the location as to where the logs are created for each meter type (e.g. DMTF format at …/audit/dmtf/xxx).
We would like to add support for auditing APIs access using the DMTF format as a reference implementation as it provides normative data for the key “auditor questions” needed for any compliance regime known in compliance circles as the “7 Ws” and also provides guidance on classifying data by resource, correlation of events (across transactions and service layers) and guidance for federation/merging with other logs (e.g. use UUIDs, normalized timestamps). The format also supports optionally providing geolocation information (per ISO 27017 standards) and Metric/Measurement information that aligns with the developing NIST cloud model/standard.
Below is a sample of a representative DMTF format mappings to show it provides data similar to that in Ceilometer today, but also know that it provides for extended fields/structured data for many of the target regulatory standards mentioned standards above. In this (Cinder, volume) example, since a “metric” and “measurement” are carried these fields appear in the DMTF format.
