EfficientMetering/ArchitectureProposalV1
Metering Architecture Proposal 1
This document is based on EfficientMetering#Architecture
Goals
- Provide efficient collection of metering data, in terms of CPU and network costs.
- Allow deployers to integrate with the metering system directly or by replacing components.
- Allow deployers to configure the type of data collected to meet their operating requirements.
High Level Description
- An agent runs on each OpenStack node ( Bare Metal machine ) and harvests the data localy
- A storage daemon communicates with the agents to collect their data and aggregate them
- The agents collecting data are authenticated to avoid pollution of the metering service
- The data is sent from agents to the storage daemon via a trusted messaging system (RabbitMQ?)
- The data / messages exchanged between agents and the storage daemon use a common messages format
- The content of the storage is made available thru a REST API providing aggregation
- The message queue is separate from other queues (such as the nova queue)
- The messages in queue are signed and non repudiable (http://en.wikipedia.org/wiki/Non-repudiation)
Note: document some use case scenarios to really nail down the architecture. Who signals the metering service? The API service or nova, quantum, swift, glance, volume?
Note: ideally, all meters are available from the OpenStack component responsible for a given resource (for instance the disk I/O for an ephemeral disk is made available in nova). However, it is not realistic to assume it can always be the case. Standalone ceilometer agents runing on OpenStack nodes provide access to the meters when the OpenStack component don't. The meter implemented in ceilometer agents should always be contributed to the OpenStack component. This kind of incubation for each given meter ( first implemented in ceilometer agents and then in the OpenStack component ) is both practical for short term purposes and a sound long term practice that avoids forking code.
