Ceilometer/blueprints/transportation-pipeline-monitoring-metrics

Summary

Compared to metering data, the monitoring metrics need to be collected in a more frequent fashion and the amount of data is way bigger. Does the monitoring metrics necessarily need to go through the message bus as metering data does. Or the option worth being considered is to write them directly to data store.

The current metering pipeline

---
-
    name: meter_pipeline
    interval: 60
    counters:
        - "*"
    transformers:
    publishers:
        - meter_publisher

Alternative for metrics pipeline

---
-
    name: metrics_pipeline
    interval: 5   // more frequent polling might be needed by monitoring
    counters:
        - "*"
    transformers:
    publishers:
        - metrics_publisher     // metrics publisher might directly write to backend storage (avoid MQ)
        - metrics_notifier(publisher)     // inline pre-processing metrics notification