Ceilometer/blueprints/publisher-counters-frequency


 * Launchpad Entry: CeilometerSpec:publisher-counters-frequency
 * Created: 05 Jan 2013
 * Contributors: yunhong jiang

Summary
Different publishers may have different interval requirement for counters. For example, meter may require 10 minutes interval while monitor require 1 minute interval.

Release Note
This will enable the different frequency for multiple publishers environment.

Design
This is just one possible design for this feature (keep that in mind).

1. The pollster will check all pipelines through pipeline manager for interval requirements for all the counters that the pollster will emit. If no interval requirement, then a default value will be assumed. The pollster will decide the polling interval based on the GCD of interval requirement for all of it's counters.

2. Instead of using periodic task, the compute/central service will create a timer for each pollster based on it's polling interval. Please notice that this in fact requires the pipeline be thread safe. It should be OK for transformer, since the transformer should be CPU workload. However, publishers possibly has I/O and possibly have issues in thread situation. But in green-thread situation, it should be OK.

3. A interval transformer provided to support different interval. Considering followed counters sequence as "c0, c1, c2, c3 ..." for resource_1 and c3 happens "interval time" after c0. The transformer will publish c0, c3, while discard c1, c2. The transformer will keep information for each resource_id for each counter_name. Special consideration needed to handle jitters and counter lost.

Implementation
This section should describe a plan of action (the "how") to implement the changes discussed. Could include subsections like:

UI Changes
The UI changes including the pipeline.yaml config file and also the parameters to the tranformer.

Several options exists:

1) The interval specified followed the counter name:

--- -    counters: - counter_1:interval_1 - counter_2:interval_2 transformers: -       name: Transformer_1 parameters: p1: value publishers: - publisher_1 - publisher_2

-   counters: - counter_3:interval_3 - counter_4:interval_4 transformers: -       name: Transformer_2 parameters: p1: value publishers: - publisher_1 - publisher_2

2) An interval transformer specified explicitly, with the counter name and corresponding interval as parameters.

--- -    counters: - counter_1 - counter_2 transformers: -       name: Transformer_1 parameters: p1: value -       name: Interval_transformer parameters: counter_1:interval_1 counter_2:interval_2 publishers: - publisher_1 - publisher_2

-   counters: - counter_3 - counter_4 transformers: -       name: Transformer_2 parameters: p1: value -       name: Interval_transformer parameters: counter_3:interval_3 counter_4:interval_4

publishers: - publisher_1 - publisher_2

Code Changes
If we use option 1) for UI changes, an interval transformer needed, and pipeline manager will decode the config, create the interval transformer internally, and attach it to the head of the pipeline's transformer list. In fact, in such situation, we can also merge the interval transformer logic into pipeline itself, but that's not clean IMHO.

If we use option 2), then only an interval transformer needed. Also in such situation, the interval_transformer need a special API to consult the interval time for a counter, since the pipeline manager will have no idea of it.

Personally, I prefer to option 2, since that will be more flexible. For example, an special extended interval transformer can consult some config option to decide the interval, or check the hyper-visor type to decide the interval.

Migration
Include:
 * data migration, if any
 * redirects from old URLs to new ones, if any
 * how users will be pointed to the new way of doing things, if necessary.

Test/Demo Plan
This need not be added or completed until the specification is nearing beta.

Unresolved issues
This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.

BoF agenda and discussion
Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarising what was discussed and note any options that were rejected.