Ceilometer/blueprints/multi-dimensions

• Launchpad Entry: tbd
• Created: 27 Nov 2012
• Contributors: Nick Barcet& Julien Danjou

Summary

In order to be able to perform smart and fast aggregation in the API, we need to be able to perform queries that would aggregate counters based on additional keypair values than the simple tenant/user/ressource triplet. This spec proposes to handle meta data in a new and different way that would allow to solve this.

Release Note

It is now possible to request aggregated value based on additional "dimensions" for each counters/meters in Ceilometer

User stories

• John would like to retrieve the aggregated number of objects accross all of his containers named xxx
• Peter would like to know his sum cpu usage for all instance type yyyy
• Andrew would like to retrieve max value of cpu usage for a list of instance-ids

Assumptions

• We assume here that the way metadata have been used contains a majority of keypairs at first level in the json, and that this will constitute our future dimensions.

Design

This blueprint assumes 3 changes:

1. Switching from meta-data to dimensions

The proposal is to assume in the current json formated meta data that all first leve keypairs in the form of <string>:<string> are what we need to consider as dimensions.

2. Adding dimension filters to the API

Each meters related API call would have one additional optional parameter which would be a list of Keypairs that need all need to be statistisfied (Logical AND) which would allow to perform agreggated sum, max and min function on the matching records (another blueprint will specify the addition of an average function).

• dimension: keypair possibly expressed in the URL as multiple arguments (ie: &dimension="key1:value1"&dimension="key2:value2"...)

In a future iteration of the feature, another parameter could allow nested conditional expresion mixing logical AND and OR:

• dimensionExpr -- Nested condition on keypairs as ( (("key1"="value1") & ("key2"="valueN")) | (("key1"="value3") & ("key2"="value4")) )

The WSME package supports complex data structures as arguments to GET requests such as these queries using the syntax:

 q.dimensions[0].name=dim1&q.dimensions[0].value=dim1val&q.dimensions[1].name=dim2&q.dimensions[1].value=dim2val

the method being called would receive a "q" argument that would be an instance of a Query class that would look something like:

  class Dimension(Base):
    name = text
    value = text

  class Query(Base):
    dimensions = [Dimension]

3. Implicit dimensions

The existing Source, ResourceID, UserID and TenantID are considered implicit dimensions, which do not need to be specified in the meta-data, but can be queried as such.

Implementation

Strorage design

In the case of a relational database storage engine, the dimensions would be stored in a separate table as follow

+------------+       +------------+
| Dimensions |       | Event      |
+------------+       +------------+
| EventID    |N-----1| EvenID     |
| Key        |       | ResourceID |
| Value      |       | .....      |
+------------+       | .....      |

Code Changes

TBD

Migration

If we decide to modify the structure of the existing meta-data, we need to provide a procedure to translate existing meta-data information to the new format.

Test/Demo Plan

This need not be added or completed until the specification is nearing beta.

Unresolved issues

This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.

BoF agenda and discussion

Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarising what was discussed and note any options that were rejected.

---