Jump to: navigation, search

Difference between revisions of "Ceilometer/ComplexFilterExpressionsInAPIQueries/ComplexFilterExpressionsInStatistics"

(Possible extensions)
(Possible extensions)
Line 82: Line 82:
 
== Possible extensions ==
 
== Possible extensions ==
 
Request: Return the last sample for each resource for a meter
 
Request: Return the last sample for each resource for a meter
 +
 
Problems:
 
Problems:
 
* The "last" function is not a valid aggregate function as it does not generate a single field for every row in the result. We can fix that by changing the request to "Return the last 'field_name' for each resource for a meter". Where 'field_name' is a field of the sample (e.g. counter_volume). This way last(field_name) becomes a valid aggregate function.  
 
* The "last" function is not a valid aggregate function as it does not generate a single field for every row in the result. We can fix that by changing the request to "Return the last 'field_name' for each resource for a meter". Where 'field_name' is a field of the sample (e.g. counter_volume). This way last(field_name) becomes a valid aggregate function.  
 
* SQL does not provide "last" kind of aggregate out of the box. To implement it we need to issue a subquery select t1.* from samples t1 left outer join samples t2 on t1.resource_id=t2.resource and t1.timestamp<t2.timestamp where t2.resource_id is null . However the speed of this query is really bad for big tables.
 
* SQL does not provide "last" kind of aggregate out of the box. To implement it we need to issue a subquery select t1.* from samples t1 left outer join samples t2 on t1.resource_id=t2.resource and t1.timestamp<t2.timestamp where t2.resource_id is null . However the speed of this query is really bad for big tables.
 +
 +
Generic subquery support to support multiple subsequent group by:
 +
* How to make the API abstract enough to fit for both SQL and NoSQL

Revision as of 16:43, 6 May 2014

Architecture

REST resource

The new solution includes new REST resources for provide complex queries for sample statistics:

REST Resource Functionality Returned Object Type
'/query/samples/statistics' retrieve statistics of samples for meter(s) Statistics
'/query/alarms/statistics' retrieve statistics of alarms AlarmStatistics (New)
'/query/alarms/history/statistics' retrieve statistics of alarm changes AlarmStatistics

AlarmStatistics

Example:

{ "aggregate": {"count": 19.0},
  "duration": 328.478029,
  "duration_start": "2014-01-31T10:00:41.823919",
  "duration_end": "2014-01-31T10:06:10.301948",
  "period": 900,
  "period_start": "2014-01-31T10:00:00",
  "period_end": "2014-01-31T10:15:00",
  "groupby": {"project_id": "061a5c91811e4044b7dc86c6136c4f99"}
}

As new resource is defined, the simple statistics query support remained as it is now, which means that the API is backward compatible and also a new query grammar could be defined without limitations.

Filter Expression Language

The same expression language is used as in Ceilometer/ComplexFilterExpressionsInAPIQueries#Filter_Expression_Language

Supported DB drivers

Planned to be supported:

  • SqlAlchemy
  • MongoDB

API examples

Consider a requirement for determining, for some tenant, the number of distinct instances (cardinality) as well as the total number of instance samples (count). You might also want to see this information with 15 minute long intervals. Then, using current API the query would look like the following:

GET /v2/meters/instance/statistics?aggregate.func=cardinality
                                  &aggregate.param=resource_id
                                  &aggregate.func=count
                                  &groupby=project_id&period=900

With the new API it would look like the following

POST /v2/query/samples/statistics

With the body:

{
filter: {"=": {"counter_name": "instance"}},
aggregates: [{"func": "cardinality", "param": ["resource_id"]}, {"func": "count", "param": []}],
groupby: ["project_id"],
period: 900,
}

You can also limit and order the result with the following extended body:

{
filter: {"=": {"counter_name": "instance"}},
aggregates: [{"func": "cardinality", "param": ["resource_id"]}, {"func": "count", "param": []}],
groupby: ["project_id"],
period: 900,
orderby: [{"count": "ASC"}, {"cardinality/resource_id": "DESC"}],
limit: 10
}

Possible extensions

Request: Return the last sample for each resource for a meter

Problems:

  • The "last" function is not a valid aggregate function as it does not generate a single field for every row in the result. We can fix that by changing the request to "Return the last 'field_name' for each resource for a meter". Where 'field_name' is a field of the sample (e.g. counter_volume). This way last(field_name) becomes a valid aggregate function.
  • SQL does not provide "last" kind of aggregate out of the box. To implement it we need to issue a subquery select t1.* from samples t1 left outer join samples t2 on t1.resource_id=t2.resource and t1.timestamp<t2.timestamp where t2.resource_id is null . However the speed of this query is really bad for big tables.

Generic subquery support to support multiple subsequent group by:

  • How to make the API abstract enough to fit for both SQL and NoSQL