Jump to: navigation, search

Ceilometer/ComplexFilterExpressionsInAPIQueries

< Ceilometer
Revision as of 15:53, 16 January 2014 by Gibi (talk | contribs) (Expression Language)

Architecture

The purpose of the improvement is to support rich query functionality in Ceilometer API. The current query support is limited, it's grammar cannot be extended easily.

REST Resources

The new solution includes new REST resources for provide complex queries for both the meter and alarm related data:

REST Resource Functionality Returned Object Type
'/query/samples' retrieve samples for meter(s) Sample
'/query/alarms' retrieve alarms Alarm
'/query/alarms/history' retrieve alarm history AlarmChange

As new resources were defined, the simple query support remained as it is now, which means that the API is backward compatible and also a new query grammar could be defined without limitations.

POST request

The new solution uses POST request, as the functionality of the query part of the GET request is very limited. The body of the POST request contains the query parameters in JSON format, which makes the processing of the query easier on the API level.

Expression Language

 filter := expression
 expression := simple_expression | complex_expression
 simple_expression := {simple_operator: {field_name: value}}
 simple_operator := = | != | < | <= | > | >= | in
 complex_expression := {complex_operator: [expression, expression, ...]} | not_expression
 not_expression := {not: expression}
 complex_operator := and | or

Where

  • field_name is the name of a field of the Sample, Alarm or AlarmChange object
    • currently directly mapped to db column name
    • validated against the db models
  • value can be int, float, string, or iso date string for timestamp and state_timestamp fields
  • metadata fields of the Sample object can be used in the query with metadata.<field_name> syntax. For example: {"=": {"metadata.display_name" : "server1"}}
    • metadata field names are not validated and therefore filtering on a nonexistent metadata field does not return an error but an empty result instead

Orderby

List of field_name and sort instruction pairs:

orderby: [{"field_name1": "ASC"}, {"field_name2": "DESC"}, {"field_name3": "ASC"}, ...]

Limit

Number of rows returned, defined value should be a positive int.

Supported DB drivers

  • SQLAlchmey
  • MongoDB

Further improvements

  • Expression language operators
    • not
  • Metadata query
  • Statistics
    • Support complex filter functionality for Statistics
    • Separate blueprint
  • Stored query
    • The architecture of this new REST resource provides the possibility to implement stored query support
    • Future blueprint

API example

Filter expression

Check for cpu_util samples reported between 18:00-18:15 or between 18:30 - 18:45 where the utilization is between 23 and 26 percent.

 cpu_util > 23% AND cpu_util < 26% AND ((timestamp < 2013-12-01T18:15:00 AND timestamp > 2013-12-01T18:00:00) OR (timestamp < 2013-12-02T18:45:00 AND timestamp > 2013-12-02T18:30:00))

Pretty printed json:

{"and":
 [{"and":
  [{"=": {"counter_name": "cpu_util"}},
   {">": {"counter_volume": 0.23}},
   {"<": {"counter_volume": 0.26}}]},
 [{"or":
  [{"and":
   [{">": {"timestamp": "2013-12-01T18:00:00"}},
    {"<": {"timestamp": "2013-12-01T18:15:00"}}]},
    {"and":
     [{">": {"timestamp": "2013-12-01T18:30:00"}},
      {"<": {"timestamp": "2013-12-01T18:45:00"}}]}]}]]}

Filter expression in the POST request body:

{
"filter" : "{\"and\":[{\"and\": [{\"=\": {\"counter_name\": \"cpu_util\"}}, {\">\": {\"counter_volume\": 0.23}}, {\"<\": {\"counter_volume\": 0.26}}]}, [{\"or\": [{\"and\": [{\">\": {\"timestamp\": \"2013-12-01T18:00:00\"}}, {\"<\": {\"timestamp\": \"2013-12-01T18:15:00\"}}]}, {\"and\": [{\">\": {\"timestamp\": \"2013-12-01T18:30:00\"}}, {\"<\": {\"timestamp\": \"2013-12-01T18:45:00\"}}]}]}]]}"
}

Samples query

The current implementation supports to retrieve limited number of values and using multiple orderby fields with the possibility of specifying the ordering direction too. An example query can be to request the first 4 samples with the filter specified above, with ordering the result by the values of the counter_volume field with ascending order and then by the timestamp with descending order.

POST request body of the query:

{
"filter" : "{\"and\":[{\"and\": [{\"=\": {\"counter_name\": \"cpu_util\"}}, {\">\": {\"counter_volume\": 0.23}}, {\"<\": {\"counter_volume\": 0.26}}]}, [{\"or\": [{\"and\": [{\">\": {\"timestamp\": \"2013-12-01T18:00:00\"}}, {\"<\": {\"timestamp\": \"2013-12-01T18:15:00\"}}]}, {\"and\": [{\">\": {\"timestamp\": \"2013-12-01T18:30:00\"}}, {\"<\": {\"timestamp\": \"2013-12-01T18:45:00\"}}]}]}]]}",
"orderby" : "[{\"counter_volume\": \"ASC\"}, {\"timestamp\": \"DESC\"}]",
"limit" : 4
}

After saving the request body into a file, e.g: body.txt, the POST request will be the following:

curl -X POST -H 'X-Auth-Token: <inserttokenhere>' -H 'Content-Type: application/json' -d @body.txt \
  http://localhost:8777/v2/query/meters