Difference between revisions of "Ceilometer/blueprints/api-group-by"

Revision as of 00:15, 16 January 2013

Summary

The API should allow to do GROUP BY type operation.

User stories

• I had an instance running for 6h. It started as a m1.tiny flavor during the first 2 hours and then grew up to a m1.large flavor for the next 4 hours. I need to get this two durations so I can bill them with different rates.

Design

Enhance API v2 so it implements a new arguments to do GROUP BY operations.

For example add:

g[]=<field name>

That solves the user story above with:

/v2/meters/instance?
 q[0].field=resource&
 q[0].op=eq&
 q[0].value=<my-resource-id>&
 q[1].field=timestamp&
 q[1].op=lt&
 q[1].value=<now>&
 q[2].field=timestamp&
 q[2].op=gt&
 q[2].value=<now - 6 hours>&
 g[0]=metadata.flavor&
 c[0]=sum&
 period=360

Would return

{[
  { "m1.tiny": 1 },
  { "m1.tiny": 1 },
  { "m1.large": 1 },
  { "m1.large": 1 },
  { "m1.large": 1 },
  { "m1.large": 1 },
]}

NOTES:

• Each return value should include the time range for the accounting systems to record. It may be simpler to just extend the statistics API we have (without the "c" argument), and adding a "g" field with the grouping column names and values so the recipient can figure out what each result means. -- dhellmann
• That's a problem that APIv2 needs to solve, not this blueprint -- jd

Further more, dropping the q[0] request that narrows the search to only one resource allows to retrieve this information for all instances over that period of time:

/v2/meters/instance?
 q[0].field=timestamp&
 q[0].op=lt&
 q[0].value=<now>&
 q[1].field=timestamp&
 q[1].op=gt&
 q[1].value=<now - 6 hours>&
 g[0]=metadata.flavor&
 g[1]=resource&
 c[0]=sum&
 period=360

If there was another large instance, that would return:

{[
  { "m1.tiny": 1, "m1.large": 1 },
  { "m1.tiny": 1, "m1.large": 1 },
  { "m1.large": 2 },
  { "m1.large": 2 },
  { "m1.large": 2 },
  { "m1.large": 2 },
]}

Be careful that specifying a c[] is mandatory in a GROUP BY operation (as in SQL).

Angus's comments/ramblings

Just to update with current impl.

/v2/meters/instance/statistics?
 q[0].field=resource&
 q[0].op=eq&
 q[0].value=<my-resource-id>&
 q[1].field=timestamp&
 q[1].op=lt&
 q[1].value=<now>&
 q[2].field=timestamp&
 q[2].op=gt&
 q[2].value=<now - 6 hours>&
 g[0]=metadata.flavor&
 period=360

1) I assume we can't group by more than one field? If so this should be (not an array):

groupby=metadata.flavor&

2) period is not yet impl. - I'd better get on that ;)

3) Currently we return:

{[
  { "min": 1,
    "max": 1,
    "avg": 1,
    "sum": 1,
    "count": 1,
    "duration": 1,
  },
]}

To show the groupby we could return the following:

{[
  { "min": 1,
    "max": 1,
    "avg": 1,
    "sum": 1,
    "count": 1,
    "duration": 1,
    "groupby": "m1.tiny",
  },
]}

If there is no groupby that can just be None.

4) from an impl. pov (mongo) we have:

   MAP_STATS = bson.code.Code("""
	    function () {
-	        emit('statistics', { min : this.counter_volume,
+	        emit(groupby_field, { min : this.counter_volume,
	                             max : this.counter_volume,
	                             qty : this.counter_volume,
	                             count : 1,
	                             timestamp_min : this.timestamp,
	                             timestamp_max : this.timestamp } )
	    }
	    """)

If we can pass in the groupby field into the above function then this will be super easy. Can we generate this bcode dynamically?