Ceilometer/blueprints/api-v2-improvements

There are a lot of new requests for api features/extensions, so thought it might make sense to put the combined use cases here:

Currently supported

 * 1) List the resources (filter by any field)
 * 2) List the available meters (filter by any field)
 * 3) List the Samples (filter by any field)
 * 4) Get Statistics (filter by any field) [max, min, avg, count, total]

Restrict access based on Requestors tenant
In the v1 API we have "acl.get_limited_to_project(rq.headers)" but this has not yet been ported to the v2 API.

New requested features
Some of the below requests are very high level and need better justification/explaining (IMO).

From https://blueprints.launchpad.net/ceilometer/+spec/api-v2-improvement

 * Give me the maximum usage of a resource that lasted more than 1h
 * Give me the use of a resource over a period of time, listing changes by increment of X volume over a period of Y time
 * Provide additional statistical function (Deviation, Median, Variation, Distribution, Slope, etc...) which could be given as multiple results for a given data set collection

From https://wiki.openstack.org/wiki/Ceilometer/blueprints/api-group-by

 * I had an instance running for 6h. It started as a m1.tiny flavor during the first 2 hours and then grew up to a m1.large flavor for the next 4 hours. I need to get this two durations so I can bill them with different rates.

From https://wiki.openstack.org/wiki/Ceilometer/blueprints/Ceilometer-api-extensions
Usage Reporting is our primary use case for the proposed API extensions. We would like to use an implementation of the Ceilometer API as the front end for our reporting service. The following requests illustrate the need for this functionality:


 * End Customer Usage Reporting
 * Ability to see a summary of my month-to-date (MTD) usage on a Dashboard - this would include one summary per Region with MTD totals for one or more usage metrics for each service
 * Ability to see a graphical and/or tabular summary of my usage per service for a specified month - I can specifiy the interval that the data is displayed at (i.e. per week, day, hour).
 * Ability to further drill down into the usage report to see reports by
 * service characteristics (i.e. compute flavor, application type, operating system type, volume type, ...)
 * location (region and availability zone)
 * instance level (specific instance for the service, i.e. VM, volume, storage container, ...)
 * Internal Provider Reporting
 * Ability to generate a monthly report of service usage (all usage by service plus drill down by service and customer characteristics)
 * Ability to generate licensing related reports (number of nodes, VM, tenants running a specified operating system or application type)

(asalkeld) further on the dashboard idea
So the fedora infrastructure guys use OpenStack and they had this request:
 * instances get created and destroyed fairly quickly
 * so every ~ 10 mins they want a report of what got created and destroyed with in that timeframe
 * also, I think it would be useful to have the resource type recorded so it is easy to seperate out volumes/instances (without having to guess based on the meter names)

As a mockup of the kind of output I was thinking (something like a "top -n1" for OpenStack):

Resources added [type]    [instance]    [owner - user-id] [time created] compute swift ... Resources destroyed [type]    [instance]    [owner - user-id] [time created] [time destroyed] compute volume  

5 Most active Instances [instance]   [owner - user-id] [time created] [cpu_util] [mem_util] [net_util]                 x%          y%            z%                  x%          y%            z%                   x%          y%            z% ...

5 Fullest Volumes [volume]   [owner - user-id] [time created] [util]                     x% ...

5 Most active Users [user-id] [number of resources] 23 ...

To achieve this we need to:
 * 1) record the creation and deletion timestamps
 * 2) record the object type (or service type) this could just be the basename of the get_event_type.
 * 3) implement the groupby feature to query the stats of all instances within the time period grouped by id.