Zaqar/bp/ops-stats

Stats Collector
We need to gather several stats. Some of these will need to be exposed via the API (bp/advanced-queue-stats), but the first step is recording, which is what this bp is all about.

Note: Use a storage driver middleware approach for metering/stats initially (ala Swift)

Here are some possible metrics (starred items are most important).

Per Cluster

 * Storage stats
 * Storage health

Per account

 * Number of queues*
 * Total messages/day
 * Average messages/sec*

Per queue

 * Average messages/sec enqueued*
 * Number of messages*
 * Life of queue (how long since created)
 * Number of producers
 * Number of workers
 * Number of subscribers
 * Average message size
 * Average message TTL
 * Average message lifetime (explicitly deleted)
 * Average claim TTL
 * Average claim lifetime
 * Average number of messages posted per request
 * Average number of messages requested per GET or claim

Operations should be able to aggregate and graph these metrics across acounts.

See also: http://eavesdrop.openstack.org/meetings/marconi/2013/marconi.2013-06-27-19.05.log.html