Jump to: navigation, search

Extend volume notifications to include usage statistics

Summary

Nova administrators want data on nova volume usage(number of reads, bytes read, number of writes, bytes written) for billing, chargeback, or monitoring purposes.

User Stories

As a systems integrator, I need to retrieve volume usage data so that I can properly bill my customers.

Design

1. Add database table to store volume usage statistics (volume_usage_cache).

2. Add function to virt driver to get all volume usage statistics given a list of volumes (raise Not implemented).

3. Add periodic task to compute manager that calls get_all_volume_usage and write statistics to volume_usage_cache table (_poll_volume_usage)

4. Add usage notification in attach/detach in volume/manager.py to write usage statistics to volume usage table.

5. Extend volume-usage-auditor to read statistics from volume_usage_cache table and generate notification.

  • This volume usage design will follow the same model as currently implemented network usage model

Thinks to consider when using libvirt and domblkstat:

  • On an instance reboot/shutdown/kill the results of domblkstat will be reset to zero
  • After a detach domblkstat will return “not found” for a the device

Volume_usage_cache table design

ID(Primay key)

Nova Volume Name(Foreign key)

Total last updated(timestamp)

Total Reads(long int)

Total Read Bytes(long int)

Total Writes(long int)

Total Write Bytes(long int)

Current last updated(timestamp)

Current Reads(long int)

Current Read Bytes(long int)

Current Writes(long int)

Current Write Bytes(long int)

Volume_usage_cache table interaction

1. On a first attach of a volume to vm(create then attach). Nova volume id will not be found in volume_usage_cache table, so create. Stats columns will default to zero

2. Periodic task (_poll_volume_usage) will run and update Current last updated, Current Reads, Current Read Bytes, Current Writes, Current Write Bytes with usage information for attach volumes on a specific compute host.

3. On a detach update database. Get current stats(Current Reads, Current Read Bytes, Current Writes, Current Write Bytes) and add to total stats(Total Reads, Total Read Bytes, Total Writes, Total Write Bytes) putting result in total stats. Set current stats to zero

4. On a subsequent attach verify that current stats are zero

5. Periodic task (_poll_volume_usage) will run and update current stats and so on

_poll_volume_usage implementation using libvirt domblkstat

  • _poll_volume_usage will call self.db.volume_get_all_by_host(ctxt, self.host) - returns a list of volumes for this compute host
  • Using this volume list call get_all_vol_usage (will have a very similar signature as get_all_bw_usage) - function will take the list of volumes mounted on this compute host as an arg
  • add get_all_vol_usage in virt driver (NO OP), add implementation of get_all_vol_usage in libvirt driver
  • get_all_vol_usage in libvirt will take a list of volumes as an arg (return of self.db.volume_get_all_by_host(ctxt, self.host)), iterates over the list of volumes running bock_stats on each and append to a list of stats
  • returns the list of vol stats once completed
  • return up to poll_volume_usage with list of stats
  • _poll_volume_usage iterate over list of stats and updates volume_usage_cache table.