OpenStack - User contributions [en]

Stackforge Namespace Retirement

2015-09-28T15:41:42Z

Belliott:

=== Background ===

The stackforge/ git namespace is being retired, and active projects are being moved to the openstack/ namespace. See this mailing list post for [http://lists.openstack.org/pipermail/openstack-dev/2015-August/071816.html full background]. These changes are scheduled to occur on October 17, 2015.

There are two lists below, one for active projects that should be moved, and a second for inactive projects that should become read-only. Please update them to add your project to the correct list.

=== Active Projects to Move ===
Active stackforge projects that wish to move into the openstack/ namespace should be added to this list. Projects in this list will be automatically moved by the Infrastructure team to openstack/ on October 17. Please note that no other renames can happen during this move -- projects will strictly be moved from stackforge/ to openstack/ with their existing names.

* aeromancer
* anvil
* bansho
* blazar
* blazar-nova
* ceilometer-powervm
* ceilometer-zvm
* cerberus
* cerberus-dashboard
* cl-openstack-client
* cloud-init
* cloudbase-init
* clouddocs-maven-plugin
* cloudkitty
* cloudkitty-dashboard
* cloudpulse
* cognitive
* compass-adapters
* compass-core
* compass-specs
* compass-web
* compute-hyperv
* designate-msdnsagent
* devstack-plugin-glusterfs
* devstack-plugin-sheepdog
* doc8
* dox
* drbd-devstack
* driverlog
* ec2-api
* ec2-driver
* faafo
* flame
* freezer
* freezer-api
* freezer-web-ui
* fuel-agent
* fuel-astute
* fuel-dev-tools
* fuel-devops
* fuel-docs
* fuel-library
* fuel-main
* fuel-mirror
* fuel-nailgun-agent
* fuel-octane
* fuel-ostf
* fuel-ostf-plugin
* fuel-plugin-calamari
* fuel-plugin-calico
* fuel-plugin-ceilometer-redis
* fuel-plugin-cinder-netapp
* fuel-plugin-cisco-aci
* fuel-plugin-contrail
* fuel-plugin-dbaas-trove
* fuel-plugin-detach-database
* fuel-plugin-detach-keystone
* fuel-plugin-detach-rabbitmq
* fuel-plugin-elasticsearch-kibana
* fuel-plugin-external-emc
* fuel-plugin-external-glusterfs
* fuel-plugin-external-zabbix
* fuel-plugin-glance-nfs
* fuel-plugin-ha-fencing
* fuel-plugin-influxdb-grafana
* fuel-plugin-ironic
* fuel-plugin-ldap
* fuel-plugin-lma-collector
* fuel-plugin-lma-infrastructure-alerting
* fuel-plugin-mellanox
* fuel-plugin-midonet
* fuel-plugin-neutron-fwaas
* fuel-plugin-neutron-lbaas
* fuel-plugin-neutron-vpnaas
* fuel-plugin-nova-nfs
* fuel-plugin-nsxv
* fuel-plugin-opendaylight
* fuel-plugin-saltstack
* fuel-plugin-solidfire-cinder
* fuel-plugin-swiftstack
* fuel-plugin-tintri-cinder
* fuel-plugin-tls
* fuel-plugin-vmware-dvs
* fuel-plugin-vxlan
* fuel-plugin-zabbix-monitoring-emc
* fuel-plugin-zabbix-monitoring-extreme-networks
* fuel-plugin-zabbix-snmptrapd
* fuel-plugins
* fuel-qa
* fuel-specs
* fuel-stats
* fuel-upgrade
* fuel-web
* gce-api
* gerrit-dash-creator
* git-upstream
* golang-client
* group-based-policy
* group-based-policy-automation
* group-based-policy-specs
* group-based-policy-ui
* intel-nfv-ci-tests
* merlin
* monasca-agent
* monasca-api
* monasca-ceilometer
* monasca-common
* monasca-log-api
* monasca-notification
* monasca-persister
* monasca-statsd
* monasca-thresh
* monasca-ui
* monasca-vagrant
* monitoring-for-openstack
* namos
* nerd-reviewer
* networking-6wind
* networking-bagpipe-l2
* networking-hyperv
* networking-ovs-dpdk
* networking-zvm
* nova-docker
* nova-powervm
* nova-solver-scheduler
* nova-zvm-virt-driver
* ooi
* ops-tags-team
* osprofiler
* ospurge
* packstack
* poppy
* proliantutils
* puppet-autossh
* puppet-ceph
* puppet-n1k-vsm
* puppet-setproxy
* puppet-surveil
* python-blazarclient
* python-cerberusclient
* python-cloudkittyclient
* python-cloudpulseclient
* python-cognitiveclient
* python-fuelclient
* python-group-based-policy-client
* python-jenkins
* python-monascaclient
* python-openstacksdk
* python-rackclient
* python-senlinclient
* python-sticksclient
* python-surveilclient
* python-tackerclient
* python-watcherclient
* rack
* requests-mock
* sahara-ci-config
* senlin
* senlin-dashboard
* shaker
* sqlalchemy-migrate
* surveil
* surveil-specs
* stackalytics
* sticks
* sticks-dashboard
* swift3
* swiftonfile
* swift-ceph-backend
* tacker
* tacker-horizon
* tacker-specs
* tap-as-a-service
* telcowg-usecases
* terracotta
* third-party-ci-tools
* tricircle
* vmtp
* watcher
* wsme
* xenapi-os-testing
* xstatic-d3
* xstatic-angular-sanitize
* xstatic-bootstrap-datepicker
* xstatic-angular-gettext
* xstatic-bootswatch
* xstatic-angular-cookies
* xstatic-bootstrap-scss
* xstatic-angular-smart-table
* xstatic-angular-fileupload
* xstatic-angular-bootstrap
* xstatic-angular
* xstatic-angular-mock
* xstatic-angular-lrdragndrop
* xstatic-roboto-fontface
* xstatic-jquery.quicksearch
* xstatic-mdi
* xstatic-rickshaw
* xstatic-magic-search
* xstatic-font-awesome
* xstatic-hogan
* xstatic-spin
* xstatic-jasmine
* xstatic-jquery-migrate
* xstatic-jquery-migrate
* xstatic-jsencrypt
* yaql

=== Inactive Projects to Retire ===
Inactive projects that should be retired should be added to this list. These projects will have a commit merged removing their content and replacing it with a message indicating the project is no longer maintained and will become read-only in Gerrit.

* cachemonkey
* compass-monit
* congressmiddleware
* fuel-plugin-availability-zones
* fuel-provision
* fuel-tasklib
* mercador-pub
* mercador-sub
* MRaaS
* networking-portforwarding
* openstackdroid
* python-mercadorclient
* rubick
* sahara-guestagent
* libra
* logaas
* python-libraclient
* python-rallyclient
* cookbook-pacemaker
* puppet-openstack_dev_env
* puppet_openstack_builder
* puppet-openstack-cloud
* tripleo-ansible
* kickstack
* packstack-vagrant
* haos
* novaimagebuilder

Meetings/Nova

2015-08-17T14:56:35Z

Belliott:

= Weekly Nova team meeting =
'''MEETING TIME: Thursdays alternating 14:00 UTC (#openstack-meeting) and 21:00 UTC (#openstack-meeting)'''

This meeting is a weekly gathering of developers working on [[Nova|OpenStack Compute (Nova)]]. We cover topics such as release planning and status, bugs, reviews, and other current topics worthy of real-time discussion.

NOTE: this wiki page should be 'emptied' at the end of each meeting.

== Agenda for next meeting ==

Next meetings scheduled for:
* August 20th 2015 1400 UTC, #openstack-meeting (http://www.timeanddate.com/worldclock/fixedtime.html?iso=20150820T140000)
* August 27th 2015 2100 UTC, #openstack-meeting (http://www.timeanddate.com/worldclock/fixedtime.html?iso=20150827T210000)

Add your IRC nick to this list to be pinged at the start of the meeting:
<code>
mikal tjones cburgess jgrimm adrian_otto funzo mjturek jcookekhugen
irina_pov krtaylor danpb alexpilotti flip214 jaypipes garyk edleafe dims
moshele anteaya Nisha sileht claudiub lxsli neiljerram markus_z swamireddy alevine
tonyb andreykurilin ndipanov sc68cal akuriata artom jlvillal mnestratov kashyap
aloga rgeragnov bauzas xyang tpatil med_ nic scottda nagyz dannywilson belliott
</code>

Please note "stuck review" means a review where there is some disagreement that needs resolving.
Its not for reviews that just haven't had attention, except for exceptional cases.
Where you see "?" feel free to just edit the wiki and add your item.

Here is the agenda for the next meeting:
* Release Status
** August 18: Feature Proposal Freeze (see https://wiki.openstack.org/wiki/FeatureProposalFreeze)
** Updates on feature freeze in here: https://etherpad.openstack.org/p/liberty-nova-non-priority-feature-freeze
** Nova specific things: https://wiki.openstack.org/wiki/Nova/Liberty_Release_Schedule
* Bugs (stuck/critical)
** Gate status
*** http://status.openstack.org/elastic-recheck/index.html
** Critical bugs
*** ?
** Stable branch status: https://etherpad.openstack.org/p/stable-tracker
*** stable/kilo: https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:stable/kilo,n,z
*** stable/juno: https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:stable/juno,n,z
* Regular Review Reminders
** all please look here for what patches to review: https://etherpad.openstack.org/p/liberty-nova-priorities-tracking
** we have a list of actions: https://etherpad.openstack.org/p/YVR-nova-liberty-summit-action-items and https://etherpad.openstack.org/p/liberty-nova-midcycle
* Stuck Reviews
** Please note "stuck review" means a review where there is some disagreement that needs resolving. Its not for reviews that just haven't had attention, except for exceptional cases. Where you see "?" feel free to just edit the wiki and add your item.
** Any stuck code reviews to discuss:
*** ?
* Open discussion
** ?

== Sub-teams ==

There are also some Nova subteam meetings. See [[Nova#Active_Sub-teams:]] for details.

== Previous meetings ==

* [http://eavesdrop.openstack.org/meetings/nova/ All other meetings are here]
* [http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-08-16-21.01.html 2012-08-16]
* [http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-08-09-21.00.html 2012-08-09]
* [http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-08-02-21.40.html 2012-08-02]

[[category: compute]]
[[category: meetings]]

ImagePrecacher

2014-03-12T21:42:38Z

Belliott:

= Goal =

Introduce a new, optional nova service to cache high priority images on compute hosts prior to the image being requested during the usual instance creation process.

= Current Caching Strategy =

In the current icehouse release of Nova, images are cached on demand. As instances are created, compute hosts need to retrieve the requested image in order to build each instance. The host may then cache that image for some period of time locally. This solution speeds up subsequent requests to build an instance from the same image, on the same host.

== Limitations ==
However, there are some limitations with the current approach:
* Inconsistent build times - the first build on a host for each image is slower than subsequent ones
* Network throttling - If creation of a large number of instances is requested, many simultaneous transfers of images to hosts will be initiated. This may have a crippling effect on network performance.

= Proposed Caching Service =

To rectify the current limitations, we can add a simple nova service to aid in efficient distribution of high priority images to compute hosts out of the control path of the instance creation process. The service would consist of a periodic task performing the following tasks:

# List - Obtain a list of images to pre-cache.
# Fetch - Obtain a copy of each image's data.
# Serve - Setup each image's data for distribution to hosts.
# Distribute - Trigger downloads of each image by the hosts.

== List ==

Use a pluggable strategy for obtaining a list of images to pre-cache. A default implementation might query Glance to obtain a list of images for certain tenants.

<nowiki>
class ImageLister(object):
def list(self):
...
return image_list # return a list of images to pre-cache</nowiki>

== Fetch ==

Use a pluggable strategy for fetching the images from their backing storage. One option might be a class capable of fetching image data from swift.

<nowiki>
class ImageFetcher(object):
def fetch(self, image_id):
...
return image_filename # return local filesystem path to image</nowiki>

== Serve ==

Perform work necessary to make the image available to compute hosts. In the BitTorrent case, this would mean providing an initial seed of the image.

<nowiki>
class ImageServer(object):
def serve(self, image_id, image_filename):
...
return meta # return metadata necessary for a compute host to download the image</nowiki>

== Distribute ==

Pre-cache service would send a RPC fanout message on the "compute" topic for each image. This would cause each host to fetch the image and pre-cache it uses virt driver specific mechanisms.

RPC API method:
<nowiki>
def cache(image_id, meta)
... # meta contains all details necessary to fetch image from pre-cache service</nowiki>

ImagePrecacher

2014-03-12T20:50:23Z

Belliott: Created page with "= Goal = Introduce a new, optional nova service to cache high priority images on compute hosts prior to the image being requested during the usual instance creation process...."

XenServer/Overhead

2013-12-04T21:58:39Z

Belliott:

= Xen API driver overhead =

Per-VM virtualization overhead calculations were added to the xenapi driver in this review: https://review.openstack.org/#/c/35380/

These values were updated to take into account the number of VCPUs assigned to a VM in this review: https://review.openstack.org/60087

== Estimating Overhead ==

The overhead data below is collected from guest instances in the Rackspace cloud via XenAPI.

The data was plotted on a graph as follows:
* X-axis: memory size (GB) of the VM
* Y-axis: number of VCPUs allocated to the VM
* Z-axis: memory overhead size (MB) of the VM as observed in XenAPI.

An interpolated equation is fitted over top of the data to predict the overhead based on VM memory size and number of VCPUs.

== Data ==
The formula to calculate overhead is derived from the following data:

{|class="wikitable" style="text-align: center; color: black;"
! align="left"| VM memory size (GB)
! VM number of VCPUs
! VM memory overhead
! Overhead/MB
! Predicted Overhead
|-
| 256.0
| 1
| 4.0
| 0.015625
| 6.49936
|-
| 512.0
| 1
| 7.0
| 0.013671875
| 8.49872
|-
| 1024.0
| 1
| 10.0
| 0.009765625
| 12.49744
|-
| 1024.0
| 1
| 11.0
| 0.0107421875
| 12.49744
|-
| 512.0
| 1
| 6.0
| 0.01171875
| 8.49872
|-
| 2048.0
| 1
| 19.0
| 0.00927734375
| 20.49488
|-
| 3972.02233887
| 2
| 35.0
| 0.00881163221503
| 37.0214944666
|-
| 3895.72839355
| 2
| 34.0
| 0.00872750781503
| 36.4256387537
|-
| 1508.39562988
| 2
| 15.0
| 0.00994434066424
| 17.7805698694
|-
| 4086.0
| 2
| 36.0
| 0.00881057268722
| 37.91166
|-
| 2048.0
| 2
| 19.0
| 0.00927734375
| 21.99488
|-
| 4048.31628418
| 2
| 35.0
| 0.00864556955117
| 37.6173501794
|-
| 2048.0
| 2
| 20.0
| 0.009765625
| 21.99488
|-
| 4057.85302734
| 2
| 36.0
| 0.0088716865193
| 37.6918321436
|-
| 4073.77954102
| 2
| 35.0
| 0.0085915302111
| 37.8162182153
|-
| 4096.0
| 2
| 35.0
| 0.008544921875
| 37.98976
|-
| 4096.0
| 2
| 36.0
| 0.0087890625
| 37.98976
|-
| 4072.0
| 2
| 35.0
| 0.0085952848723
| 37.80232
|-
| 4096.0
| 4
| 38.0
| 0.00927734375
| 40.98976
|-
| 4096.0
| 4
| 37.0
| 0.009033203125
| 40.98976
|-
| 15360.0
| 4
| 125.0
| 0.00813802083333
| 128.9616
|-
| 512.0
| 4
| 9.0
| 0.017578125
| 12.99872
|-
| 8192.0
| 4
| 70.0
| 0.008544921875
| 72.97952
|-
| 8192.0
| 4
| 69.0
| 0.0084228515625
| 72.97952
|-
| 1024.0
| 4
| 14.0
| 0.013671875
| 16.99744
|-
| 1024.0
| 4
| 13.0
| 0.0126953125
| 16.99744
|-
| 2048.0
| 4
| 21.0
| 0.01025390625
| 24.99488
|-
| 256.0
| 4
| 7.0
| 0.02734375
| 10.99936
|-
| 15360.0
| 4
| 126.0
| 0.008203125
| 128.9616
|-
| 15258.7890625
| 6
| 127.0
| 0.008323072
| 131.171142578
|-
| 15324.7539062
| 6
| 127.0
| 0.00828724564041
| 131.686328008
|-
| 15169.2651367
| 6
| 126.0
| 0.00830626921373
| 130.471960718
|-
| 15264.6325684
| 6
| 127.0
| 0.00831988581653
| 131.216780359
|-
| 15267.1328125
| 6
| 127.0
| 0.00831852329836
| 131.236307266
|-
| 15360.0
| 6
| 127.0
| 0.00826822916667
| 131.9616
|-
| 15335.0830078
| 6
| 128.0
| 0.00834687363184
| 131.766998291
|-
| 15360.0
| 6
| 128.0
| 0.00833333333333
| 131.9616
|-
| 14819.2524414
| 6
| 124.0
| 0.00836749360268
| 127.738361567
|-
| 15256.3276367
| 6
| 128.0
| 0.00838996140145
| 131.151918843
|-
| 30720.0
| 8
| 249.0
| 0.00810546875
| 254.9232
|-
| 8192.0
| 8
| 74.0
| 0.009033203125
| 78.97952
|-
| 8192.0
| 8
| 73.0
| 0.0089111328125
| 78.97952
|-
| 30720.0
| 8
| 250.0
| 0.00813802083333
| 254.9232
|-
| 61440.0
| 16
| 497.0
| 0.00808919270833
| 506.8464
|-
| 61440.0
| 16
| 498.0
| 0.00810546875
| 506.8464
|-
| 92160.0
| 24
| 745.0
| 0.00808376736111
| 758.7696
|-
| 92160.0
| 24
| 746.0
| 0.00809461805556
| 758.7696
|-
| 122880.0
| 32
| 994.0
| 0.00808919270833
| 1010.6928
|-
| 122880.0
| 32
| 993.0
| 0.0080810546875
| 1010.6928
|}

== Graph ==

The following graph was produced via matplotlib:

[[File:Overheadgraph2.jpg|framed|left|VM Memory Size (X) vs Number of VCPUs (Y) vs VM Memory Overhead Size (Z)]]

== Equation ==

The following equation is an interpolation of the above data done via the Affine method in Mac OS X's Grapher app:

overhead = (a * flavor_mb) + (b * num_vcpus) + c

a and b are the constants:
a = 0.0078
b = 0.9674
c = 1.6162

XenServer/Overhead

2013-12-04T18:02:04Z

Belliott:

= Xen API driver overhead =

Per-VM virtualization overhead calculations were added to the xenapi driver in this review: https://review.openstack.org/#/c/35380/

These values were updated to take into account the number of VCPUs assigned to a VM in this review: TBD

== Estimating Overhead ==

The overhead data below is collected from guest instances in the Rackspace cloud via XenAPI.

The data was plotted on a graph as follows:
* X-axis: memory size (GB) of the VM
* Y-axis: number of VCPUs allocated to the VM
* Z-axis: memory overhead size (MB) of the VM as observed in XenAPI.

An interpolated equation is fitted over top of the data to predict the overhead based on VM memory size and number of VCPUs.

== Data ==
The formula to calculate overhead is derived from the following data:

{|class="wikitable" style="text-align: center; color: black;"
! align="left"| VM memory size (GB)
! VM number of VCPUs
! VM memory overhead
! Overhead/MB
! Predicted Overhead
|-
| 256.0
| 1
| 4.0
| 0.015625
| 6.49936
|-
| 512.0
| 1
| 7.0
| 0.013671875
| 8.49872
|-
| 1024.0
| 1
| 10.0
| 0.009765625
| 12.49744
|-
| 1024.0
| 1
| 11.0
| 0.0107421875
| 12.49744
|-
| 512.0
| 1
| 6.0
| 0.01171875
| 8.49872
|-
| 2048.0
| 1
| 19.0
| 0.00927734375
| 20.49488
|-
| 3972.02233887
| 2
| 35.0
| 0.00881163221503
| 37.0214944666
|-
| 3895.72839355
| 2
| 34.0
| 0.00872750781503
| 36.4256387537
|-
| 1508.39562988
| 2
| 15.0
| 0.00994434066424
| 17.7805698694
|-
| 4086.0
| 2
| 36.0
| 0.00881057268722
| 37.91166
|-
| 2048.0
| 2
| 19.0
| 0.00927734375
| 21.99488
|-
| 4048.31628418
| 2
| 35.0
| 0.00864556955117
| 37.6173501794
|-
| 2048.0
| 2
| 20.0
| 0.009765625
| 21.99488
|-
| 4057.85302734
| 2
| 36.0
| 0.0088716865193
| 37.6918321436
|-
| 4073.77954102
| 2
| 35.0
| 0.0085915302111
| 37.8162182153
|-
| 4096.0
| 2
| 35.0
| 0.008544921875
| 37.98976
|-
| 4096.0
| 2
| 36.0
| 0.0087890625
| 37.98976
|-
| 4072.0
| 2
| 35.0
| 0.0085952848723
| 37.80232
|-
| 4096.0
| 4
| 38.0
| 0.00927734375
| 40.98976
|-
| 4096.0
| 4
| 37.0
| 0.009033203125
| 40.98976
|-
| 15360.0
| 4
| 125.0
| 0.00813802083333
| 128.9616
|-
| 512.0
| 4
| 9.0
| 0.017578125
| 12.99872
|-
| 8192.0
| 4
| 70.0
| 0.008544921875
| 72.97952
|-
| 8192.0
| 4
| 69.0
| 0.0084228515625
| 72.97952
|-
| 1024.0
| 4
| 14.0
| 0.013671875
| 16.99744
|-
| 1024.0
| 4
| 13.0
| 0.0126953125
| 16.99744
|-
| 2048.0
| 4
| 21.0
| 0.01025390625
| 24.99488
|-
| 256.0
| 4
| 7.0
| 0.02734375
| 10.99936
|-
| 15360.0
| 4
| 126.0
| 0.008203125
| 128.9616
|-
| 15258.7890625
| 6
| 127.0
| 0.008323072
| 131.171142578
|-
| 15324.7539062
| 6
| 127.0
| 0.00828724564041
| 131.686328008
|-
| 15169.2651367
| 6
| 126.0
| 0.00830626921373
| 130.471960718
|-
| 15264.6325684
| 6
| 127.0
| 0.00831988581653
| 131.216780359
|-
| 15267.1328125
| 6
| 127.0
| 0.00831852329836
| 131.236307266
|-
| 15360.0
| 6
| 127.0
| 0.00826822916667
| 131.9616
|-
| 15335.0830078
| 6
| 128.0
| 0.00834687363184
| 131.766998291
|-
| 15360.0
| 6
| 128.0
| 0.00833333333333
| 131.9616
|-
| 14819.2524414
| 6
| 124.0
| 0.00836749360268
| 127.738361567
|-
| 15256.3276367
| 6
| 128.0
| 0.00838996140145
| 131.151918843
|-
| 30720.0
| 8
| 249.0
| 0.00810546875
| 254.9232
|-
| 8192.0
| 8
| 74.0
| 0.009033203125
| 78.97952
|-
| 8192.0
| 8
| 73.0
| 0.0089111328125
| 78.97952
|-
| 30720.0
| 8
| 250.0
| 0.00813802083333
| 254.9232
|-
| 61440.0
| 16
| 497.0
| 0.00808919270833
| 506.8464
|-
| 61440.0
| 16
| 498.0
| 0.00810546875
| 506.8464
|-
| 92160.0
| 24
| 745.0
| 0.00808376736111
| 758.7696
|-
| 92160.0
| 24
| 746.0
| 0.00809461805556
| 758.7696
|-
| 122880.0
| 32
| 994.0
| 0.00808919270833
| 1010.6928
|-
| 122880.0
| 32
| 993.0
| 0.0080810546875
| 1010.6928
|}

== Graph ==

The following graph was produced via matplotlib:

[[File:Overheadgraph2.jpg|framed|left|VM Memory Size (X) vs Number of VCPUs (Y) vs VM Memory Overhead Size (Z)]]

== Equation ==

The following equation is an interpolation of the above data done via the Affine method in Mac OS X's Grapher app:

overhead = (a * flavor_mb) + (b * num_vcpus) + c

a and b are the constants:
a = 0.0078
b = 0.9674
c = 1.6162

XenServer/Overhead

2013-12-04T17:58:22Z

Belliott:

= Xen API driver overhead =

Per-VM virtualization overhead calculations were added to the xenapi driver in this review: https://review.openstack.org/#/c/35380/

These values were updated to take into account the number of VCPUs assigned to a VM in this review: TBD

== Estimating Overhead ==

The overhead data below is collected from guest instances in the Rackspace cloud via XenAPI.

The data was plotted on a graph as follows:
* X-axis: memory size (GB) of the VM
* Y-axis: number of VCPUs allocated to the VM
* Z-axis: memory overhead size (MB) of the VM as observed in XenAPI.

An interpolated equation is fitted over top of the data to predict the overhead based on VM memory size.

== Data ==
The formula to calculate overhead is derived from the following data:

{|class="wikitable" style="text-align: center; color: black;"
! align="left"| VM memory size (GB)
! VM number of VCPUs
! VM memory overhead
! Overhead/MB
! Predicted Overhead
|-
| 256.0
| 1
| 4.0
| 0.015625
| 6.49936
|-
| 512.0
| 1
| 7.0
| 0.013671875
| 8.49872
|-
| 1024.0
| 1
| 10.0
| 0.009765625
| 12.49744
|-
| 1024.0
| 1
| 11.0
| 0.0107421875
| 12.49744
|-
| 512.0
| 1
| 6.0
| 0.01171875
| 8.49872
|-
| 2048.0
| 1
| 19.0
| 0.00927734375
| 20.49488
|-
| 3972.02233887
| 2
| 35.0
| 0.00881163221503
| 37.0214944666
|-
| 3895.72839355
| 2
| 34.0
| 0.00872750781503
| 36.4256387537
|-
| 1508.39562988
| 2
| 15.0
| 0.00994434066424
| 17.7805698694
|-
| 4086.0
| 2
| 36.0
| 0.00881057268722
| 37.91166
|-
| 2048.0
| 2
| 19.0
| 0.00927734375
| 21.99488
|-
| 4048.31628418
| 2
| 35.0
| 0.00864556955117
| 37.6173501794
|-
| 2048.0
| 2
| 20.0
| 0.009765625
| 21.99488
|-
| 4057.85302734
| 2
| 36.0
| 0.0088716865193
| 37.6918321436
|-
| 4073.77954102
| 2
| 35.0
| 0.0085915302111
| 37.8162182153
|-
| 4096.0
| 2
| 35.0
| 0.008544921875
| 37.98976
|-
| 4096.0
| 2
| 36.0
| 0.0087890625
| 37.98976
|-
| 4072.0
| 2
| 35.0
| 0.0085952848723
| 37.80232
|-
| 4096.0
| 4
| 38.0
| 0.00927734375
| 40.98976
|-
| 4096.0
| 4
| 37.0
| 0.009033203125
| 40.98976
|-
| 15360.0
| 4
| 125.0
| 0.00813802083333
| 128.9616
|-
| 512.0
| 4
| 9.0
| 0.017578125
| 12.99872
|-
| 8192.0
| 4
| 70.0
| 0.008544921875
| 72.97952
|-
| 8192.0
| 4
| 69.0
| 0.0084228515625
| 72.97952
|-
| 1024.0
| 4
| 14.0
| 0.013671875
| 16.99744
|-
| 1024.0
| 4
| 13.0
| 0.0126953125
| 16.99744
|-
| 2048.0
| 4
| 21.0
| 0.01025390625
| 24.99488
|-
| 256.0
| 4
| 7.0
| 0.02734375
| 10.99936
|-
| 15360.0
| 4
| 126.0
| 0.008203125
| 128.9616
|-
| 15258.7890625
| 6
| 127.0
| 0.008323072
| 131.171142578
|-
| 15324.7539062
| 6
| 127.0
| 0.00828724564041
| 131.686328008
|-
| 15169.2651367
| 6
| 126.0
| 0.00830626921373
| 130.471960718
|-
| 15264.6325684
| 6
| 127.0
| 0.00831988581653
| 131.216780359
|-
| 15267.1328125
| 6
| 127.0
| 0.00831852329836
| 131.236307266
|-
| 15360.0
| 6
| 127.0
| 0.00826822916667
| 131.9616
|-
| 15335.0830078
| 6
| 128.0
| 0.00834687363184
| 131.766998291
|-
| 15360.0
| 6
| 128.0
| 0.00833333333333
| 131.9616
|-
| 14819.2524414
| 6
| 124.0
| 0.00836749360268
| 127.738361567
|-
| 15256.3276367
| 6
| 128.0
| 0.00838996140145
| 131.151918843
|-
| 30720.0
| 8
| 249.0
| 0.00810546875
| 254.9232
|-
| 8192.0
| 8
| 74.0
| 0.009033203125
| 78.97952
|-
| 8192.0
| 8
| 73.0
| 0.0089111328125
| 78.97952
|-
| 30720.0
| 8
| 250.0
| 0.00813802083333
| 254.9232
|-
| 61440.0
| 16
| 497.0
| 0.00808919270833
| 506.8464
|-
| 61440.0
| 16
| 498.0
| 0.00810546875
| 506.8464
|-
| 92160.0
| 24
| 745.0
| 0.00808376736111
| 758.7696
|-
| 92160.0
| 24
| 746.0
| 0.00809461805556
| 758.7696
|-
| 122880.0
| 32
| 994.0
| 0.00808919270833
| 1010.6928
|-
| 122880.0
| 32
| 993.0
| 0.0080810546875
| 1010.6928
|}

== Graph ==

The graph is produced in Mac OS X's Grapher app. The equation shown is interpolated via the Affine method built into the app. Two lines are shown; one of the actual observed data and one of the predicted overhead.

[[File:Overheadgraph2.jpg|framed|left|VM Memory Size (X) vs Number of VCPUs (Y) vs VM Memory Overhead Size (Z)]]

File:Overheadgraph2.jpg

2013-12-04T17:57:22Z

Belliott: XenServer overhead graph

XenServer overhead graph

XenServer/Overhead

2013-12-04T17:55:29Z

Belliott:

= Xen API driver overhead =

Per-VM virtualization overhead calculations were added to the xenapi driver in this review: https://review.openstack.org/#/c/35380/

These values were updated to take into account the number of VCPUs assigned to a VM in this review: TBD

== Estimating Overhead ==

The overhead data below is collected from guest instances in the Rackspace cloud via XenAPI.

The data was plotted on a graph as follows:
* X-axis: memory size (GB) of the VM
* Y-axis: number of VCPUs allocated to the VM
* Z-axis: memory overhead size (MB) of the VM as observed in XenAPI.

An interpolated equation is fitted over top of the data to predict the overhead based on VM memory size.

== Data ==
The formula to calculate overhead is derived from the following data:

{|class="wikitable" style="text-align: center; color: black;"
! align="left"| VM memory size (GB)
! VM number of VCPUs
! VM memory overhead
! Overhead/MB
! Predicted Overhead
|-
| 256.0
| 1
| 4.0
| 0.015625
| 6.49936
|-
| 512.0
| 1
| 7.0
| 0.013671875
| 8.49872
|-
| 1024.0
| 1
| 10.0
| 0.009765625
| 12.49744
|-
| 1024.0
| 1
| 11.0
| 0.0107421875
| 12.49744
|-
| 512.0
| 1
| 6.0
| 0.01171875
| 8.49872
|-
| 2048.0
| 1
| 19.0
| 0.00927734375
| 20.49488
|-
| 3972.02233887
| 2
| 35.0
| 0.00881163221503
| 37.0214944666
|-
| 3895.72839355
| 2
| 34.0
| 0.00872750781503
| 36.4256387537
|-
| 1508.39562988
| 2
| 15.0
| 0.00994434066424
| 17.7805698694
|-
| 4086.0
| 2
| 36.0
| 0.00881057268722
| 37.91166
|-
| 2048.0
| 2
| 19.0
| 0.00927734375
| 21.99488
|-
| 4048.31628418
| 2
| 35.0
| 0.00864556955117
| 37.6173501794
|-
| 2048.0
| 2
| 20.0
| 0.009765625
| 21.99488
|-
| 4057.85302734
| 2
| 36.0
| 0.0088716865193
| 37.6918321436
|-
| 4073.77954102
| 2
| 35.0
| 0.0085915302111
| 37.8162182153
|-
| 4096.0
| 2
| 35.0
| 0.008544921875
| 37.98976
|-
| 4096.0
| 2
| 36.0
| 0.0087890625
| 37.98976
|-
| 4072.0
| 2
| 35.0
| 0.0085952848723
| 37.80232
|-
| 4096.0
| 4
| 38.0
| 0.00927734375
| 40.98976
|-
| 4096.0
| 4
| 37.0
| 0.009033203125
| 40.98976
|-
| 15360.0
| 4
| 125.0
| 0.00813802083333
| 128.9616
|-
| 512.0
| 4
| 9.0
| 0.017578125
| 12.99872
|-
| 8192.0
| 4
| 70.0
| 0.008544921875
| 72.97952
|-
| 8192.0
| 4
| 69.0
| 0.0084228515625
| 72.97952
|-
| 1024.0
| 4
| 14.0
| 0.013671875
| 16.99744
|-
| 1024.0
| 4
| 13.0
| 0.0126953125
| 16.99744
|-
| 2048.0
| 4
| 21.0
| 0.01025390625
| 24.99488
|-
| 256.0
| 4
| 7.0
| 0.02734375
| 10.99936
|-
| 15360.0
| 4
| 126.0
| 0.008203125
| 128.9616
|-
| 15258.7890625
| 6
| 127.0
| 0.008323072
| 131.171142578
|-
| 15324.7539062
| 6
| 127.0
| 0.00828724564041
| 131.686328008
|-
| 15169.2651367
| 6
| 126.0
| 0.00830626921373
| 130.471960718
|-
| 15264.6325684
| 6
| 127.0
| 0.00831988581653
| 131.216780359
|-
| 15267.1328125
| 6
| 127.0
| 0.00831852329836
| 131.236307266
|-
| 15360.0
| 6
| 127.0
| 0.00826822916667
| 131.9616
|-
| 15335.0830078
| 6
| 128.0
| 0.00834687363184
| 131.766998291
|-
| 15360.0
| 6
| 128.0
| 0.00833333333333
| 131.9616
|-
| 14819.2524414
| 6
| 124.0
| 0.00836749360268
| 127.738361567
|-
| 15256.3276367
| 6
| 128.0
| 0.00838996140145
| 131.151918843
|-
| 30720.0
| 8
| 249.0
| 0.00810546875
| 254.9232
|-
| 8192.0
| 8
| 74.0
| 0.009033203125
| 78.97952
|-
| 8192.0
| 8
| 73.0
| 0.0089111328125
| 78.97952
|-
| 30720.0
| 8
| 250.0
| 0.00813802083333
| 254.9232
|-
| 61440.0
| 16
| 497.0
| 0.00808919270833
| 506.8464
|-
| 61440.0
| 16
| 498.0
| 0.00810546875
| 506.8464
|-
| 92160.0
| 24
| 745.0
| 0.00808376736111
| 758.7696
|-
| 92160.0
| 24
| 746.0
| 0.00809461805556
| 758.7696
|-
| 122880.0
| 32
| 994.0
| 0.00808919270833
| 1010.6928
|-
| 122880.0
| 32
| 993.0
| 0.0080810546875
| 1010.6928
|}

== Graph ==

The graph is produced in Mac OS X's Grapher app. The equation shown is interpolated via the Affine method built into the app. Two lines are shown; one of the actual observed data and one of the predicted overhead.

[[File:Overheadgraph.jpg|framed|left|VM Memory Size (X) vs VM Memory Overhead Size (Y)]]

XenServer/Overhead

2013-07-23T16:38:15Z

Belliott:

= Xen API driver overhead =

Per-VM virtualization overhead calculations were added to the xenapi driver in this review: https://review.openstack.org/#/c/35380/

== Estimating Overhead ==

The overhead data below is collected from guest instances in the Rackspace cloud via XenAPI.

The data was plotted on a graph as follows:
* X-axis: memory size (MB) of the VM
* Y-axis: memory overhead size (MB) of the VM as observed in XenAPI.

An interpolated equation is fitted over top of the data to predict the overhead based on VM memory size.

== Data ==
The formula to calculate overhead is derived from the following data:

{|class="wikitable" style="text-align: center; color: black;"
! align="left"| XenServer version
! VM type
! VM memory size
! VM memory overhead
! Overhead/MB
! Predicted Overhead
|-
| 6.0.0
| guest
| 512.0
| 7.0
| 0.013671875
| 7.1472
|-
| 6.0.0
| guest
| 512.0
| 6.0
| 0.01171875
| 7.1472
|-
| 6.0.0
| guest
| 1024.0
| 11.0
| 0.0107421875
| 11.2944
|-
| 6.0.0
| guest
| 1024.0
| 10.0
| 0.009765625
| 11.2944
|-
| 6.0.0
| guest
| 2048.0
| 20.0
| 0.009765625
| 19.5888
|-
| 6.0.0
| guest
| 2048.0
| 19.0
| 0.00927734375
| 19.5888
|-
| 6.0.0
| guest
| 4096.0
| 36.0
| 0.0087890625
| 36.1776
|-
| 6.0.0
| guest
| 4096.0
| 35.0
| 0.008544921875
| 36.1776
|-
| 6.0.0
| guest
| 8192.0
| 70.0
| 0.008544921875
| 69.3552
|-
| 6.0.0
| guest
| 8192.0
| 69.0
| 0.0084228515625
| 69.3552
|-
| 6.0.0
| guest
| 15360.0
| 128.0
| 0.00833333333333
| 127.416
|-
| 6.0.0
| guest
| 15360.0
| 127.0
| 0.00826822916667
| 127.416
|-
| 6.0.0
| guest
| 30720.0
| 250.0
| 0.00813802083333
| 251.832
|-
| 6.0.0
| guest
| 30720.0
| 249.0
| 0.00810546875
| 251.832
|}

== Graph ==

The graph is produced in Mac OS X's Grapher app. The equation shown is interpolated via the Affine method built into the app. Two lines are shown; one of the actual observed data and one of the predicted overhead.

[[File:Overheadgraph.jpg|framed|left|VM Memory Size (X) vs VM Memory Overhead Size (Y)]]

XenServer/Overhead

2013-07-23T16:36:30Z

Belliott:

= Xen API driver overhead =

Per-VM virtualization overhead calculations were added to the xenapi driver in this review: https://review.openstack.org/#/c/35380/

== Estimating Overhead ==

The overhead data below is collected from guest instances in the Rackspace cloud via XenAPI.

The data was plotted on a graph as follows:
* X-axis: memory size (MB) of the VM
* Y-axis: memory overhead size (MB) of the VM as observed in XenAPI.

An interpolated equation is fitted over top of the data to predict the overhead based on VM memory size.

== Data ==
The formula to calculate overhead is derived from the following data:

{|class="wikitable" style="text-align: center; color: black;"
! align="left"| XenServer version
! VM type
! VM memory size
! VM memory overhead
! Overhead/MB
! Predicted Overhead
|-
| 6.0.0
| guest
| 512.0
| 7.0
| 0.013671875
| 7.1472
|-
| 6.0.0
| guest
| 512.0
| 6.0
| 0.01171875
| 7.1472
|-
| 6.0.0
| guest
| 1024.0
| 11.0
| 0.0107421875
| 11.2944
|-
| 6.0.0
| guest
| 1024.0
| 10.0
| 0.009765625
| 11.2944
|-
| 6.0.0
| guest
| 2048.0
| 20.0
| 0.009765625
| 19.5888
|-
| 6.0.0
| guest
| 2048.0
| 19.0
| 0.00927734375
| 19.5888
|-
| 6.0.0
| guest
| 4096.0
| 36.0
| 0.0087890625
| 36.1776
|-
| 6.0.0
| guest
| 4096.0
| 35.0
| 0.008544921875
| 36.1776
|-
| 6.0.0
| guest
| 8192.0
| 70.0
| 0.008544921875
| 69.3552
|-
| 6.0.0
| guest
| 8192.0
| 69.0
| 0.0084228515625
| 69.3552
|-
| 6.0.0
| guest
| 15360.0
| 128.0
| 0.00833333333333
| 127.416
|-
| 6.0.0
| guest
| 15360.0
| 127.0
| 0.00826822916667
| 127.416
|-
| 6.0.0
| guest
| 30720.0
| 250.0
| 0.00813802083333
| 251.832
|-
| 6.0.0
| guest
| 30720.0
| 249.0
| 0.00810546875
| 251.832
|}

== Graph ==

The graph is produced in Mac OS X's Grapher app. The equation shown is interpolated via the Affine method built into the app.

[[File:Overheadgraph.jpg|framed|left|VM Memory Size (X) vs VM Memory Overhead Size (Y)]]

File:Overheadgraph.jpg

2013-07-23T16:31:47Z

Belliott: XenServer memory overhead graph

XenServer memory overhead graph

XenServer/Overhead

2013-07-23T16:30:19Z

Belliott:

XenServer/Overhead

2013-07-19T15:32:45Z

Belliott: Created page with "= Xen API driver overhead = Per-VM virtualization overhead calculations were added to the xenapi driver in this review: https://review.openstack.org/#/c/35380/ == Data == Th..."

= Xen API driver overhead =

Per-VM virtualization overhead calculations were added to the xenapi driver in this review: https://review.openstack.org/#/c/35380/

== Data ==
The formula to calculate overhead is derived from the following data:

{|class="wikitable" style="text-align: center; color: black;"
! align="left"| XenServer version
! VM type
! VM memory size
! VM memory overhead
! Overhead/MB
! Predicted Overhead
|-
| 6.0.0
| guest
| 512.0
| 7.0
| 0.013671875
| 7.1472
|-
| 6.0.0
| guest
| 512.0
| 6.0
| 0.01171875
| 7.1472
|-
| 6.0.0
| guest
| 1024.0
| 11.0
| 0.0107421875
| 11.2944
|-
| 6.0.0
| guest
| 1024.0
| 10.0
| 0.009765625
| 11.2944
|-
| 6.0.0
| guest
| 2048.0
| 20.0
| 0.009765625
| 19.5888
|-
| 6.0.0
| guest
| 2048.0
| 19.0
| 0.00927734375
| 19.5888
|-
| 6.0.0
| guest
| 4096.0
| 36.0
| 0.0087890625
| 36.1776
|-
| 6.0.0
| guest
| 4096.0
| 35.0
| 0.008544921875
| 36.1776
|-
| 6.0.0
| guest
| 8192.0
| 70.0
| 0.008544921875
| 69.3552
|-
| 6.0.0
| guest
| 8192.0
| 69.0
| 0.0084228515625
| 69.3552
|-
| 6.0.0
| guest
| 15360.0
| 128.0
| 0.00833333333333
| 127.416
|-
| 6.0.0
| guest
| 15360.0
| 127.0
| 0.00826822916667
| 127.416
|-
| 6.0.0
| guest
| 30720.0
| 250.0
| 0.00813802083333
| 251.832
|-
| 6.0.0
| guest
| 30720.0
| 249.0
| 0.00810546875
| 251.832

XenServer

2013-07-19T15:08:50Z

Belliott:

For general information about [http://www.citrix.com/XenServer XenServer] and [[OpenStack]] see:
* [[XenServer/GettingStarted|Getting Started with XenServer and OpenStack]]
* [[XenServer/XenXCPAndXenServer|Introduction to using Xen, XCP, Kronos and Citrix XenServer with OpenStack]]

To start developing, consider using a Virtual Box based development environment:
* [[XenServer/VirtualBox]]

Information on configuration of [[XenServer]]:
* [[XenServer/PostInstall|Post Install steps to prepare XenServer to work with OpenStack]]
* [[XenServer/NetworkingFlags|Configuring networking with XenServer]]

Other topics:
* [[XenServer/Install|Installing XenServer]]
* [[XenServer/Install/PXE|PXE Installing XenServer]]
* [[XenServer/DevStack|XenServer and DevStack]]
* [[XenServer/VLANManager|XenServer and VLAN networking]]
* [[XenServer/Tempest|XenServer and Tempest]]
* [[QuantumDevstackOvsXcp]]
* [[XenServer/Overhead|XenServer Virtualization Overhead]]

SystemUsageData

2013-04-26T22:01:23Z

Belliott:

== Summary ==
Nova administrators want data on system usage for billing, chargeback, or monitoring purposes.

== Release Note ==
System usage data is emitted from Nova in the form of usage events from Nova's [[NotificationSystem]]. The notifications system can be configured to emit events either through nova's logging facility, or send them to a series of AMQP queues (one per notification priority). System usages are emitted as notification events with the INFO priority. Different types of usage events are distinguished via the notifications' event_type, which is a hierarchical dotted string such as ''compute.instance.create'', which allows usages to be easily grouped for aggregation. Usage notifications can be immediate, created when a specific increment of usage occurs (such as creation of an instance) or periodic, generated by a periodic task, like a cron job, and covering usage for a certain period of time. Besides the standard Nova Notification priority, notification timestamp, and event_type, usage notifications contain a payload of data that will vary depending on the event_type. This is presented as a json-formatted hash of key-value pairs. Some of the keys, such as tenant_id will always be present in any usage notification, others will be data relevent to that event_type (For example, instance related notifications will contain data describing the instance.)

For more indepth examples see [[NotificationEventExamples]].

An external application (https://github.com/rackspace/yagi) is available that can read notifications from AMQP queues and provide an Atom API. This application generates Atom feeds of the notifications, and can publish feeds via PubSubHubbub (PSH) for efficient distribution of data to subscribers.

== Rationale ==
As a systems integrator, we need the ability for 3rd party systems to query usage information from Nova to determine how to charge customers for use of the platform.

Enterprise customers also need the ability to determine usage data, though for different reasons. For example, a cloud administrator may want to determine VMs that have not been used in several month so that they can be reclaimed and the resources reallocated. Some IT departments may implement a quota system for their users.

Note that Nova itself should not be concerned with billing, but it needs to present a feed of usage data that an external application could use to aggregate the billing data for a requested time period.

== User stories ==
# As a systems integrator, I need to retrieve usage data so that I can properly bill my customers.
# As a systems integrator or enterprise cloud administrator, I want to monitor usage data over time so that I can optimize the utilization of my resources.
# As an business or agency that supports multiple projects, I need to account for the resources consumed by each project so that I can properly meet accounting and budget standards.
# As a systems integrator, I need to provide usage data to multiple third-party systems without building custom interfaces to each one so that I can efficiently utilize my manpower.

== Assumptions ==
Usage data can be aggregated by Tenant ID (as defined in the [[openstack-accounting]] blueprint).(i.e. Tenant ID is always present)

There is a billing system that is not a part of [[OpenStack]]. Invoicing, billing, and customer management are handled externally to [[OpenStack]]/Nova. Nova only has knowledge of an "tenant ID" (as per the [[openstack-accounting]] blueprint).

The Tenant ID is the same as the tenant id concept in Keystone, and is known internally to Nova as the "Project ID".

If Nova is installed in multiple regions (aka "zones"), usage data will not span zones; even if the same tenant ID is used in multiple zones, that data must be aggregated external to Nova.

== Design and Implementation ==
There are two events generated for every activity. Previously, events were only generated on activity completion, but now there is an event when the activity starts and, as before, when the activity ends. These are marked as 'compute.instance.<activity>.start' and 'compute.instance.<activity>.end'. The '.end' event is the same as the event previously expected.

=== Event Types and Payload data: ===
==== compute.instance.create.{start,error,end}: ====
* Usage notification upon creation of a new instance.
* '''tenant_id''': Tenant ID that owns the this instance (string)
* '''user_id''': User ID that owns this instance (string)
* '''instance_id''': Nova instance ID of this instance (string)
* '''instance_type''': Name of the instance type ('flavor') of this instance. (string)
* '''instance_type_id''': Nova ID for instance type ('flavor') of this instance. (string)
* '''display_name''': User selected display name for instance.
* '''created_at''': Timestamp for when this instance's record was created in Nova (string, formatted "YYYY-MM-DD hh:mm:ss.ssssss")
* '''launched_at''': Timestamp for when this instance was last launched by hypervisor. (string, formatted "YYYY-MM-DD hh:mm:ss.ssssss")
* '''image_ref_url''': Image URL (from Glance) that this instance was created from. (string)
* '''state''': Current state of instance. (string, such as 'active' or 'deleted')
* '''state_description''': Additional human readable description of current state of instance.
* '''fixed_ips''': list of ip addresses formatted like [{"floating_ips": [], "meta": {}, "type": "fixed", "version": 4, "address": "10.0.0.9", "label": "public"}] assigned to instance.
* '''memory_mb''': memory allocation for this instance (in mb)
* '''disk_gb''': disk allocation for this instance (in gb)
* '''message''': High-level message describing the notification. If the event type is "create.error", it will contain error details.

==== compute.instance.delete.start/.end: ====
* Usage notification upon deletion of an instance.
* '''tenant_id''': Tenant ID that owns the this instance (string)
* '''user_id''': User ID that owns this instance (string)
* '''instance_id''': Nova instance ID of this instance (string)
* '''instance_type''': Name of the instance type ('flavor') of this instance. (string)
* '''instance_type_id''': Nova ID for instance type ('flavor') of this instance. (string)
* '''display_name''': User selected display name for instance.
* '''created_at''': Timestamp for when this instance's record was created in Nova (string, formatted "YYYY-MM-DD hh:mm:ss.ssssss")
* '''launched_at''': Timestamp for when this instance was last launched by hypervisor. (string, formatted "YYYY-MM-DD hh:mm:ss.ssssss")
* '''image_ref_url''': Image URL (from Glance) that this instance was created from. (string)
* '''state''': Current state of instance. (string, such as 'active' or 'deleted')
* '''state_description''': Additional human readable description of current state of instance.
* '''fixed_ips''': list of ip addresses formatted like [{"floating_ips": [], "meta": {}, "type": "fixed", "version": 4, "address": "10.0.0.9", "label": "public"}] assigned to instance.
* '''memory_mb''': memory allocation for this instance (in mb)
* '''disk_gb''': disk allocation for this instance (in gb)

==== compute.instance.rebuild.start/.end: ====
* Usage notification upon rebuild of instance.
* '''tenant_id''': Tenant ID that owns the this instance (string)
* '''user_id''': User ID that owns this instance (string)
* '''instance_id''': Nova instance ID of this instance (string)
* '''instance_type''': Name of the instance type ('flavor') of this instance. (string)
* '''instance_type_id''': Nova ID for instance type ('flavor') of this instance. (string)
* '''display_name''': User selected display name for instance.
* '''created_at''': Timestamp for when this instance's record was created in Nova (string, formatted "YYYY-MM-DD hh:mm:ss.ssssss")
* '''launched_at''': Timestamp for when this instance was last launched by hypervisor. (string, formatted "YYYY-MM-DD hh:mm:ss.ssssss")
* '''image_ref_url''': Image URL (from Glance) that this instance is being rebuilt from. (string)
* '''state''': Current state of instance. (string, such as 'active' or 'deleted')
* '''state_description''': Additional human readable description of current state of instance.
* '''fixed_ips''': list of ip addresses formatted like [{"floating_ips": [], "meta": {}, "type": "fixed", "version": 4, "address": "10.0.0.9", "label": "public"}] assigned to instance.
* '''memory_mb''': memory allocation for this instance (in mb)
* '''disk_gb''': disk allocation for this instance (in gb)

==== compute.instance.resize.prep.start/.end: ====
* Usage notification when a resize/migrate of an instance is begun.
* '''tenant_id''': Tenant ID that owns the this instance (string)
* '''user_id''': User ID that owns this instance (string)
* '''instance_id''': Nova instance ID of this instance (string)
* '''instance_type''': Name of the current (before resize) instance type ('flavor') of this instance. (string)
* '''instance_type_id''': Nova ID for current (before resize) instance type ('flavor') of this instance. (string)
* '''new_instance_type''': Name of the new (after resize) instance type ('flavor') of this instance. (string)
* '''new_instance_type_id''': Nova ID for new (after resize) instance type ('flavor') of this instance. (string)
* '''display_name''': User selected display name for instance.
* '''created_at''': Timestamp for when this instance's record was created in Nova (string, formatted "YYYY-MM-DD hh:mm:ss.ssssss")
* '''launched_at''': Timestamp for when this instance was last launched by hypervisor. (string, formatted "YYYY-MM-DD hh:mm:ss.ssssss")
* '''image_ref_url''': Image URL (from Glance) that this instance was created from. (string)
* '''state''': Current state of instance. (string, such as 'active' or 'deleted')
* '''state_description''': Additional human readable description of current state of instance.
* '''fixed_ips''': list of ip addresses formatted like [{"floating_ips": [], "meta": {}, "type": "fixed", "version": 4, "address": "10.0.0.9", "label": "public"}] assigned to instance.
* '''memory_mb''': memory allocation for this instance (in mb)
* '''disk_gb''': disk allocation for this instance (in gb)

==== compute.instance.resize.confirm.start/.end: ====
* Usage notification when a resize/migration of an instance has been confirmed by the user.
* '''tenant_id''': Tenant ID that owns the this instance (string)
* '''user_id''': User ID that owns this instance (string)
* '''instance_id''': Nova instance ID of this instance (string)
* '''instance_type''': Name of the current instance type ('flavor') of this instance. (i.e. the one it has been resized to.) (string)
* '''instance_type_id''': Nova ID for the current instance type ('flavor') of this instance. (i.e. the one it has been resized to.) (string)
* '''display_name''': User selected display name for instance.
* '''created_at''': Timestamp for when this instance's record was created in Nova (string, formatted "YYYY-MM-DD hh:mm:ss.ssssss")
* '''launched_at''': Timestamp for when this instance was last launched by hypervisor. (string, formatted "YYYY-MM-DD hh:mm:ss.ssssss")
* '''image_ref_url''': Image URL (from Glance) that this instance was created from. (string)
* '''state''': Current state of instance. (string, such as 'active' or 'deleted')
* '''state_description''': Additional human readable description of current state of instance.
* '''fixed_ips''': list of ip addresses formatted like [{"floating_ips": [], "meta": {}, "type": "fixed", "version": 4, "address": "10.0.0.9", "label": "public"}] assigned to instance.
* '''memory_mb''': memory allocation for this instance (in mb)
* '''disk_gb''': disk allocation for this instance (in gb)

==== compute.instance.resize.revert.start/.end: ====
* Usage notification when a resize/migration of an instance has been reverted by the user.
* '''tenant_id''': Tenant ID that owns the this instance (string)
* '''user_id''': User ID that owns this instance (string)
* '''instance_id''': Nova instance ID of this instance (string)
* '''instance_type''': Name of the current instance type ('flavor') of this instance. (i.e. the one it has been reverted back to.) (string)
* '''instance_type_id''': Nova ID for the current instance type ('flavor') of this instance. (i.e. the one it has been reverted back to.) (string)
* '''display_name''': User selected display name for instance.
* '''created_at''': Timestamp for when this instance's record was created in Nova (string, formatted "YYYY-MM-DD hh:mm:ss.ssssss")
* '''launched_at''': Timestamp for when this instance was last launched by hypervisor. (string, formatted "YYYY-MM-DD hh:mm:ss.ssssss")
* '''image_ref_url''': Image URL (from Glance) that this instance was created from. (string)
* '''state''': Current state of instance. (string, such as 'active' or 'deleted')
* '''state_description''': Additional human readable description of current state of instance.
* '''fixed_ips''': list of ip addresses formatted like [{"floating_ips": [], "meta": {}, "type": "fixed", "version": 4, "address": "10.0.0.9", "label": "public"}] assigned to instance.
* '''memory_mb''': memory allocation for this instance (in mb)
* '''disk_gb''': disk allocation for this instance (in gb)

==== compute.instance.exists: ====
There is no .start/.end event for this activity ... just the 'compute.instance.exists' event.

* Periodic usage notification generated by the instance-usage-audit cron job. These usages are generated for '''each''' instance that was active during the specified audit period.
* '''tenant_id''': Tenant ID that owns the this instance (string)
* '''user_id''': User ID that owns this instance (string)
* '''instance_id''': Nova instance ID of this instance (string)
* '''instance_type''': Name of the instance type ('flavor') of this instance. (string)
* '''instance_type_id''': Nova ID for instance type ('flavor') of this instance. (string)
* '''display_name''': User selected display name for instance.
* '''created_at''': Timestamp for when this instance's record was created in Nova (string, formatted "YYYY-MM-DD hh:mm:ss.ssssss")
* '''launched_at''': Timestamp for when this instance was last launched by hypervisor. (string, formatted "YYYY-MM-DD hh:mm:ss.ssssss")
* '''image_ref_url''': Image URL (from Glance) that this instance was created from. (string)
* '''image_meta''': Dictionary of key-value pairs representing metadata from the image the instance was built from.
* '''audit_period_begining''': Timestamp of beginning of audit period. (string, formatted "YYYY-MM-DD hh:mm:ss.ssssss")
* '''audit_period_ending''': Timestamp of end of audit period. (string, formatted "YYYY-MM-DD hh:mm:ss.ssssss")
* '''bandwidth''': Hash listing bandwidth used for each network this instance is connected to. Keys will be network labels, values will be hashes containing the keys ''''bw_in'''', and ''''bw_out'''', listing the incoming, and outgoing bandwith, respectively, used by that instance, on that network, for that audit period. Bandwith is reported in bytes.
* '''state''': Current state of instance. (string, such as 'active' or 'deleted')
* '''state_description''': Additional human readable description of current state of instance.
* '''fixed_ips''': list of ip addresses formatted like [{"floating_ips": [], "meta": {}, "type": "fixed", "version": 4, "address": "10.0.0.9", "label": "public"}] assigned to instance.
* '''memory_mb''': memory allocation for this instance (in mb)
* '''disk_gb''': disk allocation for this instance (in gb)

==== compute.instance.update: ====

This event is generated whenever the VM or task state of the server changes; for example, when the server moves from BUILDING to either ACTIVE or ERROR.

This notification is disabled by default.

To enable notifications when the VM state changes, set configuration flag "notify_on_state_change" to "vm_state".

To enable notifications when either VM state or task state changes, set configuration flag "notify_on_state_change" to "vm_and_task_state".

* '''tenant_id''': Tenant ID that owns the this instance (string)
* '''user_id''': User ID that owns this instance (string)
* '''instance_id''': Nova instance ID of this instance (string)
* '''instance_type''': Name of the instance type ('flavor') of this instance. (string)
* '''instance_type_id''': Nova ID for instance type ('flavor') of this instance. (string)
* '''display_name''': User selected display name for instance.
* '''created_at''': Timestamp for when this instance's record was created in Nova (string, formatted "YYYY-MM-DD hh:mm:ss.ssssss")
* '''launched_at''': Timestamp for when this instance was last launched by hypervisor. (string, formatted "YYYY-MM-DD hh:mm:ss.ssssss")
* '''image_ref_url''': Image URL (from Glance) that this instance was created from. (string)
* '''image_metadata''': array of key-value pairs representing the metadata from the image from which the instance was built (array)
* '''audit_period_begining''': Timestamp of beginning of audit period. (string, formatted "YYYY-MM-DD hh:mm:ss.ssssss")
* '''audit_period_ending''': Timestamp of end of audit period. (string, formatted "YYYY-MM-DD hh:mm:ss.ssssss")
* '''bandwidth''': Hash listing bandwidth used for each network this instance is connected to. Keys will be network labels, values will be hashes containing the keys ''''bw_in'''', and ''''bw_out'''', listing the incoming, and outgoing bandwith, respectively, used by that instance, on that network, for that audit period. Bandwith is reported in bytes.
* '''old_state''': Prior state of instance. (string, such as 'active' or 'deleted')
* '''state''': Current state of instance. (string, such as 'active' or 'deleted')
* '''state_description''': Additional human readable description of current state of instance.
* '''fixed_ips''': list of ip addresses formatted like [{"floating_ips": [], "meta": {}, "type": "fixed", "version": 4, "address": "10.0.0.9", "label": "public"}] assigned to instance.
* '''memory_mb''': memory allocation for this instance (in mb)
* '''disk_gb''': disk allocation for this instance (in gb)

==== volume.create.start/.end: ====
* Usage notification upon creation of a new volume.
* '''tenant_id''': Tenant ID that owns the this volume (string)
* '''user_id''': User ID that owns this volume (string)
* '''volume_id''': Nova volume ID of this volume (string)
* '''volume_type''': Name of the volume type of this volume. (string)
* '''display_name''': User selected display name for volume.
* '''created_at''': Timestamp for when this volume's record was created in Nova (string, formatted "YYYY-MM-DD hh:mm:ss.ssssss")
* '''launched_at''': Timestamp for when this instance was last launched by hypervisor. (string, formatted "YYYY-MM-DD hh:mm:ss.ssssss")
* '''status''': Current status of volume. (string, such as 'available' or 'attached')
* '''size''': size of disk allocation for this volume.
* '''snapshot_id''': Nova id of the volume snapshot.

==== volume.delete.start/.end: ====
* Usage notification upon deletion of a volume.
* '''tenant_id''': Tenant ID that owns the this volume (string)
* '''user_id''': User ID that owns this volume (string)
* '''volume_id''': Nova volume ID of this volume (string)
* '''volume_type''': Name of the volume type of this volume. (string)
* '''display_name''': User selected display name for volume.
* '''created_at''': Timestamp for when this volume's record was created in Nova (string, formatted "YYYY-MM-DD hh:mm:ss.ssssss")
* '''launched_at''': Timestamp for when this instance was last launched by hypervisor. (string, formatted "YYYY-MM-DD hh:mm:ss.ssssss")
* '''status''': Current status of volume. (string, such as 'available' or 'attached')
* '''size''': size of disk allocation for this volume.
* '''snapshot_id''': Nova id of the volume snapshot.

==== volume.exists: ====
There is no .start/.end event for this activity ... just the 'volume.exists' event.

* Periodic usage notification generated by the volume-usage-audit cron job. These usages are generated for '''each''' volume that was active during the specified audit period.
* '''tenant_id''': Tenant ID that owns the this volume (string)
* '''user_id''': User ID that owns this volume (string)
* '''volume_id''': Nova volume ID of this volume (string)
* '''volume_type''': Name of the volume type of this volume. (string)
* '''display_name''': User selected display name for volume.
* '''created_at''': Timestamp for when this volume's record was created in Nova (string, formatted "YYYY-MM-DD hh:mm:ss.ssssss")
* '''launched_at''': Timestamp for when this instance was last launched by hypervisor. (string, formatted "YYYY-MM-DD hh:mm:ss.ssssss")
* '''audit_period_begining''': Timestamp of beginning of audit period. (string, formatted "YYYY-MM-DD hh:mm:ss.ssssss")
* '''audit_period_ending''': Timestamp of end of audit period. (string, formatted "YYYY-MM-DD hh:mm:ss.ssssss")
* '''status''': Current status of volume. (string, such as 'available' or 'attached')
* '''size''': size of disk allocation for this volume.
* '''snapshot_id''': Nova id of the volume snapshot.

=== additional events ===
These are generated for operations monitoring/profiling purposes and shouldn't directly affect usage/billing. They all follow the same signature as the above calls. Some of these notifications may be called from the above operations (for example, shutdown from delete).

==== compute.instance.shutdown.start/.end ====
==== compute.instance.power_off.start/.end ====
==== compute.instance.power_on.start/.end ====
==== compute.instance.snapshot.start/.end ====
==== compute.instance.resize.start/.end ====
==== compute.instance.finish_resize.start/.end ====

Design Summit/Havana/Etherpads

2013-04-14T16:17:54Z

Belliott:

(Please add links to the Havana Design Summit etherpad documents here)

=== Documentation ===

* Monday 2013-04-15 11:00 - [https://etherpad.openstack.org/restructure-docs-session Restructure documentation]
* Monday 2013-04-15 11:50 - [https://etherpad.openstack.org/openstack-ops-session Review Operations Manual and Plan Future Work]
* Monday 2013-04-15 13:50 - [https://etherpad.openstack.org/openstack-docs-translation Translation management enhancement]
* Monday 2013-04-15 14:40 - [https://etherpad.openstack.org/openstack-docs-new-projects Documentation for Newly Integrated Projects]

=== Heat ===

* [https://etherpad.openstack.org/heat-multiple-engines 9:50am How to run multiple heat-engines and scaling]
* [https://etherpad.openstack.org/heat-credentials-management 11:00am Heat credentials management/delegation]
* 11:50am Adding support for OASIS TOSCA to Heat
* 1:50pm Abstracting the AWS out of Heat
* [https://etherpad.openstack.org/heat-autoscaling 2:40pm Autoscaling API for Heat]
* 3:40pm Concurrent Resource Scheduling in Heat
* 4:30pm Rolling Updates and Instance Specific Metadata

=== Horizon ===

* [https://etherpad.openstack.org/HavanaHorizonTesting 11:00am Testing in Horizon]
* 11:50am Keystone v3 and multi endpoint/region support
* 1:50pm Quantum/Horizon integration
* 2:40pm Heat GUI integration into Horizon
* 3:40pm Ceilometer/Horizon Integration
* 4:30pm Are we ready for real-time data in Horizon?
* 5:20pm BoF: Building on Horizon

=== Keystone ===
* [https://etherpad.openstack.org/havana-saml-oauth-scim SAML, OAuth 2, and SCIM]
* [https://etherpad.openstack.org/havana-external-auth Generic support for external authn/authz]
* [https://etherpad.openstack.org/havana-keystone-performance Scaling & Performance]
* [https://etherpad.openstack.org/havana-availability-zone-and-region-management Availability zone and region management]
* [https://etherpad.openstack.org/havana-key-manager Key Manager]
* [https://etherpad.openstack.org/havana-ldap-integration LDAP Integration]
* [https://etherpad.openstack.org/HavanaCentralizedQuotas Centralized Quotas]
* [https://etherpad.openstack.org/havana-endpoint-filtering Endpoint Filtering]
* [https://etherpad.openstack.org/havana-fine-grained-access-control Fine-grained access control]

=== Nova ===
* [https://etherpad.openstack.org/ZeroDowntimeUpgrades Zero down upgrades]
* [https://etherpad.openstack.org/HavanaContinuingRoadToLiveUpgrade The (continuing) road to live upgrade]
* [https://etherpad.openstack.org/HavanaVersionedInternalObjects Versioned internal objects]
* [https://etherpad.openstack.org/HavanaDatabaseRedux Database Status Redux]
* [https://etherpad.openstack.org/HavanaTripleO Openstack-on-Openstack]
* [https://etherpad.openstack.org/HavanaBaremetalNextSteps Baremetal Driver Next Steps]
* [https://etherpad.openstack.org/roadmap-for-ceph-integration-with-openstack Roadmap for Ceph integration with Openstack]
* [https://etherpad.openstack.org/NovaAPIExtensionFramework Nova API Extension framework]
* [https://etherpad.openstack.org/NovaV3APIHavana Nova v3 API]
* [https://etherpad.openstack.org/NovaApiValidationFramework Nova API Validation Framework]
* [https://etherpad.openstack.org/UnifiedResourcePlacement Unified Resource Placement Module]
* [https://etherpad.openstack.org/HavanaSchedulerFeatures Scheduler Features]
* [https://etherpad.openstack.org/HavanaMultiHypervisor Multi-hypervisor clouds]
* [https://etherpad.openstack.org/HavanaLongLivedPeriodicTasks Long lived periodic tasks]
* [https://etherpad.openstack.org/API_for_block_devices Richer and more flexible APIs for block devices]

=== Oslo ===

* [https://etherpad.openstack.org/havana-oslo Oslo Status and Plans]
* [https://etherpad.openstack.org/havana-common-wsgi Pecan/WSME Status]
* [https://etherpad.openstack.org/HavanaNoDowntimeDBMigrations No-downtime DB migrations]
* [https://etherpad.openstack.org/HavanaRootwrap Rootwrap improvements for the Havana cycle]
* [https://etherpad.openstack.org/havana-oslo-packaging-and-hacking Common packaging support and code analysis tools]
* [https://etherpad.openstack.org/havana-rpc-api-review RPC API review]
* [https://etherpad.openstack.org/havana-rpc-zmq-for-ceilometer-and-quantum ZeroMQ RPC for Ceilometer and Quantum]
* [https://etherpad.openstack.org/havana-rpc-access-control Message queue access control]
* [https://etherpad.openstack.org/havana-rpc-signing-and-encryption RPC Message Signing and Encryption]
* [https://etherpad.openstack.org/havana-zipkin-tracing Zipkin tracing in OpenStack]
* [https://etherpad.openstack.org/havana-oslo-i18n-strategy i18n strategy for OpenStack services]
* [https://etherpad.openstack.org/havana-common-xenapi-library Common XenAPI libary]

=== Quantum ===
* [https://etherpad.openstack.org/HavanaVPNaaS VPNaaS]
* [https://etherpad.openstack.org/havana-quantum-lbaas LBaaS]
* [https://etherpad.openstack.org/HavanaBareMetalVLAN BareMetal VLAN]
* [https://etherpad.openstack.org/HavanaHADHCP HA DHCP]
* [https://etherpad.openstack.org/quantum-qos-api QoS API]
* [https://etherpad.openstack.org/Quantum_Firewall_As_A_Service FWaaS]
* [https://etherpad.openstack.org/Quantum_Services'_Chaining_and_Insertion Services' Chaining & Insertion]
* [https://etherpad.openstack.org/havana-networking-db-improvements Openstack networking DB improvements]
* [https://etherpad.openstack.org/havana-quantum-unittests Reviewing Openstack networking unit tests]

=== QA ===
'''Tuesday'''
* [https://etherpad.openstack.org/havana-tempest-scope Tempest Scope]
* [https://etherpad.openstack.org/havana-gating-strategies Strategies for Gating in a growing project]
* [https://etherpad.openstack.org/havana-multi-node multi-node openstack testing]
* [https://etherpad.openstack.org/havana-fits-testing FITS testing of public clouds]
* [https://etherpad.openstack.org/havana-gating-validation Gating/Validation of OpenStack Deployments]
* [https://etherpad.openstack.org/havana-upgrade-testing Upgrade testing and Grenade]
* [https://etherpad.openstack.org/havana-beyond-api Beyond the API - End to End Testing of OpenStack]

'''Wednesday'''
* [https://etherpad.openstack.org/havana-adding-integrated Adding new integrated projects to Tempest]
* [https://etherpad.openstack.org/havana-testr-feedback testr / testtools feedback/next-steps]
* [https://etherpad.openstack.org/havana-tempest-best-practice-guide Tempest Best Practice Guide]
* [https://etherpad.openstack.org/havan-rackspace-engine Rackspace testing engine Case Study/Overview]
* [https://etherpad.openstack.org/havana-gap-analysis Tempest - Gap Analysis - Identify new testsdevelop]

=== Swift ===
'''Monday'''

* [https://etherpad.openstack.org/swift-openstack-havana Swift and OpenStack what's next]
* [http://etherpad.openstack.org/HavanaSwiftBenchmarking Benchmarking Swift]
* [https://etherpad.openstack.org/swift_api Swift API Cleanup]

'''Tuesday (scheduled on-site for the unconference)'''
* [http://etherpad.openstack.org/Multi_namespaces Swift Multi Namespaces, Swift Multi - Tenants]
* [http://etherpad.openstack.org/Acls_metadata Extending Swift ACLs and Metadata]

=== Process / Infra ===
* Calling all Interns and Mentors: Ideas to improve
* Python3 in OpenStack
* [https://etherpad.openstack.org/havana-stable-branch Stable Branch]
* [http://etherpad.openstack.org/HavanaVulnerabilityManagement Vulnerability management: infra needs, scoring...]
* [http://etherpad.openstack.org/TCMembershipEvolution Technical Committee membership evolution]
* Continuous-deployment for upstream Openstack
* Dependency Management
* Sorting out test runners, wrappers and venvs B119
* Failure management in the gate
* Bare Metal Testing
* OpenStack CI logging
* [http://etherpad.openstack.org/NonInfraManagedTooling Review of non-infra-managed tooling]
* [http://etherpad.openstack.org/ProjectsReNaming Projects (re)naming]
* [http://etherpad.openstack.org/HavanaReleaseSchedule Havana release schedule]
* [http://etherpad.openstack.org/HavanaContinuousDeployment Continuous Deployment]

=== Ceilometer ===

'''Wednesday'''

* [https://etherpad.openstack.org/havana-ceilometer-architecture-overview Intro to Ceilometer Architecture]
* Feedback from ceilometer users

'''Thursday'''

* [https://etherpad.openstack.org/havana-ceilometer-incremental-grab-bag Incremental improvement grab-bag]
* Double Entry Auditing of collected metrics in CM
* API improvements for ceilometer
* Logical combination of alarm states
* Time series data manipulation in nosql stores
* Simple messaging action for Alerting
* Alarm state and history management
* Ceilometer support for advanced billing models
* Supporting rich data types and aggregation

=== Glance ===
* [https://etherpad.openstack.org/havana-getting-glance-ready-for-public-clouds Getting glance ready for Public Clouds]

=== Cinder ===

'''Wednesday'''
* [https://etherpad.openstack.org/Cinder-Havana-Refactor-Attach-Code Refactor Cinder Attach Code]
* [https://etherpad.openstack.org/havana-cinder-api2-a-new-hope API V2 a new hope for better validation]

'''Thursday'''
* [https://etherpad.openstack.org/summit-havana-cinder-multi-attach-and-ro-volumes Multi-Attach and Read Only Volumes]
* [https://etherpad.openstack.org/Summit-Havana-Cinder-Safe-Shutdown Safe Shutdown]