OpenStack - User contributions [en]

Stackforge Namespace Retirement

2015-09-30T20:49:42Z

Monsyne Dragon: /* Active Projects to Move */

=== Background ===

The stackforge/ git namespace is being retired, and active projects are being moved to the openstack/ namespace. See this mailing list post for [http://lists.openstack.org/pipermail/openstack-dev/2015-August/071816.html full background]. These changes are scheduled to occur on October 17, 2015.

There are two lists below, one for active projects that should be moved, and a second for inactive projects that should become read-only. Please update them to add your project to the correct list.

=== Active Projects to Move ===
Active stackforge projects that wish to move into the openstack/ namespace should be added to this list. Projects in this list will be automatically moved by the Infrastructure team to openstack/ on October 17. Please note that no other renames can happen during this move -- projects will strictly be moved from stackforge/ to openstack/ with their existing names.

* aeromancer
* anvil
* bansho
* blazar
* blazar-nova
* cathead
* ceilometer-powervm
* ceilometer-zvm
* cerberus
* cerberus-dashboard
* cl-openstack-client
* cloudcafe
* cloud-init
* cloudbase-init
* clouddocs-maven-plugin
* cloudkitty
* cloudkitty-dashboard
* cloudpulse
* cloudroast
* cloudv-ostf-adapter
* cognitive
* compass-adapters
* compass-core
* compass-specs
* compass-web
* compute-hyperv
* designate-msdnsagent
* devstack-plugin-glusterfs
* devstack-plugin-sheepdog
* doc8
* dox
* drbd-devstack
* driverlog
* ec2-api
* ec2-driver
* entropy
* faafo
* flame
* freezer
* freezer-api
* freezer-web-ui
* fuel-agent
* fuel-astute
* fuel-dev-tools
* fuel-devops
* fuel-docs
* fuel-library
* fuel-main
* fuel-mirror
* fuel-nailgun-agent
* fuel-octane
* fuel-ostf
* fuel-ostf-plugin
* fuel-plugin-availability-zones
* fuel-plugin-calamari
* fuel-plugin-calico
* fuel-plugin-ceilometer-redis
* fuel-plugin-cinder-netapp
* fuel-plugin-cisco-aci
* fuel-plugin-contrail
* fuel-plugin-dbaas-trove
* fuel-plugin-detach-database
* fuel-plugin-detach-keystone
* fuel-plugin-detach-rabbitmq
* fuel-plugin-elasticsearch-kibana
* fuel-plugin-external-emc
* fuel-plugin-external-glusterfs
* fuel-plugin-external-zabbix
* fuel-plugin-glance-nfs
* fuel-plugin-ha-fencing
* fuel-plugin-influxdb-grafana
* fuel-plugin-ironic
* fuel-plugin-ldap
* fuel-plugin-lma-collector
* fuel-plugin-lma-infrastructure-alerting
* fuel-plugin-mellanox
* fuel-plugin-midonet
* fuel-plugin-neutron-fwaas
* fuel-plugin-neutron-lbaas
* fuel-plugin-neutron-vpnaas
* fuel-plugin-nova-nfs
* fuel-plugin-nsxv
* fuel-plugin-opendaylight
* fuel-plugin-saltstack
* fuel-plugin-solidfire-cinder
* fuel-plugin-swiftstack
* fuel-plugin-tintri-cinder
* fuel-plugin-tls
* fuel-plugin-vmware-dvs
* fuel-plugin-vxlan
* fuel-plugin-zabbix-monitoring-emc
* fuel-plugin-zabbix-monitoring-extreme-networks
* fuel-plugin-zabbix-snmptrapd
* fuel-plugins
* fuel-qa
* fuel-specs
* fuel-stats
* fuel-upgrade
* fuel-web
* gce-api
* gerrit-dash-creator
* git-upstream
* golang-client
* group-based-policy
* group-based-policy-automation
* group-based-policy-specs
* group-based-policy-ui
* intel-nfv-ci-tests
* merlin
* monasca-agent
* monasca-api
* monasca-ceilometer
* monasca-common
* monasca-log-api
* monasca-notification
* monasca-persister
* monasca-statsd
* monasca-thresh
* monasca-ui
* monasca-vagrant
* monitoring-for-openstack
* namos
* nerd-reviewer
* networking-6wind
* networking-bagpipe-l2
* networking-brocade
* networking-hyperv
* networking-ovs-dpdk
* networking-zvm
* nova-docker
* nova-powervm
* nova-solver-scheduler
* nova-zvm-virt-driver
* ooi
* opencafe
* ops-tags-team
* osprofiler
* ospurge
* packstack
* poppy
* proliantutils
* puppet-autossh
* puppet-ceph
* puppet-n1k-vsm
* puppet-setproxy
* puppet-surveil
* python-blazarclient
* python-cerberusclient
* python-cloudkittyclient
* python-cloudpulseclient
* python-cognitiveclient
* python-fuelclient
* python-group-based-policy-client
* python-jenkins
* python-monascaclient
* python-openstacksdk
* python-rackclient
* python-senlinclient
* python-sticksclient
* python-surveilclient
* python-tackerclient
* python-watcherclient
* rack
* requests-mock
* sahara-ci-config
* senlin
* senlin-dashboard
* shaker
* sqlalchemy-migrate
* surveil
* surveil-specs
* stackalytics
* stacktach
* stacktach-klugman
* stacktach-notification-utils
* stacktach-notigen
* stacktach-quince
* stacktach-quincy
* stacktach-sandbox
* stacktach-shoebox
* stacktach-simport
* stacktach-stackdistiller
* stacktach-timex
* stacktach-winchester
* sticks
* sticks-dashboard
* swift3
* swiftonfile
* swift-ceph-backend
* tacker
* tacker-horizon
* tacker-specs
* tap-as-a-service
* telcowg-usecases
* terracotta
* third-party-ci-tools
* tricircle
* turbo-hipster
* vmtp
* watcher
* wsme
* xenapi-os-testing
* xstatic-d3
* xstatic-angular-sanitize
* xstatic-bootstrap-datepicker
* xstatic-angular-gettext
* xstatic-bootswatch
* xstatic-angular-cookies
* xstatic-bootstrap-scss
* xstatic-angular-smart-table
* xstatic-angular-fileupload
* xstatic-angular-bootstrap
* xstatic-angular
* xstatic-angular-mock
* xstatic-angular-lrdragndrop
* xstatic-roboto-fontface
* xstatic-jquery.quicksearch
* xstatic-mdi
* xstatic-rickshaw
* xstatic-magic-search
* xstatic-font-awesome
* xstatic-hogan
* xstatic-spin
* xstatic-jasmine
* xstatic-jquery-migrate
* xstatic-jquery-migrate
* xstatic-jsencrypt
* yaql

=== Inactive Projects to Retire ===
Inactive projects that should be retired should be added to this list. These projects will have a commit merged removing their content and replacing it with a message indicating the project is no longer maintained and will become read-only in Gerrit.

* cachemonkey
* compass-monit
* congressmiddleware
* fuel-provision
* fuel-tasklib
* mercador-pub
* mercador-sub
* MRaaS
* networking-portforwarding
* openstackdroid
* python-mercadorclient
* rubick
* sahara-guestagent
* libra
* logaas
* python-libraclient
* python-rallyclient
* cookbook-pacemaker
* puppet-openstack
* puppet-openstack_dev_env
* puppet_openstack_builder
* puppet-openstack-cloud
* tripleo-ansible
* kickstack
* packstack-vagrant
* haos
* novaimagebuilder
* radar

Ceilometer/blueprints/event-triggers

2013-09-30T21:02:32Z

Monsyne Dragon: Design of the Event Triggers system for Ceilometer

= Introduction: =
Triggers are designed to make it easy to aggregate data from events that occur over time. Many processes in OpenStack will create Notifications as they proceed. These will be represented as Events within Ceilometer. Such processes may take place over a long period of time, possibly hours, and the notifications may be emitted from many different OpenStack nodes, and even different systems within OpenStack, such as Compute, or Image storage. They may be processed by multiple different Collector nodes in a Ceilometer installation, and due to the trade-offs of reliable messaging within AMQP, may not be received in order.

In order to properly aggregate statistics from these sets of events, we need to collect a set of events we are interested in over time, once we have the ones we need, we need to order them by their timestamps, and process them as a batch. If we never receive all of the events we need, such as when there is a failure in the process we are trying to gather information on, we will need to expire the list we have collected, and possibly perform alternate processing, such as updating failure statistics.

Triggers accomplish this. They allow for collecting a list of relevant events over a period of time, detecting when we have all of the events we need, properly ordering them, then processing them using an event pipeline (as defined in the Event Pipelines blueprint: https://blueprints.launchpad.net/ceilometer/+spec/notification-pipelines). They also allow for processing of what events we may have collected through an alternate event pipeline if we fail to collect all of the events we need in an appropriate period of time.

= Terminology: =
== Trigger definition: ==
A configuration for a class of triggers. The definition acts as a template that trigger instances are created from. It defines the name, distinguished_by traits, pipelines, expiration, matching criteria, etc.

== Trigger instance: ==
A specific instance of a trigger, persisted in the datastore, with a list of events. It has:

* '''first_event''': The timestamp of the earliest event in the event list.
* '''last_event''': The timestamp of the latest event in the event list. This is updated when a new event is matched.
* '''expire_timestamp''': The actual time when this trigger instance will expire. It is derived from either the first_event timestamp or the last_event timestamp via the time expression set in the definition's expiration field. It is updated if the timestamp it's based on changes.
* '''fire_timestamp''': The actual time when the trigger will fire. This is set based on the current time, plus any fire_delay, if the trigger's firing criteria are met.
* '''name''': The name of the trigger from it's definition.
* '''state''': Triggers can be in ''ACTIVE'', ''READY_TO_FIRE'', ''READY_TO_EXPIRE'', ''FIRING'', ''EXPIRING'', ''ERROR'', or ''EXPIRE_ERROR'' states.
* and list of '''distinguished_by''' traits and values.

== Time expression: ==
A simple DSL, like a regex for time comparisons. Time expressions can be absolute like ''first@00:00'' (meaning midnight on the day of the first event), or relative, like ''last+3h ''(3 hours after the last event). They can be based on the last event's timestamp or the first event's timestamp. These can be used to set trigger expiration, or matching values for datetime traits. For matches, you can use a single time expression to match a specific timestamp, or a time range (which is just two time expressions separated by “to”, such as “''first@00:00 to last+2h''” )

== Criteria: ==
A description of a pattern to match events against. These are basically a hash which includes: '''event_type''', which can be wildcarded, '''timestamp''', which can be matched with a time expression, and a list of trait names and values, which can be compared to constants, or expressions. Fire criteria can also have '''number''', which is the number of events matching a given criteria that must be in the trigger's list of events for it to fire. Criteria are used for: ''match_criteria'', which determine what events to collect, ''fire_criteria'', which determine when we have the events we need to fire, and ''load_criteria'', which describe historical events we may need to load from the datastore.

== Event pipeline: ==
A pipeline for processing events (see [https://blueprints.launchpad.net/ceilometer/+spec/notification-pipelines https://blueprints.launchpad.net/ceilometer/+spec/notification-pipelines]) Once we have the needed events, the event pipelines do the actual processing. Triggers can send events to a pipeline listed as a ''fire_pipeline'', when the triggers fire criteria is matched, or optionally, an ''expire_pipeline'', if the trigger expires without firing.

= Defining the triggers: =
A trigger definition has the following fields:

* '''name''': A unique name to identify the trigger
* '''distinguished_by''': A list of traits which distinguish this trigger. A unique trigger instance (with it's own list of events) is created for each unique combination of distinguishing traits. If trigger “''instance_create”'' is distinguished by ''instance_id'', then there will be a separate “''instance_create''" trigger instance created for each unique ''instance_id'' in events that match the trigger's criteria.
* '''expiration''': Time expression, sets when the trigger expires. can be specified relative to the first (earliest) event in the event list or the last (latest) For example, if the expiration is set to "first+4h" , the trigger will expire 4 hours after the first event matching its match criteria is received. If it's set to "last+2h", the trigger will expire 2 hours after the last event matching it's criteria is received (and it's expiration clock will be reset every time it receives an event)
* '''fire_delay''': Number of seconds to wait after the trigger is ready to fire, before it actually fires. (This allows collection of any out-of-order 'straggler' events.) Defaults to 0
* '''match_criteria''': A list of criteria for events to match. Each matched event will be placed on the trigger's event list. An event matches if it matches any of the criteria. Each criteria consists of:
:* '''event_type''': Type of event to match. May be wild carded using glob-style wildcards.
:* '''timestamp''': Time range of event timestamp to match
:* '''traits''': Hash of traits to match. hash is trait_name: value. Value may be a constant or an expression, such as:
::* "> 10"
::* <nowiki>"< 1" </nowiki>
::* "is NULL" (event does NOT have the trait)
::* "is not NULL" (event has the trait, regardless of value)
::* a time expression (for date time traits)
::* a time range (also for date time traits)<br/> The distinguished_by traits for the trigger are automatically added to each criterion's traits. To match a given criterion, all of an the traits specified must match.
* '''fire_criteria''': Describes what events must be present in the trigger's event list for the trigger to fire. Similar to the match criteria, but fire criteria has one additional option:
:* '''number''': There must be at least this many events that match this criterion for the trigger to fire. Defaults to 1.
* '''load_criteria''': Additional, historical, events may be loaded from the datastore prior to the trigger's running of it's event pipelines, (either firing or expiring). The syntax is the same as for match_criteria. These events will be added to the trigger's event list just before running the pipeline(s)
* '''fire_pipeline''': Name of an event pipeline to send the events to when the trigger fires. The events in the event list are sorted according to timestamp, and sent to that event pipeline
* '''expire_pipeline''': An optional name of a pipeline that events will be sent to if the trigger expires. If such a pipeline is listed, then events are sent to that pipeline when the trigger's expiration time has passed.

= Trigger Flow: =
There are four main processes to the trigger system: Processing incoming events, time-based execution for trigger firing and expiration, running the fire pipeline for firing triggers, and running the expire pipeline for expiring triggers.

== Incoming Event: ==
An Event comes into the system, either from a converted Notification, or from other systems (such as Alarms) that may generate Events. It is passed to the Dispatchers, and the DB dispatcher persists the event to the datastore (rejecting duplicates by message_id along the way). It is then passed to the Trigger Manager. The trigger manager checks the event against the list of trigger definitions to see if the event matches any. For each definition the event matches, the trigger manager then determines the set of distinguishing trait values from that event, and checks the datastore to see if there is an existing, non-expired trigger instance in the ''ACTIVE'' state that matches. If there is no existing trigger instance, one will be created in the ''ACTIVE'' state and persisted to the datastore, with it's distinguishing trait values and first event timestamp derived from the Event. In either case, the Event's id is then added to that trigger instance's event list, and the trigger instance's last event timestamp is set from the event's timestamp. The trigger instance's expire_timestamp will then be (re)calculated based on it's first_event or last_event timestamp, as appropriate.

Afterwords, any trigger instances created/changed by the event, that do not already have their fire_timestamp set, are checked to see if their firing criteria is met. For any that have met their firing criteria, the trigger instance's fire_timestamp is set to the current time, plus any fire delay period. Firing of the triggers is done through a timestamp in order to allow for a firing delay to collect any out-of order events.

== Time-Based Trigger Firing/Expiration: ==
The fire/expire job is run as a background thread in the Collector. (or it could be a separate process). It performs two tasks in a loop indefinitely.

First it fires triggers whose fire_timestamp has past. A limited sized batch of triggers whose state is ''ACTIVE'' and whose fire_timestamp is less than the current time are loaded from the datastore. The list of triggers is iterated in random order, each one is set to the ''READY_TO_FIRE'' state, and an rpc cast is made to the collector's fire_trigger method with the trigger's id. An rpc cast is made to spread the work across multiple workers.

Similarly, it then expires triggers whose expire time has past. A batch of triggers whose state is ''ACTIVE'', and whose expire time has past (and that have a null fire_timestamp), are loaded, and iterated in random order. Each is set to the ''READY_TO_EXPIRE'' state, and an rpc cast to expire_trigger with the trigger's id is made.

If neither of the jobs two tasks has anything to process, it sleeps for a short period (a second or so). The job is designed so that multiple jobs can run simultaneously, for scaling. Because the trigger states use a mechanism to synchronize between workers (see Trigger States below), the jobs will not step on each other. Also, the methods called via rpc are deliberately designed so that calling expire or fire multiple times on the same trigger will not run its fire or expiration more than once.

== Firing trigger: ==
An rpc request comes in to fire a given trigger instance. The trigger instance is loaded from the datastore, and it is checked to make sure it is in a ''READY_TO_FIRE'' state. (If not, the request is ignored). The trigger instance is set to a ''FIRING'' state. All of the events in the trigger's event list are loaded from the datastore. If there is any load criteria, the criteria is used to construct a query to the datastore to load additional events. The combined list of events is sorted by timestamp, earliest to latest, and any duplicates removed. The firing pipeline is loaded by the Event Pipeline manager, and the combined list of events is sent to that pipeline. Once the events are successfully finished processing through the pipeline, the trigger instance is deleted. If the pipeline throws an error while processing, the error is logged, and the trigger instance is set to an ''ERROR'' state. (An error in the pipeline is likely due to a code error, and should be investigated to file a bug.)

== Expiring Trigger: ==
An rpc request is received to expire a given trigger instance. The trigger instance is loaded from the datastore, and checked to make sure it is in the ''READY_TO_EXPIRE'' state. (if not, the request is ignored). The trigger is set to the ''EXPIRING'' state. If there is an expire pipeline, events are loaded from the event list and any load_criteria. The event loading proceeds the same as when firing, and the events are sent to the expire pipeline. If there is an error, the trigger is set to an ''EXPIRE_ERROR'' state. If the trigger is not in an error state, it is then deleted.

== Note: Trigger States: ==
The trigger instance's state is synchronized to prevent multiple workers stepping on each other. This can be done with a simple optimistic locking protocol in the datastore, or through some form of external atomic state machine if a suitable one is available.

DuplicateWorkCeilometer

2013-08-26T18:32:55Z

Monsyne Dragon:

== Preventing Duplicate Work in Ceilometer ==
=== aka "Retry Semantics" ===

Or ... "What happens when Step 4 of a 10 Step Pipeline fails?"

Here's a common use case in Ceilometer (and, most parts of OpenStack for that matter):

# We have some work to do that will take several steps. We could fail anywhere along the way.
# If we fail, we may get called again by the service that initiated the action.
# We don't want to redo the steps we've already done. Instead, we'd like to retry just the part that failed. Redo'ing previous work could have disastrous ramifications:
:* writing duplicate records to a database,
:* sending a notice to a downstream user/service thousands of times,
:* performing an expensive calculation over and over DDOS'ing ourselves, etc.

Currently in CM, we have the the following major components:
* The collector
* The dispatcher
* The pipeline
* Alarm states
* and, coming soon, the trigger pipeline

The problem with the above use case manifests itself in the dispatcher, the pipeline and will in the trigger pipeline. The collector is pretty dumb now thanks to the dispatcher.

The way I was hoping to prevent this problem is:
# the collector would re-publish to a new CM exchange for each downstream component that needs to do some unit of work. This could be a pipeline plugin, a dispatcher plugin or a trigger pipeline plugin.
:''dhellmann'' - Does ceilometer need to republish, or should we set up a different queue for each pipeline? That way the message broker will handle the messages and redelivery if the pipeline raises an exception, but the collector won't be a bottleneck to creating those new messages.
::''swalsh'': depends. If it's the same datatype, there would be 1 queue per notification. If the pipeline component produced a new datatype (like an Event producing 3 new Samples), there would be a publication required for each Sample.
# Each plugin would do its work and use the normal queue ack(), reject(), requeue() semantics of the queuing engine to deal with failures.
# This means that each plugin is responsible for detecting if it has already run or not. For some entities, such as events, this is easy since there is a unique message_id per event. But let's say that event generates 10 samples, we would need to repeatably produce a unique id for each sample based on that event id. For things like publishing, that gets very difficult since there is no related database record.

The downside with always going back to the queuing system is the large amount of chatter it will produce. Consider, 1 event that produces 10 meters, each with their own 3-step pipeline doing complex calculations and outbound publishing. That gets expensive quickly.

What we need is some way to say "I've done this step already".

Some possibilities:
# Each pipeline would manage the retry semantics for the pipeline. If any step fails, the pipeline manager would retry starting at the failed step. This is tricky since we would need to have a context object to persist and pass back to the failed plugin during the retry.
# Ditch the pipeline model. The collector would call a dispatcher when a particular event comes in. The dispatcher would do some work and either:
## ack and republish a new event to a new exchange, or
## requeue the event, or
## reject the event
# For non-pipeline stuff (the alarm state), we sort of need a persistent state-machine anyway. Rather than reinvent this for every component a reusable state machine library would be preferred (perhaps like the one the orchestration/state-management team is working on). Something that has schema-defined transitions with proper timeouts & persistence handled along each transition. A collection of enums and a snarl of if-statements is not a good state machine. I think this state machine library could also be used for the pipeline management as well.

:''dhellmann'' - The steps of the pipeline are meant to be relatively inexpensive transformations, not full processing. So each pipeline is massaging the data before taking the final action, which is the thing that might be reasonably expected to fail (publishing, writing to a db, whatever). Do we have (or envision) any transformations that might "fail" if they see the same data twice? Is it good enough to document that they are expected to be implemented in a way so they can avoid such issues?
::''sandywalsh'' - in our experience the intermediate steps are generally the expensive ones, the final steps are relatively lightweight. We've seen that there are some pretty complex queries involved when a significant event occurs (a .end or the end of a long sequence, for example). These can fail in a number of ways, but db timeouts are common, likewise with uncaught divide-by-zeros, etc. The final step (writing to the db, emailing a summary, etc) are pretty light relatively. That said, I think we have to assume things could fail anywhere for any reason. Currently, I'm happy with it failing and the queue growing until we can figure it out, but as we get more distributed (more collectors, more pipelines to carry the load) things are going to have to self-heal.
:::''mdragon'' - Having some kind of persistent state machine will definitely be needed for the triggers work, so that will help there. I see the pipelines (both the current sample pipelines, and the forthcoming event-pipelines) as fairly idempotent up to the point of the final persist/publish step. Basically, you do the various transformations, altering/adding items, and until you persist those changes, they can be redone safely. Part of the idea of the triggers is to encourage this. Instead of a transformer loading a bunch of related events from a persistant store directly, the trigger waits til you have all of the needed events, loads them from the datastore, and sends them into a processing pipeline as a batch, so you have (hopefully) all of the data there that you need. That said, some of the transformations may be involved. My worry about persisting each step along the way is: What happens if step 5 fails because step 2 screwed up?

A lightweight state machine engine that supports atomic updates is what we need. This could be backed by a database with good transaction support or something like zookeeper. Something like memcache wouldn't have locking we need. Either way the backend for this piece should be pluggable. We should chat with Josh Harlow to find out where they are with their effort and help out there if possible.

Additional problems:
* the problem of a single plugin generating N items before failing on the N+1 th item. My gut says we simply need to persist for each step of the generator. :/

Ceilometer/blueprints/configurable-event-definitions

2013-08-22T22:44:22Z

Monsyne Dragon: Created page with "== Introduction == Ceilometer's Events are currently fairly limited, as the traits added are pretty minimal, and the conversion is hardcoded, which makes things difficult for..."

== Introduction ==

Ceilometer's Events are currently fairly limited, as the traits added are pretty minimal, and the conversion is hardcoded, which makes things difficult for people deploying Ceilometer, who may have different needs for what data they want to process and store.

In order to make it easier to allow users to extract what they need, we should add a method of converting Notifications to Events that can be driven from a configuration file.
This will include descriptions of how to map fields in the notification body to Traits, and optional plugins for doing any programmatic translations (splitting a string, forcing case, etc.)
The mapping of notification to events will be defined per event_type, which can be wildcarded. Traits are added to events if the corresponding fields in the notification exist and are non-null. (As a special case, an empty string is considered null for non-text traits. This is due to some openstack projects (mostly Nova) using empty string for null dates.)

Current behavior is preserved: If the definitions file is not present, a warning will be logged, but an empty set of definitions will be assumed.
By default, any notifications that do not have an event definition in the definitions file for them will be converted to events with a set of minimal, default traits.
This can be changed by setting a flag in the ceilometer.conf file. If this is set to True, then any notifications that don't have events defined for them in the file will be dropped. This can be what you want, the notification system is quite chatty by design (notifications philosophy is "tell us everything, we'll ignore what we don't need"), so you may want to ignore the noisier ones if you don't use them.

There is a set of default traits (all are TEXT type) that will be added to all events if the notification has the relevant data:
* message_id: (All notifications should have this) notification's uuid
* service: (All notifications should have this) notification's publisher
* tenant_id
* request_id
These do not have to be specified in the event definition, they are automatically added, but their definitions can be overridden for a given event_type.

== Configuration Settings ==

There are two configuration settings in ceilometer.conf that affect event conversions:
* allow_dropping_of_notifications: (Default: False) If set to True, then notifications with no matching event definition will be dropped. (Notifications will '''only''' be dropped if this is True)
* event_definitions_cfg_file: (Default: "event_definitions.yaml") Name of event definitions config file (yaml format)

== Definitions file format ==

The event definitions file is in YAML format. It consists of a list of event definitions, which are mappings. Order is significant, the list of definitions is scanned in '''reverse''' order (last definition in the file to the first), to find a definition which matches the notification's event_type. That definition will be used to generate the Event. The reverse ordering is done because it is common to want the have a more general wildcarded definition (such as ''compute.instance.*'' ) with a set of traits common to all of those events, with a few more specific event definitions (like ''compute.instance.exists'') afterward that have all of the above traits, plus a few more. This lets you put the general definition first, followed by the specific ones, and use YAML mapping include syntax to avoid copying all of the trait definitions.

=== Event Definitions ===

Each event definition is a mapping with two keys (both required):
* '''event_type''': This is a list (or a string, which will be taken as a 1 element list) of event_types this definition will handle. These can be wildcarded with unix shell glob syntax. An exclusion listing (starting with a '!') will exclude any types listed from matching. If ONLY exclusions are listed, the definition will match anything not matching the exclusions.
* '''traits''': This is a mapping, the keys are the trait names, and the values are trait definitions.

=== Trait Definitions ===

Each trait definition is a mapping with the following keys:
* '''type''' (optional): The data type for this trait. (as a string). Valid options are: ''text'', ''int'', ''float'', and ''datetime''. defaults to ''text'' if not specified.
* '''fields''': A path specification for the field(s) in the notification you wish to extract for this trait. Specifications can be written to match multiple possible fields, the value for the trait will be derived from the matching fields that exist and have a non-null values in the notification. By default the value will be the first such field. (plugins can alter that, if they wish). This is nomally a string, but, for convenience, it can be specified as a list of specifications, which will match the fields for all of them. (See Field Path Specifications for more info on this syntax.)
* '''plugin''' (optional): This is a mapping (For convenience, this value can also be specified as a string, which is interpreted as a the name of a plugin, to be loaded with no parameters) with the following keys:
** '''name''': (string) name of a plugin to load
** '''parameters''': (optional) Mapping of keyword arguments to pass to the plugin on initialization. (See documentation on each plugin to see what arguments it accepts.)

=== Field Path Specifications ===

The path specifications define which fields in the JSON notification body are extracted to provide the value for a given trait.
The paths can be specified with a dot syntax (e.g. ''payload.host''). Square bracket syntax (e.g. ''payload[host]'') is also supported. In either case, if the key for the field you are looking for contains special characters, like '.', it will need to be quoted (with double or single quotes) like so:

payload.image_meta.'org.openstack__1__architecture'

The syntax used for the field specification is a variant of JSONPath, and is fairly flexible. (see: https://github.com/kennknowles/python-jsonpath-rw for more info)

=== Example Definitions file ===

<pre><nowiki>
---
- event_type: compute.instance.*
traits: &instance_traits
user_id:
fields: payload.user_id
instance_id:
fields: payload.instance_id
host:
fields: publisher_id
plugin:
name: split
parameters:
segment: 1
max_split: 1
service_name:
fields: publisher_id
plugin: split
instance_type_id:
type: int
fields: payload.instance_type_id
os_architecture:
fields: payload.image_meta.'org.openstack__1__architecture'
launched_at:
type: datetime
fields: payload.launched_at
deleted_at:
type: datetime
fields: payload.deleted_at
- event_type:
- compute.instance.exists
- compute.instance.update
traits:
<<: *instance_traits
audit_period_beginning:
type: datetime
fields: payload.audit_period_beginning
audit_period_ending:
type: datetime
fields: payload.audit_period_ending
</nowiki></pre>

== Trait plugins ==

Trait plugins can be used to do simple programmatic conversions on the value in a notification field, like splitting a string, lowercasing a value, converting a screwball date into ISO format, or the like. They are initialized with the parameters from the trait definition, if any, which can customize their behavior for a given trait. They are called with a list of all matching fields from the notification, so they can derive a value from multiple fields. The plugin will be called even if there is no fields found matching the field path(s), this lets a plugin set a default value, if needed. A plugin can also reject a value by returning ''None'', which will cause the trait not to be added. If the plugin returns anything other than ''None'', the trait's value will be set from whatever the plugin returned (coerced to the appropriate type for the trait).

NotificationEventExamples

2013-04-10T15:56:49Z

Monsyne Dragon: /* Periodic Notifications: */

__TOC__

= Notification Examples and Quick Reference: =
== Tips: ==
* Notifications generated by Nova are generated in JSON format, and placed on an AMQP queue.
* An external app, such as Yagi can format them into ATOM format and publish them as a feed.
* Notifications can be immediate notifications, which are generated when a specific event happens, or periodic notifications, which cover usage over a period of time.
* Currently most notifications are immediate, with the "compute.instance.exists" event being the only periodic event so far.
* Notifications all contain some metadata, (such as event type, and message id), and payload which varies depending on message type.
* Nova will often emit two immediate notifications for many tasks, one as the beginning of a lengthy task, such as instance create, and one upon successful completion of that task. The event types will distinguish the two.

== Examples: ==
=== Immediate Notifications: ===

<pre><nowiki>
{"event_type": "compute.instance.resize.confirm.start",
"timestamp": "2012-03-12 17:01:29.899834",
"message_id": "1234653e-ce46-4a82-979f-a9286cac5258",
"priority": "INFO",
"publisher_id": "compute.compute-1-2-3-4",
"payload": {"state_description": "",
"display_name": "testserver",
"memory_mb": 512,
"disk_gb": 20,
"tenant_id": "12345",
"created_at": "2012-03-12 16:55:17",
"instance_type_id": 2,
"instance_id": "abcbd165-fd41-4fd7-96ac-d70639a042c1",
"instance_type": "512MB instance",
"state": "active",
"user_id": "67890",
"launched_at": "2012-03-12 16:57:29",
"image_ref_url": "http://127.0.0.1:9292/images/a1b2c3b4-575f-4381-9c6d-fcd3b7d07d17"}}
</nowiki></pre>

<pre><nowiki>
{"event_type": "compute.instance.create.end",
"timestamp": "2012-03-12 17:00:24.156710",
"message_id": "00004e00-8da5-4c39-8ffb-c94ed0b5278c",
"priority": "INFO",
"publisher_id": "compute.compute-1-5-6-7",
"payload": {"state_description": "",
"display_name": "testserver",
"memory_mb": 512,
"disk_gb": 20,
"tenant_id": "12345",
"created_at": "2012-03-12 16:58:32",
"instance_type_id": 2,
"instance_id": "abcdef01-7b76-4b43-9143-fb2385df48a3",
"instance_type": "512MB instance",
"state": "active",
"user_id": "67890",
"fixed_ips":
[{"floating_ips": [],
"meta": {},
"type": "fixed",
"version": 6,
"address": "fe80::1234:5678"},
{"floating_ips": [],
"meta": {},
"type": "fixed",
"version": 4, "address": "127.0.1.1"},
{"floating_ips": [],
"meta": {},
"type": "fixed",
"version": 4,
"address": "10.180.0.151"}],
"launched_at": "2012-03-12 17:00:23.998518",
"image_ref_url": "http://127.0.0.1:9292/images/12345678-201f-4600-b5a1-0b97e2b1cb31"}}
</nowiki></pre>

=== Periodic Notifications: ===
* These notifications cover events, such as usage over a range of time. The period covered by such an event is called the audit period. Generally, these periodic notifications are generated by a a periodic task in the worker process (i.e. nova-compute) or a cronjob once per audit period from cron. Nova has the notion of a system-wide audit period set in it's config file. Nova will collect and aggregate statistics, such as bandwidth usage over that period. When the periodic events are generated, they will be generated for the most recently completed audit period. (Note that the timing of audit periods is *not* affected by when the script or task is run.)

System audit period: This is configured in Novas config file under the "instance_usage_audit_period" option.

* Valid audit periods are:
hour Each hour, for example "2012-03-01 01:00UTC" to "2012-03-01 02:00UTC".
* May be suffixed with @<number> to specify minute of hour. (for example hour@30 for 01:30UTC-02:30UTC)
day Daily. For example "2012-03-01 00:00UTC" to "2012-03-02 00:00UTC"
* May be suffixed with @<number> to specify hour of day. (for example day@6 for "2012-03-01 06:00UTC" to "2012-03-02 06:00UTC")
month Monthly. For example: "2012-03-01 00:00UTC" to "2012-04-01 00:00UTC".
* May be suffixed with @<number> to specify day of month. (for example month@15 for "2012-03-15 00:00UTC" to "2012-04-15 00:00UTC")
year Yearly. For example: "2011-01-01 00:00UTC" to "2012-01-01 00:00UTC".
* May be suffixed with @<number> to specify month. (for example year@4 for "2011-04-01 00:00UTC" to "2012-04-01 00:00UTC")

Examples:

<pre><nowiki>
{"event_type": "compute.instance.exists",
"timestamp": "2012-03-13 17:01",
"message_id": "0ab1ac0a-2867-402d-83c7-d7087262470c",
"priority": "INFO",
"publisher_id": "api.node-127-0-1-42",
"payload": {"state_description": "",
"display_name": "testserver",
"memory_mb": 512,
"disk_gb": 20,
"audit_period_beginning": "2012-03-12 06:00:00",
"audit_period_ending": "2012-03-13 06:00:00",
"tenant_id": "12345",
"created_at": "2012-03-12 16:55:17",
"instance_type_id": 2,
"bandwidth": {"public": {"bw_in": 12342000, "bw_out": 987654321},
"private": {"bw_in": 0, "bw_out": 0}},
"instance_id": "8888eeee-f1d0-4fd7-96ac-d70639a042c1",
"instance_type": "512MB instance",
"state": "active",
"user_id": "67890",
"launched_at": "2012-03-12 16:57:29",
"image_ref_url": "http://127.0.0.1:9292/images/a3a2a1a0-575f-4381-9c6d-fcd3b7d07d17"}}
</nowiki></pre>

Detailed explanation of example:

This is a compute.instance.exists event (currently the only periodic event we have).

These are generated periodically, one event for each instance active during the most recently completed audit period.

These can be used to synchronize an external system's picture of what instances a nova installation has running. This info can be obtained from the compute.instance.(create|delete).end events, but exists allows you to do instance accounting without looking at notification data more than 1 audit period old. This event also contains bandwidth usage data.

Some of the data in this specific event:

* '''timestamp''': This specific event was generated at 17:01 utc on March 13th (presumably when the cronjob ran)
* '''audit_period_(beginning|ending)''': It covers the audit period from 06:00 utc on March 12th, thru the same time on March 13th. (our example system is set for an audit period of "day@6")
* '''created_at''': Our example instance was requested to be created at 16:55:17 on March 12th
* '''launched_at''': It actually booted up at 16:57:29 on March 12th
* '''bandwidth''': Our example instance has two virtual network interfaces. In this audit period, the interface plugged into the network named "public" received 12.3MB of traffic. It transmitted 987.6MB. The interface plugged into the network named "private" did not receive or transmit any data.

NotificationEventExamples

2013-04-10T15:54:58Z

Monsyne Dragon: /* Periodic Notifications: */

__TOC__

= Notification Examples and Quick Reference: =
== Tips: ==
* Notifications generated by Nova are generated in JSON format, and placed on an AMQP queue.
* An external app, such as Yagi can format them into ATOM format and publish them as a feed.
* Notifications can be immediate notifications, which are generated when a specific event happens, or periodic notifications, which cover usage over a period of time.
* Currently most notifications are immediate, with the "compute.instance.exists" event being the only periodic event so far.
* Notifications all contain some metadata, (such as event type, and message id), and payload which varies depending on message type.
* Nova will often emit two immediate notifications for many tasks, one as the beginning of a lengthy task, such as instance create, and one upon successful completion of that task. The event types will distinguish the two.

== Examples: ==
=== Immediate Notifications: ===

<pre><nowiki>
{"event_type": "compute.instance.resize.confirm.start",
"timestamp": "2012-03-12 17:01:29.899834",
"message_id": "1234653e-ce46-4a82-979f-a9286cac5258",
"priority": "INFO",
"publisher_id": "compute.compute-1-2-3-4",
"payload": {"state_description": "",
"display_name": "testserver",
"memory_mb": 512,
"disk_gb": 20,
"tenant_id": "12345",
"created_at": "2012-03-12 16:55:17",
"instance_type_id": 2,
"instance_id": "abcbd165-fd41-4fd7-96ac-d70639a042c1",
"instance_type": "512MB instance",
"state": "active",
"user_id": "67890",
"launched_at": "2012-03-12 16:57:29",
"image_ref_url": "http://127.0.0.1:9292/images/a1b2c3b4-575f-4381-9c6d-fcd3b7d07d17"}}
</nowiki></pre>

<pre><nowiki>
{"event_type": "compute.instance.create.end",
"timestamp": "2012-03-12 17:00:24.156710",
"message_id": "00004e00-8da5-4c39-8ffb-c94ed0b5278c",
"priority": "INFO",
"publisher_id": "compute.compute-1-5-6-7",
"payload": {"state_description": "",
"display_name": "testserver",
"memory_mb": 512,
"disk_gb": 20,
"tenant_id": "12345",
"created_at": "2012-03-12 16:58:32",
"instance_type_id": 2,
"instance_id": "abcdef01-7b76-4b43-9143-fb2385df48a3",
"instance_type": "512MB instance",
"state": "active",
"user_id": "67890",
"fixed_ips":
[{"floating_ips": [],
"meta": {},
"type": "fixed",
"version": 6,
"address": "fe80::1234:5678"},
{"floating_ips": [],
"meta": {},
"type": "fixed",
"version": 4, "address": "127.0.1.1"},
{"floating_ips": [],
"meta": {},
"type": "fixed",
"version": 4,
"address": "10.180.0.151"}],
"launched_at": "2012-03-12 17:00:23.998518",
"image_ref_url": "http://127.0.0.1:9292/images/12345678-201f-4600-b5a1-0b97e2b1cb31"}}
</nowiki></pre>

=== Periodic Notifications: ===
* These notifications cover events, such as usage over a range of time. The period covered by such an event is called the audit period. Generally, these periodic notifications are generated by a a periodic task in the worker process (i.e. nova-compute) or a cronjob once per audit period from cron. Nova has the notion of a system-wide audit period set in it's config file. Nova will collect and aggregate statistics, such as bandwidth usage over that period. When the periodic events are generated, they will be generated for the most recently completed audit period. (Note that the timing of audit periods is *not* affected by when the script or task is run.)

System audit period: This is configured in Novas config file under the "instance_usage_audit_period" option.

* Valid audit periods are:
** hour Each hour, for example "2012-03-01 01:00UTC" to "2012-03-01 02:00UTC".
*** May be suffixed with @<number> to specify minute of hour. (for example hour@30 for 01:30UTC-02:30UTC)
day Daily. For example "2012-03-01 00:00UTC" to "2012-03-02 00:00UTC"
* May be suffixed with @<number> to specify hour of day. (for example day@6 for "2012-03-01 06:00UTC" to "2012-03-02 06:00UTC")
month Monthly. For example: "2012-03-01 00:00UTC" to "2012-04-01 00:00UTC".
* May be suffixed with @<number> to specify day of month. (for example month@15 for "2012-03-15 00:00UTC" to "2012-04-15 00:00UTC")
year Yearly. For example: "2011-01-01 00:00UTC" to "2012-01-01 00:00UTC".
* May be suffixed with @<number> to specify month. (for example year@4 for "2011-04-01 00:00UTC" to "2012-04-01 00:00UTC")

Examples:

<pre><nowiki>
{"event_type": "compute.instance.exists",
"timestamp": "2012-03-13 17:01",
"message_id": "0ab1ac0a-2867-402d-83c7-d7087262470c",
"priority": "INFO",
"publisher_id": "api.node-127-0-1-42",
"payload": {"state_description": "",
"display_name": "testserver",
"memory_mb": 512,
"disk_gb": 20,
"audit_period_beginning": "2012-03-12 06:00:00",
"audit_period_ending": "2012-03-13 06:00:00",
"tenant_id": "12345",
"created_at": "2012-03-12 16:55:17",
"instance_type_id": 2,
"bandwidth": {"public": {"bw_in": 12342000, "bw_out": 987654321},
"private": {"bw_in": 0, "bw_out": 0}},
"instance_id": "8888eeee-f1d0-4fd7-96ac-d70639a042c1",
"instance_type": "512MB instance",
"state": "active",
"user_id": "67890",
"launched_at": "2012-03-12 16:57:29",
"image_ref_url": "http://127.0.0.1:9292/images/a3a2a1a0-575f-4381-9c6d-fcd3b7d07d17"}}
</nowiki></pre>

Detailed explanation of example:

This is a compute.instance.exists event (currently the only periodic event we have).

These are generated periodically, one event for each instance active during the most recently completed audit period.

These can be used to synchronize an external system's picture of what instances a nova installation has running. This info can be obtained from the compute.instance.(create|delete).end events, but exists allows you to do instance accounting without looking at notification data more than 1 audit period old. This event also contains bandwidth usage data.

Some of the data in this specific event:

* '''timestamp''': This specific event was generated at 17:01 utc on March 13th (presumably when the cronjob ran)
* '''audit_period_(beginning|ending)''': It covers the audit period from 06:00 utc on March 12th, thru the same time on March 13th. (our example system is set for an audit period of "day@6")
* '''created_at''': Our example instance was requested to be created at 16:55:17 on March 12th
* '''launched_at''': It actually booted up at 16:57:29 on March 12th
* '''bandwidth''': Our example instance has two virtual network interfaces. In this audit period, the interface plugged into the network named "public" received 12.3MB of traffic. It transmitted 987.6MB. The interface plugged into the network named "private" did not receive or transmit any data.