OpenStack - User contributions [en]

Vitrage

2016-05-23T16:05:40Z

Nofarsch:

<gallery>
File:Vitrage logo finaly.png
</gallery>

== What is Vitrage? ==
Vitrage is the OpenStack RCA (Root Cause Analysis) Engine for organizing, analyzing and expanding OpenStack alarms & events, yielding insights regarding the root cause of problems and deducing their existence before they are directly detected.

=== High Level Functionality ===
# Physical-to-Virtual entities mapping
# Deduced alarms and states (i.e., raising an alarm or modifying a state based on analysis of the system, instead of direct monitoring)
# Root Cause Analysis (RCA) for alarms/events
# Horizon plugin for the above features

== High Level Architecture ==

[[File:Vitrage-high level architecture2.png|2000px|frameless|center|Vitrage High Level Architecture]] 

'''Vitrage Data Source(s).''' Responsible for importing information from different sources, regarding the state of the system. This includes information regarding resources both physical & virtual, alarms, etc.. The information is then processed into the Vitrage Graph. Currently Vitrage comes ready with data sources for Nova, Cinder, and Aodh OpenStack projects, Nagios alarms, and a static Physical Resources data source.

'''Vitrage Graph.''' Holds the information collected by the Data Sources, as well as their inter-relations. Additionally, it implements a collection of basic graph algorithms that are used by the Vitrage Evaluator (e.g., sub-matching, BFS, DFS etc).

'''Vitrage Evaluator.''' Coordinates the analysis of (changes to) the Vitrage Graph and processes the results of this analysis. It is responsible for execution different kind of template-based actions in Vitrage, such as to add an RCA (Root Cause Analysis) relationship between alarms, raise a deduced alarm or set a deduced state.

For more information, refer to the [https://github.com/openstack/vitrage/blob/master/doc/source/vitrage-graph-design.rst low level design]

== Use Cases ==
=== Baseline ===
[[File:Rca-baseline.jpg|400px|frameless|right|Baseline]] 

We consider the following example, where a we are monitoring a Switch (id 1002), for example via Nagios, and as a problem on the Switch causes a Nagios alarm (a.k.a. Nagios test) to be activated. The following image depicts the logical relationship among different resources in the system that are related to this switch, as well as the raised alarm. Note the mapping between virtual (instance) and physical (host, switch) entities, as well as between the alarm and the switch it relates to.

=== Deduced alarms & states ===
[[File:DeducedAlarm.jpg|500px|frameless|right|Deduced Alarm]] 

Problems on the switch can, at times, have a negative impact on the virtual instances running on hosts attached to the switch. We would like to raise an alarm on those instances to indicate this impact, as shown here:

As can be seen, the problem on the switch should trigger an alarm on all instances associated with the switch. Similarly, we might want the state of all these instances to be changed to "ERROR" as well. This functionality should be supported even if we cannot directly monitor the state of the instances. Instances might not be monitored for all aspects of performance, or perhaps the problem in the switch makes monitoring them difficult or even impossible. Instead, we can '''deduce''' this problem exists on the instances based on the state of the switch, and raise alarms and change states accordingly.

=== Root Cause Indicators ===
[[File:RootCauseExample.jpg|500px|frameless|right|Root Cause Link]] 

Furthermore, we would like to be able to track this cause and effect - that the problem in the switch caused the problems experienced at the instances. In the following image, we highlight a single connection between the cause and effect for clarity - but all such links should be supported.

Important Note: not all deduced alarms are ''caused'' by the trigger - the trigger might only be an indication of correlation, not causation. In the case we are examining, however, the trigger alarm is also the cause:

Once the local "causes" links (one hop) are detected and registered, we can follow them one hop after another to track the full causal chain of a sequence of events.

== Demos and Presentations ==
=== Quick Demos ===
* [[Media:Vitrage presentation Feb 09.pdf|Vitrage Presentation]]
* [https://www.youtube.com/watch?v=tl5AD5IdzMo&feature=youtu.be Vitrage Functionalities Overview]
* [https://www.youtube.com/watch?v=GyTnMw8stXQ&feature=youtu.be Vitrage Get Topology Demo]
* [https://www.youtube.com/watch?v=w1XQATkrdmg Vitrage Alarms Demo]
* [https://www.youtube.com/watch?v=vqlOKTmYR4c Vitrage Deduced Alarms and RCA Demo]

=== Austin Presentations ===
* [https://www.youtube.com/watch?v=9Qw5coTLgMo Project Vitrage How to Organize, Analyze and Visualize your OpenStack Cloud]
* [https://www.youtube.com/watch?v=ey68KNKXc5c On the Path to Telco Cloud Openness: Nokia CloudBand Vitrage & OPNFV Doctor collaboration]

== Development (Blueprints, Roadmap, Design...) ==
* Project at Launchpad: http://launchpad.net/vitrage
* [https://blueprints.launchpad.net/vitrage Blueprints]
* Source code:
** [https://github.com/openstack/vitrage vitrage]
** [https://github.com/openstack/python-vitrageclient python-vitrageclient]
** [https://github.com/openstack/vitrage-dashboard vitrage-dashboard]

=== Open Design Discussions ===
* [https://etherpad.openstack.org/p/vitrage-overlapping-templates-support-design Supporting Overlapping Templates]

=== Documention ===
* Usage
** [https://github.com/openstack/vitrage/blob/master/doc/source/vitrage-first_steps.rst Getting Started Guide]
** [https://github.com/openstack/python-vitrageclient/blob/master/doc/source/cli.rst Vitrage CLI]
** [https://github.com/openstack/vitrage/blob/master/doc/source/vitrage-api.rst Vitrage API]
** [https://github.com/openstack/vitrage/blob/master/doc/source/vitrage-template-format.rst Evaluator Templates]
** [https://github.com/openstack/vitrage/blob/master/doc/source/installation-and-configuration.rst Installation and Configuration]
* Design Documents
** [https://github.com/openstack/vitrage/blob/master/doc/source/vitrage-graph-design.rst Vitrage Graph Design]
** [https://github.com/openstack/vitrage/blob/master/doc/source/vitrage-use-cases.rst Use Cases]
** [https://github.com/openstack/vitrage/blob/master/doc/source/scenario-evaluator.rst Scenario Evaluator]
** [https://github.com/openstack/vitrage-specs/blob/master/specs/mitaka/vitrage-synchronizer.rst Synchronizer Design]
** [https://github.com/openstack/vitrage/blob/master/doc/source/notifier-aodh-plugin.rst Aodh Notifier Design]

== Communication and Meetings ==
=== Meetings ===
* Weekly on Wednesday at 0800 UTC in #openstack-meeting-4 at freenode
* Check [https://wiki.openstack.org/wiki/Meetings/Vitrage Vitrage Meetings] for more details

=== Contact Us ===
* IRC channel for regular daily discussions: #openstack-vitrage
* Use [Vitrage] tag for Vitrage emails on [http://lists.openstack.org/pipermail/openstack-dev/ OpenStack Mailing Lists]
* Contact Persons
** Ifat Afek - PTL - [mailto:ifat.afek@nokia.com ifat.afek@nokia.com]
** Alexey Weyl - [mailto:alexey.weyl@nokia.com alexey.weyl@nokia.com]
** Elisha Rosensweig - [mailto:elisha.rosensweig@nokia.com elisha.rosensweig@nokia.com]
** Ohad Shamir - [mailto:ohad.shamir@nokia.com ohad.shamir@nokia.com]
** Idan Hefetz - [mailto:idan.hefetz@nokia.com idan.hefetz@nokia.com]
** Dan Offek - [mailto:dan.offek@nokia.com dan.offek@nokia.com]
** Eyal Bar-Ilan - [mailto:eyal.bar-ilan@nokia.com eyal.bar-ilan@nokia.com]
** Liat Har-tal - [mailto:liat.har-tal@nokia.com liat.har-tal@nokia.com]
** Nofar Schnider - [mailto:nofar.schnider.ext@nokia.com nofar.schnider.ext@nokia.com]
** Noam Bloom - [mailto:noam.bloom.ext@nokia.com noam.bloom.ext@nokia.com]

Vitrage

2016-05-23T16:01:29Z

Nofarsch:

<gallery>
File:Vitrage logo finaly.png
</gallery>

== What is Vitrage? ==
Vitrage is the OpenStack RCA (Root Cause Analysis) Engine for organizing, analyzing and expanding OpenStack alarms & events, yielding insights regarding the root cause of problems and deducing their existence before they are directly detected.

=== High Level Functionality ===
# Physical-to-Virtual entities mapping
# Deduced alarms and states (i.e., raising an alarm or modifying a state based on analysis of the system, instead of direct monitoring)
# Root Cause Analysis (RCA) for alarms/events
# Horizon plugin for the above features

== High Level Architecture ==

[[File:Vitrage-high level architecture2.png|2000px|frameless|center|Vitrage High Level Architecture]] 

'''Vitrage Data Source(s).''' Responsible for importing information from different sources, regarding the state of the system. This includes information regarding resources both physical & virtual, alarms, etc.. The information is then processed into the Vitrage Graph. Currently Vitrage comes ready with data sources for Nova, Cinder, and Aodh OpenStack projects, Nagios alarms, and a static Physical Resources data source.

'''Vitrage Graph.''' Holds the information collected by the Data Sources, as well as their inter-relations. Additionally, it implements a collection of basic graph algorithms that are used by the Vitrage Evaluator (e.g., sub-matching, BFS, DFS etc).

'''Vitrage Evaluator.''' Coordinates the analysis of (changes to) the Vitrage Graph and processes the results of this analysis. It is responsible for execution different kind of template-based actions in Vitrage, such as to add an RCA (Root Cause Analysis) relationship between alarms, raise a deduced alarm or set a deduced state.

For more information, refer to the [https://github.com/openstack/vitrage/blob/master/doc/source/vitrage-graph-design.rst low level design]

== Use Cases ==
=== Baseline ===
[[File:Rca-baseline.jpg|400px|frameless|right|Baseline]] 

We consider the following example, where a we are monitoring a Switch (id 1002), for example via Nagios, and as a problem on the Switch causes a Nagios alarm (a.k.a. Nagios test) to be activated. The following image depicts the logical relationship among different resources in the system that are related to this switch, as well as the raised alarm. Note the mapping between virtual (instance) and physical (host, switch) entities, as well as between the alarm and the switch it relates to.

=== Deduced alarms & states ===
[[File:DeducedAlarm.jpg|500px|frameless|right|Deduced Alarm]] 

Problems on the switch can, at times, have a negative impact on the virtual instances running on hosts attached to the switch. We would like to raise an alarm on those instances to indicate this impact, as shown here:

As can be seen, the problem on the switch should trigger an alarm on all instances associated with the switch. Similarly, we might want the state of all these instances to be changed to "ERROR" as well. This functionality should be supported even if we cannot directly monitor the state of the instances. Instances might not be monitored for all aspects of performance, or perhaps the problem in the switch makes monitoring them difficult or even impossible. Instead, we can '''deduce''' this problem exists on the instances based on the state of the switch, and raise alarms and change states accordingly.

=== Root Cause Indicators ===
[[File:RootCauseExample.jpg|500px|frameless|right|Root Cause Link]] 

Furthermore, we would like to be able to track this cause and effect - that the problem in the switch caused the problems experienced at the instances. In the following image, we highlight a single connection between the cause and effect for clarity - but all such links should be supported.

Important Note: not all deduced alarms are ''caused'' by the trigger - the trigger might only be an indication of correlation, not causation. In the case we are examining, however, the trigger alarm is also the cause:

Once the local "causes" links (one hop) are detected and registered, we can follow them one hop after another to track the full causal chain of a sequence of events.

== Demos and Presentations ==
=== Quick Demos ===
* [[Media:Vitrage presentation Feb 09.pdf|Vitrage Presentation]]
* [https://www.youtube.com/watch?v=tl5AD5IdzMo&feature=youtu.be Vitrage Functionalities Overview]
* [https://www.youtube.com/watch?v=GyTnMw8stXQ&feature=youtu.be Vitrage Get Topology Demo]
* [https://www.youtube.com/watch?v=w1XQATkrdmg Vitrage Alarms Demo]
* [https://www.youtube.com/watch?v=vqlOKTmYR4c Vitrage Deduced Alarms and RCA Demo]

=== Austin Presentations ===
* [https://www.youtube.com/watch?v=9Qw5coTLgMo Project Vitrage How to Organize, Analyze and Visualize your OpenStack Cloud]
* [https://www.youtube.com/watch?v=ey68KNKXc5c On the Path to Telco Cloud Openness: Nokia CloudBand Vitrage & OPNFV Doctor collaboration]

== Development (Blueprints, Roadmap, Design...) ==
* Project at Launchpad: http://launchpad.net/vitrage
* [https://blueprints.launchpad.net/vitrage Blueprints]
* Source code:
** [https://github.com/openstack/vitrage vitrage]
** [https://github.com/openstack/python-vitrageclient python-vitrageclient]
** [https://github.com/openstack/vitrage-dashboard vitrage-dashboard]

=== Open Design Discussions ===
* [https://etherpad.openstack.org/p/vitrage-overlapping-templates-support-design Supporting Overlapping Templates]

=== Documention ===
* Usage
** [https://github.com/openstack/vitrage/blob/master/doc/source/vitrage-first_steps.rst Getting Started Guide]
** [https://github.com/openstack/python-vitrageclient/blob/master/doc/source/cli.rst Vitrage CLI]
** [https://github.com/openstack/vitrage/blob/master/doc/source/vitrage-api.rst Vitrage API]
** [https://github.com/openstack/vitrage/blob/master/doc/source/vitrage-template-format.rst Evaluator Templates]
** [https://github.com/openstack/vitrage/blob/master/doc/source/installation-and-configuration.rst Installation and Configuration]
* Design Documents
** [https://github.com/openstack/vitrage/blob/master/doc/source/vitrage-graph-design.rst Vitrage Graph Design]
** [https://github.com/openstack/vitrage/blob/master/doc/source/vitrage-use-cases.rst Use Cases]
** [https://github.com/openstack/vitrage/blob/master/doc/source/scenario-evaluator.rst Scenario Evaluator]
** [https://github.com/openstack/vitrage-specs/blob/master/specs/mitaka/vitrage-synchronizer.rst Synchronizer Design]
** [https://github.com/openstack/vitrage/blob/master/doc/source/notifier-aodh-plugin.rst Aodh Notifier Design]

== Communication and Meetings ==
=== Meetings ===
* Weekly on Wednesday at 0800 UTC in #openstack-meeting-4 at freenode
* Check [https://wiki.openstack.org/wiki/Meetings/Vitrage Vitrage Meetings] for more details

=== Contact Us ===
* IRC channel for regular daily discussions: #openstack-vitrage
* Use [Vitrage] tag for Vitrage emails on [http://lists.openstack.org/pipermail/openstack-dev/ OpenStack Mailing Lists]
* Contact Persons
** Ifat Afek - PTL - [mailto:ifat.afek@nokia.com ifat.afek@nokia.com]
** Alexey Weyl - [mailto:alexey.weyl@nokia.com alexey.weyl@nokia.com]
** Elisha Rosensweig - [mailto:elisha.rosensweig@nokia.com elisha.rosensweig@nokia.com]
** Ohad Shamir - [mailto:ohad.shamir@nokia.com ohad.shamir@nokia.com]
** Idan Hefetz - [mailto:idan.hefetz@nokia.com idan.hefetz@nokia.com]
** Dan Offek - [mailto:dan.offek@nokia.com dan.offek@nokia.com]
** Eyal Bar-Ilan - [mailto:eyal.barilan@nokia.com eyal.barilan@nokia.com]
** Liat Har-tal - [mailto:liat.har-tal@nokia.com liat.har-tal@nokia.com]
** Nofar Schnider - [mailto:nofar.schnider.ext@nokia.com nofar.schnider.ext@nokia.com]
** Noam Bloom - [mailto:noam.bloom.ext@nokia.com noam.bloom.ext@nokia.com]