Jump to: navigation, search

Difference between revisions of "Activity Board"

(Toolset)
 
(3 intermediate revisions by 3 users not shown)
Line 10: Line 10:
 
* '''Metrics browser''': this product visualizes that parsed open data and provides specific visualizations. Indeed, the proposed visualization can be easily extensible by the community and third parties. Given its extensibility, in the current version of the Activity Board, there are three main frontends: General Overview of the activity, releases study and actionable dashboard.
 
* '''Metrics browser''': this product visualizes that parsed open data and provides specific visualizations. Indeed, the proposed visualization can be easily extensible by the community and third parties. Given its extensibility, in the current version of the Activity Board, there are three main frontends: General Overview of the activity, releases study and actionable dashboard.
  
 +
===Deployed Studies===
 +
Current version of the Activity Board provides several faces of the OpenStack project: Development Activity Browser, Releases and Companies study, and Actionable dashboard. Each of this parts of the Activity Board was built with a specific goal.
 +
 +
====Development Activity Browser====
 +
[http://activity.openstack.org/dash/ The Development Activity Browser] aims at providing a common place for those interested in the OpenStack Foundation development activity. This provides detailed information about each of the data sources analyzed (Gits, Launchpad, Gerrit and Mailing lists), stressing the point of the community and its evolution.
 +
 +
The list of projects are based on those that appear in the [https://wiki.openstack.org/wiki/Programs  OpenStack Foundation Programs].
 +
 +
====Releases and Companies Browser====
 +
[http://activity.openstack.org/dash/releases The Releases and Companies Browser] provides an analysis of the community with special emphasis in the companies participating in each release and their developers.
 +
 +
This browser allows to understand how the evolution of the main companies took place during specific releases (even during the whole history of the project). In addition it is possible to navigate through the several data sources analyzed and even at the level of providing information per company.
 +
 +
====Actionable Dashboard====
 +
[http://activity.openstack.org/dash/newbrowser/dashboard/ The Actionable Dashboard] is probably the most powerful of the three analysis. Indeed this allows the user to create charts from scratch selecting among metrics, companies, projects and dates (releases and years).
  
  
Line 17: Line 32:
 
* Filtering and parsing process
 
* Filtering and parsing process
 
* Visualization
 
* Visualization
 +
 +
[[File:Grimoire-arch.png|thumbnail|High level architecture]]
  
 
==== Retrieval ====
 
==== Retrieval ====
Line 22: Line 39:
  
 
The data mining toolset is based on the [http://metricsgrimoire.github.io/ Metrics Grimoire project] that provides specific tools for each of the analyzable information data sources.
 
The data mining toolset is based on the [http://metricsgrimoire.github.io/ Metrics Grimoire project] that provides specific tools for each of the analyzable information data sources.
 
The following tools are used:
 
* [https://github.com/MetricsGrimoire/CVSAnalY CVSAnalY]: this tool analyzes the versioning system. This currently supports Git, SVN and CVS. Others systems can be easily migrated to Git and later be analyzed by the tool. More information about the database schema can be found at the [https://github.com/MetricsGrimoire/CVSAnalY/wiki/Database-Schema CVSAnalY wiki page].
 
* [https://github.com/MetricsGrimoire/MailingListStats MailingListStats]: this tool analyzes mailing lists in mbox format. More information about the database schema can be found at the [https://github.com/MetricsGrimoire/MailingListStats/wiki/Database-Schema MailingListStats wiki page].
 
* [https://github.com/MetricsGrimoire/Bicho Bicho]: this tool analyzes ticketing systems. This currently analyzes different flavours of Bugzilla, Launchpad, Jira, Google Code, Redmine and others. More information about the database schema can be found at the [https://github.com/MetricsGrimoire/Bicho/wiki/Database-Schema Bicho wiki page].
 
 
==== Filtering and Parsing ====
 
After storing in databases raw data from the software development repositories, that dataset need to be filtered, cleaned and massaged in order to provide useful information for the community. [https://github.com/VizGrimoire/VizGrimoireR VizGrimoireR] is the library used for this purpose and specialized in analyzing databases from the Metrics Grimoire toolset. R is the programming language mainly used and JSON files are provided as outcomes of this step.
 
  
 
==== Visualization ====
 
==== Visualization ====
Line 35: Line 44:
  
 
=== How to contribute ===
 
=== How to contribute ===
This projects follows the general software development process of the OpenStack project. A bit more of information can be found in the [[Gerrit_Workflow|Gerrit workflow page]].
+
This projects follows the general software development process of the OpenStack project. A bit more of information can be found in the [http://docs.openstack.org/infra/manual/developers.html Developer's Guide].
  
 
Besides, you can collaborate with the project using the following channels:
 
Besides, you can collaborate with the project using the following channels:
Line 48: Line 57:
 
* Add dependencies in terms of JavaScript libraries such as Bootstrap, JQuery of VizGrimoireJS
 
* Add dependencies in terms of JavaScript libraries such as Bootstrap, JQuery of VizGrimoireJS
 
* Add specific R files to use the VizGrimoireR library that retrieves all of the information needed to run the OpenStack Activity Board.
 
* Add specific R files to use the VizGrimoireR library that retrieves all of the information needed to run the OpenStack Activity Board.
 +
 +
[[Category: resources]]

Latest revision as of 21:23, 26 March 2015

Mission

The goal of the Activity Board project is to provide a dashboard of software development activity metrics of the OpenStack community.

Description

The current version of the Activity Board project only contains the web front-end needed to navigate through the several perspectives of the OpenStack software development process. This is based on the analysis of the repositories of information used by the community as basic infrastructure: versioning system, issue tracking system, mailing lists and others.

The outcomes of this project can be divided into two main groups:

  • Open data: OpenStack software repositories are publicly available. And such repositories contain loads of dataset about how the OpenStack community behaves and evolves through time. The Activity Board project aims at providing such raw data in a more human-readable format and aggregated in databases and other formats.
  • Metrics browser: this product visualizes that parsed open data and provides specific visualizations. Indeed, the proposed visualization can be easily extensible by the community and third parties. Given its extensibility, in the current version of the Activity Board, there are three main frontends: General Overview of the activity, releases study and actionable dashboard.

Deployed Studies

Current version of the Activity Board provides several faces of the OpenStack project: Development Activity Browser, Releases and Companies study, and Actionable dashboard. Each of this parts of the Activity Board was built with a specific goal.

Development Activity Browser

The Development Activity Browser aims at providing a common place for those interested in the OpenStack Foundation development activity. This provides detailed information about each of the data sources analyzed (Gits, Launchpad, Gerrit and Mailing lists), stressing the point of the community and its evolution.

The list of projects are based on those that appear in the OpenStack Foundation Programs.

Releases and Companies Browser

The Releases and Companies Browser provides an analysis of the community with special emphasis in the companies participating in each release and their developers.

This browser allows to understand how the evolution of the main companies took place during specific releases (even during the whole history of the project). In addition it is possible to navigate through the several data sources analyzed and even at the level of providing information per company.

Actionable Dashboard

The Actionable Dashboard is probably the most powerful of the three analysis. Indeed this allows the user to create charts from scratch selecting among metrics, companies, projects and dates (releases and years).


Toolset

The Activity Board consists of three steps:

  • Retrieval process
  • Filtering and parsing process
  • Visualization
High level architecture

Retrieval

Information is parsed from software development repositories and stored in MySQL databases. Software development repositories are any type of data source with activity of interest for developers and the OpenStack community in general. Examples of this type of data sources are the versioning systems such as Git, mailing lists for discussions, the Launchpad ticketing system or the Gerrit activity.

The data mining toolset is based on the Metrics Grimoire project that provides specific tools for each of the analyzable information data sources.

Visualization

The final part of this process consists of the visualization, where those JSON files are feeded to the VizGrimoireJS library, that builds the web frontend necessary to navigate through the several aspects of the OpenStack project.

How to contribute

This projects follows the general software development process of the OpenStack project. A bit more of information can be found in the Developer's Guide.

Besides, you can collaborate with the project using the following channels:

Next Steps

The Activity Board project already contains all the code needed for the web frontend. And the following code to be added:

  • Upload config file to run the whole Metrics Grimoire chain
  • Add dependencies in terms of JavaScript libraries such as Bootstrap, JQuery of VizGrimoireJS
  • Add specific R files to use the VizGrimoireR library that retrieves all of the information needed to run the OpenStack Activity Board.