The goal of the Activity Board project is to provide a dashboard of software development activity metrics of the OpenStack community.
The current version of the Activity Board project only contains the web front-end needed to navigate through the several perspectives of the OpenStack software development process. This is based on the analysis of the repositories of information used by the community as basic infrastructure: versioning system, issue tracking system, mailing lists and others.
The outcomes of this project can be divided into two main groups:
- Open data: OpenStack software repositories are publicly available. And such repositories contain loads of dataset about how the OpenStack community behaves and evolves through time. The Activity Board project aims at providing such raw data in a more human-readable format and aggregated in databases and other formats.
- Metrics browser: this product visualizes that parsed open data and provides specific visualizations. Indeed, the proposed visualization can be easily extensible by the community and third parties. Given its extensibility, in the current version of the Activity Board, there are three main frontends: General Overview of the activity, releases study and actionable dashboard.
Current version of the Activity Board provides several faces of the OpenStack project: Development Activity Browser, Releases and Companies study, and Actionable dashboard. Each of this parts of the Activity Board was built with a specific goal.
Development Activity Browser
The Development Activity Browser aims at providing a common place for those interested in the OpenStack Foundation development activity. This provides detailed information about each of the data sources analyzed (Gits, Launchpad, Gerrit and Mailing lists), stressing the point of the community and its evolution.
The list of projects are based on those that appear in the OpenStack Foundation Programs.
Releases and Companies Browser
The Releases and Companies Browser provides an analysis of the community with special emphasis in the companies participating in each release and their developers.
This browser allows to understand how the evolution of the main companies took place during specific releases (even during the whole history of the project). In addition it is possible to navigate through the several data sources analyzed and even at the level of providing information per company.
The Actionable Dashboard is probably the most powerful of the three analysis. Indeed this allows the user to create charts from scratch selecting among metrics, companies, projects and dates (releases and years).
The Activity Board consists of three steps:
- Retrieval process
- Filtering and parsing process
Information is parsed from software development repositories and stored in MySQL databases. Software development repositories are any type of data source with activity of interest for developers and the OpenStack community in general. Examples of this type of data sources are the versioning systems such as Git, mailing lists for discussions, the Launchpad ticketing system or the Gerrit activity.
The data mining toolset is based on the Metrics Grimoire project that provides specific tools for each of the analyzable information data sources.
The following tools are used:
- CVSAnalY: this tool analyzes the versioning system. This currently supports Git, SVN and CVS. Others systems can be easily migrated to Git and later be analyzed by the tool. More information about the database schema can be found at the CVSAnalY wiki page.
- MailingListStats: this tool analyzes mailing lists in mbox format. More information about the database schema can be found at the MailingListStats wiki page.
- Bicho: this tool analyzes ticketing systems. This currently analyzes different flavours of Bugzilla, Launchpad, Jira, Google Code, Redmine and others. More information about the database schema can be found at the Bicho wiki page.
Filtering and Parsing
After storing in databases raw data from the software development repositories, that dataset need to be filtered, cleaned and massaged in order to provide useful information for the community. VizGrimoireR is the library used for this purpose and specialized in analyzing databases from the Metrics Grimoire toolset. R is the programming language mainly used and JSON files are provided as outcomes of this step.
The final part of this process consists of the visualization, where those JSON files are feeded to the VizGrimoireJS library, that builds the web frontend necessary to navigate through the several aspects of the OpenStack project.
How to contribute
This projects follows the general software development process of the OpenStack project. A bit more of information can be found in the Developer's Guide.
Besides, you can collaborate with the project using the following channels:
- Git repository: http://git.openstack.org/cgit/openstack-infra/activity-board/
- Launchpad OpenStack Community project using the tag activityboard
- OpenStack Developers mailing list, using the [metrics] topic
- If errors are found in the toolset chain, those are free software projects as OpenStack and specific ticketing systems, mailing list discussion and IRC channels are provided. For more information, please visit the main page of the Metrics Grimoire toolset.
The Activity Board project already contains all the code needed for the web frontend. And the following code to be added:
- Upload config file to run the whole Metrics Grimoire chain
- Add specific R files to use the VizGrimoireR library that retrieves all of the information needed to run the OpenStack Activity Board.