DriverLog

Mission
The DriverLog project is on a mission to provide information about drivers and plugins developed for both OpenStack itself and projects related to OpenStack.

When it comes to a production OpenStack deployment a customer wants to have an assurance that certain hardware/virtual appliances will really work with certain OpenStack components.

OpenStack community itself may be able to provide a solution to the problem of driver/version proliferation: one can be more-efficient, timely, transparent and trustworthy solution. DriverLog is an open source project that aims to provide:


 * A single, consolidated source of truth on all third party OpenStack vendor drivers.
 * Access to instructions on how to configure any vendor’s product with OpenStack.
 * Information on whether a vendor’s driver is embedded in the upstream codebase of an OpenStack release.
 * Information on whether a vendor has deployed an automated test infrastructure that provides feedback on the vendor’s driver passing or failing functional and integration tests.
 * Quick shortcuts for OpenStack deployers to contact maintainers of vendor drivers in case they run into issues.

Target audience
This initiative has three beneficiaries.

Vendors

 * Readily available, community supported driver marketplace.
 * One common and measurable version of the truth with regards to vendor driver compatibility with OpenStack releases.
 * DriverLog initiative removes some of the burden of asserting and promoting driver compatibility with different OpenStack releases by defining and aggregating one common and measurable version of Truth.

OpenStack customers

 * Standardized information OpenStack users can use and trust.
 * Improved quality and reliability of OpenStack.
 * Quick access to accurate and up-to-date information about OpenStack drivers developed and maintained by a variety of vendors.

OpenStack Community

 * Encourage more participation of vendors in the community.
 * Meet the quality, reliability and openness expected by OpenStack users.
 * Accelerate OpenStack mission of building a universal, open fabric for an heterogeneous set of components while avoiding component fragmentation and non-operable point products.

Description
DriverLog is a service that collects and processes information about OpenStack drivers, their maintainers, availability in trunk, CI testing status and makes it possible to visualize it in a convenient web dashboard. The Driver dashboard makes it possible to view data by project, vendor and other factors.

The Driverlog Dashboard
The Driverlog dashboard allows filtering drivers by relevant OpenStack release, project and vendor (e.g., Icehouse + Neutron (Networking) + Midokura) as well as search on keywords. Results can be sorted by column and display project, vendor, driver description, trunk compatibility data (e.g., Grizzly, Folsom, Havana, Icehouse), note whether the driver is CI tested, and identify the maintainer. If you click (i) icon, you will get a detailed information about the selected driver: CI test results and the maintainer's profile at Stackatytics.

Tracked projects
DriverLog stores a list of projects and their drivers in the configuration file default_data.json. Any contributor can add an untracked project. The projects section represents the list of tracked projects. It has the following format: "projects": [ {           "id": "openstack/cinder", "name": "Cinder (Block Storage)" },       {            "id": "openstack/fuel", "name": "Fuel" },       {            "id": "openstack/glance", "name": "Glance (Image Service)" },       {            "id": "openstack/ironic", "name": "Ironic (Bare Metal Provisioning)" },       {            "id": "openstack/manila", "name": "Manila (Shared File Storage)" },       {            "id": "openstack/neutron", "name": "Neutron (Networking)" },       {            "id": "openstack/nova", "name": "Nova (Compute)" },       {            "id": "openstack/sahara", "name": "Sahara (Data Processing)" }   ],

This is how these projects look in DriverLog:

Tracked releases
The releases section represents the list of OpenStack releases. It has the following format:

"releases": [ {           "id": "austin", "wiki": "https://wiki.openstack.org/wiki/ReleaseNotes/Austin" },       {            "id": "bexar", "wiki": "https://wiki.openstack.org/wiki/ReleaseNotes/Bexar" },       {            "id": "cactus", "wiki": "https://wiki.openstack.org/wiki/ReleaseNotes/Cactus" },       {            "id": "diablo", "wiki": "https://wiki.openstack.org/wiki/ReleaseNotes/Diablo" },       {            "id": "essex", "wiki": "https://wiki.openstack.org/wiki/ReleaseNotes/Essex" },       {            "id": "folsom", "wiki": "https://wiki.openstack.org/wiki/ReleaseNotes/Folsom" },       {            "id": "grizzly", "wiki": "https://wiki.openstack.org/wiki/ReleaseNotes/Grizzly" },       {            "id": "havana", "wiki": "https://wiki.openstack.org/wiki/ReleaseNotes/Havana" },       {            "id": "icehouse", "wiki": "https://wiki.openstack.org/wiki/ReleaseNotes/Icehouse" },       {            "id": "juno", "wiki": "https://wiki.openstack.org/wiki/Releases" }   ],

Tracked drivers
Any contributor can add an untracked driver. The drivers section represents the list of tracked drivers. It has the following format: {           "project_id": "openstack/neutron", "vendor": "Tail-f", "name": "Tail-f NCS ML2 Mechanism Driver", "description": "The Tail-f NCS Mechanism Driver allows OpenStack to drive the Tail-f Network Control System (NCS)", "maintainers": [ {                   "name": "Luke Gorrie", "irc": "lukego", "email": "luke@tail-f.com", "launchpad_id": "lukego" }           ],            "releases": ["Havana", "Icehouse"], "wiki": "https://wiki.openstack.org/wiki/Neutron/ML2/Tail-f-NCS-neutron-ml2-driver", "ci": { "id": "tailfncs", "success_pattern": "Successful", "failure_pattern": "Failed" }       }

For each driver the following parameters may be specified:
 * project_id - OpenStack project - one from the list of projects configured in default_data.json (mandatory)
 * vendor - a driver's vendor (mandatory)
 * name - a driver's name (mandatory)
 * description - a driver's description
 * maintainers - information about driver maintainers (there could be several of them), including:
 * name - person's name (mandatory)
 * email - person's email
 * irc - person's nick in IRC
 * launchpad_id - person's id at Launchpad
 * wiki - link to the OpenStack wiki page with driver configuration instructions
 * releases - a list of OpenStack releases the driver supported in (filled for releases not tested by CI)
 * ci - information about CI
 * id - Gerrit id of external CI
 * success_pattern - pattern to parse CI's comment for a successful run
 * failure_pattern - pattern to parse CI's comment for a failed run.

Note: CI patterns are needed for determine result for non-voting CIs or if one vote corresponds to multiple drivers

Prerequisites
Before a vendor adds a new driver to the DriverLog, a driver must meet the following requirements:
 * A driver must have a unique and clear name
 * A driver must have a complete and full description. The text should answer the questions: "What does this driver enable? What features does it have?"
 * A driver must have a maintainer. A vendor should provide a maintainer's name and contacts (email, IRC nick).
 * A driver must have a dedicated page on OpenStack wiki or vendor's web site with configuration details. It's good to add a driver's section to the OpenStack configuration overview document.
 * A driver must be accepted to the OpenStack trunk.

Note: A driver can be deprecated in the current OpenStack release.''

Instructions
DriverLog uses Gerrit for code review.

Note: Check Gerrit Workflow to get the instructions there to upload your change to Gerrit.''
 * Open the JSON file.
 * Add a new driver section to the file.

{            "project_id": "openstack/neutron", "vendor": "Extreme Networks", "name": "Neutron Plugin", "description":"The Extreme Networks OpenStack plugin runs on Extreme XOS 15.3 and later releases. This plug-in is supported on all EXOS based platforms and includes the latest OpenStack API capabilities. The Extreme Networks OpenStack 2.0 plugin enables “Networking-as-a-Service” for both Physical and Virtual Extreme Networks.", "wiki": "http://extrcdn.extremenetworks.com/wp-content/uploads/2014/02/SDN_OpenStack_Install_Guide_Final1.pdf", "releases": ["Icehouse"] }

Note: Records in config file should be sorted by key - (project_id, vendor). Drivers are sorted in alphabetical order according to the vendor's name. The configuration file automatically validated against the config file scheme.
 * Send a pull request. Provide it with detailed comments on what was done.


 * A project DriverLog Core Engineer reviews a commit. If everything is OK. he/she merges an update.
 * A driver gets displayed in the Drivers list on the DriverLog Dashboard.
 * ‘CI tested’ column will show cross mark meaning that CI is not present.
 * 'In trunk' column will show a list of releases specified in field 'releases'.

Prerequisites
Before a vendor adds a new Fuel Plugin to the DriverLog, a plugin must meet the following requirements:
 * A plugin must have a unique and clear name
 * A plugin must have a complete and full description. The text should answer the questions: "What additional functionality does this plugin bring for Fuel? What features does it have?"
 * A plugin must have a maintainer. A vendor should provide a maintainer's name and contacts (email, IRC nick, link to Launchpad profile).
 * A plugin must have a link to the plugin's repo.

Instructions
DriverLog uses Gerrit for code review.

Note: Check Gerrit Workflow to get the instructions there to upload your change to Gerrit.''
 * Open the JSON file.
 * Add a new plugin section to the file.

{  "project_id": "openstack/fuel", "repo": "openstack/fuel-plugin-calico", "vendor": "Project Calico", "name": "Calico plugin for Fuel", "description": "This plugin provides Calico's Layer 3 Virtual Networking for Highly Scalable Data Centers", "maintainers": [ {                   "name": "Emma Gordon", "email": "emma@projectcalico.org", "launchpad_id": "emma-0" }           ],            "releases": ["Juno"], "wiki": "https://github.com/openstack/fuel-plugin-calico" },

Note: Records in config file should be sorted by key - (project_id, vendor). Drivers are sorted in alphabetical order according to the vendor's name. The configuration file automatically validated against the config file scheme.
 * Send a pull request. Provide it with detailed comments on what was done.


 * A project DriverLog Core Engineer reviews a commit. If everything is OK, then the commit is merged.
 * A plugin gets displayed in the Drivers list on the DriverLog Dashboard. Note, that it will not appear immediately after the commit is merged.
 * ‘CI tested’ column will show cross mark meaning that CI is not present.
 * 'In trunk' column will show a list of releases specified in field 'releases'.

Prerequisites
To be able to add a 'CI tested' mark a vendor must deploy an external voting CI environment that runs automatic tests against a driver on every commit. Instructions for setting up external CI can be found here:
 * Understanding the OpenStack CI System
 * Setting Up an External OpenStack Testing System – Part 1
 * Setting Up an External OpenStack Testing System – Part 2

Instructions
To configure the DriverLog to poll external CI for current verification status field 'ci_id' needs to be provided. 'ci_id' is Gerrit Id of external CI, see example below: {           "project_id": "openstack/neutron", "vendor": "Embrane", "name": "Neutron Plugin", "description": "Embrane Neutron plugin enables users to interface Neutron with the Embrane heleos platform.", "maintainers": [ {                   "name": "Ivar Lazzaro", "irc": "ivar-lazzaro", "email": "ivar@embrane.com" }           ],            "wiki": "https://wiki.openstack.org/wiki/Neutron/EmbraneNeutronPlugin", "ci": { "id": "eci", "success_pattern": "Build Successful", "failure_pattern": "Build Failed" }       },
 * The actual results of driver testing are stored in Gerrit. The backend runs as a scheduled background butch performing the following logic every 4 hours:
 * The DriverLog backend by using CI environment ID (ci_id) recognizes which Gerrit reviews are related to this driver.
 * The backend parses all merged patches from new to old searching for CI's vote or comment message that matches "success_pattern" or "failure_pattern".


 * A 'CI tested’ column will show a check mark meaning that CI is present.

Note: Only known ci_id’s from default_data.json are processed by the backend.

Release Notes
Release 0.1
 * Nova, Cinder, Neutron and Sahara projects included into DriverLog.
 * Information about supported drivers from projects’ wiki pages was manually converted into JSON config file:
 * https://wiki.openstack.org/wiki/HypervisorSupportMatrix
 * https://wiki.openstack.org/wiki/CinderSupportMatrix
 * https://wiki.openstack.org/wiki/Neutron#Using_Neutron

Source
https://github.com/openstack/driverlog

Pending Code Reviews
https://review.openstack.org/#/q/status:open+driverlog,n,z

Project space
https://launchpad.net/driverlog

Blueprints
https://blueprints.launchpad.net/driverlog

Bugs
https://bugs.launchpad.net/driverlog

Web-site
http://stackalytics.com/report/driverlog