Governance/Foundation/IncubationUpdate2013

Incubation Update Committee Recommendation Report for the OpenStack Board of Directors and the Technical Committee
Date: April 3, 2013

The Incubation Update Committee (IncUp) has completed the assigned analysis of the Incubation process and definition of core and offers the following report with recommendations to the OpenStack Technical Committee (TC) and OpenStack Board of Directors (Board).

Scope and Background
Perceptions around the current defined Incubator process have grown to infer that a project must either graduate to the status of an OpenStack Core Project (Core) or it must eventually fail to make progress or become abandoned. This committee was born from the need to change that perception and update the incubator process. Perception is changing due to the actions we've already put in place. The OpenStack community will grow to contain many projects critical to the success of OpenStack but in which it will not be prudent nor necessary to be destined for Core.

Since the bylaws define that the TC exercises the authority to add, combine, delete, or split modules from the OpenStack Project and as it is the responsibility of the Board to approve or reject additions, combinations, splits and deletions from the Core OpenStack Project it is important to update the current Incubator process to enable and promote efforts within the community while facilitating the TC and Board to work together for the proper advancement of OpenStack technologies.

Software that is included within the OpenStack project is generally described at:

[1] http://www.openstack.org/software/

Working together the committee examined the current incubator process. The scope of the effort was defined as:


 * 1) Update the definition and terms of the Incubator process
 * 2) Level set Incubator status expectations
 * 3) Refine the definition of Core
 * 4) Define multiple avenues for project entry, growth and incubator exit
 * 5) Upon completion of the effort, the TC and Board will approve, publish and promote the updated process

Committee Members
Secretary:
 * Alan Clark

Board members (3):
 * Monty Taylor
 * Rob Hirshfield (Alternate)
 * Randy Bias
 * Boris Renski (Alternate)
 * Kyle MacDonald
 * Eileen Evans (Alternate)

TC members (3):
 * Anne Gentle
 * Mark McLoughlin
 * Thierry Carrez
 * Russell Bryant (Alternate)

Foundation Staff:
 * Jonathan Bryce
 * Mark Collier (Alternate)

Project Inclusion Processes
Within OpenStack there are essentially two governance bodies, the Technical Committee (TC) and OpenStack Board (Board). Project status/labels include Incubation and Core. This proposal adds Integrated. Each of these are explained in the diagram and subsequent sections of this document.

The TC will manage the Incubation and Integrated project labels. Note that the TC has already begun to apply the Integrated label, specifically to the Ceilometer and Heat projects which graduated from Incubation in February.

The Board manages the Core label. The processes for achieving either Core and/or Integrated are mostly parallel. The Core label, and its goals, are closely intertwined with the trademark program. The trademark program's goal is to manage the experience of End users of OpenStack clouds.

The current list of incubated, integrated and core projects:

Ceilometer and Heat were recently graduated from Incubation and are considered Integrated projects.

There are no project in incubation at this time.



Note that library projects and supporting projects do not go through incubation but should undergo many of the same steps, gating and processes including trademark review.

For a complete list of projects, see: [11] https://wiki.openstack.org/wiki/Projects

New Projects to the OpenStack Community
New projects enter into the OpenStack community through the OpenStack Project Expansion Process.

The process is outlined within the following wiki page:

[2] https://wiki.openstack.org/wiki/Governance/Approved/NewProjectProcess

It is common practice today for projects to use a 'project name' such as Ceilometer and Heat. The guidelines for documentation is to use a descriptive name, not the project name, as much as possible.

As projects grow and progress, they need to adopt a descriptive service name. It is recommended that each project undergo a formal trademark search as early as possible and that such a search is required prior to a project graduating from Incubation. Determining a descriptive service name should be handle in conjunction with the Trademark usage program. Typically the project will be called OpenStack X when it becomes part of the integrated release. Therefore Ceilometer and Heat will become OpenStack Metering and OpenStack Orchestration (starting with the Havana cycle).

IncUp Recommendations:

 * 1) Projects apply to the TC to join Incubation. Their "incubation request" is assessed by the TC on the basis of the project's technical maturity and the appropriateness of its scope.
 * 2) Update the document, replacing references to Project Policy Board (PPB) with Technical Committee (TC) or OpenStack Board, replacing references to core with the appropriate reference to integrated or core status and aligning the outlined process with the IncUp results.
 * 3) Add reference to core application process
 * 4) No formal trademark search today. Before a project becomes associated with OpenStack need a process for this. TC will work with Foundation staff to incorporate this into the new project process. Incubator Application wiki page [4] should be updated to reflect this policy.

Incubation Process
The new project process includes references to the Incubation process. Incubation is the process managed by the TC through which a project becomes part of the coordinated, integrated OpenStack release. Projects apply to the TC to join Incubation. Their "incubation request" is assessed by the TC on the basis of the project's technical maturity and the appropriateness of its scope. The incubation process is defined at:

[3] http://wiki.openstack.org/Governance/Approved/Incubation

As the Incubation page outlines, to enter the incubation process the project submits the following application:

[4] http://wiki.openstack.org/Projects/IncubatorApplication

Prior to Incubation projects are able to utilize the following OpenStack resources:


 * CI via stackforge

Resources used from OpenStack as part of the incubation process:


 * Release manager help and training (incl. in weekly release status meeting)
 * CI
 * CI testing
 * Infrastructure team starts caring
 * OpenStack github org namespace
 * PTL or co-leads are named (not on TC)
 * PTL guide: [5] http://wiki.openstack.org/PTLguide
 * Release cycle: [6] http://wiki.openstack.org/ReleaseCycle
 * Branch model: [7] http://wiki.openstack.org/BranchModel
 * Release process: [8] http://wiki.openstack.org/ReleaseTeam/HowToReleas
 * CI docs: [9] http://ci.openstack.org/
 * Doc: provide guidance for tooling but not doc work
 * Integration: jenkins jobs
 * Weekly release team meeting tracking time
 * Daily tracking, release definitions through blueprint tagging, an overall release schedule, and time at the weekly Project meeting - Incubated = Priority 2 for release manager
 * DevStack as possible integration testing
 * Room at Summits to 'incorporate' and discuss; have some access to rooms to collaborate, depend upon need.
 * testing gated via DevStack
 * Gate testing
 * Qualitative measurement is done per project . TC mainly gauges alignment with project processes and resources
 * If project goes through Stackforge, CLA is signed in order to use that Gerrit-based system.

By end of incubation the project should be able to be part of the devstack integration testing gate, meaning it should consistently work with and not break the other projects.

IncUp Recommendations

 * 1) Incubation is the process managed by the TC through which a project becomes part of the coordinated, integrated OpenStack release.
 * 2) Incubation wiki page [3] – replace references to PPB
 * 3) Incubator Application wiki page [4] - replace references to PPB
 * 4) That exit from the Incubation process require a project to adopt a descriptive service name rather than to continue to use a project name. The Incubation wiki page [3] be updated to reflect this policy

Integrated
Projects which complete Incubation and which are part of a coordinated release will be referred to as "Integrated" in that release. Previously this was one of the meanings of the term "Core".

At the end of every development cycle, before PTL elections are held, the TC carries out an "end of cycle graduation review" of the projects currently in Incubation. Projects will graduate from Incubation if they are considered mature, stable in design, complementary in scope and well aligned to the development cycle and processes. Graduating projects will be part of the next development cycle and fully "Integrated" in subsequent releases. Under the current 6 month release cycle, this means a graduating project will be Integrated in the release 8 months after the TC deems it ready to graduate from Incubation.

Resources used from OpenStack to support Integrated (and Core) projects:


 * 1) Track time at the Summit- integrated projects all get a track, incubated projects are all in the same track, no special casing for core projects (all projects should get the space and time they need, not really a scarce resource)
 * 2) Release management - daily tracking, release definitions through blueprint tagging, an overall release schedule, and time at the weekly Project meeting- integrated projects all get RM attention (no special casing for core projects). Incubated projects get time if there is any left, end of cycle graduation reviews should assess if the incubated projects got enough support from the rest of us to be able to succeed
 * 3) Testing gated via DevStack- Integrated projects should all be made part of the gate. When a project breaks another project, a project can expect responsiveness for a fix
 * 4) Bug triaging efforts, quality assurance assistance/systems
 * 5) Continuous integration assistance/systems
 * 6) Documentation assistance/systems

Integrated projects duties include:


 * 1) Gating – an expectation that when a project breaks another project, a project can expect responsiveness for a fix
 * 2) Following the processes developed during Incubation
 * 3) Following a defined lifecycle

IncUp Recommendations

 * 1) Projects which are part of a coordinated release should be referred to as "Integrated" in that release. Previously this was one of the meanings of the term "Core".
 * 2) The Incubation wiki page [3] will be updated to reflect the policy outlined in this section
 * 3) The following sentence will be removed from the TC's charter: "The TC recommends projects for Core status addition, combination, split or deletion of the Board of Directors, which has the sole authority to approve them".
 * 4) The TC charter

[10] http://wiki.openstack.org/Governance/Foundation/TechnicalCommittee

will be updated to replace the word "core" with "integrated" as per the policy outlined in this section. The effect of this change will be that PTLs for projects graduating from Incubation will be automatically granted a seat on the next TC.

[Note: those last two items were already implemented as part of the interim resolutions needed to proceed with the Grizzly end of cycle graduation review]

Core
Core is a label the Board can attach to a project that is part of the regular integrated release.

Bylaws definition of Core The “OpenStack Project” shall consist of a “Core OpenStack Project,” library projects, gating proj and supporting projects. The Core OpenStack Project means the software modules which are part of an integrated release and for which an OpenStack trademark may be used. The other modules which are part of the OpenStack Project, but not the Core OpenStack Project may not be identified using the OpenStack trademark except when distributed with the Core OpenStack Project. ... On formation of the Foundation, the Core OpenStack Project is the Block Storage, Compute, Dashboard, Identity Service, Image Service, Networking, and Object Storage modules.

The criterion that the board will use to guide a decision on whether to apply the Core label to an Integrated project of good standing are as follows:


 * 1) Project has passed the Incubation process and is considered an Integrated project
 * 2) Project and software is part of integrated release, in good standing with TC processes
 * 3) Interfaces are well defined as per TC processes
 * 4) Stability of component / well tested
 * 5) Well described for use
 * 6) large diverse group of contributors
 * 7) The software provides unique technology necessary for interoperability of OpenStack.
 * 8) Project is essential to part of all or virtually all OpenStack components.
 * 9) The software is a critical component, necessary for interoperability between OpenStack clouds
 * 10) It is core software necessary to power a competitive Infrastructure-as-a-Service solution complete with compute, storage, and networking capabilities in a manner that is API accessible
 * 11) The software is widely adopted by cloud service providers, distributions and other OpenStack members
 * 12) Reflects what the ecosystem vendors (such as Tool vendors) need and want to target the platform and consider essential
 * 13) The project promotes innovation
 * 14) The project enables other technologies and OpenStack projects to move forward
 * 15) The software does not preclude 3rd party enhancements or additions
 * 16) Framework coherence with core projects

To arrive at the core criteria and other recommendations outlined in this report, the IncUp committee generated and reviewed several use cases:


 * 'End User' use case,
 * 'SaaS' use case (someone who is simply using a public IaaS OpenStack cloud as the infrastructure for their SaaS app),
 * 'IaaS' use case (someone trying to instantiate services on a cloud),
 * 'Packager/Deployer, OpenStack - based Product Businesses' use case
 * 'Install Cloud from source / Joe Operator' (University/Research Centers/...) use case.

Those use cases are currently stored on the IncUp etherpad.

IncUp Recommendations

 * 1) Add wiki page outlining the application a project should use to apply for Core status. Page should include the criterion outlined above.
 * 2) Describing the branding and trademark usage for new projects, projects in incubation, integrated and core projects is not within the scope of work for the IncUp committee. This committee recommends that the board review the current branding and trademark documents to provide clear guidance to the projects on use of the OpenStack brand.
 * 3) The scope of IncUp does not address those users of OpenStack who are consuming the API and wish to claim compatibility with OpenStack. Examples include clients needing a way to claim they have been validated against OpenStack Grizzly. This issue is left to the board to address through other work efforts.

Conclusion – Board and TC Requested Actions
From the IncUp efforts and recommendations approved at the February board meeting the TC has begun to apply the Integrated term in their efforts. Specifically Ceilometer and Heat have completed the Incubation process to Integrated status.

It is important for the board to review the recommendations included in this report and that the board

spend time reviewing the proposed set of Core criteria. IncUp recommends that the board review the criteria using existing set of OpenStack projects to guide list refinement.

IncUp invites the board and TC to review and approve the recommendations in a joint meeting. If that is not feasible, then when the board has completed it's review IncUp will invite the TC to conduct a similar review. At that point the IncUp committee will implement the recommendations outlined within this report and will then consider the committee effort complete and will disband.