Difference between revisions of "Oslo/CreatingANewLibrary"

Revision as of 22:33, 10 February 2014

Creating an Initial Repository for Import

There are three sources for new Oslo libraries:

Incubated code that is graduating.
A brand new library.
An existing library that we are adopting.

These subsections cover incubated code and new libraries. Existing libraries can skip ahead to "Importing the Repository into the CI System" below.

Choosing a Name

Oslo libraries are either named "oslo.something" or just "something". The oslo prefix is reserved for libraries which are expected to only be of use to OpenStack developers. Libraries that may be useful outside of OpenStack should have more generic names.

Graduating a Library from the Incubator

Graduating a library is a multi-step process. It isn't complicated, but there are a lot of details that are easy to miss. Please tread carefully.

Updating the Incubator

Update oslo-incubator/MAINTAINERS:

Set the status of the module(s) to "Graduating".

Verify all of the correct names and contact details are present.

Extracting a Clean Copy of the History

The oslo-incubator repository includes a script in tools/graduate.sh that can be used to

Importing the Repository into the CI System

openstack-infra/config

These instructions are based on the steps for creating a new Stackforge project with some variations, and apply to changes made in the openstack-infra/config git repository.

Add the project to the master project list

Edit modules/openstack_project/files/review.projects.yaml to add a new section like:

 - project: openstack/project-name
   description: Latest and greatest cloud stuff.
   upstream: git://github.com/awesumsauce/project-name.gi

The projects in the file need to be listed in alphabetical order.

Set the "upstream" URL to the repository created earlier. If the existing library being imported is already on stackforge, the import can be handled by renaming the repository instead of using the upstream URL, so leave that line out.

Add Gerrit permissions

Each project should have 2 groups. The first, "project-name-core", is the normal core group, with permission to +2 changes. The other, "project-name-milestone", is a (usually smaller) group able to push tags to trigger releases.

Create modules/openstack_project/files/gerrit/acls/stackforge/project-name.config:

 [access "refs/heads/*"]
       label-Code-Review = -2..+2 group project-name-core
       label-Approved = +0..+1 group project-name-core
       workInProgress = group project-name-core
 [access "refs/tags/*"]
       create = group project-name-milestone
       pushTag = group project-name-milestone
 [receive]
       requireChangeId = true
       requireContributorAgreement = true
 [submit]
       mergeContent = true

Update the Gerrit Group Members

Ask the Infra team to add you to both groups in gerrit, and then you can add other members.

The "oslo-core" group should be added to every sub-project core list (as a group, not individually).

The project PTL and Oslo PTL, at least, should be added to "project-name-milestone", and other developers who understand the release process can volunteer to be added as well.

Add Basic Jenkins Jobs

Establish the standard Python jobs, including publishing releases to PyPI and pre-release tarballs to tarballs.openstack.org.

Edit modules/openstack_project/files/jenkins_job_builder/config/projects.yaml to add:

- project:
   name: project-name
   github-org: openstack
   node: precise
   tarball-site: tarballs.openstack.org
   doc-publisher-site: docs.openstack.org
   jobs:
     - python-jobs
     - openstack-publish-jobs
     - pypi-jobs

Configure Zuul to Run Jobs

Zuul is the gate keeper. It watches for changes in gerrit to trigger the appropriate jobs. To start, establish the rules for the basic jobs already configured, but not the full devstack-gate jobs.

Edit modules/openstack_project/files/zuul/layout.yaml to add a section like:

 - name: openstack/project-name
   check:
     - gate-project-name-pep8
     - gate-project-name-python26
     - gate-project-name-python27
     - gate-project-name-python33
     - gate-project-name-pypy
   gate:
     - gate-project-name-pep8
     - gate-project-name-python26
     - gate-project-name-python27
     - gate-project-name-python33
     - gate-project-name-pypy
   post:
     - project-name-branch-tarball
   pre-release:
     - project-name-tarball
   release:
     - project-name-tarball:
       - project-name-pypi-upload:
         - post-mirror-python26
         - post-mirror-python27
         - post-mirror-python33

For projects with documentation hosted on readthedocs.org, include "hook-project-name-rtfd" under the post, pre-release, and release groups.

Add Project to the Requirements Mirror List

The global requirements repository (openstack/requirements) controls which packages are available in the PyPI mirror used for the CI system. It also automatically contributes updates to the requirements lists for OpenStack projects when the global requirements change.

Edit the projects.txt file to add the new library, adding "openstack/project-name" in the appropriate place in alphabetical order.

Add Project to the Governance Repository

Each project managed by an official program in OpenStack needs to be listed in reference/programs.yaml in the openstack/governance repository to indicate who owns the project so we know where ATCs voting rights extend.

Find the "Common Libraries" section in reference/programs.yaml and add the new project to the list of "other" projects under Oslo. (No Oslo projects are designated as core right now.)

Common Libraries:
  codename: Oslo
  ptl: Doug Hellmann (dhellmann)
  mission:
    To produce a set of python libraries containing code shared by OpenStack
    projects. The APIs provided by these libraries should be high quality,
    stable, consistent, documented and generally applicable.
  url: https://wiki.openstack.org/wiki/Oslo
  projects:
    other:
    - openstack/oslo-incubator
    - openstack/oslo.config
    - openstack/oslo.messaging
    - openstack/oslo.rootwrap
    - openstack/oslo.sphinx
    - openstack/oslo.version
    - openstack-dev/cookiecutter
    - openstack-dev/hacking
    - openstack-dev/pbr

First Testable Release

Marking Incubator Obsolete

After the first release of the new library, the status of the module(s) should be updated to "Obsolete." During this phase, only critical bug fixes will be allowed in the incubator version of the code. New features and minor bugs should be fixed in the released library, and effort should be spent focusing on having downstream projects consume the library.

After all integrated projects that use the code are using the library instead of the incubator, the module(s)_ can be deleted from the incubator.

Discussion on the mailing list here and here