Difference between revisions of "Oslo/CreatingANewLibrary"
(→Setting up Gate Testing) |
(→Launchpad Settings) |
||
(115 intermediate revisions by 9 users not shown) | |||
Line 1: | Line 1: | ||
− | == | + | == Before You Start == |
− | + | This is a long process. It's long because it has to be, not because we want it to be. If you follow it, everything will be fine. | |
− | + | It is important that you perform all of the steps, in the order they are given here. Don't skip any steps. Don't try to do things in parallel. Don't jump around. | |
− | |||
− | |||
− | + | == Project Creator's Guide == | |
− | + | The [http://docs.openstack.org/infra/manual/creators.html Project Creator's Guide] includes many of the details for creating a repository. It is maintained by the Infrastructure team, and should be up to date with the current process. | |
− | + | Read that guide before continuing. Some steps below will refer to sections of that guide, and you will need to use both documents together to complete this process. | |
− | == | + | == The Spec == |
− | + | The Oslo team reviews plans for large changes, such as graduations, using the specs process common to the feature review processes of other teams within OpenStack. To prepare to graduate or adopt a library, you will need to file a spec. | |
− | === | + | === Blueprint === |
− | + | First, create a blueprint in the oslo-incubator repository on launchpad. Name the blueprint graduate-oslo-foo, replacing foo with the name of the library. | |
− | + | === Spec === | |
− | + | In the openstack/oslo-specs repository, copy specs/graduation-template.rst to create a new spec using the name of your blueprint under the release code name directory. For example, specs/kilo/graduate-oslo-reports.rst. Edit the new file, filling in the details, then submit it for review just like any other code review. | |
− | + | The spec template includes questions that come later in this document, for which you will need to provide an answer at this point. This is the one exception to the "don't read ahead or skip around" rule mentioned earlier. Collect all of the information you need to fill in the spec. | |
− | + | == Choosing a Name == | |
− | + | In addition to the rules described in the [http://docs.openstack.org/infra/manual/creators.html Project Creator's Guide], there are currently three naming schemes used for Oslo libraries: | |
+ | # oslo.something - Used for production runtime dependencies of OpenStack projects. Using the oslo. namespace for any other type of package will cause conflicts due to some limitations in the way Python and Pip handle namespace packages. | ||
+ | # oslosomething - Used for non-production or non-runtime dependencies of OpenStack projects. If you're planning to use a name like this, please discuss it with the Oslo team first - we aren't sure we like this name scheme and may suggest an alternative. | ||
+ | # something - Used for libraries that may be generally useful outside of OpenStack. Note: you should verify that the name you are proposing is not in use on the Python Package Index (https://pypi.python.org/pypi). | ||
− | == | + | == Graduating a Library from the Incubator == |
− | + | There are three sources for new Oslo libraries: | |
− | + | # Incubated code that is graduating. | |
+ | # A brand new library. | ||
+ | # An existing library that we are adopting. | ||
− | + | Most of this document covers incubated code. Brand new and existing libraries can skip ahead to "Importing the Repository into the CI System" below. | |
− | + | ''Graduating a library is a multi-step process. It isn't complicated, but there are a lot of details that are easy to miss. Please tread carefully.'' | |
− | + | The goal with incubated code is to extract it from the incubator with all of its history, and publish the results as a new library. This means that a new repository must be created outside of gerrit, and then imported, to avoid having to re-review all of the existing changesets. A lot of that process is automated, but there are inevitably some manual steps -- especially to ensure that the tests work properly. | |
− | |||
− | |||
− | + | Perform the steps in this section before following the directions in the [http://docs.openstack.org/infra/manual/creators.html Project Creator's Guide], then use the repository you create as the "upstream" value when adding the project to gerrit. | |
− | + | === Updating the Status in the Incubator === | |
− | + | '''Update oslo-incubator/MAINTAINERS''' | |
− | + | Set the status of the module(s) to "Graduating". | |
− | + | Verify all of the correct names and contact details are present. | |
− | + | If only some modules in a sub-package under openstack/common are being graduated, add comments to those files clearly indicating that they should not be modified during graduation. | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | + | '''Remove dependencies on oslo logging code''' | |
− | + | For the files that will be graduating, submit a change to oslo-incubator that switches them to use standard Python logging instead of any Oslo logging they may be using (either from incubator or oslo.log). The logging code is now able to add context information to all third-party library log messages, so it makes sense to have graduated Oslo libraries behave the same as any other third-party library. | |
− | + | We want this done in incubator before the new repo is created so we can review the changes, but don't have to pull in logging dependencies that will just be removed immediately. This shouldn't be a problem for anyone continuing to use the incubator versions of the files either. | |
− | + | === Extracting a Clean Copy of the History === | |
− | + | Start by checking out a copy of the oslo-incubator to use for running the tool: | |
− | + | cd /tmp | |
+ | mkdir graduation | ||
+ | cd graduation | ||
+ | git clone git://git.openstack.org/openstack/oslo-incubator | ||
− | + | Now clone a second copy of the repository to be edited: | |
− | + | git clone git://git.openstack.org/openstack/oslo-incubator oslo.i18n | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | + | The graduation script works by editing the git history of the current repository, removing commits that do not touch the files you want to keep. It then rearranges the resulting files and sets up the oslo package namespace. To do these things, it needs to know the name of the new library being created and the names of the files you wish to keep. | |
− | + | cd oslo.i18n | |
+ | ../oslo-incubator/tools/graduate.sh i18n openstack/common/gettextutils.py tests/unit/test_gettext.py | ||
− | + | The script is fairly I/O intensive, and will take a while to run. It logs all of its output to /tmp/oslo-graduate.XXXX/output.log, and other scratch files are kept in the same directory. | |
− | + | === Manual Fixes === | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | + | It is not possible to completely automate the process of reorganizing the library code and ensuring that it will work, so there are several manual tasks to be performed next. | |
− | + | # Add missing dependencies | |
+ | # Fix broken imports within the current repository resulting from files being renamed | ||
+ | # Make sure the tests run -- not just make sure tox passes, make sure it is actually ''running'' the tests (moving files around makes this break silently some times) | ||
+ | # Test the new library with some of the projects that use the incubator version now | ||
− | + | At this point, make '''as few changes as possible''', just to make the test jobs pass. Larger changes to the API should go through the normal review process. | |
− | + | Allowed changes include: | |
− | + | * Broken imports | |
+ | * PEP8/test failures | ||
+ | * Syncing dependencies from the incubator | ||
− | + | Do not: | |
− | + | * Enable new tests or PEP8 checks | |
+ | * Add new library dependencies that were not already in use by the incubated code | ||
− | === | + | === Sync Required Tools from Incubator === |
− | + | Some of the tools common to all Oslo libraries reside in the oslo-incubator. These need to be synced into the new repository before it is imported. | |
− | + | # clone a copy of the oslo-incubator repository | |
+ | # Add appropriate modules and scripts to /path/to/newproject/openstack-common.conf | ||
+ | # In oslo-incubator, run: ./update.sh /path/to/newproject | ||
+ | # In /path/to/newproject, add the new files and commit any changes | ||
− | + | === Publish Your Results === | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | + | When you have a clean version of the library, publish it to a public git repository (e.g., on github). This copy of the project history will be used in subsequent steps. | |
− | + | === Receive Oslo Team Sign-off === | |
+ | |||
+ | Before proceeding to import the library into the OpenStack CI system, post to the openstack-dev mailing list asking for the Oslo team to review the public repository. | ||
− | == | + | == Adding the Repository to the CI System == |
− | + | Follow the steps in the [http://docs.openstack.org/infra/manual/creators.html Project Creator's Guide] for importing your new repository, using the repository you have created as the "upstream" value when adding the project to gerrit. | |
− | + | == Launchpad Settings == | |
− | + | The Project Creator's Guide describes [http://docs.openstack.org/infra/manual/creators.html#set-up-launchpad the steps for setting up a new Launchpad project]. | |
− | |||
− | |||
− | |||
− | + | Most Oslo projects use the [https://launchpad.net/~oslo-drivers Oslo Drivers] team for "Maintainer" and "Driver". Libraries with a large number of contributors who are not part of the Oslo core team can set up their own Maintainer/Driver team, but should include oslo-drivers as a sub-team (see [https://launchpad.net/oslo.vmware oslo.vmware] and [https://launchpad.net/oslo.policy oslo.policy] for examples). | |
− | + | == Add the Project to the Oslo Library List == | |
− | + | Edit [[Oslo]] wiki page to add the new project to the list of libraries we maintain. Include a brief description. | |
− | + | == Verify That Gerrit and the Test Jobs are Working == | |
+ | |||
+ | The next step is to verify that you can submit a change request for the repository, the tests run successfully, you have permission to approve changes, and the release process works. | ||
== Prepare an Initial Release == | == Prepare an Initial Release == | ||
Line 196: | Line 151: | ||
If you are creating a brand new library, add some code and tests to provide some minimal functionality useful to another project. | If you are creating a brand new library, add some code and tests to provide some minimal functionality useful to another project. | ||
− | === | + | === Provide Basic Documentation === |
− | |||
− | |||
− | |||
− | |||
− | + | Update the README.rst file to include a paragraph describing the new library. | |
− | + | Update the rest of the documentation with information about the public API, tips on adopting it, etc. | |
=== Tagging a Release === | === Tagging a Release === | ||
− | To verify that the release machinery works, push a signed tag to the "gerrit" remote. Use the smallest version number possible. If this is the first release, use 0.1. If other releases of the library exist, choose an appropriate next version number using | + | To verify that the release machinery works, push a signed tag to the "gerrit" remote. Use the smallest version number possible. If this is the first release, use "0.1.0". If other releases of the library exist, choose an appropriate next version number using [[Oslo/VersioningPolicy]]. |
Run: | Run: | ||
Line 221: | Line 172: | ||
git os-job $version | git os-job $version | ||
− | == | + | == Remove Incubated Code == |
− | After the first successful release of a newly graduated library, | + | After the first successful release of a newly graduated library, the module(s) should be removed from the incubator. During this phase, critical bug fixes will be merged to the previous release stable branch in the incubator version of the code. New features and minor bugs should be fixed in the released library, and effort should be spent focusing on having downstream projects consume the library. |
+ | |||
+ | === Avoid Rewriting References in Incubator's update.py === | ||
+ | |||
+ | The oslo-incubator/update.py script rewrites references to "oslo" with the name of the project receiving the copy of the code. Now that the code is no longer being consumed from the incubator, any code remaining in the incubator that refers to it needs to refer to the released library. Modify update.py so that it no longer rewrites references by adding it to the list of libraries known in the _copy_file() function. Look for "for oslo_module in ..." around line 158. | ||
== Allowing Other OpenStack Projects to Use Your Library == | == Allowing Other OpenStack Projects to Use Your Library == | ||
Line 231: | Line 186: | ||
=== Update the Global Requirements List === | === Update the Global Requirements List === | ||
− | Check out the openstack/requirements git repository and modify global-requirements.txt to | + | Check out the openstack/requirements git repository and modify global-requirements.txt to: |
− | + | # add the new library | |
+ | # add any of the library's direct dependencies that are not already listed | ||
− | == | + | === Document Migration Process === |
− | + | If the new library is graduating from code that was already being used through the incubator, create a brief description of the types of changes that are needed to adopt the library. For example, if the API has changed or if a common pattern is needed to create an integration module, describe that and provide an example. This should be a wiki page under [[Oslo]] with a name like Oslo/MigratingTo-project-name (keeping this in the wiki will make it easier for project liaisons to add tips and notes). | |
− | + | Link to the documentation under [[Oslo#Libraries]]. | |
− | + | == Updating devstack to Enable Gate Testing == | |
− | + | The devstack gate jobs install all OpenStack projects from source so that the appropriate git revisions (head, or revisions in the merge queue) are tested together. To include the new library in these tests, it needs to be included in the list of projects in the devstack gate wrapper script. For the same feature to work for developers outside of the gate, the project needs to be added to the Oslo portion of devstack. | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
Check out openstack-dev/devstack. | Check out openstack-dev/devstack. | ||
Line 259: | Line 207: | ||
NEWPROJECT_DIR=$DEST/newproject | NEWPROJECT_DIR=$DEST/newproject | ||
− | Edit the "install_oslo()" function in lib/oslo to add commands to check out the repository: | + | Edit the "install_oslo()" function in lib/oslo to add commands to check out the repository. The libraries need to be installed in order so that the lower-level packages are installed first (this avoids having a library installed from a package and then re-installed from source): |
function install_oslo() { | function install_oslo() { | ||
Line 274: | Line 222: | ||
NEWPROJECT_BRANCH=${NEWPROJECT_BRANCH:-master} | NEWPROJECT_BRANCH=${NEWPROJECT_BRANCH:-master} | ||
− | + | '''Wait for these changes to be accepted before proceeding.''' | |
+ | |||
+ | == Add Link to Your Developer Documentation == | ||
− | + | Update the http://docs.openstack.org/developer/openstack-projects.html page with a link to your documentation by checking out the openstack/openstack-manuals repository and edititing www/developer/openstack-projects.html. | |
+ | |||
+ | == Update Consuming Projects == | ||
+ | |||
+ | Now that the library is fully integrated with the rest of the OpenStack test infrastructure, projects can be updated to use it safely. Send email to the openstack-dev mailing list notifying the community that the library is available and ready for use. | ||
+ | |||
+ | Coordinate with the development teams for each project to find a time in their dev cycle when they can work with you on the update and review the necessary changes. The [[Oslo/ProjectLiaisons|liaisons]] can help with this. | ||
+ | |||
+ | Track adoption progress using the etherpad for the release cycle. For example, https://etherpad.openstack.org/p/juno-oslo-adoption-status | ||
+ | |||
+ | The PTLs of the other projects may also ask for blueprints or bugs to track the work of migrating within their projects. Use whatever process they are comfortable with. | ||
+ | |||
+ | Try to set the topic for changes in the other projects to refer to your graduation blueprint, either on the command line or by referencing the blueprint in the git commit message. This makes it easier to track all of the work for the blueprint, across all of our repositories. | ||
+ | |||
+ | == Checklist == | ||
− | + | You can paste this task list into the "Work Items" section of the blueprint to make keeping track of the steps easier. | |
− | + | Create Launchpad project: TODO | |
+ | Create Launchpad bug tracker: TODO | ||
+ | Create Launchpad blueprint tracker: TODO | ||
+ | Change owner of Launchpad project: TODO | ||
+ | Give openstackci Owner permissions on PyPI: TODO | ||
+ | Create Initial Repository: TODO | ||
+ | Update MAINTAINERS in incubator with status and name: TODO | ||
+ | Remove Oslo logging calls in incubator: TODO | ||
+ | Fix the output of graduate.sh: TODO | ||
+ | Sync tools from incubator: TODO | ||
+ | Publish git repo: TODO | ||
+ | Oslo team review new repository: TODO | ||
+ | openstack/governance reference/programs.yaml: TODO | ||
+ | openstack-infra/project-config - gerrit/projects.yaml: TODO | ||
+ | openstack-infra/project-config - gerrit/acls/openstack/project-name.config: TODO | ||
+ | openstack-infra/project-config - jenkins/jobs/projects.yaml: TODO | ||
+ | openstack-infra/project-config - zuul/layout.yaml: TODO | ||
+ | openstack-infra/project-config - gerritbot/channels.yaml: TODO | ||
+ | Update Gerrit Groups and ACLs: TODO | ||
+ | openstack-infra/devstack-gate - devstack-vm-gate-wrap.sh: TODO | ||
+ | openstack/requirements projects.txt: TODO | ||
+ | Update list of libraries on Oslo wiki page: TODO | ||
+ | Make the library do something: TODO | ||
+ | Update the README.rst: TODO | ||
+ | Tag a release: TODO | ||
+ | Remove graduated code from oslo-incubator: TODO | ||
+ | Update oslo-incubator/update.py to not rewrite references to the library: TODO | ||
+ | openstack/requirements - global-requirements.txt: TODO | ||
+ | Document Migration Process: TODO | ||
+ | openstack-dev/devstack - lib/oslo: TODO | ||
+ | openstack-dev/devstack - stackrc: TODO | ||
+ | Update project list on docs.openstack.org: TODO |
Latest revision as of 21:18, 12 January 2015
Contents
- 1 Before You Start
- 2 Project Creator's Guide
- 3 The Spec
- 4 Choosing a Name
- 5 Graduating a Library from the Incubator
- 6 Adding the Repository to the CI System
- 7 Launchpad Settings
- 8 Add the Project to the Oslo Library List
- 9 Verify That Gerrit and the Test Jobs are Working
- 10 Prepare an Initial Release
- 11 Remove Incubated Code
- 12 Allowing Other OpenStack Projects to Use Your Library
- 13 Updating devstack to Enable Gate Testing
- 14 Add Link to Your Developer Documentation
- 15 Update Consuming Projects
- 16 Checklist
Before You Start
This is a long process. It's long because it has to be, not because we want it to be. If you follow it, everything will be fine.
It is important that you perform all of the steps, in the order they are given here. Don't skip any steps. Don't try to do things in parallel. Don't jump around.
Project Creator's Guide
The Project Creator's Guide includes many of the details for creating a repository. It is maintained by the Infrastructure team, and should be up to date with the current process.
Read that guide before continuing. Some steps below will refer to sections of that guide, and you will need to use both documents together to complete this process.
The Spec
The Oslo team reviews plans for large changes, such as graduations, using the specs process common to the feature review processes of other teams within OpenStack. To prepare to graduate or adopt a library, you will need to file a spec.
Blueprint
First, create a blueprint in the oslo-incubator repository on launchpad. Name the blueprint graduate-oslo-foo, replacing foo with the name of the library.
Spec
In the openstack/oslo-specs repository, copy specs/graduation-template.rst to create a new spec using the name of your blueprint under the release code name directory. For example, specs/kilo/graduate-oslo-reports.rst. Edit the new file, filling in the details, then submit it for review just like any other code review.
The spec template includes questions that come later in this document, for which you will need to provide an answer at this point. This is the one exception to the "don't read ahead or skip around" rule mentioned earlier. Collect all of the information you need to fill in the spec.
Choosing a Name
In addition to the rules described in the Project Creator's Guide, there are currently three naming schemes used for Oslo libraries:
- oslo.something - Used for production runtime dependencies of OpenStack projects. Using the oslo. namespace for any other type of package will cause conflicts due to some limitations in the way Python and Pip handle namespace packages.
- oslosomething - Used for non-production or non-runtime dependencies of OpenStack projects. If you're planning to use a name like this, please discuss it with the Oslo team first - we aren't sure we like this name scheme and may suggest an alternative.
- something - Used for libraries that may be generally useful outside of OpenStack. Note: you should verify that the name you are proposing is not in use on the Python Package Index (https://pypi.python.org/pypi).
Graduating a Library from the Incubator
There are three sources for new Oslo libraries:
- Incubated code that is graduating.
- A brand new library.
- An existing library that we are adopting.
Most of this document covers incubated code. Brand new and existing libraries can skip ahead to "Importing the Repository into the CI System" below.
Graduating a library is a multi-step process. It isn't complicated, but there are a lot of details that are easy to miss. Please tread carefully.
The goal with incubated code is to extract it from the incubator with all of its history, and publish the results as a new library. This means that a new repository must be created outside of gerrit, and then imported, to avoid having to re-review all of the existing changesets. A lot of that process is automated, but there are inevitably some manual steps -- especially to ensure that the tests work properly.
Perform the steps in this section before following the directions in the Project Creator's Guide, then use the repository you create as the "upstream" value when adding the project to gerrit.
Updating the Status in the Incubator
Update oslo-incubator/MAINTAINERS
Set the status of the module(s) to "Graduating".
Verify all of the correct names and contact details are present.
If only some modules in a sub-package under openstack/common are being graduated, add comments to those files clearly indicating that they should not be modified during graduation.
Remove dependencies on oslo logging code
For the files that will be graduating, submit a change to oslo-incubator that switches them to use standard Python logging instead of any Oslo logging they may be using (either from incubator or oslo.log). The logging code is now able to add context information to all third-party library log messages, so it makes sense to have graduated Oslo libraries behave the same as any other third-party library.
We want this done in incubator before the new repo is created so we can review the changes, but don't have to pull in logging dependencies that will just be removed immediately. This shouldn't be a problem for anyone continuing to use the incubator versions of the files either.
Extracting a Clean Copy of the History
Start by checking out a copy of the oslo-incubator to use for running the tool:
cd /tmp mkdir graduation cd graduation git clone git://git.openstack.org/openstack/oslo-incubator
Now clone a second copy of the repository to be edited:
git clone git://git.openstack.org/openstack/oslo-incubator oslo.i18n
The graduation script works by editing the git history of the current repository, removing commits that do not touch the files you want to keep. It then rearranges the resulting files and sets up the oslo package namespace. To do these things, it needs to know the name of the new library being created and the names of the files you wish to keep.
cd oslo.i18n ../oslo-incubator/tools/graduate.sh i18n openstack/common/gettextutils.py tests/unit/test_gettext.py
The script is fairly I/O intensive, and will take a while to run. It logs all of its output to /tmp/oslo-graduate.XXXX/output.log, and other scratch files are kept in the same directory.
Manual Fixes
It is not possible to completely automate the process of reorganizing the library code and ensuring that it will work, so there are several manual tasks to be performed next.
- Add missing dependencies
- Fix broken imports within the current repository resulting from files being renamed
- Make sure the tests run -- not just make sure tox passes, make sure it is actually running the tests (moving files around makes this break silently some times)
- Test the new library with some of the projects that use the incubator version now
At this point, make as few changes as possible, just to make the test jobs pass. Larger changes to the API should go through the normal review process.
Allowed changes include:
- Broken imports
- PEP8/test failures
- Syncing dependencies from the incubator
Do not:
- Enable new tests or PEP8 checks
- Add new library dependencies that were not already in use by the incubated code
Sync Required Tools from Incubator
Some of the tools common to all Oslo libraries reside in the oslo-incubator. These need to be synced into the new repository before it is imported.
- clone a copy of the oslo-incubator repository
- Add appropriate modules and scripts to /path/to/newproject/openstack-common.conf
- In oslo-incubator, run: ./update.sh /path/to/newproject
- In /path/to/newproject, add the new files and commit any changes
Publish Your Results
When you have a clean version of the library, publish it to a public git repository (e.g., on github). This copy of the project history will be used in subsequent steps.
Receive Oslo Team Sign-off
Before proceeding to import the library into the OpenStack CI system, post to the openstack-dev mailing list asking for the Oslo team to review the public repository.
Adding the Repository to the CI System
Follow the steps in the Project Creator's Guide for importing your new repository, using the repository you have created as the "upstream" value when adding the project to gerrit.
Launchpad Settings
The Project Creator's Guide describes the steps for setting up a new Launchpad project.
Most Oslo projects use the Oslo Drivers team for "Maintainer" and "Driver". Libraries with a large number of contributors who are not part of the Oslo core team can set up their own Maintainer/Driver team, but should include oslo-drivers as a sub-team (see oslo.vmware and oslo.policy for examples).
Add the Project to the Oslo Library List
Edit Oslo wiki page to add the new project to the list of libraries we maintain. Include a brief description.
Verify That Gerrit and the Test Jobs are Working
The next step is to verify that you can submit a change request for the repository, the tests run successfully, you have permission to approve changes, and the release process works.
Prepare an Initial Release
Make Your Library Useful
Before going any farther, make the library do something useful.
If you are importing an existing library with features, you can go ahead.
If you are creating a brand new library, add some code and tests to provide some minimal functionality useful to another project.
Provide Basic Documentation
Update the README.rst file to include a paragraph describing the new library.
Update the rest of the documentation with information about the public API, tips on adopting it, etc.
Tagging a Release
To verify that the release machinery works, push a signed tag to the "gerrit" remote. Use the smallest version number possible. If this is the first release, use "0.1.0". If other releases of the library exist, choose an appropriate next version number using Oslo/VersioningPolicy.
Run:
git tag -s -m "descriptive message" $version git push gerrit $version
Wait a little while for the pypi job to run and publish the release.
If you need to check the logs, you can use the "git-os-job" plugin (https://pypi.python.org/pypi/git-os-job):
git os-job $version
Remove Incubated Code
After the first successful release of a newly graduated library, the module(s) should be removed from the incubator. During this phase, critical bug fixes will be merged to the previous release stable branch in the incubator version of the code. New features and minor bugs should be fixed in the released library, and effort should be spent focusing on having downstream projects consume the library.
Avoid Rewriting References in Incubator's update.py
The oslo-incubator/update.py script rewrites references to "oslo" with the name of the project receiving the copy of the code. Now that the code is no longer being consumed from the incubator, any code remaining in the incubator that refers to it needs to refer to the released library. Modify update.py so that it no longer rewrites references by adding it to the list of libraries known in the _copy_file() function. Look for "for oslo_module in ..." around line 158.
Allowing Other OpenStack Projects to Use Your Library
OpenStack projects share a common global requirements list so that all components can be installed together on the same system. To allow other projects to use your library, you need to update that requirements list.
Update the Global Requirements List
Check out the openstack/requirements git repository and modify global-requirements.txt to:
- add the new library
- add any of the library's direct dependencies that are not already listed
Document Migration Process
If the new library is graduating from code that was already being used through the incubator, create a brief description of the types of changes that are needed to adopt the library. For example, if the API has changed or if a common pattern is needed to create an integration module, describe that and provide an example. This should be a wiki page under Oslo with a name like Oslo/MigratingTo-project-name (keeping this in the wiki will make it easier for project liaisons to add tips and notes).
Link to the documentation under Oslo#Libraries.
Updating devstack to Enable Gate Testing
The devstack gate jobs install all OpenStack projects from source so that the appropriate git revisions (head, or revisions in the merge queue) are tested together. To include the new library in these tests, it needs to be included in the list of projects in the devstack gate wrapper script. For the same feature to work for developers outside of the gate, the project needs to be added to the Oslo portion of devstack.
Check out openstack-dev/devstack.
Edit lib/oslo to add a variable defining where the source should go:
NEWPROJECT_DIR=$DEST/newproject
Edit the "install_oslo()" function in lib/oslo to add commands to check out the repository. The libraries need to be installed in order so that the lower-level packages are installed first (this avoids having a library installed from a package and then re-installed from source):
function install_oslo() { ... git_clone $NEWPROJECT_REPO $NEWPROJECT_DIR $NEWPROJECT_BRANCH setup_develop $NEWPROJECT_DIR ... }
Edit stackrc to add the other variables needed for configuring the new library:
# new-project NEWPROJECT_REPO=${NEWPROJECT_REPO:-${GIT_BASE}/openstack/new-project.git} NEWPROJECT_BRANCH=${NEWPROJECT_BRANCH:-master}
Wait for these changes to be accepted before proceeding.
Add Link to Your Developer Documentation
Update the http://docs.openstack.org/developer/openstack-projects.html page with a link to your documentation by checking out the openstack/openstack-manuals repository and edititing www/developer/openstack-projects.html.
Update Consuming Projects
Now that the library is fully integrated with the rest of the OpenStack test infrastructure, projects can be updated to use it safely. Send email to the openstack-dev mailing list notifying the community that the library is available and ready for use.
Coordinate with the development teams for each project to find a time in their dev cycle when they can work with you on the update and review the necessary changes. The liaisons can help with this.
Track adoption progress using the etherpad for the release cycle. For example, https://etherpad.openstack.org/p/juno-oslo-adoption-status
The PTLs of the other projects may also ask for blueprints or bugs to track the work of migrating within their projects. Use whatever process they are comfortable with.
Try to set the topic for changes in the other projects to refer to your graduation blueprint, either on the command line or by referencing the blueprint in the git commit message. This makes it easier to track all of the work for the blueprint, across all of our repositories.
Checklist
You can paste this task list into the "Work Items" section of the blueprint to make keeping track of the steps easier.
Create Launchpad project: TODO Create Launchpad bug tracker: TODO Create Launchpad blueprint tracker: TODO Change owner of Launchpad project: TODO Give openstackci Owner permissions on PyPI: TODO Create Initial Repository: TODO Update MAINTAINERS in incubator with status and name: TODO Remove Oslo logging calls in incubator: TODO Fix the output of graduate.sh: TODO Sync tools from incubator: TODO Publish git repo: TODO Oslo team review new repository: TODO openstack/governance reference/programs.yaml: TODO openstack-infra/project-config - gerrit/projects.yaml: TODO openstack-infra/project-config - gerrit/acls/openstack/project-name.config: TODO openstack-infra/project-config - jenkins/jobs/projects.yaml: TODO openstack-infra/project-config - zuul/layout.yaml: TODO openstack-infra/project-config - gerritbot/channels.yaml: TODO Update Gerrit Groups and ACLs: TODO openstack-infra/devstack-gate - devstack-vm-gate-wrap.sh: TODO openstack/requirements projects.txt: TODO Update list of libraries on Oslo wiki page: TODO Make the library do something: TODO Update the README.rst: TODO Tag a release: TODO Remove graduated code from oslo-incubator: TODO Update oslo-incubator/update.py to not rewrite references to the library: TODO openstack/requirements - global-requirements.txt: TODO Document Migration Process: TODO openstack-dev/devstack - lib/oslo: TODO openstack-dev/devstack - stackrc: TODO Update project list on docs.openstack.org: TODO