Oslo/JunoGraduationPlans

= Oslo Juno Library Graduation Plan =

Over the course of the Icehouse release, the Oslo team has invested a lot of time in creating processes and tools to prepare for releasing code from the incubator as a set of new libraries. Besides the release process itself, we have also thought about ways to make consuming Oslo code easier for other projects. This is the plan we have put together, considering as many factors as we can anticipate, for creating 9 new libraries during the Juno release.

Graduation Order
We need to be careful about the order we extract each library, to avoid circular dependencies and further drift from libraries needing to have modules from the incubator. We also need to balance the complexity of the work for creating some of the planned libraries, and integrating them with consuming projects.

With those two requirements in mind, the proposed order for graduation is:


 * oslo.db - database access (models, sessions, and migrations)
 * oslo.i18n - internationalization tools
 * oslo.local - thread-local implementation
 * oslo.log - logging setup and context-aware formatter
 * oslo.text - generic text-processing tools
 * oslo.language - python utilities
 * oslo.serialization - JSON and XML utilities
 * oslo.utils - miscellaneous
 * oslo.concurrency - process and thread management tools

Although oslo.db depends on several of the other planned libraries, migrating to it will take more time than most of the other libraries, so we will be trying to release it early in the cycle. That means oslo.db will sync changes from the incubator for a while, but since we will be trying to minimize those changes we do not anticipate having a lot of trouble keeping it up to date. As the code that oslo.db depends on is moved from the incubator into libraries, oslo.db will be updated to use the new libraries.

The other libraries are listed roughly in order based on their position in the dependency graph. We may not finish the entire list in time to have them released during Juno, but we do not anticipate creating more libraries than this.

Minimizing the Pain
We have done a lot of planning to prepare, but this migration is not going to be without bumps. We will try to make it as painless as possible.

Cross-Project Unit Test Gates
The new oslotest library, created during Icehouse, gave us a chance to figure out how to run cross-project unit test gate jobs. As a result, changes in Oslo libraries can now be tested with the unit tests of other projects, to detect regressions before a library is released.

We will set up a voting gate job for each project/library combination before beginning the integration, so that after integration is complete the Oslo library will not change in a way that breaks the project and so the project will not change in a way that breaks its use of the Oslo library.

More Frequent Alpha Releases
New features added to Oslo libraries will not be available for developers immediately, because while we are running gate tests for the libraries using source, and devstack gate using source, the projects specify dependencies on released packages. That means "tox -e py27" will not find the latest version of the source from master, only the latest release. To reduce the time between a feature landing in a library and being useful in a project, we plan to release alphas of the libraries throughout the Juno cycle. The exact process for creating these releases is still being worked out.

Dependency Management
As each new library is integrated with a consuming project, any dependencies needed only by that library that were previously listed in the requirements for the main project will be removed from the requirements file for the project. For example, if a project needs sqlalchemy for openstack.common.db then, when the project is updated to use oslo.db, sqlalchemy will be removed from the project's requirements.txt and the dependency will be inherited from oslo.db. This will make managing our common list of dependencies easier, and prevent issues caused by conflicting requirements from libraries and projects.

API Changes During Graduation
Each new library meets the stability requirement for graduation, and we are going to try to minimize API changes, but some changes are unavoidable. For example, oslo.i18n will need to add a public API to gettextutils so multiple translation domains can be used in the same process (to support translations within Oslo libraries). As much as possible we will make these changes in a way to require the fewest changes from consuming projects, without requiring that each project have complex integration modules.

To avoid excessive syncing of incubator modules into projects, most of these API changes will be made after the library is moved outside of the incubator. That means a typical "adoption" patch for a new library will include removing the incubator versions of the module, possibly adding an integration module (to hold globals that were previously managed by the incubated copy of the code, or to pass project-specific options into the library), updating any callers of the module within the project, and updating the dependencies of the project.

Cross-Program Coordination
To make the graduation and adoption processes go more smoothly, we are planning some changes to the way Oslo interacts with the other Programs.

Library Leads
Similar to the way Nova has sub-section leads, and the Oslo incubator has a designated maintainer for each module, we will also be designating a lead for each new library. This will give consuming projects a primary contact for each library, to answer questions or provide assistance.

Project Liaisons
There are now more projects consuming code from the Oslo incubator than we have Oslo contributors. That means we are going to need your help to make these migrations happen. We are asking for one person from each project to serve as a liaison between the project and Oslo, and to assist with integrating changes as we move code out of the incubator into libraries.

The liaison should be a core committer to the project, but does not need to be the PTL. The liaison should be prepared to assist with writing and reviewing patches in their project as libraries are adopted, and with discussions of API changes to the libraries to make them easier to use within the project.

Adoption Order
We plan to work with the smaller integrated projects first, and then move to the larger projects like nova and cinder. Working on the smaller code bases will make it easier to work out some of the kinks in the process of migrating to each library, without large disruptive patches that need to be re-based constantly in the faster-moving projects. However, this is not a hard rule. If the liaison from a project wants to start migrating to a library as soon as it is released, we will work with them.

Incubator Code After Graduation
As discussed in Oslo, after a library's first release, the version of the code in the incubator will be treated as a "stable maintenance branch" for one full cycle. For example, that means that the RPC code now in oslo.messaging is a candidate for removal from the incubator at the end of Juno, and the new libraries created during Juno will likely be removed at the end of the K release. Stable branches in the incubator will still hold older copies of the code after it is removed from master.