Jump to: navigation, search

Difference between revisions of "Oslo"

(add program description)
(The Oslo Team)
(203 intermediate revisions by 26 users not shown)
Line 1: Line 1:
 +
[[Category:Commonlibraries]]
 +
 +
'''Revised on:''' {{REVISIONMONTH1}}/{{REVISIONDAY}}/{{REVISIONYEAR}} by {{REVISIONUSER}}
 +
 +
[[File:OpenStack_Project_Oslo_mascot.jpg|200px|thumbnail|right]]
 +
 
'''Official Title:''' OpenStack Common Libraries<br />
 
'''Official Title:''' OpenStack Common Libraries<br />
  
'''PTL:''' Mark McLoughlin <markmc@redhat.com><br />
+
'''PTL:''' Ben Nemec <openstack@nemebean.com><br />
  
'''Mission Statement:''' <blockquote>To produce a set of python libraries containing infrastructure code shared by OpenStack projects. The APIs provided by these libraries should be high quality, stable, consistent and generally applicable.</blockquote>
+
'''Mission Statement:''' <blockquote>To produce a set of python libraries containing code shared by OpenStack projects. The APIs provided by these libraries should be high quality, stable, consistent, documented and generally applicable.</blockquote>
  
 
== The Oslo Team ==
 
== The Oslo Team ==
Line 15: Line 21:
 
=== Specialist API Maintainers ===
 
=== Specialist API Maintainers ===
  
Each library or incubating API has one or more specialist maintainers who have responsibility for evolving the API in question. They work to ensure the API meets the needs of all OpenStack projects and helps to ensure the APIs are widely adopted across the project, wherever the APIs can reduce duplication of functionality.
+
Each API typically has one or more specialist maintainers who have responsibility for evolving the API in question. They work to ensure the API meets the needs of all OpenStack projects, and once the API is stable and useful. Each library has its own core team, which can include specialists in the area of the library who are not general Oslo cores.  Because the scope of the Oslo project has grown so large, these library-specific cores are **critical** to the long-term health of the projects and anyone with an interest in a library is encouraged to get involved.
 +
 
 +
=== Help Wanted ===
 +
 
 +
Over the years, contributors to Oslo projects have come and gone.  In some cases this has left us with expertise gaps on important projects.  New contributors to the following projects would be greatly appreciated:
 +
 
 +
* oslo.privsep
 +
* oslo.service
 +
* taskflow
  
 
=== Project Meetings ===
 
=== Project Meetings ===
Line 23: Line 37:
 
=== Getting in Touch ===
 
=== Getting in Touch ===
  
We use the [http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev openstack-dev@lists.openstack.org] mailing list for discussions and we all hang out on #openstack-dev.
+
We use the [http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev openstack-dev@lists.openstack.org] mailing list for discussions and we all hang out in #openstack-oslo and #openstack-dev on freenode.
 +
 
 +
Each project also designates a liaison for handling integration issues. See [https://wiki.openstack.org/wiki/CrossProjectLiaisons#Oslo Oslo/ProjectLiaisons].
 +
 
 +
== Backwards Compatibility ==
 +
 
 +
As of the Newton summit, where a vote of 10 Oslo cores was 7 in favour, 0 against, 3 abstained, Oslo projects maintain backwards compatibility for one release cycle: a maximum period of 12 months (start of one release cycle to the end of the next). We do this so that important improvements in master that aren't suitable for backporting to a stable branch are still able to be used by deployers. For instance, we fixed heartbeat in oslo.messaging in this manner, and have an upcoming kafka support change that requires upgrading support libraries we use across incompatible versions: our users won't suffer, but folk using python-kafka directly would, making it unsuitable for stable backports.
 +
 
 +
To test this, we will be implementing both stable jobs on master oslo changes (run stable devstack on master oslo) and unstable jobs on stable server changes (run the server with latest-oslo-release).
 +
 
 +
We are not specifically aiming to support partial mixed upgrades (e.g. Nova Mitaka + Neutron Newton) in a single Python environment: there are no obvious hurdles to that from the Oslo side, but client library Python API compatibility is not governed by Oslo policies.
 +
 
 +
== Jobs ==
 +
 
 +
'''History of (these) jobs:''' https://etherpad.openstack.org/p/dims-periodic-jobs
 +
 
 +
=== Periodic ===
 +
 
 +
[http://status.openstack.org//openstack-health/#/job/periodic-tempest-dsvm-oslo-latest-full-master Oslo (dvsm) latest against master]
 +
 
 +
[http://status.openstack.org/openstack-health/#/?groupKey=build_name&resolutionKey=hour&searchProject=-with-oslo Oslo latest against (many projects) master]
 +
 
 +
'''Other periodic jobs:''' http://logs.openstack.org/periodic/
  
 
== Libraries ==
 
== Libraries ==
  
The following libraries are currently published by the Oslo program. Where we felt that a library had real potential for widespread use outside OpenStack, we chose not to include them in the oslo namespace.
+
The following libraries are currently published by the Oslo program. Where we felt that a library had real potential for widespread use outside OpenStack, we chose not to include them in the oslo namespace (aka prefix them with oslo).
 +
 
 +
New libraries need to be careful to avoid introducing circular dependencies. See [[Oslo/Dependencies]].
 +
 
 +
Specialized libraries have their own core-review team with members who may not be part of the main Oslo core team. Unless otherwise indicated below, an Oslo library is maintained by [https://review.openstack.org/#/admin/groups/106 oslo-core].
 +
 
 +
=== automaton ===
 +
 
 +
'''Summary''': a framework for building state machines.
 +
 
 +
'''Proposal''': [https://review.openstack.org/#/c/141961/ 141961]
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack/automaton
 +
 
 +
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/automaton automaton project in launchpad].
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/106,members oslo-core]
 +
 
 +
'''Documentation''': http://docs.openstack.org/developer/automaton/
 +
 
 +
=== castellan ===
 +
 
 +
'''Summary''':  generic key manager interface for OpenStack.
 +
 
 +
'''Proposal''': [https://blueprints.launchpad.net/oslo.utils/+spec/adopt-castellan adopt-castellan]
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack/castellan
 +
 
 +
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/castellan castellan project in launchpad].
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/1742,members,members castellan-core]
 +
 
 +
'''Documentation''': http://docs.openstack.org/developer/castellan/
 +
 
 +
=== cookiecutter ===
 +
 
 +
'''Summary''': a project that creates a skeleton OpenStack project from a set of templates.
 +
 
 +
'''Proposal''': [https://review.openstack.org/#/c/42530/ 42530]
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack-dev/cookiecutter
 +
 
 +
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/oslo oslo project in launchpad].
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/106,members oslo-core]
 +
 
 +
'''Documentation''': n/a
 +
 
 +
=== debtcollector ===
 +
 
 +
'''Summary''': a collection of python patterns that help you collect your technical debt in a non-destructive manner (following deprecation patterns and strategies and so-on).
 +
 
 +
'''Proposal''': [https://review.openstack.org/#/c/141220/ 141220]
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack/debtcollector
 +
 
 +
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/debtcollector debtcollector project in launchpad].
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/106,members oslo-core]
 +
 
 +
'''Documentation''': http://docs.openstack.org/developer/debtcollector/
 +
 
 +
=== futurist ===
 +
 
 +
'''Summary''': a collection of async functionality and additions from the future.
 +
 
 +
'''Proposal''': [https://review.openstack.org/#/c/179890/ 179890]
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack/futurist
 +
 
 +
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/futurist futurist project in launchpad].
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/106,members oslo-core]
 +
 
 +
'''Documentation''': http://docs.openstack.org/developer/futurist/
 +
 
 +
=== mox3 ===
 +
 
 +
'''Summary''': an unofficial port  of mox framework  to Python 3
 +
 
 +
'''Proposal''': n/a
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack/mox3
 +
 
 +
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/python-mox3 mox3 project in launchpad].
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/106,members oslo-core]
 +
 
 +
'''Documentation''': n/a
 +
 
 +
=== oslo-cookiecutter ===
 +
 
 +
'''Summary''': is a project that creates a skeleton Oslo library from a set of templates.
 +
 
 +
'''Proposal''': n/a
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack-dev/oslo-cookiecutter
 +
 
 +
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/oslo-cookiecutter oslo-cookiecutter project in launchpad].
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/106,members oslo-core]
 +
 
 +
'''Documentation''': ??
 +
 
 +
=== oslo.cache ===
 +
 
 +
'''Summary''': a library for caching based on [https://pypi.python.org/pypi/dogpile.cache dogpile].
 +
 
 +
'''Proposal''': n/a
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack/oslo.cache
 +
 
 +
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/oslo.cache oslo.cache project in launchpad].
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/106,members oslo-core]
 +
 
 +
'''Documentation''': http://docs.openstack.org/developer/oslo.cache/
 +
 
 +
=== oslo.concurrency ===
 +
 
 +
'''Summary''': a project with helpers managing external processes and task synchronization.
 +
 
 +
'''Proposal''': n/a
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack/oslo.concurrency
 +
 
 +
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/oslo.concurrency oslo.concurrency project in launchpad].
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/106,members oslo-core]
 +
 
 +
'''Documentation''': http://docs.openstack.org/developer/oslo.concurrency/
  
 
=== oslo.config ===
 
=== oslo.config ===
  
[http://pypi.python.org/pypi/oslo.config oslo.config] is a library for parsing configuration files and command line arguments. It is maintained by Mark McLoughlin.
+
'''Summary''': a ''extensive'' library for parsing configuration files and command line arguments.
 +
 
 +
'''Proposal''': see [https://wiki.openstack.org/wiki/Oslo/Config this historical blueprint] describing the initial requirements for the API.
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack/oslo.config
 +
 
 +
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/oslo.config oslo.config project in launchpad].
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/106,members oslo-core]
 +
 
 +
'''Documentation''': http://docs.openstack.org/developer/oslo.config/
 +
 
 +
=== oslo.context ===
 +
 
 +
'''Summary''': a project with helpers to maintain useful information about a request context (aka associated information/data bound to a request).
 +
 
 +
'''Proposal''': see [https://wiki.openstack.org/wiki/Oslo/Context this historical blueprint] describing the initial requirements for the API.
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack/oslo.context
 +
 
 +
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/oslo.context oslo.context project in launchpad].
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/106,members oslo-core]
 +
 
 +
'''Documentation''': http://docs.openstack.org/developer/oslo.context/
 +
 
 +
=== oslo.db ===
 +
 
 +
'''Summary''': a ''extensive'' library to aid in database interactions and/or handling.
 +
 
 +
'''Proposal''': n/a
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack/oslo.db
 +
 
 +
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/oslo.db oslo.db project in launchpad].
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/331,members oslo-db-core]
 +
 
 +
'''Documentation''': http://docs.openstack.org/developer/oslo.db/
 +
 
 +
=== oslo.i18n ===
 +
 
 +
'''Summary''': a wrapper library around Python's gettext module for string translation and other internationalization features.
 +
 
 +
'''Proposal''': n/a
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack/oslo.i18n
 +
 
 +
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/oslo.i18n oslo.i18n project in launchpad]
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/106,members oslo-core]
 +
 
 +
'''Documentation''': http://docs.openstack.org/developer/oslo.i18n/
 +
 
 +
=== oslo.log ===
 +
 
 +
'''Summary''': a logging configuration library.
 +
 
 +
'''Proposal''': n/a
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack/oslo.logging
 +
 
 +
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/oslo.log oslo.log project in launchpad].
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/106,members oslo-core]
 +
 
 +
'''Documentation''': http://docs.openstack.org/developer/oslo.log/
 +
 
 +
=== oslo.messaging ===
 +
 
 +
'''Summary''': a library that provides a messaging API which supports RPC and notifications over a number of different messaging transports.
 +
 
 +
'''Proposal''': [https://etherpad.openstack.org/HavanaOsloMessaging this etherpad] captures the latest status and background to this project.
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack/oslo.messaging
 +
 
 +
'''Devstack plugin''':  [http://git.openstack.org/cgit/openstack/devstack-plugin-amqp1 devstack-plugin-amqp1] [http://git.openstack.org/cgit/openstack/devstack-plugin-kafka devstack-plugin-kafka]  [http://git.openstack.org/cgit/openstack/devstack-plugin-pika devstack-plugin-pika]  [http://git.openstack.org/cgit/openstack/devstack-plugin-zmq devstack-plugin-zmq]
 +
 
 +
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/oslo.messaging oslo.messaging project in launchpad].
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/318 oslo-messaging-core]
 +
 
 +
'''Documentation''': http://docs.openstack.org/developer/oslo.messaging/
 +
 
 +
=== oslo.middleware ===
 +
 
 +
'''Summary''': provides a collection of WSGI middleware for web service development.
 +
 
 +
'''Proposal''': n/a
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack/oslo.middleware
 +
 
 +
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/oslo.middleware oslo.middleware project in launchpad].
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/106,members oslo-core]
 +
 
 +
'''Documentation''': http://docs.openstack.org/developer/oslo.middleware/
 +
 
 +
=== oslo.policy ===
 +
 
 +
'''Summary''': provides a rules engine for enforcing policy.
 +
 
 +
'''Proposal''': n/a
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack/oslo.policy
 +
 
 +
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/oslo.policy oslo.policy project in launchpad].
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/556,members oslo-policy-core]
 +
 
 +
'''Documentation''': http://docs.openstack.org/developer/oslo.policy/
 +
 
 +
=== oslo.privsep ===
 +
 
 +
'''Summary''': provides a mechanism for running selected python code with elevated privileges
 +
 
 +
'''Proposal''': [https://review.openstack.org/#/c/204073/ 204073]
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack/oslo.privsep
 +
 
 +
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/oslo.privsep oslo.privsep project in launchpad]
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/1135 oslo-privsep-core]
 +
 
 +
'''Documentation''': http://docs.openstack.org/developer/oslo.privsep/
 +
 
 +
=== oslo.reports ===
 +
 
 +
'''Summary''': allows projects to generate Guru Meditation Reports for debugging the current state of OpenStack processes. It can also be used to generating general reports on the fly that are serializable as plain text, JSON, or XML.
 +
 
 +
'''Proposal''': n/a
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack/oslo.reports
 +
 
 +
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/oslo.reports oslo.reports project in launchpad].
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/106,members oslo-core]
 +
 
 +
'''Documentation''': http://docs.openstack.org/developer/oslo.reports/
 +
 
 +
=== oslo.rootwrap ===
 +
 
 +
'''Deprecated (with preference to oslo.privsep)'''
 +
 
 +
'''Summary''': rootwrap allows fine filtering of shell commands to run as root from OpenStack services.
 +
 
 +
'''Proposal''': n/a
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack/oslo.rootwrap
 +
 
 +
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/oslo.rootwrap oslo.rootwrap project in launchpad]
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/293 oslo-rootwrap-core]
 +
 
 +
'''Documentation''': http://docs.openstack.org/developer/oslo.rootwrap/
 +
 
 +
=== oslo.serialization ===
 +
 
 +
'''Summary''': is a library that provides serialization functionality with special handling for some common types used in OpenStack.
 +
 
 +
'''Proposal''': n/a
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack/oslo.serialization
 +
 
 +
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/oslo.serialization oslo.serialization project in launchpad].
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/106,members oslo-core]
 +
 
 +
'''Documentation''': http://docs.openstack.org/developer/oslo.serialization/
 +
 
 +
=== oslo.service ===
 +
 
 +
'''Summary''': is a helper library that provides functionality for running OpenStack services.
 +
 
 +
'''Proposal''': n/a
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack/oslo.service
 +
 
 +
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/oslo.service oslo.service project in launchpad].
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/106,members oslo-core]
 +
 
 +
'''Documentation''': http://docs.openstack.org/developer/oslo.service/
 +
 
 +
=== oslo.tools ===
 +
 
 +
'''Summary''': Tools for working in the openstack oslo community
 +
 
 +
'''Proposal''': n/a
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack/oslo.tools
 +
 
 +
'''Bugs''': n/a
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/106,members oslo-core]
 +
 
 +
'''Documentation''': n/a
 +
 
 +
=== oslo.utils ===
 +
 
 +
'''Summary''': is a helper library that provides various low-level utility modules/code (that doesn't have a home anywhere else).
 +
 
 +
'''Proposal''': n/a
  
Please file bugs in the [https://bugs.launchpad.net/oslo oslo project in launchpad].
+
'''Source''': http://git.openstack.org/cgit/openstack/oslo.utils
  
See [https://wiki.openstack.org/wiki/Oslo/Config this historical blueprint] describing the initial requirements for the API.
+
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/oslo.utils oslo.utils project in launchpad].
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/106,members oslo-core]
 +
 
 +
'''Documentation''': http://docs.openstack.org/developer/oslo.utils/
 +
 
 +
=== oslo.versionedobjects ===
 +
 
 +
'''Summary''': is a library that helps deal with DB schema being at different versions than the code expects, allowing services to be operated safely during upgrades (among other things).
 +
 
 +
'''Proposal''': [https://review.openstack.org/#/c/127532/ 127532]
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack/oslo.versionedobjects
 +
 
 +
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/oslo.versionedobjects oslo.versionedobjects project in launchpad].
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/106,members oslo-core]
 +
 
 +
'''Documentation''': http://docs.openstack.org/developer/oslo.versionedobjects/
 +
 
 +
=== oslo.version ===
 +
 
 +
'''!!Retired!!'''
 +
 
 +
'''Summary''': is a helper library that provides for getting the version for an installed piece of software from the python metadata that already exists.
 +
 
 +
'''Proposal''': [https://review.openstack.org/#/c/40498/ 40498]
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack/oslo.version
 +
 
 +
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/oslo oslo project in launchpad].
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/106,members oslo-core]
 +
 
 +
'''Documentation''': ??
 +
 
 +
=== oslo.vmware ===
 +
 
 +
'''Summary''': provides for a shared location for code common to the VMware drivers in several projects.
 +
 
 +
'''Proposal''': n/a
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack/oslo.vmware
 +
 
 +
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/oslo.vmware oslo.vmware project in launchpad].
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/271,members  oslo-vmware-core]
 +
 
 +
'''Documentation''': http://docs.openstack.org/developer/oslo.vmware/
 +
 
 +
=== oslosphinx ===
 +
 
 +
'''Summary''': is a sphinx add-on library that provides theme and extension support for Sphinx documentation from the OpenStack project.
 +
 
 +
'''Proposal''': n/a
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack/oslosphinx
 +
 
 +
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/oslosphinx oslosphinx project in launchpad].
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/106,members oslo-core]
 +
 
 +
'''Documentation''': http://docs.openstack.org/developer/oslosphinx/
 +
 
 +
=== oslotest ===
 +
 
 +
'''Summary''': is a helper library that provides base classes and fixtures for creating unit and functional tests.
 +
 
 +
'''Proposal''': n/a
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack/oslotest
 +
 
 +
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/oslotest oslotest project in launchpad].
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/106,members oslo-core]
 +
 
 +
'''Documentation''': http://docs.openstack.org/developer/oslotest/
 +
 
 +
=== osprofiler ===
 +
 
 +
'''Summary''': an OpenStack cross-project profiling library.
 +
 
 +
'''Proposal''': [https://review.openstack.org/#/c/103825/ 103825]
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack/osprofiler
 +
 
 +
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/osprofiler osprofiler project in launchpad].
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/1222,members osprofiler-core]
 +
 
 +
'''Documentation''': http://docs.openstack.org/developer/osprofiler
  
 
=== pbr ===
 
=== pbr ===
  
[https://pypi.python.org/pypi/pbr pbr] (or Python Build Reasonableness) is a set of sensible default setuptools behaviours. It is maintained by Monty Taylor.
+
'''Summary''': pbr (or Python Build Reasonableness) is a add-on library that helps provide (and enforce) a set of sensible default setuptools behaviours.
  
Please file bugs in the [https://bugs.launchpad.net/pbr pbr project in launchpad].
+
'''Proposal''': n/a
  
=== hacking ===
+
'''Source''': http://git.openstack.org/cgit/openstack-dev/pbr
  
[https://pypi.python.org/pypi/hacking hacking] is a set of tools for enforcing coding style guidelines. It is maintained by Joe Gordon and Sean Dague.
+
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/pbr pbr project in launchpad].
  
Please file bugs in the [https://bugs.launchpad.net/hacking hacking project in launchpad].
+
'''Core review team''': [https://review.openstack.org/#/admin/groups/154 pbr-core]
  
=== oslo.messaging ===
+
'''Documentation''': http://docs.openstack.org/developer/pbr/
 +
 
 +
=== pylockfile ===
 +
 
 +
'''Deprecated'''
 +
 
 +
'''Summary''': legacy (and adopted) inter-process lock management library.
 +
 
 +
'''Proposal''':[https://review.openstack.org/#/c/102202/ 102202]
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack/pylockfile
 +
 
 +
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/pylockfile pylockfile project in launchpad].
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/106,members oslo-core]
 +
 
 +
'''Documentation''': http://docs.openstack.org/developer/pylockfile/
 +
 
 +
=== stevedore ===
 +
 
 +
'''Summary''': a library for managing plugins for Python applications.
 +
 
 +
'''Proposal''': n/a
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack/stevedore
 +
 
 +
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/python-stevedore python-stevedore project in launchpad].
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/247 stevedore-core]
 +
 
 +
'''Documentation''': http://docs.openstack.org/developer/stevedore/
 +
 
 +
=== taskflow ===
 +
 
 +
'''Summary''': a library that helps create applications that handle state/failures... in a reasonable manner.
 +
 
 +
'''Proposal''': n/a
 +
 
 +
'''Source''': http://git.openstack.org/cgit/openstack/taskflow
 +
 
 +
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/taskflow taskflow project in launchpad].
 +
 
 +
'''Core review team''': [https://review.openstack.org/#/admin/groups/173 taskflow-core]
 +
 
 +
'''Documentation''': http://docs.openstack.org/developer/taskflow/
 +
 
 +
=== tooz ===
 +
 
 +
'''Summary''': a library that aims at centralizing the most common distributed primitives like group membership protocol, lock service and leader election by providing a coordination API helping developers to build distributed applications.
  
The [https://github.com/openstack/oslo.messaging oslo.messaging] will provide a messaging API supports RPC and notifications over a number of different messaging transports. It is not yet ready for use. It is maintained by Mark McLoughlin.
+
'''Proposal''': [https://review.openstack.org/#/c/122439/ 122439]
  
[https://etherpad.openstack.org/HavanaOsloMessaging This etherpad] captures the latest status and background to this project.
+
'''Source''': http://git.openstack.org/cgit/openstack/tooz
  
=== oslo.sphinx===
+
'''Bugs''': please file bugs in the [https://bugs.launchpad.net/python-tooz python-tooz project in launchpad].
  
[https://github.com/openstack/oslo.sphinx oslo.sphinx] provides theme and extension support for Sphinx documentation from the OpenStack project. It is maintained by Doug Hellmann.
+
'''Core review team''': [https://review.openstack.org/#/admin/groups/246,members tooz-core]
  
Please file bugs in the [https://bugs.launchpad.net/oslo oslo project in launchpad].
+
'''Documentation''': http://docs.openstack.org/developer/tooz/
  
 
== Principles ==
 
== Principles ==
Line 76: Line 582:
 
== Incubation ==
 
== Incubation ==
  
The process of developing a new Oslo API usually begins by taking code which is common to some OpenStack projects and moving it into the [https://github.com/openstack/oslo-incubator oslo-incubator] repository. New APIs live in the incubator until they have matured to meet the criteria described above.
+
'''The Oslo Incubator is now dead. Refer to the [http://lists.openstack.org/pipermail/openstack-dev/2015-November/thread.html#79343 announcement] and the [https://review.openstack.org/#/c/245461/ change clearing out the repository].'''
 +
 
 +
== FAQs ==
 +
 
 +
=== Why aren't alpha releases of oslo.config published to PyPI? ===
 +
 
 +
See [http://specs.openstack.org/openstack/oslo-specs/specs/policy/versioning.html Choosing Version Numbers] for the current policies related to versioning and releases.
 +
 
 +
=== Why does oslo.config have a CONF object? Global object SUCK! ===
 +
 
 +
Indeed. Well, it's a long story and well documented in mailing list archives if anyone cares to dig up some links.
  
While incubating, all APIs should have a specialist API maintainer. The responsibility of these maintainers and the list of maintainers for each incubating API is documented in the [https://github.com/openstack/oslo-incubator/blob/master/MAINTAINERS MAINTAINERS file in oslo-incubator].
+
Around the time of the Folsom Design Summit, an attempt was made to remove our dependence on a global object like this. There was massive debate and, in the end, the rough consensus was to stick with using this approach.
  
Developers making major changes to incubating APIs in oslo-incubator must be prepared to update the copies in the projects which have previously imported the code.
+
Nova, through its use of the gflags library, used this approach from [https://github.com/openstack/nova/blob/bf6e6e7/nova/flags.py#L27 commit zero]. Some OpenStack projects didn't initially use this approach, but most now do. The idea is that having all projects use the same approach is more important than the objections to the approach. Sharing code between projects is great, but by also having projects use the same idioms for stuff like this it makes it much easier for people to work on multiple projects.
  
Incubation shouldn't be seen as a long term option for any API - it is merely a stepping stone to inclusion into a published Oslo library.
+
This debate will probably never completely go away, though. See [http://lists.openstack.org/pipermail/openstack-dev/2014-August/044050.html this latest discussion in August, 2014].
  
Please file bugs for incubating APIs in the [https://bugs.launchpad.net/oslo oslo project in launchpad].
+
=== Why does Oslo observe feature freeze ===
  
=== Syncing Code from Incubator ===
+
Feature freeze is a time to stabilize all of the new features that were added during a development cycle, but since Oslo projects don't necessarily release on the same six month schedule as the other OpenStack projects (or at all in the case of oslo-incubator) it might seem odd that Oslo observes feature freeze.
  
APIs which are incubating can be copied into individual openstack projects from oslo-incubator using the <code><nowiki>update.py</nowiki></code> script provided. An <code><nowiki>openstack-common.conf</nowiki></code> configuration file in the project describes which modules to copy and where they should be copied to.
+
For the graduated libraries this serves the same purpose as for any of the other projects - it's a time for focusing on bug fixes and stability.
  
Usually the API maintainer or those making significant changes to an API take responsibility for syncing that specific module into the projects which use it by doing e.g.:
+
For oslo-incubator, the primary motivation is making last-minute fixes needed by other projects easier to sync.  If a new feature lands in oslo-incubator and an unrelated bug is discovered by one of the consuming projects, it becomes a problem to sync just the bug fix to the project.  When 11th hour bug fixes are needed it's best if the sync is as simple and small as possible. To avoid problems, oslo-incubator respects the feature freeze period just like any other project.
  
<pre><nowiki>
+
=== How does Oslo manage versions? ===
$> cd ../
 
$> git clone .../oslo-incubator
 
$> cd oslo-incubator
 
$> python update.py --nodeps --base nova --dest-dir ../nova --modules jsonutils,gettextutils
 
</nowiki></pre>
 
  
Alternatively, it can make sense for someone to batch sync more minor changes into a project by doing e.g.:
+
See [[Oslo/VersioningPolicy]].
To sync all code for a specific project, you can do:
 
  
<pre><nowiki>
+
== Resources ==
$> python update.py ../nova
 
Copying the config,exception,extensions,utils,wsgi modules under the nova module in ../nova
 
</nowiki></pre>
 
  
in this latter case, the <code><nowiki>update.py</nowiki></code> script uses the <code><nowiki>openstack-common.conf</nowiki></code> config file to determine which modules to copy. The format of that file is e.g.
+
=== Review policies ===
  
<pre><nowiki>
+
These overlay the regular review rules for OpenStack as a whole.
$> cd ../nova
 
$> cat openstack-common.conf
 
[DEFAULT]
 
  
# The list of modules to copy from oslo-incubator
+
* Automated changes - patches from 'openstack proposal bot' and 'transifex' which have passed CI checks can be +2+A by a single core reviewer
modules=cfg,iniparser
 
  
# The base module to hold the copy of openstack.common
+
=== Review Links ===
base=nova
 
</nowiki></pre>
 
  
Projects which are using such incubating APIs must avoid ever modifying their copies of the code. All changes should be made in oslo-incubator itself and copied into the project.
+
(See http://git.openstack.org/cgit/openstack/oslo.tools/tree/dashboards for the source files to create these links)
  
== FAQs ==
+
* [https://review.openstack.org/#/dashboard/?foreach=%28project%3A%5Eopenstack%2Foslo.%2A+OR+project%3Aopenstack%2Fdebtcollector+OR%0Aproject%3Aopenstack%2Fpylockfile+OR+project%3Aopenstack%2Fcastellan+OR%0Aproject%3Aopenstack%2Ffuturist+OR+project%3Aopenstack%2Fautomaton+OR%0Aproject%3Aopenstack%2Fstevedore+OR+project%3Aopenstack%2Ftaskflow+OR%0Aproject%3Aopenstack%2Ftooz+OR+project%3Aopenstack%2Ddev%2Fcookiecutter+OR%0Aproject%3Aopenstack%2Ddev%2Fpbr+OR+project%3Aopenstack%2Fdebtcollector+OR%0Aproject%3Aopenstack%2Ddev%2Foslo%2Dcookiecutter+OR+project%3Aopenstack%2Fmox3%29%0Astatus%3Aopen+NOT+owner%3Aself+NOT+label%3AWorkflow%3C%3D%2D1+label%3AVerified%3E%3D1%0ANOT+label%3ACode%2DReview%3C%3D%2D1%2Cself+NOT+label%3ACode%2DReview%3E%3D1%2Cself&title=Oslo+Review+Inbox&Oslo+Specs=project%3Aopenstack%2Foslo%2Dspecs&Bug+Fixes=topic%3A%5Ebug%2F.%2A&Blueprints=message%3A%22Blueprint%22&Needs+Feedback+%28Changes+older+than+5+days+that+have+not+been+reviewed+by+anyone%29=NOT+label%3ACode%2DReview%3C%3D2+age%3A5d&You+are+a+reviewer%2C+but+haven%27t+voted+in+the+current+revision=reviewer%3Aself&Needs+final+%2B2=label%3ACode%2DReview%3E%3D2+limit%3A50&New+Contributors=reviewer%3A10068&Passed+Jenkins%2C+No+Negative+Feedback=NOT+label%3ACode%2DReview%3E%3D2+NOT+label%3ACode%2DReview%3C%3D%2D1+limit%3A50&Wayward+Changes+%28Changes+with+no+code+review+in+the+last+2days%29=NOT+label%3ACode%2DReview%3C%3D2+age%3A2d Oslo General Review Dashboard]
 +
* [https://review.openstack.org/#/dashboard/?foreach=status%3Aopen+NOT+owner%3Aself&title=Oslo+Review+Inbox%28Part+One%29&Oslo+Specs=project%3Aopenstack%2Foslo%2Dspecs&automaton=project%3Aopenstack%2Fautomaton&castellan=project%3Aopenstack%2Fcastellan&cookiecutter=project%3Aopenstack%2Ddev%2Fcookiecutter&debtcollector=project%3Aopenstack%2Fdebtcollector&futurist=project%3Aopenstack%2Ffuturist&mox3=project%3Aopenstack%2Fmox3&oslo%2Dcookiecutter=project%3Aopenstack%2Ddev%2Foslo%2Dcookiecutter&oslo.cache=project%3Aopenstack%2Foslo.cache  Part One Review Dashboard]
 +
* [https://review.openstack.org/#/dashboard/?foreach=status%3Aopen+NOT+owner%3Aself&title=Oslo+Review+Inbox%28Part+Two%29&oslo.privsep=project%3Aopenstack%2Foslo.privsep&oslo.reports=project%3Aopenstack%2Foslo.reports&oslo.rootwrap=project%3Aopenstack%2Foslo.rootwrap&oslo.serialization=project%3Aopenstack%2Foslo.serialization&oslo.service=project%3Aopenstack%2Foslo.service&oslo.tools=project%3Aopenstack%2Foslo.tools&oslo.utils=project%3Aopenstack%2Foslo.utils&oslo.versionedobjects=project%3Aopenstack%2Foslo.versionedobjects&oslo.vmware=project%3Aopenstack%2Foslo.vmware  Part Two Review Dashboard]
 +
* [https://review.openstack.org/#/dashboard/?foreach=status%3Aopen+NOT+owner%3Aself&title=Oslo+Review+Inbox%28Part+Three%29&oslo.concurrency=project%3Aopenstack%2Foslo.concurrency&oslo.config=project%3Aopenstack%2Foslo.config&oslo.context=project%3Aopenstack%2Foslo.context&oslo.db=project%3Aopenstack%2Foslo.db&oslo.i18n=project%3Aopenstack%2Foslo.i18n&oslo.log=project%3Aopenstack%2Foslo.log&oslo.messaging=project%3Aopenstack%2Foslo.messaging&oslo.middleware=project%3Aopenstack%2Foslo.middleware&oslo.policy=project%3Aopenstack%2Foslo.policy Part Three Review Dashboard]
 +
* [https://review.openstack.org/#/dashboard/?foreach=status%3Aopen+NOT+owner%3Aself&title=Oslo+Review+Inbox%28Part+Four%29&oslosphinx=project%3Aopenstack%2Foslosphinx&oslotest=project%3Aopenstack%2Foslotest&osprofiler=project%3Aopenstack%2Fosprofiler&pbr=project%3Aopenstack%2Ddev%2Fpbr&pylockfile=project%3Aopenstack%2Fpylockfile&stevedore=project%3Aopenstack%2Fstevedore&taskflow=project%3Aopenstack%2Ftaskflow&tooz=project%3Aopenstack%2Ftooz  Part Four Review Dashboard]
 +
* [https://bugs.launchpad.net/oslo/+bugs?field.searchtext=&orderby=-importance&search=Search&field.status%3Alist=INPROGRESS&assignee_option=any&field.assignee=&field.bug_reporter=&field.bug_commenter=&field.subscriber=&field.structural_subscriber=&field.tag=&field.tags_combinator=ANY&field.has_cve.used=&field.omit_dupes.used=&field.omit_dupes=on&field.affects_me.used=&field.has_patch.used=&field.has_branches.used=&field.has_branches=on&field.has_no_branches.used=&field.has_no_branches=on&field.has_blueprints.used=&field.has_blueprints=on&field.has_no_blueprints.used=&field.has_no_blueprints=on In Progress Bugs]
 +
 
 +
=== Security Team ===
 +
 
 +
In addition to OpenStack's Vulnerability Management team, some members of the Oslo team have indicated their willingness to help with security related issues in Oslo code. See [[Oslo/Security]] for the current list.
 +
 
 +
=== Design Proposals ===
 +
 
 +
We use the [http://git.openstack.org/cgit/openstack/oslo-specs oslo-specs repo] to track design proposals across all Oslo projects.
 +
 
 +
See [http://specs.openstack.org/openstack/oslo-specs/specs/policy/spec-approval.html the Spec Approval policy] for details.
  
=== Why aren't alpha releases of oslo.config published to PyPI? ===
+
The [https://blueprints.launchpad.net/oslo blueprints on launchpad] detail the changes currently underway to implement these specs.
  
oslo.config is considered part of the OpenStack coordinated release and follows the same release cadence.
+
[http://specs.openstack.org/openstack/oslo-specs/ Approved Specs] are published separately.
  
The thinking behind this is that any major development that happens in oslo.config is done in support of the projects in OpenStack's coordinated release. Similar benefits accrue from having nova and oslo.config on the same release schedule as say nova and glance. For example, we front-load the major, risky development towards the start of the release cycle and towards the end of the cycle we restrict ourselves to bugfixes. This reduces the risk of major regressions blocking us from doing a release. That's not to say that trunk should ever intentionally be broken. Nor should nova trunk ever be broken. We are committed to supporting folks who wish to deploy from trunk but still recognize that (for the forseeable future, at least) releases are less risky to consume than trunk.
+
=== Release Instructions ===
  
The point is simple - oslo.config is part of the OpenStack release-based development process.
+
[[Oslo/ReleaseProcess]]
  
Users of OpenStack releases may just be using 'pip install' to install OpenStack dependencies. If we published oslo.config development releases to PyPI, we'd find ourselves inadvertently breaking working configurations for some users and having to scramble to release fixes for those issues. During our heavy development phase, we really want to avoid the pressure of knowing that any release we make may immediately break working installations of the previous OpenStack release.
+
== Summits ==
  
You might think that with the level of test coverage we have and our commitment to API stability, the risk of breaking working setups should be minimal. Our experience with oslo.config in Havana teaches us otherwise. At the beginning of the Havana development cycle we expected to be making minimal changes to oslo.config but actually made pretty massive changes to fix some broken semantics. This, in turn, caused issues for Quantum like [https://bugs.launchpad.net/bugs/1196084 this] and [https://review.openstack.org/30250 this]. In one case, we subtly changed a public oslo.config API we really didn't consider public and in another case Quantum was using a clearly internal oslo.config API. One of the issues was caught by Quantum unit tests, the other issue wasn't.
+
===Pike===
  
Ok, so why not restrict the versions of oslo.config used by releases? e.g. why didn't Grizzly restrict itself to '>=1.1.0,<1.2'? It's quite simple - we need to be able to run a mixture of projects from different releases in the same environment. You may choose to upgrade Glance before Nova, for example. If Havana Glance only works with 1.2 and Grizzly Nova only works with 1.1, that doesn't work. This is also the reason why Oslo libraries need to make such a strong commitment to API stability.
+
*[https://etherpad.openstack.org/p/oslo-ptg-pike Pike PTG etherpads]
  
So, our requirement is that during the development cycle, the development versions of OpenStack projects should be able to use the development versions of Oslo libraries. But that users of existing stable releases should not be exposed to development versions of Oslo libraries.
+
===Ocata===
  
The best solution we've come up with to date is to publish oslo.config pre-releases (aka alphas) using the X.Y.ZaN numbering scheme to tarballs.openstack.org and reference them in requirements.txt using:
+
*[https://etherpad.openstack.org/p/ocata-oslo-summit-planning Ocata Summit Planning etherpads]
  
<pre><nowiki>
+
=== Newton ===
-f http://tarballs.openstack.org/oslo.config/oslo.config-1.2.0a3.tar.gz#egg=oslo.config-1.2.0a3
 
oslo.config>=1.2.0a3
 
</nowiki></pre>
 
  
The rationale for using this exact method is [https://review.openstack.org/35296 explained in this review].
+
* [https://etherpad.openstack.org/p/newton-oslo-summit-planning  Newton Summit Planning etherpads]
  
We're still looking for better solutions.
+
=== Mitaka ===
  
We've discussed only pushing alpha releases to OpenStack's pypi and including a --extra-index-url in our requirements.txt pointing to this mirror during the development cycle. While we're only talking about a very small number of libraries, that really isn't much different to the above though.
+
* [https://wiki.openstack.org/wiki/Design_Summit/Mitaka/Etherpads#Oslo Mitaka Summit Planning etherpads]
  
The [https://github.com/pypa/pip/commit/4d5c5f8 new --pre feature in pip 1.4] looked like a very promising solution. However, we realized that there will be people using older versions of pip for a long time to come. If we push pre-releases to PyPI, pip 1.4 users wouldn't be exposed to them by default but users of older pip would.
+
=== Liberty ===
  
We are currently scheming about the possibility of only publishing pre-releases in wheel format, which would mean they could only be consumed by pip 1.4 users. This solution does indeed look promising, but we're still thinking through the implications.
+
* [https://etherpad.openstack.org/p/liberty-oslo-summit-planning Liberty Summit Planning etherpad]
  
=== Why does oslo.config have a CONF object? Global object SUCK! ===
+
=== Kilo ===
  
Indeed. Well, it's a long story and well documented in mailing list archives if anyone cares to dig up some links.
+
* [[Summit/Kilo/Etherpads#Oslo]]
  
Around the time of the Folsom Design Summit, an attempt was made to remove our dependence on a global object like this. There was massive debate and, in the end, the rough consensus was to stick with using this approach.
+
=== Juno ===
  
Nova, through its use of the gflags library, used this approach from [https://github.com/openstack/nova/blob/bf6e6e7/nova/flags.py#L27 commit zero]. Some OpenStack projects didn't initially use this approach, but most now do. The idea is that having all projects use the same approach is more important than the objections to the approach. Sharing code between projects is great, but by also having projects use the same idioms for stuff like this it makes it much easier for people to work on multiple projects.
+
* [[Oslo/JunoGraduationPlans]]
  
This debate will probably never completely go away, though.
+
=== Juno Etherpads ===
  
== Resources ==
+
* [https://etherpad.openstack.org/p/juno-infra-library-testing|Testing pre-releases of Oslo libs with apps]
  
=== Blueprints ===
+
=== Icehouse Etherpads ===
  
We use [https://blueprints.launchpad.net/oslo/ launchpad blueprints] to track design proposals.
+
Etherpads from sessions at the Icehouse Design summit.
  
The [https://blueprints.launchpad.net/oslo/havana havana blueprints] detail the work currently underway.
+
* [https://etherpad.openstack.org/p/icehouse-oslo-pecan-wsme-tips Creating REST services with Pecan/WSME]
 +
* [https://etherpad.openstack.org/p/icehouse-oslo-openstack-client-update OpenStack Client Update]
 +
* [https://etherpad.openstack.org/p/icehouse-oslo-hacking-updates Updates to hacking, our code style enforcement tool]
 +
* [https://etherpad.openstack.org/p/icehouse-oslo-i18n-policies I18n policies of messages]
 +
* [https://etherpad.openstack.org/p/IcehouseOsloMessaging oslo.messaging - API design, plans for Icehouse]
 +
* [https://etherpad.openstack.org/p/icehouse-oslo-config-import-side-effects oslo.config enhancements, including removing import side-effects from consumers]
 +
* [https://etherpad.openstack.org/p/icehouse-oslo-rootwrap Rootwrap: Icehouse plans]
 +
* [https://etherpad.openstack.org/p/icehouse-oslo-db-migrations State of affairs in DB schema migrations]
 +
* [https://etherpad.openstack.org/p/icehouse-oslo-structured-notifications Towards more structured & qualified notifications]
 +
* [https://etherpad.openstack.org/p/icehouse-oslo-logging-and-notifications Merge logging and notifications]
 +
* [https://etherpad.openstack.org/p/icehouse-oslo-service-synchronization Writing a service synchronisation library]
 +
* [https://etherpad.openstack.org/p/icehouse-oslo-status Oslo incubated libraries status]
 +
* [https://etherpad.openstack.org/p/icehouse-oslo-splitting-the-incubator Aggressively split oslo-incubator]
  
 
=== Messaging Related Work in Havana ===
 
=== Messaging Related Work in Havana ===

Revision as of 18:31, 17 July 2018


Revised on: 7/17/2018 by Bnemec

OpenStack Project Oslo mascot.jpg

Official Title: OpenStack Common Libraries

PTL: Ben Nemec <openstack@nemebean.com>

Mission Statement:
To produce a set of python libraries containing code shared by OpenStack projects. The APIs provided by these libraries should be high quality, stable, consistent, documented and generally applicable.

The Oslo Team

The Oslo program brings together generalist code reviewers and specialist API maintainers. They share a common interest in tackling copy-and-paste technical debt across the OpenStack project.

Generalist Code Reviewers

Oslo's core reviewers take on a generalist role on the project. They are folks with good taste in Python code, provide constructive input in their reviews and make time to review any patches submitted to the project, irrespective of the area which a given patch targets.

Specialist API Maintainers

Each API typically has one or more specialist maintainers who have responsibility for evolving the API in question. They work to ensure the API meets the needs of all OpenStack projects, and once the API is stable and useful. Each library has its own core team, which can include specialists in the area of the library who are not general Oslo cores. Because the scope of the Oslo project has grown so large, these library-specific cores are **critical** to the long-term health of the projects and anyone with an interest in a library is encouraged to get involved.

Help Wanted

Over the years, contributors to Oslo projects have come and gone. In some cases this has left us with expertise gaps on important projects. New contributors to the following projects would be greatly appreciated:

  • oslo.privsep
  • oslo.service
  • taskflow

Project Meetings

See Meetings/Oslo.

Getting in Touch

We use the openstack-dev@lists.openstack.org mailing list for discussions and we all hang out in #openstack-oslo and #openstack-dev on freenode.

Each project also designates a liaison for handling integration issues. See Oslo/ProjectLiaisons.

Backwards Compatibility

As of the Newton summit, where a vote of 10 Oslo cores was 7 in favour, 0 against, 3 abstained, Oslo projects maintain backwards compatibility for one release cycle: a maximum period of 12 months (start of one release cycle to the end of the next). We do this so that important improvements in master that aren't suitable for backporting to a stable branch are still able to be used by deployers. For instance, we fixed heartbeat in oslo.messaging in this manner, and have an upcoming kafka support change that requires upgrading support libraries we use across incompatible versions: our users won't suffer, but folk using python-kafka directly would, making it unsuitable for stable backports.

To test this, we will be implementing both stable jobs on master oslo changes (run stable devstack on master oslo) and unstable jobs on stable server changes (run the server with latest-oslo-release).

We are not specifically aiming to support partial mixed upgrades (e.g. Nova Mitaka + Neutron Newton) in a single Python environment: there are no obvious hurdles to that from the Oslo side, but client library Python API compatibility is not governed by Oslo policies.

Jobs

History of (these) jobs: https://etherpad.openstack.org/p/dims-periodic-jobs

Periodic

Oslo (dvsm) latest against master

Oslo latest against (many projects) master

Other periodic jobs: http://logs.openstack.org/periodic/

Libraries

The following libraries are currently published by the Oslo program. Where we felt that a library had real potential for widespread use outside OpenStack, we chose not to include them in the oslo namespace (aka prefix them with oslo).

New libraries need to be careful to avoid introducing circular dependencies. See Oslo/Dependencies.

Specialized libraries have their own core-review team with members who may not be part of the main Oslo core team. Unless otherwise indicated below, an Oslo library is maintained by oslo-core.

automaton

Summary: a framework for building state machines.

Proposal: 141961

Source: http://git.openstack.org/cgit/openstack/automaton

Bugs: please file bugs in the automaton project in launchpad.

Core review team: oslo-core

Documentation: http://docs.openstack.org/developer/automaton/

castellan

Summary: generic key manager interface for OpenStack.

Proposal: adopt-castellan

Source: http://git.openstack.org/cgit/openstack/castellan

Bugs: please file bugs in the castellan project in launchpad.

Core review team: castellan-core

Documentation: http://docs.openstack.org/developer/castellan/

cookiecutter

Summary: a project that creates a skeleton OpenStack project from a set of templates.

Proposal: 42530

Source: http://git.openstack.org/cgit/openstack-dev/cookiecutter

Bugs: please file bugs in the oslo project in launchpad.

Core review team: oslo-core

Documentation: n/a

debtcollector

Summary: a collection of python patterns that help you collect your technical debt in a non-destructive manner (following deprecation patterns and strategies and so-on).

Proposal: 141220

Source: http://git.openstack.org/cgit/openstack/debtcollector

Bugs: please file bugs in the debtcollector project in launchpad.

Core review team: oslo-core

Documentation: http://docs.openstack.org/developer/debtcollector/

futurist

Summary: a collection of async functionality and additions from the future.

Proposal: 179890

Source: http://git.openstack.org/cgit/openstack/futurist

Bugs: please file bugs in the futurist project in launchpad.

Core review team: oslo-core

Documentation: http://docs.openstack.org/developer/futurist/

mox3

Summary: an unofficial port of mox framework to Python 3

Proposal: n/a

Source: http://git.openstack.org/cgit/openstack/mox3

Bugs: please file bugs in the mox3 project in launchpad.

Core review team: oslo-core

Documentation: n/a

oslo-cookiecutter

Summary: is a project that creates a skeleton Oslo library from a set of templates.

Proposal: n/a

Source: http://git.openstack.org/cgit/openstack-dev/oslo-cookiecutter

Bugs: please file bugs in the oslo-cookiecutter project in launchpad.

Core review team: oslo-core

Documentation: ??

oslo.cache

Summary: a library for caching based on dogpile.

Proposal: n/a

Source: http://git.openstack.org/cgit/openstack/oslo.cache

Bugs: please file bugs in the oslo.cache project in launchpad.

Core review team: oslo-core

Documentation: http://docs.openstack.org/developer/oslo.cache/

oslo.concurrency

Summary: a project with helpers managing external processes and task synchronization.

Proposal: n/a

Source: http://git.openstack.org/cgit/openstack/oslo.concurrency

Bugs: please file bugs in the oslo.concurrency project in launchpad.

Core review team: oslo-core

Documentation: http://docs.openstack.org/developer/oslo.concurrency/

oslo.config

Summary: a extensive library for parsing configuration files and command line arguments.

Proposal: see this historical blueprint describing the initial requirements for the API.

Source: http://git.openstack.org/cgit/openstack/oslo.config

Bugs: please file bugs in the oslo.config project in launchpad.

Core review team: oslo-core

Documentation: http://docs.openstack.org/developer/oslo.config/

oslo.context

Summary: a project with helpers to maintain useful information about a request context (aka associated information/data bound to a request).

Proposal: see this historical blueprint describing the initial requirements for the API.

Source: http://git.openstack.org/cgit/openstack/oslo.context

Bugs: please file bugs in the oslo.context project in launchpad.

Core review team: oslo-core

Documentation: http://docs.openstack.org/developer/oslo.context/

oslo.db

Summary: a extensive library to aid in database interactions and/or handling.

Proposal: n/a

Source: http://git.openstack.org/cgit/openstack/oslo.db

Bugs: please file bugs in the oslo.db project in launchpad.

Core review team: oslo-db-core

Documentation: http://docs.openstack.org/developer/oslo.db/

oslo.i18n

Summary: a wrapper library around Python's gettext module for string translation and other internationalization features.

Proposal: n/a

Source: http://git.openstack.org/cgit/openstack/oslo.i18n

Bugs: please file bugs in the oslo.i18n project in launchpad

Core review team: oslo-core

Documentation: http://docs.openstack.org/developer/oslo.i18n/

oslo.log

Summary: a logging configuration library.

Proposal: n/a

Source: http://git.openstack.org/cgit/openstack/oslo.logging

Bugs: please file bugs in the oslo.log project in launchpad.

Core review team: oslo-core

Documentation: http://docs.openstack.org/developer/oslo.log/

oslo.messaging

Summary: a library that provides a messaging API which supports RPC and notifications over a number of different messaging transports.

Proposal: this etherpad captures the latest status and background to this project.

Source: http://git.openstack.org/cgit/openstack/oslo.messaging

Devstack plugin: devstack-plugin-amqp1 devstack-plugin-kafka devstack-plugin-pika devstack-plugin-zmq

Bugs: please file bugs in the oslo.messaging project in launchpad.

Core review team: oslo-messaging-core

Documentation: http://docs.openstack.org/developer/oslo.messaging/

oslo.middleware

Summary: provides a collection of WSGI middleware for web service development.

Proposal: n/a

Source: http://git.openstack.org/cgit/openstack/oslo.middleware

Bugs: please file bugs in the oslo.middleware project in launchpad.

Core review team: oslo-core

Documentation: http://docs.openstack.org/developer/oslo.middleware/

oslo.policy

Summary: provides a rules engine for enforcing policy.

Proposal: n/a

Source: http://git.openstack.org/cgit/openstack/oslo.policy

Bugs: please file bugs in the oslo.policy project in launchpad.

Core review team: oslo-policy-core

Documentation: http://docs.openstack.org/developer/oslo.policy/

oslo.privsep

Summary: provides a mechanism for running selected python code with elevated privileges

Proposal: 204073

Source: http://git.openstack.org/cgit/openstack/oslo.privsep

Bugs: please file bugs in the oslo.privsep project in launchpad

Core review team: oslo-privsep-core

Documentation: http://docs.openstack.org/developer/oslo.privsep/

oslo.reports

Summary: allows projects to generate Guru Meditation Reports for debugging the current state of OpenStack processes. It can also be used to generating general reports on the fly that are serializable as plain text, JSON, or XML.

Proposal: n/a

Source: http://git.openstack.org/cgit/openstack/oslo.reports

Bugs: please file bugs in the oslo.reports project in launchpad.

Core review team: oslo-core

Documentation: http://docs.openstack.org/developer/oslo.reports/

oslo.rootwrap

Deprecated (with preference to oslo.privsep)

Summary: rootwrap allows fine filtering of shell commands to run as root from OpenStack services.

Proposal: n/a

Source: http://git.openstack.org/cgit/openstack/oslo.rootwrap

Bugs: please file bugs in the oslo.rootwrap project in launchpad

Core review team: oslo-rootwrap-core

Documentation: http://docs.openstack.org/developer/oslo.rootwrap/

oslo.serialization

Summary: is a library that provides serialization functionality with special handling for some common types used in OpenStack.

Proposal: n/a

Source: http://git.openstack.org/cgit/openstack/oslo.serialization

Bugs: please file bugs in the oslo.serialization project in launchpad.

Core review team: oslo-core

Documentation: http://docs.openstack.org/developer/oslo.serialization/

oslo.service

Summary: is a helper library that provides functionality for running OpenStack services.

Proposal: n/a

Source: http://git.openstack.org/cgit/openstack/oslo.service

Bugs: please file bugs in the oslo.service project in launchpad.

Core review team: oslo-core

Documentation: http://docs.openstack.org/developer/oslo.service/

oslo.tools

Summary: Tools for working in the openstack oslo community

Proposal: n/a

Source: http://git.openstack.org/cgit/openstack/oslo.tools

Bugs: n/a

Core review team: oslo-core

Documentation: n/a

oslo.utils

Summary: is a helper library that provides various low-level utility modules/code (that doesn't have a home anywhere else).

Proposal: n/a

Source: http://git.openstack.org/cgit/openstack/oslo.utils

Bugs: please file bugs in the oslo.utils project in launchpad.

Core review team: oslo-core

Documentation: http://docs.openstack.org/developer/oslo.utils/

oslo.versionedobjects

Summary: is a library that helps deal with DB schema being at different versions than the code expects, allowing services to be operated safely during upgrades (among other things).

Proposal: 127532

Source: http://git.openstack.org/cgit/openstack/oslo.versionedobjects

Bugs: please file bugs in the oslo.versionedobjects project in launchpad.

Core review team: oslo-core

Documentation: http://docs.openstack.org/developer/oslo.versionedobjects/

oslo.version

!!Retired!!

Summary: is a helper library that provides for getting the version for an installed piece of software from the python metadata that already exists.

Proposal: 40498

Source: http://git.openstack.org/cgit/openstack/oslo.version

Bugs: please file bugs in the oslo project in launchpad.

Core review team: oslo-core

Documentation: ??

oslo.vmware

Summary: provides for a shared location for code common to the VMware drivers in several projects.

Proposal: n/a

Source: http://git.openstack.org/cgit/openstack/oslo.vmware

Bugs: please file bugs in the oslo.vmware project in launchpad.

Core review team: oslo-vmware-core

Documentation: http://docs.openstack.org/developer/oslo.vmware/

oslosphinx

Summary: is a sphinx add-on library that provides theme and extension support for Sphinx documentation from the OpenStack project.

Proposal: n/a

Source: http://git.openstack.org/cgit/openstack/oslosphinx

Bugs: please file bugs in the oslosphinx project in launchpad.

Core review team: oslo-core

Documentation: http://docs.openstack.org/developer/oslosphinx/

oslotest

Summary: is a helper library that provides base classes and fixtures for creating unit and functional tests.

Proposal: n/a

Source: http://git.openstack.org/cgit/openstack/oslotest

Bugs: please file bugs in the oslotest project in launchpad.

Core review team: oslo-core

Documentation: http://docs.openstack.org/developer/oslotest/

osprofiler

Summary: an OpenStack cross-project profiling library.

Proposal: 103825

Source: http://git.openstack.org/cgit/openstack/osprofiler

Bugs: please file bugs in the osprofiler project in launchpad.

Core review team: osprofiler-core

Documentation: http://docs.openstack.org/developer/osprofiler

pbr

Summary: pbr (or Python Build Reasonableness) is a add-on library that helps provide (and enforce) a set of sensible default setuptools behaviours.

Proposal: n/a

Source: http://git.openstack.org/cgit/openstack-dev/pbr

Bugs: please file bugs in the pbr project in launchpad.

Core review team: pbr-core

Documentation: http://docs.openstack.org/developer/pbr/

pylockfile

Deprecated

Summary: legacy (and adopted) inter-process lock management library.

Proposal:102202

Source: http://git.openstack.org/cgit/openstack/pylockfile

Bugs: please file bugs in the pylockfile project in launchpad.

Core review team: oslo-core

Documentation: http://docs.openstack.org/developer/pylockfile/

stevedore

Summary: a library for managing plugins for Python applications.

Proposal: n/a

Source: http://git.openstack.org/cgit/openstack/stevedore

Bugs: please file bugs in the python-stevedore project in launchpad.

Core review team: stevedore-core

Documentation: http://docs.openstack.org/developer/stevedore/

taskflow

Summary: a library that helps create applications that handle state/failures... in a reasonable manner.

Proposal: n/a

Source: http://git.openstack.org/cgit/openstack/taskflow

Bugs: please file bugs in the taskflow project in launchpad.

Core review team: taskflow-core

Documentation: http://docs.openstack.org/developer/taskflow/

tooz

Summary: a library that aims at centralizing the most common distributed primitives like group membership protocol, lock service and leader election by providing a coordination API helping developers to build distributed applications.

Proposal: 122439

Source: http://git.openstack.org/cgit/openstack/tooz

Bugs: please file bugs in the python-tooz project in launchpad.

Core review team: tooz-core

Documentation: http://docs.openstack.org/developer/tooz/

Principles

APIs included in Oslo should reflect a rough consensus across the project on the requirements and design for that use case. New OpenStack projects should be able to use an Oslo API safe in the knowledge that, by doing so, the project is being a good OpenStack citizen and building upon established best practice.

To that end, we keep a number of principles in mind when designing and evolving Oslo APIs:

  1. The API should be generally useful and a "good fit" - e.g. it shouldn't encode any assumptions specific to the project it originated from, it should follow a style consistent with other Oslo APIs and should fit generally in a theme like error handling, configuration options, time and date, notifications, WSGI, etc.
  2. The API should already be in use by a number of OpenStack projects
  3. There should be a commitment to adopt the API in all other OpenStack projects (where appropriate) and there should be no known major blockers to that adoption
  4. The API should represents the "rough consensus" across OpenStack projects
  5. There should be no other API in OpenStack competing for this "rough consensus"
  6. It should be possible for the API to evolve while continuing to maintain backwards compatibility with older versions for a reasonable period - e.g. compatibility with an API deprecated in release N may only be removed in release N+2

Incubation

The Oslo Incubator is now dead. Refer to the announcement and the change clearing out the repository.

FAQs

Why aren't alpha releases of oslo.config published to PyPI?

See Choosing Version Numbers for the current policies related to versioning and releases.

Why does oslo.config have a CONF object? Global object SUCK!

Indeed. Well, it's a long story and well documented in mailing list archives if anyone cares to dig up some links.

Around the time of the Folsom Design Summit, an attempt was made to remove our dependence on a global object like this. There was massive debate and, in the end, the rough consensus was to stick with using this approach.

Nova, through its use of the gflags library, used this approach from commit zero. Some OpenStack projects didn't initially use this approach, but most now do. The idea is that having all projects use the same approach is more important than the objections to the approach. Sharing code between projects is great, but by also having projects use the same idioms for stuff like this it makes it much easier for people to work on multiple projects.

This debate will probably never completely go away, though. See this latest discussion in August, 2014.

Why does Oslo observe feature freeze

Feature freeze is a time to stabilize all of the new features that were added during a development cycle, but since Oslo projects don't necessarily release on the same six month schedule as the other OpenStack projects (or at all in the case of oslo-incubator) it might seem odd that Oslo observes feature freeze.

For the graduated libraries this serves the same purpose as for any of the other projects - it's a time for focusing on bug fixes and stability.

For oslo-incubator, the primary motivation is making last-minute fixes needed by other projects easier to sync. If a new feature lands in oslo-incubator and an unrelated bug is discovered by one of the consuming projects, it becomes a problem to sync just the bug fix to the project. When 11th hour bug fixes are needed it's best if the sync is as simple and small as possible. To avoid problems, oslo-incubator respects the feature freeze period just like any other project.

How does Oslo manage versions?

See Oslo/VersioningPolicy.

Resources

Review policies

These overlay the regular review rules for OpenStack as a whole.

  • Automated changes - patches from 'openstack proposal bot' and 'transifex' which have passed CI checks can be +2+A by a single core reviewer

Review Links

(See http://git.openstack.org/cgit/openstack/oslo.tools/tree/dashboards for the source files to create these links)

Security Team

In addition to OpenStack's Vulnerability Management team, some members of the Oslo team have indicated their willingness to help with security related issues in Oslo code. See Oslo/Security for the current list.

Design Proposals

We use the oslo-specs repo to track design proposals across all Oslo projects.

See the Spec Approval policy for details.

The blueprints on launchpad detail the changes currently underway to implement these specs.

Approved Specs are published separately.

Release Instructions

Oslo/ReleaseProcess

Summits

Pike

Ocata

Newton

Mitaka

Liberty

Kilo

Juno

Juno Etherpads

Icehouse Etherpads

Etherpads from sessions at the Icehouse Design summit.

Messaging Related Work in Havana

During the Havana cycle, work is going on to re-design our messaging APIs and to add signatures and encryption to our messages.

See this etherpad for yet more details.

Havana Etherpads

Etherpads from sessions at the Havana Design summit.

Grizzly Etherpads

Etherpads from sessions at the Grizzly Design summit.

Folsom Etherpads

Etherpads from sessions at the Folsom Design summit.