Oslo Library Graduation Status

cliff

S: Released

cliff

https://launchpad.net/python-cliff

M: Doug Hellmann <doug.hellmann@dreamhost.com>

S: Maintained

F: openstack/cliff repository

Depends on: (none)

oslo.cache

bp: https://blueprints.launchpad.net/oslo/+spec/graduate-oslo-cache

cache

M: Flavio Percoco <flavio@redhat.com>

M: Morgan Fainberg <m@metacloud.com>

S: Maintained

F: cache/

Depends on: lockutils, timeutils

memorycache

M:

S: Orphan

F: memorycache.py

Depends on: timeutils

oslo.client

bp: https://blueprints.launchpad.net/oslo/+spec/graduate-oslo-client

apiclient

M: Alessio Ababilov <aababilov@griddynamics.com>

S: Maintained

F: apiclient/

Depends on: strutils, importutils

oslo.concurrency

S: Next

bp: https://blueprints.launchpad.net/oslo/+spec/graduate-oslo-concurrency

lockutils

M: Michael Still <mikal@stillhq.com>

S: Maintained

F: lockutils.py

Depends on: oslo.config, fileutils, gettextutils

This seems pretty stable now. I see no reason it couldn't be released as a library.

Comments from bnemec via ML:

 It would be nice to get lockutils graduated to solve some of the issues mentioned in the oslo.db section, but I believe we do have an outstanding question regarding its behavior without lock_path being set.  I think Clint was on board with Sean's proposed solution after quite a bit of discussion (http://lists.openstack.org/pipermail/openstack-dev/2013-December/021620.html), so it's possible we could just restore that patch and call it done, but it should probably be addressed somehow before graduation.

The issue with lock path is resolved by moving most locks to POSIX IPC instead of file locks, except when the lock is across servers. tooz should provide a better solution for that. - dhellmann

TODO: Remove the dependency of fixture on lockutils by moving fixture/lockutils.py into oslo.concurrency

TODO: Evaluate the relationship between this module and tooz, before graduation.

processutils

M: Michael Still <mikal@stillhq.com>

S: Maintained

F: processutils.py

Depends on: gettextutils

change status from maintained?

needs some code cleanup (greenthreads and random sleeps)

oslo.config

S: Released

cfgfilter

M: Mark McLoughlin <markmc@redhat.com>

S: Maintained

F: cfgfilter.py

Depends on: oslo.config

bp: https://blueprints.launchpad.net/oslo/+spec/graduate-cfgfilter

Moved to oslo.config, remove from this list? - dhellmann

keep for filter

config

M: Julien Danjou <julien@danjou.info>

M: Zhongyue Luo <zhongyue.nah@intel.com>

S: Maintained

F: config/

Depends on: oslo.config, gettextutils, importutils

bp: https://blueprints.launchpad.net/oslo/+spec/graduate-config-generator

This is far from being optimal, but it's the best we can do. There's a lot of problem with some stuff like dynamically registered option, but I'm not sure we can really do anything about it.

We will be talking about this issue at the summit

oslo.config

(Released library)

Depends on: (none)

oslo.context

S: Next

context

M:

S: Orphan

F: context.py

There is some confusion about whether this should be obsoleted or not, based on notes from the icehouse summit (https://etherpad.openstack.org/p/icehouse-oslo-status). Given some trouble we had late in icehouse with projects not having consistent parameters in their contexts, it seems like what we want is to provide a context base class which the projects can subclass and then customize to add data and fill in the common values in an app-specific way (https://blueprints.launchpad.net/oslo/+spec/app-agnostic-logging-parameters). --doug-hellmann (talk) 17:54, 11 March 2014 (UTC)

oslo.crypto

crypto

M: Simo Sorce <simo@redhat.com>

S: Maintained

F: crypto/

Depends on: gettextutils, importutils

bp: https://blueprints.launchpad.net/oslo/+spec/graduate-oslo-crypto

KDS's REST API is being slightly changed after review by the Keystone developers. This will require some minor changes to the SecureMessage internal code used to fetch and tickets from the KDS.

This code will be used in oslo.messaging and keystone (markmc)

Should it eventually live in oslo.crypto or should it be part of a library that the keystone devs maintain? - dhellmann

It is mostly used in KDS to encrypt keys before storage and in this capacity it should be reusable for swift/cinder or projects looking to do encrypted storage. It's fairly low level at the moment but a simpler interface to this can be added when there is reuse of the module. --jamielennox

oslo.db

S: Released

bp: https://blueprints.launchpad.net/oslo/+spec/oslo-db-lib

db

M: Boris Pavlovic <boris@pavlovic.me>

M: Roman Podolyaka <rpodolyaka@mirantis.com>

M: Victor Sergeyev <vsergeyev@mirantis.com>

S: Maintained

F: db/

Depends on: importutils, timeutils, gettextutils

On the API side we've been pretty stable so far.

The big code changes worth mentioning are:

using of [database] group for DB options instead of [DEFAULT] (must be handled gracefully by oslo.config deprecated_name|group options)
new convention for naming of unique constraints (it's needed for providing developers with informative exceptions on integrity errors)
support of slave engines (the ability to distribute SELECT queries among slave DBs -- this only extends the existing API a bit)
moving of useful sqlalchemy and migrations utils developed in Nova to common code

It would be nice to complete these tasks before making common db code a separate library, but they aren't blockers in any way:

   https://blueprints.launchpad.net/oslo/+spec/tests-given-db-backend
   https://bugs.launchpad.net/oslo/+bug/1224898
   https://blueprints.launchpad.net/nova/+spec/db-reconnect

openstack.common.db should become a separate library - oslo.db We did a PoC a few months ago - (https://review.openstack.org/#/c/42159/) Ironic test suite passed when oslo.db was installed as a separate library (https://github.com/malor/oslo.db)

The challenge with making oslo.db a separate library is that common modules we use (e.g. lockutils) might possibly conflict with the ones in OpenStack projects due to defining the same oslo.config options, but having different definitions for those (e.g. oslo.config raised an exception, because lockutils version in Ironic had a config option, that didn't provide a help string, but the newer version of lockutils, we used in oslo.db, did)

Move lockutils out of incubation first? - dhellmann

db api must be cleaned up to remove the eventlet dependency before graduating

needs to be synced with nova's version

oslo.hooks

bp: https://blueprints.launchpad.net/oslo/+spec/graduate-oslo-hooks

hooks

M: Andrey Kurilin <akurilin@mirantis.com>

S: Maintained

F: hooks.py

Depends on: gettextutils, stevedore

oslo.i18n

S: Released

bp: https://blueprints.launchpad.net/oslo/+spec/graduate-oslo-i18n

gettextutils

M: Mark McLoughlin <markmc@redhat.com>

S: Maintained

F: gettextutils.py

Depends on: (none)

I think this is mostly being maintained by the various folks working on delayed-message-translation now (markmc)

Set a goal to have this ready to move out of the incubator during J? - dhellmann

oslo.image

bp: https://blueprints.launchpad.net/oslo/+spec/graduate-oslo-image

imageutils

M: Zhongyue Luo <zhongyue.nah@intel.com>

S: Maintained

F: imageutils.py

Depends on: gettextutils, strutils

This module has just been added. Next steps are to add common image manipulation helper functions used in Nova and Cinder.

oslo.io

bp: https://blueprints.launchpad.net/oslo/+spec/graduate-oslo-io

fileutils

M: Zhongyue Luo <zhongyue.nah@intel.com>

S: Maintained

F: fileutils.py

Depends on: excutils, gettextutils

There are still changes being made to function signatures to meet the needs of mocking when writing test scripts. Needs a cycle or two.

oslo.log

S: Next

bp: https://blueprints.launchpad.net/oslo/+spec/graduate-oslo-log

NOTE: The dependencies 'on' this library have been removed from the other modules because we assume we will update them to use python's logging directly as described below and in the blueprints for graduating oslo.log.

local

M:

S: Orphan

F: local.py

Depends on: (none)

This module is only used to manage thread-locals for the logging context objects, and we may want to make it a private implementation detail of the logging package. However, something similar is used in oslo.messaging.localcontext, and if the intent there is to make sure the context is available for logging then something needs to make the 2 modules work together. See https://review.openstack.org/#/c/110070/ --doug-hellmann (talk) 16:28, 5 August 2014 (UTC)

log

M:

S: Orphan

F: log.py

Depends on: oslo.config, gettextutils, importutils, jsonutils, local, context

flaper87: Agreed!

TODO: Change handling of publish_errors configuration option so the log_handler module isn't imported with importutils, since that module is moving to oslo.messaging and we do not want a circular dependency.

TODO: Install the formatter to do translation, but don't require everyone to use the ContextAdapter, so we can remove dependencies on this lib from other libs.

The log_handler module has already moved to oslo.messaging, but I don't see the configuration option. We need to change log.py to import the module from the new library, or come up with a more generic way to let things hook into the logging setup phase. --doug-hellmann (talk) 22:21, 6 March 2014 (UTC)

24 Oct 2014 - dhellmann - The logging code will define an API for the context class to support for providing logging data for the context. We also need something to provide the context to the log, either by calling a callback when a context is created or by exposing an API to ask for the context explicitly.

oslo.messaging

S: Released

oslo.messaging

M: Mark McLoughlin <markmc@redhat.com>

S: Maintained

F: (released library)

Depends on: excutils, gettextutils, importutils, jsonutils, network_utils, sslutils, timeutils

log_handler

M:

S: Orphan

F: log.py

Depends on: oslo.config, notifier

notifier

M: Julien Danjou <julien@danjou.info>

S: Maintained

F: notifier/

F: middleware/notifier.py

Depends on: context, gettextutils, importutils, jsonutils, timeutils, rpc

bp: https://blueprints.launchpad.net/oslo/+spec/graduate-notifier-middleware

Will be in oslo.messaging and removed from incubator in icehouse (markmc) http://docs.openstack.org/developer/oslo.messaging/notifier.html

rpc

M: Mark McLoughlin <markmc@redhat.com>

M: Russell Bryant <rbryant@redhat.com>

S: Maintained

F: rpc/

Depends on: gettextutils, excutils, importutils, jsonutils, local, network_utils, rpc, service, sslutils, versionutils

bp: https://blueprints.launchpad.net/oslo/+spec/remove-rpc-from-incubator

Will be moving projects from this to oslo.messaging in Icehouse so that this can be removed (markmc)

zmq

M: Mike Wilson <geekinutah@gmail.com>

M: Paul Mathews <pmathews@bluehost.com>

S: Maintained

F: rpc/impl_zmq.py

F: rpc/matchmaker*.py

F: rpc/zmq_receiver.py

bp: https://blueprints.launchpad.net/oslo/+spec/graduate-zmq-driver

Moving to oslo.messaging (markmc)

oslo.middleware

bp: https://blueprints.launchpad.net/oslo/+spec/graduate-oslo-middleware

middleware

M:

S: Orphan

F: middleware/

Depends on: gettextutils, context, notifier

oslo.policy

bp: https://blueprints.launchpad.net/oslo/+spec/graduate-oslo-policy

The keystone team is also interested in owning the policy management code. - dhellmann, 7 July 2014

policy

M: Flavio Percoco <flavio@redhat.com>

M: Adam Young <ayoung@redhat.com>

S: Maintained

F: policy.py

Depends on: fileutils, gettextutils, jsonutils, oslo.config

We could get rid of the oslo.config dependency.

(Agreed on the stability of policy.py)

oslo.quota

bp: https://blueprints.launchpad.net/oslo/+spec/graduate-oslo-quota

quota

M: Sergey Skripnick <sskripnick@mirantis.com>

S: Maintained

F: quota.py

Depends on: gettextutils, importutils, timeutils

This file is merged from nova and cinder, but not used in any project yet.

Wait to see how merging it back into nova and cinder works? --doug-hellmann (talk) 23:09, 6 March 2014 (UTC)

oslo.reports

bp: https://blueprints.launchpad.net/oslo/+spec/graduate-oslo-reports

reports

M: Solly Ross <sross@redhat.com>

S: Maintained

F: report/

Depends on: jsonutils

Where is this being used? - dhellmann

We are planning to use the pattern and some of the code here for Ceilometer reporting. Most of the existing stuff is pretty specific to the Guru Mediation Report, it looks like. Link: https://wiki.openstack.org/wiki/OutreachProgramForWomen/Ideas#Reporting_Framework_for_Ceilometer_based_on_Oslo_Reporting - thomasem

oslo.rootwrap

S: Released

bp: https://blueprints.launchpad.net/oslo/+spec/standalone-rootwrap

rootwrap

M: Thierry Carrez <thierry@openstack.org>

S: Maintained

F: (released)

Depends on: (none)

Graduated as oslo.rootwrap standalone library during the Icehouse cycle.

oslo.serialization

S: Released

bp: https://blueprints.launchpad.net/oslo/+spec/graduate-oslo-serialization

jsonutils

M:

S: Orphan

F: jsonutils.py

Depends on: gettextutils, importutils, timeutils

xmlutils

M:

S: Orphan

F: xmlutils.py

Depends on: (none)

This was a security fix. Can we rely on our dependencies having this fix now? - dhellmann

oslo.server

bp: https://blueprints.launchpad.net/oslo/+spec/graduate-oslo-server

eventlet_backdoor

M:

S: Orphan

F: eventlet_backdoor.py

Depends on: gettextutils

should eventually go into the service library (move in the incubator to the service stuff)

loopingcall

M:

S: Orphan

F: loopingcall.py

Depends on: gettextutils, timeutils

periodic_task

M: Michael Still <mikal@stillhq.com>

S: Maintained

F: periodic_task.py

Depends on: gettextutils, timeutils

This one seems stable too.

requestutils

M: Sandy Walsh <sandy.walsh@rackspace.com> S: Maintained F: request_utils.py

Depends on: gettextutils

service

M: Michael Still <mikal@stillhq.com>

S: Maintained

F: service.py

Depends on: eventlet_backdoor, gettextutils, importutils, threadgroup

sslutils

M:

S: Orphan

F: sslutils.py

Depends on: gettextutils

threadgroup

M:

S: Orphan

F: threadgroup.py

Depends on: loopingcall

fixture

M: Monty Taylor <mordred@inaugust.com>

S: Maintained

F: fixture/

Depends on: (none)

TODO: Remove the dependency on lockutils by moving fixture/lockutils.py into oslo.concurrency

TODO: Remove the dependency on oslo.config by moving fixture/config.py into oslo.config

test

M:

S: Orphan

F: test.py

Depends on: (none)

oslo.text

S: Next

bp: https://blueprints.launchpad.net/oslo/+spec/graduate-oslo-text

oslo.units

units

M:

S: Orphan

F: units.py

Depends on: (none)

oslo.utils

S: Released

bp: https://blueprints.launchpad.net/oslo/+spec/graduate-oslo-utils

excutils

M: S: Orphan F: excutils.py

Depends on: gettextutils

Other than its use of our gettextutils, this could be pulled out into a separate library without needing the oslo name. - dhellmann

can we use the stdlib? should we remove the translations? (2 messages)

look for similar code elsewhere

contribute to stdlib?

How do we handle translations for the log messages if this goes into its own lib?

network_utils

M:

S: Orphan

F: network_utils.py

Depends on: (none)

I'm trying to put these things in stdlib see http://bugs.python.org/issue18191 -- jd

won't graduate?

strutils

M: Flavio Percoco <flavio@redhat.com>

S: Maintained

F: strutils.py

Depends on: gettextutils

I think strutils is stable enough, however, I'd like to:

   1) Find a common place for this and other modules. I don't think it is worth having oslo.strutils. We once talk about having oslo.text and group strutils and other modules there.
   2) Make sure it's PY3K support. Even though python 3 support is not an OS requirement, this module should be fairly simple to port and we could have a voting PY3K gate for it.

It may make sense to keep the encoding/decoding functions. to_slug() is used in the clients, maybe that can move to the new client section of the incubator? I would like to replace the units parsing code with another library if we can find one suitable. Pint does not have python 2.6 support, but would work other than that. - dhellmann

timeutils

M: Zhongyue Luo <zhongyue.nah@intel.com>

S: Maintained

F: timeutils.py

Depends on: (none)

The problem with timeutils is that code for testing coexist. For example, IMO set_time_override should be removed and use mock or fixture to replace its functionality.

Do most of the functions here only apply for testing? The parsing and formatting functions may be useful, to ensure consistent timestamp formats.

uuidutils

M: Zhongyue Luo <zhongyue.nah@intel.com>

S: Maintained

F: uuidutils.py

Depends on: (none)

This module is stable.

Generating the UUID string seems trivial enough to not need a function. There is some work going into WSME to validate UUID strings at the API layer. Is the validation code used anywhere else? - dhellmann

nova uses this all over the place to check if a string is a uuid; can we move that function into nova and deprecate/delete this module?

neutron also uses the same function

oslo.version

S: Released

oslo.version

M:

S: Orphan

F:

Depends on: (none)

Utilities for consuming the version from pkg_resources.

oslo.versionutils

Should this move to oslo.version instead? No, that would add extra dependencies to oslo.version that would make it harder to use in installation scripts. - dhellmann

versionutils

M:

S: Orphan

F: versionutils.py

Depends on: gettextutils

TODO: Break circular dependency between versionutils and log by moving deprecation logging function out of log and into versionutils.

oslo.vmware

S: Released

oslo.vmware

M: Davanum Srinivas <davanum@gmail.com>

S: Maintained

F: openstack/oslo.vmware repo

Depends on: gettextutils, importutils, jsonutils, loopingcall, timeutils

pbr

S: Released

pbr

M: Monty Taylor <mordred@inaugust.com>

S: Maintained

PyCADF

S: Released

pycadf

https://launchpad.net/pycadf

M: Gordon Chung <chungg@ca.ibm.com>

S: Maintained

F: openstack/pycadf repository

Depends on: oslo.messaging, oslo.config

The auditing middleware in pycadf depends on oslo.messaging.

See also middleware/audit.py in oslo-incubator.

stevedore

S: Released

stevedore

https://launchpad.net/python-stevedore

M: Doug Hellmann <doug.hellmann@dreamhost.com>

S: Maintained

F: openstack/stevedore repository

Depends on: (none)

taskflow

S: Released

taskflow

https://launchpad.net/taskflow

M: Joshua Harlow <harlowja@yahoo-inc.com>

S: Maintained

F: openstack/taskflow repository

Depends on: gettextutils, stevedore, excutils, importutils, jsonutils, timeutils, uuidutils

authutils

S: Deleting

authutils

M:

S: Orphan

F: authutils.py

I found no users of this module outside of its tests under openstack/* It was apparently brought in from keystone by russelb - dhellmann

fixes a vulnerability using constant time string comparison

Added by https://review.openstack.org/6221

See https://bugs.launchpad.net/keystone/+bug/942644

Was strcmp_const_time() in Nova when Nova had its own auth code removed by https://review.openstack.org/9579

Swift uses it too (https://review.openstack.org/4643) but swift hates Oslo, so ....

 swift/common/utils.py:1866:def streq_const_time(s1, s2)

Conclusion: only keystone needs this, so we can remove it from oslo-incubator

cliutils

S: Deleting

cliutils

M:

S: Orphan

F: cliutils.py

Ignoring Depends on: apiclient, gettextutils, importutils, strutils, uuidutils

These functions should move into the python-openstackclient project, if they are still needed. - dhellmann

middleware/context

S: Deleting

middleware/context

M:

S: Obsolete

F: context.py

F: middleware/context.py

Deprecate? Delete?

delete, but look at the notifier code and logging -- also reviews in nova that were trying to rebase on top of this?

used in context middleware (markmc says delete) and rpc (obsoleted by oslo.messaging)

scheduler

S: Deleting

scheduler

M:

S: Orphan

F: scheduler/

boris-42: We should get rid of this. There is a way to build scalable one scheduler as a service. And solve a lot of current issues.

INCUBATOR:funcutils

funcutils

M: Joshua Harlow <harlowja@yahoo-inc.com>

S: Maintained

F: funcutils.py

Depends on: (none)

3 June 2014 - Staying in the incubator until 2.6 support is no longer needed. -- dhellmann

https://etherpad.openstack.org/p/juno-oslo-release-plan

INCUBATOR:importutils

importutils

M:

S: Orphan

F: importutils.py

Depends on: (none)

Deprecate the functions replaced by stevedore & pkg_resources? the try_import() function looks trivial, do we need it? - dhellmann

3 June 2014 -- Staying in the incubator until everyone has moved to stevedore. -- dhellmann

keystonemiddleware

The keystonemiddleware library will be owned by the Keystone team, and is only listed here to reflect the new location of the audit middleware library.

audit

M: Gordon Chung <chungg@ca.ibm.com>

S: Maintained

F: middleware/audit.py

Depends on: (none)

this middleware is relatively lightweight as it utilises notifier middleware to handle most of the actions... outside of that it uses only pycadf library to append audit info. currently making improvements to pycadf library... we have plans to extend its functionality but the overall model is relatively stable. (ref: https://launchpad.net/pycadf)

The audit middleware in pycadf has its own copy of the notifier middleware. That will need to change to use the one in oslo.messaging. --doug-hellmann (talk) 22:31, 6 March 2014 (UTC)

Oslo/GraduationStatus

Contents

Oslo Library Graduation Status

cliff

cliff

oslo.cache

cache

memorycache

oslo.client

apiclient

oslo.concurrency

lockutils

processutils

oslo.config

cfgfilter

config

oslo.config

oslo.context

context

oslo.crypto

crypto

oslo.db

db

oslo.hooks

hooks

oslo.i18n

gettextutils

oslo.image

imageutils

oslo.io

fileutils

oslo.log

local

log

oslo.messaging

oslo.messaging

log_handler

notifier

rpc

zmq

oslo.middleware

middleware

oslo.policy

policy

oslo.quota

quota

oslo.reports

reports

oslo.rootwrap

rootwrap

oslo.serialization

jsonutils

xmlutils

oslo.server

eventlet_backdoor

loopingcall

periodic_task

requestutils

service

sslutils

threadgroup

fixture

test

oslo.text

oslo.units

units

oslo.utils

excutils

network_utils

strutils

timeutils

uuidutils

oslo.version

oslo.version

oslo.versionutils

versionutils

oslo.vmware

oslo.vmware

pbr

pbr