The current trove impl for loading API extensions is based on a single path which is specified in the trove conf file and then searched at trove-api start-up time to find and load extensions. While this approach is sufficient for loading trove proper extensions it becomes more painful for consumers who wish to package and / or consume trove extensions not shipped with trove proper. With the current approach, a consumer must copy his / her extension class (a concrete impl of trove.common.extensions.ExtensionDescriptor ) into the path specified by the api_extensions_path conf property in trove.conf in order for it to be picked up the the trove-api extension loading . The act of copying py files into another modules install path is a bad practice and this is not a very robust / flexible approach.
What's proposed here to move to a more dynamic means to discover and load extensions using the stevedore package: http://stevedore.readthedocs.org/en/latest/index.html
-  https://github.com/openstack/trove/blob/master/trove/openstack/common/extensions.py#L35
-  https://github.com/openstack/trove/blob/master/trove/common/cfg.py#L40
-  https://github.com/openstack/trove/blob/master/trove/openstack/common/extensions.py#L407
A more dynamic / robust means to discover and load trove extensions will provide greater ease of use and flexibility for trove consumers as well as for trove proper. Moreover using stevedore to discover and load extensions will move trove proper more in line with how other core projects do things (e.g. nova and co). In particular:
- Consumers can package and distribute trove extensions in separate python packages and the trove framework will automatically discover and load them without any changes to trove code or conf.
- Trove proper development will not be constrained to only defining extensions in the fixed path / package of trove.extensions.routes
- Use of stevedore will be more in line with with other core projects do and put trove in a closer position to updating our extension handling (https://blueprints.launchpad.net/trove/+spec/extensions-update).
A simple PoC has been coded up and can be viewed here: https://github.com/bodenr/trove/commit/fa06e1d96e6a49a2a54057e8feb8e624edeaf728
As shown in the PoC minimal code changes are necessary to trove proper hence minimal risk from a code churn POV. The major impacts are as follows:
- The api_extensions_path conf property is no longer needed.
- Extensions are defined in the setup.cfg of python projects under a well defined extension key.
- As per the PoC the base Extension class (ExtensionDescriptor) is now a "proper" abstract class (using py package abc) which provides stronger runtime checking... However this change is not required per say.
- The extension loading code no longer needs to check a path and load init python classes -- this is done via stevedore.
However a potential impact which is not shown in the PoC is -- how do we handle upgrade for extensions. The concern here is that consumers may have extensions using the old (current) means whereupon they copied them into the extension path, but with the new approach these extensions need to be put into a well defined section of a setup.cfg. There are a few options for this consideration:
- Doc the change assuming not many consumers have custom extensions.
- Retain a legacy path in the code whereupon in addition to using the stevedore method to discover plugins we also still check the old-school path and only load plugins which were not loaded with stevedore... This provides backwards compatibility.
- Have a simple CLI or code in the "upgrade" path which writes out a custom setup.cfg to capture any non-standard extensions and effectively migrate them to the new approach.
- trove.conf will no longer need the api_extensions_path unless we keep it for backwards compat.
- The setup.cfg will have a new section to define the trove proper extensions (see PoC for example)
No DB changes.
No public API changes.
No CLI changes.
No REST API changes.
Nothing new other than whats described in the above sections w/r/t to internal impl of extension discovery and loading with stevedore.
RPC API description