From a vendor / consumer perspective, there are likely cases where consumers want to leverage the existing trove sqlachemy DB models and framework abstractions for their own extensions / features within the trove framework. For example a consumer may want to develop custom in-house (proprietary) add-ons which use persistence for trove which either they do not wish to contribute upstream, or they want to build out a PoC in-house before upstreaming. In such cases a more rapid time to value and lower risk investment can be achieved by leveraging the existing trove db framework and having the ability to plug-in their own schema / migration / etc..
The current trove sqlalchemy implementation contains the plumbing and support necessary to allow consumers to "plug-in" their own database mappers which in turn can define their own ORM mappings, schema, etc.. The hook point for such extensions in python can be found in trove.db.sqlalchemy.api.py in the configure_db() method which looks like this:
def configure_db(options, *plugins): session.configure_db(options) configure_db_for_plugins(options, *plugins)
Here any number of "plugins" can be passed to the configure_db() function allowing consumers to plug into trove's sqlalchemy ORM engine. In the current impl, the plugin is just a python object which containers a 'mapper' attribute which defines the map(self, engine) method which of course defines ORM mappings atop troves sqlalchemy engine.
For example you could define this simple custom DB plugin / mapper (fictional):
class Mapper(object): def map(self, engine): meta = MetaData() meta.bind = engine if mappers.mapping_exists(my_models.Person): return orm.mapper(my_models.Person, Table('person', meta, autoload=True))
class DBPlugins(object): def __init__(self): self.mapper = Mapper()
which can then be added to trove's ORM using:
from trove.db import get_db_api get_db_api().configure_db(CONF, DBPlugins())
This is all fine, and everything shown above exists already in trove to date. However what's missing is the ability for consumers to pass in their 'plugins' via the main entry points. Instead the entry points do not permit passing any plugins to def configure_db(options, *plugins). See: https://github.com/openstack/trove/blob/master/trove/cmd/common.py#L52
Whats being propose here is: (a) Support a comma list property on CONF.DEFAULT in the trove conf files. e.g.
[DEFAULT] db_plugins = org.foo.bar.sqlalchemy.BarPlugins,org.yadda.sqlalchemy.MyPlugins
(b) Update the common.py entry point (see link above) to load each of the CONF.db_plugins as an object and pass them to configure_db() in the common.py (linked above).
a-b above would permit consumers to plug into troves ORM.
Obviously at this point you are probably wondering about migration and munking up troves tables / schema. However the way I've done this in a PoC allows the db plugin schema and migration impl to exist in parallel to trove's. Here's an overview of how that would look from a consumer perspective wanting to plug into trove's db: - Consumer creates their own migrate_repo artifacts (separate from troves) including:
- migrate.cnf with own version_table and repository_id - schema.py if needed - migrate version py files.. for example com.foo.dbaas.db.migate_repo.versions.001_myplugin.py
- Consumer defines their plugin mapper. for example: com.foo.dbaas.db.plugin.py (see my previous description on what the mapper does) - Consumer adds their plugin to the trove.conf: db_plugins = com.foo.dbaas.db.plugin.DBPlugins - Consumer uses trove-manage to create their schema (1-time action upon install). e.g. trove-manage --config-file=/etc/trove/trove.conf db_sync --repo_path=/path/to/com/foo/dbaas/db/plugin/migrate_repo - Now when trove is started it imports the plugin object from CONF.db_plugins and passes that to configure_db() method... everything just works now
Note in the above, trove-manage already supports pointing to a different migrate_repo path.
As you can see in the above, the consumer manages their own db schema, migration, etc. and it just resides happily in trove's db next to the trove proper schema.
- What is the driving force behind this change?
- Does it allow for great flexibility? Stability? Security?
- Does this impact any configuration files? If so, which ones?
- Does this impact any existing tables? If so, which ones?
- Are the changes forward and backward compatible?
- Be sure to include the expected migration process
- Does this change any API that an end-user has access to?
- Are there any exceptions in terms of consistency with other APIs?
- How the command will look like?
- Does it extends the already existed command interfaces ?
- Which HTTP methods added ?
- Which routes were added/modified/extended?
- How does the Request body look like?
- How does the Response object look like?
- Does this change any internal messages between API and Task Manager or Task Manager to Guest
RPC API description
- Method name.
- Method parameters.
- Message type (cast/call).
- Does this change behavior on the Guest Agent? If so, is it backwards compatible with API and Task Manager?