Jump to: navigation, search


Revision as of 13:45, 28 March 2013 by Dripton (talk | contribs) (proposing switching from SQLAlchemy-migrate to Alembic for DB migrations)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

Nova has historically used SQLAlchemy-migrate for database migrations. We should consider switching to Alembic.

A problem with SQLAlchemy-migrate is that migration scripts have a fixed order, with a version number embedded at the beginning of each script's filename. For example, 158_add_networks_uc.py is hardcoded to run before 159_revert_ip_column_length.py SQLAlchemy-migrate stores the current version number in the migrate_version table of the database, and requires that migration scripts use consecutive version numbers with none missing.

This is fine for straight-line development, but a problem when selectively backporting only a few features or bugfixes to a stable branch. One option would be to renumber scripts in the stable branch, but that's error-prone. Another option, which we have gone with so far, is creating a range of do-nothing placeholder scripts (currently versions 162-171 for backporting from Havana to Grizzly), and then filling in the content of those scripts with whatever we need to backport later.

Alembic has a better design for backports. Each Alembic migration script gets a uuid rather than a fixed integer version number. And then Alembic has the concept of branches, which are ordered chains of these uuids. So it's possible to have a "master" branch with all the migrations in their original order, and then a "stable" branch with only a subset of the migrations. The design of Alembic mirrors the design of Git here, and so is a better match to Nova's development model.

Of course, neither tool can perform miracles. To support backports, we need to write good migration scripts that are as robust as possible and make as few assumptions as possible. If scripts only work if all of the preceding scripts have already run, then they'll fail when backported to an environment where that assumption does not hold, regardless of how nice the backporting scheme is.