Jump to: navigation, search


Revision as of 13:54, 28 March 2013 by Dripton (talk | contribs)

Nova has historically used SQLAlchemy-migrate for database migrations. We should consider switching to Alembic.

A problem with SQLAlchemy-migrate is that migration scripts have a fixed order, with a version number embedded at the beginning of each script's filename. For example, 158_add_networks_uc.py is hardcoded to run before 159_revert_ip_column_length.py SQLAlchemy-migrate stores the current version number in the migrate_version table of the database, and requires that migration scripts use consecutive version numbers with none missing.

This is fine for straight-line development, but a problem when selectively backporting only a few features or bugfixes to a stable branch. One option would be to renumber scripts in the stable branch, but that's error-prone. Another option, which we have gone with so far, is creating a range of do-nothing placeholder scripts (currently versions 162-171 for backporting from Havana to Grizzly), and then filling in the content of those scripts with whatever we need to backport later.

Alembic has a better design for backports. Each Alembic migration script gets a uuid rather than a fixed integer version number. And then Alembic has the concept of branches, which are ordered chains of these uuids. So it's possible to have a "master" branch with all the migrations in their original order, and then a "stable" branch with only a subset of the migrations. The design of Alembic mirrors the design of Git here, and so is a better match to Nova's development model.

Of course, neither tool can perform miracles. To support backports, we need to write good migration scripts that are as robust as possible and make as few assumptions as possible. If scripts only work if all of the preceding scripts have already run, then they'll fail when backported to an environment where that assumption does not hold, regardless of how nice the backporting scheme is.

Switching tools would involve some pain. At the beginning of the Havana cycle, Nova has 39 migration scripts, of which 10 are placeholders. If we switch to Alembic, the 29 non-trivial scripts would need to be converted to Alembic syntax. There's an autoconversion tool at https://review.openstack.org/#/c/15196/, but it only does most of the work, not all. If we made the change later in the Havana cycle, there would presumably be other changes to migration scripts in flight, and we'd have to convert those as well. So if we make the switch, we should do it as soon as possibly, and as atomically as possible, and with loud advance warning to all Nova developers so they know about the disruption to their own migration scripts.