Revision as of 16:57, 27 October 2011

Stable Branch

At the Essex Design Summit it was agreed that a working group would be established to maintain stable/backport/fixes branches for the Diablo release.

This effort is essentially an experiment to establish:

• That the idea makes sense
• That there is enough people willing to help maintain it for it to be viable
• The policies for what belongs in such a stable branch
• How changes should be proposed to the branch
• The workflow for maintaining the branch
• Whether updates should be released from the branch

This page tries to cover most of these topics and should be considered an RFC.

<<TableOfContents()>>

Overview

Each project will have a branch named after the previous release. For example, the stable branch for the diablo release will be called 'stable/diablo'.

The stable branch will only be maintained until the next release is out. This period may be extended if there are volunteers to maintain it beyond this point.

The maintainers, in conjunction with the release team, may choose to cut a new release from the branch if there are enough worthwhile fixes and the branch has been tested enough.

The branch maintainer(s) will monitor master for changes and periodically cherry-pick patches into the branch. They may choose to stop actively monitoring master after a period (e.g. 2 months) if they feel there are so few backport candidates that it is no longer worth the effort. If so, they will notify developers via the mailing list.

All changes to the stable branch will go through gerrit. The branch maintainers and core teams of each project can +2 the changes.

Every change on the stable branch must pass all the tests.

Security patches will follow a different process, whereby the patch can be embargoed and pushed at the same time as the patch on master, once the issue has been made public.

In Essex, the RBP will be when the RCs are considered "good enough". It is at that point the stable maintainers need to start monitoring master. For Diablo, the PTLs and release manager did this work. That implies that the PTLs and RM will initially take an active interest in the stable branch, but that interest will want after a time. Which makes perfect sense.

Policy

The most obvious question we need to answer is "What belongs in a stable branch?"

However, we don't need to ignore prior art here. Plenty of projects maintain such a branch and there's a very rough consensus across projects on what the policy should be.

As an example, stable_kernel_rules.txt says:

• It must be obviously correct and tested.
• It cannot be bigger than 100 lines, with context.
• It must fix only one thing.
• It must fix a real bug that bothers people (not a, "This could be a problem..." type thing).
• It must fix a problem that causes a build error [..], an oops, a hang, data corruption, a real security issue, or some "oh, that's not good" issue. In short, something critical.
• No "theoretical race condition" issues, unless an explanation of how the race can be exploited is also provided.
• It cannot contain any "trivial" fixes in it (spelling changes, whitespace cleanups, etc).
• It or an equivalent fix must [...] (upstream).

We can also add some rules specific to OpenStack:

• No changes to the external HTTP APIs
• No changes to the internal AMQP API
• No DB schema changes

The branch is intended to be a safe source of fixes for significant issues which have been fixed on master since the release. New features, regressions, compatibility issues, subtle behavioural changes, etc. have no place here.

The maintainers need to balance the risk of any given patch with the value that it will provide to users of the stable branch. A large, risky patch for a major data corruption issue might make sense. As might a trivial fix for a fairly obscure error handling case.

Workflow

Proposing Fixes

Anyone can propose a cherry-pick to the maintainers. This helps ensure that the maintainers don't miss anything. To catch the maintainers attention, simply add

Cc: stable-maintainers

to the commit message. If the patch you're proposing will not cherry-pick cleanly, you can help by resolving the conflicts yourself and proposing the resulting patch.

You can use git-review to propose a change to the stable branch with:

git review stable/diablo

If a bug in launchpad looks like a good candidate for backporting - e.g. if it's a significant bug with the previous release - then just tagging it with 'diablo-backport' will bring it to the attention of the maintainers e.g. for nova and for glance.

Email Notifications

If you want to be notified of these patches you can create a watch on this screen: https://review.openstack.org/#settings,projects with the settings:

Project Name: All-Projects
     Only If: message:"stable-maintainers"

And then check the "Email Notifications - New Changes" checkbox. That will cause gerrit to send an email whenever a matching change is proposed, and better yet, the change shows up in your 'watched changes' list in gerrit.

See the docs for Watched Projects configuration and the search syntax.

Generating New Change-Ids

When cherry-picking a commit, in order to ensure a new Change-Id, you need to do:

 $> git cherry-pick -e -x $commit
 <remove the Change-Id tag>
 $> git commit --amend
 <this will add a new Change-Id>

Alternatively, you can skip the latter step and use 'git rebase -i' and the 'reword' action to add Change-Ids later.

We could probably fix this in the commit-msg hook ... perhaps by recognizing when its a cherry-pick. Or maybe there's a hook specific to cherry-picking.

Stale .pyc Files From Migrations

Switching between master and diablo is a pain because you end up with .pyc files for migrations that exist only on master and diablo blows up when trying to run them. Delete them with:

 $> rm -f nova/db/sqlalchemy/migrate_repo/versions/*.pyc

Reviewing Changes On Master

Try e.g.

 $> git log --graph 2011.3..origin/master

If you're looking at a sequence of commits on a branch which got merged into master and you want to see exactly what changes were merged in, then look at the merge commit:

 commit f5f429bac6446db084ca3f5d86c64127a1e539f2
 Merge: e95e923 f3fb16a

and do:

 $> git diff e95e923..f5f429b

Keeping Notes On Your Reviews

When reviewing the master branch for candidates to backport, you probably want to keep notes on which commits do and don't look good for cherry-picking. Even if it's just a case of keeping track of which was the latest commit on master you reviewed. You may also be curious about another maintainers opinion on a given patch.

git notes allows you to maintain notes on commits in the same git repository as the commits themselves. It's exactly what's needed here, but it can be a bit of a pain to use.

To fetch my notes on nova, do:

 $> git remote add markmc-notes git://github.com/markmc/nova.git
 $> git config remote.markmc-notes.fetch +refs/notes/diablo-notes:refs/notes/markmc/diablo-notes
 $> git fetch markmc-notes

You now have my notes in ref/notes/markmc/diablo-notes. To look at them, do either:

 $> git log --notes=markmc/diablo-notes

Or:

 $> export GIT_NOTES_DISPLAY_REF=refs/notes/markmc/diablo-notes
 $> git log

You'll see something like:

commit 1b7fba648aa3eb4cdda345237c9f77dc0b229329
Author: Naveed Massjouni [...]
Date:   Wed Oct 26 14:32:48 2011 -0400

    Adding support for retrying glance image downloads.
    
    Change-Id: Ifff40d90f7dc61a6d41ae2d6908d6e1e6f0aea7e

Notes (markmc/diablo-notes):
    New feature.
    
    Action: pass

(The values of this Action tag I'm using are: pass = "Don't cherry-pick", pick = "Do cherry-pick", proposed = "Already cherry-picked and proposed in gerrit", merged = "Already cherry-picked and merged into stable" and defer = "Consider cherry-picking later")

If you want to start maintaining your own notes, you can do e.g.

 $> export GIT_NOTES_REF=refs/notes/diablo-notes
 $> git notes add 261b4111d4
 $> git notes edit 261b4111d4
 $> git notes show 261b4111d4
 $> git show 261b4111d4

and to push them to a remote repo:

 $> git push ${myrepo} refs/notes/diablo-notes:refs/notes/diablo-notes