Jump to: navigation, search

Difference between revisions of "Obsolete:GerritJenkinsGit"

m (Test Failures: Remove reference to jenkins as now we have 3 of them and no easy way to see which one is running your test.)
m (Smaffulli moved page GerritJenkinsGit to Obsolete:GerritJenkinsGit: old content)
 
(35 intermediate revisions by 15 users not shown)
Line 1: Line 1:
 
+
Please see the new Developer's Guide here: http://docs.openstack.org/infra/manual/developers.html
= Gerrit, Jenkins, and Git =
 
For a quick reference, please see [[GerritWorkflow]] instead.
 
 
 
We use [[http://git-scm.com/|Git]] for managing code repositories and interacting with other developers. [http://jenkins-ci.org/ Jenkins] is used to continuously test all of the components of [[OpenStack]] to ensure functionality and to verify that each change to the code base works as intended. [http://code.google.com/p/gerrit/ Gerrit] is a code review system originally developed for use by the Android Open Source Project and allows us to build a workflow where every change is peer-reviewed and tested by Jenkins before being merged into the main repository.
 
 
 
After making a change in their local Git repository, developers can easily push that change to Gerrit as a proposed change for the project.  Jenkins will automatically run functional tests on the code and provide feedback on the change in Gerrit.  Any [[OpenStack]] developer can provide feedback (in the form of a comment, or even line-by-line annotations) using Gerrit, and the core developers of the project can indicate whether they approve of the patch as is, or would like to see changes before it is integrated.  Once patches are merged by Gerrit, the repository is mirrored to the canonical public repository at [https://git.openstack.org/cgit/ git.openstack.org] and to GitHub.
 
 
 
== Using Gerrit ==
 
The next sections describe how Gerrit fits into the developer workflow.
 
 
 
=== Gerrit Accounts ===
 
Gerrit lives at https://review.openstack.org/. Visit and click the '''Sign In''' link at the top-right corner of the page. Log in with your Launchpad ID.
 
 
 
Because Gerrit uses Launchpad OpenID single sign-on, you won't need a separate password for Gerrit, and once you log in to one of Launchpad,  Gerrit, or Jenkins, you won't have to enter your password for the others.
 
 
 
Gerrit accounts are automatically synchronized with Launchpad, so your Gerrit account should already have the same username, full name,  email address, ssh keys, and group membership.
 
 
 
Some information in Launchpad is not publicly available and so may not be copied over.  The first time you log into Gerrit, you should click the '''Settings''' link at the top of the page, and then make sure that your '''Contact Information''', '''SSH Public Keys''', and '''Groups''' look correct.  If not, please register your email address and SSH keys.  If your group membership is not correct, please email openstack-ci-admins@lists.launchpad.net .
 
 
 
Ensure that you have run these steps to let git know about your email address:
 
 
 
<pre><nowiki>
 
git config --global user.name "Firstname Lastname"
 
git config --global user.email "your_email@youremail.com"
 
</nowiki></pre>
 
 
 
=== Setting up Git for Use with Gerrit ===
 
For a more comprehensive look at using Gerrit, see [https://review.openstack.org/Documentation/user-upload.html the Gerrit manual].
 
 
 
==== Change-Id Hook ====
 
Gerrit uses a '''Change-Id''' footer in commits so that it can link Git commits to changes stored in its database.  When you upload a revised change (to correct a problem or respond to code review comments), Gerrit will use the Change-Id footer to attach the commit as a new patchset on the existing gerrit change.  This works best if the Change-Id is already in the original commit message, before it is even sent to Gerrit.
 
 
 
"git review" installs a commit hook into your repository that automatically adds Change-Id lines to your commits..
 
 
 
The Gerrit manual goes into more detail about [https://review.openstack.org/Documentation/user-changeid.html change IDs].
 
 
 
Install the git-review tool, which is the tool [[OpenStack]] uses to simplify submission of reviews to Gerrit.
 
 
 
<pre><nowiki>
 
sudo pip install git-review
 
</nowiki></pre>
 
 
 
==== Pushing Changes from Git ====
 
Simply running '''git review''' should be sufficient to push your changes to Gerrit, assuming your repository is set up as described above, you don't need to read the rest of this section unless you want to use an alternate workflow.
 
 
 
If you want to push your changes without using git-review, you can push changes to gerrit like you would any other git repository, using the following syntax (assuming "gerrit" is configured as a remote repository):
 
 
 
<pre><nowiki>
 
git push gerrit HEAD:refs/for/$BRANCH[/$TOPIC]
 
</nowiki></pre>
 
 
 
Where $BRANCH is the name of the Gerrit branch to push to (usually "master"), and you may optionally specify a Gerrit topic by appending it after a slash character.
 
 
 
==== Git SSH Commands ====
 
If you find you are frequently executing Gerrit commands via SSH, you may wish to add something like the following to your '''~/.ssh/config''' file:
 
 
 
<pre><nowiki>
 
Host review
 
  Hostname review.openstack.org
 
  Port 29418
 
  User USERNAME
 
</nowiki></pre>
 
 
 
Which may shorten some SSH commands; the following are equivalent:
 
 
 
<pre><nowiki>
 
ssh -p 29418 review.openstack.org gerrit ls-projects
 
ssh review gerrit ls-projects
 
</nowiki></pre>
 
 
 
== Reviewing a Change ==
 
Log in to https://review.openstack.org/ to see proposed changes, and review them.
 
 
 
To provide a review for a proposed change in the Gerrit UI, click on the Review  button (it will be next to the buttons that will provide unified or side-by-side  diffs in the browser). In the code review, you can add a message, as well as a  vote (+1,0,-1).
 
 
 
Any Openstack developer may propose or comment on a change (including voting +1/0/-1 on it).  Openstack projects have a policy requiring two positive reviews from core reviewers.  A vote of +2 is allowed from core reviewers, and should be used to indicate that they are a core reviewer and are leaving a vote that should be counted as such.
 
 
 
When a review has two +2 reviews and one of the core team believes it is ready to be merged, he or she should leave a +1 vote in the "Approved" category.  You may do so by clicking the "Review" button again, with or without changing your code review vote and optionally leaving a comment.  When a +1 Approved review is received, Jenkins will run tests on the change, and if they pass, it will be merged.
 
 
 
=== Gerrit Best Practices ===
 
If you are working on unrelated changes, you should use a [http://progit.org/book/ch3-4.html topic branch] so that there  isn't a dependency between the changes.
 
 
 
When you start working on a new change, make sure you have the current repository head from git.openstack.org (or GitHub).
 
 
 
For more information about uploading changes to gerrit, see the [https://review.openstack.org/Documentation/user-upload.html Uploading Changes] section of the Gerrit manual.
 
 
 
=== Gerrit Errors ===
 
==== missing Change-Id in commit message ====
 
If you see an error like this:
 
 
 
<pre><nowiki>
 
! [remote rejected] HEAD -> refs/for/master (missing Change-Id in commit message)
 
</nowiki></pre>
 
 
 
Make sure that you have the [[#Change-Id_Hook|Change-Id hook]] installed.  If you don't, install it now, and the run '''git commit --amend''' and re-save your commit message.  The hook will then add a Change-Id line.
 
 
 
If you did have the hook installed, there may be a syntax error with the Change-Id line.  It must be in the last paragraph of the commit message, and it must be at the beginning of the line.  Your commit message should look like this in your editor:
 
 
 
<pre><nowiki>
 
The text of your commit message is here.
 
 
 
Change-Id: I5f55e68d1bdb42a0fa6f0b1a5432786d0395da51
 
</nowiki></pre>
 
 
 
==== squash commits first ====
 
If you see this message:
 
 
 
<pre><nowiki>
 
! [remote rejected] HEAD -> refs/for/master (squash commits first)
 
</nowiki></pre>
 
 
 
It means that you are trying to update an existing change in Gerrit, but you created two separate commits.  Normally to update a change you should amend an existing commit (see [[GerritWorkflow#Updating_a_Change|Updating a Change]]).  If you have already made a second commit, you will need squash the last two commits in your tree.  To do that, run:
 
 
 
<pre><nowiki>
 
git rebase -i HEAD~2
 
</nowiki></pre>
 
 
 
Your editor should appear with two commits listed, one per line. Change the word "pick" on the second line to "squash", so that it looks like:
 
 
 
<pre><nowiki>
 
 
 
pick 1458567 Fix bug 1014727
 
pick 7284e97 Document underlying technologies
 
pick fac4698 Clean up on section
 
pick xxxxxxx 2nd commit back
 
squash yyyyyyy head
 
 
 
</nowiki></pre>
 
 
 
And save.  You should then be able to upload your commit with '''git review'''.
 
 
 
==== can not create new references ====
 
 
 
If you try to send a draft with '''git review -D''' and it fails like this:
 
 
 
<pre><nowiki>
 
! [remote rejected] HEAD -> refs/draft/master (can not create new references)
 
</nowiki></pre>
 
 
 
Then check that you are running git-review version 1.16 or later. You can install the lastest version with pip:
 
 
 
<pre><nowiki>
 
pip install git-review
 
</nowiki></pre>
 
 
 
=== Gerrit Merge Problems ===
 
Gerrit will fast-forward or merge changes as necessary when they are approved.  If a conflict would be created by a merge, gerrit will not merge the change and will record an error message in the comments for the change.  In these cases, you may need to [http://www.kernel.org/pub/software/scm/git/docs/git-rebase.html rebase] or [http://www.kernel.org/pub/software/scm/git/docs/git-merge.html merge] the change, or if the repository head has changed significantly, you may need to change the patch.
 
 
 
If you don't already have the patch in your local repository, Gerrit provides commands on the web page for each change indicating how to download that change.  You can then use git to correct the problem.
 
 
 
If you encounter other error messages from Gerrit, the  [https://review.openstack.org/Documentation/error-messages.html Error Messages]  section of the Gerrit manual may offer some tips.
 
 
 
=== Test Failures ===
 
When a new patchset is uploaded to Gerrit, that project's "check" tests are run on the patchset by Jenkins. Once completed the test results are reported to Gerrit by Jenkins in the form of a Verified: +/-1 vote. After code reviews have been completed and a change receives an Approved: +1 vote that project's "gate" tests are run on the change by Jenkins. Jenkins reports the results of these tests back to Gerrit in the form of a Verified: +/-2 vote. Code merging will only occur after the gate tests have passed successfully and received a Verified: +2. You can view the state of tests currently being run on the [http://status.openstack.org/zuul/ Zuul Status].
 
 
 
If a change fails tests in Jenkins, please follow the steps below:
 
 
 
# Jenkins leaves a comment in the review with links to the log files for the test run.  Follow those links and examine the output from the test.  It will include a console log, and in the case of unit tests, HTML output from the test runner, or in the case of a devstack-gate test, it may contain quite a large number of system logs.
 
# Examine the console log or other relevant log files to determine the cause of the error.  If it is related to your change, you should fix the problem and upload a new patchset.
 
# If the problem is due to non-deterministic behavior already merged, and is unrelated to your change, you should do the following to help other developers who may be affected by the same issue, and to focus attention of QA, CI, and other developers working to fix high-impact bugs and improve test systems:
 
# Visit http://status.openstack.org/rechecks/ to see if one of the bugs listed there matches the error you've seen.  If your error isn't there, then:
 
#* Identify which project(s) are affected, and search for a related bug on [https://bugs.launchpad.net/ Launchpad].  If you do not find an existing bug, file a new one (and be sure to include the error message).  If the problem is due to an infrastructure problem (such as Jenkins, Gerrit, etc.), file (or search for) the bug against the "openstack-ci" project.
 
# Once you have a bug number in hand:
 
#* To re-run the check jobs (before a change has been approved), leave a comment with the form "recheck bug ####".
 
#* To re-run the gate jobs (after a change has been approved), leave a comment with the form "reverify bug ####".
 
# This will update the [http://status.openstack.org/rechecks/ rechecks status page] and help others quickly identify the issue if it occurs again.
 
 
 
If you need to re-run tests and it does not make sense to include a bug number (perhaps there is no error, you've just made a draft change visible and want it tested or you're updating test results because you know that a related branch has changed since the last time they were run), you may leave a comment with the form "recheck no bug" (or "reverify no bug").  Please only do this if you are certain there is no bug that needs to be addressed.
 
 
 
==== Test failures due to changes in the code ====
 
 
 
There might be some cases in which the submitted patch changes the way a tempest test behaves. In this case, the tempest test needs to be modified, so that it tests the new behaviour instead of the old one. The route for this is propose the tempest change, and get core contributors from the core project to +1 it, saying that they agree with the change. Once the tempest change has been accepted and merged, the original change could be tested again.
 
 
 
=== Merge Commits ===
 
When a branch is pushed to gerrit (either manually, or via git-review), each commit in the branch that is not in Gerrit's copy of the tree is turned into a new change for review (unless that commit has a Change-Id that corresponds to an existing change, in which case, the change is updated).  This means that if a developer merges a branch from another source into their local repository and pushes to Gerrit, hundreds of new changes may be created.  This is almost certainly not what was intended.  For that reason, Gerrit is configured not to accept merge commits in the general case.
 
 
 
'''Caution: The following describes an experimental process and is not generally applicable yet:'''
 
 
 
However, merge commits are very useful for feature branches -- development branches that fork from the main line of development, occasionally receiving updates from the main line, and when development is complete, merging back into the main line.  For these purposes, members of the $PROJECT-milestone group are permitted to upload merge commits to Gerrit.  These users must take special care when dealing with merge commits to avoid the problems mentioned above.  The following tips will help avoid problems with merge commits:
 
 
 
* Don't merge branches from gerrit and outside of gerrit (eg, GitHub or personal branches).
 
* When developing, follow the branching model described in [[GerritWorkflow]], this will avoid creating merge commits locally.
 
* Use git-review to upload your changes to Gerrit.  By default, if git-review would create or update more than one commit, it will list the commits and ask you if that is what you intend to do.  Pay attention to that, and if you see that too many commits are being uploaded, say "no" and clean up your local branch.
 
* To merge master into a feature branch, do the following:
 
 
 
<pre><nowiki>
 
git checkout feature-branch
 
git pull --ff-only origin feature-branch
 
git checkout -b merge-branch
 
git merge origin/master
 
# Amend the merge commit to automatically add a Change-ID to the commit message:
 
GIT_EDITOR=/bin/true git commit --amend
 
git review -R feature-branch
 
git checkout master
 
git branch -D merge-branch
 
</nowiki></pre>
 
 
 
Note that follows the same model described in [[GerritWorkflow]] of using a topic branch for each independent unit of local development.  In this case, the "merge-branch" branch is being used to generate the merge commit, and once submitted, is no longer needed and can be (optionally) removed.  You may continue to use your local copy of the feature-branch as normal, pulling fast-forward only commits from Gerrit.  The following similar process may be used to merge the feature branch into master, terminating development on that branch:
 
 
 
<pre><nowiki>
 
git checkout master
 
git pull --ff-only origin master
 
git checkout -b merge-branch
 
git merge origin/feature-branch
 
# Amend the merge commit to automatically add a Change-ID to the commit message:
 
GIT_EDITOR=/bin/true git commit --amend
 
git review -R
 
git checkout master
 
git branch -D merge-branch
 
</nowiki></pre>
 
 
 
= Release Management =
 
== Milestones ==
 
Between the Milestone Branch Point and the release of the corresponding milestone, there needs to be a separate branch in Gerrit for changes destined for the milestone release.  Meanwhile, development on the master branch should continue as normal (with the addition that changes proposed for the milestone should also be proposed for master, and some changes for master may need to be applied to milestone-proposed).
 
 
 
This process creates an ephemeral milestone-proposed branch that is only available in Gerrit during the milestone process.  When the milestone is released, a tag is applied to the final commit to record the state of the branch at the time.
 
 
 
=== Create milestone-proposed Branch ===
 
This step should be performed by the [[OpenStack]] Release Manager at the Release Branch Point.
 
 
 
* Go to https://review.openstack.org/ and sign in
 
* Select '''Admin''', '''Projects''', then the project
 
* Select '''Branches'''
 
* Enter '''milestone-proposed''' in the '''Branch Name''' field, and '''HEAD''' as the Initial Revision, then press '''Create Branch'''
 
 
 
In your local checkout:
 
 
 
<pre><nowiki>
 
git checkout master
 
git pull
 
git checkout milestone-proposed
 
</nowiki></pre>
 
 
 
=== Authoring Changes for milestone-proposed ===
 
Create topic branches as normal, but branch them from milestone-proposed rather than master.
 
 
 
<pre><nowiki>
 
git checkout milestone-proposed
 
git pull
 
git checkout -b MY-TOPIC-BRANCH
 
</nowiki></pre>
 
 
 
Changes for milestone-proposed should be submitted with:
 
 
 
<pre><nowiki>
 
git push gerrit HEAD:refs/for/milestone-proposed
 
</nowiki></pre>
 
 
 
=== Submit Changes in master to milestone-proposed ===
 
If a change to master should also be included in milestone-proposed, use this procedure to cherry-pick that change and submit it for review.
 
 
 
<pre><nowiki>
 
git checkout milestone-proposed
 
git pull
 
git checkout -b master-to-mp
 
git cherry-pick -x <SHA1 or "master">
 
git push gerrit HEAD:refs/for/milestone-proposed
 
git branch -D master-to-mp
 
</nowiki></pre>
 
 
 
'''git cherry-pick master''' will pick the most recent commit from master to apply, if you want a different patch, use the SHA1 of the commit instead.
 
 
 
The '-x' flag will ensure the commit message records the SHA1 hash of the original commit in master.
 
 
 
If there are conflicts when cherry-picking, do not delete the 'Conflicts' lines GIT adds to the commit message. These are valuable to reviewers to identify files which need extra attention.
 
 
 
=== Submitting Changes in milestone-proposed to master ===
 
Changes that originate in milestone-proposed should also be submitted to master.  Use these commands to make an up-to-date topic branch from master, then cherry-pick changes from milestone-proposed to be applied to master.
 
 
 
<pre><nowiki>
 
git checkout master
 
git pull
 
git checkout -b mp-to-master
 
git cherry-pick <SHA1 or milestone-proposed>
 
git review
 
git branch -D mp-to-master
 
</nowiki></pre>
 
 
 
'''git cherry-pick milestone-proposed''' will pick the most recent commit from milestone-proposed to apply, if you want a different patch, use the SHA1 of the commit instead.
 
 
 
=== Tagging a Release ===
 
This step should be performed by the [[OpenStack]] Release Manager when the release is made.
 
 
 
To tag the tip of the milestone-proposed branch with a release tag and push that tag to gerrit, git.openstack.org and GitHub, run the following commands:
 
 
 
<pre><nowiki>
 
git checkout milestone-proposed
 
git pull
 
git tag -s RELEASE-TAG-NAME
 
git push gerrit RELEASE-TAG-NAME
 
</nowiki></pre>
 
 
 
Running `git tag -s` signs the tag using GPG, so it's important to ensure that the person making the release have a valid GPG key.
 
 
 
Make sure you're only adding a single tag when pushing to gerrit.
 
 
 
=== End of Milestone ===
 
This step should be performed by the [[OpenStack]] Release Manager after the release is tagged.
 
 
 
When the milestone process is complete and the released commit is tagged, remove the milestone-proposed branch.  The tag will persist, even after the branch is deleted, making it possible to restore the state of the tree.
 
 
 
* Go to https://review.openstack.org/ and sign in
 
* Select '''Admin''', '''Projects''', then the project
 
* Select '''Branches'''
 
* Select the checkbox next to '''milestone-proposed''' and hit '''Delete'''
 
 
 
= Resources =
 
See the [https://review.openstack.org/Documentation/index.html Gerrit documentation],  especially the User Guide, for more information on how to use Gerrit.  It is also available within Gerrit by clicking on the '''Documentation''' link on the top of the page.
 
 
 
The Mahara Project also [https://wiki.mahara.org/index.php/Developer_Area/Developer_Tools uses Git, Gerrit, and Jenkins] in a similar manner.
 
 
 
A description of many of the elements of the [http://sandofsky.com/blog/git-workflow.html git workflow]
 

Latest revision as of 17:58, 31 March 2015

Please see the new Developer's Guide here: http://docs.openstack.org/infra/manual/developers.html