Jump to: navigation, search

Difference between revisions of "Chef/Contributing"

m
 
(23 intermediate revisions by 4 users not shown)
Line 1: Line 1:
===Contributing to the OpenStack Chef Cookbooks===
+
==Contributing to the OpenStack Chef Cookbooks==
  
See [http://www.joinfu.com/2013/05/working-with-the-openstack-code-review-and-ci-system-chef-edition/ this blog entry] for an introduction to contributing and pushing code to for the OpenStack Chef cookbooks housed on Stackforge.
+
See [http://www.joinfu.com/2013/05/working-with-the-openstack-code-review-and-ci-system-chef-edition/ this blog entry] for an introduction to contributing and pushing code to for the OpenStack Chef cookbooks housed on OpenStack. And look here for more details: http://docs.openstack.org/infra/manual/developers.html
  
 +
How to create a new cookbook: https://wiki.openstack.org/wiki/Chef/Contributing/NewCookbook
  
===Guidelines for Commit Messages===
+
==Guidelines for Commit Messages==
  
 
Please try to make commit messages useful. Read [http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html this excellent blog post], as well as the [[GitCommitMessages|standard commit message guidelines for OpenStack projects]] before making your first code push.
 
Please try to make commit messages useful. Read [http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html this excellent blog post], as well as the [[GitCommitMessages|standard commit message guidelines for OpenStack projects]] before making your first code push.
  
===Guidelines for Blueprints===
+
The cookbooks projects and the cookbook testing repo fully support cross dependencies using the [http://docs.openstack.org/infra/manual/developers.html#cross-repository-dependencies Depends-On support]. This support basically comes from the Common cookbook bootstrap.sh file which uses the zuul-cloner to get the other cookbook dependencies cloned under the jenkins gate workspace: /home/jenkins/workspace/openstack-cookbook-xxxxx.  For the cookbooks, the bootstrap.sh automatically adjusts the Berksfile to look for these
 +
dependencies in that local path.  For the testing repo, the Berksfile already supports a flag, REPO_DEV=On, that enable it to search for cookbooks in a local path.
 +
 
 +
To use the Depends-On support, just add the gerrit Change-Id of the dependency patch to your commit message. Use the full Change-Id (‘I’ + 40 characters).  For example:
 +
 
 +
<pre><nowiki>
 +
  commit ae878fc8b9761d099a4145617e4a48cbeb390623
 +
  Author: [removed]
 +
  Date:  Fri Jun 26 01:44:02 2015 +0000
 +
 
 +
  The commit message short title line
 +
 
 +
  The description of the patch with all the details one could
 +
  ever hope for and maybe even words of wisdom on how
 +
  to actually use it.
 +
  * something on interest
 +
  * more stuff
 +
 
 +
  Blueprint my-very-important-freature
 +
 
 +
  Depends-on: I4e1cb4f1caaee61683dc9fb0ffc67e8f9e06dc65
 +
  for cookbook-common
 +
  Depends-on: I321cb4f1caaee61683dc9fb0ffc67e8f9e06dc11
 +
  for cookbook-identity
 +
 
 +
  Change-Id: I2072f93ad6c889d534b04009671147af653048e7
 +
</nowiki></pre>
 +
 
 +
==Guidelines for Blueprints==
 +
 
 
If you ever want to change something that can or should not be changed in just one commit or affects multiple cookbooks, please go ahead and [https://blueprints.launchpad.net/openstack/+addspec register a blueprint] for 'OpenStack + Chef'. After you read this wiki page on [[Blueprints|Blueprints]] you can basically go ahead and follow these steps for all openstack-cookbooks.
 
If you ever want to change something that can or should not be changed in just one commit or affects multiple cookbooks, please go ahead and [https://blueprints.launchpad.net/openstack/+addspec register a blueprint] for 'OpenStack + Chef'. After you read this wiki page on [[Blueprints|Blueprints]] you can basically go ahead and follow these steps for all openstack-cookbooks.
  
Line 27: Line 57:
  
 
cookbook-openstack-dashboard: TODO
 
cookbook-openstack-dashboard: TODO
 +
 +
cookbook-openstack-database: TODO
 +
 +
cookbook-openstack-data-processing: TODO
  
 
cookbook-openstack-identity: TODO
 
cookbook-openstack-identity: TODO
Line 48: Line 82:
 
openstack-chef-repo: TODO
 
openstack-chef-repo: TODO
  
===Guidelines for Code Reviewers===
+
==Guidelines for Code Reviewers==
  
 
Here are some simple rules for reviewers of code on the [https://review.openstack.org Gerrit Review site]:
 
Here are some simple rules for reviewers of code on the [https://review.openstack.org Gerrit Review site]:
Line 62: Line 96:
 
'''Rule #5''': If the patch '''''adds''''' additional functionality to a '''''library''''' cookbook, a corresponding bump in version number in the metadata.rb file should accompany the patch. Failure to do so should prevent a +1A by a core review.
 
'''Rule #5''': If the patch '''''adds''''' additional functionality to a '''''library''''' cookbook, a corresponding bump in version number in the metadata.rb file should accompany the patch. Failure to do so should prevent a +1A by a core review.
  
===Guidelines for Version Changes===
+
==Guidelines for Version Changes==
  
 
When submitting cookbook patches, it is generally required that the version
 
When submitting cookbook patches, it is generally required that the version
number (within metadata.rb) is incremented in a manor reflective of the level
+
number (within metadata.rb) is incremented in a manner reflective of the level
 
of the change.  Any patch should also update the CHANGELOG.md and if appropriate
 
of the change.  Any patch should also update the CHANGELOG.md and if appropriate
 
the README.md should reflect the changes and any relevant how-to instructions. The
 
the README.md should reflect the changes and any relevant how-to instructions. The
Line 76: Line 110:
 
version requirement in some simple cases to allow for increased productivity.
 
version requirement in some simple cases to allow for increased productivity.
  
====Semantic Versioning====
+
===Semantic Versioning===
  
The cookbooks use the Semantic Versioning system (see http://semver.org)
+
The cookbooks use the [http://semver.org Semantic Versioning system].
 
The system uses a three part version number, Major.Minor.Patch.
 
The system uses a three part version number, Major.Minor.Patch.
 
For example: 9.2.33
 
For example: 9.2.33
  
 
The Major number shouldn't change within a development branch. It will reflect the
 
The Major number shouldn't change within a development branch. It will reflect the
number that is the Alphabetized Letter of the base Openstack release,
+
number that is the Alphabetized Letter of the base,
see: https://wiki.openstack.org/wiki/Releases.  An example would be Icehouse
+
[https://wiki.openstack.org/wiki/Releases OpenStack release].  An example would be Icehouse
 
being the 9th letter and the 9th release all the stable cookbooks would be 9.Y.Z.
 
being the 9th letter and the 9th release all the stable cookbooks would be 9.Y.Z.
 
When the Master branch becomes stabilized, a new Stable branch will be created from
 
When the Master branch becomes stabilized, a new Stable branch will be created from
Line 92: Line 126:
 
sections below.
 
sections below.
  
====Stable Branch====
+
===Stable Branch===
 +
 
 +
Creating a stable branch for Openstack+Chef here are the [[OpenstackChefStablebranchCreateNotes|notes]].
 +
 
 +
See https://wiki.openstack.org/wiki/StableBranch#Gerrit for information about Cherry picking patches to stable branches. In most cases
 +
the Changelog and metadata.rb will also need to be updated with patch level increment.  Then after the patch is merged, please consider
 +
updating the https://github.com/openstack/openstack-chef-repo to pull in your change.  The Changelog, Berksfile and lock will need to be updated.
  
 
The Stable branch cookbooks must leverage the Semantic Versioning system exclusively.
 
The Stable branch cookbooks must leverage the Semantic Versioning system exclusively.
Line 124: Line 164:
 
what part of the version number to increment for Stable branches.
 
what part of the version number to increment for Stable branches.
  
=====Version Locking=====
+
====Version Locking====
  
 
The Stable branches are also locked down by added Berksfile.lock and Gemfile.lock to each
 
The Stable branches are also locked down by added Berksfile.lock and Gemfile.lock to each
Line 131: Line 171:
 
and metadata.rb files would need to be updated accordingly.
 
and metadata.rb files would need to be updated accordingly.
  
=====Cookbook Dependencies=====
+
====Cookbook Dependencies====
  
 
When a change requires hits to multiple cookbooks, like when adding attributes to Common, the
 
When a change requires hits to multiple cookbooks, like when adding attributes to Common, the
Line 137: Line 177:
 
Berksfile.lock would also need to be updated with the commit id of the dependent change.
 
Berksfile.lock would also need to be updated with the commit id of the dependent change.
  
====Master Branch====
+
===Master Branch===
  
 
The master cookbook should leverage the Semantic Versioning system lightly.
 
The master cookbook should leverage the Semantic Versioning system lightly.
Line 158: Line 198:
 
| change Gemfile or Berksfile || Increment Minor
 
| change Gemfile or Berksfile || Increment Minor
 
|-
 
|-
| add attribute for a value in a configuration file with the same default || Increment Patch**
+
| add attribute for a value in a configuration file with the same default || Increment Patch'''<big><big>**</big></big>'''
 
|-
 
|-
| changing a resource option || Increment Patch**
+
| changing a resource option || Increment Patch'''<big><big>**</big></big>'''
 
|-
 
|-
| add a test || Increment Patch**
+
| add a test || Increment Patch'''<big><big>**</big></big>'''
 
|-
 
|-
| fix a broken recipe || Increment Patch**
+
| fix a broken recipe || Increment Patch'''<big><big>**</big></big>'''
 
|-
 
|-
| re-factoring recipe or test || Increment Patch**
+
| re-factoring recipe or test || Increment Patch'''<big><big>**</big></big>'''
 
|}
 
|}
  
Line 172: Line 212:
 
what part of the version number to increment for Master branches.
 
what part of the version number to increment for Master branches.
  
(**) There are cases where incrementing the Patch number is not necessary and only updating the
+
'''<big><big>**</big></big>''' There are cases where incrementing the Patch number is not necessary and only updating the
 
CHANGELOG.md is required.  To avoid re-base collisions on Patch number changes and allow more
 
CHANGELOG.md is required.  To avoid re-base collisions on Patch number changes and allow more
 
rapid development, if the change falls within these guidelines, incrementing the Patch version
 
rapid development, if the change falls within these guidelines, incrementing the Patch version
Line 181: Line 221:
 
* When a patch only effect the README or other comments
 
* When a patch only effect the README or other comments
  
=====Version Locking=====
+
====Version Locking====
  
 
The Master branch is NOT locked down a Berksfile.lock or Gemfile.lock.  Changes to the Berksfile
 
The Master branch is NOT locked down a Berksfile.lock or Gemfile.lock.  Changes to the Berksfile
 
and Gemfile can be made directly.
 
and Gemfile can be made directly.
  
=====Cookbook Dependencies=====
+
====Cookbook Dependencies====
  
 
When a change requires hits to multiple cookbooks, like when adding attributes to Common, the
 
When a change requires hits to multiple cookbooks, like when adding attributes to Common, the
 
metadata.rb file would need to be updated to reflect that required version level.
 
metadata.rb file would need to be updated to reflect that required version level.
 +
 +
==Gate jobs==
 +
 +
The OpenStack Chef cookbooks use infra gate jobs to check and verify all patches.
 +
For the master the branch, the gate used is: gate-{name}-chef-rake, where name is the name of one of the cookbooks.
 +
This rake job uses a common/shared bootstrap script which is located in the cookbook-openstack-common project
 +
In the other cookbooks, you will note that their bootstrap scripts simple load and use the common one.
 +
 +
The bootstrap script will setup the development test environment by installing [https://downloads.chef.io/chef-dk/ ChefDK].  All Ruby gems required for the
 +
tests come from the ChefDK package, we don't use Gemfile or Bundler anymore.
 +
The rake gate job will simply execute the Rakefile located in each of the cookbooks.
 +
 +
The Rakefiles will by default run three tests with '''chef exec rake test'''
 +
* unit - RSpec
 +
* lint - FoodCritic
 +
* style - RuboCop
 +
 +
 +
See the TESTING.md file in each cookbook for more details on testing cookbook patches.
 +
 +
==Core Reviewers==
 +
 +
To become a core reviewer for the OpenStack Chef project you should:
 +
* show up at all/most of the IRC meetings
 +
* be available via IRC
 +
* have sufficient knowledge and experience in reviewing and contributing
 +
* ask to be a core member or be nominated and willing
 +
* acknowledge the ([https://wiki.openstack.org/wiki/Chef/Contributing#Guidelines_for_Code_Reviewers Guidelines])
 +
 +
 +
Finally you should put in a review to become a core member in the specs repo MAINTAINER file (https://github.com/openstack/openstack-chef-specs) and get the needed +2s from the other core members
 +
 +
back to [https://wiki.openstack.org/wiki/Chef/GettingStarted GettingStarted page]

Latest revision as of 09:47, 29 January 2018

Contributing to the OpenStack Chef Cookbooks

See this blog entry for an introduction to contributing and pushing code to for the OpenStack Chef cookbooks housed on OpenStack. And look here for more details: http://docs.openstack.org/infra/manual/developers.html

How to create a new cookbook: https://wiki.openstack.org/wiki/Chef/Contributing/NewCookbook

Guidelines for Commit Messages

Please try to make commit messages useful. Read this excellent blog post, as well as the standard commit message guidelines for OpenStack projects before making your first code push.

The cookbooks projects and the cookbook testing repo fully support cross dependencies using the Depends-On support. This support basically comes from the Common cookbook bootstrap.sh file which uses the zuul-cloner to get the other cookbook dependencies cloned under the jenkins gate workspace: /home/jenkins/workspace/openstack-cookbook-xxxxx. For the cookbooks, the bootstrap.sh automatically adjusts the Berksfile to look for these dependencies in that local path. For the testing repo, the Berksfile already supports a flag, REPO_DEV=On, that enable it to search for cookbooks in a local path.

To use the Depends-On support, just add the gerrit Change-Id of the dependency patch to your commit message. Use the full Change-Id (‘I’ + 40 characters). For example:

  commit ae878fc8b9761d099a4145617e4a48cbeb390623
  Author: [removed]
  Date:   Fri Jun 26 01:44:02 2015 +0000

  The commit message short title line

  The description of the patch with all the details one could
  ever hope for and maybe even words of wisdom on how
  to actually use it.
  * something on interest
  * more stuff
  
  Blueprint my-very-important-freature

  Depends-on: I4e1cb4f1caaee61683dc9fb0ffc67e8f9e06dc65
  for cookbook-common
  Depends-on: I321cb4f1caaee61683dc9fb0ffc67e8f9e06dc11
  for cookbook-identity

  Change-Id: I2072f93ad6c889d534b04009671147af653048e7

Guidelines for Blueprints

If you ever want to change something that can or should not be changed in just one commit or affects multiple cookbooks, please go ahead and register a blueprint for 'OpenStack + Chef'. After you read this wiki page on Blueprints you can basically go ahead and follow these steps for all openstack-cookbooks.

  1. Write down the basic idea in some words.
  2. Elaborate in some sentences on how you think your idea can be integrated in the cookbooks and why it is important.
  3. Create a list of 'Work Items', including all the cookbooks your idea should be applied to and the order you think would be best.
  4. Maybe check back with the people in the irc channel #openstack-chef to refine the things mentioned above.


If you think your idea concerns all the cookbooks, please go ahead and copy this complete list of openstack-cookbooks to avoid missing one during the process:


cookbook-openstack-block-storage: TODO

cookbook-openstack-common: TODO

cookbook-openstack-compute: TODO

cookbook-openstack-dashboard: TODO

cookbook-openstack-database: TODO

cookbook-openstack-data-processing: TODO

cookbook-openstack-identity: TODO

cookbook-openstack-image: TODO

cookbook-openstack-network: TODO

cookbook-openstack-object-storage: TODO

cookbook-openstack-ops-database: TODO

cookbook-openstack-ops-messaging: TODO

cookbook-openstack-orchestration: TODO

cookbook-openstack-telemetry: TODO

cookbook-openstack-integration-test: TODO

openstack-chef-repo: TODO

Guidelines for Code Reviewers

Here are some simple rules for reviewers of code on the Gerrit Review site:

Rule #1: Never +1/+2R or +1A your own patch.

Rule #2: All patches must have a commit message that meets the standard commit message guidelines for OpenStack projects. Failure of the commit message to meet these guidelines should prevent a +1A by a core reviewer.

Rule #3: If the patch is more than just stylistic or typo fixes, it requires at least 2 core reviewers to add a +2R to the review before any core reviewer can +1A the review.

Rule #4: If the patch changes existing behavior of any cookbook in a backwards-incompatible way, a corresponding bump in the version in the cookbook's metadata.rb must be included in patch set. Failure to do so should prevent a +1A by a core reviewer.

Rule #5: If the patch adds additional functionality to a library cookbook, a corresponding bump in version number in the metadata.rb file should accompany the patch. Failure to do so should prevent a +1A by a core review.

Guidelines for Version Changes

When submitting cookbook patches, it is generally required that the version number (within metadata.rb) is incremented in a manner reflective of the level of the change. Any patch should also update the CHANGELOG.md and if appropriate the README.md should reflect the changes and any relevant how-to instructions. The CHANGELOG.md is our executive summary of changes, it should inform what the change was in a quick manner.

There are some differences between the development of patches on the Master and Stable branches. There is more restrictive and vigorous oversight given to changes on the Stable branches. The Master branch is bleeding edge and can relax the version requirement in some simple cases to allow for increased productivity.

Semantic Versioning

The cookbooks use the Semantic Versioning system. The system uses a three part version number, Major.Minor.Patch. For example: 9.2.33

The Major number shouldn't change within a development branch. It will reflect the number that is the Alphabetized Letter of the base, OpenStack release. An example would be Icehouse being the 9th letter and the 9th release all the stable cookbooks would be 9.Y.Z. When the Master branch becomes stabilized, a new Stable branch will be created from it and the Major number in the Master will be incremented by the core team.

The Minor and Patch numbers will be incremented as described in the branch specific sections below.

Stable Branch

Creating a stable branch for Openstack+Chef here are the notes.

See https://wiki.openstack.org/wiki/StableBranch#Gerrit for information about Cherry picking patches to stable branches. In most cases the Changelog and metadata.rb will also need to be updated with patch level increment. Then after the patch is merged, please consider updating the https://github.com/openstack/openstack-chef-repo to pull in your change. The Changelog, Berksfile and lock will need to be updated.

The Stable branch cookbooks must leverage the Semantic Versioning system exclusively. All patches must update the metadata.rb and the CHANGELOG.md at a minimum. All patches should try to be backwards compatible.

Stable Branch Example Situations Level of Change
add a recipe Increment Minor
add a function or method Increment Minor
change Gemfile or Berksfile Increment Minor
backport a fix from Master branch Increment Minor
add attribute for a value in a configuration file with the same default Increment Patch
changing a resource option Increment Patch
add a test Increment Patch
fix a broken recipe Increment Patch
re-factoring recipe or test Increment Patch

The table above shows some examples of different levels of changes introduced by a patch and what part of the version number to increment for Stable branches.

Version Locking

The Stable branches are also locked down by added Berksfile.lock and Gemfile.lock to each cookbook. If any changes are made to the Gemfile, the Gemfile.lock would also have to be updated. If any changes are dependent upon other cookbook changes, then the Berkfile.lock and metadata.rb files would need to be updated accordingly.

Cookbook Dependencies

When a change requires hits to multiple cookbooks, like when adding attributes to Common, the metadata.rb file would need to be updated to reflect that required version level. The Berksfile.lock would also need to be updated with the commit id of the dependent change.

Master Branch

The master cookbook should leverage the Semantic Versioning system lightly. We consider the master branch a fast paced "Work in Progress" until we come to the Release Candidate 1 date (RC-1). This means it will be under rapid and active development where versioning isn't always required. All patches must update the the CHANGELOG.md at a minimum.


Master Branch Example Situations Level of Change
Branching for a new Stable branch Increment Major
add a recipe Increment Minor
add a function or method Increment Minor
change Gemfile or Berksfile Increment Minor
add attribute for a value in a configuration file with the same default Increment Patch**
changing a resource option Increment Patch**
add a test Increment Patch**
fix a broken recipe Increment Patch**
re-factoring recipe or test Increment Patch**

The table above shows some examples of different levels of changes introduced by a patch and what part of the version number to increment for Master branches.

** There are cases where incrementing the Patch number is not necessary and only updating the CHANGELOG.md is required. To avoid re-base collisions on Patch number changes and allow more rapid development, if the change falls within these guidelines, incrementing the Patch version would not be required.

  • When the change only effects a single cookbook
  • When the change is just a simple addition of a new attribute for a template and no other logic change is required
  • When a patch only effects the tests
  • When a patch only effect the README or other comments

Version Locking

The Master branch is NOT locked down a Berksfile.lock or Gemfile.lock. Changes to the Berksfile and Gemfile can be made directly.

Cookbook Dependencies

When a change requires hits to multiple cookbooks, like when adding attributes to Common, the metadata.rb file would need to be updated to reflect that required version level.

Gate jobs

The OpenStack Chef cookbooks use infra gate jobs to check and verify all patches. For the master the branch, the gate used is: gate-{name}-chef-rake, where name is the name of one of the cookbooks. This rake job uses a common/shared bootstrap script which is located in the cookbook-openstack-common project In the other cookbooks, you will note that their bootstrap scripts simple load and use the common one.

The bootstrap script will setup the development test environment by installing ChefDK. All Ruby gems required for the tests come from the ChefDK package, we don't use Gemfile or Bundler anymore. The rake gate job will simply execute the Rakefile located in each of the cookbooks.

The Rakefiles will by default run three tests with chef exec rake test

  • unit - RSpec
  • lint - FoodCritic
  • style - RuboCop


See the TESTING.md file in each cookbook for more details on testing cookbook patches.

Core Reviewers

To become a core reviewer for the OpenStack Chef project you should:

  • show up at all/most of the IRC meetings
  • be available via IRC
  • have sufficient knowledge and experience in reviewing and contributing
  • ask to be a core member or be nominated and willing
  • acknowledge the (Guidelines)


Finally you should put in a review to become a core member in the specs repo MAINTAINER file (https://github.com/openstack/openstack-chef-specs) and get the needed +2s from the other core members

back to GettingStarted page