Revision as of 03:11, 17 June 2012

<<TableOfContents()>>

How To Work on OpenStack Documentation

At the core of OpenStack is the community and collaboration that we do - the same rules for the code apply to documentation too. Ideally any code contribution that is merged into the base has documentation to go with it. Anne Gentle is the coordinator for all documentation efforts, both community-based and "official" docs. This page describes the methods we use to create the basis for world-class documentation for OpenStack developers and users.

First-time Contributors

First, thanks for contributing to docs! Secondly, you're just like a developer. Here are the overall steps to get started.

1. Sign the CLA. See HowToContribute for details on that. Once the CLA is signed and you have an Echosign number, update the Contributors page to add your name and Echosign number.

2. Create a Launchpad account and add SSH keys to your account profile.

3. Join the openstack-cla Launchpad group. To do this, navigate to https://launchpad.net/~openstack-cla and click 'Join the team' on the right side of the page. Your request will be forwarded to the core OpenStack developers, who can then approve your request. Keep in mind, you must have completed the first two steps for your membership to be approved. It can take 15 minutes after approval to sync with the systems.

4. Create a Github account and add SSH keys to your account profile (same one as used in Launchpad, they must match). If this is your first time setting up git and Github, be sure to run these steps in a Terminal window:

$ git config --global user.name "Firstname Lastname"
$ git config --global user.email "your_email@youremail.com"

5. Install git-review. If pip is not already installed, use "sudo easy_install pip" to install it on a Mac.

sudo pip install git-review

6. Enter your Launchpad account information after testing for ssh key setup with git review -s.

git review -s

Note: If you see any "Permission denied" errors at this step, go to review.openstack.org, log in, and check through all the items in your Settings link to ensure that everything matches. You may need to enter your SSH key in the Gerrit site (review.openstack.org).

If you get the error "We don't know where your gerrit is.", you will need to add a new git remote. The url should be in the error message. Copy that and create the new remote.

git remote add gerrit ssh://<username>@review.openstack.org:29418/openstack/openstack-manuals.git

7. Now you can clone the openstack-manuals project and work on it locally. Refer to the GerritWorkflow steps to contribute your changes.

git clone git://github.com/openstack/openstack-manuals.git

That should get you started contributing to docs just like code. Read more for the details of what types of documentation go where.

Tools Overview

Currently, we use this wiki, RST files stored in source code, (ReStructured Text, it has a similar text-based markup as wikitext) and DocBook to create OpenStack's web documentation.

The wiki is easy-peasy, you just request an account on wiki.openstack.org using your Launchpad user ID, create a new page, and start editing.

Editing RST or DocBook has a few more steps and depends you getting the code for the project. To work on the DocBook source files, a working project exists in Github in the http://github.com/openstack/openstack-manuals project since DocBook documentation encompasses all three OpenStack projects. All API documentation has been moved to Github, and is housed in image-api, identity-api, compute-api, and object-api projects there. For Github hosted docs, you get a Launchpad account plus a Github account and use git for checking in and out docs.

If you're on a Mac, here are the quick steps for getting started on the openstack-manuals project:

1. First install the Apache Maven project.

With Macports:

sudo port install maven3

With Homebrew:

brew install maven

on Ubuntu:

sudo apt-get install maven

on Fedora Core:

sudo yum install maven3

2. Install Git by referring to http://help.github.com/mac-set-up-git/.

3. Build the docs.

git clone https://github.com/openstack/compute-api.git
cd compute-api/openstack-compute-api-2
mvn clean generate-sources

Note: for admin manuals, we have now added properties to pass in to the pom file prior to building. So for an example, to build the Essex Compute Admin manual, you would use:

mvn -Drelease.path.name=essex -Dcomments.enabled=1 clean generate-sources

As long as you see BUILD SUCCESSFUL, your docs have built.

You may see these image loading errors when using Maven 2, but the build is still successful and the image is correctly placed:

Failed to load image:
/home/anne.gentle/openstack-manuals/doc/src/docbkx/openstack-compute-admin/../common/images/cc/by-sa.png

The PDF version will be generated at:

compute-api/openstack-compute-api-2/target/docbkx/webhelp/api/openstack-compute/2/os-compute-devguide-2.pdf

The root of the HTML version will be generated at:

compute-api/openstack-compute-api-2/target/docbkx/webhelp/api/openstack-compute/2/content/index.html

The RST source is built into html using Sphinx so that it gets displayed on nova.openstack.org, swift.openstack.org, and glance.openstack.org.

The DocBook source is built into html (webhelp) and PDF using XSLT transforms included with the DocBook project.

A super light-weight drafting option, easy as a click-through, is to write with other collaborators on an Etherpad at http://etherpad.openstack.org/. We can invite others as we go, and choose to write it as wikitext, RST, or docbook.

wiki.openstack.org (wikitext or RST)

The OpenStack wiki contains draft documentation but should ideally contain project docs, specs, doc drafts, and outlines. Any dev or user doc on the wiki is subject to constant change so if there's a page you want to keep an eye on (like Nova installations for example), add it to your Notifications list (under User > Settings > Notifications in the wiki). I've also begun a copy/paste effort to put RST in wiki pages to avoid multiple maintenance on pages that are also housed on nova.openstack.org, for example. There are great pages on the wiki that I want to take to the other doc sites, for example the Nova deploy page on the wiki should be highlighted in other locations as well.

nova.openstack.org, swift.openstack.org, glance.openstack.org (RST)

The RST pages stored with the project code should be written with a developer audience in mind, although many times you'll find there is overlap in what an admin needs to know and what a developer needs to know. High priorities for those sites are wider coverage of doc strings, API doc, i18N methodology, and architecture concepts that'll help developers.

docs.openstack.org (DocBook 5)

As mentioned above, the source for this site is housed in Github in the http://github.com/openstack/openstack-manuals project, the Launchpad page for this project is https://launchpad.net/openstack-manuals. You can either build the output locally if you want, or just submit changes to the source XML through merge proposals. The OpenStack Jenkins build server builds the doc and copies the output to docs.openstack.org/trunk automatically thanks to the pom.xml files included in the source files.

• doc/source/docbkx/$BOOK contains the DocBook XML source files and images

When editing DocBook documentation, please adhere to the DocBook 5 syntax. If you've used DocBook version 4 or earlier before, and you're not familiar with the changes that DocBook saw in V5.0, the Transition Guide is a helpful read.

To generate the PDF and webhelp within your local environment, install Maven. Then, to generate documentation for one of the manuals, go to the root directory of that manual (e.g., doc/src/docbkx/openstack-compute-admin) and do:

mvn generate-sources

The html file starts at doc/src/docbkx/openstack-compute-admin/target/docbkx/webhelp/trunk/openstack-compute/admin/index.html and the PDF will be doc/src/docbkx/openstack-compute-admin/target/docbkx/webhelp/trunk/openstack-compute/admin/os-compute-adminguide-trunk.pdf.

You may need a modified settings.xml and if you already have M2_HOME set, you might see an error about "java.lang.ClassNotFoundException: org.codehaus.plexus.classworlds.launcher.Launcher" - just run unset M2_HOME.

Versions of Doc

For RST-based documentation, you can get to a point-release of a docs site by going to http://swift.openstack.org/1.4, for example. We'll keep doing that for ongoing releases. For Swift, we now have archived sites for the 1.4.0 and 1.4.1 release at swift.openstack.org/1.4.0 and swift.openstack.org/1.4.1. To make those archived sites, you can run a build on the Jenkins server at http://jenkins.openstack.org/view/Swift/job/swift-docs-archive/ and enter the version number and the Launchpad branch that you want to use to build the archived site.

For DocBook documentation in openstack-manuals, we now have two branches available for authoring - the StableBranch for Diablo (nova 2011.3 and swift 1.4.3) and HEAD which corresponds to docs.openstack.org/trunk. The Continuous Integration work keeps docs building to docs.openstack.org/trunk and docs.openstack.org/diablo based on the branch they are checked into.

To work on edits that affect only Diablo, you need to:

• install and use the git-review tool, read more about it here: https://github.com/openstack-ci/git-review.
• follow the normal Gerrit workflow with these two changes:
  • - instead of 'git checkout master' use git checkout gerrit/stable/diablo - instead of 'git review' use git review gerrit/stable/diablo
• be very considerate of what you are documenting specifically for Diablo and why. If it applies to Diablo and ongoing Essex releases, you have to make your edits twice.
• note that the reviewers with +2 permissions for the stable/diablo branch are doc-core members and the stable maintenance team.

Doc Bug Triaging Guidelines

Doc bugs are logged and triaged at http://bugs.launchpad.net/openstack-manuals. You can see how well we're doing with bugs at this webnumbr. Here are some definitions for Status and Importance so you can triage incoming doc bugs.

Status:

New - Recently logged by a non-triaging person

Incomplete - Needs additional information before it can be triaged

Opinion - (not sure what to do with this one)

Invalid - Not an issue for docs

Won't Fix - Doc fixes won't fix the issue

Confirmed - Acknowledged that it's a doc bug

Triaged - Comments in the bug indicate its scope and amount of work to be done

In Progress - Someone is working on it

Fix Committed - A fix is in the repository; Gerrit sets this automatically

Fix Released - A fix is published to the site; would like the CI team to set this automatically, bug filed bug 943413

Importance:

Critical - data will be lost if this bug stays in; or it's so bad that we're better off fixing it than dealing with all the incoming questions about it. Also items on the website itself that prevent access are Critical doc bugs.

High - Definitely need docs about this or a fix to current docs; docs are incomplete without this. Work on these first if possible.

Medium - Need docs about this within a six-month release timeframe.

Low - Docs are fine without this but could be enhanced by fixing this bug.

Wishlist - Would like this doc task done "some day" Would prefer to use this for "tasks" instead of "bugs" - mark a bug as low rather than putting it on the wishlist. When something is wrong with the doc, mark it as Low rather than Wishlist.

Undecided - Recently logged by a non-triaging person or requires more research before deciding its importance.

Audiences and Targets

OpenStack has three main projects - Compute (nova), Object Storage (swift), and now an imaging service (glance). While the goals you'd achieve using each project vary, the three projects can also be used together. Those three projects have two basic audiences - developers and sysadmins (think operations or dev-ops). The RST-based documentation, because it automatically generates doc from docstrings in the code, is much more for a developer audience. There are two types of developers that the developer documentation serves - Python developers who want to work on OpenStack code, and web developers who work with the OpenStack API.

Workflow

The workflow for documentation matches the workflow for code - you download a branch, work on it locally, then commit the changes and propose a merge. Your changes are merged into the main, published doc base, and the doc is built to its varying outputs.

Detailed Workflow with Github and Gerrit

The page I always refer to for using Git and Gerrit for daily tasks is http://wiki.openstack.org/GerritWorkflow.

The workflow takes some startup work to get the accounts you need. First, you make a Launchpad account, and upload a public key. Then, you identify your Launchpad account to the git command line and away you go. You also need a Github account. The Launchpad account contains the signon and permissions information for the Gerrit workflow that we use for reviews.

Here's a sample for the documentation project specifically, once you get your Launchpad and Github accounts set up.

Create the directory where you want to store the doc source files, then switch to it.

mkdir ~/src/docs

cd ~/src/docs

Follow the steps in GerritWorkflow, using openstack-manuals as the project name for the Admin manuals, and project-api, such as compute-api, for the project name for API guides.

Now you can check out all the files for the admin doc project:

git clone git://github.com/openstack/openstack-manuals.git

Set up the Gerrit Change-Id hook, which is used for reviews, and run git review to run a script in the /tools directory which sets up the remote repository correctly:

cd $PROJECT
git review

Now make sure you have the latest from Github (docs move more slowly than code, but this is a good practice).

git checkout master
git pull origin master

Lastly, create a branch to do your work in that you'll do commits from:

git checkout -b TOPIC-BRANCH

Edit happily! You can use Oxygen since you're working on an open source project and we acknowledge their support.

When you're done with edits, if you've added any files do this in the working directory:

git add .

Next, you commit your changes with a message that'll be displayed in review.openstack.org, creating a change set:

git commit -a -m 'bug xxxxxxx Here's what I did in this set of changes'

Next, push your changes for review with:

git review

Go to the review site itself to see the review votes for the openstack-manuals project, at review.openstack.org. One of the core doc team members will review your edits and let you know if any changes are needed prior to merging them in.

Acknowledgment

Thanks to the generosity of Oxygen in supporting open source projects, you can edit DocBook XML within the Oxygen Author or Editor by downloading a copy of the software from http://www.oxygenxml.com/. Send an email to support @ oxygenxml dot com requesting the license extension beyond the 30-day trial.

Rackbook Framework

Thanks to Rackspace maintaining Oxygen tools that make authoring DocBook and WADL, we can offer a Rackbook framework. Download the attachment:com.oxygenxml.author.zip file from the Attachments on this page. Extract it to ~/Library/Preferences (i.e. so you end up with /Users/your.username/Library/Preferences/com.oxygenxml.author). You should see a new bar of buttons and a Rackbook menu item.

Translation

Translation efforts should occur either in the Launchpad Translations area (for strings in the code, error messages and the like) or on the wiki, which has translations enabled.

Figures

Here are some conventions for including figures and images in DocBook and RST documents.

In DocBook source, here is how you include images that have both a scalable-for-print-resolution in the PDF and an online-resolution for the HTML output. In this case the two types of images are SVG and PNG formats. The scale="60" attribute ensures that the image does not overlap print margins nor take up too much screen space.

<figure xml:id="CFinterfaces">
      <title>Cloud Files System Interfaces</title>
      <mediaobject>
        <imageobject role="fo">
          <imagedata scale="60" fileref="figures/CFinterfaces.svg"/>
        </imageobject>
        <imageobject role="html">
          <imagedata scale="60" fileref="figures/CFinterfaces.png"/>
        </imageobject>
      </mediaobject>
    </figure>

For DocBook, the convention is to use the /src/figures/ directory to store both the source image and any other formats of that same image. The pom.xml file copies the files in the /figures/ directory into the output directory required for HTML in the post processing section.

Also, when you add the image to the /src/figures directory, be sure to tell the source control system that you've added the image. For example, use "bzr add" to ensure the images get added to source control so the HTML and PDF output will be built correctly by the Jenkins continuous integration server.

For RST documents, this is the syntax for including an image in the output:

    .. image:: /images/dating_service.png

The /images directory is relative to the doc/source directory, within /source. There's also an images_src directory in the doc/source directory that contains the source images so that they can be updated when needed.

For any figure you create, please also include the source files, even if the image was not created with open source tools, for maintenance purposes. While all OpenStack docs are created with open source in mind, including open-licensed fonts in the output, we are willing to allow non-open authoring or image creation tools if it's more efficient.

Markup Conventions

Refer to Documentation/Conventions for markup information.

How to amend a review-in-progress

Within the repository for which you have a patch, do these steps:

git review -d 5829

where 5829 is the four-digit identifier found in the review.openstack.org URL for the patch set.

A new branch is created and checked out that you'll work within.

Then make your edits in the new branch that is created. Once edits are complete, do:

git commit -a --amend

and in the vi editor, add more lines to your commit message to explain the reasons for the patch set.

Press Esc :wq to save the message and quit the VI editor.

Then do: git review -v

How to a cherry-pick a change to a stable branch

Let's say a docfix was submitted to the master openstack-manuals branch, and you want this change to be reflected in the Essex documentation, which has already closed. You can cherry pick the change and submit it.

The simple way

Let's say we want to cherry-pick bug/948242 (review 7480) into essex.

1. Create a topic branch by branching from essex:

git checkout -b essex-bug/948242

2. Click "cherry-pick" (It's under Patch Set N, in the Download row), copy the text, and then patse it. For example, for review 7480, it looks like this:

git fetch https://review.openstack.org/openstack/openstack-manuals refs/changes/80/7480/2 && git cherry-pick FETCH_HEAD

3. Amend the commit log (Add something like "Cherry picked from review.openstack.org/748) and submit for review for the essex branch:

git commit --amend
git review stable/essex

The slightly more complicated way

You'll need the SHA of the commit, which you can find on the review page next to "Patch Set #". For example, for https://review.openstack.org/7178 the SHA is d5af1ec63f6209b578af0a7af4d256e1b883a069.

Assume you are in the openstack-manuals repository. First, set up a branch that tracks the essex branch on github:

git checkout -b stable/essex remotes/origin/stable/essex

Here's an example of doing a cherry pick and submitting to the essex branch

git checkout -b essex-rados stable/essex
git cherry-pick d5af1ec63f6209b578af0a7af4d256e1b883a069
git commit --amend
#  Change the comment to mention where you cherry-picked from
git review stable/essex

How to create PDF links on webhelp landing pages

You need these items to make the PDF link work:

<?rax pdf.url="../os-install-guide-trunk.pdf"?> above the title element in the book file, where the filename and path point to the PDF file name.

and

<canonicalUrlBase>http://docs.openstack.org/trunk/openstack-compute/admin/content/</canonicalUrlBase>

in the pom.xml file.

And at least <version>1.0.9</version>

in the pom.xml file for the plugin com.rackspace.cloud.api.