Jump to: navigation, search


Doc Migration from DocBook to RST Overview

Many of the API reference documents were meant for the contributor developer to know what was going into an API. So, the history of that document was forged as a specification. As projects beyond swift and nova were added, projects made similar documents. They're output to HTML at http://docs.openstack.org/api/api-specs.html. However, as we now have a <project>-specs repo, it makes sense to move to those repos. In that move, also it makes sense to migrate to RST rather than DocBook/WADL.

We are also migrating the End User Guide and Admin User Guide to RST. Feel free to sign up for a chapter in the section below.

Migration Conventions

Follow the existing existing markup conventions for RST.

Follow the existing file name conventions.

When in doubt, use simpler markup. This guidance means revise the text to not use tables, and avoid cross-references that are difficult to maintain.

Use :ref: when doing cross references so that you don't have to have the exact title. This means you will have to add markup to create anchors, such as:

.. _dashboard-project-tab:

Please use the existing XML:Ids if possible for these cross references.

Store the figures in a /figures/ directory with the deliverable itself.

If the conversion outputs Note as a heading, change it to .. note::.

If your file has lots of code blocks, be sure to set up line numbering for the entire file. Each code-block must have :linenos: after the .. code-block:: directive, the code itself must be indented at the same level as the :lineos: line, and you must have at the start of the file

.. highlight: python :linenothreshold: 5

so that any code-blocks longer than five lines long will have line numbers.

Migration Issues

How to get numbered list continuation to work when you have a table in a step?

How to get numbered list continuation to work when you have a bulleted list between list items?

How to get get embedded .. note: directives to work between numbered list items?

Can you have line breaks in something with inline semantic markup like :guilabel: at the end of the line? For example:

#. Select the volume to add to an instance and click :guilabel:`Edit Attachments`.

That line is longer than 79 characters, so Attachments`. must be on the second line, but when you do that, the HTML output is incorrect.

Migration How To

Note: If you don't want to use tox, install these prereqs locally: pip install sphinx; pip install openstackdocstheme and then switch to the directory containing a conf.py and run sphinx-build /path/to/source/ path/to/build/ to get html output by default.

Because Sphinx builds have dependent requirements it is best to work with in a virtual environment. Fortunately the openstack-manuals project already has tox set up so that you can create a virtualenv that contains the required dependencies. To use it, do this on a Mac or Ubuntu machine with Python and pip already installed:

Clone the openstack-manuals repo.

Change directories into openstack-manuals.

Run this:

tox -e py27

When you get a congratulations message, run this:


source .tox/py27/bin/activate


source .tox/py27/Scripts/activate

Your prompt should now have (py27) as a preface.

Now you have all the pre-requisites installed and can run additional tox commands. To see the list of commands, view or edit tox.ini in the openstack-manuals directory. To build the RST docs, run:

tox -e docs

This will run sphinx-build doc/playground-user-guide/source/ doc/playground-user-guide/build/html.

When the build is finished, you can open doc/playground-user-guide/build/html/index.html to view the resulting output.

End User Guide Migration

First, get a local copy of the End User Guide from openstack-manuals. Create a branch to work within.

Next, either use the process documented on this wiki page or your own methods with pandoc (http://johnmacfarlane.net/pandoc/try/) to convert from DocBook to RST. Pick any chapter or section and look for errors in the automation, then manually clean up. Once you have cleaned up the RST, build it within playground-user-guide by running tox -echeckbuild or if you already have Sphinx installed, run sphinx-build doc/playground-user-guide/source/ doc/playground-user-guide/build/html from within openstack-manuals. This will apply the oslosphinx theme currently.

Sign up below for a chapter, then create a patch with RST in doc/playground-user-guide/source for others to review. Use the xml:id from the DocBook file for the RST file name. Once the patch merges, change the Status from In Review to Merged.

Chapter/Section (old filename) Chapter/Section (new filename) Name Patch URL Status
xxx section_dashboard_access_and_security.rst Andreas Jaeger https://review.openstack.org/#/c/142437/ Merged
xxx section_dashboard_access Anne Gentle https://review.openstack.org/151289 In Progress
xxx launching_instances_using_dashboard Anne Gentle https://review.openstack.org/151818 In Progress
section_dashboard_databases.xml dashboard_databases Anne Gentle https://review.openstack.org/#/c/151846/ In Progress
xxx dashboard_create_networks Anne Gentle https://review.openstack.org/#/c/152582/ In Progress
xxx dashboard_manage_containers Anne Gentle https://review.openstack.org/#/c/153349/ In Progress
xxx dashboard_manage_volumes Anne Gentle https://review.openstack.org/#/c/153577 In Progress
section_cli_trove.xml trove-manage-db.rst Laurel Michaels https://review.openstack.org/#/c/156087/ In Progress
section_cli_trove.xml create_db.rst Laurel Michaels https://review.openstack.org/156266/ In Progress
ch_cli.xml ch_cli Karin Levenstein https://review.openstack.org/#/c/154235/ In Progress
section_cli_nova_boot cli_launch_instances Karin Levenstein https://review.openstack.org/#/c/154235/ In Progress
ch_sdk.xml ch_sdk Darren Chan https://review.openstack.org/#/c/154412/ In Progress
section_sdk_install sdk_install Darren Chan https://review.openstack.org/#/c/154412/ In Progress
section_sdk_authenticate sdk_authenticate Darren Chan https://review.openstack.org/#/c/154412/ In Progress
section_sdk_manage_images.xml sdk_manage _images Darren Chan https://review.openstack.org/#/c/154412/ In Progress
section_sdk_configure_instances.xml sdk_configure _access_security_instances Darren Chan https://review.openstack.org/#/c/154412/ In Progress
section_sdk_neutron.xml sdk_authenticate_networking_endpoint Darren Chan https://review.openstack.org/#/c/154412/ In Progress
section_sdk_nova.xml sdk_authenticate_compute_endpoint Darren Chan https://review.openstack.org/#/c/154412/ In Progress
section_cli_swift_howto.xml managing-openstack-object-storage-with-swift-cli Erik Wilson local In Progress

Admin User Guide Migration

Sign up below for a chapter, then create a patch with RST in doc/playground-user-guide/source for others to review.

Chapter/Section (old filename) Chapter/Section (new filename) Name Patch URL Status
section_cli_keystone_manage_projects_users_roles.xml admin_cli_manage_projects_users.rst Anne Gentle local In Progress
section_cli_keystone_manage_projects_users_roles.xml nova_cli_manage_projects_security.rst Anne Gentle local In Progress

Doc Migration Plan

The Juno projects where this document needs to migrate to a specification are: nova (compute-api) v2, v3 swift (object-api) v1 glance (image-api) v1, v1.1, v2 keystone (identity-api) v2.0, v3 neutron (netconn-api) v1.0, v2.0 cinder (volume-api) v1.0, v2.0

Juno projects that have this type of document in a separate <service>-api repo are: trove (trove)

Juno projects that do not provide this type of prose-based spec for their API are: ceilometer heat sahara

Incubating projects, this information is just a heads-up so you know how we're thinking about API documentation going forward. ironic zaqar (marconi) barbican designate

I'm going to do the migration work with pandoc and propose the patch to the project's repo. Be on the lookout for those patches.

Doc Migration Steps

Open docbook book file in Oxygen.

Choose Document > Transformation > Configure Transformation Scenario(s).

Select DocBook XHTML - Chunk.

Click Apply associated (1).

Within the /out/xhtml-chunks/ directory that's generated, run the following script:

for i in *.xhtml
  # Convert from XHTML to RST
  pandoc -s -t rst $i -o $file_name
  sed -i -e '4,16d' $file_name 
  sed -i -e '/+--------------------------+$/,$d' $file_name
  sed -i -e '$d' $file_name
  sed -i -e '$d' $file_name

  # Rename file to second line of new RST content, but lowercase and
  # all non-alphanumeric chars renamed to underscores
  real_file_name=$(sed 's/[^a-zA-Z0-9\-]/_/g;2q;d' $file_name | awk '{print tolower($0)}').rst
  mv $file_name $real_file_name
  # Replace all cross-refs to xhtml files to renamed rst files
  sed -i '' -e "s/\<$i/fixmefixmefixme/g" *.rst
#  sed -i '' -e "s/\ fixmefixmefixme.*\`__//g" *.rst

Clean up where the fredfredfred is output, it indicates where a cross-reference cannot exist any longer.

Clean up tables where the pandoc conversion just outputs paragraphs.

Remove numbering from Example titles and Table titles.

Rename chapter_ files and ensure they are titled to match the contents of the file, such as "Networking API 2.0 Overview" to networking_api_2.0_overview" for example.

Remove "programlisting" "screen" and "literallayout" from .. code:: lines.