Jump to: navigation, search

Difference between revisions of "Documentation/Migrate"

(Operations Guide)
(Operations Guide)
Line 226: Line 226:
 
| ch_arch_examples.xml ||  ||  ||  ||  
 
| ch_arch_examples.xml ||  ||  ||  ||  
 
|-
 
|-
| section_arch_example-neutron.xml ||section_arch_example-neutron.rst  || venkatamahesh ||  ||  
+
| section_arch_example-neutron.xml || section_arch_example-neutron.rst  || venkatamahesh || https://review.openstack.org/#/c/291965 || Inprogress
 
|-
 
|-
| section_arch_example-nova.xml ||section_arch_example-nova.rst  || venkatamahesh ||https://review.openstack.org/#/c/291783  || InProgress  
+
| section_arch_example-nova.xml || section_arch_example-nova.rst  || venkatamahesh   || https://review.openstack.org/#/c/291783  || InProgress  
 
|-
 
|-
| ch_arch_network_design.xml ||  ||  ||  ||  
+
| ch_arch_network_design.xml ||  || venkatamahesh ||  ||  
 
|-
 
|-
| ch_arch_provision.xml ||  ||  ||  ||  
+
| ch_arch_provision.xml ||  || venkatamahesh ||  ||  
 
|-
 
|-
 
| ch_arch_scaling.xml ||  ||  ||  ||  
 
| ch_arch_scaling.xml ||  ||  ||  ||  

Revision as of 01:28, 12 March 2016

Doc Migration from DocBook to RST Overview

Previously, we migrated the <service>-api repos from DocBook to RST. These were API reference documents that were meant for the contributor developer to know what was going into an API. So, the history of that document was forged as a specification. As projects beyond swift and nova were added, projects made similar documents. They're output to HTML at http://docs.openstack.org/api/api-specs.html. However, as we now have a <project>-specs repo, it makes sense to move to those repos. In that move, also it makes sense to migrate to RST rather than DocBook/WADL.

Migration Conventions

Follow the RST markup conventions in the OpenStack Documentation Contributor Guide .

Headings

First title in an RST file, use equals signs above and below. Second title, use tilde. Third title, use a series of dashes. Try not to need a Heading 4 if you can help it by rewriting or reorganizing.



========
Heading 1
========

Heading 2
~~~~~~~~~

Heading 3
---------

File names

As a rule, we want to keep the xml:id identical so that the output HTML files do not need redirection. However, we also at the same time want to get rid of ch_ and section_ nomenclatures while going to a page-based, topical approach. So, first use the XML:ID, and if the XML:ID has ch_ or section_ in it, remove the prefix.

Cross references

Use :ref: when doing cross references so that you don't have to have the exact title. This means you will have to add markup to create anchors, such as:

.. _dashboard-project-tab:

Please use the existing XML:Ids if possible for these cross references.

Also, while intersphinx is an enabled extension for many of the contributor developer docs, we don't have an intersphinx requirement yet for the end user guide or admin user guide. We'll investigate that as we add more guides.

Figures and images

Store the figures in a /figures/ directory with the deliverable itself.

Notes and admonitions

If the conversion outputs Note as a heading, change it to use the .. note:: directive.

Line numbers in code blocks

If your file has lots of code blocks, be sure to set up line numbering for the entire file. Each code-block must have :linenos: after the .. code-block:: directive, the code itself must be indented at the same level as the :lineos: line, and you must have at the start of the file this directive:

.. highlight: python
   :linenothreshold: 5

so that any code-blocks longer than five lines long will have line numbers.

Migration Issues

List of bugs or missing features for the Sphinx template, openstackdocstheme: https://bugs.launchpad.net/openstack-manuals/+bugs?field.tag=openstackdocstheme

Can you have line breaks in something with inline semantic markup like :guilabel: at the end of the line? For example:


#. Select the volume to add to an instance and click :guilabel:`Edit Attachments`.

That line is longer than 79 characters, so Attachments`. must be on the second line, but when you do that, the HTML output is incorrect.

Since RST documents impose a 79 character limit when building files in .rst format. When tables exceed that limit, a list table role can help create content in a new table without exceeding the character limit. The issue is when a table is generated by an OpenStack service, and returned as a result of a command. These tables are constructed with +, -, and | characters. The table created by a list table role has solid lines forming rows, columns, and cells. The content is similar, however users might be surprised by a table with solid lines rather than a table built with +, -, and | characters.

Large scale content inside tables.

When building tables with a list table role, some items that appear inside the table cells might exceed the 79 character limit. Is there a solution to recreating this large scale content in a table such that it does not affect the character limit?

The :orphan: role and references.

Files with an :orphan: role at the top of the file, not included in the toc tree, will not link to another file when the :ref: role is used to link to the file from a different file. Is there a solution to this linking issue?

Pandoc conversion problems.

When converting files with Pandoc, some entries in the .xml file will not convert accurately. Procedure titles and sub section titles will not parse correctly, and will be deleted from the converted document. Pandoc also removes procedure numbering. Currently, the solution is to compare the complete .rst file to the built .xml version, and check the headings, subheadings, and procedure numbers to make sure they are correct, and line up with original. This is an issue that could also be discussed.

  • Contact details for RST issues that we can't solve:
    • Author: David Goodger
    • Contact: docutils-develop@lists.sourceforge.net

Migration How To

Note: If you don't want to use tox, install these prereqs locally: pip install sphinx; pip install openstackdocstheme and then switch to the directory containing a conf.py and run sphinx-build /path/to/source/ path/to/build/ to get html output by default.

Because Sphinx builds have dependent requirements it is best to work with in a virtual environment. Fortunately the openstack-manuals project already has tox set up so that you can create a virtualenv that contains the required dependencies. To use it, do this on a Mac or Ubuntu machine with Python and pip already installed:

  1. Clone the openstack-manuals repository.
  2. Change directories into openstack-manuals.
  3. Run this:
    tox -e py27
  4. When you get a congratulations message, run this:

Mac/Ubuntu:

source .tox/py27/bin/activate

Windows:

 source .tox/py27/Scripts/activate

Your prompt should now have (py27) as a preface.

Now you have all the pre-requisites installed and can run additional tox commands. To see the list of commands, view or edit tox.ini in the openstack-manuals directory. To build the RST docs, run:

tox -e docs

This will run sphinx-build doc/playground-user-guide/source/ doc/playground-user-guide/build/html.

When the build is finished, you can open doc/playground-user-guide/build/html/index.html to view the resulting output.

Migration Using Oxygen

  1. Open DocBook book file in Oxygen.
  2. Choose Document > Transformation > Configure Transformation Scenario(s).
  3. Select DocBook XHTML - Chunk.
  4. Click Apply associated (1).
  5. Within the /out/xhtml-chunks/ directory that's generated, run the following script:
    for i in *.xhtml
    do
    # Convert from XHTML to RST
    file_name=${i%.*l}.rst
    pandoc -s -t rst $i -o $file_name
    sed -i -e '4,16d' $file_name
    sed -i -e '/+--------------------------+$/,$d' $file_name
    sed -i -e '$d' $file_name
    sed -i -e '$d' $file_name
    # Rename file to second line of new RST content, but lowercase and all non-alphanumeric chars renamed to underscores:
    real_file_name=$(sed 's/[^a-zA-Z0-9\-]/_/g;2q;d' $file_name | awk '{print tolower($0)}').rst
    mv $file_name $real_file_name<pre><nowiki>
    # Replace all cross-refs to xhtml files to renamed rst files:
    sed -i '' -e "s/\<$i/fixmefixmefixme/g" *.rst
    # sed -i '' -e "s/\ fixmefixmefixme.*\`__//g" *.rst
    done
  6. Clean up where the fixmefixmefixme is output, it indicates where a cross-reference cannot exist any longer.
  7. Clean up tables where the pandoc conversion just outputs paragraphs.
  8. Remove numbering from Example titles and Table titles.
  9. Rename chapter_ files and ensure they are titled to match the contents of the file, such as "Networking API 2.0 Overview" to networking-api-2.0-overview" for example.
  10. Remove "programlisting" "screen" and "literallayout" from .. code:: lines.
  11. When commiting conversion patches, include the related blueprint in your commit message. For example:
    Convert ch_compute_focus.xml to RST
    Implements: blueprint archguide-mitaka-rst

Migration Using a Text Editor

  1. Install pandoc.
  2. Update your openstack-manuals master branch, and create a branch.
  3. Run the pandoc command to convert the .xml file to .rst. For example:
    pandoc -f docbook -t rst -s ch_compute_focus.xml -o compute-focus.rst
  4. Move converted files to the <guide>/source folder.
  5. Rename converted RST files and ensure they are titled to match the contents of the file, such as "Networking API 2.0 Overview" to networking-api-2.0-overview" for example.
  6. Pandoc has its quirks, so check all the content has been migrated from the source file.
  7. Use a text editor to complete the following tasks:
    1. Clean up tables where the pandoc conversion just outputs paragraphs.
    2. Remove numbering from Example titles and Table titles.
    3. Remove "programlisting" "screen" and "literallayout" from .. code:: lines.
    4. Check the converted RST file follow conventions described in the OpenStack Documentation Contributor Guide
  8. Build the guide locally, to check for errors:
    tox -e docs
  9. Commit your changes, including the related blueprint in your commit message. For example:
    Convert ch_compute_focus.xml to RST
    Implements: blueprint archguide-mitaka-rst

Doc Migration Plan

For the Mitaka release, we are migrating the Configuration Reference Guide and the Architecture Design Guide. Refer to the detailed specification for details.

The Juno projects where this document needs to migrate to a specification are: nova (compute-api) v2, v3 swift (object-api) v1 glance (image-api) v1, v1.1, v2 keystone (identity-api) v2.0, v3 neutron (netconn-api) v1.0, v2.0 cinder (volume-api) v1.0, v2.0

Juno projects that have this type of document in a separate <service>-api repo are: trove (trove)

Juno projects that do not provide this type of prose-based spec for their API are: ceilometer heat sahara

Incubating projects, this information is just a heads-up so you know how we're thinking about API documentation going forward. ironic zaqar (marconi) barbican designate

I'm going to do the migration work with pandoc and propose the patch to the project's repo. Be on the lookout for those patches.

Operations Guide

Sign up below for a chapter, then create a patch with RST in doc/ops-guide/source for others to review. Contributions are welcome. Please add "Implements: blueprint ops-guide-rst" in the commit message.

Please make sure this book follows O'Reilly conventions.

Chapter/Section (old filename) Chapter/Section (new filename) Name Patch URL Status
bk_ops_guide.xml
preface_ops.xml
part_architecture.xml
part_operations.xml
ch_arch_cloud_controller.xml
ch_arch_compute_nodes.xml
ch_arch_examples.xml
section_arch_example-neutron.xml section_arch_example-neutron.rst venkatamahesh https://review.openstack.org/#/c/291965 Inprogress
section_arch_example-nova.xml section_arch_example-nova.rst venkatamahesh https://review.openstack.org/#/c/291783 InProgress
ch_arch_network_design.xml venkatamahesh
ch_arch_provision.xml venkatamahesh
ch_arch_scaling.xml
ch_arch_storage.xml
ch_ops_advanced_configuration.xml
ch_ops_backup_recovery.xml
ch_ops_customize.xml
ch_ops_lay_of_land.xml
ch_ops_log_monitor.xml
ch_ops_maintenance.xml
ch_ops_network_troubleshooting.xml
ch_ops_projects_users.xml
ch_ops_resources.xml
ch_ops_upgrades.xml
ch_ops_upstream.xml
ch_ops_user_facing.xml

Completing conversion to RST

Once we have reviewed the draft guide and think it's ready to publish, we need to do the following steps:

  1. Create a patch with the following steps (for example, see https://review.openstack.org/#/c/248577/ or https://review.openstack.org/211766)
    1. Delete the old guide, here security-guide
    2. Move the RST guide to the location of new guide (security-guide-rst -> security-guide).
    3. Update tools/build-all-rst.sh for the change
    4. Delete the entry from doc/pom.xml
    5. Update RELEASENOTES.rst
    6. If the repository has no further DocBook guides in it, update tox.ini
    7. Update doc-tools-check-languages.conf
    8. Rename localization files to new directory name
    9. If there is a ".tx/config" file, update it (remove RST guide and update paths)
  2. If the repository has no further DocBook guides in it:
  3. Update .gitignore. For example: https://review.openstack.org/212685
  4. If the repository is not the openstack-manuals repo, stop syncing of XML files: https://review.openstack.org/211842 and remove the copied files once that patch is merged: https://review.openstack.org/211904
  5. Fix links on docs.openstack.org and in other guides: https://review.openstack.org/212044
    • Create redirects from content directory to top-level index.html file
    • Update all links so that they go to new guide (security-guide, not security-guide/content)
    • Remove links to PDF of guide
  6. Sync translations from old guide with new guide (needs to be done in transifex by Andreas)
  7. Tell i18n team that conversion is finished and which resource is active.
  8. Remove old guide and draft RST guides from docs.openstack.org (needs docs.openstack.org admin access)
  9. Regenerate sitemap.xml after all changes are in: https://review.openstack.org/212689 and blacklist the change: https://review.openstack.org/212690