Jump to: navigation, search

Difference between revisions of "Documentation/Migrate"

m (Conversion Using a Text Editor)
m (Conversion Using a Text Editor)
Line 504: Line 504:
 
## Check the converted RST file follows conventions described in the [http://docs.openstack.org/contributor-guide/rst-conv.html OpenStack Documentation Contributor Guide]
 
## Check the converted RST file follows conventions described in the [http://docs.openstack.org/contributor-guide/rst-conv.html OpenStack Documentation Contributor Guide]
 
# Commit your changes, including the related blueprint in your commit message. For example:
 
# Commit your changes, including the related blueprint in your commit message. For example:
  Convert ch_compute_focus.xml to RST
+
#:  Convert ch_compute_focus.xml to RST
 
    
 
    
 
   Implements: blueprint archguide-mitaka-rst
 
   Implements: blueprint archguide-mitaka-rst

Revision as of 03:33, 3 November 2015

Doc Migration from DocBook to RST Overview

Previously, we migrated the <service>-api repos from DocBook to RST. These were API reference documents that were meant for the contributor developer to know what was going into an API. So, the history of that document was forged as a specification. As projects beyond swift and nova were added, projects made similar documents. They're output to HTML at http://docs.openstack.org/api/api-specs.html. However, as we now have a <project>-specs repo, it makes sense to move to those repos. In that move, also it makes sense to migrate to RST rather than DocBook/WADL.

Migration Conventions

Follow the RST markup conventions in the OpenStack Documentation Contributor Guide .

Headings

First title in an RST file, use equals signs above and below. Second title, use tilde. Third title, use a series of dashes. Try not to need a Heading 4 if you can help it by rewriting or reorganizing.



========
Heading 1
========

Heading 2
~~~~~~~~~

Heading 3
---------

File names

As a rule, we want to keep the xml:id identical so that the output HTML files do not need redirection. However, we also at the same time want to get rid of ch_ and section_ nomenclatures while going to a page-based, topical approach. So, first use the XML:ID, and if the XML:ID has ch_ or section_ in it, remove the prefix.

Cross references

Use :ref: when doing cross references so that you don't have to have the exact title. This means you will have to add markup to create anchors, such as:

.. _dashboard-project-tab:

Please use the existing XML:Ids if possible for these cross references.

Also, while intersphinx is an enabled extension for many of the contributor developer docs, we don't have an intersphinx requirement yet for the end user guide or admin user guide. We'll investigate that as we add more guides.

Figures and images

Store the figures in a /figures/ directory with the deliverable itself.

Notes and admonitions

If the conversion outputs Note as a heading, change it to use the .. note:: directive.

Line numbers in code blocks

If your file has lots of code blocks, be sure to set up line numbering for the entire file. Each code-block must have :linenos: after the .. code-block:: directive, the code itself must be indented at the same level as the :lineos: line, and you must have at the start of the file this directive:

.. highlight: python
   :linenothreshold: 5

so that any code-blocks longer than five lines long will have line numbers.

Migration Issues

List of bugs or missing features for the Sphinx template, openstackdocstheme: https://bugs.launchpad.net/openstack-manuals/+bugs?field.tag=openstackdocstheme

Can you have line breaks in something with inline semantic markup like :guilabel: at the end of the line? For example:


#. Select the volume to add to an instance and click :guilabel:`Edit Attachments`.

That line is longer than 79 characters, so Attachments`. must be on the second line, but when you do that, the HTML output is incorrect.

Migration How To

Note: If you don't want to use tox, install these prereqs locally: pip install sphinx; pip install openstackdocstheme and then switch to the directory containing a conf.py and run sphinx-build /path/to/source/ path/to/build/ to get html output by default.

Because Sphinx builds have dependent requirements it is best to work with in a virtual environment. Fortunately the openstack-manuals project already has tox set up so that you can create a virtualenv that contains the required dependencies. To use it, do this on a Mac or Ubuntu machine with Python and pip already installed:

  1. Clone the openstack-manuals repository.
  2. Change directories into openstack-manuals.
  3. Run this:
  tox -e py27
  1. When you get a congratulations message, run this:

Mac/Ubuntu:

 source .tox/py27/bin/activate

Windows:

 source .tox/py27/Scripts/activate

Your prompt should now have (py27) as a preface.

Now you have all the pre-requisites installed and can run additional tox commands. To see the list of commands, view or edit tox.ini in the openstack-manuals directory. To build the RST docs, run:

tox -e docs

This will run sphinx-build doc/playground-user-guide/source/ doc/playground-user-guide/build/html.

When the build is finished, you can open doc/playground-user-guide/build/html/index.html to view the resulting output.


Doc Migration Plan

For the Mitaka release, we are migrating the Configuration Reference Guide and the Architecture Design Guide. Refer to the detailed specification for the plan.

The Juno projects where this document needs to migrate to a specification are: nova (compute-api) v2, v3 swift (object-api) v1 glance (image-api) v1, v1.1, v2 keystone (identity-api) v2.0, v3 neutron (netconn-api) v1.0, v2.0 cinder (volume-api) v1.0, v2.0

Juno projects that have this type of document in a separate <service>-api repo are: trove (trove)

Juno projects that do not provide this type of prose-based spec for their API are: ceilometer heat sahara

Incubating projects, this information is just a heads-up so you know how we're thinking about API documentation going forward. ironic zaqar (marconi) barbican designate

I'm going to do the migration work with pandoc and propose the patch to the project's repo. Be on the lookout for those patches.

Configuration Reference Guide Migration

Sign up below for a chapter, then create a patch with RST in doc/config-ref-rst/source for others to review. While the Config Ref Specialty team will be giving the conversion attention, contributions are welcome.

Chapter/Section (old filename) Chapter/Section (new filename) Name Patch URL Status
app_firewalls-ports.xml
app_policy_json.xml
bk-config-ref.xml
block-storage/backup/ceph-backup-driver.xml
block-storage/backup/nfs-backup-driver.xml
block-storage/backup/swift-backup-driver.xml
block-storage/backup/tsm-backup-driver.xml
block-storage/drivers/ceph-rbd-volume-driver.xml
block-storage/drivers/datera-volume-driver.xml
block-storage/drivers/dell-equallogic-driver.xml
block-storage/drivers/dell-storagecenter-driver.xml
block-storage/drivers/dothill-driver.xml
block-storage/drivers/emc-scaleio-driver.xml
block-storage/drivers/emc-vmax-driver.xml
block-storage/drivers/emc-vnx-driver.xml
block-storage/drivers/emc-xtremio-driver.xml
block-storage/drivers/figures/xenapinfs/local_config.png
block-storage/drivers/figures/xenapinfs/remote_config.png
block-storage/drivers/figures/xenapinfs/xenapinfs.svg
block-storage/drivers/glusterfs-driver.xml
block-storage/drivers/hds-hnas-driver.xml
block-storage/drivers/hitachi-storage-volume-driver.xml
block-storage/drivers/hp-3par-driver.xml
block-storage/drivers/hp-lefthand-driver.xml
block-storage/drivers/hp-msa-driver.xml
block-storage/drivers/huawei-storage-driver.xml
block-storage/drivers/ibm-flashsystem-volume-driver.xml
block-storage/drivers/ibm-gpfs-volume-driver.xml
block-storage/drivers/ibm-sonas-7k-driver.xml
block-storage/drivers/ibm-storwize-svc-driver.xml
block-storage/drivers/ibm-xiv-volume-driver.xml
block-storage/drivers/lenovo-driver.xml
block-storage/drivers/lvm-volume-driver.xml
block-storage/drivers/netapp-volume-driver.xml
block-storage/drivers/nfs-volume-driver.xml
block-storage/drivers/nimble-volume-driver.xml
block-storage/drivers/prophetstor-dpl-driver.xml
block-storage/drivers/pure-storage-driver.xml
block-storage/drivers/scality-sofs-driver.xml
block-storage/drivers/sheepdog-driver.xml
block-storage/drivers/smbfs-volume-driver.xml
block-storage/drivers/solidfire-volume-driver.xml
block-storage/drivers/tintri-volume-driver.xml
block-storage/drivers/violin-v6000-driver.xml
block-storage/drivers/violin-v7000-driver.xml
block-storage/drivers/vmware-vmdk-driver.xml
block-storage/drivers/windows-iscsi-volume-driver.xml
block-storage/drivers/xio-volume-driver.xml
block-storage/drivers/zfssa-iscsi-driver.xml
block-storage/drivers/zfssa-nfs-driver.xml
block-storage/section_backup-drivers.xml
block-storage/section_block-storage-overview.xml
block-storage/section_block-storage-sample-configuration-files.xml
block-storage/section_cinder-log-files.xml
block-storage/section_fc-zoning.xml
block-storage/section_misc.xml
block-storage/section_volume-drivers.xml
block-storage/section_volume-encryption.xml
ch_baremetalconfigure.xml
ch_blockstorageconfigure.xml
ch_computeconfigure.xml
ch_config-overview.xml
ch_dashboardconfigure.xml
ch_databaseserviceconfigure.xml
ch_dataprocessingserviceconfigure.xml
ch_identityconfigure.xml
ch_imageservice.xml
ch_networkingconfigure.xml
ch_objectstorageconfigure.xml
ch_orchestrationconfigure.xml
ch_sharedfilesystemsconfigure.xml
ch_telemetryconfigure.xml
compute/samples/cells-resp.json
compute/section_compute-cells.xml
compute/section_compute-conductor.xml
compute/section_compute-config-samples.xml
compute/section_compute-configure-backing-storage.xml
compute/section_compute-configure-db.xml
compute/section_compute-configure-xapi.xml
compute/section_compute-hypervisors.xml
compute/section_compute-iscsioffload.xml
compute/section_compute-options-reference.xml
compute/section_compute-sample-configuration-files.xml
compute/section_compute-scheduler.xml
compute/section_hypervisor_hyper-v.xml
compute/section_hypervisor_kvm.xml
compute/section_hypervisor_lxc.xml
compute/section_hypervisor_qemu.xml
compute/section_hypervisor_vmware.xml
compute/section_hypervisor_xen_api.xml
compute/section_hypervisor_xen_libvirt.xml
compute/section_nova-conf.xml
compute/section_nova-log-files.xml
compute/section_rpc.xml
compute/section_xapi-ami-setup.xml
compute/section_xapi-install-plugins.xml
compute/section_xapi-install.xml
compute/section_xapi-resize-setup.xml
conf-changes/ceilometer.xml Automated
conf-changes/cinder.xml Automated
conf-changes/glance.xml Automated
conf-changes/heat.xml Automated
conf-changes/ironic.xml Automated
conf-changes/keystone.xml Automated
conf-changes/manila.xml Automated
conf-changes/neutron.xml Automated
conf-changes/nova.xml Automated
conf-changes/README.txt
conf-changes/sahara.xml Automated
conf-changes/swift.xml Automated
conf-changes/trove.xml Automated
dashboard/section_dashboard-log-files.xml
dashboard/section_dashboard-sample-configuration-files.xml
database-service/section-databaseservice-db.xml
database-service/section-databaseservice-rpc.xml
identity/section_keystone-sample-conf-files.xml
image-service/section_image-service-api.xml
image-service/section_image-service-backends.xml
image-service/section_image-service-backend-vmware.xml
image-service/section_image-service-ISO-support.xml
image-service/section_image-service-rpc.xml
image-service/section_image-service-sample-configuration-files.xml
networking/section_networking-log-files.xml
networking/section_networking-options-reference.xml
networking/section_networking-plugins-ml2.xml
networking/section_networking-plugins.xml
networking/section_networking-sample-configuration-files.xml
networking/section_rpc-for-networking.xml
object-storage/section_configure_s3.xml
object-storage/section_object-storage-cors.xml
object-storage/section_object-storage-features.xml
object-storage/section_object-storage-general-service-conf.xml
object-storage/section_object-storage-listendpoints.xml
orchestration/section_orchestration-api.xml
orchestration/section_orchestration-clients.xml
orchestration/section_orchestration-rpc.xml
roadmap.rst
shared-file-systems/drivers/emc-isilon-driver.xml
shared-file-systems/drivers/emc-vnx-driver.xml
shared-file-systems/drivers/generic-driver.xml
shared-file-systems/drivers/glusterfs-driver.xml
shared-file-systems/drivers/glusterfs-native-driver.xml
shared-file-systems/drivers/hdfs-native-driver.xml
shared-file-systems/drivers/hp-3par-share-driver.xml
shared-file-systems/drivers/huawei-nas-driver.xml
shared-file-systems/drivers/ibm-gpfs-driver.xml
shared-file-systems/drivers/netapp-cluster-mode-driver.xml
shared-file-systems/section_manila-log-files.xml
shared-file-systems/section_manila-misc.xml
shared-file-systems/section_manila-sample-configuration-files.xml
shared-file-systems/section_shared-file-systems-overview.xml
shared-file-systems/section_share-drivers.xml
table_default-ports-peripheral-services.xml
table_default-ports-primary-services.xml
telemetry/section_telemetry-alarming-service-config-opts.xml
telemetry/section_telemetry-sample-configuration-files.xml
telemetry/section_telemetry-service-config-opts.xml

Architecture Design Guide Migration

Doc Migration

Conversion Using Oxygen

  1. Open DocBook book file in Oxygen.
  2. Choose Document > Transformation > Configure Transformation Scenario(s).
  3. Select DocBook XHTML - Chunk.
  4. Click Apply associated (1).
  5. Within the /out/xhtml-chunks/ directory that's generated, run the following script:
for i in *.xhtml
   do
   # Convert from XHTML to RST
   file_name=${i%.*l}.rst
   pandoc -s -t rst $i -o $file_name
   sed -i -e '4,16d' $file_name 
   sed -i -e '/+--------------------------+$/,$d' $file_name
   sed -i -e '$d' $file_name
   sed -i -e '$d' $file_name
  # Rename file to second line of new RST content, but lowercase and all non-alphanumeric chars renamed to underscores:
   <pre><nowiki>real_file_name=$(sed 's/[^a-zA-Z0-9\-]/_/g;2q;d' $file_name | awk '{print tolower($0)}').rst
   mv $file_name $real_file_name<pre><nowiki>
  # Replace all cross-refs to xhtml files to renamed rst files:
   sed -i '' -e "s/\<$i/fixmefixmefixme/g" *.rst
  #  sed -i '' -e "s/\ fixmefixmefixme.*\`__//g" *.rst
  done
  1. Clean up where the fixmefixmefixme is output, it indicates where a cross-reference cannot exist any longer.
  2. Clean up tables where the pandoc conversion just outputs paragraphs.
  3. Remove numbering from Example titles and Table titles.
  4. Rename chapter_ files and ensure they are titled to match the contents of the file, such as "Networking API 2.0 Overview" to networking_api_2.0_overview" for example.
  5. Remove "programlisting" "screen" and "literallayout" from .. code:: lines.
  6. When commiting conversion patches, include the related blueprint in your commit message. For example:
  Convert ch_compute_focus.xml to RST
  
  Implements: blueprint archguide-mitaka-rst

Conversion Using a Text Editor

  1. Install pandoc
  2. Update your openstack-manuals master branch, and create a branch.
  3. Run the pandoc command to convert the .xml file to .rst. For example:
    pandoc -f docbook -t rst -s ch_compute_focus.xml -o compute_focus.rst
  4. Move converted files to the <guide>/source folder.
  5. Rename converted RST files and ensure they are titled to match the contents of the file, such as "Networking API 2.0 Overview" to networking_api_2.0_overview" for example.
  6. Use a text editor to complete the following tasks:
    1. Clean up tables where the pandoc conversion just outputs paragraphs.
    2. Remove numbering from Example titles and Table titles.
    3. Remove "programlisting" "screen" and "literallayout" from .. code:: lines.
    4. Check the converted RST file follows conventions described in the OpenStack Documentation Contributor Guide
  7. Commit your changes, including the related blueprint in your commit message. For example:
    Convert ch_compute_focus.xml to RST
  Implements: blueprint archguide-mitaka-rst

Completing conversion to RST

Note that this includes links to how it was done for the Security Guide as reference.

Once we have reviewed the draft guide and think it's ready to publish, we need to do the following steps:

  1. Create a patch with the following steps: https://review.openstack.org/211766
    1. Delete old guide, here security-guide
    2. Move RST guide to location of new guide (security-guide-rst -> security-guide).
    3. Update tools/build-all-rst.sh for the change
    4. If the repository has no further DocBook guides in it, update tox.ini
    5. Update doc-tools-test-languages.conf
    6. Rename localization files to new directory name
    7. If there is a ".tx/config" file, update it (remove rst guide, update paths)
  2. If the repository has no further DocBook guides in it:
  3. Update .gitignore: https://review.openstack.org/212685
  4. If the repository is not the openstack-manuals repo, stop syncing of XML files: https://review.openstack.org/211842 and remove the copied files once that patch is merged: https://review.openstack.org/211904
  5. Fix links on docs.openstack.org and in other guides: https://review.openstack.org/212044
    • Create redirects from content directory to top-level index.html file
    • Update all links so that they go to new guide (security-guide, not security-guide/content)
    • Remove links to PDF of guide
  6. Sync translations from old guide with new guide (needs to be done in transifex by Andreas)
  7. Tell i18n team that conversion is finished and which resource is active.
  8. Remove old guide and draft RST guides from docs.openstack.org (needs docs.openstack.org admin access)
  9. Regenerate sitemap.xml after all changes are in: https://review.openstack.org/212689 and blacklist the change: https://review.openstack.org/212690