I18n/Tools

I18N uses many different tools, across several OpenStack Projects to meet its goals.

Tools related with document translation

OpenStack documents are in DocBook format. In order to make the document translation management consistent with message translation, we need to convert DocBook into PO format.

The document translation management include following steps:

• Slicing: slice DocBook into string segment with a Python script, and generate a PO template
• Uploading: Upload the translation resources to Zanata by automation scripts
• Translating: manage the translation in Zanata including the translation memory and glossary management
• Merging: merge the translated string into Docbook with a python script
• Uploading: Download the translated results by automation scripts.

Slicing script

The slicing script has dependency to package "xml2po". So you have to install gnome-doc-utils before running it. On Ubuntu, do:

 sudo apt-get install gnome-doc-utils

or on openSUSE:

 sudo zypper install xml2po

The script can scan a specific folder under openstack-manuals/doc/src/docbkx, collect all the text segment in *.xml and combine them into a PO template, with command:

./tools/generatepot [foldername]

The PO template will be put in a "locale" folder under the specific docbook folder, i.e. openstack-manuals/doc/src/docbkx/[bookname]/locale.

Merging script

The merging script has dependency to package "xml2po". So you have to install gnome-doc-utils before running it. On Ubuntu, do:

 sudo apt-get install gnome-doc-utils

or on openSUSE:

 sudo zypper install xml2po

The script can merge the translation strings into DocBook and generate a DocBook in the specific language.

tox -e buildlang -- $LANG

Uploading script

The uploading script is an automation job running in Jenkins, which will be triggered by "commit" event of openstack-manuals repository. See here.

Downloading script

The downloading script is an automation job running daily in Jenkins. See here.

Tools related with RST document translation

Some of OpenStack documents are using RST format. The steps to translate RST documents are similar with DocBook documents, including:

• Slicing: generate PO templates from RST documents
• Uploading: Upload the translation resources to Zanata
• Translating: manage the translation in Zanata, including the translation memory and glossary management
• Downloading: Download the translated results by automation scripts.
• Building: Build HTML from RST documents and the translated results.

Sphinx is a tool to translate RST source files to various output formats, including POT and HTML. You need to install Sphinx before you go to below steps.

pip install Sphinx

All of this is automated from Jenkins jobs.

Slicing

We use sphinx-build to translate RST files to POT files. Because we want to have a single POT file per document, we use msgcat to merge those POTs after sphinx-build.

sphinx-build -b gettext doc/[docname]/source/ doc/[docname]/source/locale/
msgcat doc/[docname]/source/locale/*.pot > doc/[docname]/source/locale/[docname].pot

Uploading

We use Zanata client to upload the POT file to the Zanata instance.

Downloading

We use Zanata client to download the translated PO files from Zanata instance.

Building

Before use sphinx-build to build HTML file, we need to feed the translations from the single PO file into those small PO files. For example:

msgmerge -o doc/[docname]/source/locale/zh_CN/LC_MESSAGES/A.po doc/[docname]/source/locale/zh_CN/LC_MESSAGES/[docname].po doc/[docname]/source/locale/A.pot

Then, for evey PO file, we should execute the following command to build into MO file:

msgfmt "doc/[docname]/source/locale/zh_CN/LC_MESSAGES/A.po" -o "doc/[docname]/source/locale/zh_CN/LC_MESSAGES/A.mo"

Finally, we could generate HTML files by

sphinx-build -D "language='zh_CN' doc/[docname]/source/ doc/[docname]/build/html

Tools related with message translation

For most of the Python projects, the preferred tools for I18N are gettext and babel. The gettext module provides internationalization (I18N) and localization (L10N) services for your Python modules and applications. Babel are a collection of tools for internationalizing Python applications.

You can extract the messages in code to PO template with pybabel:

pybabel extract -o nova/locale/nova.pot nova/

Uploading script

For each Python project in OpenStack, there is an automation job to extract the messages , generate PO template and upload to Zanata, which is triggered by the "commit" event. See here.

Downloading script

For each Python project in OpenStack, there is an automation job daily to download the translations in po file to the "locale" folder under the source folder of each project. See here. It will generate a review request in Gerrit. After review, the translation in po file will be merged.