Translations/Infrastructure - Revision history

Ian Y. Choi at 16:03, 19 November 2018

2018-11-19T16:03:30Z

Ian Y. Choi: Adds a caution message

2018-11-19T16:00:29Z

Adds a caution message

StevenK: Update URLs for Zanata push/pull scripts to openstack-zuul-jobs / prepare-zanata-client.

2018-11-15T09:02:24Z

Update URLs for Zanata push/pull scripts to openstack-zuul-jobs / prepare-zanata-client.

Ian Y. Choi: Updates translation percentage and common_translation_update.sh

2016-12-03T01:11:14Z

Updates translation percentage and common_translation_update.sh

Ian Y. Choi: /* Translation percentage */

2016-10-10T16:20:42Z

‎Translation percentage

Ian Y. Choi: Explicitly described percent values

2016-10-10T16:20:01Z

Explicitly described percent values

Jaegerandi: /* New proposal */

2015-10-19T07:09:10Z

‎New proposal

Jaegerandi: /* New proposal */

2015-10-07T18:14:00Z

‎New proposal

Jaegerandi: /* New proposal */

2015-10-07T18:07:49Z

‎New proposal

Jaegerandi: /* New proposal */

2015-10-07T18:06:50Z

‎New proposal

← Older revision		Revision as of 16:03, 19 November 2018
Line 1:		Line 1:

−	{{caution\|This page contains the mix of current and old information on Translation infrastructure with old Transifex. Please visit [https://docs.openstack.org/i18n/latest/infra.html Translation infrastructure] page for up-to-date information.}}	+	{{caution \| header=Caution\|body=This page contains the mix of current and old information on Translation infrastructure with old Transifex. Please visit [https://docs.openstack.org/i18n/latest/infra.html Translation infrastructure] page for up-to-date information.}}

	= Translation Infrastructure workflow and setup =		= Translation Infrastructure workflow and setup =

← Older revision		Revision as of 16:00, 19 November 2018
Line 1:		Line 1:
		+
		+	{{caution\|This page contains the mix of current and old information on Translation infrastructure with old Transifex. Please visit [https://docs.openstack.org/i18n/latest/infra.html Translation infrastructure] page for up-to-date information.}}
		+
	= Translation Infrastructure workflow and setup =		= Translation Infrastructure workflow and setup =

@@ Line 43: / Line 43: @@
 The process is as follows :
 # A patch to $PROJECT merges to git (this applies to each and every patch)
-# A post job called $PROJECT-upstream-translation-update is run that calls [http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/scripts/upstream_translation_update.sh project-config/jenkins/scripts/upstream_translation_update.sh]
+# A post job called $PROJECT-upstream-translation-update is run that calls [http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/roles/prepare-zanata-client/files/upstream_translation_update.sh upstream_translation_update.sh]
 # The script extracts any string that is [[#Marked for translation|marked for translation]] and updates the pot files in $PROJECT/locale/
 # The script sends these changes to the Zanata instance
@@ Line 50: / Line 50: @@
 Every morning at 6:00 UTC our periodic scripts are executed:
-# The job $PROJECT-propose-translation-update is executed, it checks out the repository and calls the script [http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/scripts/propose_translation_update.sh project-config/jenkins/propose_translation_update.sh].
+# The job $PROJECT-propose-translation-update is executed, it checks out the repository and calls the script [http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/roles/prepare-zanata-client/files/propose_translation_update.sh propose_translation_update.sh].
 # The script connects to Zanata and asks to download for each resource all new "sufficiently translated" files. We define "sufficiently" as at least '''X''' percent of strings are translated.
 # The script connects to Zanata and asks to download updates for each existing resource.

@@ Line 1: / Line 1: @@
 = Translation Infrastructure workflow and setup =
-The OpenStack Infrastructure team uses a series of scripts to interact with  [https://translate.openstack.org/ Zanata] in order to manage translations changes. This document explains the infrastructure workflow and scripts used to accomplish this.
+The OpenStack Infrastructure team uses a series of scripts to interact with  [https://translate.openstack.org/ Zanata], a translation platform in order to manage translations changes. This document explains the infrastructure workflow and scripts used to accomplish this.
 These scripts are run after each change of a project to push changes to Zanata. Every day at 6:00 UTC, the "OpenStack Proposal Bot" imports these changes with subject "Imported Translations from Zanata".
@@ Line 33: / Line 33: @@
 == Translation percentage ==
-All translations are stored in Zanata, the copies in $PROJECT are just copies.For consistency, we want a translation file that is sufficiently translated, there's no sense if one out of 5 strings is translated and the rest is in English. Therefore, we defined three type of percentages:
+All translations are stored in Zanata, the copies in $PROJECT are just copies.For consistency, we want a translation file that is sufficiently translated, there's no sense if one out of 5 strings is translated and the rest is in English. Therefore, we defined type of percentages with I18n team, and the percentage values are described in http://docs.openstack.org/developer/i18n/infra.html#translation-jobs .
 == Workflow ==
@@ Line 88: / Line 80: @@
 == Horizon ==
-Horizon and django_openstack_auth use their own scripts ([http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/scripts/propose_translation_update_django_openstack_auth.sh propose_translation_update_django_openstack_auth.sh], [http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/scripts/propose_translation_update_horizon.sh propose_translation_update_horizon.sh], upstream_translation_django_openstack_auth.sh, upstream_translation_horizon.sh) for the translation process which basically do the same as mentioned above.
 == Documentation projects ==
-The process is the same as for the Python projects but separate scripts are used ([http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/scripts/propose_translation_update_manuals.sh propose_translation_update_manuals.sh] and [http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/scripts/upstream_translation_update_manuals.sh upstream_translation_update_manuals.sh]). Documentation files do not mark strings, all strings are extracted. We have special scripts that extract all strings from the DocBook and RST files and place them in pot files. The extraction is done by tools/generatepot which lives in each documentation repository. There is also a script openstack-generate-docbook (created by [http://git.openstack.org/cgit/openstack/openstack-doc-tools/tree/os_doc_tools/handle_pot.py os_doc_tools/handle_pot.py] when you run `python setup.py install`) that is used to merge strings back into DocBook XML.
+For documentation repositories like api-site, ha-guide, openstack-manuals, operations-guide, and security-guide, a separate project is used as well.
 All documentation projects with translations have also a separate Jenkins check which builds the localized manuals - and these are also published on docs.openstack.org.
-For building localized documents, the script `doc-tools-check-languages` is used, it lives in the openstack-doc-tools repository. It is configured via `doc-tools-check-languages.conf`. The syncing scripts to our translation server also parse this file.
+For building localized documents, the script `doc-tools-check-languages` is used, it lives in the [http://git.openstack.org/cgit/openstack/openstack-doc-tools openstack-doc-tools] repository. It is configured via `doc-tools-check-languages.conf`. The syncing scripts to our translation server also parse this file.
 In `doc-tools-check-languages.config`, the array `SPECIAL_BOOKS` is used by the syncing scripts. The other values are used by the `doc-tools-check-languages`:
@@ Line 208: / Line 208: @@
 Note that after the RC2 merge, there is no automatic copying done from stable branch to master. Translation teams need to translate strings in both master and stable, if they choose to continue translating stable. If a merge is needed, it needs to be requested by the translation team for a specific project.
-Note that stable branches will only be created  for repositories that have already at least *one* translation of the main file. Repositories with only translated log files or with no translations at all are not branched. We consider a file translated if at least ''X'' percent of its messages are translated.
+Note that stable branches will only be created for repositories that have already at least *one* translation of the main file. Repositories with only translated log files or with no translations at all are not branched. We consider a file translated if at least ''X'' percent of its messages are translated.
 Note: Around RC1 time, a patch to clean up "old" translation files gets proposed to master - and stable branch if already cut - to remove any translations in git that are less than '''Y''' % translated (the infra jobs only deletes files from git less than '''Y''' % to avoid to frequent deletion).
 ----
 [[Category:i18n]]

@@ Line 35: / Line 35: @@
 All translations are stored in Zanata, the copies in $PROJECT are just copies.For consistency, we want a translation file that is sufficiently translated, there's no sense if one out of 5 strings is translated and the rest is in English. Therefore, we defined three type of percentages:
-*  only download new files that are at least '''X''' percent translated and if files grow over time
+*  Only download new files that are at least '''X''' percent translated and if files grow over time
 *  However if there is no new translations (or strings change too much), we remove them again from the project with a threshold of '''Y''' percent. This reduces the amount of patches that go into a project. Without these changes, each day a large patch would go in.
-*  there is also a lower threshold for releases of '''Z''' percent of messages translated as policy - which is only manually enforced.
+*  There is also a lower threshold for releases of '''Z''' percent of messages translated as policy - which is only manually enforced.

@@ Line 30: / Line 30: @@
 UI projects like horizon and documentation projects have different documents to translate.
 == Workflow ==
@@ Line 47: / Line 59: @@
 # The job $PROJECT-propose-translation-update is executed, it checks out the repository and calls the script [http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/scripts/propose_translation_update.sh project-config/jenkins/propose_translation_update.sh].
-# The script connects to Zanata and asks to download for each resource all new "sufficiently translated" files. We define "sufficiently" as at least 75 per cent of strings are translated.
+# The script connects to Zanata and asks to download for each resource all new "sufficiently translated" files. We define "sufficiently" as at least '''X''' percent of strings are translated.
 # The script connects to Zanata and asks to download updates for each existing resource.
 # The script generates the pot files for the project since the pot files are outdated (upstream_translation_update does not touch the git repo)
 # The script merges translations and pot files
-# The script removes translation files that have no strings anymore or are less than 20 per cent translated (See: [[#Translation percentage changes|Translation percentage changes]])
+# The script removes translation files that have no strings anymore or are less than '''Y''' percent translated
 # The script filters the translated files and does some heuristics to not update too often (for example, if a project has no update for translated files, then it does not update the pot files)
 # The script removes location information and empty strings from the language files.
@@ Line 74: / Line 86: @@
 * For logging, it's <code>LOG.info(_LI("info message"))</code> - and there's also <code>_LE</code> for <code>LOG.error</code>, <code>_LW</code> for <code>LOG.warning</code>, <code>_LC</code> for <code>LOG.critical</code>
 * Normal strings like <code>print("Do not translate")</code> are not translated
 == Horizon ==
@@ Line 88: / Line 96: @@
 All documentation projects with translations have also a separate Jenkins check which builds the localized manuals - and these are also published on docs.openstack.org.
 For building localized documents, the script `doc-tools-check-languages` is used, it lives in the openstack-doc-tools repository. It is configured via `doc-tools-check-languages.conf`. The syncing scripts to our translation server also parse this file.
@@ Line 110: / Line 116: @@
 ''Goal:''
-* Updated translations of files that are sufficiently translate (> 75 %)
+* Updated translations of files that are sufficiently translate (> '''X''' %)
 * Updated po source file that reflect state of repository
-* No empty/mainly untranslated language files (< 66 % is rule of thumb)
+* No empty/mainly untranslated language files (< '''Z''' % is rule of thumb)
 === Project list ===
@@ Line 185: / Line 191: @@
 #. Open master version to accept translations.
-Note: Around RC1 time, a patch to cleanup "old" translations gets proposed to both stable and master to remove any translations in git that are less than 66 % translated (we normally only delete less than 20 % to avoid to frequent deletion).
+Note: Around RC1 time, a patch to cleanup "old" translations gets proposed to both stable and master to remove any translations in git that are less than '''Z''' % translated (we normally only delete less than '''Y''' % to avoid to frequent deletion).
 === New proposal ===
@@ Line 202: / Line 208: @@
 Note that after the RC2 merge, there is no automatic copying done from stable branch to master. Translation teams need to translate strings in both master and stable, if they choose to continue translating stable. If a merge is needed, it needs to be requested by the translation team for a specific project.
-Note that stable branches will only be created  for repositories that have already at least *one* translation of the main file. Repositories with only translated log files or with no translations at all are not branched. We consider a file translated if at least 75 per cent of its messages are translated.
+Note that stable branches will only be created  for repositories that have already at least *one* translation of the main file. Repositories with only translated log files or with no translations at all are not branched. We consider a file translated if at least ''X'' percent of its messages are translated.
-Note: Around RC1 time, a patch to clean up "old" translation files gets proposed to master - and stable branch if already cut - to remove any translations in git that are less than 66 % translated (the infra jobs only deletes files from git less than 20 % to avoid to frequent deletion).
+Note: Around RC1 time, a patch to clean up "old" translation files gets proposed to master - and stable branch if already cut - to remove any translations in git that are less than '''Y''' % translated (the infra jobs only deletes files from git less than '''Y''' % to avoid to frequent deletion).
 ----
 [[Category:i18n]]

← Older revision		Revision as of 07:09, 19 October 2015
Line 188:		Line 188:

	=== New proposal ===		=== New proposal ===
		+
		+	The following proposal needs buy-in from stable maintainer team and I18n team:

	# Before RC1 is cut, translators work with master version.		# Before RC1 is cut, translators work with master version.

@@ Line 189: / Line 189: @@
 === New proposal ===
-#. Before RC1 is cut, translators work with master version.
+# Before RC1 is cut, translators work with master version.
-#. When stable branch is cut, branches get created in Zanata as well. Syncing to projects will get enabled for them.
+# When stable branch is cut, branches get created in Zanata as well. Syncing to projects will get enabled for them.
-#. Translators focus on translation of stable branch. Master branch is made read-only.
+# Translators focus on translation of stable branch. Master branch is made read-only.
-#. The release manager will cut a RC2 including the current state of translators work, RC2 is normally cut 10 days after RC1.
+# The release manager will cut a RC2 including the current state of translators work, RC2 is normally cut 10 days after RC1.
-#. Once RC2 is created, a copy of new translations from stable branch to master is done in Zanata.
+# Once RC2 is created, a copy of new translations from stable branch to master is done in Zanata.
-#. Master branch is made writeable and translation can now be done for both stable branch and master.
+# Master branch is made writeable and translation can now be done for both stable branch and master.
-#. Translations of stable branch will continue to be synced and translators can continue translation.
+# Translations of stable branch will continue to be synced and translators can continue translation.
-#. Briefly before RC1 of next stable release is cut, the stable branch in Zanata is closed. So, there is always a single stable branch (latest release) and the master to translate on.
+# Briefly before RC1 of next stable release is cut, the stable branch in Zanata is closed. So, there is always a single stable branch (latest release) and the master to translate on.
 Note that after the RC2 merge, there is no automatic copying done from stable branch to master. Translation teams need to translate strings in both master and stable, if they choose to continue translating stable. If a merge is needed, it needs to be requested by the translation team for a specific project.

@@ Line 196: / Line 196: @@
 #. Master branch is made writeable and translation can now be done for both stable branch and master.
 #. Translations of stable branch will continue to be synced and translators can continue translation.
-#. Alternatives:
+#. Open Question when to close the stable branch for translation, two alternative suggestions:
 #* Once stable release team cuts the last release for the stable branch, the stable branch in Zanata is closed.
 #* Briefly before RC1 of next stable release is cut, the stable branch in Zanata is closed. So, there is always a single stable branch (latest release) and the master to translate on.