From 118a5511bda97edbf3270cc5e6323a0525ba4911 Mon Sep 17 00:00:00 2001 From: Omar Al-Ithawi Date: Mon, 4 Sep 2023 09:15:04 +0300 Subject: [PATCH 1/7] doc: design for `atlas pull` for the edx-platform and its plugins --- .../0019-oep-58-atlas-translations-design.rst | 282 ++++++++++++++++++ 1 file changed, 282 insertions(+) create mode 100644 docs/decisions/0019-oep-58-atlas-translations-design.rst diff --git a/docs/decisions/0019-oep-58-atlas-translations-design.rst b/docs/decisions/0019-oep-58-atlas-translations-design.rst new file mode 100644 index 000000000000..ee5cc4fef2fc --- /dev/null +++ b/docs/decisions/0019-oep-58-atlas-translations-design.rst @@ -0,0 +1,282 @@ +Design for Refactoring Translations ``pull`` to use Atlas +########################################################## + +Status +====== + +Pending + +Context +======= + +OEP-58 Translation Management overview +-------------------------------------- + +The `Translation Management update OEP-58`_ proposal has been merged with +the following changes to the way translations are managed in Open edX: + +- Move Translation Files to the `openedx-translations repo`_ +- Add `Transifex GitHub App `_ + to openedx Organization +- Connect the `openedx-translations repo`_ to the + `openedx-translations project`_ +- Copy Transifex's Translation Memory and Combine Translators +- Get Translations Back for Deployment/Development and introduce the new + `openedx-atlas`_ translation tool. + +If you're new to the `Translation Management update OEP-58`_ proposal, please +review it in addition to the +`Approach Memo and Technical Discovery - Translations Infrastructure Implementation`_ +document before continuing. + +Current Architecture/Implementation for XBlocks and Plugins +----------------------------------------------------------- +As of now the Open edX XBlocks and Plugins are installed via ``pip`` with +their translations embedded within the Python package. + +Bundling translations helps to ensure its always available. However, it also +means that in order to update translations, a full opensource contribution +process needs to happen. Somtimes the process takes a full month to +complete such as in the example below: + +- `Added French (Canada) and Japanese - xblock-drag-and-drop-v2 #220`_ + + +XBlockI18nService +----------------- + +Once the translations are bundled and the XBlock/Plugin is installed the +XBlockI18nService will be able to find the Python translations and use load +them via the ``__init__`` method of the `XBlockI18nService`_ which finds +the ``text.mo`` files and make it available to the ``edx-platform`` +during the execution of the XBlock. + +The `XBlockI18nService implementation pull request`_ (2016) introduced the +support for XBlock translations in ``edx-platform`` and has the full +context of the implementation. + +JavaScript Translations for XBocks +---------------------------------- + +As of September 2023, there is no centralized method to bundle JavaScript +translations in XBlocks. Non-XBlock plugins lacks JavaScript translations +support altogether. + +The de-factor stadnard method for bundling JavaScript translations in XBlocks +is to use ``web_fragment`` and load the translations as part of the XBlock +frontend static files on every XBlock load. + +The LTI Consumer XBlock embeds the translations in its ``web_fragment`` via +the `LtiConsumerXBlock._get_statici18n_js_url`_ and +`LtiConsumerXBlock.student_view`_ methods. + +This design has it's trade-offs but it's been widely adopted and out of the +scope of this proposal. Therefore, we'll improve on the existing design to +load the ``atlas`` JavaScript translations instead of the bundled ones. + +Decisions +========= + +Proposed Design for edX Platform ``conf/locale`` translations +------------------------------------------------------------- + +Update the ``make pull_translations`` command to use the ``atlas pull`` +if the ``OPENEDX_ATLAS_PULL`` environment variable is set. + +This has been the standard for all repositories as seen in both +`course-discovery atlas integration`_ and +`frontend-app-learning atlas integration`_. + +The updated `edx-platform pull_translations`_ would look like the following:: + + pull_translations: + git clean -fdX conf/locale + ifeq ($(OPENEDX_ATLAS_PULL),) + find conf/locale -type d -mindepth 1 -maxdepth 1 -exec rm -rf {} + # Remove stale translation files + atlas pull $(ATLAS_PULL_ARGS) translations/edx-platform/conf/locale:conf/locale + else + i18n_tool transifex pull + endif + # The commands below are simplified for demonistration purposes + i18n_tool generate --verbose 1 + i18n_tool validate --verbose + paver i18n_compilejs + + +This mostly is a non-controversial change that has been done in other repos +already. + +The next section is a little more intricate and requires more discussion. + +Proposed Design for XBlocks and Plugins +--------------------------------------- + +The proposed design for XBlocks and Plugins is to use the ``atlas pull`` +in a centrally managed way for all XBlocks and Plugins to circumvent the +the need for managing the translations in each XBlock. + +The XBlock translations is already stored in the `openedx-translations repo`_ +and is accessible by the `openedx-atlas`_ command-line. + + +New ``ENABLE_ATLAS_TRANSLATIONS`` Waffle Switch +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The ``ENABLE_ATLAS_TRANSLATIONS`` switch will be disabled by default. +It will be used by the ``XBlockI18nService`` to determine which translations +to use for XBlocks until the `OEP-58`_ is fully implemented and +the non-atlas translations are removed. + +The non-XBlock plugins such as `edx-val`_ are out of the scope of this +proposal and will be handled separately as stated in the :ref:`non-goals` +section. + +New ``pull_plugins_translations`` command +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Introduce new Django commands to the ``edx-platform``: + +- ``manage.py lms pull_plugins_translations --list-only``: This command + will list all the XBlocks and Plugins that are installed in the + ``edx-platform`` virtual environment regardless of whether its run + in Docker, devstack or Native Installation. + + If the command is executed with the ``--list-only`` flag it will print a + list of Python *module names* (as opposed to git repository names) of the + installed XBlocks and Plugins e.g.:: + + $ manage.py lms atlas_pull_plugins_translations --list-only + drag_and_drop_v2 + done + eox_tenant + + This list doesn't include plugins that are bundled within the + ``edx-platform`` repository itself such as the Video XBlock, the ``capa`` + module and others. The reason for this is that their translations are + already included in the ``edx-platform`` translations. + +- ``manage.py lms atlas_pull_plugins_translations``: This command + will craft and executes the ``atlas pull`` command for the XBlocks and + Plugins listed in the previous command. This command is will be added + to the ``Makefile`` and can be executed for both development and production + deployments. + + This command will run and ``atlas pull`` with the arguments below to pull + the translations by module name:: + + $ atlas pull \ + translations/edx-platform-links/drag_and_drop_v2:conf/plugins-locale/drag_and_drop_v2 \ + translations/edx-platform-links/done:conf/plugins-locale/done \ + translations/edx-platform-links/eox_tenant:conf/plugins-locale/eox_tenant + + **Note:** The command above is for demonstration purposes and may not work + properly yet. + +Changes to the `openedx-translations repo`_ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The `openedx-translations repo`_ organizes the translations by the +GitHub repository name. However, once an XBlock is installed the repository +name is no longer known to the ``edx-platform``. Therefore, we provide two +ways to fetch the XBlock translations: + +- By repo name: e.g. `translations/xblock-drag-and-drop-v2 directory`_. +- By module name: e.g. + `translations/edx-platform-links/drag_and_drop_v2 directory`_. + +This update is already implemented in the `openedx-translations repo`_ as of +`edx-platform-links PR #353`_ which includes the details of the changes. + + +BlockI18nService support for ``atlas`` Python translations +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +**TBD:** + +XBlockI18nService support for ``atlas`` JavaScript translations +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +**TBD:** + +Dismissed Proposals +=================== + + +XBlocks and plugins have their own "atlas pull" command +------------------------------------------------------- + +This dismissed proposal intends to have each XBlock and Plugin have their +own ``make pull_translations`` and be responsible for managing pulling their +own translations from the `openedx-translations repo`_. + +This proposal has been dismissed because it would require substantial work +to get into the details for the ``lib/python3.8/site-packages/`` directory +and ensure that the ``make pull_translations`` command won't corrupt the +virtual environment. + +This is a non-trivial task and appears to add more complexity than necessary +for not much added benefit. + + +Goals +===== +#. Use ``atlas pull`` for the ``edx-platform`` repo. +#. Use ``atlas pull`` for the XBlocks and Plugins. +#. Allow Tutor and other advanced uses to craft their own ``atlas pull`` + commands by making the the plugins list available via Django commands. +#. Allow ``atlas pull`` to use the Python module names instead of the + repository name of XBlocks and Plugins. This is already done in the + `openedx-translations repo`_ via the + ``extract-translation-source-files.yml``_ as described in the + `edx-platform translations links`_ document. + +.. _non-goals: + +Non-Goals +========= + +The following are non-goals for this proposal, although some are going to +be tackled in the future as part of the +`Translation Management update OEP-58`_ proposal. + +#. Provide a fool-proof method for managing named-release translations. + This will be a separate discussion. +#. Discuss the merge/segment strategy of the ``edx-platform``. This is being + discussed in the + `decision no. 0018 `_. +#. Design a new XBlock frontend architecture. Instead this proposal works + with the existing architecture. +#. Provide a new translation method for theme translations. This will be + tackled later on. +#. Provide a new translation method for non-XBlock plugins such as + ``edx-val``. This will be tackled later on as part of the `OEP-58`_ + proposal. + +.. _Translation Management update OEP-58: https://open-edx-proposals.readthedocs.io/en/latest/architectural-decisions/oep-0058-arch-translations-management.html#specification +.. _OEP-58: https://open-edx-proposals.readthedocs.io/en/latest/architectural-decisions/oep-0058-arch-translations-management.html#specification +.. _openedx-atlas: https://github.com/openedx/openedx-atlas +.. _openedx-translations repo: https://github.com/openedx/openedx-translations +.. _extract-translation-source-files.yml: https://github.com/openedx/openedx-translations/blob/2566e0c9a30d033e5dd8d05d4c12601c8e37b4ef/.github/workflows/extract-translation-source-files.yml#L36-L43 +.. _openedx-translations project: https://app.transifex.com/open-edx/openedx-translations/dashboard/ + +.. _Approach Memo and Technical Discovery - Translations Infrastructure Implementation: https://docs.google.com/document/d/11dFBCnbdHiCEdZp3pZeHdeH8m7Glla-XbIin7cnIOzU/edit +.. _Added French (Canada) and Japanese - xblock-drag-and-drop-v2 #220: https://github.com/openedx/xblock-drag-and-drop-v2/pull/220 +.. _edx-platform translations links: https://github.com/openedx/openedx-translations/tree/main/translations/edx-platform-links +.. _XBlockI18nService: https://github.com/openedx/edx-platform/blob/6e28ba329e0a5354d7264ea834861bf0cae4ceb3/xmodule/modulestore/django.py#L359-L395 +.. _XBlockI18nService implementation pull request: https://github.com/openedx/edx-platform/pull/11575/files#diff-0bbcc6c13d9bfc9d88fbe2fdf4fd97f6066a7a0f0bfffb82bc942378b7cf33e0R248 + +.. _course-discovery atlas integration: https://github.com/openedx/course-discovery/pull/4037 +.. _frontend-app-learning atlas integration: https://github.com/openedx/frontend-app-learning/pull/1093 +.. _edx-platform pull_translations: https://github.com/openedx/edx-platform/blob/0137881b8199701b2af7d07c9a01200e358e3d86/Makefile#L55-L64 + +.. _drag-and-drop-v2 xblock: https://github.com/openedx/xblock-drag-and-drop-v2/ +.. _LTI Consumer XBlock: https://github.com/openedx/xblock-lti-consumer/ +.. _edx-val: https://github.com/openedx/edx-val + +.. _LtiConsumerXBlock._get_statici18n_js_url: https://github.com/openedx/xblock-lti-consumer/blob/7a142310a78ac393286c1e9e77c535ea520ab90b/lti_consumer/lti_xblock.py#L663-L677 +.. _LtiConsumerXBlock.student_view: https://github.com/openedx/xblock-lti-consumer/blob/7a142310a78ac393286c1e9e77c535ea520ab90b/lti_consumer/lti_xblock.py#L1215C24-L1217 + + +.. _edx-platform-links PR #353: https://github.com/openedx/openedx-translations/pull/353 +.. _translations/xblock-drag-and-drop-v2 directory: https://github.com/openedx/openedx-translations/tree/8a01424fd8f42e9e76aed34e235c82ab654cdfc5/translations/xblock-drag-and-drop-v2 +.. _translations/edx-platform-links/drag_and_drop_v2 directory: https://github.com/openedx/openedx-translations/blob/8a01424fd8f42e9e76aed34e235c82ab654cdfc5/translations/edx-platform-links/drag_and_drop_v2 From 54d589290b3b12a573b8c527a2d9163843d5e517 Mon Sep 17 00:00:00 2001 From: Omar Al-Ithawi Date: Tue, 28 Mar 2023 16:43:23 +0300 Subject: [PATCH 2/7] feat: experimental oep-58 i18n service updates This adds support for including i18n service files for XBlocks that are pulled with atlas. References ---------- This pull request is part of the [FC-0012 project](https://openedx.atlassian.net/l/cp/XGS0iCcQ) which is sparked by the [Translation Infrastructure update OEP-58](https://open-edx-proposals.readthedocs.io/en/latest/architectural-decisions/oep-0058-arch-translations-management.html#specification). Check the links above for full information about the overall project. --- .gitignore | 1 + Makefile | 11 ++- cms/envs/production.py | 2 + common/djangoapps/xblock_django/api.py | 9 +++ .../commands/pull_plugins_translations.py | 71 ++++++++++++++++++ .../0019-oep-58-atlas-translations-design.rst | 6 +- xmodule/modulestore/django.py | 73 +++++++++++++++---- xmodule/waffle.py | 20 +++++ 8 files changed, 174 insertions(+), 19 deletions(-) create mode 100644 common/djangoapps/xblock_django/management/commands/pull_plugins_translations.py create mode 100644 xmodule/waffle.py diff --git a/.gitignore b/.gitignore index d6f1202bf4ce..5ac88bad430e 100644 --- a/.gitignore +++ b/.gitignore @@ -61,6 +61,7 @@ conf/locale/fake*/LC_MESSAGES/*.po conf/locale/fake*/LC_MESSAGES/*.mo # this was a mistake in i18n_tools, now fixed. conf/locale/messages.mo +conf/plugins-locale/ ### Testing artifacts .testids/ diff --git a/Makefile b/Makefile index 8664111ebefb..5eba534e59cb 100644 --- a/Makefile +++ b/Makefile @@ -43,7 +43,16 @@ extract_translations: ## extract localizable strings from sources push_translations: ## push source strings to Transifex for translation i18n_tool transifex push -pull_translations: ## pull translations from Transifex +pull_xblock_translations: +ifeq ($(OPENEDX_ATLAS_PULL),) + @echo "atlas is not used due to empty OPENEDX_ATLAS_PULL environment variable" +else + rm -rf conf/xblocks/locale + mkdir conf/xblocks/locale + python manage.py lms xblocks_atlas_pull_module +endif + +pull_translations: pull_xblock_translations ## pull translations from Transifex git clean -fdX conf/locale i18n_tool transifex pull i18n_tool extract diff --git a/cms/envs/production.py b/cms/envs/production.py index d04dfcd8acc0..760bd3f9522f 100644 --- a/cms/envs/production.py +++ b/cms/envs/production.py @@ -262,6 +262,8 @@ def get_env_setting(setting): # ], PREPEND_LOCALE_PATHS = ENV_TOKENS.get('PREPEND_LOCALE_PATHS', []) +PLUGINS_TRANSLATIONS_ROOT = REPO_ROOT / 'conf/plugins-locale' + #Timezone overrides TIME_ZONE = ENV_TOKENS.get('CELERY_TIMEZONE', CELERY_TIMEZONE) diff --git a/common/djangoapps/xblock_django/api.py b/common/djangoapps/xblock_django/api.py index 163683ab0c38..19dd4a8b74ef 100644 --- a/common/djangoapps/xblock_django/api.py +++ b/common/djangoapps/xblock_django/api.py @@ -2,6 +2,7 @@ API methods related to xblock state. """ +import pkg_resources from openedx.core.lib.cache_utils import CacheInvalidationManager from common.djangoapps.xblock_django.models import XBlockConfiguration, XBlockStudioConfiguration @@ -27,6 +28,14 @@ def disabled_xblocks(): return XBlockConfiguration.objects.current_set().filter(enabled=False) +def get_xblocks_entry_points(): + return [entry_point for entry_point in pkg_resources.iter_entry_points(group='xblock.v1')] + +def get_xblocks(): + return [entry_point.resolve() for entry_point in get_xblocks_entry_points()] + + + def authorable_xblocks(allow_unsupported=False, name=None): """ This method returns the QuerySet of XBlocks that can be created in Studio (by default, only fully supported diff --git a/common/djangoapps/xblock_django/management/commands/pull_plugins_translations.py b/common/djangoapps/xblock_django/management/commands/pull_plugins_translations.py new file mode 100644 index 000000000000..c2d10022cfd5 --- /dev/null +++ b/common/djangoapps/xblock_django/management/commands/pull_plugins_translations.py @@ -0,0 +1,71 @@ +""" +This command downloads the translations for the XBlocks that are installed. +""" +import shutil +import subprocess +import os.path + +from django.conf import settings + +from django.core.management.base import BaseCommand + +from ...api import get_xblocks_entry_points + + +class Command(BaseCommand): + + def add_arguments(self, parser): + parser.add_argument( + '--verbose|-v', + action='store_true', + default=False, + dest='verbose', + help='Verbose output.' + ) + + parser.add_argument( + '--list|-l', + action='store_true', + default=False, + dest='list', + help='List plugins module names.' + ) + + def handle(self, *args, **options): + # Remove previous translations + for dir_name in os.listdir(settings.PLUGINS_TRANSLATIONS_ROOT): + dir_path = os.path.join(settings.PLUGINS_TRANSLATIONS_ROOT, dir_name) + + if os.path.isdir(dir_path): + shutil.rmtree(dir_path, ignore_errors=True) + + xblock_module_names = set() + for entry_point in get_xblocks_entry_points(): + xblock = entry_point.resolve() + module_import_path = xblock.__module__ + parent_module, _rest = module_import_path.split('.', maxsplit=1) + if parent_module == 'xmodule': + if options['verbose']: + self.stdout.write(f'INFO: Skipped edx-platform XBlock "{entry_point.name}" ' + f'in module={module_import_path}') + else: + xblock_module_names.add(parent_module) + + sorted_xblock_module_names = list(sorted(xblock_module_names)) + + if options['list']: + self.stdout.write('\n'.join(sorted_xblock_module_names)) + else: + xblock_atlas_args_list = [ + f'translations/edx-platform-plugins/{module_name}:{module_name}' + for module_name in sorted_xblock_module_names + ] + + subprocess.run( + [ + 'atlas', 'pull', + '--repository=Zeit-Labs/openedx-translations', + '--branch=plugins-fix' + ] + xblock_atlas_args_list, + cwd=settings.PLUGINS_TRANSLATIONS_ROOT, + ) diff --git a/docs/decisions/0019-oep-58-atlas-translations-design.rst b/docs/decisions/0019-oep-58-atlas-translations-design.rst index ee5cc4fef2fc..728fff0deda7 100644 --- a/docs/decisions/0019-oep-58-atlas-translations-design.rst +++ b/docs/decisions/0019-oep-58-atlas-translations-design.rst @@ -136,16 +136,16 @@ New ``pull_plugins_translations`` command Introduce new Django commands to the ``edx-platform``: -- ``manage.py lms pull_plugins_translations --list-only``: This command +- ``manage.py lms pull_plugins_translations --list``: This command will list all the XBlocks and Plugins that are installed in the ``edx-platform`` virtual environment regardless of whether its run in Docker, devstack or Native Installation. - If the command is executed with the ``--list-only`` flag it will print a + If the command is executed with the ``--list`` flag it will print a list of Python *module names* (as opposed to git repository names) of the installed XBlocks and Plugins e.g.:: - $ manage.py lms atlas_pull_plugins_translations --list-only + $ manage.py lms atlas_pull_plugins_translations --list drag_and_drop_v2 done eox_tenant diff --git a/xmodule/modulestore/django.py b/xmodule/modulestore/django.py index 9e78b21a4ce6..4d87f740d216 100644 --- a/xmodule/modulestore/django.py +++ b/xmodule/modulestore/django.py @@ -8,6 +8,7 @@ from importlib import import_module import gettext import logging +from os import path from pkg_resources import resource_filename import re # lint-amnesty, pylint: disable=wrong-import-order @@ -29,6 +30,7 @@ from xmodule.modulestore.draft_and_published import BranchSettingMixin # lint-amnesty, pylint: disable=wrong-import-position from xmodule.modulestore.mixed import MixedModuleStore # lint-amnesty, pylint: disable=wrong-import-position from xmodule.util.xmodule_django import get_current_request_hostname # lint-amnesty, pylint: disable=wrong-import-position +from xmodule.waffle import ENABLE_ATLAS_TRANSLATIONS # We also may not always have the current request user (crum) module available try: @@ -369,8 +371,7 @@ class XBlockI18nService: def __init__(self, block=None): """ Attempt to load an XBlock-specific GNU gettext translator using the XBlock's own domain - translation catalog, currently expected to be found at: - /conf/locale//LC_MESSAGES/.po|mo + translation catalog. If we can't locate the domain translation catalog then we fall-back onto django.utils.translation, which will point to the system's own domain translation catalog This effectively achieves translations by coincidence for an XBlock which does not provide @@ -378,21 +379,63 @@ def __init__(self, block=None): """ self.translator = django.utils.translation if block: - xblock_class = getattr(block, 'unmixed_class', block.__class__) - xblock_resource = xblock_class.__module__ + xblock_domain = 'django' + selected_language = get_language() + + xblock_locale_path = self.get_python_locale_directory(block) + if xblock_locale_path: + try: + self.translator = gettext.translation( + xblock_domain, + xblock_locale_path, + [to_locale(selected_language if selected_language else settings.LANGUAGE_CODE)] + ) + except OSError: + # Fall back to the default Django translator if the XBlock translator is not found. + pass + + @staticmethod + def get_python_locale_directory(block): + """ + Return the XBlock locale directory with support for OEP-58 updated translation infrastructure. + + This function works in two modes: + 1. With ENABLE_ATLAS_TRANSLATIONS enabled it loads OEP-58 external translations. + 2. With ENABLE_ATLAS_TRANSLATIONS disabled it uses the old in-xblock translations which are + typically found at /conf/locale//LC_MESSAGES/.po|mo + + The second mode is going to be eventually deprecated when OEP-58 is fully in use. + """ + xblock_class = getattr(block, 'unmixed_class', block.__class__) + xblock_resource = xblock_class.__module__ + + if ENABLE_ATLAS_TRANSLATIONS.is_enabled(): + xblock_module_name = xblock_resource + xblock_locale_path = path.join(settings.PLUGINS_TRANSLATIONS_ROOT, xblock_module_name, 'conf/locale') + else: xblock_locale_dir = 'translations' xblock_locale_path = resource_filename(xblock_resource, xblock_locale_dir) - xblock_domain = 'text' - selected_language = get_language() - try: - self.translator = gettext.translation( - xblock_domain, - xblock_locale_path, - [to_locale(selected_language if selected_language else settings.LANGUAGE_CODE)] - ) - except OSError: - # Fall back to the default Django translator if the XBlock translator is not found. - pass + + return xblock_locale_path + + @staticmethod + def get_javascript_locale_path(block): + """ + Return the XBlock compiled javascript i18n path with support for OEP-58 updated translation infrastructure. + + This function only works With ENABLE_ATLAS_TRANSLATIONS waffle switch enabled. + """ + xblock_class = getattr(block, 'unmixed_class', block.__class__) + xblock_resource = xblock_class.__module__ + selected_language = get_language() + + if ENABLE_ATLAS_TRANSLATIONS.is_enabled(): + xblock_module_name = xblock_resource + xblock_locale_dir = 'conf/locale/{language}/LC_MESSAGES'.format(language=selected_language) + translations_dir = settings.XBLOCK_TRANSLATIONS_DIRECTORY + xblock_locale_path = path.join(translations_dir, xblock_module_name, xblock_locale_dir, 'text.js') + if path.exists(xblock_locale_path): + return xblock_locale_path def __getattr__(self, name): name = 'gettext' if name == 'ugettext' else name diff --git a/xmodule/waffle.py b/xmodule/waffle.py new file mode 100644 index 000000000000..c224090a393b --- /dev/null +++ b/xmodule/waffle.py @@ -0,0 +1,20 @@ +""" +This module contains configuration settings via waffle flags for the xmodule modulestore. +""" + +from edx_toggles.toggles import WaffleSwitch + +# XModule Namespace +WAFFLE_NAMESPACE = 'xmodule' + +# .. toggle_name: xmodule.enable_atlas_translations +# .. toggle_implementation: WaffleSwitch +# .. toggle_default: False +# .. toggle_description: Waffle switch for loading XBlock translations from external directory in line with to OEP-58. +# .. toggle_use_cases: temporary +# .. toggle_creation_date: 2023-03-28 +# .. toggle_tickets: TODO: add here the docs.openedx.org document link. + +ENABLE_ATLAS_TRANSLATIONS = WaffleSwitch( + f'{WAFFLE_NAMESPACE}.enable_atlas_translations', __name__ +) From 235f264a25a777083a3ca05d5554fd5bf57831d5 Mon Sep 17 00:00:00 2001 From: Omar Al-Ithawi Date: Thu, 7 Sep 2023 11:42:46 +0300 Subject: [PATCH 3/7] doc: finished the proposal --- .../0019-oep-58-atlas-translations-design.rst | 155 +++++++++++++++++- xmodule/modulestore/django.py | 11 +- 2 files changed, 155 insertions(+), 11 deletions(-) diff --git a/docs/decisions/0019-oep-58-atlas-translations-design.rst b/docs/decisions/0019-oep-58-atlas-translations-design.rst index 728fff0deda7..dde56a85c34d 100644 --- a/docs/decisions/0019-oep-58-atlas-translations-design.rst +++ b/docs/decisions/0019-oep-58-atlas-translations-design.rst @@ -55,6 +55,8 @@ The `XBlockI18nService implementation pull request`_ (2016) introduced the support for XBlock translations in ``edx-platform`` and has the full context of the implementation. +.. _js-translations: + JavaScript Translations for XBocks ---------------------------------- @@ -70,6 +72,15 @@ The LTI Consumer XBlock embeds the translations in its ``web_fragment`` via the `LtiConsumerXBlock._get_statici18n_js_url`_ and `LtiConsumerXBlock.student_view`_ methods. +In order to separate the XBlock translations from the platform, it's isolated +in a separate ``gettext`` name space. For example, the Drag and Drop XBlock +namespace is ``DragAndDropI18N``` which is hardcoded in multiple places such +as: + +- `XBlock Makefile compile_translations rule`_ +- `XBlock compiled JavaScript text.js translations`_ +- `XBlock main JavaScript file`_ + This design has it's trade-offs but it's been widely adopted and out of the scope of this proposal. Therefore, we'll improve on the existing design to load the ``atlas`` JavaScript translations instead of the bundled ones. @@ -165,12 +176,63 @@ Introduce new Django commands to the ``edx-platform``: the translations by module name:: $ atlas pull \ - translations/edx-platform-links/drag_and_drop_v2:conf/plugins-locale/drag_and_drop_v2 \ - translations/edx-platform-links/done:conf/plugins-locale/done \ - translations/edx-platform-links/eox_tenant:conf/plugins-locale/eox_tenant - - **Note:** The command above is for demonstration purposes and may not work + translations/edx-platform-links/drag_and_drop_v2/conf/locale:conf/plugins-locale/drag_and_drop_v2 \ + translations/edx-platform-links/done/conf/locale:conf/plugins-locale/done \ + translations/edx-platform-links/edx_proctoring/conf/locale:conf/plugins-locale/edx_proctoring + + + It will pull from the `edx-platform-links`_ directory to create the + following file tree:: + + $ tree conf/plugins-locale/ + conf/plugins-locale/ + ├── done + │ ├── ar + │ │ └── LC_MESSAGES + │ │ └── django.po + │ ├── de + │ │ └── LC_MESSAGES + │ │ └── django.po + │ ├── en + │ │ └── LC_MESSAGES + │ │ └── django.po + │ └── fr_CA + │ └── LC_MESSAGES + │ └── django.po + ├── drag_and_drop_v2 + │ ├── ar + │ │ └── LC_MESSAGES + │ │ └── django.po + │ ├── en + │ │ └── LC_MESSAGES + │ │ └── django.po + │ └── fr_CA + │ └── LC_MESSAGES + │ └── django.po + └── edx_proctoring + ├── ar + │ └── LC_MESSAGES + │ └── djangojs.po + ├── de + │ └── LC_MESSAGES + │ └── djangojs.po + ├── en + │ └── LC_MESSAGES + │ ├── djangojs.po + │ └── django.po + └── fr_CA + └── LC_MESSAGES + └── djangojs.po + + + +**Notes:** + +- The command above is for demonstration purposes and may not work properly yet. +- The directory name may change from ``edx-platform-links`` to + ``edx-platform-modules`` but this is out of the scope of this proposal and + have little to no impact on the rest of the proposal. Changes to the `openedx-translations repo`_ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -191,12 +253,84 @@ This update is already implemented in the `openedx-translations repo`_ as of BlockI18nService support for ``atlas`` Python translations ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -**TBD:** +The ``get_python_locale_directory`` will support two modes: + +#. When ``ENABLE_ATLAS_TRANSLATIONS`` is disabled, the XBlock translations + from the ``pip`` packages will be used such as the + ``lib/python3.8/site-packages/drag_and_drop_v2/translations/ar/LC_MESSAGES/text.po`` + path for the Drag and Drop XBlock. + +#. When ``ENABLE_ATLAS_TRANSLATIONS`` is enabled, the atlas translations will + be used which is located in the ``edx-platform`` in an the git-ignored + ``edx-platform/conf/plugins-locale/drag_and_drop_v2/ar/LC_MESSAGES/text.po`` + path. This file pulled by the ``pull_plugins_translations`` command. + XBlockI18nService support for ``atlas`` JavaScript translations ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -**TBD:** +The ``XBlockI18nService`` will provide a new centralized +``get_javascript_locale_path`` method to get the JavaScript translations +``django.js`` file. + +This function needs to be used by the XBlocks in an opt-in backward-compatible +manner. + +A new ``i18n_js_namespace`` property is needed for the :ref:`compile-js-command` +to generate JavaScript translations in a centrally managed manner for all +XBlocks as described in the :ref:`js-translations` section. + +The ``i18n_js_namespace`` property will eliminate the need to hardcode the +namespace the `XBlock Makefile compile_translations rule`_. + + +For example, the `Drag and Drop XBlock get_static_i18n_js_url`_ will need to +be updated to support both the ``XBlockI18nService`` new +``get_javascript_locale_path`` method and the namespace. + +.. code:: diff + + class DragAndDropBlock(XBlock): + + + i18n_js_namespace = 'DragAndDropI18N' + + @staticmethod + def _get_statici18n_js_url(): + """ + Returns the Javascript translation file for the currently selected language, if any found by + `pkg_resources` + """ + lang_code = translation.get_language() + if not lang_code: + return None + + + # TODO: Make this the default once OEP-58 is implemented. + + if hasattr(self.i18n_service, 'get_javascript_locale_path'): + + atlas_locale_path = self.i18n_service.get_javascript_locale_path() + + if atlas_locale_path: + + return atlas_locale_path + + text_js = 'public/js/translations/{lang_code}/text.js' + country_code = lang_code.split('-')[0] + for code in (translation.to_locale(lang_code), lang_code, country_code): + if pkg_resources.resource_exists(loader.module_name, text_js.format(lang_code=code)): + return text_js.format(lang_code=code) + return None + + +New ``compile_plugins_js_translations`` command +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +This command will loop over XBlock modules that has the ``i18n_js_namespace`` +property and compile the JavaScript translations. + +For example if the Drag and Drop XBlock has the ``i18n_js_namespace`` +the ``compile_plugins_js_translations`` command will execute the following +commands:: + + i18n_tool generate -v # Generate the .mo files + python manage.py compilejsi18n --namespace DragAndDropI18N --output conf/plugins-locale/drag_and_drop_v2/js/ + Dismissed Proposals =================== @@ -275,8 +409,15 @@ be tackled in the future as part of the .. _LtiConsumerXBlock._get_statici18n_js_url: https://github.com/openedx/xblock-lti-consumer/blob/7a142310a78ac393286c1e9e77c535ea520ab90b/lti_consumer/lti_xblock.py#L663-L677 .. _LtiConsumerXBlock.student_view: https://github.com/openedx/xblock-lti-consumer/blob/7a142310a78ac393286c1e9e77c535ea520ab90b/lti_consumer/lti_xblock.py#L1215C24-L1217 +.. _Drag and Drop XBlock get_static_i18n_js_url: https://github.com/openedx/xblock-drag-and-drop-v2/blob/66e8d3517fe8c0db55c1a3907ff253c2a4562a7e/drag_and_drop_v2/drag_and_drop_v2.py#L318-L332 + +.. _XBlock compiled JavaScript text.js translations: https://github.com/openedx/xblock-drag-and-drop-v2/blob/b8ab1ecd9168ab1dba21f994ee4bfedb6a57d11f/drag_and_drop_v2/public/js/translations/tr/text.js#L3 +https://github.com/Zeit-Labs/xblock-drag-and-drop-v2/blob/b8ab1ecd9168ab1dba21f994ee4bfedb6a57d11f/drag_and_drop_v2/public/js/translations/tr/text.js#L3 +.. _XBlock Makefile compile_translations rule: https://github.com/openedx/xblock-drag-and-drop-v2/blob/66e8d3517fe8c0db55c1a3907ff253c2a4562a7e/Makefile#L41 +.. _XBlock main JavaScript file: https://github.com/openedx/xblock-drag-and-drop-v2/blob/b8ab1ecd9168ab1dba21f994ee4bfedb6a57d11f/drag_and_drop_v2/public/js/drag_and_drop.js#L6 .. _edx-platform-links PR #353: https://github.com/openedx/openedx-translations/pull/353 .. _translations/xblock-drag-and-drop-v2 directory: https://github.com/openedx/openedx-translations/tree/8a01424fd8f42e9e76aed34e235c82ab654cdfc5/translations/xblock-drag-and-drop-v2 .. _translations/edx-platform-links/drag_and_drop_v2 directory: https://github.com/openedx/openedx-translations/blob/8a01424fd8f42e9e76aed34e235c82ab654cdfc5/translations/edx-platform-links/drag_and_drop_v2 +.. _edx-platform-links: https://github.com/openedx/openedx-translations/tree/8a01424fd8f42e9e76aed34e235c82ab654cdfc5/translations/edx-platform-links diff --git a/xmodule/modulestore/django.py b/xmodule/modulestore/django.py index 4d87f740d216..87d99e180b88 100644 --- a/xmodule/modulestore/django.py +++ b/xmodule/modulestore/django.py @@ -379,7 +379,11 @@ def __init__(self, block=None): """ self.translator = django.utils.translation if block: - xblock_domain = 'django' + if ENABLE_ATLAS_TRANSLATIONS.is_enabled(): + xblock_domain = 'django' + else: + xblock_domain = 'text' + selected_language = get_language() xblock_locale_path = self.get_python_locale_directory(block) @@ -411,7 +415,7 @@ def get_python_locale_directory(block): if ENABLE_ATLAS_TRANSLATIONS.is_enabled(): xblock_module_name = xblock_resource - xblock_locale_path = path.join(settings.PLUGINS_TRANSLATIONS_ROOT, xblock_module_name, 'conf/locale') + xblock_locale_path = path.join(settings.PLUGINS_TRANSLATIONS_ROOT, xblock_module_name) else: xblock_locale_dir = 'translations' xblock_locale_path = resource_filename(xblock_resource, xblock_locale_dir) @@ -431,9 +435,8 @@ def get_javascript_locale_path(block): if ENABLE_ATLAS_TRANSLATIONS.is_enabled(): xblock_module_name = xblock_resource - xblock_locale_dir = 'conf/locale/{language}/LC_MESSAGES'.format(language=selected_language) translations_dir = settings.XBLOCK_TRANSLATIONS_DIRECTORY - xblock_locale_path = path.join(translations_dir, xblock_module_name, xblock_locale_dir, 'text.js') + xblock_locale_path = path.join(translations_dir, xblock_module_name, selected_language, 'django.js') if path.exists(xblock_locale_path): return xblock_locale_path From 08c01b5057dc15c31ea03626b271443473637cf0 Mon Sep 17 00:00:00 2001 From: Omar Al-Ithawi Date: Wed, 13 Sep 2023 14:07:29 +0300 Subject: [PATCH 4/7] fix: apply suggestions from code review Co-authored-by: Brian Smith <112954497+brian-smith-tcril@users.noreply.github.com> --- .../0019-oep-58-atlas-translations-design.rst | 102 ++++++------------ 1 file changed, 34 insertions(+), 68 deletions(-) diff --git a/docs/decisions/0019-oep-58-atlas-translations-design.rst b/docs/decisions/0019-oep-58-atlas-translations-design.rst index dde56a85c34d..3b670bf83e2f 100644 --- a/docs/decisions/0019-oep-58-atlas-translations-design.rst +++ b/docs/decisions/0019-oep-58-atlas-translations-design.rst @@ -13,16 +13,15 @@ OEP-58 Translation Management overview -------------------------------------- The `Translation Management update OEP-58`_ proposal has been merged with -the following changes to the way translations are managed in Open edX: +the following changes to the way translations are managed in the Open edX platform: - Move Translation Files to the `openedx-translations repo`_ -- Add `Transifex GitHub App `_ +- Add the `Transifex GitHub App `_ to openedx Organization - Connect the `openedx-translations repo`_ to the `openedx-translations project`_ - Copy Transifex's Translation Memory and Combine Translators -- Get Translations Back for Deployment/Development and introduce the new - `openedx-atlas`_ translation tool. +- Utilize `openedx-atlas`_ to pull translations for development/deployment. If you're new to the `Translation Management update OEP-58`_ proposal, please review it in addition to the @@ -61,10 +60,10 @@ JavaScript Translations for XBocks ---------------------------------- As of September 2023, there is no centralized method to bundle JavaScript -translations in XBlocks. Non-XBlock plugins lacks JavaScript translations +translations in XBlocks. Non-XBlock plugins lack JavaScript translation support altogether. -The de-factor stadnard method for bundling JavaScript translations in XBlocks +The de-facto standard method for bundling JavaScript translations in XBlocks is to use ``web_fragment`` and load the translations as part of the XBlock frontend static files on every XBlock load. @@ -91,7 +90,7 @@ Decisions Proposed Design for edX Platform ``conf/locale`` translations ------------------------------------------------------------- -Update the ``make pull_translations`` command to use the ``atlas pull`` +Update the ``make pull_translations`` command to use ``atlas pull`` if the ``OPENEDX_ATLAS_PULL`` environment variable is set. This has been the standard for all repositories as seen in both @@ -122,12 +121,8 @@ The next section is a little more intricate and requires more discussion. Proposed Design for XBlocks and Plugins --------------------------------------- -The proposed design for XBlocks and Plugins is to use the ``atlas pull`` -in a centrally managed way for all XBlocks and Plugins to circumvent the -the need for managing the translations in each XBlock. - -The XBlock translations is already stored in the `openedx-translations repo`_ -and is accessible by the `openedx-atlas`_ command-line. +Instead of storing translation files for each XBlock and Plugin in their respective repositories, +we will use `openedx-atlas`_ to pull them from the `openedx-translations repo`_. New ``ENABLE_ATLAS_TRANSLATIONS`` Waffle Switch @@ -147,42 +142,30 @@ New ``pull_plugins_translations`` command Introduce new Django commands to the ``edx-platform``: -- ``manage.py lms pull_plugins_translations --list``: This command - will list all the XBlocks and Plugins that are installed in the - ``edx-platform`` virtual environment regardless of whether its run - in Docker, devstack or Native Installation. - - If the command is executed with the ``--list`` flag it will print a +- ``manage.py lms atlas_pull_plugin_translations --list``: List all XBlocks and + Plugins installed in the ``edx-platform`` virtual environment. This will list of Python *module names* (as opposed to git repository names) of the installed XBlocks and Plugins e.g.:: - $ manage.py lms atlas_pull_plugins_translations --list + $ manage.py lms atlas_pull_plugin_translations --list drag_and_drop_v2 done eox_tenant This list doesn't include plugins that are bundled within the - ``edx-platform`` repository itself such as the Video XBlock, the ``capa`` - module and others. The reason for this is that their translations are - already included in the ``edx-platform`` translations. - -- ``manage.py lms atlas_pull_plugins_translations``: This command - will craft and executes the ``atlas pull`` command for the XBlocks and - Plugins listed in the previous command. This command is will be added - to the ``Makefile`` and can be executed for both development and production - deployments. - - This command will run and ``atlas pull`` with the arguments below to pull - the translations by module name:: + ``edx-platform`` repository itself. Translations for bundled plugins + are included in the ``edx-platform`` translation files. +- ``manage.py lms atlas_pull_plugin_translations``: This command + will pull translations for installed XBlocks and Plugins by module name:: + $ atlas pull \ translations/edx-platform-links/drag_and_drop_v2/conf/locale:conf/plugins-locale/drag_and_drop_v2 \ translations/edx-platform-links/done/conf/locale:conf/plugins-locale/done \ translations/edx-platform-links/edx_proctoring/conf/locale:conf/plugins-locale/edx_proctoring - It will pull from the `edx-platform-links`_ directory to create the - following file tree:: + Resulting in the following file tree:: $ tree conf/plugins-locale/ conf/plugins-locale/ @@ -226,55 +209,38 @@ Introduce new Django commands to the ``edx-platform``: -**Notes:** - -- The command above is for demonstration purposes and may not work - properly yet. -- The directory name may change from ``edx-platform-links`` to - ``edx-platform-modules`` but this is out of the scope of this proposal and - have little to no impact on the rest of the proposal. Changes to the `openedx-translations repo`_ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -The `openedx-translations repo`_ organizes the translations by the -GitHub repository name. However, once an XBlock is installed the repository -name is no longer known to the ``edx-platform``. Therefore, we provide two -ways to fetch the XBlock translations: - -- By repo name: e.g. `translations/xblock-drag-and-drop-v2 directory`_. -- By module name: e.g. - `translations/edx-platform-links/drag_and_drop_v2 directory`_. - -This update is already implemented in the `openedx-translations repo`_ as of -`edx-platform-links PR #353`_ which includes the details of the changes. +The `openedx-translations repo`_ directory structure organizes translation files by +repository name. ``edx-platform`` has no record of plugin repository names. As of +`edx-platform-links PR #353`_, it is also possible to use ``atlas`` to pull translations by +module name. BlockI18nService support for ``atlas`` Python translations ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -The ``get_python_locale_directory`` will support two modes: +``get_python_locale_directory`` will support two modes: -#. When ``ENABLE_ATLAS_TRANSLATIONS`` is disabled, the XBlock translations - from the ``pip`` packages will be used such as the +#. When ``ENABLE_ATLAS_TRANSLATIONS`` is disabled, the XBlock translation files + included in the ``pip`` packages will be used. For example, the ``lib/python3.8/site-packages/drag_and_drop_v2/translations/ar/LC_MESSAGES/text.po`` path for the Drag and Drop XBlock. -#. When ``ENABLE_ATLAS_TRANSLATIONS`` is enabled, the atlas translations will - be used which is located in the ``edx-platform`` in an the git-ignored +#. When ``ENABLE_ATLAS_TRANSLATIONS`` is enabled, the translation files pulled by ``atlas`` + from the `openedx-translations repo`_ will be used. For example, the ``edx-platform/conf/plugins-locale/drag_and_drop_v2/ar/LC_MESSAGES/text.po`` - path. This file pulled by the ``pull_plugins_translations`` command. + path for the Drag and Drop XBlock. XBlockI18nService support for ``atlas`` JavaScript translations ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -The ``XBlockI18nService`` will provide a new centralized -``get_javascript_locale_path`` method to get the JavaScript translations -``django.js`` file. - -This function needs to be used by the XBlocks in an opt-in backward-compatible -manner. +A ``get_javascript_locale_path`` method will be added to the ``XBlockI18nService`` to provide XBlocks the +appropriate path to ``django.js`` translation files. This will allow XBlocks to utilize legacy packaged translations +or ``atlas`` pulled translations depending on configuration settings. A new ``i18n_js_namespace`` property is needed for the :ref:`compile-js-command` to generate JavaScript translations in a centrally managed manner for all @@ -318,13 +284,13 @@ be updated to support both the ``XBlockI18nService`` new return None -New ``compile_plugins_js_translations`` command +New ``compile_js_plugin_translations`` command ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -This command will loop over XBlock modules that has the ``i18n_js_namespace`` +This command will loop over XBlock modules that have the ``i18n_js_namespace`` property and compile the JavaScript translations. -For example if the Drag and Drop XBlock has the ``i18n_js_namespace`` +For example, if the Drag and Drop XBlock has the ``i18n_js_namespace`` property, the ``compile_plugins_js_translations`` command will execute the following commands:: @@ -349,7 +315,7 @@ and ensure that the ``make pull_translations`` command won't corrupt the virtual environment. This is a non-trivial task and appears to add more complexity than necessary -for not much added benefit. +due to the fact that XBlocks and plugins won't be used outside the context of ``edx-platform``. Goals From 0b9b45f0fbc587c54506b13271e54d54140f2ee1 Mon Sep 17 00:00:00 2001 From: Omar Al-Ithawi Date: Mon, 18 Sep 2023 17:56:36 +0300 Subject: [PATCH 5/7] fix: review comments by brian --- .../0019-oep-58-atlas-translations-design.rst | 228 ++++++++---------- 1 file changed, 103 insertions(+), 125 deletions(-) diff --git a/docs/decisions/0019-oep-58-atlas-translations-design.rst b/docs/decisions/0019-oep-58-atlas-translations-design.rst index 3b670bf83e2f..55380c3bfde1 100644 --- a/docs/decisions/0019-oep-58-atlas-translations-design.rst +++ b/docs/decisions/0019-oep-58-atlas-translations-design.rst @@ -19,8 +19,10 @@ the following changes to the way translations are managed in the Open edX platfo - Add the `Transifex GitHub App `_ to openedx Organization - Connect the `openedx-translations repo`_ to the - `openedx-translations project`_ -- Copy Transifex's Translation Memory and Combine Translators + `openedx-translations Transifex project`_ +- Copy `Transifex Translation Memory`_ into from the both of the + `edx-platform Transifex project`_ and the `xblocks Transifex project`_ into + the new `openedx-translations Transifex project`_ - Utilize `openedx-atlas`_ to pull translations for development/deployment. If you're new to the `Translation Management update OEP-58`_ proposal, please @@ -28,26 +30,30 @@ review it in addition to the `Approach Memo and Technical Discovery - Translations Infrastructure Implementation`_ document before continuing. -Current Architecture/Implementation for XBlocks and Plugins ------------------------------------------------------------ -As of now the Open edX XBlocks and Plugins are installed via ``pip`` with -their translations embedded within the Python package. +Pre-OEP-58 Architecture/Implementation for XBlocks and Plugins +-------------------------------------------------------------- -Bundling translations helps to ensure its always available. However, it also -means that in order to update translations, a full opensource contribution -process needs to happen. Somtimes the process takes a full month to -complete such as in the example below: +Before `OEP-58`_, Open edX XBlocks and Open edX plugins had the following: -- `Added French (Canada) and Japanese - xblock-drag-and-drop-v2 #220`_ + - Translations live in the GitHub repository. + - Translations are packaged with the rest of the code when published to pypi +Pros: + + - Translations are always available after installation. + +Cons: + + - This can mean a complex integration with Transifex + - This can mean a lengthy manual PR review process up to a month such as in + the following example: + `Added French (Canada) and Japanese - xblock-drag-and-drop-v2 #220`_ XBlockI18nService ----------------- -Once the translations are bundled and the XBlock/Plugin is installed the -XBlockI18nService will be able to find the Python translations and use load -them via the ``__init__`` method of the `XBlockI18nService`_ which finds -the ``text.mo`` files and make it available to the ``edx-platform`` +The `XBlockI18nService`_ loads translations for installed XBlocks via its +``__init__`` method. XBlock translations are only used during the during the execution of the XBlock. The `XBlockI18nService implementation pull request`_ (2016) introduced the @@ -80,9 +86,9 @@ as: - `XBlock compiled JavaScript text.js translations`_ - `XBlock main JavaScript file`_ -This design has it's trade-offs but it's been widely adopted and out of the -scope of this proposal. Therefore, we'll improve on the existing design to -load the ``atlas`` JavaScript translations instead of the bundled ones. +`OEP-58`_ does not change this structure, it just makes the necessary changes +to pull translations from the `openedx-translations repo`_ via ``atlas`` +instead of having them live in the XBlock repository itself. Decisions ========= @@ -90,39 +96,17 @@ Decisions Proposed Design for edX Platform ``conf/locale`` translations ------------------------------------------------------------- -Update the ``make pull_translations`` command to use ``atlas pull`` -if the ``OPENEDX_ATLAS_PULL`` environment variable is set. - -This has been the standard for all repositories as seen in both +We're going to use ``atlas`` in ``make pull_translations`` like we do in `course-discovery atlas integration`_ and `frontend-app-learning atlas integration`_. -The updated `edx-platform pull_translations`_ would look like the following:: - - pull_translations: - git clean -fdX conf/locale - ifeq ($(OPENEDX_ATLAS_PULL),) - find conf/locale -type d -mindepth 1 -maxdepth 1 -exec rm -rf {} + # Remove stale translation files - atlas pull $(ATLAS_PULL_ARGS) translations/edx-platform/conf/locale:conf/locale - else - i18n_tool transifex pull - endif - # The commands below are simplified for demonistration purposes - i18n_tool generate --verbose 1 - i18n_tool validate --verbose - paver i18n_compilejs - - -This mostly is a non-controversial change that has been done in other repos -already. - -The next section is a little more intricate and requires more discussion. - Proposed Design for XBlocks and Plugins --------------------------------------- -Instead of storing translation files for each XBlock and Plugin in their respective repositories, -we will use `openedx-atlas`_ to pull them from the `openedx-translations repo`_. +Instead of storing translation files for each XBlock and Plugin in their +respective repositories, +we will use `openedx-atlas`_ to pull them from the +`openedx-translations repo`_. New ``ENABLE_ATLAS_TRANSLATIONS`` Waffle Switch @@ -153,70 +137,59 @@ Introduce new Django commands to the ``edx-platform``: eox_tenant This list doesn't include plugins that are bundled within the - ``edx-platform`` repository itself. Translations for bundled plugins + ``edx-platform`` repository itself. Translations for bundled plugins are included in the ``edx-platform`` translation files. - ``manage.py lms atlas_pull_plugin_translations``: This command will pull translations for installed XBlocks and Plugins by module name:: - - $ atlas pull \ - translations/edx-platform-links/drag_and_drop_v2/conf/locale:conf/plugins-locale/drag_and_drop_v2 \ - translations/edx-platform-links/done/conf/locale:conf/plugins-locale/done \ - translations/edx-platform-links/edx_proctoring/conf/locale:conf/plugins-locale/edx_proctoring + $ atlas pull --expand-glob \ + 'translations/*/drag_and_drop_v2/conf/locale:conf/plugins-locale/drag_and_drop_v2' \ + 'translations/*/done/conf/locale:conf/plugins-locale/done' \ + 'translations/*/edx_proctoring/conf/locale:conf/plugins-locale/edx_proctoring' Resulting in the following file tree:: $ tree conf/plugins-locale/ conf/plugins-locale/ ├── done - │ ├── ar - │ │ └── LC_MESSAGES - │ │ └── django.po - │ ├── de - │ │ └── LC_MESSAGES - │ │ └── django.po - │ ├── en - │ │ └── LC_MESSAGES - │ │ └── django.po - │ └── fr_CA - │ └── LC_MESSAGES - │ └── django.po + │ ├── ar + │ │ └── LC_MESSAGES + │ │ └── django.po + │ ├── de + │ │ └── LC_MESSAGES + │ │ └── django.po + │ ├── en + │ │ └── LC_MESSAGES + │ │ └── django.po + │ └── fr_CA + │ └── LC_MESSAGES + │ └── django.po ├── drag_and_drop_v2 - │ ├── ar - │ │ └── LC_MESSAGES - │ │ └── django.po - │ ├── en - │ │ └── LC_MESSAGES - │ │ └── django.po - │ └── fr_CA - │ └── LC_MESSAGES - │ └── django.po + │ ├── ar + │ │ └── LC_MESSAGES + │ │ └── django.po + │ ├── en + │ │ └── LC_MESSAGES + │ │ └── django.po + │ └── fr_CA + │ └── LC_MESSAGES + │ └── django.po └── edx_proctoring ├── ar - │ └── LC_MESSAGES - │ └── djangojs.po + │ └── LC_MESSAGES + │ └── djangojs.po ├── de - │ └── LC_MESSAGES - │ └── djangojs.po + │ └── LC_MESSAGES + │ └── djangojs.po ├── en - │ └── LC_MESSAGES - │ ├── djangojs.po - │ └── django.po + │ └── LC_MESSAGES + │ ├── djangojs.po + │ └── django.po └── fr_CA └── LC_MESSAGES - └── djangojs.po - - - - -Changes to the `openedx-translations repo`_ -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The `openedx-translations repo`_ directory structure organizes translation files by -repository name. ``edx-platform`` has no record of plugin repository names. As of -`edx-platform-links PR #353`_, it is also possible to use ``atlas`` to pull translations by -module name. + ├── djangojs.po + └── django.po BlockI18nService support for ``atlas`` Python translations @@ -230,26 +203,45 @@ BlockI18nService support for ``atlas`` Python translations path for the Drag and Drop XBlock. #. When ``ENABLE_ATLAS_TRANSLATIONS`` is enabled, the translation files pulled by ``atlas`` - from the `openedx-translations repo`_ will be used. For example, the + from the `openedx-translations repo`_ will be used. For example, the ``edx-platform/conf/plugins-locale/drag_and_drop_v2/ar/LC_MESSAGES/text.po`` path for the Drag and Drop XBlock. +New ``compile_plugins_js_translations`` command +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +An ``XBlock.i18n_js_namespace`` property will be added for +the ``compile_plugins_js_translations`` to generate JavaScript translations +in a centrally managed manner for installed XBlocks. + +A ``compile_plugins_js_translations`` command will loop over XBlock +modules that has the ``i18n_js_namespace`` +property set and compile the JavaScript translations via the `compilejsi18n`_ +command. + +For example if the Drag and Drop XBlock has +``i18n_js_namespace = 'DragAndDropI18N'``, the +``compile_plugins_js_translations`` command will execute the following +commands:: + + i18n_tool generate -v # Generate the .mo files + python manage.py compilejsi18n --namespace DragAndDropI18N --output conf/plugins-locale/drag_and_drop_v2/js/ + + XBlockI18nService support for ``atlas`` JavaScript translations ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -A ``get_javascript_locale_path`` method will be added to the ``XBlockI18nService`` to provide XBlocks the -appropriate path to ``django.js`` translation files. This will allow XBlocks to utilize legacy packaged translations -or ``atlas`` pulled translations depending on configuration settings. +A ``get_javascript_locale_path`` method will be added to the +``XBlockI18nService`` to provide XBlocks the +appropriate path to ``django.js`` translation files. This method +will allow XBlocks to utilize legacy packaged translations +or ``atlas``. -A new ``i18n_js_namespace`` property is needed for the :ref:`compile-js-command` +A ``i18n_js_namespace`` property will be added to generate JavaScript translations in a centrally managed manner for all XBlocks as described in the :ref:`js-translations` section. -The ``i18n_js_namespace`` property will eliminate the need to hardcode the -namespace the `XBlock Makefile compile_translations rule`_. - - For example, the `Drag and Drop XBlock get_static_i18n_js_url`_ will need to be updated to support both the ``XBlockI18nService`` new ``get_javascript_locale_path`` method and the namespace. @@ -284,24 +276,9 @@ be updated to support both the ``XBlockI18nService`` new return None -New ``compile_js_plugin_translations`` command -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -This command will loop over XBlock modules that have the ``i18n_js_namespace`` -property and compile the JavaScript translations. - -For example, if the Drag and Drop XBlock has the ``i18n_js_namespace`` property, -the ``compile_plugins_js_translations`` command will execute the following -commands:: - - i18n_tool generate -v # Generate the .mo files - python manage.py compilejsi18n --namespace DragAndDropI18N --output conf/plugins-locale/drag_and_drop_v2/js/ - - Dismissed Proposals =================== - XBlocks and plugins have their own "atlas pull" command ------------------------------------------------------- @@ -315,7 +292,8 @@ and ensure that the ``make pull_translations`` command won't corrupt the virtual environment. This is a non-trivial task and appears to add more complexity than necessary -due to the fact that XBlocks and plugins won't be used outside the context of ``edx-platform``. +due to the fact that XBlocks and plugins won't be used outside the +context of ``edx-platform``. Goals @@ -325,10 +303,8 @@ Goals #. Allow Tutor and other advanced uses to craft their own ``atlas pull`` commands by making the the plugins list available via Django commands. #. Allow ``atlas pull`` to use the Python module names instead of the - repository name of XBlocks and Plugins. This is already done in the - `openedx-translations repo`_ via the - ``extract-translation-source-files.yml``_ as described in the - `edx-platform translations links`_ document. + repository name of XBlocks and Plugins which is supported via the + `atlas pull --expand-glob`_ option. .. _non-goals: @@ -357,11 +333,10 @@ be tackled in the future as part of the .. _openedx-atlas: https://github.com/openedx/openedx-atlas .. _openedx-translations repo: https://github.com/openedx/openedx-translations .. _extract-translation-source-files.yml: https://github.com/openedx/openedx-translations/blob/2566e0c9a30d033e5dd8d05d4c12601c8e37b4ef/.github/workflows/extract-translation-source-files.yml#L36-L43 -.. _openedx-translations project: https://app.transifex.com/open-edx/openedx-translations/dashboard/ +.. _openedx-translations Transifex project: https://app.transifex.com/open-edx/openedx-translations/dashboard/ .. _Approach Memo and Technical Discovery - Translations Infrastructure Implementation: https://docs.google.com/document/d/11dFBCnbdHiCEdZp3pZeHdeH8m7Glla-XbIin7cnIOzU/edit .. _Added French (Canada) and Japanese - xblock-drag-and-drop-v2 #220: https://github.com/openedx/xblock-drag-and-drop-v2/pull/220 -.. _edx-platform translations links: https://github.com/openedx/openedx-translations/tree/main/translations/edx-platform-links .. _XBlockI18nService: https://github.com/openedx/edx-platform/blob/6e28ba329e0a5354d7264ea834861bf0cae4ceb3/xmodule/modulestore/django.py#L359-L395 .. _XBlockI18nService implementation pull request: https://github.com/openedx/edx-platform/pull/11575/files#diff-0bbcc6c13d9bfc9d88fbe2fdf4fd97f6066a7a0f0bfffb82bc942378b7cf33e0R248 @@ -383,7 +358,10 @@ https://github.com/Zeit-Labs/xblock-drag-and-drop-v2/blob/b8ab1ecd9168ab1dba21f9 .. _XBlock main JavaScript file: https://github.com/openedx/xblock-drag-and-drop-v2/blob/b8ab1ecd9168ab1dba21f994ee4bfedb6a57d11f/drag_and_drop_v2/public/js/drag_and_drop.js#L6 -.. _edx-platform-links PR #353: https://github.com/openedx/openedx-translations/pull/353 .. _translations/xblock-drag-and-drop-v2 directory: https://github.com/openedx/openedx-translations/tree/8a01424fd8f42e9e76aed34e235c82ab654cdfc5/translations/xblock-drag-and-drop-v2 -.. _translations/edx-platform-links/drag_and_drop_v2 directory: https://github.com/openedx/openedx-translations/blob/8a01424fd8f42e9e76aed34e235c82ab654cdfc5/translations/edx-platform-links/drag_and_drop_v2 -.. _edx-platform-links: https://github.com/openedx/openedx-translations/tree/8a01424fd8f42e9e76aed34e235c82ab654cdfc5/translations/edx-platform-links +.. _atlas pull --expand-glob: https://github.com/openedx/openedx-atlas/blob/main/docs/decisions/0001-support-glob-pattern.rst + +.. _compilejsi18n: https://django-statici18n.readthedocs.io/en/latest/commands.html#compilejsi18n +.. _Transifex Translation Memory: https://help.transifex.com/en/articles/6224636-introduction-to-translation-memory +.. _edx-platform Transifex project: https://www.transifex.com/open-edx/edx-platform/ +.. _xblocks Transifex project: https://www.transifex.com/open-edx/xblocks/ From 662fa894428761193ff8d06aa091a4c0b7857425 Mon Sep 17 00:00:00 2001 From: Omar Al-Ithawi Date: Mon, 18 Sep 2023 18:03:36 +0300 Subject: [PATCH 6/7] fix: add deep links --- docs/decisions/0019-oep-58-atlas-translations-design.rst | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/decisions/0019-oep-58-atlas-translations-design.rst b/docs/decisions/0019-oep-58-atlas-translations-design.rst index 55380c3bfde1..4865ecd38c3a 100644 --- a/docs/decisions/0019-oep-58-atlas-translations-design.rst +++ b/docs/decisions/0019-oep-58-atlas-translations-design.rst @@ -25,8 +25,9 @@ the following changes to the way translations are managed in the Open edX platfo the new `openedx-translations Transifex project`_ - Utilize `openedx-atlas`_ to pull translations for development/deployment. -If you're new to the `Translation Management update OEP-58`_ proposal, please -review it in addition to the +If you're new to the `OEP-58`_ proposal, please +review the `OEP-58 Specifications`_ in addition to the +Key Metrics and Expected Results section in the `Approach Memo and Technical Discovery - Translations Infrastructure Implementation`_ document before continuing. @@ -328,6 +329,7 @@ be tackled in the future as part of the ``edx-val``. This will be tackled later on as part of the `OEP-58`_ proposal. +.. _OEP-58 Specifications: https://open-edx-proposals.readthedocs.io/en/latest/architectural-decisions/oep-0058-arch-translations-management.html#specification .. _Translation Management update OEP-58: https://open-edx-proposals.readthedocs.io/en/latest/architectural-decisions/oep-0058-arch-translations-management.html#specification .. _OEP-58: https://open-edx-proposals.readthedocs.io/en/latest/architectural-decisions/oep-0058-arch-translations-management.html#specification .. _openedx-atlas: https://github.com/openedx/openedx-atlas From f2c0f15f0f866bfb97197c4fe396b15839834320 Mon Sep 17 00:00:00 2001 From: Omar Al-Ithawi Date: Mon, 18 Sep 2023 18:08:38 +0300 Subject: [PATCH 7/7] doc: delete the proposal --- .../0019-oep-58-atlas-translations-design.rst | 369 ------------------ 1 file changed, 369 deletions(-) delete mode 100644 docs/decisions/0019-oep-58-atlas-translations-design.rst diff --git a/docs/decisions/0019-oep-58-atlas-translations-design.rst b/docs/decisions/0019-oep-58-atlas-translations-design.rst deleted file mode 100644 index 4865ecd38c3a..000000000000 --- a/docs/decisions/0019-oep-58-atlas-translations-design.rst +++ /dev/null @@ -1,369 +0,0 @@ -Design for Refactoring Translations ``pull`` to use Atlas -########################################################## - -Status -====== - -Pending - -Context -======= - -OEP-58 Translation Management overview --------------------------------------- - -The `Translation Management update OEP-58`_ proposal has been merged with -the following changes to the way translations are managed in the Open edX platform: - -- Move Translation Files to the `openedx-translations repo`_ -- Add the `Transifex GitHub App `_ - to openedx Organization -- Connect the `openedx-translations repo`_ to the - `openedx-translations Transifex project`_ -- Copy `Transifex Translation Memory`_ into from the both of the - `edx-platform Transifex project`_ and the `xblocks Transifex project`_ into - the new `openedx-translations Transifex project`_ -- Utilize `openedx-atlas`_ to pull translations for development/deployment. - -If you're new to the `OEP-58`_ proposal, please -review the `OEP-58 Specifications`_ in addition to the -Key Metrics and Expected Results section in the -`Approach Memo and Technical Discovery - Translations Infrastructure Implementation`_ -document before continuing. - -Pre-OEP-58 Architecture/Implementation for XBlocks and Plugins --------------------------------------------------------------- - -Before `OEP-58`_, Open edX XBlocks and Open edX plugins had the following: - - - Translations live in the GitHub repository. - - Translations are packaged with the rest of the code when published to pypi - -Pros: - - - Translations are always available after installation. - -Cons: - - - This can mean a complex integration with Transifex - - This can mean a lengthy manual PR review process up to a month such as in - the following example: - `Added French (Canada) and Japanese - xblock-drag-and-drop-v2 #220`_ - -XBlockI18nService ------------------ - -The `XBlockI18nService`_ loads translations for installed XBlocks via its -``__init__`` method. XBlock translations are only used during the -during the execution of the XBlock. - -The `XBlockI18nService implementation pull request`_ (2016) introduced the -support for XBlock translations in ``edx-platform`` and has the full -context of the implementation. - -.. _js-translations: - -JavaScript Translations for XBocks ----------------------------------- - -As of September 2023, there is no centralized method to bundle JavaScript -translations in XBlocks. Non-XBlock plugins lack JavaScript translation -support altogether. - -The de-facto standard method for bundling JavaScript translations in XBlocks -is to use ``web_fragment`` and load the translations as part of the XBlock -frontend static files on every XBlock load. - -The LTI Consumer XBlock embeds the translations in its ``web_fragment`` via -the `LtiConsumerXBlock._get_statici18n_js_url`_ and -`LtiConsumerXBlock.student_view`_ methods. - -In order to separate the XBlock translations from the platform, it's isolated -in a separate ``gettext`` name space. For example, the Drag and Drop XBlock -namespace is ``DragAndDropI18N``` which is hardcoded in multiple places such -as: - -- `XBlock Makefile compile_translations rule`_ -- `XBlock compiled JavaScript text.js translations`_ -- `XBlock main JavaScript file`_ - -`OEP-58`_ does not change this structure, it just makes the necessary changes -to pull translations from the `openedx-translations repo`_ via ``atlas`` -instead of having them live in the XBlock repository itself. - -Decisions -========= - -Proposed Design for edX Platform ``conf/locale`` translations -------------------------------------------------------------- - -We're going to use ``atlas`` in ``make pull_translations`` like we do in -`course-discovery atlas integration`_ and -`frontend-app-learning atlas integration`_. - -Proposed Design for XBlocks and Plugins ---------------------------------------- - -Instead of storing translation files for each XBlock and Plugin in their -respective repositories, -we will use `openedx-atlas`_ to pull them from the -`openedx-translations repo`_. - - -New ``ENABLE_ATLAS_TRANSLATIONS`` Waffle Switch -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The ``ENABLE_ATLAS_TRANSLATIONS`` switch will be disabled by default. -It will be used by the ``XBlockI18nService`` to determine which translations -to use for XBlocks until the `OEP-58`_ is fully implemented and -the non-atlas translations are removed. - -The non-XBlock plugins such as `edx-val`_ are out of the scope of this -proposal and will be handled separately as stated in the :ref:`non-goals` -section. - -New ``pull_plugins_translations`` command -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Introduce new Django commands to the ``edx-platform``: - -- ``manage.py lms atlas_pull_plugin_translations --list``: List all XBlocks and - Plugins installed in the ``edx-platform`` virtual environment. This will - list of Python *module names* (as opposed to git repository names) of the - installed XBlocks and Plugins e.g.:: - - $ manage.py lms atlas_pull_plugin_translations --list - drag_and_drop_v2 - done - eox_tenant - - This list doesn't include plugins that are bundled within the - ``edx-platform`` repository itself. Translations for bundled plugins - are included in the ``edx-platform`` translation files. - -- ``manage.py lms atlas_pull_plugin_translations``: This command - will pull translations for installed XBlocks and Plugins by module name:: - - $ atlas pull --expand-glob \ - 'translations/*/drag_and_drop_v2/conf/locale:conf/plugins-locale/drag_and_drop_v2' \ - 'translations/*/done/conf/locale:conf/plugins-locale/done' \ - 'translations/*/edx_proctoring/conf/locale:conf/plugins-locale/edx_proctoring' - - Resulting in the following file tree:: - - $ tree conf/plugins-locale/ - conf/plugins-locale/ - ├── done - │ ├── ar - │ │ └── LC_MESSAGES - │ │ └── django.po - │ ├── de - │ │ └── LC_MESSAGES - │ │ └── django.po - │ ├── en - │ │ └── LC_MESSAGES - │ │ └── django.po - │ └── fr_CA - │ └── LC_MESSAGES - │ └── django.po - ├── drag_and_drop_v2 - │ ├── ar - │ │ └── LC_MESSAGES - │ │ └── django.po - │ ├── en - │ │ └── LC_MESSAGES - │ │ └── django.po - │ └── fr_CA - │ └── LC_MESSAGES - │ └── django.po - └── edx_proctoring - ├── ar - │ └── LC_MESSAGES - │ └── djangojs.po - ├── de - │ └── LC_MESSAGES - │ └── djangojs.po - ├── en - │ └── LC_MESSAGES - │ ├── djangojs.po - │ └── django.po - └── fr_CA - └── LC_MESSAGES - ├── djangojs.po - └── django.po - - -BlockI18nService support for ``atlas`` Python translations -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -``get_python_locale_directory`` will support two modes: - -#. When ``ENABLE_ATLAS_TRANSLATIONS`` is disabled, the XBlock translation files - included in the ``pip`` packages will be used. For example, the - ``lib/python3.8/site-packages/drag_and_drop_v2/translations/ar/LC_MESSAGES/text.po`` - path for the Drag and Drop XBlock. - -#. When ``ENABLE_ATLAS_TRANSLATIONS`` is enabled, the translation files pulled by ``atlas`` - from the `openedx-translations repo`_ will be used. For example, the - ``edx-platform/conf/plugins-locale/drag_and_drop_v2/ar/LC_MESSAGES/text.po`` - path for the Drag and Drop XBlock. - - -New ``compile_plugins_js_translations`` command -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -An ``XBlock.i18n_js_namespace`` property will be added for -the ``compile_plugins_js_translations`` to generate JavaScript translations -in a centrally managed manner for installed XBlocks. - -A ``compile_plugins_js_translations`` command will loop over XBlock -modules that has the ``i18n_js_namespace`` -property set and compile the JavaScript translations via the `compilejsi18n`_ -command. - -For example if the Drag and Drop XBlock has -``i18n_js_namespace = 'DragAndDropI18N'``, the -``compile_plugins_js_translations`` command will execute the following -commands:: - - i18n_tool generate -v # Generate the .mo files - python manage.py compilejsi18n --namespace DragAndDropI18N --output conf/plugins-locale/drag_and_drop_v2/js/ - - -XBlockI18nService support for ``atlas`` JavaScript translations -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -A ``get_javascript_locale_path`` method will be added to the -``XBlockI18nService`` to provide XBlocks the -appropriate path to ``django.js`` translation files. This method -will allow XBlocks to utilize legacy packaged translations -or ``atlas``. - -A ``i18n_js_namespace`` property will be added -to generate JavaScript translations in a centrally managed manner for all -XBlocks as described in the :ref:`js-translations` section. - -For example, the `Drag and Drop XBlock get_static_i18n_js_url`_ will need to -be updated to support both the ``XBlockI18nService`` new -``get_javascript_locale_path`` method and the namespace. - -.. code:: diff - - class DragAndDropBlock(XBlock): - - + i18n_js_namespace = 'DragAndDropI18N' - - @staticmethod - def _get_statici18n_js_url(): - """ - Returns the Javascript translation file for the currently selected language, if any found by - `pkg_resources` - """ - lang_code = translation.get_language() - if not lang_code: - return None - - + # TODO: Make this the default once OEP-58 is implemented. - + if hasattr(self.i18n_service, 'get_javascript_locale_path'): - + atlas_locale_path = self.i18n_service.get_javascript_locale_path() - + if atlas_locale_path: - + return atlas_locale_path - - text_js = 'public/js/translations/{lang_code}/text.js' - country_code = lang_code.split('-')[0] - for code in (translation.to_locale(lang_code), lang_code, country_code): - if pkg_resources.resource_exists(loader.module_name, text_js.format(lang_code=code)): - return text_js.format(lang_code=code) - return None - - -Dismissed Proposals -=================== - -XBlocks and plugins have their own "atlas pull" command -------------------------------------------------------- - -This dismissed proposal intends to have each XBlock and Plugin have their -own ``make pull_translations`` and be responsible for managing pulling their -own translations from the `openedx-translations repo`_. - -This proposal has been dismissed because it would require substantial work -to get into the details for the ``lib/python3.8/site-packages/`` directory -and ensure that the ``make pull_translations`` command won't corrupt the -virtual environment. - -This is a non-trivial task and appears to add more complexity than necessary -due to the fact that XBlocks and plugins won't be used outside the -context of ``edx-platform``. - - -Goals -===== -#. Use ``atlas pull`` for the ``edx-platform`` repo. -#. Use ``atlas pull`` for the XBlocks and Plugins. -#. Allow Tutor and other advanced uses to craft their own ``atlas pull`` - commands by making the the plugins list available via Django commands. -#. Allow ``atlas pull`` to use the Python module names instead of the - repository name of XBlocks and Plugins which is supported via the - `atlas pull --expand-glob`_ option. - -.. _non-goals: - -Non-Goals -========= - -The following are non-goals for this proposal, although some are going to -be tackled in the future as part of the -`Translation Management update OEP-58`_ proposal. - -#. Provide a fool-proof method for managing named-release translations. - This will be a separate discussion. -#. Discuss the merge/segment strategy of the ``edx-platform``. This is being - discussed in the - `decision no. 0018 `_. -#. Design a new XBlock frontend architecture. Instead this proposal works - with the existing architecture. -#. Provide a new translation method for theme translations. This will be - tackled later on. -#. Provide a new translation method for non-XBlock plugins such as - ``edx-val``. This will be tackled later on as part of the `OEP-58`_ - proposal. - -.. _OEP-58 Specifications: https://open-edx-proposals.readthedocs.io/en/latest/architectural-decisions/oep-0058-arch-translations-management.html#specification -.. _Translation Management update OEP-58: https://open-edx-proposals.readthedocs.io/en/latest/architectural-decisions/oep-0058-arch-translations-management.html#specification -.. _OEP-58: https://open-edx-proposals.readthedocs.io/en/latest/architectural-decisions/oep-0058-arch-translations-management.html#specification -.. _openedx-atlas: https://github.com/openedx/openedx-atlas -.. _openedx-translations repo: https://github.com/openedx/openedx-translations -.. _extract-translation-source-files.yml: https://github.com/openedx/openedx-translations/blob/2566e0c9a30d033e5dd8d05d4c12601c8e37b4ef/.github/workflows/extract-translation-source-files.yml#L36-L43 -.. _openedx-translations Transifex project: https://app.transifex.com/open-edx/openedx-translations/dashboard/ - -.. _Approach Memo and Technical Discovery - Translations Infrastructure Implementation: https://docs.google.com/document/d/11dFBCnbdHiCEdZp3pZeHdeH8m7Glla-XbIin7cnIOzU/edit -.. _Added French (Canada) and Japanese - xblock-drag-and-drop-v2 #220: https://github.com/openedx/xblock-drag-and-drop-v2/pull/220 -.. _XBlockI18nService: https://github.com/openedx/edx-platform/blob/6e28ba329e0a5354d7264ea834861bf0cae4ceb3/xmodule/modulestore/django.py#L359-L395 -.. _XBlockI18nService implementation pull request: https://github.com/openedx/edx-platform/pull/11575/files#diff-0bbcc6c13d9bfc9d88fbe2fdf4fd97f6066a7a0f0bfffb82bc942378b7cf33e0R248 - -.. _course-discovery atlas integration: https://github.com/openedx/course-discovery/pull/4037 -.. _frontend-app-learning atlas integration: https://github.com/openedx/frontend-app-learning/pull/1093 -.. _edx-platform pull_translations: https://github.com/openedx/edx-platform/blob/0137881b8199701b2af7d07c9a01200e358e3d86/Makefile#L55-L64 - -.. _drag-and-drop-v2 xblock: https://github.com/openedx/xblock-drag-and-drop-v2/ -.. _LTI Consumer XBlock: https://github.com/openedx/xblock-lti-consumer/ -.. _edx-val: https://github.com/openedx/edx-val - -.. _LtiConsumerXBlock._get_statici18n_js_url: https://github.com/openedx/xblock-lti-consumer/blob/7a142310a78ac393286c1e9e77c535ea520ab90b/lti_consumer/lti_xblock.py#L663-L677 -.. _LtiConsumerXBlock.student_view: https://github.com/openedx/xblock-lti-consumer/blob/7a142310a78ac393286c1e9e77c535ea520ab90b/lti_consumer/lti_xblock.py#L1215C24-L1217 -.. _Drag and Drop XBlock get_static_i18n_js_url: https://github.com/openedx/xblock-drag-and-drop-v2/blob/66e8d3517fe8c0db55c1a3907ff253c2a4562a7e/drag_and_drop_v2/drag_and_drop_v2.py#L318-L332 - -.. _XBlock compiled JavaScript text.js translations: https://github.com/openedx/xblock-drag-and-drop-v2/blob/b8ab1ecd9168ab1dba21f994ee4bfedb6a57d11f/drag_and_drop_v2/public/js/translations/tr/text.js#L3 -https://github.com/Zeit-Labs/xblock-drag-and-drop-v2/blob/b8ab1ecd9168ab1dba21f994ee4bfedb6a57d11f/drag_and_drop_v2/public/js/translations/tr/text.js#L3 -.. _XBlock Makefile compile_translations rule: https://github.com/openedx/xblock-drag-and-drop-v2/blob/66e8d3517fe8c0db55c1a3907ff253c2a4562a7e/Makefile#L41 -.. _XBlock main JavaScript file: https://github.com/openedx/xblock-drag-and-drop-v2/blob/b8ab1ecd9168ab1dba21f994ee4bfedb6a57d11f/drag_and_drop_v2/public/js/drag_and_drop.js#L6 - - -.. _translations/xblock-drag-and-drop-v2 directory: https://github.com/openedx/openedx-translations/tree/8a01424fd8f42e9e76aed34e235c82ab654cdfc5/translations/xblock-drag-and-drop-v2 -.. _atlas pull --expand-glob: https://github.com/openedx/openedx-atlas/blob/main/docs/decisions/0001-support-glob-pattern.rst - -.. _compilejsi18n: https://django-statici18n.readthedocs.io/en/latest/commands.html#compilejsi18n -.. _Transifex Translation Memory: https://help.transifex.com/en/articles/6224636-introduction-to-translation-memory -.. _edx-platform Transifex project: https://www.transifex.com/open-edx/edx-platform/ -.. _xblocks Transifex project: https://www.transifex.com/open-edx/xblocks/