Skip to content

Commit

Permalink
feat: allow pulling translations by module name rather than repo name
Browse files Browse the repository at this point in the history
This pull request is part of the [FC-0012 project](https://openedx.atlassian.net/l/cp/XGS0iCcQ) which is sparked by the [Translation Infrastructure update OEP-58](https://open-edx-proposals.readthedocs.io/en/latest/architectural-decisions/oep-0058-arch-translations-management.html#specification).
  • Loading branch information
OmarIthawi committed Jun 2, 2023
1 parent c2f9d0d commit 8491d3d
Show file tree
Hide file tree
Showing 8 changed files with 141 additions and 25 deletions.
75 changes: 50 additions & 25 deletions .github/workflows/extract-translation-source-files.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,22 +39,39 @@ jobs:
max-parallel: 1
matrix:
repo:
- AudioXBlock
- completion
- course-discovery
- credentials
- DoneXBlock
- edx-ace
- edx-bulk-grades
- edx-ora2
- edx-proctoring
- RecommenderXBlock
- xblock-drag-and-drop-v2
- xblock-free-text-response
- xblock-lti-consumer
- xblock-qualtrics-survey
- xblock-sql-grader
- xblock-submit-and-compare
# Note: Add `module_name` for all edx-platform plugins and XBlocks such as DoneXBlock and completion, but not
# for micrsoervices and IDAs such as course-discovery and credentials.
- slug: AudioXBlock
module_name: audio
- slug: completion
module_name: completion
- slug: course-discovery # not a plugin
- slug: credentials # not a plugin
- slug: DoneXBlock
module_name: done
- slug: edx-ace
module_name: edx_ace
- slug: edx-bulk-grades
module_name: bulk_grades
- slug: edx-ora2
module_name: openassessment
- slug: edx-proctoring
module_name: edx_proctoring
- slug: RecommenderXBlock
module_name: recommender
- slug: xblock-drag-and-drop-v2
module_name: drag_and_drop_v2
- slug: xblock-free-text-response
module_name: freetextresponse
- slug: xblock-lti-consumer
module_name: lti_consumer
- slug: xblock-qualtrics-survey
module_name: qualtricssurvey
- slug: xblock-sql-grader
module_name: sql_grader
- slug: xblock-submit-and-compare
module_name: submit_and_compare

runs-on: ubuntu-latest
continue-on-error: true
needs: [setup-branch]
Expand All @@ -71,11 +88,11 @@ jobs:
run: sudo apt install -y gettext

# Clones the repository
- name: clone openedx/${{ matrix.repo }}
- name: clone openedx/${{ matrix.repo.slug }}
uses: actions/checkout@v3
with:
repository: openedx/${{ matrix.repo }}
path: translations/${{ matrix.repo }}
repository: openedx/${{ matrix.repo.slug }}
path: translations/${{ matrix.repo.slug }}

# Sets up Python
- name: setup python
Expand All @@ -90,13 +107,13 @@ jobs:
# Extracts the translation source files
- name: extract translation source files
run: |
cd translations/${{ matrix.repo }}
cd translations/${{ matrix.repo.slug }}
make extract_translations
# Validate compilation of translation source files
- name: validate compilation of translation source files
run: |
cd translations/${{ matrix.repo }}
cd translations/${{ matrix.repo.slug }}
django-admin compilemessages --locale en
# git adds only the translation source files, found in conf/locale/en
Expand All @@ -106,12 +123,12 @@ jobs:
# set identity
git config --global user.email "[email protected]"
git config --global user.name "edx-transifex-bot"
# Change directory to translations/${{ matrix.repo }}
cd translations/${{ matrix.repo }}
# Change directory to translations/${{ matrix.repo.slug }}
cd translations/${{ matrix.repo.slug }}
# finds the directory containing the english locale usually located in
# */*/conf/locale/en
EN_DIR=$(find . -type d -regex ".+conf\/locale\/en$")
# remove translations/${{ matrix.repo }}/.git so we don't commit a submodule
# remove translations/${{ matrix.repo.slug }}/.git so we don't commit a submodule
rm -rf .git
# finds the django.po and djangojs.po files generated by make
# extract_translations into EN_DIR and adds them
Expand All @@ -122,12 +139,20 @@ jobs:
# Check the git statuses of the translation source files
echo "GIT_STATUS=$(git status $DJANGO_PATH $DJANGOJS_PATH -s | wc -l)" >> $GITHUB_ENV
- name: create plugin/xblock link
if: ${{ matrix.repo.module_name }}
run: |
cd openedx-translations
if [ ! -d "${{ matrix.repo.module_name }}" ]; then
ln -sd ../translations/${{ matrix.repo.slug }} ${{ matrix.repo.module_name }}
fi
# Attempts to commit the translation source files if there is a difference
- name: git commit the translation source files
if: "${{ env.GIT_STATUS > 0 }}"
run: |
# commit the changes
git commit -m "chore: add extracted translation source files from ${{ matrix.repo }}"
git commit -m "chore: add extracted translation source files from ${{ matrix.repo.slug }}"
# push changes to branch
git push
Expand Down
75 changes: 75 additions & 0 deletions openedx-translations/README.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
Open edX (``edx-platform``) Translations
========================================

This directory contains links to the Python modules of plugins and XBlocks used in ``edx-platform``.

This directory is used to enable ``atlas`` fetch translations by module name rather than by repository name for two reasons:

1. When running ``make pull_translations`` in the edx-platform, the repository name of the installed XBlock isn't available. On the other hand XBlock's python module name is available.
2. Repo names can change but module names don't.


Example file structure of the ``openedx-translations`` directory:


$ pwd
~/work/openedx/openedx-translations/openedx-translations
openedx-translations $ ll
done -> ../translations/DoneXBlock/done/
drag_and_drop_v2 -> ../translations/xblock-drag-and-drop-v2/drag_and_drop_v2/
qualtricssurvey -> ../translations/xblock-qualtrics-survey/qualtricssurvey/
recommender -> ../translations/RecommenderXBlock/recommender/
submit_and_compare -> ../translations/xblock-submit-and-compare/submit_and_compare/

The links in this directory are automatically created by the ``extract-translation-source-files.yml`` GitHub Action workflow in this repo.


Pulling ``edx-platform`` translations with Atlas
------------------------------------------------

The ``atlass`` command has been integrated in the ``edx-platform`` repository therefore there's no need to run ``atlas pull`` manually. Regardless, this section shows the manual command arguments because it's needed for some advanced usage of the edX Platform such as installing custom XBlocks that's not part of the openedx github organizaiton.

The ``atlas`` command to pull from this directory for Drag and Drop v2 XBlock can be executed in two ways:


By repository name:

```
$ atlas pull translations/xblock-drag-and-drop-v2:xblock-drag-and-drop-v2
$ tree xblock-drag-and-drop-v2
xblock-drag-and-drop-v2
└── drag_and_drop_v2
└── conf
└── locale
├── ar
│ └── LC_MESSAGES
│ └── django.po
├── en
│ └── LC_MESSAGES
│ └── django.po
└── fr_CA
└── LC_MESSAGES
└── django.po
```

By python module name:

```
$ atlas pull openedx-translations/drag_and_drop_v2:drag_and_drop_v2
$ tree drag_and_drop_v2
drag_and_drop_v2
└── conf
└── locale
├── ar
│ └── LC_MESSAGES
│ └── django.po
├── en
│ └── LC_MESSAGES
│ └── django.po
└── fr_CA
└── LC_MESSAGES
└── django.po
```

The second file tree has one less level of directories because ``xblock-drag-and-drop-v2`` is completely skipped.

1 change: 1 addition & 0 deletions openedx-translations/done
1 change: 1 addition & 0 deletions openedx-translations/drag_and_drop_v2
1 change: 1 addition & 0 deletions openedx-translations/qualtricssurvey
1 change: 1 addition & 0 deletions openedx-translations/recommender
1 change: 1 addition & 0 deletions openedx-translations/submit_and_compare
11 changes: 11 additions & 0 deletions python-modules/README.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
Python Modules Links
====================

This directory contains links to the Python modules of plugins and XBlocks used in edx-platform.

This is mostly used to enable ``atlas`` fetch translations by module name rather than by repository name for two reasons:

1. When running ``make pull_translations`` in the edx-platform, the repository name of the installed XBlock isn't available. On the other hand XBlock's python module name is available.
2. Repo names can change but module names don't.

The links in this directory are created by the ``extract-translation-source-files.yml`` GitHub Action workflow in this repo.

0 comments on commit 8491d3d

Please sign in to comment.