Skip to content
This repository was archived by the owner on Aug 3, 2024. It is now read-only.

Commit 1271c34

Browse files
brawarubrawaru
committed
Switch to manual Crowdin uploading and downloading
Changes highlights ================== This commit adds two GitHub Actions workflow: i18n-pull --------- i18n-pull downloads translations from Crowdin. It is run automatically every Monday at 7 AM GMT time. It builds the Crowdin project and then downloads all the files corresponding to the branch it is running for (for scheduled runs it is the default branch of the repository). i18n-push --------- i18n-push runs the the extractor and uploads the source string files to Crowdin for translation. It is running for every push to the `master` branch and checks the changed files, trying to avoid unnecessary runs when none of the files changed contain translations. It can also be dispatched manually for every branch of the repository, which allows to create feature branches on Crowdin and give translators an early start. Translations for such branches can be downloaded by dispatching i18n-pull. They will also migrate automatically if the feature branch gets merged (see below). i18n-cleanup ------------ i18n-cleanup performs a cleanup of every repository branch deletion by deleting the translation branch on Crowdin as well Git pull branch. Deleting the translation branch after the merge allows Crowdin to migrate the translations to the other branch where updated strings now match (if no strings match, e.g., after PR rejection, then translations will just be deleted). Deletion of the pull branch closes the associated pull request which is most likely no longer relevant. *** Since translations are now extracted automatically, there is no more need in index.json file being a part of the repository, which previously caused pains when dealing with pull requests, so it is removed now. Contibutors will have to run `pnpm intl:extract` manually now. Migration strategy ================== Please take extra care when migrating to this approach to avoid the loss of translations. 1 - Creating a Crowdin token ---------------------------- The first step is to create a token in order for Crowdin CLI to perform operations. 1. Navigate to the settings for the automated account on Crowdin, and then switch to API tab. Alternatively, use the link: https://crowdin.com/settings#api-key. 2. Click on "New Token" 3. Give token a meaningful name to distinguish its requests in security logs 4. In scopes select the following: - Projects - Projects (List, Get, Create, Edit) - Source files & strings - Translations 5. Click "Create" 6. Confirm your identity by signing in again 7. Save the access token somewhere temporarily, you will need it later 2 - Creating a GitHub token --------------------------- In order for the download workflow to be able to create proper pull requests, it needs to puppeteer a user account. This is required for several reasons: - Other actions do not run on PRs created by actions - GitHub does not allow actions to create pull requests by default regardless the requested permissions in the workflow file. It requires enabling a dangerous permission in the organisation settings. - Bot's identity. Bot's cute. To create a token: 1. Log in to the GitHub account that will be automated 2. Go to Settings > Developer Settings > Personal access tokens > Tokens (classic) 3. Click "Generate new token" and select "Generate new token (classic)" from the menu 4. Give token a meaningful name and reasonable expiration time 5. In scopes select `repo` scope 6. Save the token 3 - Setting up GitHub Actions secrets and variables -------------------------------------------------- For workflows to work correctly, they need several variables. Assuming you completed the above steps, here's how you set them up. 1. Navigate to the modrinth/knossos repository 2. Switch to Settings tab 3. Go to Secrets and variables > Actions 4. Create the following repository secrets: - `CROWDIN_PERSONAL_TOKEN`: [your Crowdin token] - `CROWDIN_GH_TOKEN`: [your GitHub automated account token] 5. Create the following repository variables: - `CROWDIN_PROJECT_ID`: [Crowdin project ID (Modrinth - 518556)] 4 - Preparing Crowdin project ----------------------------- Because workflows will replace GitHub integration, that integration needs to be disabled before the merge to avoid conflics and accidental deletion of the translations due to 'deletion' of the source files. 1. Go to Modrinth Crowdin project 2. Switch to Integrations tab 3. Click on GitHub integration 4. Click on modrinth/knossos repository 5. Click "Delete integration" 6. Confirm deletion 5 - Merge --------- After following all the steps above you should be now ready to merge this pull request. Make sure to check that actions do run after you merge it.
1 parent 5ea71da commit 1271c34

File tree

7 files changed

+384
-914
lines changed

7 files changed

+384
-914
lines changed

‎.github/workflows/i18n-cleanup.yml

+103
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
name: Crowdin (cleanup)
2+
3+
on: [delete]
4+
5+
concurrency:
6+
group: i18n-management
7+
8+
env:
9+
CROWDIN_CLI_VERSION: '3.18.0'
10+
11+
jobs:
12+
preflight_check:
13+
name: 'Pre-flight check'
14+
runs-on: ubuntu-22.04
15+
concurrency:
16+
group: i18n-cleanup:${{ github.event.ref }}
17+
cancel-in-progress: true
18+
if: github.event.ref_type == 'branch' && !startsWith(github.event.ref, 'ref/heads/crowdin-pull/')
19+
steps:
20+
- name: Preflight check
21+
id: check
22+
shell: bash
23+
run: |
24+
PREFLIGHT_CHECK_RESULT=true
25+
26+
function flight_failure () {
27+
if [ "$PREFLIGHT_CHECK_RESULT" = true ]; then
28+
echo "One or more pre-flight checks failed!"
29+
echo ""
30+
PREFLIGHT_CHECK_RESULT=false
31+
fi
32+
echo "- $1"
33+
}
34+
35+
if [ "$CROWDIN_PROJECT_ID_DEFINED" != true ]; then
36+
flight_failure "CROWDIN_PROJECT_ID variable is not defined (required to push)"
37+
fi
38+
39+
if [ "$CROWDIN_PERSONAL_TOKEN_DEFINED" != true ]; then
40+
flight_failure "CROWDIN_PERSONAL_TOKEN secret is not defined (required to push)"
41+
fi
42+
43+
echo "flight_ok=$PREFLIGHT_CHECK_RESULT" >> "$GITHUB_OUTPUT"
44+
env:
45+
CROWDIN_PROJECT_ID_DEFINED: ${{ vars.CROWDIN_PROJECT_ID != '' }}
46+
CROWDIN_PERSONAL_TOKEN_DEFINED: ${{ secrets.CROWDIN_PERSONAL_TOKEN != '' }}
47+
outputs:
48+
ready: ${{ steps.check.outputs.flight_ok == 'true' }}
49+
delete_crowdin_branch:
50+
name: 'Delete Crowdin branch'
51+
runs-on: ubuntu-22.04
52+
needs: preflight_check
53+
if: needs.preflight_check.outputs.ready == 'true'
54+
concurrency:
55+
group: i18n-cleanup:${{ github.event.ref }}
56+
cancel-in-progress: true
57+
permissions:
58+
contents: write
59+
steps:
60+
- name: Checkout
61+
uses: actions/checkout@v4
62+
63+
- name: Get branch name from ref
64+
id: branch-name
65+
shell: bash
66+
run: echo "safe_branch_name=$(echo "${{ github.event.ref }}" | sed -e "s/[\\\\/\\:*?\"<>|]/_/g")" >> "$GITHUB_OUTPUT"
67+
68+
# Crowdin GitHub Action is currently broken and doesn't properly run commands that contain spaces
69+
# See https://github.com/crowdin/github-action/issues/192
70+
- name: Download Crowdin CLI
71+
uses: robinraju/[email protected]
72+
with:
73+
repository: crowdin/crowdin-cli
74+
tag: ${{ env.CROWDIN_CLI_VERSION }}
75+
fileName: 'crowdin-cli.zip'
76+
out-file-path: '.crowdin'
77+
78+
- name: Delete translations branch on Crowdin
79+
shell: bash
80+
run: |
81+
CROWDIN_DIR="$PWD/.crowdin"
82+
unzip "$CROWDIN_DIR/crowdin-cli.zip" -d "$CROWDIN_DIR"
83+
CROWDIN_CLI_JAR="$CROWDIN_DIR/${{ env.CROWDIN_CLI_VERSION }}/crowdin-cli.jar"
84+
java -jar "$CROWDIN_CLI_JAR" branch delete "[${{ github.repository_owner }}.${{ github.event.repository.name }}] ${{ steps.branch-name.outputs.safe_branch_name }}"
85+
continue-on-error: true
86+
env:
87+
CROWDIN_PROJECT_ID: ${{ vars.CROWDIN_PROJECT_ID }}
88+
CROWDIN_PERSONAL_TOKEN: ${{ secrets.CROWDIN_PERSONAL_TOKEN }}
89+
90+
# - name: Delete translations branch on Crowdin
91+
# uses: crowdin/github-action@v1
92+
# with:
93+
# command: 'branch'
94+
# command_args: 'delete "[${{ github.repository_owner }}.${{ github.event.repository.name }}] ${{ steps.branch-name.outputs.safe_branch_name }}"'
95+
# env:
96+
# CROWDIN_PROJECT_ID: ${{ vars.CROWDIN_PROJECT_ID }}
97+
# CROWDIN_PERSONAL_TOKEN: ${{ secrets.CROWDIN_PERSONAL_TOKEN }}
98+
# continue-on-error: true
99+
100+
- name: Delete Git branch
101+
shell: bash
102+
run: 'git push origin :crowdin-pull/${{ github.event.ref }}'
103+
continue-on-error: true

‎.github/workflows/i18n-pull.yml

+123
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,123 @@
1+
name: Crowdin (pull)
2+
3+
on:
4+
schedule:
5+
- cron: '0 7 * * MON' # every monday at 7 am
6+
workflow_dispatch: {}
7+
8+
concurrency:
9+
group: i18n-management
10+
11+
jobs:
12+
preflight_check:
13+
name: 'Pre-flight check'
14+
runs-on: ubuntu-22.04
15+
concurrency:
16+
group: i18n-pull:${{ github.ref }}
17+
cancel-in-progress: true
18+
steps:
19+
- name: Preflight check
20+
id: check
21+
run: |
22+
PREFLIGHT_CHECK_RESULT=true
23+
24+
function flight_failure () {
25+
if [ "$PREFLIGHT_CHECK_RESULT" = true ]; then
26+
echo "One or more pre-flight checks failed!"
27+
echo ""
28+
PREFLIGHT_CHECK_RESULT=false
29+
fi
30+
echo "- $1"
31+
}
32+
33+
if [ "$CROWDIN_PROJECT_ID_DEFINED" != true ]; then
34+
flight_failure "CROWDIN_PROJECT_ID variable is not defined (required to push)"
35+
fi
36+
37+
if [ "$CROWDIN_PERSONAL_TOKEN_DEFINED" != true ]; then
38+
flight_failure "CROWDIN_PERSONAL_TOKEN secret is not defined (required to push)"
39+
fi
40+
41+
if [ "$CROWDIN_GH_TOKEN_DEFINED" != true ]; then
42+
flight_failure "CROWDIN_GH_TOKEN secret is not defined (required to make pull requests)"
43+
fi
44+
45+
echo "flight_ok=$PREFLIGHT_CHECK_RESULT" >> "$GITHUB_OUTPUT"
46+
env:
47+
CROWDIN_PROJECT_ID_DEFINED: ${{ vars.CROWDIN_PROJECT_ID != '' }}
48+
CROWDIN_PERSONAL_TOKEN_DEFINED: ${{ secrets.CROWDIN_PERSONAL_TOKEN != '' }}
49+
CROWDIN_GH_TOKEN_DEFINED: ${{ secrets.CROWDIN_GH_TOKEN != '' }}
50+
outputs:
51+
ready: ${{ steps.check.outputs.flight_ok == 'true' }}
52+
53+
pull_translations:
54+
name: 'Pull translations from Crowdin'
55+
needs: preflight_check
56+
if: needs.preflight_check.outputs.ready == 'true'
57+
runs-on: ubuntu-22.04
58+
concurrency:
59+
group: i18n-pull:${{ github.ref }}
60+
cancel-in-progress: true
61+
steps:
62+
- name: Checkout
63+
uses: actions/checkout@v4
64+
with:
65+
ref: ${{ github.ref }}
66+
token: ${{ secrets.CROWDIN_GH_TOKEN }}
67+
68+
- name: Configure Git author
69+
id: git-author
70+
uses: MarcoIeni/[email protected]
71+
env:
72+
GITHUB_TOKEN: ${{ secrets.CROWDIN_GH_TOKEN }}
73+
74+
# Because --all flag of Crowdin CLI is currently broken we need to create a fake source file
75+
# so that the CLI won't omit translations for it. See https://github.com/crowdin/crowdin-cli/issues/724
76+
- name: Write fake sources
77+
shell: bash
78+
run: echo "{}" > locales/en-US/index.json
79+
80+
- name: Query branch name
81+
id: branch-name
82+
shell: bash
83+
run: |
84+
BRANCH_NAME=$(git rev-parse --abbrev-ref HEAD)
85+
SAFE_BRANCH_NAME=$(echo "$BRANCH_NAME" | sed -e "s/[\\\\/\\:*?\"<>|]/_/g")
86+
echo "Branch name is $BRANCH_NAME (escaped as $SAFE_BRANCH_NAME)"
87+
echo "branch_name=$BRANCH_NAME" >> "$GITHUB_OUTPUT"
88+
echo "safe_branch_name=$SAFE_BRANCH_NAME" >> "$GITHUB_OUTPUT"
89+
90+
- name: Download translations from Crowdin
91+
uses: crowdin/github-action@v1
92+
with:
93+
upload_sources: false
94+
upload_translations: false
95+
download_translations: true
96+
push_translations: false
97+
create_pull_request: false
98+
crowdin_branch_name: '[${{ github.repository_owner }}.${{ github.event.repository.name }}] ${{ steps.branch-name.outputs.safe_branch_name }}'
99+
env:
100+
CROWDIN_PROJECT_ID: ${{ vars.CROWDIN_PROJECT_ID }}
101+
CROWDIN_PERSONAL_TOKEN: ${{ secrets.CROWDIN_PERSONAL_TOKEN }}
102+
103+
- name: Fix broken permissions
104+
shell: bash
105+
run: sudo chown -R $USER:$USER locales
106+
107+
- name: Create Pull Request
108+
uses: peter-evans/create-pull-request@v6
109+
with:
110+
title: 'New translations from Crowdin (${{ steps.branch-name.outputs.branch_name }})'
111+
body: |-
112+
Here is your automated pull request with updated translations from [our Crowdin](https://crowdin.com/project/modrinth) for the `${{ steps.branch-name.outputs.branch_name }}` branch. Merge whenever you ready.
113+
114+
This pull request is created according to the `.github/workflows/i18n-pull.yml` file.
115+
116+
Want to update this pull request? [Dispatch this workflow again](https://github.com/${{ github.repository }}/actions/workflows/i18n-pull.yml).
117+
commit-message: 'New translations from Crowdin (${{ steps.branch-name.outputs.branch_name }})'
118+
branch: crowdin-pull/${{ steps.branch-name.outputs.branch_name }}
119+
add-paths: locales
120+
author: '${{ steps.git-author.outputs.name }} <${{ steps.git-author.outputs.email }}>'
121+
committer: '${{ steps.git-author.outputs.name }} <${{ steps.git-author.outputs.email }}>'
122+
token: ${{ secrets.CROWDIN_GH_TOKEN }}
123+
git-token: ${{ secrets.CROWDIN_GH_TOKEN }}

‎.github/workflows/i18n-push.yml

+149
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,149 @@
1+
name: Crowdin (push)
2+
3+
on:
4+
push: {}
5+
workflow_dispatch: {}
6+
7+
concurrency:
8+
group: i18n-management
9+
10+
jobs:
11+
preflight_check:
12+
name: 'Pre-flight check'
13+
runs-on: ubuntu-22.04
14+
concurrency:
15+
group: i18n-push:${{ github.ref }}
16+
cancel-in-progress: true
17+
steps:
18+
- name: Preflight check
19+
id: check
20+
run: |
21+
PREFLIGHT_CHECK_RESULT=true
22+
23+
function flight_failure () {
24+
if [ "$PREFLIGHT_CHECK_RESULT" = true ]; then
25+
echo "One or more pre-flight checks failed!"
26+
echo ""
27+
PREFLIGHT_CHECK_RESULT=false
28+
fi
29+
echo "- $1"
30+
}
31+
32+
if [ "$CROWDIN_PROJECT_ID_DEFINED" != true ]; then
33+
flight_failure "CROWDIN_PROJECT_ID variable is not defined (required to push)"
34+
fi
35+
36+
if [ "$CROWDIN_PERSONAL_TOKEN_DEFINED" != true ]; then
37+
flight_failure "CROWDIN_PERSONAL_TOKEN secret is not defined (required to push)"
38+
fi
39+
40+
echo "flight_ok=$PREFLIGHT_CHECK_RESULT" >> "$GITHUB_OUTPUT"
41+
env:
42+
CROWDIN_PROJECT_ID_DEFINED: ${{ vars.CROWDIN_PROJECT_ID != '' }}
43+
CROWDIN_PERSONAL_TOKEN_DEFINED: ${{ secrets.CROWDIN_PERSONAL_TOKEN != '' }}
44+
45+
- name: Check that Crowdin branch exists
46+
if: github.event_name != 'workflow_dispatch' && github.ref_name != github.event.repository.default_branch
47+
id: crowdin-branch-exists
48+
uses: GuillaumeFalourd/branch-exists@v1
49+
with:
50+
branch: ${{ format('crowdin-pull/{0}', github.ref_name) }}
51+
52+
- name: Checkout
53+
id: checkout
54+
if: github.event_name != 'workflow_dispatch' && steps.check.outputs.flight_ok == 'true' && (steps.crowdin-branch-exists.outcome == 'skipped' || steps.crowdin-branch-exists.outputs.exists == 'true')
55+
uses: actions/checkout@v4
56+
57+
- name: Confirm push necessity
58+
id: changed-files
59+
if: steps.checkout.outcome != 'skipped'
60+
uses: tj-actions/changed-files@v42
61+
with:
62+
negation_patterns_first: true
63+
files: |
64+
**
65+
locales/en-US/**
66+
crowdin.yml
67+
files_ignore: |
68+
.{github,vscode,idea}/**
69+
{assets,patches,types,public,locales}/**
70+
.{editorconfig,gitignore,npmrc,prettierignore}
71+
.*.{js,json}
72+
*.{md,yml,yaml,json}
73+
LICENSE
74+
75+
- name: Output result
76+
if: steps.changed-files.outcome != 'skipped'
77+
run: |
78+
cat << EOF
79+
Changed files are ${{ steps.changed-files.outputs.all_changed_files }}
80+
EOF
81+
outputs:
82+
ready: ${{ steps.check.outputs.flight_ok == 'true' && (github.event_name == 'workflow_dispatch' || steps.changed-files.outputs.any_changed == 'true') }}
83+
84+
push_translations:
85+
name: Push sources to Crowdin
86+
needs: preflight_check
87+
if: needs.preflight_check.outputs.ready == 'true'
88+
concurrency:
89+
group: i18n-push:${{ github.ref }}
90+
cancel-in-progress: true
91+
runs-on: ubuntu-22.04
92+
steps:
93+
- name: Checkout
94+
uses: actions/checkout@v4
95+
with:
96+
ref: ${{ github.ref }}
97+
98+
- name: Use Node.js
99+
uses: actions/setup-node@v4
100+
with:
101+
node-version: 18.x
102+
103+
- name: Install pnpm via corepack
104+
shell: bash
105+
run: |
106+
corepack enable
107+
corepack prepare --activate
108+
109+
- name: Get pnpm store directory
110+
id: pnpm-cache
111+
shell: bash
112+
run: echo "STORE_PATH=$(pnpm store path)" >> $GITHUB_OUTPUT
113+
114+
- name: Setup pnpm cache
115+
uses: actions/cache@v4
116+
with:
117+
path: ${{ steps.pnpm-cache.outputs.STORE_PATH }}
118+
key: ${{ runner.os }}-pnpm-store-${{ hashFiles('**/pnpm-lock.yaml') }}
119+
restore-keys: |
120+
${{ runner.os }}-pnpm-store-
121+
122+
- name: Install dependencies
123+
run: pnpm install
124+
125+
- name: Extract translations
126+
run: pnpm intl:extract
127+
128+
- name: Query branch name
129+
id: branch-name
130+
shell: bash
131+
run: |
132+
BRANCH_NAME=$(git rev-parse --abbrev-ref HEAD)
133+
SAFE_BRANCH_NAME=$(echo "$BRANCH_NAME" | sed -e "s/[\\\\/\\:*?\"<>|]/_/g")
134+
echo "Branch name is $BRANCH_NAME (escaped as $SAFE_BRANCH_NAME)"
135+
echo "branch_name=$BRANCH_NAME" >> "$GITHUB_OUTPUT"
136+
echo "safe_branch_name=$SAFE_BRANCH_NAME" >> "$GITHUB_OUTPUT"
137+
138+
- name: Upload translations to Crowdin
139+
uses: crowdin/github-action@v1
140+
with:
141+
upload_sources: true
142+
upload_translations: false
143+
download_translations: false
144+
push_translations: false
145+
create_pull_request: false
146+
crowdin_branch_name: '[${{ github.repository_owner }}.${{ github.event.repository.name }}] ${{ steps.branch-name.outputs.safe_branch_name }}'
147+
env:
148+
CROWDIN_PROJECT_ID: ${{ vars.CROWDIN_PROJECT_ID }}
149+
CROWDIN_PERSONAL_TOKEN: ${{ secrets.CROWDIN_PERSONAL_TOKEN }}

‎.gitignore

+3
Original file line numberDiff line numberDiff line change
@@ -83,3 +83,6 @@ sw.*
8383

8484
# Vim swap files
8585
*.swp
86+
87+
# Automatically generated locale files
88+
/locales/en-US/index.json

‎crowdin.yml

+5-2
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,11 @@
1-
project_id: 518556
21
preserve_hierarchy: true
32
commit_message: '[ci skip]'
43

54
files:
6-
- source: /locales/en-US/*
5+
- source: /locales/en-US/*.json
76
dest: /%original_file_name%
87
translation: /locales/%locale%/%original_file_name%
8+
skip_untranslated_strings: true
9+
10+
project_id_env: CROWDIN_PROJECT_ID
11+
api_token_env: CROWDIN_PERSONAL_TOKEN

0 commit comments

Comments
 (0)