Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Internationalization & localization #1822

Merged
merged 14 commits into from
Nov 9, 2023
1 change: 1 addition & 0 deletions babel.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ module.exports = function (api) {
},
},
],
'macros',
'react-native-reanimated/plugin', // NOTE: this plugin MUST be last
],
env: {
Expand Down
113 changes: 113 additions & 0 deletions docs/internationalization.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
# Internationalization

We want the official Bluesky app to be supported in as many languages as possible. If you want to help us translate the app, please open a PR or issue on the [Bluesky app repo on GitHub](https://github.com/bluesky-social/social-app)

## Tools
We are using Lingui to manage translations. You can find the documentation [here](https://lingui.dev/).

### Adding new strings
When adding a new string, do it as follows:
```jsx
// Before
import { Text } from "react-native";

<Text>Hello World</Text>
```

```jsx
// After
import { Text } from "react-native";
import { Trans } from "@lingui/macro";

<Text><Trans>Hello World</Trans></Text>
```

The `<Trans>` macro will extract the string and add it to the catalog. It is not really a component, but a macro. Further reading [here](https://lingui.dev/ref/macro.html)

However sometimes you will run into this case:
```jsx
// Before
import { Text } from "react-native";

const text = "Hello World";
<Text accessibilityLabel="Label is here">{text}</Text>
```
In this case, you cannot use the `useLingui()` hook:
```jsx
import { msg } from "@lingui/macro";
import { useLingui } from "@lingui/react";

const { _ } = useLingui();
return <Text accessibilityLabel={_(msg`Label is here`)}>{text}</Text>
```

If you want to do this outside of a React component, you can use the `t` macro instead (note: this won't react to changes if the locale is switched dynamically within the app):
```jsx
import { t } from "@lingui/macro";

const text = t`Hello World`;
```

We can then run `yarn intl:extract` to update the catalog in `src/locale/locales/{locale}/messages.po`. This will add the new string to the catalog.
We can then run `yarn intl:compile` to update the translation files in `src/locale/locales/{locale}/messages.js`. This will add the new string to the translation files.
The configuration for translations is defined in `lingui.config.js`

So the workflow is as follows:
1. Wrap messages in Trans macro
2. Run `yarn intl:extract` command to generate message catalogs
3. Translate message catalogs (send them to translators usually)
4. Run `yarn intl:compile` to create runtime catalogs
5. Load runtime catalog
6. Enjoy translated app!

### Common pitfalls
These pitfalls are memoization pitfalls that will cause the components to not re-render when the locale is changed -- causing stale translations to be shown.

```jsx
import { msg } from "@lingui/macro";
import { i18n } from "@lingui/core";

const welcomeMessage = msg`Welcome!`;

// ❌ Bad! This code won't work
ansh marked this conversation as resolved.
Show resolved Hide resolved
export function Welcome() {
const buggyWelcome = useMemo(() => {
return i18n._(welcomeMessage);
}, []);

return <div>{buggyWelcome}</div>;
}

// ❌ Bad! This code won't work either because the reference to i18n does not change
export function Welcome() {
const { i18n } = useLingui();

const buggyWelcome = useMemo(() => {
return i18n._(welcomeMessage);
}, [i18n]);

return <div>{buggyWelcome}</div>;
}

// ✅ Good! `useMemo` has i18n context in the dependency
export function Welcome() {
const linguiCtx = useLingui();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I understand correctly that these pitfalls only matter if we allow changing the locale without reloading the app? I guess that's unavoidable on native?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that is correct.

If we use the hook and properly memoize any functions that use data from the hook, then the UI will react to changes in the locale.

When it's implemented in the whole app, I doubt it'll be be instantaneous as the new locale will need to be loaded, then all components will need to be re-rendered. It'll be like switching from light mode -> dark mode, I think.


const welcome = useMemo(() => {
return linguiCtx.i18n._(welcomeMessage);
}, [linguiCtx]);

return <div>{welcome}</div>;
}

// 🤩 Better! `useMemo` consumes the `_` function from the Lingui context
export function Welcome() {
const { _ } = useLingui();

const welcome = useMemo(() => {
return _(welcomeMessage);
}, [_]);

return <div>{welcome}</div>;
}
```
11 changes: 11 additions & 0 deletions lingui.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
/** @type {import('@lingui/conf').LinguiConfig} */
module.exports = {
locales: ['en', 'cs', 'fr', 'hi', 'es'],
catalogs: [
{
path: '<rootDir>/src/locale/locales/{locale}/messages',
include: ['src'],
},
],
format: 'po',
}
10 changes: 8 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,9 @@
"perf:test:measure": "NODE_ENV=test flashlight test --bundleId xyz.blueskyweb.app --testCommand 'yarn perf:test' --duration 150000 --resultsFilePath .perf/results.json",
"perf:test:results": "NODE_ENV=test flashlight report .perf/results.json",
"perf:measure": "NODE_ENV=test flashlight measure",
"build:apk": "eas build -p android --profile dev-android-apk"
"build:apk": "eas build -p android --profile dev-android-apk",
"intl:extract": "lingui extract",
"intl:compile": "lingui compile"
},
"dependencies": {
"@atproto/api": "^0.6.23",
Expand All @@ -42,6 +44,7 @@
"@fortawesome/free-solid-svg-icons": "^6.1.1",
"@fortawesome/react-native-fontawesome": "^0.3.0",
"@gorhom/bottom-sheet": "^4.5.1",
"@lingui/react": "^4.5.0",
"@mattermost/react-native-paste-input": "^0.6.4",
"@miblanchard/react-native-slider": "^2.3.1",
"@react-native-async-storage/async-storage": "1.18.2",
Expand Down Expand Up @@ -164,10 +167,12 @@
},
"devDependencies": {
"@atproto/dev-env": "^0.2.5",
"@babel/core": "^7.20.0",
"@babel/core": "^7.23.2",
"@babel/preset-env": "^7.20.0",
"@babel/runtime": "^7.20.0",
"@did-plc/server": "^0.0.1",
"@lingui/cli": "^4.5.0",
"@lingui/macro": "^4.5.0",
"@pmmmwh/react-refresh-webpack-plugin": "^0.5.11",
"@react-native-community/eslint-config": "^3.0.0",
"@testing-library/jest-native": "^5.4.1",
Expand All @@ -192,6 +197,7 @@
"@typescript-eslint/parser": "^5.48.2",
"babel-jest": "^29.4.2",
"babel-loader": "^9.1.2",
"babel-plugin-macros": "^3.1.0",
"babel-plugin-module-resolver": "^5.0.0",
"babel-plugin-react-native-web": "^0.18.12",
"detox": "^20.13.0",
Expand Down
12 changes: 9 additions & 3 deletions src/App.web.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@ import {Shell} from 'view/shell/index'
import {ToastContainer} from 'view/com/util/Toast.web'
import {ThemeProvider} from 'lib/ThemeContext'
import {queryClient} from 'lib/react-query'
import {i18n} from '@lingui/core'
import {I18nProvider} from '@lingui/react'
import {defaultLocale, dynamicActivate} from './locale/i18n'
import {Provider as ShellStateProvider} from 'state/shell'
import {Provider as ModalStateProvider} from 'state/modals'
import {Provider as MutedThreadsProvider} from 'state/muted-threads'
Expand All @@ -34,6 +37,7 @@ const InnerApp = observer(function AppImpl() {
setRootStore(store)
analytics.init(store)
})
dynamicActivate(defaultLocale) // async import of locale data
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks concerning to me. What will it be doing while the locale data is loading? Is this blocking? If it's not blocking, how does the UI work in the meantime?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can check the i18n.ts code -- it's just a dynamic import

Copy link
Collaborator

@gaearon gaearon Nov 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand it's a dynamic import — what I don't understand is what strings will we be showing until the default locale loads?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can either show nothing or the default locale (English)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's see if any screen flashing becomes an issue...

Copy link
Collaborator

@gaearon gaearon Nov 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Things to verify on mobile:

  • we shouldn't flash the wrong locale
  • we shouldn't introduce any new async delays; all files are already on the disk

Things to verify on web:

  • we shouldn't flash the wrong locale
  • we shouldn't introduce a waterfall (fetch code -> try to render -> discover we don't have a locale -> download locale). if locale is known at the request time, it should download in parallel with the code.

At FB I think we'd just create a complete JS bundle per locale and force a full reload on locale change. This is nice because you don't have to worry about locale changing dynamically at all.

I wonder if an approach like this would work for us? My concern with locales as separate files is that they grow unbounded: we'd have to download all strings even if we use code splitting for the actual code (and so the initial bundle doesn't need all strings). I think that's the motivation for replacing content in the bundles instead.

Its docs are not great, but I strongly recommend investigating how https://facebook.github.io/fbt/ handles this. I wonder if the open source version does bundle replacement... Can you check how this works? https://github.com/facebook/fbt/tree/main/demo-app

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lingui works very similar to FBT from what I can tell. I'll take a deeper look and then we can adjust the method that we use. However, I do think that not reloading the app on locale change would be a better user experience.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm inclined to say we should bundle the english locale and then dynamically load the preferred locale once we have the preference. It's not ideal but it may be the least worst option - ensures there are strings and keeps the bundle as small as possible.

The calculus is likely different between native and web. Web is the harder one. Caching may help us -- though we also need to ensure dynamic loading works in the prod build

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On mobile, everything will be bundled automatically and then switch when we want it to. I expect our locale messages.js files to be around 20kb. Right now its 14kb for English.

On web, it will be fetched when required. We can cache it and that will definitely help. Here is how it works on web: https://lingui.dev/guides/dynamic-loading-catalogs#conclusion

On RN, we would have to implement our own solution. There has been some prior art on this: lingui/js-lingui#503

}, [])

// show nothing prior to init
Expand All @@ -47,9 +51,11 @@ const InnerApp = observer(function AppImpl() {
<RootSiblingParent>
<analytics.Provider>
<RootStoreProvider value={rootStore}>
<SafeAreaProvider>
<Shell />
</SafeAreaProvider>
<I18nProvider i18n={i18n}>
<SafeAreaProvider>
<Shell />
</SafeAreaProvider>
</I18nProvider>
<ToastContainer />
</RootStoreProvider>
</analytics.Provider>
Expand Down
20 changes: 20 additions & 0 deletions src/locale/i18n.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
import {i18n} from '@lingui/core'

export const locales = {
en: 'English',
cs: 'Česky',
fr: 'Français',
hi: 'हिंदी',
es: 'Español',
}
export const defaultLocale = 'en'

/**
* We do a dynamic import of just the catalog that we need
* @param locale any locale string
*/
export async function dynamicActivate(locale: string) {
const {messages} = await import(`./locales/${locale}/messages`)
i18n.load(locale, messages)
i18n.activate(locale)
}
Loading