Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support loading translation files with importlib.resources files() API #30

Open
MyreMylar opened this issue Nov 21, 2021 · 0 comments
Open

Comments

@MyreMylar
Copy link

Right now the library lets me pass in a string path to a directory containing translation files & internally the files are loaded using standard python file system (os.path) and file loading (open()) code.

I'd also like to be able to pass in a python style package and then the library would go away and use importlib.resources to load all the translation files in that package.

The main reason for my use case is that importlib.resources should work more reliably with various freezing programs such as PyOxidiser & PyInstaller in the future, which is useful for distributing games. Right now when including data files in my GUI library package (in this case the translations for default text on GUI elements) I have to fall back on using the __file__ hack to get a path without importlib.resources . There may be other additional benefits to importlib.resources as it is a fairly recent addition to the python standard library.

Desired user API:

i18n.load_package_path.append("data.translations")

Example of some potential importlib.resources usage internally:

from importlib.resources import files

def load_translation_file_from_package(file_name, package, locale=config.get('locale')):
    skip_locale_root_data = config.get('skip_locale_root_data')
    root_data = None if skip_locale_root_data else locale
    translations_dic = load_resource(package.joinpath(file_name), root_data)

def load_package(package_name, locale=config.get('locale')):
    package = files(package_name)
    for file in package.iterdir():
        if file.is_file() and file.name.endswith(config.get('file_format')):
            if '{locale}' in config.get('filename_format') and not locale in file.name:
                continue
            load_translation_file_from_package(file.name, package, locale)

I've skipped a bunch of the required code there, but from a swift glance over it - it should be possible to adapt what's here already to load using importlib.resources methods like .read_text() which creates a StringIO that can be parsed by the various .json and .yaml parsers.

There are also some shenanigans with configuration needed to support this as:

a) the new files() API to importlib.resources was only added in Python 3.9 replacing an older API.
b) the older API itself only existed in the standard library since Python 3.7
c) However, there is a backport called importlib_resources for pre 3.9 versions of Python - but that has the downside of introducing an additional dependency if you are leery of those (though at least this one is in the standard library past 3.7/3.9).

I'm hoping this is the last time the standard resource loading APIs for python are going to change.

Extra note in case you try testing this: data packages, just like regular python packages, need an __init__.py file in their directory. However since the 3.9 files() API you don't need an __init__.py file in parent directories of your data directory any more so you can do:

main-package
|___code_file.py
|___another_code_file.py
|___data
    |___translations
        |___ __init__.py
        |___ main-package.en.json
        |___ main-pacakge.fr.json
    

Here is also a helpful video on why importlib.resources exists if you want more information than I have provided here:
https://pyvideo.org/pycon-us-2018/get-your-resources-faster-with-importlibresources.html

Useful details on the config required here:
https://discuss.python.org/t/deprecating-importlib-resources-legacy-api/11386

Docs for importlib.resources from the backport:
https://importlib-resources.readthedocs.io/en/latest/index.html

Official documentation from Python 3.9:
https://docs.python.org/3.9/library/importlib.html#module-importlib.resources

P.S. I'm enjoying using the library, making light work of a task I've been wary of, so thanks for creating it!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant