Idea: Add support for lazy module-level properties

[Erotemic]

I have a use-case where I want my kwplot module to have a "module-level property". In other words, I want to define a function in the module and when the name of that function is accessed as an attribute, the function should execute and give the return value. Specifically this is to provide users quick access to the pyplot and seaborn libraries, which have import time side effects I would like to avoid until I explicitly use them.

Previously I have users do something like this when they need plt.

import kwplot
plt = kwplot.autoplt()
plt.figure()

which appropriately sets the backend based on hueristics. However, I've found it much more convenient to have something like:

import kwplot
kwplot.plt.figure()

But this requires hooking up plt as a module-level property so it calls autoplt exactly when needed.

My original proof of concept was something very simple:

     __all__ += ['plt', 'sns']
     def __getattr__(key):
         # Make these special auto-backends top-level dynamic properties of kwplot
         if key == 'plt':
             import kwplot
             return kwplot.autoplt()
         if key == 'sns':
             import kwplot
             return kwplot.autosns()
         raise AttributeError(key)

And this was not used in conjunction with lazy-loader. But now I'm dropping 3.6 and 3.7 as supported versions, so lazy-loader is becoming much more appealing.

I have a proof of concept for this in mkinit, where in __init__.py, the user defines something like:

class __module_properties__:
    """
    experimental mkinit feature for handling module level properties.
    """

    @property
    def plt(self):
        import kwplot
        return kwplot.autoplt()

    @property
    def sns(self):
        import kwplot
        return kwplot.autosns()

    @property
    def Color(self):
        # Backwards compat
        from kwimage import Color
        return Color

And then mkinit does static parsing to parse the names of all properties in that special class if it sees it and then munges the lazy_import (equivalent to lazy_loader.attach) function and effectively inject

def lazy_import(module_name, submodules, submod_attrs, eager='auto'):
    ... ommitted ...
    module_property_names = {'Color', 'plt', 'sns'}
    modprops = __module_properties__()
    def __getattr__(name):
        if name in module_property_names:
            return getattr(modprops, name)
        ... ommitted ...
        return attr
    ... ommitted ...
    return __getattr__

Which would not work here, because we probably don't want to dynamically generate the body of lazy_loader.attach at import time. However, it would not be hard to have lazy_loader.attach take a __module_properties__ class and dynamically infer what its properties are. Perhaps something like this:

-def attach(package_name, submodules=None, submod_attrs=None):
+def attach(package_name, submodules=None, submod_attrs=None, __module_properties__=None):
     """Attach lazily loaded submodules, functions, or other attributes.
 
     Typically, modules import submodules and attributes as follows::
@@ -68,7 +68,13 @@ def attach(package_name, submodules=None, submod_attrs=None):
         attr: mod for mod, attrs in submod_attrs.items() for attr in attrs
     }
 
-    __all__ = sorted(submodules | attr_to_modules.keys())
+    if __module_properties__ is not None:
+        modprops = __module_properties__()
+        property_names = {k for k in dir(__module_properties__) if not k.startswith('__')}
+    else:
+        property_names = {}
+
+    __all__ = sorted(submodules | attr_to_modules.keys() | property_names)
 
     def __getattr__(name):
         if name in submodules:
@@ -86,6 +92,8 @@ def attach(package_name, submodules=None, submod_attrs=None):
                 pkg.__dict__[name] = attr
 
             return attr
+        elif name in property_names:
+            return getattr(modprops, name)
         else:
             raise AttributeError(f"No {package_name} attribute {name}")

Then the user would be responsible for passing the class they want to expose as module-level properties to the attach function.

Another option is to real properties similar to how it is done in this example:

"""
Demo how to add real module level properties

The following code follows [SO1060796]_ to enrich a module with class features
like properties, `__call__()`, and `__iter__()`, etc... for Python versions
3.5+.  In the future, if [PEP713]_ is accepted then that will be preferred.
Note that type checking is ignored here because mypy cannot handle callable
modules [MyPy9240]_.

References
----------
.. [SO1060796] https://stackoverflow.com/questions/1060796/callable-modules
.. [PEP713] https://peps.python.org/pep-0713/
.. [MyPy9240] https://github.com/python/mypy/issues/9240

Example
-------
>>> # Assuming this file is in your cwd named "demo_module_properties.py"
>>> import demo_module_properties
>>> print(demo_module_properties.MY_GLOBAL)
0
>>> print(demo_module_properties.our_property1)
we made a module level property with side effects
>>> print(demo_module_properties.MY_GLOBAL)
1
>>> demo_module_properties()
YOU CALLED ME!
>>> print(list(demo_module_properties))
[1, 2, 3]

"""
import sys

MY_GLOBAL = 0


class OurModule(sys.modules[__name__].__class__):  # type: ignore

    def __iter__(self):
        yield from [1, 2, 3]

    def __call__(self, *args, **kwargs):
        print('YOU CALLED ME!')

    @property
    def our_property1(self):
        global MY_GLOBAL
        MY_GLOBAL += 1
        return 'we made a module level property with side effects'


sys.modules[__name__].__class__ = OurModule
del sys, OurModule

But that seems more heavy handed than the alternative of using the existing module-level __getattr__ functionality.

In any case, I'm experimenting with this feature in mkinit and thought I would at least put a writeup of my ideas here in case others had similar use-cases.