Abstract

It is proposed to support__getattr__ and__dir__ function definedon modules to provide basic customization of module attribute access.

Rationale

It is sometimes convenient to customize or otherwise have control overaccess to module attributes. A typical example is managing deprecationwarnings. Typical workarounds are assigning__class__ of a module objectto a custom subclass oftypes.ModuleType or replacing thesys.modulesitem with a custom wrapper instance. It would be convenient to simplify thisprocedure by recognizing__getattr__ defined directly in a module thatwould act like a normal__getattr__ method, except that it will be definedon moduleinstances. For example:

# lib.pyfromwarningsimportwarndeprecated_names=["old_function",...]def_deprecated_old_function(arg,other):...def__getattr__(name):ifnameindeprecated_names:warn(f"{name} is deprecated",DeprecationWarning)returnglobals()[f"_deprecated_{name}"]raiseAttributeError(f"module{__name__!r} has no attribute{name!r}")# main.pyfromlibimportold_function# Works, but emits the warning

Another widespread use case for__getattr__ would be lazy submoduleimports. Consider a simple example:

# lib/__init__.pyimportimportlib__all__=['submod',...]def__getattr__(name):ifnamein__all__:returnimportlib.import_module("."+name,__name__)raiseAttributeError(f"module{__name__!r} has no attribute{name!r}")# lib/submod.pyprint("Submodule loaded")classHeavyClass:...# main.pyimportliblib.submod.HeavyClass# prints "Submodule loaded"

There is a related proposalPEP 549 that proposes to support instanceproperties for a similar functionality. The difference is this PEP proposesa faster and simpler mechanism, but provides more basic customization.An additional motivation for this proposal is thatPEP 484 already definesthe use of module__getattr__ for this purpose in Python stub files,seePEP 484.

In addition, to allow modifying result of adir() call on a moduleto show deprecated and other dynamically generated attributes, it isproposed to support module level__dir__ function. For example:

# lib.pydeprecated_names=["old_function",...]__all__=["new_function_one","new_function_two",...]defnew_function_one(arg,other):...defnew_function_two(arg,other):...def__dir__():returnsorted(__all__+deprecated_names)# main.pyimportlibdir(lib)# prints ["new_function_one", "new_function_two", "old_function", ...]

Specification

The__getattr__ function at the module level should accept one argumentwhich is the name of an attribute and return the computed value or raiseanAttributeError:

def__getattr__(name:str)->Any:...

If an attribute is not found on a module object through the normal lookup(i.e.object.__getattribute__), then__getattr__ is searched inthe module__dict__ before raising anAttributeError. If found, it iscalled with the attribute name and the result is returned. Looking up a nameas a module global will bypass module__getattr__. This is intentional,otherwise calling__getattr__ for builtins will significantly harmperformance.

The__dir__ function should accept no arguments, and returna list of strings that represents the names accessible on module:

def__dir__()->List[str]:...

If present, this function overrides the standarddir() search ona module.

The reference implementation for this PEP can be found in[2].

Backwards compatibility and impact on performance

This PEP may break code that uses module level (global) names__getattr__and__dir__. (But the language reference explicitly reservesallundocumented dunder names, and allows “breakage without warning”; see[3].)The performance implications of this PEP are minimal, since__getattr__is called only for missing attributes.

Some tools that perform module attributes discovery might not expect__getattr__. This problem is not new however, since it is already possibleto replace a module with a module subclass with overridden__getattr__ and__dir__, but with this PEP such problems can occur more often.

Discussion

Note that the use of module__getattr__ requires care to keep the referredobjects pickleable. For example, the__name__ attribute of a functionshould correspond to the name with which it is accessible via__getattr__:

defkeep_pickleable(func):func.__name__=func.__name__.replace('_deprecated_','')func.__qualname__=func.__qualname__.replace('_deprecated_','')returnfunc@keep_pickleabledef_deprecated_old_function(arg,other):...

One should be also careful to avoid recursion as one would do witha class level__getattr__.

To use a module global with triggering__getattr__ (for example if onewants to use a lazy loaded submodule) one can access it as:

sys.modules[__name__].some_global

or as:

from.importsome_global

Note that the latter sets the module attribute, thus__getattr__ will becalled only once.

References

Copyright

This document has been placed in the public domain.