This PEP proposes a feature to transparently defer the finding and execution ofimported modules until the moment when an imported object is first used. SincePython programs commonly import many more modules than a single invocation ofthe program is likely to use in practice, lazy imports can greatly reduce theoverall number of modules loaded, improving startup time and memory usage. Lazyimports also mostly eliminate the risk of import cycles.
Common Python code styleprefers imports at modulelevel, so they don’t have to be repeated within each scope the imported objectis used in, and to avoid the inefficiency of repeated execution of the importsystem at runtime. This means that importing the main module of a programtypically results in an immediate cascade of imports of most or all of themodules that may ever be needed by the program.
Consider the example of a Python command line program (CLI) with a number ofsubcommands. Each subcommand may perform different tasks, requiring the importof different dependencies. But a given invocation of the program will onlyexecute a single subcommand, or possibly none (i.e. if just--help usageinfo is requested). Top-level eager imports in such a program will result in theimport of many modules that will never be used at all; the time spent (possiblycompiling and) executing these modules is pure waste.
To improve startup time, some large Python CLIs make imports lazy by manuallyplacing imports inline into functions to delay imports of expensive subsystems.This manual approach is labor-intensive and fragile; one misplaced import orrefactor can easily undo painstaking optimization work.
The Python standard library already includes built-in support for lazy imports,viaimportlib.util.LazyLoader.There are also third-party packages such asdemandimport. These provide a “lazy moduleobject” which delays its own import until first attribute access. This is notsufficient to make all imports lazy: imports such asfromfooimporta,bwill still eagerly import the modulefoo since they immediately access anattribute from it. It also imposes noticeable runtime overhead on every moduleattribute access, since it requires a Python-level__getattr__ or__getattribute__ implementation.
Authors of scientific Python packages have also made extensive use of lazyimports to allow users to write e.g.importscipyassp and then easilyaccess many different submodules with e.g.sp.linalg, without requiring allthe many submodules to be imported up-front.SPEC 1 codifies this practice in theform of alazy_loader library that can be used explicitly in a package__init__.py to provide lazily accessible submodules.
Users of static typing also have to import names for use in type annotationsthat may never be used at runtime (ifPEP 563 or possibly in futurePEP 649 are used to avoid eager runtime evaluation of annotations). Lazyimports are very attractive in this scenario to avoid overhead of unneededimports.
This PEP proposes a more general and comprehensive solution for lazy importsthat can encompass all of the above use cases and does not impose detectableoverhead in real-world use. The implementation in this PEP has alreadydemonstratedstartup time improvements up to 70% and memory-use reductions up to 40% onreal-world Python CLIs.
Lazy imports also eliminate most import cycles. With eager imports, “falsecycles” can easily occur which are fixed by simply moving an import to thebottom of a module or inline into a function, or switching fromfromfooimportbar toimportfoo. With lazy imports, these “cycles” just work.The only cycles which will remain are those where two modules actually each usea name from the other at module level; these “true” cycles are only fixable byrefactoring the classes or functions involved.
The aim of this feature is to make imports transparently lazy. “Lazy” meansthat the import of a module (execution of the module body and addition of themodule object tosys.modules) should not occur until the module (or a nameimported from it) is actually referenced during execution. “Transparent” meansthat besides the delayed import (and necessarily observable effects of that,such as delayed import side effects and changes tosys.modules), there isno other observable change in behavior: the imported object is present in themodule namespace as normal and is transparently loaded whenever first used: itsstatus as a “lazy imported object” is not directly observable from Python orfrom C extension code.
The requirement that the imported object be present in the module namespace asusual, even before the import has actually occurred, means that we need somekind of “lazy object” placeholder to represent the not-yet-imported object.The transparency requirement dictates that this placeholder must never bevisible to Python code; any reference to it must trigger the import and replaceit with the real imported object.
Given the possibility that Python (or C extension) code may pull objectsdirectly out of a module__dict__, the only way to reliably preventaccidental leakage of lazy objects is to have the dictionary itself beresponsible to ensure resolution of lazy objects on lookup.
When a lookup finds that the key references a lazy object, it resolves the lazyobject immediately before returning it. To avoid side effects mutatingdictionaries midway through iteration, all lazy objects in a dictionary areresolved prior to starting an iteration; this could incur a performance penaltywhen using bulk iterations (iter(dict),reversed(dict),dict.__reversed__(),dict.keys(),iter(dict.keys()) andreversed(dict.keys())). To avoid this performance penalty on the vastmajority of dictionaries, which never contain any lazy objects, we steal a bitfrom thedk_kind field for a newdk_lazy_imports flag to keep track ofwhether a dictionary may contain lazy objects or not.
This implementation comprehensively prevents leakage of lazy objects, ensuringthey are always resolved to the real imported object before anyone can get holdof them for any use, while avoiding any significant performance impact ondictionaries in general.
Lazy imports are opt-in, and they can be globally enabled either via a new-L flag to the Python interpreter, or via a call to a newimportlib.set_lazy_imports() function. This function takes two arguments, abooleanenabled and anexcluding container. Ifenabled is true, lazyimports will be turned on from that point forward. If it is false, they will beturned off from that point forward. (Use of theexcluding keyword isdiscussed below under “Per-module opt-out.”)
When the flag-L is passed to the Python interpreter, a newsys.flags.lazy_imports is set toTrue, otherwise it exists asFalse.This flag is used to propagate-L to new Python subprocesses.
The flag insys.flags.lazy_imports does not necessarily reflect the currentstatus of lazy imports, only whether the interpreter was started with the-Loption. Actual current status of whether lazy imports are enabled or not at anymoment can be retrieved usingimportlib.is_lazy_imports_enabled(), whichwill returnTrue if lazy imports are enabled at the call point orFalseotherwise.
When lazy imports are enabled, the loading and execution of all (and only)top-level imports is deferred until the imported name is first used. This couldhappen immediately (e.g. on the very next line after the import statement) ormuch later (e.g. while using the name inside a function being called by someother code at some later time.)
For these top level imports, there are two contexts which will make them eager(not lazy): imports insidetry /except /finally orwithblocks, and star imports (fromfooimport*.) Imports insideexception-handling blocks (this includeswith blocks, since those can also“catch” and handle exceptions) remain eager so that any exceptions arising fromthe import can be handled. Star imports must remain eager since performing theimport is the only way to know which names should be added to the namespace.
Imports inside class definitions or inside functions/methods are not “toplevel” and are never lazy.
Dynamic imports using__import__() orimportlib.import_module() arealso never lazy.
Lazy imports state (i.e. whether they have been enabled, and any excludedmodules; see below) is per-interpreter, but global within the interpreter (i.e.all threads will be affected).
Say we have a modulespam.py:
# simulate some workimporttimetime.sleep(10)print("spam loaded")
And a moduleeggs.py which imports it:
importspamprint("imports done")
If we runpython-Leggs.py, thespam module will never be imported(because it is never referenced after the import),"spamloaded" will neverbe printed, and there will be no 10 second delay.
But ifeggs.py simply references the namespam after importing it, thatwill be enough to trigger the import ofspam.py:
importspamprint("imports done")spam
Now if we runpython-Leggs.py, we will see the output"importsdone"printed first, then a 10 second delay, and then"spamloaded" printed afterthat.
Of course, in real use cases (especially with lazy imports), it’s notrecommended to rely on import side effects like this to trigger real work. Thisexample is just to clarify the behavior of lazy imports.
Another way to explain the effect of lazy imports is that it is as if each lazyimport statement had instead been written inline in the source code immediatelybefore each use of the imported name. So one can think of lazy imports assimilar to transforming this code:
importfoodeffunc1():returnfoo.bar()deffunc2():returnfoo.baz()
To this:
deffunc1():importfooreturnfoo.bar()deffunc2():importfooreturnfoo.baz()
This gives a good sense of when the import offoo will occur under lazyimports, but lazy import is not really equivalent to this code transformation.There are several notable differences:
foo still doesexist in the module’s global namespace, and can be imported or referenced byother modules that import this one. (Such references would also trigger theimport.)foo which triggers the import, subsequentreferences will have zero import system overhead; they are indistinguishablefrom a normal name reference.In a sense, lazy imports turn the import statement into just a declaration of animported name or names, to later be fully resolved when referenced.
An import in the stylefromfooimportbar can also be made lazy. When theimport occurs, the namebar will be added to the module namespace as a lazyimport. The first reference tobar will importfoo and resolvebartofoo.bar.
Since lazy imports are a potentially-breaking semantic change, they should beenabled only by the author or maintainer of a Python application, who isprepared to thoroughly test the application under the new semantics, ensure itbehaves as expected, and opt-out any specific imports as needed (see below).Lazy imports should not be enabled speculatively by the end user of a Pythonapplication with any expectation of success.
It is the responsibility of the application developer enabling lazy imports fortheir application to opt-out any library imports that turn out to need to beeager for their application to work correctly; it is not the responsibility oflibrary authors to ensure that their library behaves exactly the same under lazyimports.
The documentation of the feature, the-L flag, and the newimportlibAPIs will be clear about the intended usage and the risks of adoption withouttesting.
Lazy imports are represented internally by a “lazy import” object. When a lazyimport occurs (sayimportfoo orfromfooimportbar), the key"foo"or"bar" is immediately added to the module namespace dictionary, but withits value set to an internal-only “lazy import” object that preserves all thenecessary metadata to execute the import later.
A new boolean flag inPyDictKeysObject (dk_lazy_imports) is set tosignal that this particular dictionary may contain lazy import objects. Thisflag is only used to efficiently resolve all lazy objects in “bulk” operations,when a dictionary may contain lazy objects.
Anytime a key is looked up in a dictionary to extract its value, thevalue is checked to see if it is a lazy import object. If so, the lazy objectis immediately resolved, the relevant imported modules executed, the lazyimport object is replaced in the dictionary (if possible) by the actualimported value, and the resolved value is returned from the lookup function. Adictionary could mutate as part of an import side effect while resolving a lazyimport object. In this case it is not possible to efficiently replace the keyvalue with the resolved object. In this case, the lazy import object will gaina cached pointer to the resolved object. On next access that cached referencewill be returned and the lazy import object will be replaced in the dict withthe resolved value.
Because this is all handled internally by the dictionary implementation, lazyimport objects can never escape from the module namespace to become visible toPython code; they are always resolved at their first reference. No stub, dummyor thunk objects are ever visible to Python code or placed insys.modules.If a module is imported lazily, no entry for it will appear insys.modulesat all until it is actually imported on first reference.
If two different modules (moda andmodb) both contain a lazyimportfoo, each module’s namespace dictionary will have an independent lazy importobject under the key"foo", delaying import of the samefoo module. Thisis not a problem. When there is first a reference to, say,moda.foo, themodulefoo will be imported and placed insys.modules as usual, and thelazy object under the keymoda.__dict__["foo"] will be replaced by theactual modulefoo. At this pointmodb.__dict__["foo"] will remain a lazyimport object. Whenmodb.foo is later referenced, it will also try toimportfoo. This import will find the module already present insys.modules, as is normal for subsequent imports of the same module inPython, and at this point will replace the lazy import object atmodb.__dict__["foo"] with the actual modulefoo.
There are two cases in which a lazy import object can escape a dictionary:
dict.update() anddict.copy(), they do not check for or resolvelazy import objects. However, if the source dict has thedk_lazy_importsflag set that indicates it may contain lazy objects, that flag will bepassed on to the updated/copied dictionary. This still ensures that the lazyimport object can’t escape into Python code without being resolved.gc.get_objects(). If a lazy object becomesvisible to Python code in this way, it is opaque and inert; it has no usefulmethods or attributes. Arepr() of it would be shown as something like:<lazy_object'fully.qualified.name'>.When a lazy object is added to a dictionary, the flagdk_lazy_imports is set.Once set, the flag is only cleared ifall lazy import objects in thedictionary are resolved, e.g. prior to dictionary iteration.
All dictionary iteration methods involving values (such asdict.items(),dict.values(),PyDict_Next() etc.) will attempt to resolveall lazyimport objects in the dictionary prior to starting the iteration. Since only(some) module namespace dictionaries will ever havedk_lazy_imports set, theextra overhead of resolving all lazy import objects inside a dictionary is onlypaid by those dictionaries that need it. Minimizing the overhead on normalnon-lazy dictionaries is the sole purpose of thedk_lazy_imports flag.
PyDict_Next will attempt to resolve all lazy import objects the first timeposition0 is accessed, and those imports could fail with exceptions. SincePyDict_Next cannot set an exception,PyDict_Next will return0immediately in this case, and any exception will be printed to stderr as anunraisable exception.
For this reason, this PEP introducesPyDict_NextWithError, which works inthe same way asPyDict_Next, but which can set an error when returning0and this should be checked viaPyErr_Occurred() after the call.
The eagerness of imports withintry /except /with blocks or withinclass or function bodies is handled in the compiler via a newEAGER_IMPORT_NAME opcode that always imports eagerly. Top-level imports useIMPORT_NAME, which may be lazy or eager depending on-L and/orimportlib.set_lazy_imports().
Debug logging frompython-v will include logging whenever an importstatement has been encountered but execution of the import will be deferred.
Python’s-Ximporttime feature for profiling import costs adapts naturallyto lazy imports; the profiled time is the time spent actually importing.
Although lazy import objects are not generally visible to Python code, in somedebugging cases it may be useful to check from Python code whether the value ata given key in a given dictionary is a lazy import object, without triggeringits resolution. For this purpose,importlib.is_lazy_import() can be used:
fromimportlibimportis_lazy_importimportfoois_lazy_import(globals(),"foo")foois_lazy_import(globals(),"foo")
In this example, if lazy imports have been enabled the first call tois_lazy_import will returnTrue and the second will returnFalse.
Due to the backwards compatibility issues mentioned below, it may be necessaryfor an application using lazy imports to force some imports to be eager.
In first-party code, since imports inside atry orwith block are neverlazy, this can be easily accomplished:
try:# force these imports to be eagerimportfooimportbarfinally:pass
This PEP proposes to add a newimportlib.eager_imports() context manager,so the above technique can be less verbose and doesn’t require comments toclarify its intent:
fromimportlibimporteager_importswitheager_imports():importfooimportbar
Since imports within context managers are always eager, theeager_imports()context manager can just be an alias to a null context manager. The contextmanager’s effect is not transitive:foo andbar will be importedeagerly, but imports within those modules will still follow the usual lazinessrules.
The more difficult case can occur if an import in third-party code that can’teasily be modified must be forced to be eager. For this purpose,importlib.set_lazy_imports() takes a second optional keyword-onlyexcluding argument, which can be set to a container of module names withinwhich all imports will be eager:
fromimportlibimportset_lazy_importsset_lazy_imports(excluding=["one.mod","another"])
The effect of this is also shallow: all imports withinone.mod will beeager, but not imports in all modules imported byone.mod.
Theexcluding parameter ofset_lazy_imports() can be a container of anykind that will be checked to see whether it contains a module name. If themodule name is contained in the object, imports within it will be eager. Thus,arbitrary opt-out logic can be encoded in a__contains__ method:
importrefromimportlibimportset_lazy_importsclassChecker:def__contains__(self,name):returnre.match(r"foo\.[^.]+\.logger",name)set_lazy_imports(excluding=Checker())
If Python was executed with the-L flag, then lazy imports will already beglobally enabled, and the only effect of callingset_lazy_imports(True,excluding=...) will be to globally set the eager module names/callback. Ifset_lazy_imports(True) is called with noexcluding argument, theexclusion list/callback will be cleared and all eligible imports (module-levelimports not intry/except/with, and notimport*) will be lazy from thatpoint forward.
This opt-out system is designed to maintain the possibility of local reasoningabout the laziness of an import. You only need to see the code of one module,and theexcluding argument toset_lazy_imports, if any, to know whethera given import will be eager or lazy.
The CPython test suite will pass with lazy imports enabled (with some testsskipped). One buildbot should run the test suite with lazy imports enabled.
For authors of C extension modules, the proposed public C API is as follows:
| C API | Python API |
|---|---|
PyObject*PyImport_SetLazyImports(PyObject*enabled,PyObject*excluding) | importlib.set_lazy_imports(enabled:bool=True,*,excluding:typing.Container[str]|None=None) |
intPyDict_IsLazyImport(PyObject*dict,PyObject*name) | importlib.is_lazy_import(dict:typing.Dict[str,object],name:str)->bool |
intPyImport_IsLazyImportsEnabled() | importlib.is_lazy_imports_enabled()->bool |
voidPyDict_ResolveLazyImports(PyObject*dict) | |
PyDict_NextWithError() |
voidPyDict_ResolveLazyImports(PyObject*dict) resolves all lazy objectsin a dictionary, if any. To be used prior callingPyDict_NextWithError()orPyDict_Next().PyDict_NextWithError(), works the same way asPyDict_Next(), withthe exception it propagates any errors to the caller by returning0 andsetting an exception. The caller should usePyErr_Occurred() to check for anyerrors.This proposal preserves full backwards compatibility when the feature isdisabled, which is the default.
Even when enabled, most code will continue to work normally without anyobservable change (other than improved startup time and memory usage.)Namespace packages are not affected: they work just as they do currently,except lazily.
In some existing code, lazy imports could produce currently unexpected resultsand behaviors. The problems that we may see when enabling lazy imports in anexisting codebase are related to:
Import side effects that would otherwise be produced by the execution ofimported modules during the execution of import statements will be deferreduntil the imported objects are used.
These import side effects may include:
A relevant and typical affected case is theclick library for building Python command-lineinterfaces. If e.g.cli=click.group() is defined inmain.py, andsub.py importscli frommain and adds subcommands to it viadecorator (@cli.command(...)), but the actualcli() call is inmain.py, then lazy imports may prevent the subcommands from beingregistered, since in this case Click is depending on side effects of the importofsub.py. In this case the fix is to ensure the import ofsub.py iseager, e.g. by using theimportlib.eager_imports() context manager.
There could be issues related to dynamic Python import paths; particularly,adding (and then removing after the import) paths fromsys.path:
sys.path.insert(0,"/path/to/foo/module")importfoodelsys.path[0]foo.Bar()
In this case, with lazy imports enabled, the import offoo will not actuallyoccur while the addition tosys.path is present.
An easy fix for this (which also improves the code style and ensures cleanup)would be to place thesys.path modifications in a context manager. Thisresolves the issue, since imports inside awith block are always eager.
Exceptions that occur during a lazy import bubble up and erase thepartially-constructed module(s) fromsys.modules, just as exceptions duringnormal import do.
Since errors raised during a lazy import will occur later than they would ifthe import were eager (i.e. wherever the name is first referenced), it is alsopossible that they could be accidentally caught by exception handlers that didnot expect the import to be running within theirtry block, leading toconfusion.
Downsides of this PEP include:
Lazy import semantics are already possible and even supported today in thePython standard library, so these drawbacks are not newly introduced by thisPEP. So far, existing usage of lazy imports by some applications has not provena problem. But this PEP could make the usage of lazy imports more popular,potentially exacerbating these drawbacks.
These drawbacks must be weighed against the significant benefits offered by thisPEP’s implementation of lazy imports. Ultimately these costs will be higher ifthe feature is widely used; but wide usage also indicates the feature provides alot of value, perhaps justifying the costs.
Deferred execution of code could produce security concerns if process owner,shell path,sys.path, or other sensitive environment or contextual stateschange between the time theimport statement is executed and the time theimported object is first referenced.
The reference implementation has shown that the feature has negligibleperformance impact on existing real-world codebases (Instagram Server, severalCLI programs at Meta, Jupyter notebooks used by Meta researchers), whileproviding substantial improvements to startup time and memory usage.
The reference implementation showsno measurable changein aggregate performance on thepyperformance benchmark suite.
Since the feature is opt-in, beginners should not encounter it by default.Documentation of the-L flag andimportlib.set_lazy_imports() canclarify the behavior of lazy imports.
The documentation should also clarify that opting into lazy imports is optinginto a non-standard semantics for Python imports, which could cause Pythonlibraries to break in unexpected ways. The responsibility to identify thesebreakages and work around them with an opt-out (or stop using lazy imports)rests entirely with the person choosing to enable lazy imports for theirapplication, not with the library author. Python libraries are under noobligation to support lazy import semantics. Politely reporting anincompatibility may be useful to the library author, but they may choose tosimply say their library does not support use with lazy imports, and this is avalid choice.
Some best practices to deal with some of the issues that could arise and tobetter take advantage of lazy imports are:
from.importbar infoo/__init__.py, always doimportfoo.bar;foo.bar.Baz, notimportfoo;foo.bar.Baz. The latter only works(unreliably) because the attributefoo.bar is added as a side effect offoo.bar being imported somewhere else. With lazy imports this may not alwayshappen in time.The initial implementation is available as part ofCinder. This reference implementationis in use within Meta and has proven to achieve improvements in startup time(and total runtime for some applications) in the range of 40%-70%, as well assignificant reduction in memory footprint (up to 40%), thanks to not needing toexecute imports that end up being unused in the common flow.
Anupdated reference implementation based on CPython main branch is also available.
To reduce the potential for confusion, exceptions raised in thecourse of executing a lazy import could be replaced by aLazyImportErrorexception (a subclass ofImportError), with a__cause__ set to theoriginal exception.
Ensuring that all lazy import errors are raised asLazyImportError wouldreduce the likelihood that they would be accidentally caught and mistaken for adifferent expected exception. However, in practice we have seen cases, e.g.inside tests, where failing modules raiseunittest.SkipTest exception andthis too would end up being wrapped inLazyImportError, making such testsfail because the true exception type is hidden. The drawbacks here seem tooutweigh the hypothetical case where unexpected deferred exceptions are caughtby mistake.
A per-module opt-in using future imports (i.e.from__future__importlazy_imports) does not make sense because__future__ imports are not feature flags, they are for transition tobehaviors which will become default in the future. It is not clear if lazyimports will ever make sense as the default behavior, so we should notpromise this with a__future__ import.
There are other cases where a library might desire to locally opt-in to lazyimports for a particular module; e.g. a lazy top-level__init__.py for alarge library, to make its subcomponents accessible as lazy attributes. For now,to keep the feature simpler, this PEP chooses to focus on the “application” usecase and does not address the library use case. The underlying lazinessmechanism introduced in this PEP could be used in the future to address this usecase as well.
If the primary objective of lazy imports were solely to work around importcycles and forward references, an explicitly-marked syntax for particulartargeted imports to be lazy would make a lot of sense. But in practice it wouldbe very hard to get robust startup time or memory use benefits from thisapproach, since it would require converting most imports within your code base(and in third-party dependencies) to use the lazy import syntax.
It would be possible to aim for a “shallow” laziness where only the top-levelimports of subsystems from the main module are made explicitly lazy, but thenimports within the subsystems are all eager. This is extremely fragile, though– it only takes one mis-placed import to undo the carefully constructedshallow laziness. Globally enabling lazy imports, on the other hand, providesin-depth robust laziness where you always pay only for the imports you use.
There may be use cases (e.g. for static typing) where individually-marked lazyimports are desirable to avoid forward references, but the perf/memory benefitsof globally lazy imports are not needed. Since this is a different set ofmotivating use cases and requires new syntax, we prefer not to include it inthis PEP. Another PEP could build on top of this implementation and propose theadditional syntax.
Providing an environment variable opt-in lends itself too easily to abuse of thefeature. It may seem tempting for a Python user to, for instance, globally setthe environment variable in their shell in the hopes of speeding up all thePython programs they run. This usage with untested programs is likely to lead tospurious bug reports and maintenance burden for the authors of those tools. Toavoid this, we choose not to provide an environment variable opt-in at all.
-L flagWe do provide the-L CLI flag, which could in theory be abused in a similarway by an end user running an individual Python program that is run withpythonsomescript.py orpython-msomescript (rather than distributedvia Python packaging tools). But the potential scope for misuse is much lesswith-L than an environment variable, and-L is valuable for someapplications to maximize startup time benefits by ensuring that all imports fromthe start of a process will be lazy, so we choose to keep it.
It is already the case that running arbitrary Python programs with command lineflags they weren’t intended to be used with (e.g.-s,-S,-E, or-I) can have unexpected and breaking results.-L is nothing new in thisregard.
It would be possible to eagerly run the import loader to the point of findingthe module source, but then defer the actual execution of the module andcreation of the module object. The advantage of this would be that certainclasses of import errors (e.g. a simple typo in the module name) would becaught eagerly instead of being deferred to the use of an imported name.
The disadvantage would be that the startup time benefits of lazy imports wouldbe significantly reduced, since unused imports would still require a filesystemstat() call, at least. It would also introduce a possibly non-obvious splitbetweenwhich import errors are raised eagerly and which are delayed, whenlazy imports are enabled.
This idea is rejected for now on the basis that in practice, confusion aboutimport typos has not been an observed problem with the referenceimplementation. Generally delayed imports are not delayed forever, and errorsshow up soon enough to be caught and fixed (unless the import is truly unused.)
Another possible motivation for half-lazy imports would be to allow modulesthemselves to control via some flag whether they are imported lazily or eagerly.This is rejected both on the basis that it requires half-lazy imports, giving upsome of the performance benefits of import laziness, and because in generalmodules do not decide how or when they are imported, the module importing themdecides that. There isn’t clear rationale for this PEP to invert that control;instead it just provides more options for the importing code to make thedecision.
It would be possible to add alazy=True or similar option to__import__() and/orimportlib.import_module(), to enable them toperform lazy imports. That idea is rejected in this PEP for lack of a clearuse case. Dynamic imports are already far outside thePEP 8 code stylerecommendations for imports, and can easily be made precisely as lazy asdesired by placing them at the desired point in the code flow. These aren’tcommonly used at module top level, which is where lazy imports applies.
The proposedimportlib.eager_imports() context manager and excluded modulesin theimportlib.set_lazy_imports(excluding=...) override all have shalloweffects: they only force eagerness for the location they are applied to, nottransitively. It would be possible to provide a deep/transitive version of oneor both. That idea is rejected in this PEP because the implementation would becomplex (taking into account threads and async code), experience with thereference implementation has not shown it to be necessary, and because itprevents local reasoning about laziness of imports.
A deep override can lead to confusing behavior because thetransitively-imported modules may be imported from multiple locations, some ofwhich use the “deep eager override” and some of which don’t. Thus those modulesmay still be imported lazily initially, if they are first imported from alocation that doesn’t have the override.
With deep overrides it is not possible to locally reason about whether a givenimport will be lazy or eager. With the behavior specified in this PEP, suchlocal reasoning is possible.
Making lazy imports the default/sole behavior of Python imports, instead ofopt-in, would have some long-term benefits, in that library authors would(eventually) no longer need to consider the possibility of both semantics.
However, the backwards-incompatibilies are such that this could only beconsidered over a long time frame, with a__future__ import. It is not atall clear that lazy imports should become the default import semantics forPython.
This PEP takes the position that the Python community needs more experience withlazy imports before considering making it the default behavior, so that isentirely left to a possible future PEP.
This document is placed in the public domain or under theCC0-1.0-Universal license, whichever is more permissive.
Source:https://github.com/python/peps/blob/main/peps/pep-0690.rst
Last modified:2025-02-01 08:55:40 GMT