Since Python 1.5 (1997), CPython users can run multiple interpretersin the same process. However, interpreters in the same processhave always shared a significantamount of global state. This is a source of bugs, with a growingimpact as more and more people use the feature. Furthermore,sufficient isolation would facilitate true multi-core parallelism,where interpreters no longer share the GIL. The changes outlined inthis proposal will result in that level of interpreter isolation.
At a high level, this proposal changes CPython in the following ways:
The GIL protects concurrent access to most of CPython’s runtime state.So all that GIL-protected global state must move to each interpreterbefore the GIL can.
(In a handful of cases, other mechanisms can be used to ensurethread-safe sharing instead, such as locks or “immortal” objects.)
Properly isolating interpreters requires that most of CPython’sruntime state be stored in thePyInterpreterState struct. Currently,only a portion of it is; the rest is found either in C global variablesor in_PyRuntimeState. Most of that will have to be moved.
This directly coincides with an ongoing effort (of many years) to greatlyreduce internal use of global variables and consolidate the runtimestate into_PyRuntimeState andPyInterpreterState.(SeeConsolidating Runtime Global State below.) That project hassignificant merit on its ownand has faced little controversy. So, while a per-interpreter GILrelies on the completion of that effort, that project should not beconsidered a part of this proposal–only a dependency.
CPython’s interpreters must be strictly isolated from each other, withfew exceptions. To a large extent they already are. Each interpreterhas its own copy of all modules, classes, functions, and variables.The CPython C-API docsexplain further.
However, aside from what has already been mentioned (e.g. the GIL),there are a couple of ways in which interpreters still share some state.
First of all, some process-global resources (e.g. memory,file descriptors, environment variables) are shared. There are noplans to change this.
Second, some isolation is faulty due to bugs or implementations thatdid not take multiple interpreters into account. This includesCPython’s runtime and the stdlib, as well as extension modules thatrely on global variables. Bugs should be opened in these cases,as some already have been.
PEP 683 introduces immortal objects as a CPython-internal feature.With immortal objects, we can share any otherwise immutable globalobjects between all interpreters. Consequently, this PEP does notneed to address how to deal with the various objectsexposed in the public C-API.It also simplifies the question of what to do about the builtinstatic types. (SeeGlobal Objects below.)
Both issues have alternate solutions, but everything is simpler withimmortal objects. If PEP 683 is not accepted then this one will beupdated with the alternatives. This lets us reduce noise in thisproposal.
The fundamental problem we’re solving here is a lack of true multi-coreparallelism (for Python code) in the CPython runtime. The GIL is thecause. While it usually isn’t a problem in practice, at the very leastit makes Python’s multi-core story murky, which makes the GILa consistent distraction.
Isolated interpreters are also an effective mechanism to supportcertain concurrency models.PEP 554 discusses this in more detail.
Most of the effort needed for a per-interpreter GIL has benefits thatmake those tasks worth doing anyway:
Include/internal)Furthermore, much of that work benefits other CPython-related projects:
The C-API for multiple interpreters has been used for many years.However, until relatively recently the feature wasn’t widely known,nor extensively used (with the exception of mod_wsgi).
In the last few years use of multiple interpreters has been increasing.Here are some of the public projects using the feature currently:
Note that, withPEP 554, multiple interpreter usage would likelygrow significantly (via Python code rather than the C-API).
PEP 554 is strictly about providing a minimal stdlib moduleto give users access to multiple interpreters from Python code.In fact, it specifically avoids proposing any changes related tothe GIL. Consider, however, that users of that module would benefitfrom a per-interpreter GIL, which makes PEP 554 more appealing.
During initial investigations in 2014, a variety of possible solutionsfor multi-core Python were explored, but each had its drawbackswithout simple solutions:
multiprocessingEven in 2014, it was fairly clear that a solution using isolatedinterpreters did not have a high level of technical risk and thatmost of the work was worth doing anyway.(The downside was the volume of work to be done.)
Assummarized above, this proposal involves thefollowing changes, in the order they must happen:
_PyRuntimeStatePyInterpreterStatePyInterpreterStateThe following runtime state will be moved toPyInterpreterState:
Furthermore, a portion of the full global state has already beenmoved to the interpreter, including GC, warnings, and atexit hooks.
The following runtime state will not be moved:
constPyUnicodeObject) are idempotently calculatedwhen first needed and then cachedmain()This is one of the most sensitive parts of the work to isolate interpreters.The simplest solution is to move the global state of the internal“small block” allocator toPyInterpreterState, as we are doing withnearly all other runtime state. The following elaborates on the detailsand rationale.
CPython provides a memory management C-API, withthree allocator domains:“raw”, “mem”, and “object”. Each provides the equivalent ofmalloc(),calloc(),realloc(), andfree(). A custom allocator for eachdomain can be set during runtime initialization and the current allocatorcan be wrapped with a hook using the same API (for example, the stdlibtracemalloc module). The allocators are currently runtime-global,shared by all interpreters.
The “raw” allocator is expected to be thread-safe and defaults to glibc’sallocator (malloc(), etc.). However, the “mem” and “object” allocatorsare not expected to be thread-safe and currently may rely on the GIL forthread-safety. This is partly because the default allocator for both,AKA “pyobject”,is not thread-safe. This is due to how all state forthat allocator is stored in C global variables.(SeeObjects/obmalloc.c.)
Thus we come back to the question of isolating runtime state. In orderfor interpreters to stop sharing the GIL, allocator thread-safetymust be addressed. If interpreters continue sharing the allocatorsthen we need some other way to get thread-safety. Otherwise interpretersmust stop sharing the allocators. In both cases there are a number ofpossible solutions, each with potential downsides.
To keep sharing the allocators, the simplest solution is to usea granular runtime-global lock around the calls to the “mem” and “object”allocators inPyMem_Malloc(),PyObject_Malloc(), etc. This wouldimpact performance, but there are some ways to mitigate that (e.g. onlystart locking once the first subinterpreter is created).
Another way to keep sharing the allocators is to require that the “mem”and “object” allocators be thread-safe. This would mean we’d have tomake the pyobject allocator implementation thread-safe. That couldeven involve re-implementing it using an extensible allocator likemimalloc. The potential downside is in the cost to re-implementthe allocator and the risk of defects inherent to such an endeavor.
Regardless, a switch to requiring thread-safe allocators would impactanyone that embeds CPython and currently sets a thread-unsafe allocator.We’d need to consider who might be affected and how we reduce anynegative impact (e.g. add a basic C-API to help make an allocatorthread-safe).
If we did stop sharing the allocators between interpreters, we’d haveto do so only for the “mem” and “object” allocators. We might also needto keep a full set of global allocators for certain runtime-level usage.There would be some performance penalty due to looking up the currentinterpreter and then pointer indirection to get the allocators.Embedders would also likely have to provide a new allocator contextfor each interpreter. On the plus side, allocator hooks (e.g. tracemalloc)would not be affected.
Ultimately, we will go with the simplest option:
PyInterpreterStateWe experimented witha rough implementation and found it was fairlystraightforward, and the performance penalty was essentially zero.
Internally, the interpreter state will now track how the import systemshould handle extension modules which do not support use with multipleinterpreters. SeeRestricting Extension Modules below. We’ll referto that setting here as “PyInterpreterState.strict_extension_compat”.
The following API will be made public, if they haven’t been already:
PyInterpreterConfig (struct)PyInterpreterConfig_INIT (macro)PyInterpreterConfig_LEGACY_INIT (macro)PyThreadState*Py_NewInterpreterFromConfig(PyInterpreterConfig*)We will add two new fields toPyInterpreterConfig:
intown_gilintstrict_extensions_compatWe may add other fields over time, as needed (e.g. “own_initial_thread”).
Regarding the initializer macros,PyInterpreterConfig_INIT wouldbe used to get an isolated interpreter that also avoidssubinterpreter-unfriendly features. It would be the default forinterpreters created throughPEP 554. The unrestricted (status quo)will continue to be available throughPyInterpreterConfig_LEGACY_INIT,which is already used for the main interpreter andPy_NewInterpreter().This will not change.
A note about the “main” interpreter:
Below, we mention the “main” interpreter several times. This refersto the interpreter created during runtime initialization, for whichthe initialPyThreadState corresponds to the process’s main thread.It is has a number of unique responsibilities (e.g. handling signals),as well as a special role during runtime initialization/finalization.It is also usually (for now) the only interpreter.(Also seehttps://docs.python.org/3/c-api/init.html#sub-interpreter-support.)
Iftrue (1) then the new interpreter will have its own “global”interpreter lock. This means the new interpreter can run withoutgetting interrupted by other interpreters. This effectively unblocksfull use of multiple cores. That is the fundamental goal of this PEP.
Iffalse (0) then the new interpreter will use the maininterpreter’s lock. This is the legacy (pre-3.12) behavior in CPython,where all interpreters share a single GIL. Sharing the GIL like thismay be desirable when using extension modules that still dependon the GIL for thread safety.
InPyInterpreterConfig_INIT, this will betrue.InPyInterpreterConfig_LEGACY_INIT, this will befalse.
Also, to play it safe, for now we will not allowown_gil to be trueif a custom allocator was set during runtime init. Wrapping the allocator,a la tracemalloc, will still be fine.
PyInterpreterConfig.strict_extension_compat is basically the initialvalue used for “PyInterpreterState.strict_extension_compat”.
Extension modules have many of the same problems as the runtime whenstate is stored in global variables.PEP 630 covers all the detailsof what extensions must do to support isolation, and thus safely run inmultiple interpreters at once. This includes dealing with their globals.
If an extension implements multi-phase init (seePEP 489) it isconsidered compatible with multiple interpreters. All other extensionsare considered incompatible. (SeeExtension Module Thread Safetyfor more details about how a per-interpreter GIL may affect thatclassification.)
If an incompatible extension is imported and the current“PyInterpreterState.strict_extension_compat” value istrue then the importsystem will raiseImportError. (Forfalse it simply doesn’t check.)This will be done throughimportlib._bootstrap_external.ExtensionFileLoader (really, through_imp.create_dynamic(),_PyImport_LoadDynamicModuleWithSpec(), andPyModule_FromDefAndSpec2()).
Such imports will never fail in the main interpreter (or in interpreterscreated throughPy_NewInterpreter()) since“PyInterpreterState.strict_extension_compat” initializes tofalse in bothcases. Thus the legacy (pre-3.12) behavior is preserved.
We will work with popular extensions to help them support use inmultiple interpreters. This may involve adding to CPython’s public C-API,which we will address on a case-by-case basis.
As noted inExtension Modules, many extensions work fine in multipleinterpreters (and under a per-interpreter GIL) without needing anychanges. The import system will still fail if such a module doesn’texplicitly indicate support. At first, not many extension moduleswill, so this is a potential source of frustration.
We will address this by adding a context manager to temporarily disablethe check on multiple interpreter support:importlib.util.allow_all_extensions(). More or less, it will modifythe current “PyInterpreterState.strict_extension_compat” value (e.g. througha privatesys function).
If a module supports use with multiple interpreters, that mostly impliesit will work even if those interpreters do not share the GIL. The onecaveat is where a module links against a library with internal globalstate that isn’t thread-safe. (Even something as innocuous as a staticlocal variable as a temporary buffer can be a problem.) With a sharedGIL, that state is protected. Without one, such modules must wrap anyuse of that state (e.g. through calls) with a lock.
Currently, it isn’t clear whether or not supports-multiple-interpretersis sufficiently equivalent to supports-per-interpreter-gil, such thatwe can avoid any special accommodations. This is still a point ofmeaningful discussion and investigation. The practical distinctionbetween the two (in the Python community, e.g. PyPI) is not yetunderstood well enough to settle the matter. Likewise, it isn’t clearwhat we might be able to do to help extension maintainers mitigatethe problem (assuming it is one).
In the meantime, we must proceed as though the difference would belarge enough to cause problems for enough extension modules out there.The solution we would apply is:
PyModuleDef slot that indicates an extension can be importedunder a per-interpreter GIL (i.e. opt in)The downside is that not a single extension module will be able to takeadvantage of the per-interpreter GIL without extra effort by the modulemaintainer, regardless of how minor that effort. This compounds theproblem described inExtension Module Compatibility and the sameworkaround applies. Ideally, we would determine that there isn’t enoughdifference to matter.
If we do end up requiring an opt-in for imports under a per-interpreterGIL, and later determine it isn’t necessary, then we can switch thedefault at that point, make the old opt-in slot a noop, and add a newPyModuleDef slot for explicitly optingout. In fact, it makessense to add that opt-out slot from the beginning.
Doc/c-api/init.rstwill detail the updated APIExtensionFileLoader entry will note importmay fail in subinterpretersimportlib.util.allow_all_extensions()No behavior or APIs are intended to change due to this proposal,with two exceptions:
The existing C-API for managing interpreters will preserve its currentbehavior, with new behavior exposed through new API. No other APIor runtime behavior is meant to change, including compatibility withthe stable ABI.
SeeObjects Exposed in the C-API below for related discussion.
Currently the most common usage of Python, by far, is with the maininterpreter running by itself. This proposal has zero impact onextension modules in that scenario. Likewise, for better or worse,there is no change in behavior under multiple interpreters createdusing the existingPy_NewInterpreter().
Keep in mind that some extensions already break when used in multipleinterpreters, due to keeping module state in global variables (ordue to theinternal state of linked libraries). Theymay crash or, worse, experience inconsistent behavior. That was partof the motivation forPEP 630 and friends, so this is not a newsituation nor a consequence of this proposal.
In contrast, when theproposed API is used tocreate multiple interpreters, with the appropriate settings,the behavior will change for incompatible extensions. In that case,importing such an extension will fail (outside the main interpreter),as explained inRestricting Extension Modules. For extensions thatalready break in multiple interpreters, this will be an improvement.
Additionally, some extension modules link against libraries withthread-unsafe internal global state.(SeeExtension Module Thread Safety.)Such modules will have to start wrapping any direct or indirect useof that state in a lock. This is the key difference from other modulesthat also implement multi-phase init and thus indicate support formultiple interpreters (i.e. isolation).
Now we get to the break in compatibility mentioned above. Someextensions are safe under multiple interpreters (and a per-interpreterGIL), even though they haven’t indicated that. Unfortunately, there isno reliable way for the import system to infer that such an extensionis safe, so importing them will still fail. This case is addressedinExtension Module Compatibility above.
One related consideration is that a per-interpreter GIL will likelydrive increased use of multiple interpreters, particularly ifPEP 554is accepted. Some maintainers of large extension modules have expressedconcern about the increased burden they anticipate due to increaseduse of multiple interpreters.
Specifically, enabling support for multiple interpreters will requiresubstantial work for some extension modules (albeit likely not many).To add that support, the maintainer(s) of such a module (oftenvolunteers) would have to set aside their normal priorities andinterests to focus on compatibility (seePEP 630).
Of course, extension maintainers are free to not add support for usein multiple interpreters. However, users will increasingly demandsuch support, especially if the feature grows in popularity.
Either way, the situation can be stressful for maintainers of suchextensions, particularly when they are doing the work in their sparetime. The concerns they have expressed are understandable, and we addressthe partial solution in theRestricting Extension Modules andExtension Module Compatibility sections.
Other Python implementation are not required to provide support formultiple interpreters in the same process (though some do already).
There is no known impact to security with this proposal.
On the one hand, this proposal has already motivated a number ofimprovements that make CPythonmore maintainable. That is expectedto continue. On the other hand, the underlying work has alreadyexposed various pre-existing defects in the runtime that have hadto be fixed. That is also expected to continue as multiple interpretersreceive more use. Otherwise, there shouldn’t be a significant impacton maintainability, so the net effect should be positive.
The work to consolidate globals has already provided a number ofimprovements to CPython’s performance, both speeding it up and usingless memory, and this should continue. The performance benefits of aper-interpreter GIL specifically have not been explored. At the veryleast, it is not expected to make CPython slower(as long as interpreters are sufficiently isolated). And, obviously,it enable a variety of multi-core parallelism in Python code.
UnlikePEP 554, this is an advanced feature meant for a narrow setof users of the C-API. There is no expectation that the specifics ofthe API nor its direct application will be taught.
That said, if it were taught then it would boil down to the following:
In addition to Py_NewInterpreter(), you can usePy_NewInterpreterFromConfig() to create an interpreter.The config you pass it indicates how you want thatinterpreter to behave.
Furthermore, the maintainers of any extension modules that createisolated interpreters will likely need to explain the consequencesof a per-interpreter GIL to their users. The first thing to explainis whatPEP 554 teaches about the concurrency model that isolatedinterpreters enables. That leads into the point that Python softwarewritten using that concurrency model can then take advantageof multi-core parallelism, which is currentlyprevented by the GIL.
<TBD>
allow_all_extensions?PyInterpreterConfig option to always run the interpreter in a new threadPyInterpreterConfig option to assign a “main” thread to the interpreterand only run in that thread<TBD>
We are sharing some global objects between interpreters.This is an implementation detail and relates more toglobals consolidationthan to this proposal, but it is a significant enough detailto explain here.
The alternative is to share no objects between interpreters, ever.To accomplish that, we’d have to sort out the fate of all our statictypes, as well as deal with compatibility issues for the many objectsexposed in the public C-API.
That approach introduces a meaningful amount of extra complexityand higher risk, though prototyping has demonstrated valid solutions.Also, it would likely result in a performance penalty.
Immortal objects allow us toshare the otherwise immutable global objects. That way we avoidthe extra costs.
The C-API (including the limited API) exposes all the builtin types,including the builtin exceptions, as well as the builtin singletons.The exceptions are exposed asPyObject* but the rest are exposedas the static values rather than pointers. This was one of the fewnon-trivial problems we had to solve for per-interpreter GIL.
With immortal objects this is a non-issue.
As noted inCPython Runtime State above, there is an active effort(separate from this PEP) to consolidate CPython’s global state into the_PyRuntimeState struct. Nearly all the work involves moving thatstate from global variables. The project is particularly relevant tothis proposal, so below is some extra detail.
Consolidating the globals has a variety of benefits:
Furthermore all the benefits listed inIndirect Benefits above alsoapply here, and the same projects listed there benefit.
The number of global variables to be moved is large enough to matter,but most are Python objects that can be dealt with in large groups(likePy_IDENTIFIER). In nearly all cases, moving these globalsto the interpreter is highly mechanical. That doesn’t requirecleverness but instead requires someone to put in the time.
The remaining global variables can be categorized as follows:
Those globals are spread between the core runtime, the builtin modules,and the stdlib extension modules.
For a breakdown of the remaining globals, run:
./pythonTools/c-analyzer/table-file.pyTools/c-analyzer/cpython/globals-to-fix.tsv
As mentioned, this work has been going on for many years. Here are someof the things that have already been done:
_PyRuntimeState_Py_IDENTIFIER()As already indicated, there are several tools to help identify theglobals and reason about them.
Tools/c-analyzer/cpython/globals-to-fix.tsv - the list of remaining globalsTools/c-analyzer/c-analyzer.pyanalyze - identify all the globalscheck - fail if there are any unsupported globals that aren’t ignoredTools/c-analyzer/table-file.py - summarize the known globalsAlso, the check for unsupported globals is incorporated into CI so thatno new globals are accidentally added.
Global objects that are safe to be shared (without a GIL) betweeninterpreters can stay on_PyRuntimeState. Not only must the objectbe effectively immutable (e.g. singletons, strings), but not even therefcount can change for it to be safe. Immortality (PEP 683)provides that. (The alternative is that no objects are shared, whichadds significant complexity to the solution, particularly for theobjectsexposed in the public C-API.)
Builtin static types are a special case of global objects that will beshared. They are effectively immutable except for one part:__subclasses__ (AKAtp_subclasses). We expect that nothingelse on a builtin type will change, even the contentof__dict__ (AKAtp_dict).
__subclasses__ for the builtin types will be dealt with by makingit a getter that does a lookup on the currentPyInterpreterStatefor that type.
Related:
This document is placed in the public domain or under theCC0-1.0-Universal license, whichever is more permissive.
Source:https://github.com/python/peps/blob/main/peps/pep-0684.rst
Last modified:2024-06-04 17:05:36 GMT