Important
This PEP is a historical document. The up-to-date, canonical documentation can now be found atC API Stability (user docs) andChanging Python’s C API (development docs).
×
SeePEP 1 for how to propose changes.
Currently, each feature release introduces a new name for thePython DLL on Windows, and may cause incompatibilities for extensionmodules on Unix. This PEP proposes to define a stable set of APIfunctions which are guaranteed to be available for the lifetimeof Python 3, and which will also remain binary-compatible acrossversions. Extension modules and applications embedding Pythoncan work with different feature releases as long as they restrictthemselves to this stable ABI.
The primary source of ABI incompatibility are changes to the lay-outof in-memory structures. For example, the way in which string interningworks, or the data type used to represent the size of an object, havechanged during the life of Python 2.x. As a consequence, extensionmodules making direct access to fields of strings, lists, or tuples,would break if their code is loaded into a newer version of theinterpreter without recompilation: offsets of other fields may havechanged, making the extension modules access the wrong data.
In some cases, the incompatibilities only affect internal objects ofthe interpreter, such as frame or code objects. For example, the wayline numbers are represented has changed in the 2.x lifetime, as hasthe way in which local variables are stored (due to the introductionof closures). Even though most applications probably never used theseobjects, changing them had required to change the PYTHON_API_VERSION.
On Linux, changes to the ABI are often not much of a problem: thesystem will provide a default Python installation, and many extensionmodules are already provided pre-compiled for that version. If additionalmodules are needed, or additional Python versions, users can typicallycompile them themselves on the system, resulting in modules that usethe right ABI.
On Windows, multiple simultaneous installations of different Pythonversions are common, and extension modules are compiled by theirauthors, not by end users. To reduce the risk of ABI incompatibilities,Python currently introduces a new DLL name pythonXY.dll for eachfeature release, whether or not ABI incompatibilities actually exist.
With this PEP, it will be possible to reduce the dependency of binaryextension modules on a specific Python feature release, and applicationsembedding Python can be made work with different releases.
The ABI specification falls into two parts: an API specification,specifying what function (groups) are available for use with theABI, and a linkage specification specifying what libraries to linkwith. The actual ABI (layout of structures in memory, functioncalling conventions) is not specified, but implied by thecompiler. As a recommendation, a specific ABI is recommended forselected platforms.
During evolution of Python, new ABI functions will be added.Applications using them will then have a requirement on a minimumversion of Python; this PEP provides no mechanism for suchapplications to fall back when the Python library is too old.
Applications and extension modules that want to use this ABIare collectively referred to as “applications” from here on.
Applications shall only include the header file Python.h (beforeincluding any system headers), or, optionally, include pyconfig.h, andthen Python.h.
During the compilation of applications, the preprocessor macroPy_LIMITED_API must be defined. Doing so will hide all definitionsthat are not part of the ABI.
Only the following structures and structure fields are accessible toapplications:
The accessor macros to these fields (Py_REFCNT, Py_TYPE, Py_SIZE)are also available to applications.
The following types are available, but opaque (i.e. incomplete):
The structure of type objects is not available to applications;declaration of “static” type objects is not possible anymore(for applications using this ABI).Instead, type objects get created dynamically. To allow aneasy creation of types (in particular, to be able to fill outfunction pointers easily), the following structures and functionsare available:
typedefstruct{intslot;/*slotid,seebelow*/void*pfunc;/*functionpointer*/}PyType_Slot;typedefstruct{constchar*name;intbasicsize;intitemsize;unsignedintflags;PyType_Slot*slots;/*terminatedbyslot==0.*/}PyType_Spec;PyObject*PyType_FromSpec(PyType_Spec*);
To specify a slot, a unique slot id must be provided. New Pythonversions may introduce new slot ids, but slot ids will never berecycled. Slots may get deprecated, but continue to be supportedthroughout Python 3.x.
The slot ids are named like the field names of the structures thathold the pointers in Python 3.1, with an addedPy_ prefix (i.e.Py_tp_dealloc instead of just tp_dealloc):
The following fields cannot be set during type definition:- tp_dict tp_mro tp_cache tp_subclasses tp_weaklist tp_print- tp_weaklistoffset tp_dictoffset
In addition to the typedefs for structs listed above, the followingtypedefs are available. Their inclusion in the ABI means that theunderlying type must not change on a platform (even though it maydiffer across platforms).
Most notably, Py_UNICODE is not available as a typedef,since the same Python version may use different definitionsof it on the same platform (depending on whether it uses narrowor wide code units). Applications that need to access the contentsof a Unicode string can convert it to wchar_t.
By default, all functions are available, unless they are excludedbelow.Whether a function is documented or not does not matter.
Function-like macros (in particular, field access macros) remainavailable to applications, but get replaced by function calls(unless their definition only refers to features of the ABI, suchas the various _Check macros)
ABI function declarations will not change their parameters or returntypes. If a change to the signature becomes necessary, a new functionwill be introduced. If the new function is source-compatible (e.g. ifjust the return type changes), an alias macro may get added toredirect calls to the new function when the applications isrecompiled.
If continued provision of the old function is not possible, it may getdeprecated, then removed, causingapplications that use that function to break.
All functions starting with _Py are not available to applications.Also, all functions that expect parameter types that are unavailableto applications are excluded from the ABI, such as PyAST_FromNode(which expects anode*).
Functions declared in the following header files are not partof the ABI:
In addition, functions expectingFILE* are not part ofthe ABI, to avoid depending on a specific version of theMicrosoft C runtime DLL on Windows.
Module and type initializer and finalizer functions are not available(PyByteArray_Init, PyOS_FiniInterruptsand all functions ending in _Fini or _ClearFreeList).
Several functions dealing with interpreter implementationdetails are not available:
PyStructSequence_InitType is not available, as it requiresthe caller to provide a static type object.
Py_FatalError will be moved from pydebug.h into some otherheader file (e.g. pyerrors.h).
The exact list of functions being available is givenin the Windows module definition file for python3.dll[1].
Global variables representing types and exceptions are availableto applications. In addition, selected global variables referencedin macros (such as Py_True and Py_False) are available.
A complete list of global variable definitions is given in thepython3.def file[1]; those declared DATA denote variables.
All macros defining symbolic constants are available to applications;the numeric values will not change.
In addition, the following macros are available:
The buffer interface (type Py_buffer, type slots bf_getbuffer andbf_releasebuffer, etc) has been omitted from the ABI, since the stabilityof the Py_buffer structure is not clear at this time. Inclusion in theABI can be considered in future releases.
A number of functions currently expect a specific struct, even thoughcallers typically have PyObject* available. These have been changedto expect PyObject* as the parameter; this will cause warnings inapplications that currently explicitly cast to the parameter type.These functions are PySlice_GetIndices, PySlice_GetIndicesEx,PyUnicode_AsWideChar, and PyEval_EvalCode.
On Windows, applications shall link with python3.dll; an importlibrary python3.lib will be available. This DLL will redirect all ofits API functions through /export linker options to the fullinterpreter DLL, i.e. python3y.dll.
On Unix systems, the ABI is typically provided by the pythonexecutable itself. PyModule_Create is changed to pass3 as the APIversion if the extension module was compiled with Py_LIMITED_API; theversion check for the API version will accept either 3 or the currentPYTHON_API_VERSION as conforming. If Python is compiled as a sharedlibrary, it is installed as both libpython3.so, and libpython3.y.so;applications conforming to this PEP should then link to the former(extension modules can continue to link with no libpython shared object,but rather rely on runtime linking).The ABI version is symbolically available asPYTHON_ABI_VERSION.
Also on Unix, thePEP 3149 tag abi<PYTHON_ABI_VERSION> is acceptedin file names of extension modules. No checking is performed thatfiles named in this way are actually restricted to the limited API,and no support for building such files will be added to distutilsdue to the distutils code freeze.
This PEP will be implemented in a branch[2], allowing users to checkwhether their modules conform to the ABI. To avoid users having torewrite their type definitions, a script to convert C source codecontaining type definitions will be provided[3].
This document has been placed in the public domain.
Source:https://github.com/python/peps/blob/main/peps/pep-0384.rst
Last modified:2025-02-01 08:55:40 GMT