Stable ABI

The CPythonStable ABI is a promise that extensions built againsta specific Stable ABI version will be compatible with any newerinterpreter of the same major version.

The Stable ABI does not define a complete binary interface:important details like the layout of structures in memory or functioncalling conventions are determined by the platform and the compiler andits settings.The Stable ABI promise only applies if these lower-details are also stable.

For example, an extension built with the CPython 3.10 Stable ABI will be usablewith CPython 3.11, 3.12, etc.It will not necessarily be compatible with CPython 4.0, nor with CPython 3.10on a different platform.

The Stable ABI is not generally forward-compatible: an extension built andtested with CPython 3.10 will not generally be compatible with CPython 3.9.

The Stable ABI trades performance for its stability.For example, extensions built for a specific CPython version will automaticallyuse faster macros instead of functions in the Stable ABI.

Future Python versions may deprecate some members of the Stable ABI.Deprecated members will still work, but may suffer from issues like reducedperformance or, in the most extreme cases, memory/resource leaks.

Limited API

The Stable ABI promise holds for extensions compiled from code that restrictsitself to theLimited API (application programming interface).The Limited API is a subset of CPython’s C API.

Extensions that target the Limited API should define the preprocessor macroPy_LIMITED_API to either3 or the currentPYTHON_API_VERSION.This will enable Stable ABI versions of several functions and limit definitionsto the Limited API.(However, note that the macro is not perfect: due to technical issues oroversights, some non-limited API might be exposed even with it defined.)

The Limited API is not guaranteed to bestable.In the future, parts of the Limited API may be deprecated.They may even be removed, as long as theStable ABI is keptstable and Python’s general backwards compatibility policy,PEP 387,is followed.

The goal for the Limited API is to cover everything needed to interactwith the interpreter.The main reason to not include a public API in the Limited subsetshould be that it needs implementation details that change between CPythonversions (like struct memory layouts) – usually for performance reasons.

The Limited API is not limited to CPython. Other implementations areencouraged to implement it and help drive its design.

Stable ABI Manifest

All members of the Stable ABI – functions, typedefs, structs, data, macros,and constants – will be explicitly listed in a single “manifest” file,Misc/stable_abi.txt.

For structs, any fields that users of the Stable ABI are allowed to accesswill be listed explicitly.

The manifest will also serve as the definitive list of the Limited API.Members that are not part of the Limited API, but are part of the Stable ABI(e.g.PyObject.ob_type, which is accessible by thePy_TYPE macro),will be annotated as such.

For items that are only available on some systems, the manifest will record thefeature macro that determines their presence (such asMS_WINDOWS orHAVE_FORK).To make the implementation (and usage from non-C languages) easier,all such macros will be simple names.If a future item needs a “negative” macro or complex expression (such as ahypothetical#ifndefMACOSX or#ifdefined(POSIX)&&!defined(LINUX)),a new feature macro will be derived.

The format of the manifest will be subject to change whenever needed.It should be consumed only by scripts in the CPython repository.If a stable list is needed, a script can be added to generate it.

The following will be generated from the ABI manifest:

• Source for the Windows shared library,PC/python3dll.c.
• Input for documentation (see below).
• Test case that checks the runtime availability of symbols (see below).

The following will be checked against the Stable ABI manifest as part ofcontinuous integration:

• The reference count summary,Doc/data/refcounts.txt, includes allfunction in the Stable ABI (among others).
• The functions/structs declared and constants/macros definedwhenPython.h is included withPy_LIMITED_API set.(Initially Linux only; checks on other systems may be added in the future.)

After the initial implementation, details such as function arguments will beadded and the manifest will be checked for internal consistency (e.g. alltypes used in function signatures are part of the API).

Contents of the Stable ABI

The initial Stable ABI manifest will include:

• The Stable ABI specified inPEP 384.
• Everything listed inPC/python3dll.c.
• All structs (struct typedefs) which these functions return or take asarguments. (Fields of such structs will not necessarily be added.)
• New type slots, such asPy_am_aiter.
• The type flagsPy_TPFLAGS_DEFAULT,Py_TPFLAGS_BASETYPE,Py_TPFLAGS_HAVE_GC,Py_TPFLAGS_METHOD_DESCRIPTOR.
• The calling conventionsMETH_* (except deprecated ones).
• All API needed by macros is the Stable ABI (annotated as not being part ofthe Limited API).

Items that are no longer in CPython when this PEP is accepted will be removedfrom the list.

Additional items may be added to the initial manifest according tothe checklist below.

Documenting the Limited API

Notes saying “Part of the Limited API” will be added to Python’s documentationautomatically, in a way similar to the notes on functions that return borrowedreferences.

A complete list of all members of the Limited API will also be added tothe documentation.

Testing the Stable ABI

An automatically generated test module will be added to ensure that all symbolsincluded in the Stable ABI are available at compile time.

Changing the Limited API

A checklist for changing the Limited API, including adding new items to itand removing existing ones, will be added to theDevguide.The checklist will 1) mention best practices and common pitfalls in PythonC API design and 2) guide the developer around the files that need changing andscripts that need running when the Limited API is changed.

Below is the initial proposal for the checklist.(After the PEP is accepted, see the Devguide for the current version.)

Note that the checklist applies to new changes; several itemsin theexisting Limited API are grandfathered and couldn’t be added today.

Design considerations:

• Make sure the change does not break the Stable ABI of any version of Pythonsince 3.5.
• Make sure no exposed names are private (i.e. begin with an underscore).
• Make sure the new API is well documented.
• Make sure the types of all parameters and return values of the addedfunction(s) and all fields of the added struct(s) are be part of theLimited API (or standard C).
• Make sure the new API and its intended use follows standard C, not justfeatures of currently supported platforms.Specifically, follow the C dialect specified inPEP 7.
  • Do not cast a function pointer tovoid* (a data pointer) or vice versa.
• Make sure the new API follows reference counting conventions. (Following themmakes the API easier to reason about, and easier use in other Pythonimplementations.)
  • Do not return borrowed references from functions.
  • Do not steal references to function arguments.
• Make sure the ownership rules and lifetimes of all applicable struct fields,arguments and return values are well defined.
• Think about ease of use for the user. (In C, ease of use itself is not veryimportant; whatis useful is reducing boilerplate code needed to use theAPI. Bugs like to hide in boiler plates.)
  • If a function will be often called with specific value for an argument,consider making it default (used whenNULL is passed in).
• Think about future extensions: for example, if it’s possible that futurePython versions will need to add a new field to your struct,how will that be done?
• Make as few assumptions as possible about details that might change infuture CPython versions or differ across C API implementations:

  • The GIL
  • Garbage collection
  • Memory layout of PyObject, lists/tuples and other structures

If following these guidelines would hurt performance, add a fast function(or macro) to the non-limited API and a stable equivalent to the Limited API.

If anything is unclear, or you have a good reason to break the guidelines,consider discussing the change at thecapi-sig mailing list.

Procedure:

• Move the declaration to a header file directly underInclude/, into a#if!defined(Py_LIMITED_API)||Py_LIMITED_API+0>=0x03yy0000 block(with theyy corresponding to the target CPython version).
• Make an entry in the Stable ABI manifest,Misc/stable_abi.txt.
• Regenerate the autogenerated files usingmakeregen-all.(or the alternative for non-make platforms)
• Build Python and run checks usingmakecheck-abi.(or the alternative for non-make platforms)

Advice for Extenders and Embedders

The following notes will be added to documentation, along with betterinformation regarding this topic and what guarantees do we offer:

Extension authors should test with all Python versions they support,and preferably build with the lowest such version.

Compiling withPy_LIMITED_API defined isnot a guarantee that your codeconforms to the Limited API or the Stable ABI.Py_LIMITED_API only covers definitions, but an API also includes otherissues, such as expected semantics.

Examples of issues thatPy_LIMITED_API does not guard against are:

• Calling a function with invalid arguments
• A function that started acceptingNULL values for an argumentin Python 3.9 will fail ifNULL is passed to it under Python 3.8.Only testing with 3.8 (or lower versions) will uncover this issue.
• Some structs include a few fields that are part of the Stable ABI and otherfields that aren’t.Py_LIMITED_API does not filter out such “private” fields.
• Code that uses something that is not documented as part of the Stable ABI,but exposed even withPy_LIMITED_API defined, may break in the future.Despite the team’s best efforts, such issues may happen.

Note for Redistributors of Python

The Stable ABI promise relies on stable underlying ABI details, such as thelayout of structures in memory and function calling conventions, whichare affected by the compiler and its settings.For the promise to hold, these details must not change between CPython 3.xreleases on a particular platform.

Defining a process for deprecations/removals

While this PEP acknowledges that parts of the Limited API might be deprecatedor removed in the future, a process to do this is not in scope, and is leftto a possible future PEP.

C syntax for the ABI manifest

It might be useful to have the ABI manifest be a C header file, or togenerate header files from the manifest.Again, either are options for the future.