With PyAPI_FUNC or PyAPI_DATA¶

Functions or structures inInclude/internal/ defined withPyAPI_FUNC orPyAPI_DATA are internal functions which areexposed only for specific use cases like debuggers and profilers.Ideally, these should be migrated to theUnstable C API.

With the extern keyword¶

Functions inInclude/internal/ defined with theextern keywordmust not and can not be used outside the CPython code base. Onlybuilt-in stdlib extensions (built with thePy_BUILD_CORE_BUILTINmacro defined) can use such functions.

When in doubt, new internal C functions should be defined inInclude/internal using theextern keyword.

Private names¶

Any API named with a leading underscore is also considered internal.There is currently only one main use case for using such names rather thanputting the definition inInclude/internal/ (or directly in a.c file):

• Internal helpers for other public APIs, which users should not call directly.

Note that historically, underscores were used for APIs that are better served bytheUnstable C API:

• “provisional” APIs, included in a Python release to test real-worldusage of new APIs;

• APIs for very specialized uses like JIT compilers.

Internal API tests¶

C tests for the internal C API live inModules/_testinternalcapi.c.Functions namedtest_* are used as tests directly.Python parts of the tests live in various places inLib/test.

Guidelines for expanding/changing the public API¶

• Make sure the new API follows reference counting conventions.(Following them makes the API easier to reason about, and easier usein other Python implementations.)

  • Functionsmust not steal references

  • Functionsmust not return borrowed references

  • Functions returning referencesmust return a strong reference

• Make sure the ownership rules and lifetimes of all applicable structfields, arguments and return values are well defined.

• Functions returningPyObject* must return a valid pointer on success,andNULL with an exception raised on error.Most other API must return-1 with an exception raised on error,and0 on success.

• APIs with lesser and greater results must return0 for the lesser result,and1 for the greater result.Consider a lookup function with a three-way return:

  • return-1: internal error or API misuse; exception raised

  • return0: lookup succeeded; no item was found

  • return1: lookup succeeded; item was found

Please start a public discussion if these guidelines won’t work for your API.

C API tests¶

Tests for the public C API live in the_testcapi module.Functions namedtest_* are used as tests directly.Tests that need Python code (or are just easier to partially write in Python)live inLib/test, mainly inLib/test/test_capi.

Due to its size, the_testcapi module is defined in several sourcefiles.To add a new set of tests (or extract a set out of the monolithicModules/_testcapimodule.c):

• Create a C file namedModules/_testcapi/yourfeature.c

• The file should define a module as usual, except:

  • Instead of<Python.h>, include"parts.h".

  • Instead ofPyInit_modname, define a_PyTestCapi_Init_yourfeaturefunction thattakes the_testcapi module and adds functions/classesto it. (You can usePyModule_AddFunctions to add functions.)

• Add the_PyTestCapi_Init_* function toModules/_testcapi/parts.h

• Call the_PyTestCapi_Init_* fromPyInit__testcapi inModules/_testcapimodule.c.

• Add the new C file toModules/Setup.stdlib.in,PCbuild/_testcapi.vcxproj andPCbuild/_testcapi.vcxproj.filters,alongside the other_testcapi/*.c entries.

Note that allModules/_testcapi/*.c sources initialize the same module,so be careful about name collisions.

When moving existing tests, feel free to replaceTestError withPyExc_AssertionError unless actually testing custom exceptions.

Moving an API from the public tier to Unstable¶

• Expose the API under its new name, with thePyUnstable_ prefix.ThePyUnstable_ prefix must be used for all symbols (functions, macros,variables, etc.).

• Make the old name an alias (for example, astaticinline function calling thenew function).

• Deprecate the old name, typically usingPy_DEPRECATED.

• Announce the change in the “What’s New”.

The old name should continue to be available until an incompatible change ismade. Per Python’s backwards compatibility policy (PEP 387),this deprecation needs to last at least two releases(modulo Steering Council exceptions).

The rules are relaxed for APIs that were introduced in Python versionsbefore 3.12, when the official Unstable tier was added.You can make an incompatible change (and remove the old name)as if the function was already part of the Unstable tierfor APIs introduced before Python 3.12 that are either:

• Documented to be less stable than default.

• Named with a leading underscore.

Moving an API from the private tier to unstable¶

• Expose the API under its new name, with thePyUnstable_ prefix.

• If the old name is documented, or widely used externally,make it an alias and deprecate it (typically withPy_DEPRECATED).It should continue to be available until an incompatible change is made,as if it was previously public.

  This applies even to underscored names. Python wasn’t always strict withthe leading underscore.

• Announce the change in What’s New.

Moving an API from unstable to public¶

• Expose the API under its new name, without thePyUnstable_ prefix.

• Make the oldPyUnstable_* name be an alias (for example, astaticinlinefunction calling the new function).

• Announce the change in What’s New.

The old name should remain available until thenew public name is deprecated or removed.There’s no need to deprecate the old name (it was unstable to begin with),but there’s also no need to break working code just because some functionis now ready for a wider audience.

Guidelines for changing the Limited API, and removing items from it¶

While theStable ABI must not be broken, the existing Limited API can bechanged, and items can be removed from it, if:

• the Backwards Compatibility Policy (PEP 387) is followed, and

• the Stable ABI is not broken – that is, extensions compiled withLimited API of older versions of Python continue to work onnewer versions of Python.

This is tricky to do and requires careful thought.Some examples:

• Functions, structs etc. accessed by macros inany version of theLimited API are part of the Stable ABI, even if they are named withan underscore. They must not be removed and their signature must not change.(Their implementation may change, though.)

• Structs members cannot be rearranged if they were part of any version ofthe Limited API.

• If the Limited API allows users to allocate a struct directly,its size must not change.

• Exported symbols (functions and data) must continue to be availableas exported symbols. Specifically, a function can only be convertedto astaticinline function (or macro) if Python also continues toprovide the actual function.For an example, see thePy_NewRefmacro andredefinition in 3.10.

It is possible to remove items marked as part of the Stable ABI, but onlyif there was no way to use them in any past version of the Limited API.

Guidelines for adding to the Limited API¶

• Guidelines for the generalPublic C API apply.SeeGuidelines for expanding/changing the public API.

• New Limited API should only be defined ifPy_LIMITED_API is setto the version the API was added in or higher.(See below for the proper#if guard.)

• All parameter types, return values, struct members, etc. need to be partof the Limited API.

  • Functions that deal withFILE* (or other types with ABI portabilityissues) should not be added.

• Think twice when defining macros.

  • Macros should not expose implementation details

  • Functions must be exported as actual functions, not (only)as functions-like macros.

  • If possible, avoid macros. This makes the Limited API more usable inlanguages that don’t use the C preprocessor.

• Please start a public discussion before expanding the Limited API

• The Limited API and must follow standard C, not just features of currentlysupported platforms. The exact C dialect is described inPEP 7.

  • Documentation examples (and more generally: the intended use of the API)should also follow standard C.

  • In particular, do not cast a function pointer tovoid* (a data pointer)or vice versa.

• Think about ease of use for the user.

  • In C, ease of use itself is not very important; what is useful isreducing boilerplate code needed to use the API. Bugs like to hide inboiler plates.

  • If a function will be often called with specific value for an argument,consider making it default (used whenNULL is passed in).

  • The Limited API needs to be well documented.

• Think about future extensions

  • If it’s possible that future Python versions will need to add a newfield to your struct, make sure it can be done.

  • Make as few assumptions as possible about implementation details thatmight change in future CPython versions or differ across C APIimplementations. The most important CPython-specific implementationdetails involve:

    • The GIL

    • Garbage collection

    • Memory layout of PyObject, lists/tuples and other structures

If following these guidelines would hurt performance, add a fast function(or macro) to the non-limited API and a stable equivalent to the LimitedAPI.

If anything is unclear, or you have a good reason to break the guidelines,consider discussing the change at thecapi-sig mailing list.

Adding a new definition to the Limited API¶

• Add the declaration to a header file directly underInclude/, into ablock guarded with the following:

  #if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x03yy0000

  with theyy corresponding to the target CPython version, for example,0x030A0000 for Python 3.10.

• Append an entry to the Stable ABI manifest,Misc/stable_abi.toml

• Regenerate the autogenerated files usingmakeregen-limited-abi.On platforms withoutmake, run this command directly:

  ./python./Tools/build/stable_abi.py--generate-all./Misc/stable_abi.toml

• Build Python and check the usingmakecheck-limited-abi.On platforms withoutmake, run this command directly:

  ./python./Tools/build/stable_abi.py--all./Misc/stable_abi.toml

• Add tests – see below.

Limited API tests¶

Since Limited API is a subset of the C API, there’s no need to test thebehavior of individual functions. Rather, the tests could verify that sometask is possible using the exposed subset, or exercise a feature that wasremoved from the current Limited API but still needs to be supported forolder Limited API/Stable ABI versions.

To add a test file:

• Add a C fileModules/_testcapi/yourfeature_limited.c. If that filealready exists but itsPy_LIMITED_API version is too low, add a versionpostfix, for example,yourfeature_limited_3_12.c for Python 3.12+.

• #definePy_LIMITED_API to the minimum limited API version needed.

• #include"parts.h" after thePy_LIMITED_API definition

• Enclose the entire rest of the file in#ifdefLIMITED_API_AVAILABLE,so it’s skipped on incompatible builds.

• Follow the general instructions forC API tests. All additions go in thesections guarded by#ifdefLIMITED_API_AVAILABLE.

Use thetest.support.requires_limited_api decorator for Python testsinLib/test, so they’re skipped on incompatible builds.