Defining extension modules

A C extension for CPython is a shared library (for example, a.so fileon Linux,.pyd DLL on Windows), which is loadable into the Python process(for example, it is compiled with compatible compiler settings), and whichexports anexport hook function (or anold-styleinitialization function).

To be importable by default (that is, byimportlib.machinery.ExtensionFileLoader),the shared library must be available onsys.path,and must be named after the module name plus an extension listed inimportlib.machinery.EXTENSION_SUFFIXES.

Note

Building, packaging and distributing extension modules is best done withthird-party tools, and is out of scope of this document.One suitable tool is Setuptools, whose documentation can be found athttps://setuptools.pypa.io/en/latest/setuptools.html.

Extension export hook

Added in version 3.15:Support for thePyModExport_<name> export hook was added in Python3.15. The older way of defining modules is still available: consult eitherthePyInit function section or earlier versions of thisdocumentation if you plan to support earlier Python versions.

The export hook must be an exported function with the following signature:

PyModuleDef_Slot*PyModExport_modulename(void)

For modules with ASCII-only names, theexport hookmust be namedPyModExport_<name>,with<name> replaced by the module’s name.

For non-ASCII module names, the export hook must instead be namedPyModExportU_<name> (note theU), with<name> encoded usingPython’spunycode encoding with hyphens replaced by underscores. In Python:

defhook_name(name):try:suffix=b'_'+name.encode('ascii')exceptUnicodeEncodeError:suffix=b'U_'+name.encode('punycode').replace(b'-',b'_')returnb'PyModExport'+suffix

The export hook returns an array ofPyModuleDef_Slot entries,terminated by an entry with a slot ID of0.These slots describe how the module should be created and initialized.

This array must remain valid and constant until interpreter shutdown.Typically, it should usestatic storage.Prefer using thePy_mod_create andPy_mod_exec slotsfor any dynamic behavior.

The export hook may returnNULL with an exception set to signal failure.

It is recommended to define the export hook function using a helper macro:

PyMODEXPORT_FUNC
Part of theStable ABI since version 3.15.

Declare an extension module export hook.This macro:

  • specifies thePyModuleDef_Slot* return type,

  • adds any special linkage declarations required by the platform, and

  • for C++, declares the function asextern"C".

For example, a module calledspam would be defined like this:

PyABIInfo_VAR(abi_info);staticPyModuleDef_Slotspam_slots[]={{Py_mod_abi,&abi_info},{Py_mod_name,"spam"},{Py_mod_init,spam_init_function},...{0,NULL},};PyMODEXPORT_FUNCPyModExport_spam(void){returnspam_slots;}

The export hook is typically the only non-staticitem defined in the module’s C source.

The hook should be kept short – ideally, one line as above.If you do need to use Python C API in this function, it is recommended to callPyABIInfo_Check(&abi_info,"modulename") first to raise an exception,rather than crash, in common cases of ABI mismatch.

Note

It is possible to export multiple modules from a single shared library bydefining multiple export hooks.However, importing them requires a custom importer or suitably namedcopies/links of the extension file, because Python’s import machinery onlyfinds the function corresponding to the filename.See theMultiple modules in one librarysection inPEP 489 for details.

Multi-phase initialization

The process of creating an extension module follows several phases:

  • Python finds and calls the export hook to get information on how tocreate the module.

  • Before any substantial code is executed, Python can determine whichcapabilities the module supports, and it can adjust the environment orrefuse loading an incompatible extension.Slots likePy_mod_abi,Py_mod_gil andPy_mod_multiple_interpreters influence this step.

  • By default, Python itself then creates the module object – that is, it doesthe equivalent of calling__new__() when creating an object.This step can be overridden using thePy_mod_create slot.

  • Python sets initial module attributes like__package__ and__loader__, and inserts the module object intosys.modules.

  • Afterwards, the module object is initialized in an extension-specific way– the equivalent of__init__() when creating an object,or of executing top-level code in a Python-language module.The behavior is specified using thePy_mod_exec slot.

This is calledmulti-phase initialization to distinguish it from the legacy(but still supported)single-phase initialization,where an initialization function returns a fully constructed module.

Changed in version 3.5:Added support for multi-phase initialization (PEP 489).

Multiple module instances

By default, extension modules are not singletons.For example, if thesys.modules entry is removed and the moduleis re-imported, a new module object is created and, typically, populated withfresh method and type objects.The old module is subject to normal garbage collection.This mirrors the behavior of pure-Python modules.

Additional module instances may be created insub-interpretersor after Python runtime reinitialization(Py_Finalize() andPy_Initialize()).In these cases, sharing Python objects between module instances would likelycause crashes or undefined behavior.

To avoid such issues, each instance of an extension module shouldbeisolated: changes to one instance should not implicitly affect the others,and all state owned by the module, including references to Python objects,should be specific to a particular module instance.SeeIsolating Extension Modules for more details and a practical guide.

A simpler way to avoid these issues israising an error on repeated initialization.

All modules are expected to supportsub-interpreters, or otherwise explicitlysignal a lack of support.This is usually achieved by isolation or blocking repeated initialization,as above.A module may also be limited to the main interpreter usingthePy_mod_multiple_interpreters slot.

PyInit function

Deprecated since version 3.15:This functionality issoft deprecated.It will not get new features, but there are no plans to remove it.

Instead ofPyModExport_modulename(), an extension module can definean older-styleinitialization function with the signature:

PyObject*PyInit_modulename(void)

Its name should bePyInit_<name>, with<name> replaced by thename of the module.For non-ASCII module names, usePyInitU_<name> instead, with<name> encoded in the same way as for theexport hook (that is, using Punycodewith underscores).

If a module exports bothPyInit_<name> andPyModExport_<name>, thePyInit_<name> functionis ignored.

Like withPyMODEXPORT_FUNC, it is recommended to define theinitialization function using a helper macro:

PyMODINIT_FUNC

Declare an extension module initialization function.This macro:

  • specifies thePyObject* return type,

  • adds any special linkage declarations required by the platform, and

  • for C++, declares the function asextern"C".

Normally, the initialization function (PyInit_modulename) returnsaPyModuleDef instance with non-NULLm_slots. This allows Python to usemulti-phase initialization.

Before it is returned, thePyModuleDef instance must be initializedusing the following function:

PyObject*PyModuleDef_Init(PyModuleDef*def)
Part of theStable ABI since version 3.5.

Ensure a module definition is a properly initialized Python object thatcorrectly reports its type and a reference count.

Returndef cast toPyObject*, orNULL if an error occurred.

Calling this function is required before returning aPyModuleDeffrom a module initialization function.It should not be used in other contexts.

Note that Python assumes thatPyModuleDef structures are staticallyallocated.This function may return either a new reference or a borrowed one;this reference must not be released.

Added in version 3.5.

For example, a module calledspam would be defined like this:

staticstructPyModuleDefspam_module={.m_base=PyModuleDef_HEAD_INIT,.m_name="spam",...};PyMODINIT_FUNCPyInit_spam(void){returnPyModuleDef_Init(&spam_module);}

Legacy single-phase initialization

Deprecated since version 3.15:Single-phase initialization issoft deprecated.It is a legacy mechanism to initialize extensionmodules, with known drawbacks and design flaws. Extension module authorsare encouraged to use multi-phase initialization instead.

However, there are no plans to remove support for it.

In single-phase initialization, the old-styleinitialization function (PyInit_modulename)should create, populate and return a module object.This is typically done usingPyModule_Create() and functions likePyModule_AddObjectRef().

Single-phase initialization differs from thedefaultin the following ways:

  • Single-phase modules are, or rathercontain, “singletons”.

    When the module is first initialized, Python saves the contents ofthe module’s__dict__ (that is, typically, the module’s functions andtypes).

    For subsequent imports, Python does not call the initialization functionagain.Instead, it creates a new module object with a new__dict__, and copiesthe saved contents to it.For example, given a single-phase module_testsinglephase[1] that defines a functionsum and an exception classerror:

    >>>importsys>>>import_testsinglephaseasone>>>delsys.modules['_testsinglephase']>>>import_testsinglephaseastwo>>>oneistwoFalse>>>one.__dict__istwo.__dict__False>>>one.sumistwo.sumTrue>>>one.erroristwo.errorTrue

    The exact behavior should be considered a CPython implementation detail.

  • To work around the fact thatPyInit_modulename does not take aspecargument, some state of the import machinery is saved and applied to thefirst suitable module created during thePyInit_modulename call.Specifically, when a sub-module is imported, this mechanism prepends theparent package name to the name of the module.

    A single-phasePyInit_modulename function should create “its” moduleobject as soon as possible, before any other module objects can be created.

  • Non-ASCII module names (PyInitU_modulename) are not supported.

  • Single-phase modules support module lookup functions likePyState_FindModule().

  • The module’sPyModuleDef.m_slots must be NULL.

[1]

_testsinglephase is an internal module usedin CPython’s self-test suite; your installation may or may notinclude it.