Abstract

Extension module initialization currently has a few deficiencies.There is no cleanup for modules, the entry point name might givenaming conflicts, the entry functions don’t follow the usual callingconvention, and multiple interpreters are not supported well. This PEPaddresses these issues.

Problems

Specification

The module initialization routines change their signatureto:

PyObject*PyInit_<modulename>()

The initialization routine will be invoked once perinterpreter, when the module is imported. It shouldreturn a new module object each time.

In order to store per-module state in C variables,each module object will contain a block of memorythat is interpreted only by the module. The amountof memory used for the module is specified atthe point of creation of the module.

In addition to the initialization function, a modulemay implement a number of additional callbackfunctions, which are invoked when the module’stp_traverse, tp_clear, and tp_free functions areinvoked, and when the module is reloaded.

The entire module definition is combined in a structPyModuleDef:

structPyModuleDef{PyModuleDef_Basem_base;/*Tobefilledoutbytheinterpreter*/Py_ssize_tm_size;/*Sizeofper-moduledata*/PyMethodDef*m_methods;inquirym_reload;traverseprocm_traverse;inquirym_clear;freefuncm_free;};

Creation of a module is changed to expect an optionalPyModuleDef*. The module state will benull-initialized.

Each module method will be passed the module objectas the first parameter. To access the module data,a function:

void*PyModule_GetState(PyObject*);

will be provided. In addition, to lookup a modulemore efficiently than going through sys.modules,a function:

PyObject*PyState_FindModule(structPyModuleDef*);

will be provided. This lookup function will use anindex located in the m_base field, to find themodule by index, not by name.

As all Python objects should be controlled throughthe Python memory management, usage of “static”type objects is discouraged, unless the type objectitself has no memory-managed state. To simplifydefinition of heap types, a new method:

PyTypeObject*PyType_Copy(PyTypeObject*);

is added.

Example

xxmodule.c would be changed to remove the initxxfunction, and add the following code instead:

struct xxstate{  PyObject *ErrorObject;  PyObject *Xxo_Type;};#define xxstate(o) ((struct xxstate*)PyModule_GetState(o))static int xx_traverse(PyObject *m, visitproc v,                       void *arg){  Py_VISIT(xxstate(m)->ErrorObject);  Py_VISIT(xxstate(m)->Xxo_Type);  return 0;}static int xx_clear(PyObject *m){  Py_CLEAR(xxstate(m)->ErrorObject);  Py_CLEAR(xxstate(m)->Xxo_Type);  return 0;}static struct PyModuleDef xxmodule = {  {}, /* m_base */  sizeof(struct xxstate),  &xx_methods,  0,  /* m_reload */  xx_traverse,  xx_clear,  0,  /* m_free - not needed, since all is done in m_clear */}PyObject*PyInit_xx(){  PyObject *res = PyModule_New("xx", &xxmodule);  if (!res) return NULL;  xxstate(res)->ErrorObject = PyErr_NewException("xx.error", NULL, NULL);  if (!xxstate(res)->ErrorObject) {    Py_DECREF(res);    return NULL;  }  xxstate(res)->XxoType = PyType_Copy(&Xxo_Type);  if (!xxstate(res)->Xxo_Type) {    Py_DECREF(res);    return NULL;  }  return res;}

Discussion

Tim Peters reports in[1] that PythonLabs considered such a featureat one point, and lists the following additional hooks which aren’tcurrently supported in this PEP:

• when the module object is deleted from sys.modules
• when Py_Finalize is called
• when Python exits
• when the Python DLL is unloaded (Windows only)

References

Copyright

This document has been placed in the public domain.