Abstract

Python currently relies on undefined C behavior, with itsusage ofPyObject_HEAD. This PEP proposes to change thatinto standard C.

Rationale

Standard C defines that an object must be accessed only through apointer of its type, and that all other accesses are undefinedbehavior, with a few exceptions. In particular, the followingcode has undefined behavior:

structFooObject{PyObject_HEADintdata;};PyObject*foo(structFooObject*f){return(PyObject*)f;}intbar(){structFooObject*f=malloc(sizeof(structFooObject));structPyObject*o=foo(f);f->ob_refcnt=0;o->ob_refcnt=1;returnf->ob_refcnt;}

The problem here is that the storage is both accessed asif it where structPyObject, and as structFooObject.

Historically, compilers did not have any problems with thiscode. However, modern compilers use that clause as anoptimization opportunity, finding thatf->ob_refcnt ando->ob_refcnt cannot possibly refer to the same memory, andthat therefore the function should return 0, without havingto fetch the value of ob_refcnt at all in the returnstatement. For GCC, Python now uses-fno-strict-aliasingto work around that problem; with other compilers, itmay just see undefined behavior. Even with GCC, using-fno-strict-aliasing may pessimize the generated codeunnecessarily.

Specification

Standard C has one specific exception to its aliasing rules preciselydesigned to support the case of Python: a value of a struct type mayalso be accessed through a pointer to the first field. E.g. if astruct starts with anint, thestruct* may also be cast toanint*, allowing to write int values into the first field.

For Python,PyObject_HEAD andPyObject_VAR_HEAD will be changedto not list all fields anymore, but list a single field of typePyObject/PyVarObject:

typedefstruct_object{_PyObject_HEAD_EXTRAPy_ssize_tob_refcnt;struct_typeobject*ob_type;}PyObject;typedefstruct{PyObjectob_base;Py_ssize_tob_size;}PyVarObject;#define PyObject_HEAD        PyObject ob_base;#define PyObject_VAR_HEAD    PyVarObject ob_base;

Types defined as fixed-size structure will then include PyObjectas its first field, PyVarObject for variable-sized objects. E.g.:

typedefstruct{PyObjectob_base;PyObject*start,*stop,*step;}PySliceObject;typedefstruct{PyVarObjectob_base;PyObject**ob_item;Py_ssize_tallocated;}PyListObject;

The above definitions ofPyObject_HEAD are normative, so extensionauthors MAY either use the macro, or put theob_base field explicitlyinto their structs.

As a convention, the base field SHOULD be called ob_base. However, allaccesses to ob_refcnt and ob_type MUST cast the object pointer toPyObject* (unless the pointer is already known to have that type), andSHOULD use the respective accessor macros. To simplify access toob_type, ob_refcnt, and ob_size, macros:

#define Py_TYPE(o)    (((PyObject*)(o))->ob_type)#define Py_REFCNT(o)  (((PyObject*)(o))->ob_refcnt)#define Py_SIZE(o)    (((PyVarObject*)(o))->ob_size)

are added. E.g. the code blocks

#define PyList_CheckExact(op) ((op)->ob_type == &PyList_Type)returnfunc->ob_type->tp_name;

needs to be changed to:

#define PyList_CheckExact(op) (Py_TYPE(op) == &PyList_Type)returnPy_TYPE(func)->tp_name;

For initialization of type objects, the current sequence

PyObject_HEAD_INIT(NULL)0,/*ob_size*/

becomes incorrect, and must be replaced with

PyVarObject_HEAD_INIT(NULL,0)

Compatibility with Python 2.6

To support modules that compile with both Python 2.6 and Python 3.0,thePy_* macros are added to Python 2.6. The macrosPy_INCREFandPy_DECREF will be changed to cast their argument toPyObject*,so that module authors can also explicitly declare theob_basefield in modules designed for Python 2.6.

Copyright

This document has been placed in the public domain.