Parent

Thecc_parent field (accessed for example by a__parent__or__objclass__ descriptor from Python code) can be any Pythonobject, or NULL.Custom classes are free to setcc_parent to whatever they want.It is only used by the C call protocol if theCCALL_OBJCLASS flag is set.

For methods of extension types,cc_parent points to the classthat defines the method (which may be a superclass oftype(self)).This is currently non-trivial to retrieve from a method’s code.In the future, this can be used to access the module state viathe defining class. See the rationale ofPEP 573 for details.

When the flagCCALL_OBJCLASS is set (as it will be for methods ofextension types),cc_parent is used for type checks like the following:

>>>list.append({},"x")Traceback (most recent call last):  File"<stdin>", line1, in<module>TypeError:descriptor 'append' requires a 'list' object but received a 'dict'

For functions of modules,cc_parent is set to the module.Currently, this is exactly the same as__self__.However, using__self__ for the module is a quirk of the current implementation:in the future, we want to allow functions which use__self__in the normal way, for implementing methods.Such functions can still usecc_parent instead to refer to the module.

The parent would also typically be used to implement__qualname__.The new C API functionPyCCall_GenericGetQualname() does exactly that.

Using tp_print

We propose to replace the existing unused fieldtp_printbytp_ccalloffset.SincePy_TPFLAGS_HAVE_CCALL wouldnot be added toPy_TPFLAGS_DEFAULT, this ensures full backwards compatibility forexisting extension modules settingtp_print.It also means that we can require thattp_ccalloffset is a validoffset whenPy_TPFLAGS_HAVE_CCALL is specified:we do not need to checktp_ccalloffset!=0.In future Python versions, we may decide thattp_printbecomestp_ccalloffset unconditionally,drop thePy_TPFLAGS_HAVE_CCALL flag and instead check fortp_ccalloffset!=0.

NOTE: the exact layout ofPyTypeObject is not part of thestable ABI).Therefore, changing thetp_print field from aprintfunc (a function pointer)to aPy_ssize_t should not be a problem,even if this changes the memory layout of thePyTypeObject structure.Moreover, on all systems for which binaries are commonly built(Windows, Linux, macOS),the size ofprintfunc andPy_ssize_t are the same,so the issue of binary compatibility will not come up anyway.

Checking __objclass__

If theCCALL_OBJCLASS flag is set and ifcr_self is NULL(this is the case for unbound methods of extension types),then a type check is done:the function must be called with at least one positional argumentand the first (typically calledself) must be an instance ofcc_parent (which must be a class).If not, aTypeError is raised.

Self slicing

Ifcr_self is not NULL or if the flagCCALL_SELFARGis not set incc_flags, then the argument passed asselfis simplycr_self.

Ifcr_self is NULL and the flagCCALL_SELFARG is set,then the first positional argument is removed fromargs and instead passed asself argument to the C function.Effectively, the first positional argument is treated as__self__.If there are no positional arguments,TypeError is raised.

This process is called “self slicing” and a function is said to have selfslicing ifcr_self is NULL andCCALL_SELFARG is set.

Note that aCCALL_NOARGS function with self slicing effectively hasone argument, namelyself.Analogously, aCCALL_O function with self slicing has two arguments.

Descriptor behavior

Classes supporting the C call protocolmust implement the descriptor protocol in a specific way.

This is required for an efficient implementation of bound methods:if other code can make assumptions on what__get__ does,it enables optimizations which would not be possible otherwise.In particular, we want to allow sharingthePyCCallDef structure between bound and unbound methods.We also need a correct implementation of_PyObject_GetMethodwhich is used by theLOAD_METHOD/CALL_METHOD optimization.

First of all, iffunc supports the C call protocol,thenfunc.__set__ andfunc.__delete__ must not be implemented.

Second,func.__get__ must behave as follows:

• Ifcr_self is not NULL, then__get__ must be a no-opin the sense thatfunc.__get__(obj,cls)(*args,**kwds)behaves exactly the same asfunc(*args,**kwds).It is also allowed for__get__ to be not implemented at all.
• Ifcr_self is NULL, thenfunc.__get__(obj,cls)(*args,**kwds)(withobj not None)must be equivalent tofunc(obj,*args,**kwds).In particular,__get__ must be implemented in this case.This is unrelated toself slicing:obj may be passedasself argument to the C function or it may be the first positional argument.
• Ifcr_self is NULL, thenfunc.__get__(None,cls)(*args,**kwds)must be equivalent tofunc(*args,**kwds).

There are no restrictions on the objectfunc.__get__(obj,cls).The latter is not required to implement the C call protocol for example.We only specify whatfunc.__get__(obj,cls).__call__ does.

For classes that do not care about__self__ and__get__ at all,the easiest solution is to assigncr_self=Py_None(or any other non-NULL value).

The __name__ attribute

The C call protocol requires that the function has a__name__attribute which is of typestr (not a subclass).

Furthermore, the object returned by__name__ must be stored somewhere;it cannot be a temporary object.This is required becausePyEval_GetFuncNameuses a borrowed reference to the__name__ attribute(see also[2]).

Generic API functions

This section lists the new public API functions or macrosdealing with the C call protocol.

• intPyCCall_Check(PyObject*op):return true ifop implements the C call protocol.

All the functions and macros belowapply to any instance supporting the C call protocol.In other words,PyCCall_Check(func) must be true.

• PyObject*PyCCall_Call(PyObject*func,PyObject*args,PyObject*kwds):callfunc with positional argumentsargsand keyword argumentskwds (kwds may be NULL).This function is meant to be put in thetp_call slot.
• PyObject*PyCCall_FastCall(PyObject*func,PyObject*const*args,Py_ssize_tnargs,PyObject*kwds):callfunc withnargs positional arguments given byargs[0], …,args[nargs-1].The parameterkwds can be NULL (no keyword arguments),a dict withname:value items or a tuple with keyword names.In the latter case, the keyword values are stored in theargsarray, starting atargs[nargs].

Macros to access thePyCCallRoot andPyCCallDef structures:

• constPyCCallRoot*PyCCall_CCALLROOT(PyObject*func):pointer to thePyCCallRoot structure insidefunc.
• constPyCCallDef*PyCCall_CCALLDEF(PyObject*func):shorthand forPyCCall_CCALLROOT(func)->cr_ccall.
• uint32_tPyCCall_FLAGS(PyObject*func):shorthand forPyCCall_CCALLROOT(func)->cr_ccall->cc_flags.
• PyObject*PyCCall_SELF(PyOject*func):shorthand forPyCCall_CCALLROOT(func)->cr_self.

Generic getters, meant to be put into thetp_getset array:

• PyObject*PyCCall_GenericGetParent(PyObject*func,void*closure):returncc_parent.RaiseAttributeError ifcc_parent is NULL.
• PyObject*PyCCall_GenericGetQualname(PyObject*func,void*closure):return a string suitable for using as__qualname__.This uses the__qualname__ ofcc_parent if possible.It also uses the__name__ attribute.

Profiling

The profiling eventsc_call,c_return andc_exception are only generatedwhen calling actual instances ofbuiltin_function_or_method ormethod_descriptor.This is done for simplicity and also for backwards compatibility(such that the profile function does not receive objects that it does not recognize).In a future PEP, we may extend C-level profiling to arbitrary classesimplementing the C call protocol.

C API functions

The following function is added (also to thestable ABI):

• PyObject*PyCFunction_ClsNew(PyTypeObject*cls,PyMethodDef*ml,PyObject*self,PyObject*module,PyObject*parent):create a new object with object structurePyCFunctionObject and classcls.The entries of thePyMethodDef structure are used to constructthe new object, but the pointer to thePyMethodDef structureis not stored.The flags for the C call protocol are automatically determined in termsofml->ml_flags,self andparent.

The existing functionsPyCFunction_New,PyCFunction_NewEx andPyDescr_NewMethod are implemented in terms ofPyCFunction_ClsNew.

The undocumented functionsPyCFunction_GetFlagsandPyCFunction_GET_FLAGS are deprecated.They are still artificially supported by storing the originalMETH_...flags in a bitfield insidecc_flags.Despite the fact thatPyCFunction_GetFlags is technicallypart of thestable ABI,it is highly unlikely to be used that way:first of all, it is not even documented.Second, the flagMETH_FASTCALLis not part of the stable ABI but it is very common(because of Argument Clinic).So, if one cannot supportMETH_FASTCALL,it is hard to imagine a use case forPyCFunction_GetFlags.The fact thatPyCFunction_GET_FLAGS andPyCFunction_GetFlagsare not used at all by CPython outside ofObjects/call.cfurther shows that these functions are not particularly useful.

Why is this better than PEP 575?

One of the major complaints ofPEP 575 was that is was couplingfunctionality (the calling and introspection protocol)with the class hierarchy:a class could only benefit from the new featuresif it was a subclass ofbase_function.It may be difficult for existing classes to do thatbecause they may have other constraints on the layout of the C object structure,coming from an existing base class or implementation details.For example,functools.lru_cache cannot implementPEP 575 as-is.

It also complicated the implementation precisely because changeswere needed both in the implementation details and in the class hierarchy.

The current PEP does not have these problems.

Why store the function pointer in the instance?

The actual information needed for calling an objectis stored in the instance (in thePyCCallDef structure)instead of the class.This is different from thetp_call slot or earlier attemptsat implementing atp_fastcall slot[1].

The main use case is built-in functions and methods.For those, the C function to be called does depend on the instance.

Note that the current protocol makes it easy to support the casewhere the same C function is called for all instances:just use a single staticPyCCallDef structure for every instance.

Why CCALL_OBJCLASS?

The flagCCALL_OBJCLASS is meant to support various caseswhere the class of aself argument must be checked, such as:

>>>list.append({},None)Traceback (most recent call last):  File"<stdin>", line1, in<module>TypeError:append() requires a 'list' object but received a 'dict'>>>list.__len__({})Traceback (most recent call last):  File"<stdin>", line1, in<module>TypeError:descriptor '__len__' requires a 'list' object but received a 'dict'>>>float.__dict__["fromhex"](list,"0xff")Traceback (most recent call last):  File"<stdin>", line1, in<module>TypeError:descriptor 'fromhex' for type 'float' doesn't apply to type 'list'

In the reference implementation, only the first of these uses the new code.The other examples show that these kind of checks appearin multiple places, so it makes sense to add generic support for them.

Why CCALL_SELFARG?

The flagCCALL_SELFARG and the concept of self slicingare needed to support methods:the C function should not carewhether it is called as unbound method or as bound method.In both cases, there should be aself argumentand this is simply the first positional argument of an unbound method call.

For example,list.append is aMETH_O method.Both the callslist.append([],42) and[].append(42) shouldtranslate to the C calllist_append([],42).

Thanks to the proposed C call protocol, we can support this in such a waythat both the unbound and the bound method share aPyCCallDefstructure (with theCCALL_SELFARG flag set).

So,CCALL_SELFARG has two advantages:there is no extra layer of indirection for calling methodsand constructing bound methods does not require setting up aPyCCallDef structure.

Another minor advantage is that we couldmake the error messages for a wrong call signaturemore uniform between Python methods and built-in methods.In the following example, Python is undecided whethera method takes 1 or 2 arguments:

>>>classList(list):...defmyappend(self,item):...self.append(item)>>>List().myappend(1,2)Traceback (most recent call last):  File"<stdin>", line1, in<module>TypeError:myappend() takes 2 positional arguments but 3 were given>>>List().append(1,2)Traceback (most recent call last):  File"<stdin>", line1, in<module>TypeError:append() takes exactly one argument (2 given)

It is currently impossible forPyCFunction_Callto know the actual number of user-visible argumentssince it cannot distinguish at runtime betweena function (withoutself argument) and a bound method (withself argument).TheCCALL_SELFARG flag makes this difference explicit.

Why CCALL_DEFARG?

The flagCCALL_DEFARG gives the callee access to thePyCCallDef*.There are various use cases for this:

1. The callee can use thecc_parent field, which is useful forPEP 573.
2. Applications are free to extend thePyCCallDef structure with user-definedfields, which can then be accessed analogously.
3. In the case where thePyCCallDef structureis part of the object structure(this is true for example forPyCFunctionObject),an appropriate offset can be subtracted from thePyCCallDef pointerto get a pointer to the callable object defining thatPyCCallDef.

An earlier version of this PEP defined a flagCCALL_FUNCARGinstead ofCCALL_DEFARG which would pass the callable objectto the callee.This had similar use cases, but there was some ambiguity forbound methods: should the “callable object” be the bound methodobject or the original function wrapped by the method?By passing thePyCCallDef* instead, this ambiguity is gonesince the bound method uses thePyCCallDef* from the wrapped function.

Replacing tp_print

We repurposetp_print astp_ccalloffset because this makesit easier for external projects to backport the C call protocolto earlier Python versions.In particular, the Cython project has shown interest in doing that(seehttps://mail.python.org/pipermail/python-dev/2018-June/153927.html).