1. Naming

The word “built-in” is overused in Python.From a quick skim of the Python documentation, it mostly refersto things from thebuiltins module.In other words: things which are available in the global namespacewithout a need for importing them.This conflicts with the use of the word “built-in” to mean “implemented in C”.

Solution: since the C structure for built-in functions and methods is alreadycalledPyCFunctionObject,let’s use the name “cfunction” and “cmethod” instead of “built-in function”and “built-in method”.

2. Not extendable

The various classes involved (such asbuiltin_function_or_method)cannot be subclassed:

>>>fromtypesimportBuiltinFunctionType>>>classX(BuiltinFunctionType):...passTraceback (most recent call last):  File"<stdin>", line1, in<module>TypeError:type 'builtin_function_or_method' is not an acceptable base type

This is a problem because it makes it impossible to add featuressuch as introspection support to these classes.

If one wants to implement a function in C with additional functionality,an entirely new class must be implemented from scratch.The problem with this is that the existing classes likebuiltin_function_or_method are special-cased in the Python interpreterto allow faster calling (for example, by usingMETH_FASTCALL).It is currently impossible to have a custom class with the same optimizations.

Solution: make the existing optimizations available to arbitrary classes.This is done by adding a newPyTypeObject fieldtp_ccalloffset(or can we re-usetp_print for that?)specifying the offset of aPyCCallDef pointer.This is a new structure holding all information needed to calla cfunction and it would be used instead ofPyMethodDef.This implements the new “C call” protocol.

For constructing cfunctions and cmethods,PyMethodDef arrayswill still be used (for example, intp_methods) but that willbe theonly remaining purpose of thePyMethodDef structure.

Additionally, we can also make some function classes subclassable.However, this seems less important once we havetp_ccalloffset.

Reference:PEP 580

3. cfunctions do not become methods

A cfunction likerepr does not implement__get__ to bindas a method:

>>>classX:...meth=repr>>>x=X()>>>x.meth()Traceback (most recent call last):  File"<stdin>", line1, in<module>TypeError:repr() takes exactly one argument (0 given)

In this example, one would have expected thatx.meth() returnsrepr(x) by applying the normal rules of methods.

This is surprising and a needless differencebetween cfunctions and Python functions.For the standard built-in functions, this is not really a problemsince those are not meant to used as methods.But it does become a problem when one wants to implement anew cfunction with the goal of being usable as method.

Again, a solution could be to create a new class behaving justlike cfunctions but which bind as methods.However, that would lose some existing optimizations for methods,such as theLOAD_METHOD/CALL_METHOD opcodes.

Solution: the same as the previous issue.It just shows that handlingself and__get__should be part of the new C call protocol.

For backwards compatibility, we would keep the existing non-bindingbehavior of cfunctions. We would just allow it in custom classes.

Reference:PEP 580

4. Semantics of inspect.isfunction

Currently,inspect.isfunction returnsTrue only for instancesoftypes.FunctionType.That is, true Python functions.

A common use case forinspect.isfunction is checking for introspection:it guarantees for example thatinspect.getfile() will work.Ideally, it should be possible for other classes to be treated asfunctions too.

Solution: introduce a newInspectFunction abstract base classand use that to implementinspect.isfunction.Alternatively, use duck typing forinspect.isfunction(as proposed in[2]):

defisfunction(obj):returnhasattr(type(obj),"__code__")

5. C functions should have access to the function object

The underlying C function of a cfunction currentlytakes aself argument (for bound methods)and then possibly a number of arguments.There is no way for the C function to actually access the Pythoncfunction object (theself in__call__ ortp_call).This would for example allow implementing theC call protocol for Python functions (types.FunctionType):the C function which implements calling Python functionsneeds access to the__code__ attribute of the function.

This is also needed forPEP 573where all cfunctions require access to their “parent”(the module for functions of a module or the defining classfor methods).

Solution: add a newPyMethodDef flag to specifythat the C function takes an additional argument (as first argument),namely the function object.

References:PEP 580,PEP 573

6. METH_FASTCALL is private and undocumented

TheMETH_FASTCALL mechanism allows calling cfunctions and cmethodsusing a C array of Python objects instead of atuple.This was introduced in Python 3.6 for positional arguments onlyand extended in Python 3.7 with support for keyword arguments.

However, given that it is undocumented,it is presumably only supposed to be used by CPython itself.

Solution: since this is an important optimization,everybody should be encouraged to use it.Now that the implementation ofMETH_FASTCALL is stable, document it!

As part of the C call protocol, we should also add a C API function

PyObject*PyCCall_FastCall(PyObject*func,PyObject*const*args,Py_ssize_tnargs,PyObject*keywords)

Reference:PEP 580

7. Allowing native C arguments

A cfunction always takes its arguments as Python objects(say, an array ofPyObject pointers).In cases where the cfunction is really wrapping a native C function(for example, coming fromctypes or some compiler like Cython),this is inefficient: calls from C code to C code are forced to usePython objects to pass arguments.

Analogous to the buffer protocol which allows access to C data,we should also allow access to the underlying C callable.

Solution: when wrapping a C function with native arguments(for example, a Clong) inside a cfunction,we should also store a function pointer to the underlying C function,together with its C signature.

Argument Clinic could automatically do this by storinga pointer to the “impl” function.

8. Complexity

There are a huge number of classes involved to implementall variations of methods.This is not a problem by itself, but a compounding issue.

For ordinary Python classes, the table below gives the classesfor various kinds of methods.The columns refer to the class in the class__dict__,the class for unbound methods (bound to the class)and the class for bound methods (bound to the instance):

This is the analogous table for extension types (C classes):

There are a lot of classes involvedand these two tables look very different.There is no good reason why Python methods should betreated fundamentally different from C methods.Also the features are slightly different:for example,method supports__func__butbuiltin_function_or_method does not.

Since CPython has optimizations for calls to most of these objects,the code for dealing with them can also become complex.A good example of this is thecall_function function inPython/ceval.c.

Solution: all these classes should implement the C call protocol.Then the complexity in the code can mostly be fixed bychecking for the C call protocol (tp_ccalloffset!=0)instead of doing type checks.

Furthermore, it should be investigated whether some of these classes can be mergedand whethermethod can be re-used also for bound methods of extension types(seePEP 576 for the latter,keeping in mind that this may have some minor backwards compatibility issues).This is not a goal by itself but just something to keep in mindwhen working on these classes.

9. PyMethodDef is too limited

The typical way to create a cfunction or cmethod in an extension moduleis by using aPyMethodDef to define it.These are then stored in an arrayPyModuleDef.m_methods(for cfunctions) orPyTypeObject.tp_methods (for cmethods).However, because of the stable ABI (PEP 384),we cannot change thePyMethodDef structure.

So, this means that we cannot add new fields for creating cfunctions/cmethodsthis way.This is probably the reason for the hack that__doc__ and__text_signature__ are stored in the same C string(with the__doc__ and__text_signature__ descriptors extractingthe relevant part).

Solution: stop assuming that a singlePyMethodDef entryis sufficient to describe a cfunction/cmethod.Instead, we could add some flag which means that one of thePyMethodDeffields is instead a pointer to an additional structure.Or, we could add a flag to use two or more consecutivePyMethodDefentries in the array to store more data.Then thePyMethodDef array would be used only to constructcfunctions/cmethods but it would no longer be used after that.

10. Slot wrappers have no custom documentation

Right now, slot wrappers like__init__ or__lt__ only have verygeneric documentation, not at all specific to the class:

>>>list.__init__.__doc__'Initialize self.  See help(type(self)) for accurate signature.'>>>list.__lt__.__doc__'Return self<value.'

The same happens for the signature:

>>>list.__init__.__text_signature__'($self, /, *args, **kwargs)'

As you can see, slot wrappers do support__doc__and__text_signature__.The problem is that these are stored instructwrapperbase,which is common for all wrappers of a specific slot(for example, the samewrapperbase is used forstr.__eq__ andint.__eq__).

Solution: rethink the slot wrapper class to allow docstrings(and text signatures) for each instance separately.

This still leaves the question of how extension modulesshould specify the documentation.ThePyTypeObject entries liketp_init are just function pointers,we cannot do anything with those.One solution would be to add entries to thetp_methods arrayjust for adding docstrings.Such an entry could look like

{"__init__",NULL,METH_SLOTDOC,"pointer to __init__ doc goes here"}

11. Static methods and class methods should be callable

Instances ofstaticmethod andclassmethod should be callable.Admittedly, there is no strong use case for this,but it has occasionally been requested (see for example[1]).

Making static/class methods callable would increase consistency.First of all, function decorators typically add functionality or modifya function, but the result remains callable. This is not true for@staticmethod and@classmethod.

Second, class methods of extension types are already callable:

>>>fromhex=float.__dict__["fromhex"]>>>type(fromhex)<class 'classmethod_descriptor'>>>>fromhex(float,"0xff")255.0

Third, one can seefunction,staticmethod andclassmethodas different kinds of unbound methods:they all becomemethod when bound, but the implementation of__get__is slightly different.From this point of view, it looks strange thatfunction is callablebut the others are not.

Solution:when changing the implementation ofstaticmethod,classmethod,we should consider making instances callable.Even if this is not a goal by itself, it may happen naturallybecause of the implementation.