History of Positional-Only Parameter Semantics in Python

Python originally supported positional-only parameters. Early versions of thelanguage lacked the ability to call functions with arguments bound to parametersby name. Around Python 1.0, parameter semantics changed to bepositional-or-keyword. Since then, users have been able to provide argumentsto a function either positionally or by the keyword name specified in thefunction’s definition.

In current versions of Python, many CPython “builtin” and standard libraryfunctions only accept positional-only parameters. The resulting semantics can beeasily observed by calling one of these functions using keyword arguments:

>>>help(pow)...pow(x, y, z=None, /)...>>>pow(x=5,y=3)Traceback (most recent call last):  File"<stdin>", line1, in<module>TypeError:pow() takes no keyword arguments

pow() expresses that its parameters are positional-only via the/ marker. However, this is only a documentation convention; Pythondevelopers cannot use this syntax in code.

There are functions with other interesting semantics:

• range(), an overloaded function, accepts an optional parameter to theleft of its required parameter.[4]
• dict(), whose mapping/iterator parameter is optional and semanticallymust be positional-only. Any externally visible name for this parameterwould occlude that name going into the**kwarg keyword variadic parameterdict.[3]

One can emulate these semantics in Python code by accepting(*args,**kwargs) and parsing the arguments manually. However, this resultsin a disconnect between the function definition and what the functioncontractually accepts. The function definition does not match the logic of theargument handling.

Additionally, the/ syntax is used beyond CPython for specifying similarsemantics (i.e.,[1][2]); thus, indicating thatthese scenarios are not exclusive to CPython and the standard library.

Problems Without Positional-Only Parameters

Without positional-only parameters, there are challenges for library authorsand users of APIs. The following subsections outline the problemsencountered by each entity.

Benefits of Positional-Only Parameters

Positional-only parameters give more control to library authors to betterexpress the intended usage of an API and allows the API to evolve in a safe,backward-compatible way. Additionally, it makes the Python language moreconsistent with existing documentation and the behavior of various“builtin” and standard library functions.

defas_my_type(x):...

defadd_to_queue(item:QueueItem):...

defadd_to_queue(items:Union[QueueItem,List[QueueItem]]):...

defadd_to_queue(*items:QueueItem):...

io.FileIO.write(self=f,b=b"data")

>>>help(io.FileIO.write)Help on method_descriptor:write(self, b, /)    Write buffer b to file, return number of bytes written.

Performance

In addition to the aforementioned benefits, the parsing and handling ofpositional-only arguments is faster. This performance benefit can bedemonstrated in this thread about converting keyword arguments to positional:[11]. Due to this speedup, there has been a recenttrend towards moving builtins away from keyword arguments: recently,backwards-incompatible changes were made to disallow keyword arguments tobool,float,list,int,tuple.

Maintainability

Providing a way to specify positional-only parameters in Python will make iteasier to maintain pure Python implementations of C modules. Additionally,library authors defining functions will have the choice for choosingpositional-only parameters if they determine that passing a keyword argumentprovides no additional clarity.

This is a well discussed, recurring topic on the Python mailing lists:

• September 2018:Anders Hovmöller: [Python-ideas] Positional-onlyparameters
• February 2017:Victor Stinner: [Python-ideas] Positional-onlyparameters,discussion continued in March
• February 2017:[9]
• March 2012:[8]
• May 2007:George Sakkis: [Python-ideas] Positional only arguments
• May 2006:Benji York: [Python-Dev] Positional-only Arguments

Logical ordering

Positional-only parameters also have the (minor) benefit of enforcing somelogical order when calling interfaces that make use of them. For example, therange function takes all its parameters positionally and disallows formslike:

range(stop=5,start=0,step=2)range(stop=5,step=2,start=0)range(step=2,start=0,stop=5)range(step=2,stop=5,start=0)

at the price of disallowing the use of keyword arguments for the (unique)intended order:

range(start=0,stop=5,step=2)

Compatibility for Pure Python and C Modules

Another critical motivation for positional-only parameters isPEP 399:Pure Python/C Accelerator Module Compatibility Requirements. ThisPEP states that:

This PEP requires that in these instances that the C code must pass thetest suite used for the pure Python code to act as much as a drop-inreplacement as reasonably possible

If the C code is implemented using the existing capabilitiesto implement positional-only parameters using the argument clinic, and relatedmachinery, it is not possible for the pure Python counterpart to match theprovided interface and requirements. This creates a disparity between theinterfaces of some functions and classes in the CPython standard library andother Python implementations. For example:

$ python3 # CPython 3.7.2>>> import binascii; binascii.crc32(data=b'data')TypeError: crc32() takes no keyword arguments$ pypy3 # PyPy 6.0.0>>>> import binascii; binascii.crc32(data=b'data')2918445923

Other Python implementations can reproduce the CPython APIs manually, but thisgoes against the spirit ofPEP 399 to avoid duplication of effort bymandating that all modules added to Python’s standard librarymust have apure Python implementation with the same interface and semantics.

Consistency in Subclasses

Another scenario where positional-only parameters provide benefit occurs when asubclass overrides a method of the base class and changes the name of parametersthat are intended to be positional:

classBase:defmeth(self,arg:int)->str:...classSub(Base):defmeth(self,other_arg:int)->str:...deffunc(x:Base):x.meth(arg=12)func(Sub())# Runtime error

This situation could be considered a Liskov violation — the subclass cannot beused in a context when an instance of the base class is expected. Renamingarguments when overloading methods can happen when the subclass has reasons touse a different choice for the parameter name that is more appropriate for thespecific domain of the subclass (e.g., when subclassingMapping toimplement a DNS lookup cache, the derived class may not want to use the genericargument names ‘key’ and ‘value’ but rather ‘host’ and ‘address’). Having thisfunction definition with positional-only parameters can avoid this problembecause users will not be able to call the interface using keyword arguments.In general, designing for subclassing usually involves anticipating code thathasn’t been written yet and over which the author has no control. Havingmeasures that can facilitate the evolution of interfaces in abackwards-compatible would be useful for library authors.

Optimizations

A final argument in favor of positional-only parameters is that they allow somenew optimizations like the ones already present in the argument clinic due tothe fact that parameters are expected to be passed in strict order. For example, CPython’sinternalMETH_FASTCALL calling convention has been recently specialized forfunctions with positional-only parameters to eliminate the cost for handlingempty keywords. Similar performance improvements can be applied when creatingthe evaluation frame of Python functions thanks to positional-only parameters.

Syntax and Semantics

From the “ten-thousand foot view”, eliding*args and**kwargs forillustration, the grammar for a function definition would look like:

defname(positional_or_keyword_parameters,*,keyword_only_parameters):

Building on that example, the new syntax for function definitions would looklike:

defname(positional_only_parameters,/,positional_or_keyword_parameters,*,keyword_only_parameters):

The following would apply:

• All parameters left of the/ are treated as positional-only.
• If/ is not specified in the function definition, that function does notaccept any positional-only arguments.
• The logic around optional values for positional-only parameters remains thesame as for positional-or-keyword parameters.
• Once a positional-only parameter is specified with a default, thefollowing positional-only and positional-or-keyword parameters need to havedefaults as well.
• Positional-only parameters which do not have defaultvalues arerequired positional-only parameters.

Therefore, the following would be valid function definitions:

defname(p1,p2,/,p_or_kw,*,kw):defname(p1,p2=None,/,p_or_kw=None,*,kw):defname(p1,p2=None,/,*,kw):defname(p1,p2=None,/):defname(p1,p2,/,p_or_kw):defname(p1,p2,/):

Just like today, the following would be valid function definitions:

defname(p_or_kw,*,kw):defname(*,kw):

While the following would be invalid:

defname(p1,p2=None,/,p_or_kw,*,kw):defname(p1=None,p2,/,p_or_kw=None,*,kw):defname(p1=None,p2,/):

Full Grammar Specification

A simplified view of the proposed grammar specification is:

typedargslist:tfpdef['='test](','tfpdef['='test])*',''/'[','# and so onvarargslist:vfpdef['='test](','vfpdef['='test])*',''/'[','# and so on

Based on the reference implementation in this PEP, the new rule fortypedarglist would be:

typedargslist:(tfpdef['='test](','tfpdef['='test])*',''/'[','[tfpdef['='test](','tfpdef['='test])*[','['*'[tfpdef](','tfpdef['='test])*[','['**'tfpdef[',']]]|'**'tfpdef[',']]]|'*'[tfpdef](','tfpdef['='test])*[','['**'tfpdef[',']]]|'**'tfpdef[',']]])|(tfpdef['='test](','tfpdef['='test])*[','['*'[tfpdef](','tfpdef['='test])*[','['**'tfpdef[',']]]|'**'tfpdef[',']]]|'*'[tfpdef](','tfpdef['='test])*[','['**'tfpdef[',']]]|'**'tfpdef[','])

and forvarargslist would be:

varargslist:vfpdef['='test](','vfpdef['='test])*',''/'[','[(vfpdef['='test](','vfpdef['='test])*[','['*'[vfpdef](','vfpdef['='test])*[','['**'vfpdef[',']]]|'**'vfpdef[',']]]|'*'[vfpdef](','vfpdef['='test])*[','['**'vfpdef[',']]]|'**'vfpdef[','])]]|(vfpdef['='test](','vfpdef['='test])*[','['*'[vfpdef](','vfpdef['='test])*[','['**'vfpdef[',']]]|'**'vfpdef[',']]]|'*'[vfpdef](','vfpdef['='test])*[','['**'vfpdef[',']]]|'**'vfpdef[','])

Semantic Corner Case

The following is an interesting corollary of the specification.Consider this function definition:

deffoo(name,**kwds):return'name'inkwds

There is no possible call that will make it returnTrue.For example:

>>>foo(1,**{'name':2})Traceback (most recent call last):  File"<stdin>", line1, in<module>TypeError:foo() got multiple values for argument 'name'>>>

But using/ we can support this:

deffoo(name,/,**kwds):return'name'inkwds

Now the above call will returnTrue.

In other words, the names of positional-only parameters can be used in**kwds without ambiguity. (As another example, this benefits thesignatures ofdict() anddict.update().)

Origin of “/” as a Separator

Using/ as a separator was initially proposed by Guido van Rossumin 2012[8] :

Alternative proposal: how about using ‘/’ ? It’s kind of the oppositeof ‘*’ which means “keyword argument”, and ‘/’ is not a new character.

Positional-or-Keyword Arguments

If/ and* are not present in the function definition, arguments maybe passed to a function by position or by keyword.

Positional-Only Parameters

Looking at this in a bit more detail, it is possible to mark certain parametersaspositional-only. Ifpositional-only, the parameters’ order matters, andthe parameters cannot be passed by keyword. Positional-only parameters wouldbe placed before a/ (forward-slash). The/ is used to logicallyseparate the positional-only parameters from the rest of the parameters.If there is no/ in the function definition, there are no positional-onlyparameters.

Parameters following the/ may bepositional-or-keyword orkeyword-only.

Keyword-Only Arguments

To mark parameters askeyword-only, indicating the parameters must be passedby keyword argument, place an* in the arguments list just before the firstkeyword-only parameter.

Function Examples

Consider the following example function definitions paying close attention to themarkers/ and*:

>>>defstandard_arg(arg):...print(arg)...>>>defpos_only_arg(arg,/):...print(arg)...>>>defkwd_only_arg(*,arg):...print(arg)...>>>defcombined_example(pos_only,/,standard,*,kwd_only):...print(pos_only,standard,kwd_only)

The first function definitionstandard_arg, the most familiar form,places no restrictions on the calling convention and arguments may bepassed by position or keyword:

>>>standard_arg(2)2>>>standard_arg(arg=2)2

The second functionpos_only_arg is restricted to only use positionalparameters as there is a/ in the function definition:

>>>pos_only_arg(1)1>>>pos_only_arg(arg=1)Traceback (most recent call last):  File"<stdin>", line1, in<module>TypeError:pos_only_arg() got an unexpected keyword argument 'arg'

The third functionkwd_only_args only allows keyword arguments as indicatedby a* in the function definition:

>>>kwd_only_arg(3)Traceback (most recent call last):  File"<stdin>", line1, in<module>TypeError:kwd_only_arg() takes 0 positional arguments but 1 was given>>>kwd_only_arg(arg=3)3

And the last uses all three calling conventions in the same functiondefinition:

>>>combined_example(1,2,3)Traceback (most recent call last):  File"<stdin>", line1, in<module>TypeError:combined_example() takes 2 positional arguments but 3 were given>>>combined_example(1,2,kwd_only=3)1 2 3>>>combined_example(1,standard=2,kwd_only=3)1 2 3>>>combined_example(pos_only=1,standard=2,kwd_only=3)Traceback (most recent call last):  File"<stdin>", line1, in<module>TypeError:combined_example() got an unexpected keyword argument 'pos_only'

Recap

The use case will determine which parameters to use in the function definition:

deff(pos1,pos2,/,pos_or_kwd,*,kwd1,kwd2):

As guidance:

• Use positional-only if names do not matter or have no meaning, and there areonly a few arguments which will always be passed in the same order.
• Use keyword-only when names have meaning and the function definition ismore understandable by being explicit with names.

Do Nothing

Always an option — the status quo. While this was considered, theaforementioned benefits are worth the addition to the language.

Decorators

It has been suggested on python-ideas[9] toprovide a decorator written in Python for this feature.

This approach has the benefit of not polluting function definition withadditional syntax. However, we have decided to reject this idea because:

• It introduces an asymmetry with how parameter behavior is declared.
• It makes it difficult for static analyzers and type checkers tosafely identify positional-only parameters. They would need to query the ASTfor the list of decorators and identify the correct one by name or with extraheuristics, while keyword-only parameters are exposeddirectly in the AST. In order for tools to correctly identifypositional-only parameters, they would need to execute the module to accessany metadata the decorator is setting.
• Any error with the declaration will be reported only at runtime.
• It may be more difficult to identify positional-only parameters in longfunction definitions, as it forces the user to count them to know which isthe last one that is impacted by the decorator.
• The/ syntax has already been introduced for C functions. Thisinconsistency will make it more challenging to implement any tools andmodules that deal with this syntax — including but not limited to, theargument clinic, the inspect module and theast module.
• The decorator implementation would likely impose a runtime performance cost,particularly when compared to adding support directly to the interpreter.

Per-Argument Marker

A per-argument marker is another language-intrinsic option. The approach addsa token to each of the parameters to indicate they are positional-only andrequires those parameters to be placed together. Example:

def(.arg1,.arg2,arg3):

Note the dot (i.e.,.) on.arg1 and.arg2. While this approachmay be easier to read, it has been rejected because/ as an explicit markeris congruent with* for keyword-only arguments and is less error-prone.

It should be noted that some libraries already use leading underscore[12] to conventionally indicate parameters as positional-only.

Using “__” as a Per-Argument Marker

Some libraries and applications (likemypy orjinja) use namesprepended with a double underscore (i.e.,__) as a convention to indicatepositional-only parameters. We have rejected the idea of introducing__ asa new syntax because:

• It is a backwards-incompatible change.
• It is not symmetric with how the keyword-only parameters are currentlydeclared.
• Querying the AST for positional-only parameters would require checking thenormal arguments and inspecting their names, whereas keyword-only parametershave a property associated with them (FunctionDef.args.kwonlyargs).
• Every parameter would need to be inspected to know when positional-onlyarguments end.
• The marker is more verbose, forcing marking every positional-only parameter.
• It clashes with other uses of the double underscore prefix like invoking namemangling in classes.

Group Positional-Only Parameters With Parentheses

Tuple parameter unpacking is a Python 2 feature which allows the use of a tupleas a parameter in a function definition. It allows a sequence argument to beunpacked automatically. An example is:

deffxn(a,(b,c),d):pass

Tuple argument unpacking was removed in Python 3 (PEP 3113). There has been aproposition to reuse this syntax to implement positional-only parameters. Wehave rejected this syntax for indicating positional only parameters for severalreasons:

• The syntax is asymmetric with respect to how keyword-only parameters aredeclared.
• Python 2 uses this syntax which could raise confusion regarding the behaviorof this syntax. This would be surprising to users porting Python 2 codebasesthat were using this feature.
• This syntax is very similar to tuple literals. This can raise additionalconfusion because it can be confused with a tuple declaration.

After Separator Proposal

Marking positional-parameters after the/ was another idea considered.However, we were unable to find an approach which would modify the argumentsafter the marker. Otherwise, would force the parameters before the marker tobe positional-only as well. For example:

def(x,y,/,z):

If we define that/ marksz as positional-only, it would not bepossible to specifyx andy as keyword arguments. Finding a way towork around this limitation would add confusion given that at the momentkeyword arguments cannot be followed by positional arguments. Therefore,/would make both the preceding and following parameters positional-only.