This document is meant to describe the decorator syntax and theprocess that resulted in the decisions that were made. It does notattempt to cover the huge number of potential alternative syntaxes,nor is it an attempt to exhaustively list all the positives andnegatives of each form.
The current method for transforming functions and methods (for instance,declaring them as a class or static method) is awkward and can lead tocode that is difficult to understand. Ideally, these transformationsshould be made at the same point in the code where the declarationitself is made. This PEP introduces new syntax for transformations of afunction or method declaration.
The current method of applying a transformation to a function or methodplaces the actual transformation after the function body. For largefunctions this separates a key component of the function’s behavior fromthe definition of the rest of the function’s external interface. Forexample:
deffoo(self):performmethodoperationfoo=classmethod(foo)
This becomes less readable with longer methods. It also seems lessthan pythonic to name the function three times for what is conceptuallya single declaration. A solution to this problem is to move thetransformation of the method closer to the method’s own declaration.The intent of the new syntax is to replace
deffoo(cls):passfoo=synchronized(lock)(foo)foo=classmethod(foo)
with an alternative that places the decoration in the function’sdeclaration:
@classmethod@synchronized(lock)deffoo(cls):pass
Modifying classes in this fashion is also possible, though the benefitsare not as immediately apparent. Almost certainly, anything which couldbe done with class decorators could be done using metaclasses, butusing metaclasses is sufficiently obscure that there is some attractionto having an easier way to make simple modifications to classes. ForPython 2.4, only function/method decorators are being added.
PEP 3129 proposes to add class decorators as of Python 2.6.
Two decorators (classmethod() andstaticmethod()) have beenavailable in Python since version 2.2. It’s been assumed sinceapproximately that time that some syntactic support for them wouldeventually be added to the language. Given this assumption, one mightwonder why it’s been so difficult to arrive at a consensus. Discussionshave raged off-and-on at times in both comp.lang.python and thepython-dev mailing list about how best to implement function decorators.There is no one clear reason why this should be so, but a few problemsseem to be most divisive.
There is general agreement that syntactic support is desirable tothe current state of affairs. Guido mentionedsyntactic supportfor decorators in his DevDay keynote presentation at the10thPython Conference, thoughhe later said it was only one ofseveral extensions he proposed there “semi-jokingly”.Michael Hudsonraised the topic onpython-dev shortly after the conference,attributing the initial bracketed syntax to an earlier proposal oncomp.lang.python byGareth McCaughan.
Class decorations seem like an obvious next step because classdefinition and function definition are syntactically similar,however Guido remains unconvinced, and class decorators will almostcertainly not be in Python 2.4.
The discussion continued on and off on python-dev from February2002 through July 2004. Hundreds and hundreds of posts were made,with people proposing many possible syntax variations. Guido tooka list of proposals toEuroPython 2004, where a discussion tookplace. Subsequent to this, he decided that we’d have theJava-style@decorator syntax, and this appeared for the first time in 2.4a2.Barry Warsaw named this the ‘pie-decorator’ syntax, in honor of thePie-thon Parrot shootout which occurred around the same time asthe decorator syntax, and because the @ looks a little like a pie.Guidooutlined his case on Python-dev, includingthis pieceon some of the (many) rejected forms.
There’s been a number of complaints about the choice of the name‘decorator’ for this feature. The major one is that the name is notconsistent with its use in theGoF book. The name ‘decorator’probably owes more to its use in the compiler area – a syntax tree iswalked and annotated. It’s quite possible that a better name may turnup.
The new syntax should
classmethod() andstaticmethod(). Thisrequirement also means that a decorator syntax must support passingarguments to the wrapper constructorAndrew Kuchling has links to a bunch of the discussions aboutmotivations and use casesin his blog. Particularly notable isJimHuginin’s list of use cases.
The current syntax for function decorators as implemented in Python2.4a2 is:
@dec2@dec1deffunc(arg1,arg2,...):pass
This is equivalent to:
deffunc(arg1,arg2,...):passfunc=dec2(dec1(func))
without the intermediate assignment to the variablefunc. Thedecorators are near the function declaration. The @ sign makes it clearthat something new is going on here.
The rationale for theorder of application (bottom to top) is that itmatches the usual order for function-application. In mathematics,composition of functions (g o f)(x) translates to g(f(x)). In Python,@g@fdeffoo() translates tofoo=g(f(foo).
The decorator statement is limited in what it can accept – arbitraryexpressions will not work. Guido preferred this because of agutfeeling.
The current syntax also allows decorator declarations to call afunction that returns a decorator:
@decomaker(argA,argB,...)deffunc(arg1,arg2,...):pass
This is equivalent to:
func=decomaker(argA,argB,...)(func)
The rationale for having a function that returns a decorator is thatthe part after the @ sign can be considered to be an expression(though syntactically restricted to just a function), and whateverthat expression returns is called. Seedeclaration arguments.
There have beena large number of different syntaxes proposed –rather than attempting to work through these individual syntaxes, it’sworthwhile to break the syntax discussion down into a number of areas.Attempting to discusseach possible syntax individually would be anact of madness, and produce a completely unwieldy PEP.
The first syntax point is the location of the decorators. For thefollowing examples, we use the @syntax used in 2.4a2.
Decorators before the def statement are the first alternative, and thesyntax used in 2.4a2:
@classmethoddeffoo(arg1,arg2):pass@accepts(int,int)@returns(float)defbar(low,high):pass
There have been a number of objections raised to this location – theprimary one is that it’s the first real Python case where a line of codehas an effect on a following line. The syntax available in 2.4a3requires one decorator per line (in a2, multiple decorators could bespecified on the same line), and the final decision for 2.4 final stayedone decorator per line.
People also complained that the syntax quickly got unwieldy whenmultiple decorators were used. The point was made, though, that thechances of a large number of decorators being used on a single functionwere small and thus this was not a large worry.
Some of the advantages of this form are that the decorators live outsidethe method body – they are obviously executed at the time the functionis defined.
Another advantage is that a prefix to the function definition fitsthe idea of knowing about a change to the semantics of the code beforethe code itself, thus you know how to interpret the code’s semanticsproperly without having to go back and change your initial perceptionsif the syntax did not come before the function definition.
Guido decidedhe preferred having the decorators on the line beforethe ‘def’, because it was felt that a long argument list would mean thatthe decorators would be ‘hidden’
The second form is the decorators between the def and the function name,or the function name and the argument list:
def@classmethodfoo(arg1,arg2):passdef@accepts(int,int),@returns(float)bar(low,high):passdeffoo@classmethod(arg1,arg2):passdefbar@accepts(int,int),@returns(float)(low,high):pass
There are a couple of objections to this form. The first is that itbreaks easily ‘greppability’ of the source – you can no longer searchfor ‘def foo(’ and find the definition of the function. The second,more serious, objection is that in the case of multiple decorators, thesyntax would be extremely unwieldy.
The next form, which has had a number of strong proponents, is to havethe decorators between the argument list and the trailing: in the‘def’ line:
deffoo(arg1,arg2)@classmethod:passdefbar(low,high)@accepts(int,int),@returns(float):pass
Guidosummarized the arguments against this form (many of which alsoapply to the previous form) as:
The next form is that the decorator syntax goes inside the method body atthe start, in the same place that docstrings currently live:
deffoo(arg1,arg2):@classmethodpassdefbar(low,high):@accepts(int,int)@returns(float)pass
The primary objection to this form is that it requires “peeking inside”the method body to determine the decorators. In addition, even thoughthe code is inside the method body, it is not executed when the methodis run. Guido felt that docstrings were not a good counter-example, andthat it was quite possible that a ‘docstring’ decorator could help movethe docstring to outside the function body.
The final form is a new block that encloses the method’s code. For thisexample, we’ll use a ‘decorate’ keyword, as it makes no sense with the@syntax.
decorate:classmethoddeffoo(arg1,arg2):passdecorate:accepts(int,int)returns(float)defbar(low,high):pass
This form would result in inconsistent indentation for decorated andundecorated methods. In addition, a decorated method’s body would startthree indent levels in.
@decorator:@classmethoddeffoo(arg1,arg2):pass@accepts(int,int)@returns(float)defbar(low,high):pass
The major objections against this syntax are that the @ symbol isnot currently used in Python (and is used in both IPython and Leo),and that the @ symbol is not meaningful. Another objection is thatthis “wastes” a currently unused character (from a limited set) onsomething that is not perceived as a major use.
|decorator:|classmethoddeffoo(arg1,arg2):pass|accepts(int,int)|returns(float)defbar(low,high):pass
This is a variant on the @decorator syntax – it has the advantagethat it does not break IPython and Leo. Its major disadvantagecompared to the @syntax is that the | symbol looks like both a capitalI and a lowercase l.
[classmethod]deffoo(arg1,arg2):pass[accepts(int,int),returns(float)]defbar(low,high):pass
The major objection to the list syntax is that it’s currentlymeaningful (when used in the form before the method). It’s alsolacking any indication that the expression is a decorator.
<...>,[[...]], …):<classmethod>deffoo(arg1,arg2):pass<accepts(int,int),returns(float)>defbar(low,high):pass
None of these alternatives gained much traction. The alternativeswhich involve square brackets only serve to make it obvious that thedecorator construct is not a list. They do nothing to make parsing anyeasier. The ‘<…>’ alternative presents parsing problems because ‘<’and ‘>’ already parse as un-paired. They present a further parsingambiguity because a right angle bracket might be a greater than symbolinstead of a closer for the decorators.
decorate()Thedecorate() proposal was that no new syntax be implemented– instead a magic function that used introspection to manipulatethe following function. Both Jp Calderone and Philip Eby producedimplementations of functions that did this. Guido was pretty firmlyagainst this – with no new syntax, the magicness of a function likethis is extremely high:
Using functions with “action-at-a-distance” through sys.settracebackmay be okay for an obscure feature that can’t be had any otherway yet doesn’t merit changes to the language, but that’s notthe situation for decorators. The widely held view here is thatdecorators need to be added as a syntactic feature to avoid theproblems with the postfix notation used in 2.2 and 2.3. Decoratorsare slated to be an important new language feature and theirdesign needs to be forward-looking, not constrained by what can beimplemented in 2.3.
This idea was the consensus alternate from comp.lang.python (moreon this inCommunity Consensus below.) Robert Brewer wrote up adetailedJ2 proposal document outlining the arguments in favor ofthis form. The initial issues with this form are:
from__future__importdecorators statement.using emergedas the consensus choice, and is used in the proposal andimplementation.A few days later, Guidorejected the proposal on two main grounds,firstly:
… the syntactic form of an indented block stronglysuggests that its contents should be a sequence of statements, butin fact it is not – only expressions are allowed, and there is animplicit “collecting” of these expressions going on until they canbe applied to the subsequent function definition. …
and secondly:
… the keyword starting the line that heads a blockdraws a lot of attention to it. This is true for “if”, “while”,“for”, “try”, “def” and “class”. But the “using” keyword (or anyother keyword in its place) doesn’t deserve that attention; theemphasis should be on the decorator or decorators inside the suite,since those are the important modifiers to the function definitionthat follows. …
Readers are invited to readthe full response.
There are plenty of other variants and proposals onthe wiki page.
There is some history in Java using @ initially as a marker inJavadoccomments and later in Java 1.5 forannotations, which are similarto Python decorators. The fact that @ was previously unused as a tokenin Python also means it’s clear there is no possibility of such codebeing parsed by an earlier version of Python, leading to possibly subtlesemantic bugs. It also means that ambiguity of what is a decoratorand what isn’t is removed. That said, @ is still a fairly arbitrarychoice. Some have suggested using | instead.
For syntax options which use a list-like syntax (no matter where itappears) to specify the decorators a few alternatives were proposed:[|...|],*[...]*, and<...>.
Guido asked for a volunteer to implement his preferred syntax, and MarkRussell stepped up and posted apatch to SF. This new syntax wasavailable in 2.4a2.
@dec2@dec1deffunc(arg1,arg2,...):pass
This is equivalent to:
deffunc(arg1,arg2,...):passfunc=dec2(dec1(func))
though without the intermediate creation of a variable namedfunc.
The version implemented in 2.4a2 allowed multiple@decorator clauseson a single line. In 2.4a3, this was tightened up to only allowing onedecorator per line.
Aprevious patch from Michael Hudson which implements thelist-after-def syntax is also still kicking around.
After 2.4a2 was released, in response to community reaction, Guidostated that he’d re-examine a community proposal, if the communitycould come up with a community consensus, a decent proposal, and animplementation. After an amazing number of posts, collecting a vastnumber of alternatives in thePython wiki, a community consensusemerged (below). Guidosubsequently rejected this alternate form,but added:
In Python 2.4a3 (to be released this Thursday), everything remainsas currently in CVS. For 2.4b1, I will consider a change of @ tosome other single character, even though I think that @ has theadvantage of being the same character used by a similar featurein Java. It’s been argued that it’s not quite the same, since @in Java is used for attributes that don’t change semantics. ButPython’s dynamic nature makes that its syntactic elements never meanquite the same thing as similar constructs in other languages, andthere is definitely significant overlap. Regarding the impact on3rd party tools: IPython’s author doesn’t think there’s going to bemuch impact; Leo’s author has said that Leo will survive (althoughit will cause him and his users some transitional pain). I actuallyexpect that picking a character that’s already used elsewhere inPython’s syntax might be harder for external tools to adapt to,since parsing will have to be more subtle in that case. But I’mfrankly undecided, so there’s some wiggle room here. I don’t wantto consider further syntactic alternatives at this point: the buckhas to stop at some point, everyone has had their say, and the showmust go on.
This section documents the rejected J2 syntax, and is included forhistorical completeness.
The consensus that emerged on comp.lang.python was the proposed J2syntax (the “J2” was how it was referenced on the PythonDecorators wikipage): the new keywordusing prefixing a block of decorators beforethedef statement. For example:
using:classmethodsynchronized(lock)deffunc(cls):pass
The main arguments for this syntax fall under the “readability counts”doctrine. In brief, they are:
using keyword andblock transforms the single-blockdef statement into amultiple-block compound construct, akin to try/finally and others.Robert Brewer wrote adetailed proposal for this form, and MichaelSparks produceda patch.
As noted previously, Guido rejected this form, outlining his problemswith it ina message to python-dev and comp.lang.python.
Much of the discussion oncomp.lang.python and thepython-devmailing list focuses on the use of decorators as a cleaner way to usethestaticmethod() andclassmethod() builtins. This capabilityis much more powerful than that. This section presents some examples ofuse.
defonexit(f):importatexitatexit.register(f)returnf@onexitdeffunc():...
Note that this example is probably not suitable for real usage, butis for example purposes only.
python-dev.)defsingleton(cls):instances={}defgetinstance():ifclsnotininstances:instances[cls]=cls()returninstances[cls]returngetinstance@singletonclassMyClass:...
python-dev.)defattrs(**kwds):defdecorate(f):forkinkwds:setattr(f,k,kwds[k])returnfreturndecorate@attrs(versionadded="2.2",author="Guido van Rossum")defmymethod(f):...
defaccepts(*types):defcheck_accepts(f):assertlen(types)==f.func_code.co_argcountdefnew_f(*args,**kwds):for(a,t)inzip(args,types):assertisinstance(a,t), \"arg%r does not match%s"%(a,t)returnf(*args,**kwds)new_f.func_name=f.func_namereturnnew_freturncheck_acceptsdefreturns(rtype):defcheck_returns(f):defnew_f(*args,**kwds):result=f(*args,**kwds)assertisinstance(result,rtype), \"return value%r does not match%s"%(result,rtype)returnresultnew_f.func_name=f.func_namereturnnew_freturncheck_returns@accepts(int,(int,float))@returns((int,float))deffunc(arg1,arg2):returnarg1*arg2
python-dev based onexperience withPyProtocols.defprovides(*interfaces):""" An actual, working, implementation of provides for the current implementation of PyProtocols. Not particularly important for the PEP text. """defprovides(typ):declareImplementation(typ,instancesProvide=interfaces)returntypreturnprovidesclassIBar(Interface):"""Declare something about IBar here"""@provides(IBar)classFoo(object):"""Implement something here..."""
Of course, all these examples are possible today, though withoutsyntactic support.
PEP318--postingdraft) on their behalf inpython-dev. It’s exceedingly unlikely that class decoratorswill be in Python 2.4.PEP 3129 proposes to add class decorators as of Python 2.6.
@ character will be re-examined beforePython 2.4b1.In the end, the@ character was kept.
This document has been placed in the public domain.
Source:https://github.com/python/peps/blob/main/peps/pep-0318.rst
Last modified:2025-02-01 08:59:27 GMT