The importance of real code

During the development of this PEP many people (supporters and criticsboth) have had a tendency to focus on toy examples on the one hand,and on overly complex examples on the other.

The danger of toy examples is twofold: they are often too abstract tomake anyone go “ooh, that’s compelling”, and they are easily refutedwith “I would never write it that way anyway”.

The danger of overly complex examples is that they provide aconvenient strawman for critics of the proposal to shoot down (“that’sobfuscated”).

Yet there is some use for both extremely simple and extremely complexexamples: they are helpful to clarify the intended semantics.Therefore, there will be some of each below.

However, in order to becompelling, examples should be rooted inreal code, i.e. code that was written without any thought of this PEP,as part of a useful application, however large or small. Tim Petershas been extremely helpful by going over his own personal coderepository and picking examples of code he had written that (in hisview) would have beenclearer if rewritten with (sparing) use ofassignment expressions. His conclusion: the current proposal wouldhave allowed a modest but clear improvement in quite a few bits ofcode.

Another use of real code is to observe indirectly how much valueprogrammers place on compactness. Guido van Rossum searched through aDropbox code base and discovered some evidence that programmers valuewriting fewer lines over shorter lines.

Case in point: Guido found several examples where a programmerrepeated a subexpression, slowing down the program, in order to saveone line of code, e.g. instead of writing:

match=re.match(data)group=match.group(1)ifmatchelseNone

they would write:

group=re.match(data).group(1)ifre.match(data)elseNone

Another example illustrates that programmers sometimes do more work tosave an extra level of indentation:

match1=pattern1.match(data)match2=pattern2.match(data)ifmatch1:result=match1.group(1)elifmatch2:result=match2.group(2)else:result=None

This code tries to matchpattern2 even ifpattern1 has a match(in which case the match onpattern2 is never used). The moreefficient rewrite would have been:

match1=pattern1.match(data)ifmatch1:result=match1.group(1)else:match2=pattern2.match(data)ifmatch2:result=match2.group(2)else:result=None

Exceptional cases

There are a few places where assignment expressions are not allowed,in order to avoid ambiguities or user confusion:

• Unparenthesized assignment expressions are prohibited at the toplevel of an expression statement. Example:

  y:=f(x)# INVALID(y:=f(x))# Valid, though not recommended

  This rule is included to simplify the choice for the user between anassignment statement and an assignment expression – there is nosyntactic position where both are valid.

• Unparenthesized assignment expressions are prohibited at the toplevel of the right hand side of an assignment statement. Example:

  y0=y1:=f(x)# INVALIDy0=(y1:=f(x))# Valid, though discouraged

  Again, this rule is included to avoid two visually similar ways ofsaying the same thing.

• Unparenthesized assignment expressions are prohibited for the valueof a keyword argument in a call. Example:

  foo(x=y:=f(x))# INVALIDfoo(x=(y:=f(x)))# Valid, though probably confusing

  This rule is included to disallow excessively confusing code, andbecause parsing keyword arguments is complex enough already.

• Unparenthesized assignment expressions are prohibited at the toplevel of a function default value. Example:

  deffoo(answer=p:=42):# INVALID...deffoo(answer=(p:=42)):# Valid, though not great style...

  This rule is included to discourage side effects in a position whoseexact semantics are already confusing to many users (cf. the commonstyle recommendation against mutable default values), and also toecho the similar prohibition in calls (the previous bullet).

• Unparenthesized assignment expressions are prohibited as annotationsfor arguments, return values and assignments. Example:

  deffoo(answer:p:=42=5):# INVALID...deffoo(answer:(p:=42)=5):# Valid, but probably never useful...

  The reasoning here is similar to the two previous cases; thisungrouped assortment of symbols and operators composed of: and= is hard to read correctly.

• Unparenthesized assignment expressions are prohibited in lambda functions.Example:

  (lambda:x:=1)# INVALIDlambda:(x:=1)# Valid, but unlikely to be useful(x:=lambda:1)# Validlambdaline:(m:=re.match(pattern,line))andm.group(1)# Valid

  This allowslambda to always bind less tightly than:=; having aname binding at the top level inside a lambda function is unlikely to be ofvalue, as there is no way to make use of it. In cases where the name will beused more than once, the expression is likely to need parenthesizing anyway,so this prohibition will rarely affect code.

• Assignment expressions inside of f-strings require parentheses. Example:

  >>>f'{(x:=10)}'# Valid, uses assignment expression'10'>>>x=10>>>f'{x:=10}'# Valid, passes '=10' to formatter'        10'

  This shows that what looks like an assignment operator in an f-string isnot always an assignment operator. The f-string parser uses: toindicate formatting options. To preserve backwards compatibility,assignment operator usage inside of f-strings must be parenthesized.As noted above, this usage of the assignment operator is not recommended.

Scope of the target

An assignment expression does not introduce a new scope. In mostcases the scope in which the target will be bound is self-explanatory:it is the current scope. If this scope contains anonlocal orglobal declaration for the target, the assignment expressionhonors that. A lambda (being an explicit, if anonymous, functiondefinition) counts as a scope for this purpose.

There is one special case: an assignment expression occurring in alist, set or dict comprehension or in a generator expression (belowcollectively referred to as “comprehensions”) binds the target in thecontaining scope, honoring anonlocal orglobal declarationfor the target in that scope, if one exists. For the purpose of thisrule the containing scope of a nested comprehension is the scope thatcontains the outermost comprehension. A lambda counts as a containingscope.

The motivation for this special case is twofold. First, it allows usto conveniently capture a “witness” for anany() expression, or acounterexample forall(), for example:

ifany((comment:=line).startswith('#')forlineinlines):print("First comment:",comment)else:print("There are no comments")ifall((nonblank:=line).strip()==''forlineinlines):print("All lines are blank")else:print("First non-blank line:",nonblank)

Second, it allows a compact way of updating mutable state from acomprehension, for example:

# Compute partial sums in a list comprehensiontotal=0partial_sums=[total:=total+vforvinvalues]print("Total:",total)

However, an assignment expression target name cannot be the same as afor-target name appearing in any comprehension containing theassignment expression. The latter names are local to thecomprehension in which they appear, so it would be contradictory for acontained use of the same name to refer to the scope containing theoutermost comprehension instead.

For example,[i:=i+1foriinrange(5)] is invalid: thefori part establishes thati is local to the comprehension, but thei:= part insists thati is not local to the comprehension.The same reason makes these examples invalid too:

[[(j:=j)foriinrange(5)]forjinrange(5)]# INVALID[i:=0fori,jinstuff]# INVALID[i+1foriin(i:=stuff)]# INVALID

While it’s technically possible to assign consistent semantics to these cases,it’s difficult to determine whether those semantics actually makesense in theabsence of real use cases. Accordingly, the reference implementation[1] will ensurethat such cases raiseSyntaxError, rather than executing with implementationdefined behaviour.

This restriction applies even if the assignment expression is never executed:

[Falseand(i:=0)fori,jinstuff]# INVALID[ifori,jinstuffifTrueor(j:=1)]# INVALID

For the comprehension body (the part before the first “for” keyword) and thefilter expression (the part after “if” and before any nested “for”), thisrestriction applies solely to target names that are also used as iterationvariables in the comprehension. Lambda expressions appearing in thesepositions introduce a new explicit function scope, and hence may use assignmentexpressions with no additional restrictions.

Due to design constraints in the reference implementation (the symbol tableanalyser cannot easily detect when names are re-used between the leftmostcomprehension iterable expression and the rest of the comprehension), namedexpressions are disallowed entirely as part of comprehension iterableexpressions (the part after each “in”, and before any subsequent “if” or“for” keyword):

[i+1foriin(j:=stuff)]# INVALID[i+1foriinrange(2)forjin(k:=stuff)]# INVALID[i+1foriin[jforjin(k:=stuff)]]# INVALID[i+1foriin(lambda:(j:=stuff))()]# INVALID

A further exception applies when an assignment expression occurs in acomprehension whose containing scope is a class scope. If the rulesabove were to result in the target being assigned in that class’sscope, the assignment expression is expressly invalid. This case also raisesSyntaxError:

classExample:[(j:=i)foriinrange(5)]# INVALID

(The reason for the latter exception is the implicit function scope createdfor comprehensions – there is currently no runtime mechanism for afunction to refer to a variable in the containing class scope, and wedo not want to add such a mechanism. If this issue ever gets resolvedthis special case may be removed from the specification of assignmentexpressions. Note that the problem already exists forusing avariable defined in the class scope from a comprehension.)

See Appendix B for some examples of how the rules for targets incomprehensions translate to equivalent code.

Relative precedence of:=

The:= operator groups more tightly than a comma in all syntacticpositions where it is legal, but less tightly than all other operators,includingor,and,not, and conditional expressions(AifCelseB). As follows from section“Exceptional cases” above, it is never allowed at the same level as=. In case a different grouping is desired, parentheses should beused.

The:= operator may be used directly in a positional function callargument; however it is invalid directly in a keyword argument.

Some examples to clarify what’s technically valid or invalid:

# INVALIDx:=0# Valid alternative(x:=0)# INVALIDx=y:=0# Valid alternativex=(y:=0)# Validlen(lines:=f.readlines())# Validfoo(x:=3,cat='vector')# INVALIDfoo(cat=category:='vector')# Valid alternativefoo(cat=(category:='vector'))

Most of the “valid” examples above are not recommended, since humanreaders of Python source code who are quickly glancing at some codemay miss the distinction. But simple cases are not objectionable:

# Validifany(len(longline:=line)>=100forlineinlines):print("Extremely long line:",longline)

This PEP recommends always putting spaces around:=, similar toPEP 8’s recommendation for= when used for assignment, whereas thelatter disallows spaces around= used for keyword arguments.)

Change to evaluation order

In order to have precisely defined semantics, the proposal requiresevaluation order to be well-defined. This is technically not a newrequirement, as function calls may already have side effects. Pythonalready has a rule that subexpressions are generally evaluated fromleft to right. However, assignment expressions make these sideeffects more visible, and we propose a single change to the currentevaluation order:

• In a dict comprehension{X:Yfor...},Y is currentlyevaluated beforeX. We propose to change this so thatX isevaluated beforeY. (In a dict display like{X:Y} this isalready the case, and also indict((X,Y)for...) which shouldclearly be equivalent to the dict comprehension.)

Differences between assignment expressions and assignment statements

Most importantly, since:= is an expression, it can be used in contextswhere statements are illegal, including lambda functions and comprehensions.

Conversely, assignment expressions don’t support the advanced featuresfound in assignment statements:

• Multiple targets are not directly supported:

  x=y=z=0# Equivalent: (z := (y := (x := 0)))

• Single assignment targets other than a singleNAME arenot supported:

  # No equivalenta[i]=xself.rest=[]

• Priority around commas is different:

  x=1,2# Sets x to (1, 2)(x:=1,2)# Sets x to 1

• Iterable packing and unpacking (both regular or extended forms) arenot supported:

  # Equivalent needs extra parenthesesloc=x,y# Use (loc := (x, y))info=name,phone,*rest# Use (info := (name, phone, *rest))# No equivalentpx,py,pz=positionname,phone,email,*other_info=contact

• Inline type annotations are not supported:

  # Closest equivalent is "p: Optional[int]" as a separate declarationp:Optional[int]=None

• Augmented assignment is not supported:

  total+=tax# Equivalent: (total := total + tax)

Examples from the Python standard library

Simplifying list comprehensions

A list comprehension can map and filter efficiently by capturingthe condition:

results=[(x,y,x/y)forxininput_dataif(y:=f(x))>0]

Similarly, a subexpression can be reused within the main expression, bygiving it a name on first use:

stuff=[[y:=f(x),x/y]forxinrange(5)]

Note that in both cases the variabley is bound in the containingscope (i.e. at the same level asresults orstuff).

Capturing condition values

Assignment expressions can be used to good effect in the header ofanif orwhile statement:

# Loop-and-a-halfwhile(command:=input("> "))!="quit":print("You entered:",command)# Capturing regular expression match objects# See, for instance, Lib/pydoc.py, which uses a multiline spelling# of this effectifmatch:=re.search(pat,text):print("Found:",match.group(0))# The same syntax chains nicely into 'elif' statements, unlike the# equivalent using assignment statements.elifmatch:=re.search(otherpat,text):print("Alternate found:",match.group(0))elifmatch:=re.search(third,text):print("Fallback found:",match.group(0))# Reading socket data until an empty string is returnedwhiledata:=sock.recv(8192):print("Received data:",data)

Particularly with thewhile loop, this can remove the need to have aninfinite loop, an assignment, and a condition. It also creates a smoothparallel between a loop which simply uses a function call as its condition,and one which uses that as its condition but also uses the actual value.

Fork

An example from the low-level UNIX world:

ifpid:=os.fork():# Parent codeelse:# Child code

Changing the scope rules for comprehensions

A previous version of this PEP proposed subtle changes to the scoperules for comprehensions, to make them more usable in class scope andto unify the scope of the “outermost iterable” and the rest of thecomprehension. However, this part of the proposal would have causedbackwards incompatibilities, and has been withdrawn so the PEP canfocus on assignment expressions.

Alternative spellings

Broadly the same semantics as the current proposal, but spelled differently.

1. EXPRasNAME:

   stuff=[[f(x)asy,x/y]forxinrange(5)]

   SinceEXPRasNAME already has meaning inimport,except andwith statements (with different semantics), thiswould create unnecessary confusion or require special-casing(e.g. to forbid assignment within the headers of these statements).

   (Note thatwithEXPRasVAR doesnot simply assign the valueofEXPR toVAR – it callsEXPR.__enter__() and assignsthe result ofthat toVAR.)

   Additional reasons to prefer:= over this spelling include:

   • Iniff(x)asy the assignment target doesn’t jump out at you– it just reads likeiffxblahblah and it is too similarvisually toiff(x)andy.
   • In all other situations where anas clause is allowed, evenreaders with intermediary skills are led to anticipate thatclause (however optional) by the keyword that starts the line,and the grammar ties that keyword closely to the as clause:
     • importfooasbar
     • exceptExcasvar
     • withctxmgr()asvar

     To the contrary, the assignment expression does not belong to theif orwhile that starts the line, and we intentionallyallow assignment expressions in other contexts as well.

   • The parallel cadence between
     • NAME=EXPR
     • ifNAME:=EXPR

     reinforces the visual recognition of assignment expressions.

2. EXPR->NAME:

   stuff=[[f(x)->y,x/y]forxinrange(5)]

   This syntax is inspired by languages such as R and Haskell, and someprogrammable calculators. (Note that a left-facing arrowy<-f(x) isnot possible in Python, as it would be interpreted as less-than and unaryminus.) This syntax has a slight advantage over ‘as’ in that it does notconflict withwith,except andimport, but otherwise isequivalent. But it is entirely unrelated to Python’s other use of-> (function return type annotations), and compared to:=(which dates back to Algol-58) it has a much weaker tradition.

3. Adorning statement-local names with a leading dot:

   stuff=[[(f(x)as.y),x/.y]forxinrange(5)]# with "as"stuff=[[(.y:=f(x)),x/.y]forxinrange(5)]# with ":="

   This has the advantage that leaked usage can be readily detected, removingsome forms of syntactic ambiguity. However, this would be the only placein Python where a variable’s scope is encoded into its name, makingrefactoring harder.

4. Adding awhere: to any statement to create local name bindings:

   value=x**2+2*xwhere:x=spam(1,4,7,q)

   Execution order is inverted (the indented body is performed first, followedby the “header”). This requires a new keyword, unless an existing keywordis repurposed (most likelywith:). SeePEP 3150 for prior discussionon this subject (with the proposed keyword beinggiven:).

5. TARGETfromEXPR:

   stuff=[[yfromf(x),x/y]forxinrange(5)]

   This syntax has fewer conflicts thanas does (conflicting only with theraiseExcfromExc notation), but is otherwise comparable to it. Insteadof parallelingwithexprastarget: (which can be useful but can also beconfusing), this has no parallels, but is evocative.

Special-casing conditional statements

One of the most popular use-cases isif andwhile statements. Insteadof a more general solution, this proposal enhances the syntax of these twostatements to add a means of capturing the compared value:

ifre.search(pat,text)asmatch:print("Found:",match.group(0))

This works beautifully if and ONLY if the desired condition is based on thetruthiness of the captured value. It is thus effective for specificuse-cases (regex matches, socket reads that return'' when done), andcompletely useless in more complicated cases (e.g. where the condition isf(x)<0 and you want to capture the value off(x)). It also hasno benefit to list comprehensions.

Advantages: No syntactic ambiguities. Disadvantages: Answers only a fractionof possible use-cases, even inif/while statements.

Special-casing comprehensions

Another common use-case is comprehensions (list/set/dict, and genexps). Asabove, proposals have been made for comprehension-specific solutions.

1. where,let, orgiven:

   stuff=[(y,x/y)wherey=f(x)forxinrange(5)]stuff=[(y,x/y)lety=f(x)forxinrange(5)]stuff=[(y,x/y)giveny=f(x)forxinrange(5)]

   This brings the subexpression to a location in between the ‘for’ loop andthe expression. It introduces an additional language keyword, which createsconflicts. Of the three,where reads the most cleanly, but also has thegreatest potential for conflict (e.g. SQLAlchemy and numpy havewheremethods, as doestkinter.dnd.Icon in the standard library).

2. withNAME=EXPR:

   stuff=[(y,x/y)withy=f(x)forxinrange(5)]

   As above, but reusing thewith keyword. Doesn’t read too badly, and needsno additional language keyword. Is restricted to comprehensions, though,and cannot as easily be transformed into “longhand” for-loop syntax. Hasthe C problem that an equals sign in an expression can now create a namebinding, rather than performing a comparison. Would raise the question ofwhy “with NAME = EXPR:” cannot be used as a statement on its own.

3. withEXPRasNAME:

   stuff=[(y,x/y)withf(x)asyforxinrange(5)]

   As per option 2, but usingas rather than an equals sign. Alignssyntactically with other uses ofas for name binding, but a simpletransformation to for-loop longhand would create drastically differentsemantics; the meaning ofwith inside a comprehension would becompletely different from the meaning as a stand-alone statement, whileretaining identical syntax.

Regardless of the spelling chosen, this introduces a stark difference betweencomprehensions and the equivalent unrolled long-hand form of the loop. It isno longer possible to unwrap the loop into statement form without reworkingany name bindings. The only keyword that can be repurposed to this task iswith, thus giving it sneakily different semantics in a comprehension thanin a statement; alternatively, a new keyword is needed, with all the coststherein.

Lowering operator precedence

There are two logical precedences for the:= operator. Either it shouldbind as loosely as possible, as does statement-assignment; or it should bindmore tightly than comparison operators. Placing its precedence between thecomparison and arithmetic operators (to be precise: just lower than bitwiseOR) allows most uses insidewhile andif conditions to be spelledwithout parentheses, as it is most likely that you wish to capture the valueof something, then perform a comparison on it:

pos=-1whilepos:=buffer.find(search_term,pos+1)>=0:...

Once find() returns -1, the loop terminates. If:= binds as loosely as= does, this would capture the result of the comparison (generally eitherTrue orFalse), which is less useful.

While this behaviour would be convenient in many situations, it is also harderto explain than “the := operator behaves just like the assignment statement”,and as such, the precedence for:= has been made as close as possible tothat of= (with the exception that it binds tighter than comma).

Allowing commas to the right

Some critics have claimed that the assignment expressions should allowunparenthesized tuples on the right, so that these two would be equivalent:

(point:=(x,y))(point:=x,y)

(With the current version of the proposal, the latter would beequivalent to((point:=x),y).)

However, adopting this stance would logically lead to the conclusionthat when used in a function call, assignment expressions also bindless tight than comma, so we’d have the following confusing equivalence:

foo(x:=1,y)foo(x:=(1,y))

The less confusing option is to make:= bind more tightly than comma.

Always requiring parentheses

It’s been proposed to just always require parentheses around anassignment expression. This would resolve many ambiguities, andindeed parentheses will frequently be needed to extract the desiredsubexpression. But in the following cases the extra parentheses feelredundant:

# Top level in ififmatch:=pattern.match(line):returnmatch.group(1)# Short calllen(lines:=f.readlines())

Why not just turn existing assignment into an expression?

C and its derivatives define the= operator as an expression, rather thana statement as is Python’s way. This allows assignments in more contexts,including contexts where comparisons are more common. The syntactic similaritybetweenif(x==y) andif(x=y) belies their drastically differentsemantics. Thus this proposal uses:= to clarify the distinction.

With assignment expressions, why bother with assignment statements?

The two forms have different flexibilities. The:= operator can be usedinside a larger expression; the= statement can be augmented to+= andits friends, can be chained, and can assign to attributes and subscripts.

Why not use a sublocal scope and prevent namespace pollution?

Previous revisions of this proposal involved sublocal scope (restricted to asingle statement), preventing name leakage and namespace pollution. While adefinite advantage in a number of situations, this increases complexity inmany others, and the costs are not justified by the benefits. In the interestsof language simplicity, the name bindings created here are exactly equivalentto any other name bindings, including that usage at class or module scope willcreate externally-visible names. This is no different fromfor loops orother constructs, and can be solved the same way:del the name once it isno longer needed, or prefix it with an underscore.

(The author wishes to thank Guido van Rossum and Christoph Groth for theirsuggestions to move the proposal in this direction.[2])

A numeric example

I have another example that quite impressed me at the time.

Where all variables are positive integers, and a is at least as largeas the n’th root of x, this algorithm returns the floor of the n’throot of x (and roughly doubling the number of accurate bits periteration):

whilea>(d:=x//a**(n-1)):a=((n-1)*a+d)//nreturna

It’s not obvious why that works, but is no more obvious in the “loopand a half” form. It’s hard to prove correctness without building onthe right insight (the “arithmetic mean - geometric mean inequality”),and knowing some non-trivial things about how nested floor functionsbehave. That is, the challenges are in the math, not really in thecoding.

If you do know all that, then the assignment-expression form is easilyread as “while the current guess is too large, get a smaller guess”,where the “too large?” test and the new guess share an expensivesub-expression.

To my eyes, the original form is harder to understand:

whileTrue:d=x//a**(n-1)ifa<=d:breaka=((n-1)*a+d)//nreturna