In most languages that support nested scopes, code can refer to orrebind (assign to) any name in the nearest enclosing scope.Currently, Python code can refer to a name in any enclosing scope,but it can only rebind names in two scopes: the local scope (bysimple assignment) or the module-global scope (using aglobaldeclaration).
This limitation has been raised many times on the Python-Dev mailinglist and elsewhere, and has led to extended discussion and manyproposals for ways to remove this limitation. This PEP summarizesthe various alternatives that have been suggested, together withadvantages and disadvantages that have been mentioned for each.
Before version 2.1, Python’s treatment of scopes resembled that ofstandard C: within a file there were only two levels of scope, globaland local. In C, this is a natural consequence of the fact thatfunction definitions cannot be nested. But in Python, thoughfunctions are usually defined at the top level, a function definitioncan be executed anywhere. This gave Python the syntactic appearanceof nested scoping without the semantics, and yielded inconsistenciesthat were surprising to some programmers – for example, a recursivefunction that worked at the top level would cease to work when movedinside another function, because the recursive function’s own namewould no longer be visible in its body’s scope. This violates theintuition that a function should behave consistently when placed indifferent contexts. Here’s an example:
defenclosing_function():deffactorial(n):ifn<2:return1returnn*factorial(n-1)# fails with NameErrorprintfactorial(5)
Python 2.1 moved closer to static nested scoping by making visiblethe names bound in all enclosing scopes (seePEP 227). This changemakes the above code example work as expected. However, because anyassignment to a name implicitly declares that name to be local, it isimpossible to rebind a name in an outer scope (except when aglobal declaration forces the name to be global). Thus, thefollowing code, intended to display a number that can be incrementedand decremented by clicking buttons, doesn’t work as someone familiarwith lexical scoping might expect:
defmake_scoreboard(frame,score=0):label=Label(frame)label.pack()foriin[-10,-1,1,10]:defincrement(step=i):score=score+step# fails with UnboundLocalErrorlabel['text']=scorebutton=Button(frame,text='%+d'%i,command=increment)button.pack()returnlabel
Python syntax doesn’t provide a way to indicate that the namescore mentioned inincrement refers to the variablescorebound inmake_scoreboard, not a local variable inincrement.Users and developers of Python have expressed an interest in removingthis limitation so that Python can have the full flexibility of theAlgol-style scoping model that is now standard in many programminglanguages, including JavaScript, Perl, Ruby, Scheme, Smalltalk,C with GNU extensions, and C# 2.0.
It has been argued that such a feature isn’t necessary, becausea rebindable outer variable can be simulated by wrapping it in amutable object:
classNamespace:passdefmake_scoreboard(frame,score=0):ns=Namespace()ns.score=0label=Label(frame)label.pack()foriin[-10,-1,1,10]:defincrement(step=i):ns.score=ns.score+steplabel['text']=ns.scorebutton=Button(frame,text='%+d'%i,command=increment)button.pack()returnlabel
However, this workaround only highlights the shortcomings of existingscopes: the purpose of a function is to encapsulate code in its ownnamespace, so it seems unfortunate that the programmer should have tocreate additional namespaces to make up for missing functionality inthe existing local scopes, and then have to decide whether each nameshould reside in the real scope or the simulated scope.
Another common objection is that the desired functionality can bewritten as a class instead, albeit somewhat more verbosely. Onerebuttal to this objection is that the existence of a differentimplementation style is not a reason to leave a supported programmingconstruct (nested scopes) functionally incomplete. Python issometimes called a “multi-paradigm language” because it derives somuch strength, practical flexibility, and pedagogical power from itssupport and graceful integration of multiple programming paradigms.
A proposal for scoping syntax appeared on Python-Dev as far back as1994[1], long beforePEP 227’s support for nested scopes wasadopted. At the time, Guido’s response was:
This is dangerously close to introducing CSNS [classic staticnested scopes].If you were to do so, your proposed semanticsof scoped seem alright. I still think there is not enough needfor CSNS to warrant this kind of construct …
AfterPEP 227, the “outer name rebinding discussion” has reappearedon Python-Dev enough times that it has become a familiar event,having recurred in its present form since at least 2003[2].Although none of the language changes proposed in these discussionshave yet been adopted, Guido has acknowledged that a language changeis worth considering[12].
To provide some background, this section describes how some otherlanguages handle nested scopes and rebinding.
These languages use variable declarations to indicate scope. InJavaScript, a lexically scoped variable is declared with thevarkeyword; undeclared variable names are assumed to be global. InPerl, a lexically scoped variable is declared with themykeyword; undeclared variable names are assumed to be global. InScheme, all variables must be declared (withdefine orlet,or as formal parameters). In Smalltalk, any block can begin bydeclaring a list of local variable names between vertical bars.C and C# require type declarations for all variables. For all thesecases, the variable belongs to the scope containing the declaration.
Ruby is an instructive example because it appears to be the onlyother currently popular language that, like Python, tries to supportstatically nested scopes without requiring variable declarations, andthus has to come up with an unusual solution. Functions in Ruby cancontain other function definitions, and they can also contain codeblocks enclosed in curly braces. Blocks have access to outervariables, but nested functions do not. Within a block, anassignment to a name implies a declaration of a local variable onlyif it would not shadow a name already bound in an outer scope;otherwise assignment is interpreted as rebinding of the outer name.Ruby’s scoping syntax and rules have also been debated at greatlength, and changes seem likely in Ruby 2.0[28].
There have been many different proposals on Python-Dev for ways torebind names in outer scopes. They all fall into two categories:new syntax in the scope where the name is bound, or new syntax inthe scope where the name is used.
The proposals in this category all suggest a new kind of declarationstatement similar to JavaScript’svar. A few possible keywordshave been proposed for this purpose:
In all these proposals, a declaration such asvarx in aparticular scope S would cause all references tox in scopesnested within S to refer to thex bound in S.
The primary objection to this category of proposals is that themeaning of a function definition would become context-sensitive.Moving a function definition inside some other block could cause anyof the local name references in the function to become nonlocal, dueto declarations in the enclosing block. For blocks in Ruby 1.8,this is actually the case; in the following example, the two settershave different effects even though they look identical:
setter1=proc{|x|y=x}# y is local herey=13setter2=proc{|x|y=x}# y is nonlocal heresetter1.call(99)putsy# prints 13setter2.call(77)putsy# prints 77
Note that although this proposal resembles declarations in JavaScriptand Perl, the effect on the language is different because in thoselanguages undeclared variables are global by default, whereas inPython undeclared variables are local by default. Thus, movinga function inside some other block in JavaScript or Perl can onlyreduce the scope of a previously global name reference, whereas inPython with this proposal, it could expand the scope of a previouslylocal name reference.
A more radical proposal[21] suggests removing Python’s scope-guessingconvention altogether and requiring that all names be declared in thescope where they are to be bound, much like Scheme. With thisproposal,varx=3 would both declarex to belong to thelocal scope and bind it, where asx=3 would rebind the existingvisiblex. In a context without an enclosing scope containing avarx declaration, the statementx=3 would be staticallydetermined to be illegal.
This proposal yields a simple and consistent model, but it would beincompatible with all existing Python code.
There are three kinds of proposals in this category.
This type of proposal suggests a new way of referring to a variablein an outer scope when using the variable in an expression. Onesyntax that has been suggested for this is.x[7], which wouldrefer tox without creating a local binding for it. A concernwith this proposal is that in many contextsx and.x couldbe used interchangeably, which would confuse the reader[31]. Aclosely related idea is to use multiple dots to specify the numberof scope levels to ascend[8], but most consider this too error-prone[17].
This proposal suggests a new assignment-like operator that rebindsa name without declaring the name to be local[2]. Whereas thestatementx=3 both declaresx a local variable and bindsit to 3, the statementx:=3 would change the existing bindingofx without declaring it local.
This is a simple solution, but according toPEP 3099 it has beenrejected (perhaps because it would be too easy to miss or to confusewith=).
The proposals in this category suggest a new kind of declarationstatement in the inner scope that prevents a name from becominglocal. This statement would be similar in nature to theglobalstatement, but instead of making the name refer to a binding in thetop module-level scope, it would make the name refer to the bindingin the nearest enclosing scope.
This approach is attractive due to its parallel with a familiarPython construct, and because it retains context-independence forfunction definitions.
This approach also has advantages from a security and debuggingperspective. The resulting Python would not only match thefunctionality of other nested-scope languages but would do so with asyntax that is arguably even better for defensive programming. Inmost other languages, a declaration contracts the scope of anexisting name, so inadvertently omitting the declaration could yieldfarther-reaching (i.e. more dangerous) effects than expected. InPython with this proposal, the extra effort of adding the declarationis aligned with the increased risk of non-local effects (i.e. thepath of least resistance is the safer path).
Many spellings have been suggested for such a declaration:
scopedx[1]globalxinf[3] (explicitly specify which scope)freex[5]outerx[6]usex[9]globalx[10] (change the meaning ofglobal)nonlocalx[11]globalxouter[18]globalinx[18]notglobalx[18]externx[20]refx[22]referx[22]sharex[22]sharingx[22]commonx[22]usingx[22]borrowx[22]reusex[23]scopefx[25] (explicitly specify which scope)The most commonly discussed choices appear to beouter,global, andnonlocal.outer is already used as both avariable name and an attribute name in the standard library. Thewordglobal has a conflicting meaning, because “global variable”is generally understood to mean a variable with top-level scope[27].In C, the keywordextern means that a name refers to a variablein a different compilation unit. Whilenonlocal is a bit longand less pleasant-sounding than some of the other options, it doeshave precisely the correct meaning: it declares a name not local.
The solution proposed by this PEP is to add a scope overridedeclaration in the referring (inner) scope. Guido has expressed apreference for this category of solution on Python-Dev[14] and hasshown approval fornonlocal as the keyword[19].
The proposed declaration:
nonlocalx
preventsx from becoming a local name in the current scope. Alloccurrences ofx in the current scope will refer to thexbound in an outer enclosing scope. As withglobal, multiplenames are permitted:
nonlocalx,y,z
If there is no pre-existing binding in an enclosing scope, thecompiler raises a SyntaxError. (It may be a bit of a stretch tocall this a syntax error, but so far SyntaxError is used for allcompile-time errors, including, for example, __future__ importwith an unknown feature name.) Guido has said that this kind ofdeclaration in the absence of an outer binding should be consideredan error[16].
If anonlocal declaration collides with the name of a formalparameter in the local scope, the compiler raises a SyntaxError.
A shorthand form is also permitted, in whichnonlocal isprepended to an assignment or augmented assignment:
nonlocalx=3
The above has exactly the same meaning asnonlocalx;x=3.(Guido supports a similar form of theglobal statement[24].)
On the left side of the shorthand form, only identifiers are allowed,not target expressions likex[0]. Otherwise, all forms ofassignment are allowed. The proposed grammar of thenonlocalstatement is:
nonlocal_stmt::="nonlocal"identifier(","identifier)*["="(target_list"=")+expression_list]|"nonlocal"identifieraugopexpression_list
The rationale for allowing all these forms of assignment is that itsimplifies understanding of thenonlocal statement. Separatingthe shorthand form into a declaration and an assignment is sufficientto understand what it means and whether it is valid.
This PEP targets Python 3000, as suggested by Guido[19]. However,others have noted that some options considered in this PEP may besmall enough changes to be feasible in Python 2.x[26], in whichcase this PEP could possibly be moved to be a 2.x series PEP.
As a (very rough) measure of the impact of introducing a new keyword,here is the number of times that some of the proposed keywords appearas identifiers in the standard library, according to a scan of thePython SVN repository on November 5, 2006:
nonlocal0use2using3reuse4free8outer147
global appears 214 times as an existing keyword. As a measureof the impact of usingglobal as the outer-scope keyword, thereare 18 files in the standard library that would break as a resultof such a change (because a function declares a variableglobalbefore that variable has been introduced in the global scope):
cgi.pydummy_thread.pymhlib.pymimetypes.pyidlelib/PyShell.pyidlelib/run.pymsilib/__init__.pytest/inspect_fodder.pytest/test_compiler.pytest/test_decimal.pytest/test_descr.pytest/test_dummy_threading.pytest/test_fileinput.pytest/test_global.py(notcounted:thisteststhekeyworditself)test/test_grammar.py(notcounted:thisteststhekeyworditself)test/test_itertools.pytest/test_multifile.pytest/test_scope.py(notcounted:thisteststhekeyworditself)test/test_threaded_import.pytest/test_threadsignals.pytest/test_warnings.py
[15] Explicit Lexical Scoping (pre-PEP?) (Guido van Rossum)https://mail.python.org/pipermail/python-dev/2006-July/066995.html
The ideas and proposals mentioned in this PEP are gleaned fromcountless Python-Dev postings. Thanks to Jim Jewett, Mike Orr,Jason Orendorff, and Christian Tanzer for suggesting specificedits to this PEP.
This document has been placed in the public domain.
Source:https://github.com/python/peps/blob/main/peps/pep-3104.rst
Last modified:2025-02-01 08:55:40 GMT