Abstract

In most languages that support nested scopes, code can refer to or rebind (assign to) any name in the nearest enclosing scope. Currently, Python code can refer to a name in any enclosing scope, but it can only rebind names in two scopes: the local scope (by simple assignment) or the module-global scope (using a global declaration).

This limitation has been raised many times on the Python-Dev mailing list and elsewhere, and has led to extended discussion and many proposals for ways to remove this limitation. This PEP summarizes the various alternatives that have been suggested, together with advantages and disadvantages that have been mentioned for each.

Rationale

Before version 2.1, Python’s treatment of scopes resembled that of standard C: within a file there were only two levels of scope, global and local. In C, this is a natural consequence of the fact that function definitions cannot be nested. But in Python, though functions are usually defined at the top level, a function definition can be executed anywhere. This gave Python the syntactic appearance of nested scoping without the semantics, and yielded inconsistencies that were surprising to some programmers – for example, a recursive function that worked at the top level would cease to work when moved inside another function, because the recursive function’s own name would no longer be visible in its body’s scope. This violates the intuition that a function should behave consistently when placed in different contexts. Here’s an example:

def enclosing_function():
    def factorial(n):
        if n < 2:
            return 1
        return n * factorial(n - 1)  # fails with NameError
    print factorial(5)

Python 2.1 moved closer to static nested scoping by making visible the names bound in all enclosing scopes (see PEP 227). This change makes the above code example work as expected. However, because any assignment to a name implicitly declares that name to be local, it is impossible to rebind a name in an outer scope (except when a global declaration forces the name to be global). Thus, the following code, intended to display a number that can be incremented and decremented by clicking buttons, doesn’t work as someone familiar with lexical scoping might expect:

def make_scoreboard(frame, score=0):
    label = Label(frame)
    label.pack()
    for i in [-10, -1, 1, 10]:
        def increment(step=i):
            score = score + step  # fails with UnboundLocalError
            label['text'] = score
        button = Button(frame, text='%+d' % i, command=increment)
        button.pack()
    return label

Python syntax doesn’t provide a way to indicate that the name score mentioned in increment refers to the variable score bound in make_scoreboard, not a local variable in increment. Users and developers of Python have expressed an interest in removing this limitation so that Python can have the full flexibility of the Algol-style scoping model that is now standard in many programming languages, including JavaScript, Perl, Ruby, Scheme, Smalltalk, C with GNU extensions, and C# 2.0.

It has been argued that such a feature isn’t necessary, because a rebindable outer variable can be simulated by wrapping it in a mutable object:

class Namespace:
    pass

def make_scoreboard(frame, score=0):
    ns = Namespace()
    ns.score = 0
    label = Label(frame)
    label.pack()
    for i in [-10, -1, 1, 10]:
        def increment(step=i):
            ns.score = ns.score + step
            label['text'] = ns.score
        button = Button(frame, text='%+d' % i, command=increment)
        button.pack()
    return label

However, this workaround only highlights the shortcomings of existing scopes: the purpose of a function is to encapsulate code in its own namespace, so it seems unfortunate that the programmer should have to create additional namespaces to make up for missing functionality in the existing local scopes, and then have to decide whether each name should reside in the real scope or the simulated scope.

Another common objection is that the desired functionality can be written as a class instead, albeit somewhat more verbosely. One rebuttal to this objection is that the existence of a different implementation style is not a reason to leave a supported programming construct (nested scopes) functionally incomplete. Python is sometimes called a “multi-paradigm language” because it derives so much strength, practical flexibility, and pedagogical power from its support and graceful integration of multiple programming paradigms.

A proposal for scoping syntax appeared on Python-Dev as far back as 1994 [1], long before PEP 227’s support for nested scopes was adopted. At the time, Guido’s response was:

This is dangerously close to introducing CSNS [classic static nested scopes]. If you were to do so, your proposed semantics of scoped seem alright. I still think there is not enough need for CSNS to warrant this kind of construct …

After PEP 227, the “outer name rebinding discussion” has reappeared on Python-Dev enough times that it has become a familiar event, having recurred in its present form since at least 2003 [2]. Although none of the language changes proposed in these discussions have yet been adopted, Guido has acknowledged that a language change is worth considering [12].

Other Languages

To provide some background, this section describes how some other languages handle nested scopes and rebinding.

Overview of Proposals

There have been many different proposals on Python-Dev for ways to rebind names in outer scopes. They all fall into two categories: new syntax in the scope where the name is bound, or new syntax in the scope where the name is used.

setter1 = proc { | x | y = x }      # y is local here
y = 13
setter2 = proc { | x | y = x }      # y is nonlocal here
setter1.call(99)
puts y                              # prints 13
setter2.call(77)
puts y                              # prints 77

Proposed Solution

The solution proposed by this PEP is to add a scope override declaration in the referring (inner) scope. Guido has expressed a preference for this category of solution on Python-Dev [14] and has shown approval for nonlocal as the keyword [19].

The proposed declaration:

nonlocal x

prevents x from becoming a local name in the current scope. All occurrences of x in the current scope will refer to the x bound in an outer enclosing scope. As with global, multiple names are permitted:

nonlocal x, y, z

If there is no pre-existing binding in an enclosing scope, the compiler raises a SyntaxError. (It may be a bit of a stretch to call this a syntax error, but so far SyntaxError is used for all compile-time errors, including, for example, __future__ import with an unknown feature name.) Guido has said that this kind of declaration in the absence of an outer binding should be considered an error [16].

If a nonlocal declaration collides with the name of a formal parameter in the local scope, the compiler raises a SyntaxError.

A shorthand form is also permitted, in which nonlocal is prepended to an assignment or augmented assignment:

nonlocal x = 3

The above has exactly the same meaning as nonlocal x; x = 3. (Guido supports a similar form of the global statement [24].)

On the left side of the shorthand form, only identifiers are allowed, not target expressions like x[0]. Otherwise, all forms of assignment are allowed. The proposed grammar of the nonlocal statement is:

nonlocal_stmt ::=
    "nonlocal" identifier ("," identifier)*
               ["=" (target_list "=")+ expression_list]
  | "nonlocal" identifier augop expression_list

The rationale for allowing all these forms of assignment is that it simplifies understanding of the nonlocal statement. Separating the shorthand form into a declaration and an assignment is sufficient to understand what it means and whether it is valid.

Backward Compatibility

This PEP targets Python 3000, as suggested by Guido [19]. However, others have noted that some options considered in this PEP may be small enough changes to be feasible in Python 2.x [26], in which case this PEP could possibly be moved to be a 2.x series PEP.

As a (very rough) measure of the impact of introducing a new keyword, here is the number of times that some of the proposed keywords appear as identifiers in the standard library, according to a scan of the Python SVN repository on November 5, 2006:

nonlocal    0
use         2
using       3
reuse       4
free        8
outer     147

global appears 214 times as an existing keyword. As a measure of the impact of using global as the outer-scope keyword, there are 18 files in the standard library that would break as a result of such a change (because a function declares a variable global before that variable has been introduced in the global scope):

cgi.py
dummy_thread.py
mhlib.py
mimetypes.py
idlelib/PyShell.py
idlelib/run.py
msilib/__init__.py
test/inspect_fodder.py
test/test_compiler.py
test/test_decimal.py
test/test_descr.py
test/test_dummy_threading.py
test/test_fileinput.py
test/test_global.py (not counted: this tests the keyword itself)
test/test_grammar.py (not counted: this tests the keyword itself)
test/test_itertools.py
test/test_multifile.py
test/test_scope.py (not counted: this tests the keyword itself)
test/test_threaded_import.py
test/test_threadsignals.py
test/test_warnings.py

References

[15] Explicit Lexical Scoping (pre-PEP?) (Guido van Rossum) https://mail.python.org/pipermail/python-dev/2006-July/066995.html

Acknowledgements

The ideas and proposals mentioned in this PEP are gleaned from countless Python-Dev postings. Thanks to Jim Jewett, Mike Orr, Jason Orendorff, and Christian Tanzer for suggesting specific edits to this PEP.

Copyright

This document has been placed in the public domain.