Python Enhancement Proposals

Python »
PEP Index »
PEP 3114

PEP 3114 – Renaming iterator.next() to iterator.__next__()

PEP 3114 – Renaming iterator.next() to iterator.next()

Author:: Ka-Ping Yee <ping at zesty.ca>
Status:

Abstract

The iterator protocol in Python 2.x consists of two methods:__iter__() called on an iterable object to yield an iterator, andnext() called on an iterator object to yield the next item in thesequence. Using afor loop to iterate over an iterable objectimplicitly calls both of these methods. This PEP proposes that thenext method be renamed to__next__, consistent with all theother protocols in Python in which a method is implicitly called aspart of a language-level protocol, and that a built-in function namednext be introduced to invoke__next__ method, consistent withthe manner in which other protocols are explicitly invoked.

Names With Double Underscores

In Python, double underscores before and after a name are used todistinguish names that belong to the language itself. Attributes andmethods that are implicitly used or created by the interpreter employthis naming convention; some examples are:

__file__ - an attribute automatically created by the interpreter
__dict__ - an attribute with special meaning to the interpreter
__init__ - a method implicitly called by the interpreter

Note that this convention applies to methods such as__init__ thatare explicitly defined by the programmer, as well as attributes such as__file__ that can only be accessed by naming them explicitly, so itincludes names that are usedor created by the interpreter.

(Not all things that are called “protocols” are made of methods withdouble-underscore names. For example, the__contains__ method hasdouble underscores because the language constructxiny implicitlycalls__contains__. But even though theread method is part ofthe file protocol, it does not have double underscores because there isno language construct that implicitly invokesx.read().)

The use of double underscores creates a separate namespace for namesthat are part of the Python language definition, so that programmersare free to create variables, attributes, and methods that start withletters, without fear of silently colliding with names that have alanguage-defined purpose. (Colliding with reserved keywords is stilla concern, but at least this will immediately yield a syntax error.)

The naming of thenext method on iterators is an exception tothis convention. Code that nowhere contains an explicit call to anext method can nonetheless be silently affected by the presenceof such a method. Therefore, this PEP proposes that iterators shouldhave a__next__ method instead of anext method (with nochange in semantics).

Double-Underscore Methods and Built-In Functions

The Python language defines several protocols that are implemented orcustomized by defining methods with double-underscore names. In eachcase, the protocol is provided by an internal method implemented as aC function in the interpreter. For objects defined in Python, thisC function supports customization by implicitly invoking a Python methodwith a double-underscore name (it often does a little bit of additionalwork beyond just calling the Python method.)

Sometimes the protocol is invoked by a syntactic construct:

x[y] –> internaltp_getitem –>x.__getitem__(y)
x+y –> internalnb_add –>x.__add__(y)
-x –> internalnb_negative –>x.__neg__()

Sometimes there is no syntactic construct, but it is still useful to beable to explicitly invoke the protocol. For such cases Python offers abuilt-in function of the same name but without the double underscores.

len(x) –> internalsq_length –>x.__len__()
hash(x) –> internaltp_hash –>x.__hash__()
iter(x) –> internaltp_iter –>x.__iter__()

Following this pattern, the natural way to handlenext is to add anext built-in function that behaves in exactly the same fashion.

next(x) –> internaltp_iternext –>x.__next__()

Further, it is proposed that thenext built-in function accept asentinel value as an optional second argument, following the style ofthegetattr anditer built-in functions. When called with twoarguments,next catches the StopIteration exception and returnsthe sentinel value instead of propagating the exception. This createsa nice duality betweeniter andnext:

iter(function, sentinel) <–> next(iterator, sentinel)

Previous Proposals

This proposal is not a new idea. The idea proposed here was supportedby the BDFL on python-dev[1] and is even mentioned in the originaliterator PEP,PEP 234:

(Inretrospect,itmighthavebeenbettertogofor__next__()andhaveanewbuilt-in,next(it),whichcallsit.__next__().Butalas,it's too late; this has been deployed in Python 2.2sinceDecember2001.)

Objections

There have been a few objections to the addition of more built-ins.In particular, Martin von Loewis writes[2]:

Idisliketheintroductionofmorebuiltinsunlesstheyhaveatruegenerality(i.e.arelikelytobeneededinmanyprograms).Forthisone,Ithinkthenormalusageof__next__willbewithaforloop,soIdon't think one would often need an explicit next() invocation.Itisalsonottruethatmostprotocolsareexplicitlyinvokedthroughbuiltinfunctions.Instead,mostprotocolsarecanbeexplicitlyinvokedthroughmethodsintheoperatormodule.Sofollowingtradition,itshouldbeoperator.next....Asanalternative,Iproposethatobjectgrowsa.next()method,whichcalls__next__bydefault.

Transition Plan

Two additional transformations will be added to the 2to3 translationtool[3]:

Method definitions namednext will be renamed to__next__.
Explicit calls to thenext method will be replaced with callsto the built-innext function. For example,x.next() willbecomenext(x).

Collin Winter looked into the possibility of automatically decidingwhether to perform the second transformation depending on the presenceof a module-level binding tonext[4] and found that it would be“ugly and slow”. Instead, the translation tool will emit warningsupon detecting such a binding. Collin has proposed warnings for thefollowing conditions[5]: