Introduction

This PEP describes the “range literal” proposal for Python 2.0.This PEP tracks the status and ownership of this feature, slatedfor introduction in Python 2.0. It contains a description of thefeature and outlines changes necessary to support the feature.This PEP summarizes discussions held in mailing list forums, andprovides URLs for further information, where appropriate. The CVSrevision history of this file contains the definitive historicalrecord.

List ranges

Ranges are sequences of numbers of a fixed stepping, often used infor-loops. The Python for-loop is designed to iterate over asequence directly:

>>>l=['a','b','c','d']>>>foriteminl:...printitemabcd

However, this solution is not always prudent. Firstly, problemsarise when altering the sequence in the body of the for-loop,resulting in the for-loop skipping items. Secondly, it is notpossible to iterate over, say, every second element of thesequence. And thirdly, it is sometimes necessary to process anelement based on its index, which is not readily available in theabove construct.

For these instances, and others where a range of numbers isdesired, Python provides therange builtin function, whichcreates a list of numbers. Therange function takes threearguments,start,end andstep.start andstep areoptional, and default to 0 and 1, respectively.

Therange function creates a list of numbers, starting atstart, with a step ofstep, up to, but not includingend, sothatrange(10) produces a list that has exactly 10 items, thenumbers 0 through 9.

Using therange function, the above example would look likethis:

>>>foriinrange(len(l)):...printl[i]abcd

Or, to start at the second element ofl and processing onlyevery second element from then on:

>>>foriinrange(1,len(l),2):...printl[i]bd

There are several disadvantages with this approach:

• Clarity of purpose: Adding another function call, possibly withextra arithmetic to determine the desired length and step of thelist, does not improve readability of the code. Also, it ispossible to “shadow” the builtinrange function by supplying alocal or global variable with the same name, effectivelyreplacing it. This may or may not be a desired effect.
• Efficiency: because therange function can be overridden, thePython compiler cannot make assumptions about the for-loop, andhas to maintain a separate loop counter.
• Consistency: There already is a syntax that is used to denoteranges, as shown below. This syntax uses the exact samearguments, though all optional, in the exact same way. It seemslogical to extend this syntax to ranges, to form “rangeliterals”.

Slice Indices

In Python, a sequence can be indexed in one of two ways:retrieving a single item, or retrieving a range of items.Retrieving a range of items results in a new object of the sametype as the original sequence, containing zero or more items fromthe original sequence. This is done using a “range notation”:

>>>l[2:4]['c', 'd']

This range notation consists of zero, one or two indices separatedby a colon. The first index is thestart index, the second theend. When either is left out, they default to respectively thestart and the end of the sequence.

There is also an extended range notation, which incorporatesstep as well. Though this notation is not currently supportedby most builtin types, if it were, it would work as follows:

>>>l[1:4:2]['b', 'd']

The third “argument” to the slice syntax is exactly the same asthestep argument torange(). The underlying mechanisms of thestandard, and these extended slices, are sufficiently differentand inconsistent that many classes and extensions outside ofmathematical packages do not implement support for the extendedvariant. While this should be resolved, it is beyond the scope ofthis PEP.

Extended slices do show, however, that there is already aperfectly valid and applicable syntax to denote ranges in a waythat solve all of the earlier stated disadvantages of the use oftherange() function:

• It is clearer, more concise syntax, which has already proven tobe both intuitive and easy to learn.
• It is consistent with the other use of ranges in Python(e.g. slices).
• Because it is built-in syntax, instead of a builtin function, itcannot be overridden. This means both that a viewer can becertain about what the code does, and that an optimizer will nothave to worry aboutrange() being “shadowed”.

The Proposed Solution

The proposed implementation of range-literals combines the syntaxfor list literals with the syntax for (extended) slices, to formrange literals:

>>>[1:10][1, 2, 3, 4, 5, 6, 7, 8, 9]>>>[:5][0, 1, 2, 3, 4]>>>[5:1:-1][5, 4, 3, 2]

There is one minor difference between range literals and the slicesyntax: though it is possible to omit all ofstart,end andstep in slices, it does not make sense to omitend in rangeliterals. In slices,end would default to the end of the list,but this has no meaning in range literals.

Reference Implementation

The proposed implementation can be found on SourceForge[1]. Itadds a new bytecode,BUILD_RANGE, that takes three arguments fromthe stack and builds a list on the bases of those. The list ispushed back on the stack.

The use of a new bytecode is necessary to be able to build rangesbased on other calculations, whose outcome is not known at compiletime.

The code introduces two new functions tolistobject.c, which arecurrently hovering between private functions and full-fledged APIcalls.

PyList_FromRange() builds a list from start, end and step,returning NULL if an error occurs. Its prototype is:

PyObject*PyList_FromRange(longstart,longend,longstep)

PyList_GetLenOfRange() is a helper function used to determine thelength of a range. Previously, it was a static function inbltinmodule.c, but is now necessary in bothlistobject.c andbltinmodule.c (forxrange). It is made non-static solely to avoidcode duplication. Its prototype is:

longPyList_GetLenOfRange(longstart,longend,longstep)

Open issues

• One possible solution to the discrepancy of requiring theendargument in range literals is to allow the range syntax tocreate a “generator”, rather than a list, such as thexrangebuiltin function does. However, a generator would not be alist, and it would be impossible, for instance, to assign toitems in the generator, or append to it.

  The range syntax could conceivably be extended to include tuples(i.e. immutable lists), which could then be safely implementedas generators. This may be a desirable solution, especially forlarge number arrays: generators require very little in the wayof storage and initialization, and there is only a smallperformance impact in calculating and creating the appropriatenumber on request. (TBD: is there any at all? Cursory testingsuggests equal performance even in the case of ranges of length1)

  However, even if idea was adopted, would it be wise to “specialcase” the second argument, making it optional in one instance ofthe syntax, and non-optional in other cases ?

• Should it be possible to mix range syntax with normal listliterals, creating a single list? E.g.:

  >>>[5,6,1:6,7,9]

  to create:

  [5,6,1,2,3,4,5,7,9]

• How should range literals interact with another proposed newfeature,“list comprehensions”? Specifically, should it bepossible to create lists in list comprehensions? E.g.:

  >>>[x:yforxin(1,2)yin(3,4)]

  Should this example return a single list with multiple ranges:

  [1,2,1,2,3,2,2,3]

  Or a list of lists, like so:

  [[1,2],[1,2,3],[2],[2,3]]

  However, as the syntax and semantics of list comprehensions arestill subject of hot debate, these issues are probably bestaddressed by the “list comprehensions” PEP.

• Range literals accept objects other than integers: it performsPyInt_AsLong() on the objects passed in, so as long as theobjects can be coerced into integers, they will be accepted.The resulting list, however, is always composed of standardintegers.

  Should range literals create a list of the passed-in type? Itmight be desirable in the cases of other builtin types, such aslongs and strings:

  >>>[1L:2L<<64:2<<32L]>>>["a":"z":"b"]>>>["a":"z":2]

  However, this might be too much “magic” to be obvious. It mightalso present problems with user-defined classes: even if thebase class can be found and a new instance created, the instancemay require additional arguments to__init__, causing thecreation to fail.

• ThePyList_FromRange() andPyList_GetLenOfRange() functions needto be classified: are they part of the API, or should they bemade private functions?

Copyright

This document has been placed in the Public Domain.

References