Introduction

This PEP describes the ‘lockstep iteration’ proposal. This PEP tracksthe status and ownership of this feature, slated for introduction inPython 2.0. It contains a description of the feature and outlineschanges necessary to support the feature. This PEP summarizesdiscussions held in mailing list forums, and provides URLs for furtherinformation, where appropriate. The CVS revision history of this filecontains the definitive historical record.

Motivation

Standard for-loops in Python iterate over every element in a sequenceuntil the sequence is exhausted[1]. However, for-loops iterate overonly a single sequence, and it is often desirable to loop over morethan one sequence in a lock-step fashion. In other words, in a waysuch that the i-th iteration through the loop returns an objectcontaining the i-th element from each sequence.

The common idioms used to accomplish this are unintuitive. This PEPproposes a standard way of performing such iterations by introducing anew builtin function calledzip.

While the primary motivation for zip() comes from lock-step iteration,by implementing zip() as a built-in function, it has additionalutility in contexts other than for-loops.

Lockstep For-Loops

Lockstep for-loops are non-nested iterations over two or moresequences, such that at each pass through the loop, one element fromeach sequence is taken to compose the target. This behavior canalready be accomplished in Python through the use of the map() built-infunction:

>>>a=(1,2,3)>>>b=(4,5,6)>>>foriinmap(None,a,b):printi...(1, 4)(2, 5)(3, 6)>>>map(None,a,b)[(1, 4), (2, 5), (3, 6)]

The for-loop simply iterates over this list as normal.

While the map() idiom is a common one in Python, it has severaldisadvantages:

• It is non-obvious to programmers without a functional programmingbackground.
• The use of the magicNone first argument is non-obvious.
• It has arbitrary, often unintended, and inflexible semantics whenthe lists are not of the same length: the shorter sequences arepadded withNone:

  >>>c=(4,5,6,7)>>>map(None,a,c)[(1, 4), (2, 5), (3, 6), (None, 7)]

For these reasons, several proposals were floated in the Python 2.0beta time frame for syntactic support of lockstep for-loops. Here aretwo suggestions:

forxinseq1,yinseq2:# stuff

forx,yinseq1,seq2:# stuff

Neither of these forms would work, since they both already meansomething in Python and changing the meanings would break existingcode. All other suggestions for new syntax suffered the same problem,or were in conflict with other another proposed feature called ‘listcomprehensions’ (seePEP 202).

The Proposed Solution

The proposed solution is to introduce a new built-in sequencegenerator function, available in the__builtin__ module. Thisfunction is to be calledzip and has the following signature:

zip(seqa,[seqb,[...]])

zip() takes one or more sequences and weaves their elementstogether, just asmap(None,...) does with sequences of equallength. The weaving stops when the shortest sequence is exhausted.

Return Value

zip() returns a real Python list, the same waymap() does.

Examples

Here are some examples, based on the reference implementation below:

>>>a=(1,2,3,4)>>>b=(5,6,7,8)>>>c=(9,10,11)>>>d=(12,13)>>>zip(a,b)[(1, 5), (2, 6), (3, 7), (4, 8)]>>>zip(a,d)[(1, 12), (2, 13)]>>>zip(a,b,c,d)[(1, 5, 9, 12), (2, 6, 10, 13)]

Note that when the sequences are of the same length,zip() isreversible:

>>>a=(1,2,3)>>>b=(4,5,6)>>>x=zip(a,b)>>>y=zip(*x)# alternatively, apply(zip, x)>>>z=zip(*y)# alternatively, apply(zip, y)>>>x[(1, 4), (2, 5), (3, 6)]>>>y[(1, 2, 3), (4, 5, 6)]>>>z[(1, 4), (2, 5), (3, 6)]>>>x==z1

It is not possible to reverse zip this way when the sequences are notall the same length.

Reference Implementation

Here is a reference implementation, in Python of the zip() built-infunction. This will be replaced with a C implementation after finalapproval:

defzip(*args):ifnotargs:raiseTypeError('zip() expects one or more sequence arguments')ret=[]i=0try:while1:item=[]forsinargs:item.append(s[i])ret.append(tuple(item))i=i+1exceptIndexError:returnret

BDFL Pronouncements

Note: the BDFL refers to Guido van Rossum, Python’s BenevolentDictator For Life.

• The function’s name. An earlier version of this PEP included anopen issue listing 20+ proposed alternative names tozip(). Inthe face of no overwhelmingly better choice, the BDFL stronglypreferszip() due to its Haskell[2] heritage. See version 1.7of this PEP for the list of alternatives.
• zip() shall be a built-in function.
• Optional padding. An earlier version of this PEP proposed anoptionalpad keyword argument, which would be used when theargument sequences were not the same length. This is similarbehavior to themap(None,...) semantics except that the userwould be able to specify pad object. This has been rejected by theBDFL in favor of always truncating to the shortest sequence, becauseof the KISS principle. If there’s a true need, it is easier to addlater. If it is not needed, it would still be impossible to deleteit in the future.
• Lazy evaluation. An earlier version of this PEP proposed thatzip() return a built-in object that performed lazy evaluationusing__getitem__() protocol. This has been strongly rejectedby the BDFL in favor of returning a real Python list. If lazyevaluation is desired in the future, the BDFL suggests anxzip()function be added.
• zip() with no arguments. the BDFL strongly prefers this raise aTypeError exception.
• zip() with one argument. the BDFL strongly prefers that thisreturn a list of 1-tuples.
• Inner and outer container control. An earlier version of this PEPcontains a rather lengthy discussion on a feature that some peoplewanted, namely the ability to control what the inner and outercontainer types were (they are tuples and list respectively in thisversion of the PEP). Given the simplified API and implementation,this elaboration is rejected. For a more detailed analysis, seeversion 1.7 of this PEP.

Subsequent Change tozip()

In Python 2.4, zip() with no arguments was modified to return an emptylist rather than raising a TypeError exception. The rationale for theoriginal behavior was that the absence of arguments was thought toindicate a programming error. However, that thinking did notanticipate the use of zip() with the* operator for unpackingvariable length argument lists. For example, the inverse of zip couldbe defined as:unzip=lambdas:zip(*s). That transformationalso defines a matrix transpose or an equivalent row/column swap fortables defined as lists of tuples. The latter transformation iscommonly used when reading data files with records as rows and fieldsas columns. For example, the code:

date,rain,high,low=zip(*csv.reader(file("weather.csv")))

rearranges columnar data so that each field is collected intoindividual tuples for straightforward looping and summarization:

print"Total rainfall",sum(rain)

Usingzip(*args) is more easily coded ifzip(*[]) is handledas an allowable case rather than an exception. This is especiallyhelpful when data is either built up from or recursed down to a nullcase with no records.

Seeing this possibility, the BDFL agreed (with some misgivings) tohave the behavior changed for Py2.4.

Other Changes

• Thexzip() function discussed above was implemented in Py2.3 intheitertools module asitertools.izip(). This functionprovides lazy behavior, consuming single elements and producing asingle tuple on each pass. The “just-in-time” style saves memoryand runs faster than its list based counterpart,zip().
• Theitertools module also addeditertools.repeat() anditertools.chain(). These tools can be used together to padsequences withNone (to match the behavior ofmap(None,seqn)):

  zip(firstseq,chain(secondseq,repeat(None)))

References

Copyright

This document has been placed in the public domain.