Abstract

This PEP proposes an enhancement to generator expressions, adding a“while” clause to complement the existing “if” clause.

Rationale

A generator expression (PEP 289) is a concise method to servedynamically-generated objects to list comprehensions (PEP 202).Current generator expressions allow for an “if” clause to filterthe objects that are returned to those meeting some set ofcriteria. However, since the “if” clause is evaluated for everyobject that may be returned, in some cases it is possible that allobjects would be rejected after a certain point. For example:

g=(nforninrange(100)ifn*n<50)

which is equivalent to the using a generator function(PEP 255):

def__gen(exp):forninexp:ifn*n<50:yieldng=__gen(iter(range(10)))

would yield 0, 1, 2, 3, 4, 5, 6 and 7, but would also considerthe numbers from 8 to 99 and reject them all sincen*n>=50 fornumbers in that range. Allowing for a “while” clause would allowthe redundant tests to be short-circuited:

g=(nforninrange(100)whilen*n<50)

would also yield 0, 1, 2, 3, 4, 5, 6 and 7, but would stop at 8since the condition (n*n<50) is no longer true. This would beequivalent to the generator function:

def__gen(exp):forninexp:ifn*n<50:yieldnelse:breakg=__gen(iter(range(100)))

Currently, in order to achieve the same result, one would need toeither write a generator function such as the one above or use thetakewhile function from itertools:

fromitertoolsimporttakewhileg=takewhile(lambdan:n*n<50,range(100))

The takewhile code achieves the same result as the proposed syntax,albeit in a longer (some would say “less-elegant”) fashion. Also,the takewhile version requires an extra function call (the lambdain the example above) with the associated performance penalty.A simple test shows that:

fornin(nforninrange(100)if1):pass

performs about 10% better than:

fornintakewhile(lambdan:1,range(100)):pass

though they achieve similar results. (The first example uses agenerator; takewhile is an iterator). If similarly implemented,a “while” clause should perform about the same as the “if” clausedoes today.

The reader may ask if the “if” and “while” clauses should bemutually exclusive. There are good examples that show that thereare times when both may be used to good advantage. For example:

p=(pforpinprimes()ifp>100whilep<1000)

should return prime numbers found between 100 and 1000, assumingI have aprimes() generator that yields prime numbers.

Adding a “while” clause to generator expressions maintains thecompact form while adding a useful facility for short-circuitingthe expression.

Acknowledgements

Raymond Hettinger first proposed the concept of generatorexpressions in January 2002.

Copyright

This document has been placed in the public domain.