PEP 525 asyncgen hooks

PEP 525 proposes a set of global thread-local hooksmanaged by newsys.{get/set}_asyncgen_hooks() functions, whichallow event loops to integrate with the garbage collector to runcleanup for async generators. In principle, this proposal andPEP 525are complementary, in the same way thatwith blocks and__del__ are complementary: this proposal takes care of ensuringdeterministic cleanup in most cases, whilePEP 525’s GC hooks clean upanything that gets missed. But__aiterclose__ provides a number ofadvantages over GC hooks alone:

• The GC hook semantics aren’t part of the abstract async iteratorprotocol, but are instead restrictedspecifically to the asyncgenerator concrete type. Ifyou have an async iterator implemented using a class, like:

  classMyAsyncIterator:asyncdef__anext__():...

  then you can’t refactor this into an async generator withoutchanging its semantics, and vice-versa. This seems veryunpythonic. (It also leaves open the question of what exactlyclass-based async iterators are supposed to do, given that they faceexactly the same cleanup problems as async generators.)__aiterclose__, on the other hand, is defined at the protocollevel, so it’s duck-type friendly and works for all iterators, notjust generators.

• Code that wants to work on non-CPython implementations like PyPycannot in general rely on GC for cleanup. Without__aiterclose__, it’s more or less guaranteed that developers whodevelop and test on CPython will produce libraries that leakresources when used on PyPy. Developers who do want to targetalternative implementations will either have to take the defensiveapproach of wrapping everyfor loop in awith block, or elsecarefully audit their code to figure out which generators mightpossibly contain cleanup code and addwith blocks around thoseonly. With__aiterclose__, writing portable code becomes easyand natural.
• An important part of building robust software is making sure thatexceptions always propagate correctly without being lost. One of themost exciting things about async/await compared to traditionalcallback-based systems is that instead of requiring manual chaining,the runtime can now do the heavy lifting of propagating errors,making itmuch easier to write robust code. But, this beautifulnew picture has one major gap: if we rely on the GC for generatorcleanup, then exceptions raised during cleanup are lost. So, again,with__aiterclose__, developers who care about this kind ofrobustness will either have to take the defensive approach ofwrapping everyfor loop in awith block, or else carefullyaudit their code to figure out which generators might possiblycontain cleanup code.__aiterclose__ plugs this hole byperforming cleanup in the caller’s context, so writing more robustcode becomes the path of least resistance.
• The WSGI experience suggests that there exist importantiterator-based APIs that need prompt cleanup and cannot rely on theGC, even in CPython. For example, consider a hypothetical WSGI-likeAPI based around async/await and async iterators, where a responsehandler is an async generator that takes request headers + an asynciterator over the request body, and yields response headers + theresponse body. (This is actually the use case that got me interestedin async generators in the first place, i.e. this isn’thypothetical.) If we follow WSGI in requiring that child iteratorsmust be closed properly, then without__aiterclose__ theabsolute most minimalistic middleware in our system looks somethinglike:

  asyncdefnoop_middleware(handler,request_header,request_body):asyncwithaclosing(handler(request_body,request_body))asaiter:asyncforresponse_iteminaiter:yieldresponse_item

  Arguably in regular code one can get away with skipping thewithblock aroundfor loops, depending on how confident one is thatone understands the internal implementation of the generator. Buthere we have to cope with arbitrary response handlers, so without__aiterclose__, thiswith construction is a mandatory partof every middleware.

  __aiterclose__ allows us to eliminate the mandatory boilerplateand an extra level of indentation from every middleware:

  asyncdefnoop_middleware(handler,request_header,request_body):asyncforresponse_iteminhandler(request_header,request_body):yieldresponse_item

So the__aiterclose__ approach provides substantial advantagesover GC hooks.

This leaves open the question of whether we want a combination of GChooks +__aiterclose__, or just__aiterclose__ alone. Sincethe vast majority of generators are iterated over using afor loopor equivalent,__aiterclose__ handles most situations before theGC has a chance to get involved. The case where GC hooks provideadditional value is in code that does manual iteration, e.g.:

agen=fetch_newline_separated_json_from_url(...)whileTrue:document=awaittype(agen).__anext__(agen)ifdocument["id"]==needle:break# doesn't do 'await agen.aclose()'

If we go with the GC-hooks +__aiterclose__ approach, thisgenerator will eventually be cleaned up by GC calling the generator__del__ method, which then will use the hooks to call back intothe event loop to run the cleanup code.

If we go with the no-GC-hooks approach, this generator will eventuallybe garbage collected, with the following effects:

• its__del__ method will issue a warning that the generator wasnot closed (similar to the existing “coroutine never awaited”warning).
• The underlying resources involved will still be cleaned up, becausethe generator frame will still be garbage collected, causing it todrop references to any file handles or sockets it holds, and thenthose objects’s__del__ methods will release the actualoperating system resources.
• But, any cleanup code inside the generator itself (e.g. logging,buffer flushing) will not get a chance to run.

The solution here – as the warning would indicate – is to fix thecode so that it calls__aiterclose__, e.g. by using awithblock:

asyncwithaclosing(fetch_newline_separated_json_from_url(...))asagen:whileTrue:document=awaittype(agen).__anext__(agen)ifdocument["id"]==needle:break

Basically in this approach, the rule would be that if you want tomanually implement the iterator protocol, then it’s yourresponsibility to implement all of it, and that now includes__(a)iterclose__.

GC hooks add non-trivial complexity in the form of (a) new globalinterpreter state, (b) a somewhat complicated control flow (e.g.,async generator GC always involves resurrection, so the details of PEP442 are important), and (c) a new public API in asyncio (awaitloop.shutdown_asyncgens()) that users have to remember to call atthe appropriate time. (This last point in particular somewhatundermines the argument that GC hooks provide a safe backup toguarantee cleanup, since ifshutdown_asyncgens() isn’t calledcorrectly then Ithink it’s possible for generators to be silentlydiscarded without their cleanup code being called; compare this to the__aiterclose__-only approach where in the worst case we still atleast get a warning printed. This might be fixable.) All thisconsidered, GC hooks arguably aren’t worth it, given that the onlypeople they help are those who want to manually call__anext__ yetdon’t want to manually call__aiterclose__. But Yury disagreeswith me on this :-). And both options are viable.

Always inject resources, and do all cleanup at the top level

Several commentators on python-dev and python-ideas have suggestedthat a pattern to avoid these problems is to always pass resources infrom above, e.g.read_newline_separated_json should take a fileobject rather than a path, with cleanup handled at the top level:

defread_newline_separated_json(file_handle):forlineinfile_handle:yieldjson.loads(line)defread_users(file_handle):fordocumentinread_newline_separated_json(file_handle):yieldUser.from_json(document)withopen(path)asfile_handle:foruserinread_users(file_handle):...

This works well in simple cases; here it lets us avoid the “N+1with blocks problem”. But unfortunately, it breaks down quicklywhen things get more complex. Consider if instead of reading from afile, our generator was reading from a streaming HTTP GET request –while handling redirects and authentication via OAUTH. Then we’dreally want the sockets to be managed down inside our HTTP clientlibrary, not at the top level. Plus there are other cases wherefinally blocks embedded inside generators are important in theirown right: db transaction management, emitting logging informationduring cleanup (one of the major motivating use cases for WSGIclose), and so forth. So this is really a workaround for simplecases, not a general solution.

More complex variants of __(a)iterclose__

The semantics of__(a)iterclose__ are somewhat inspired bywith blocks, but context managers are more powerful:__(a)exit__ can distinguish between a normal exit versus exceptionunwinding, and in the case of an exception it can examine theexception details and optionally suppresspropagation.__(a)iterclose__ as proposed here does not have thesepowers, but one can imagine an alternative design where it did.

However, this seems like unwarranted complexity: experience suggeststhat it’s common for iterables to haveclose methods, and even tohave__exit__ methods that callself.close(), but I’m notaware of any common cases that make use of__exit__’s fullpower. I also can’t think of any examples where this would beuseful. And it seems unnecessarily confusing to allow iterators toaffect flow control by swallowing exceptions – if you’re in asituation where you really want that, then you should probably use arealwith block anyway.

Guiding principles

Generally,__(a)iterclose__ implementations should:

• be idempotent,
• perform any cleanup that is appropriate on the assumption that theiterator will not be used again after__(a)iterclose__ iscalled. In particular, once__(a)iterclose__ has been calledthen calling__(a)next__ produces undefined behavior.

And generally, any code which starts iterating through an iterablewith the intention of exhausting it, should arrange to make sure that__(a)iterclose__ is eventually called, whether or not the iteratoris actually exhausted.

Changes to iteration

The core proposal is the change in behavior offor loops. Giventhis Python code:

forVARinITERABLE:LOOP-BODYelse:ELSE-BODY

we desugar to the equivalent of:

_iter=iter(ITERABLE)_iterclose=getattr(type(_iter),"__iterclose__",lambda:None)try:traditional-forVARin_iter:LOOP-BODYelse:ELSE-BODYfinally:_iterclose(_iter)

where the “traditional-for statement” here is meant as a shorthand forthe classic 3.5-and-earlierfor loop semantics.

Besides the top-levelfor statement, Python also contains severalother places where iterators are consumed. For consistency, theseshould call__iterclose__ as well using semantics equivalent tothe above. This includes:

• for loops inside comprehensions
• * unpacking
• functions which accept and fully consume iterables, likelist(it),tuple(it),itertools.product(it1,it2,...),and others.

In addition, ayieldfrom that successfully exhausts the calledgenerator should as a last step call its__iterclose__method. (Rationale:yieldfrom already links the lifetime of thecalling generator to the called generator; if the calling generator isclosed when half-way through ayieldfrom, then this will alreadyautomatically close the called generator.)

Changes to async iteration

We also make the analogous changes to async iteration constructs,except that the new slot is called__aiterclose__, and it’s anasync method that getsawaited.

Modifications to basic iterator types

Generator objects (including those created by generatorcomprehensions):

• __iterclose__ callsself.close()
• __del__ callsself.close() (same as now), and additionallyissues aResourceWarning if the generator wasn’t exhausted. Thiswarning is hidden by default, but can be enabled for those who wantto make sure they aren’t inadvertently relying on CPython-specificGC semantics.

Async generator objects (including those created by async generatorcomprehensions):

• __aiterclose__ callsself.aclose()
• __del__ issues aRuntimeWarning ifaclose has not beencalled, since this probably indicates a latent bug, similar to the“coroutine never awaited” warning.

QUESTION: should file objects implement__iterclose__ to close thefile? On the one hand this would make this change more disruptive; onthe other hand people really like writingforlineinopen(...):..., and if we get used to iterators taking care of their owncleanup then it might become very weird if files don’t.

New convenience functions

Theoperator module gains two new functions, with semanticsequivalent to the following:

defiterclose(it):ifnotisinstance(it,collections.abc.Iterator):raiseTypeError("not an iterator")ifhasattr(type(it),"__iterclose__"):type(it).__iterclose__(it)asyncdefaiterclose(ait):ifnotisinstance(it,collections.abc.AsyncIterator):raiseTypeError("not an iterator")ifhasattr(type(ait),"__aiterclose__"):awaittype(ait).__aiterclose__(ait)

Theitertools module gains a new iterator wrapper that can be usedto selectively disable the new__iterclose__ behavior:

# QUESTION: I feel like there might be a better name for this one?classpreserve(iterable):def__init__(self,iterable):self._it=iter(iterable)def__iter__(self):returnselfdef__next__(self):returnnext(self._it)def__iterclose__(self):# Swallow __iterclose__ without passing it onpass

Example usage (assuming that file objects implements__iterclose__):

withopen(...)ashandle:# Iterate through the same file twice:forlineinitertools.preserve(handle):...handle.seek(0)forlineinitertools.preserve(handle):...

@contextlib.contextmanagerdefiterclosing(iterable):it=iter(iterable)try:yieldpreserve(it)finally:iterclose(it)

__iterclose__ implementations for iterator wrappers

Python ships a number of iterator types that act as wrappers aroundother iterators:map,zip,itertools.accumulate,csv.reader, and others. These iterators should define a__iterclose__ method which calls__iterclose__ in turn ontheir underlying iterators. For example,map could be implementedas:

# Helper functionmap_chaining_exceptions(fn,items,last_exc=None):foriteminitems:try:fn(item)exceptBaseExceptionasnew_exc:ifnew_exc.__context__isNone:new_exc.__context__=last_exclast_exc=new_exciflast_excisnotNone:raiselast_excclassmap:def__init__(self,fn,*iterables):self._fn=fnself._iters=[iter(iterable)foriterableiniterables]def__iter__(self):returnselfdef__next__(self):returnself._fn(*[next(it)foritinself._iters])def__iterclose__(self):map_chaining_exceptions(operator.iterclose,self._iters)defchain(*iterables):try:whileiterables:forelementiniterables.pop(0):yieldelementexceptBaseExceptionase:defiterclose_iterable(iterable):operations.iterclose(iter(iterable))map_chaining_exceptions(iterclose_iterable,iterables,last_exc=e)

In some cases this requires some subtlety; for example,itertools.teeshould not call__iterclose__ on the underlying iterator until ithas been called onall of the clone iterators.

Example / Rationale

The payoff for all this is that we can now write straightforward codelike:

defread_newline_separated_json(path):forlineinopen(path):yieldjson.loads(line)

and be confident that the file will receive deterministic cleanupwithout the end-user having to take any special effort, even incomplex cases. For example, consider this silly pipeline:

list(map(lambdakey:key.upper(),doc["key"]fordocinread_newline_separated_json(path)))

If our file contains a document wheredoc["key"] turns out to bean integer, then the following sequence of events will happen:

1. key.upper() raises anAttributeError, which propagates outof themap and triggers the implicitfinally block insidelist.
2. Thefinally block inlist calls__iterclose__() on themap object.
3. map.__iterclose__() calls__iterclose__() on the generatorcomprehension object.
4. This injects aGeneratorExit exception into the generatorcomprehension body, which is currently suspended inside thecomprehension’sfor loop body.
5. The exception propagates out of thefor loop, triggering thefor loop’s implicitfinally block, which calls__iterclose__ on the generator object representing the call toread_newline_separated_json.
6. This injects an innerGeneratorExit exception into the body ofread_newline_separated_json, currently suspended at theyield.
7. The innerGeneratorExit propagates out of thefor loop,triggering thefor loop’s implicitfinally block, whichcalls__iterclose__() on the file object.
8. The file object is closed.
9. The innerGeneratorExit resumes propagating, hits the boundaryof the generator function, and causesread_newline_separated_json’s__iterclose__() method toreturn successfully.
10. Control returns to the generator comprehension body, and the outerGeneratorExit continues propagating, allowing thecomprehension’s__iterclose__() to return successfully.
11. The rest of the__iterclose__() calls unwind without incident,back into the body oflist.
12. The originalAttributeError resumes propagating.

(The details above assume that we implementfile.__iterclose__; ifnot then add awith block toread_newline_separated_json andessentially the same logic goes through.)

Of course, from the user’s point of view, this can be simplified downto just:

1.int.upper() raises anAttributeError1. The file object is closed.2. TheAttributeError propagates out oflist

So we’ve accomplished our goal of making this “just work” without theuser having to think about it.