Feature or enhancement

We propose adding “eager” coroutine execution support toasyncio.TaskGroup via a new methodenqueue() [1].

TaskGroup.enqueue() would have the same signature asTaskGroup.create_task() but eagerly perform the first step of the passedcoroutine’s execution immediately. If thecoroutine completes without yielding, the result ofenqueue() would be an object which behaves like a completedasyncio.Task. Otherwise,enqueue() behaves the same asTaskGroup.create_task(), returning a pendingasyncio.Task.

The reason for a new method, rather than changing the implementation ofTaskGroup.create_task() is this new method introduces a small semantic difference. For example in:

async def coro(): ...async with TaskGroup() as tg:  tg.enqueue(coro())  raise Exception

The exception will cancel everthing scheduled intg, but if some or all ofcoro() completes eagerly any side-effects of this will be observable in further execution. Iftg.create_task() is used instead no part ofcoro() will be executed.

Pitch

At Instagram we’ve observed ~70% of coroutine instances passed toasyncio.gather() can run fully synchronously i.e. without performing any I/O which would suspend execution. This typically happens when there is a local cache which can elide actual I/O. We exploit this inCinder with a modifiedasyncio.gather() that eagerly executescoroutine args and skips scheduling aasyncio.Task object to an event loop if no yield occurs. Overall this optimization saved ~4% CPU on our Django webservers.

In a prototype implementation of this proposed feature [2] the overhead when schedulingTaskGroups with all fully-synchronous coroutines was decreased by ~8x. When scheduling a mixture of synchronous and asynchronouscoroutines, performance is improved by ~1.4x, and when nocoroutines can complete synchronously there is still a small improvement.

We anticipate code relying on any semantics which change betweenTaskGroup.create_task() andTaskGroup.enqueue() will be rare. So, as the TaskGroup interface is new in 3.11, we hopeenqueue() and its performance benefits can be promoted as the preferred method for scheduling coroutines in 3.12+.

Previous discussion

This new API was discussed informally at PyCon 2022, with at least some of this being between@gvanrossum,@DinoV,@markshannon, and /or@jbower-fb.

[1] The name "enqueue" came out of a discussion between@gvanrossum and@DinoV.

[2]Prototype implementation (some features missing, e.g. specifying Context), andbenchmark.

Linked PRs

• gh-97696: DRAFT asyncio eager tasks factory prototype #101613
• gh-97696: asyncio eager tasks factory #102853
• gh-97696 Remove unnecessary check for eager_start kwarg #104188
• gh-97696 Add documentation for get_coro() behavior with eager tasks #104189
• gh-97696: Remove redundant #include #104216
• gh-97696: Use PyObject_CallMethodNoArgs and inline is_loop_running check #104255
• gh-97696: Improve and fix documentation for asyncio eager tasks #104256
• gh-97696: Move around and update the whatsnew entry for asyncio eager task factory #104298
• gh-97696 Add documentation for get_coro() behavior with eager tasks #104304