Where Does Async IO Fit In?

Concurrency and parallelism are expansive subjects that are not easy to wade into. While this article focuses on async IO and its implementation in Python, it’s worth taking a minute to compare async IO to its counterparts in order to have context about how async IO fits into the larger, sometimes dizzying puzzle.

Parallelism consists of performing multiple operations at the same time.Multiprocessing is a means to effect parallelism, and it entails spreading tasks over a computer’s central processing units (CPUs, or cores). Multiprocessing is well-suited for CPU-bound tasks: tightly boundfor loops and mathematical computations usually fall into this category.

Concurrency is a slightly broader term than parallelism. It suggests that multiple tasks have the ability to run in an overlapping manner. (There’s a saying that concurrency does not imply parallelism.)

Threading is a concurrent execution model whereby multiplethreads take turns executing tasks. One process can contain multiple threads. Python has a complicated relationship with threading thanks to itsGIL, but that’s beyond the scope of this article.

What’s important to know about threading is that it’s better for IO-bound tasks. While a CPU-bound task is characterized by the computer’s cores continually working hard from start to finish, an IO-bound job is dominated by a lot of waiting on input/output to complete.

To recap the above, concurrency encompasses both multiprocessing (ideal for CPU-bound tasks) and threading (suited for IO-bound tasks). Multiprocessing is a form of parallelism, with parallelism being a specific type (subset) of concurrency. The Python standard library has offered longstandingsupport for both of these through itsmultiprocessing,threading, andconcurrent.futures packages.

Now it’s time to bring a new member to the mix. Over the last few years, a separate design has been more comprehensively built intoCPython: asynchronous IO, enabled through the standard library’sasyncio package and the newasync andawait language keywords. To be clear, async IO is not a newly invented concept, and it has existed or is being built into other languages and runtime environments, such asGo,C#, orScala.

Theasyncio package is billed by the Python documentation asa library to write concurrent code. However, async IO is not threading, nor is it multiprocessing. It is not built on top of either of these.

In fact, async IO is a single-threaded, single-process design: it usescooperative multitasking, a term that you’ll flesh out by the end of this tutorial. It has been said in other words that async IO gives a feeling of concurrency despite using a single thread in a single process. Coroutines (a central feature of async IO) can be scheduled concurrently, but they are not inherently concurrent.

To reiterate, async IO is a style of concurrent programming, but it is not parallelism. It’s more closely aligned with threading than with multiprocessing but is very much distinct from both of these and is a standalone member in concurrency’s bag of tricks.

That leaves one more term. What does it mean for something to beasynchronous? This isn’t a rigorous definition, but for our purposes here, I can think of two properties:

• Asynchronous routines are able to “pause” while waiting on their ultimate result and let other routines run in the meantime.
• Asynchronous code, through the mechanism above, facilitates concurrent execution. To put it differently, asynchronous code gives the look and feel of concurrency.

Here’s a diagram to put it all together. The white terms represent concepts, and the green terms represent ways in which they are implemented or effected:

I’ll stop there on the comparisons between concurrent programming models. This tutorial is focused on the subcomponent that is async IO, how to use it, and theAPIs that have sprung up around it. For a thorough exploration of threading versus multiprocessing versus async IO, pause here and check out Jim Anderson’soverview of concurrency in Python. Jim is way funnier than me and has sat in more meetings than me, to boot.

Async IO Explained

Async IO may at first seem counterintuitive and paradoxical. How does something that facilitates concurrent code use a single thread and a single CPU core? I’ve never been very good at conjuring up examples, so I’d like to paraphrase one from Miguel Grinberg’s 2017PyCon talk, which explains everything quite beautifully:

Chess master Judit Polgár hosts a chess exhibition in which she plays multiple amateur players. She has two ways of conducting the exhibition: synchronously and asynchronously.

Assumptions:

• 24 opponents
• Judit makes each chess move in 5 seconds
• Opponents each take 55 seconds to make a move
• Games average 30 pair-moves (60 moves total)

Synchronous version: Judit plays one game at a time, never two at the same time, until the game is complete. Each game takes(55 + 5) * 30 == 1800 seconds, or 30 minutes. The entire exhibition takes24 * 30 == 720 minutes, or12 hours.

Asynchronous version: Judit moves from table to table, making one move at each table. She leaves the table and lets the opponent make their next move during the wait time. One move on all 24 games takes Judit24 * 5 == 120 seconds, or 2 minutes. The entire exhibition is now cut down to120 * 30 == 3600 seconds, or just1 hour.(Source)

There is only one Judit Polgár, who has only two hands and makes only one move at a time by herself. But playing asynchronously cuts the exhibition time down from 12 hours to one. So, cooperative multitasking is a fancy way of saying that a program’s event loop (more on that later) communicates with multiple tasks to let each take turns running at the optimal time.

Async IO takes long waiting periods in which functions would otherwise be blocking and allows other functions to run during that downtime. (A function that blocks effectively forbids others from running from the time that it starts until the time that it returns.)

Async IO Is Not Easy

I’ve heard it said, “Use async IO when you can; use threading when you must.” The truth is that building durable multithreaded code can be hard and error-prone. Async IO avoids some of the potential speedbumps that you might otherwise encounter with a threaded design.

But that’s not to say that async IO in Python is easy. Be warned: when you venture a bit below the surface level, async programming can be difficult too! Python’s async model is built around concepts such as callbacks, events, transports, protocols, and futures—just the terminology can be intimidating. The fact that its API has been changing continually makes it no easier.

Luckily,asyncio has matured to a point where most of its features are no longer provisional, while its documentation has received a huge overhaul and some quality resources on the subject are starting to emerge as well.

Theasync/await Syntax and Native Coroutines

At the heart of async IO are coroutines. A coroutine is a specialized version of a Python generator function. Let’s start with a baseline definition and then build off of it as you progress here: a coroutine is a function that can suspend its execution before reachingreturn, and it can indirectly pass control to another coroutine for some time.

Later, you’ll dive a lot deeper into how exactly the traditional generator is repurposed into a coroutine. For now, the easiest way to pick up how coroutines work is to start making some.

Let’s take the immersive approach and write some async IO code. This short program is theHello World of async IO but goes a long way towards illustrating its core functionality:

#!/usr/bin/env python3# countasync.pyimportasyncioasyncdefcount():print("One")awaitasyncio.sleep(1)print("Two")asyncdefmain():awaitasyncio.gather(count(),count(),count())if__name__=="__main__":importtimes=time.perf_counter()asyncio.run(main())elapsed=time.perf_counter()-sprint(f"{__file__} executed in{elapsed:0.2f} seconds.")

When you execute this file, take note of what looks different than if you were to define the functions with justdef andtime.sleep():

$python3countasync.pyOneOneOneTwoTwoTwocountasync.py executed in 1.01 seconds.

The order of this output is the heart of async IO. Talking to each of the calls tocount() is a single event loop, or coordinator. When each task reachesawait asyncio.sleep(1), the function yells up to the event loop and gives control back to it, saying, “I’m going to be sleeping for 1 second. Go ahead and let something else meaningful be done in the meantime.”

Contrast this to the synchronous version:

#!/usr/bin/env python3# countsync.pyimporttimedefcount():print("One")time.sleep(1)print("Two")defmain():for_inrange(3):count()if__name__=="__main__":s=time.perf_counter()main()elapsed=time.perf_counter()-sprint(f"{__file__} executed in{elapsed:0.2f} seconds.")

When executed, there is a slight but critical change in order and execution time:

$python3countsync.pyOneTwoOneTwoOneTwocountsync.py executed in 3.01 seconds.

While usingtime.sleep() andasyncio.sleep() may seem banal, they are used as stand-ins for any time-intensive processes that involve wait time. (The most mundane thing you can wait on is asleep() call that does basically nothing.) That is,time.sleep() can represent any time-consuming blocking function call, whileasyncio.sleep() is used to stand in for a non-blocking call (but one that also takes some time to complete).

As you’ll see in the next section, the benefit of awaiting something, includingasyncio.sleep(), is that the surrounding function can temporarily cede control to another function that’s more readily able to do something immediately. In contrast,time.sleep() or any other blocking call is incompatible with asynchronous Python code, because it will stop everything in its tracks for the duration of the sleep time.

The Rules of Async IO

At this point, a more formal definition ofasync,await, and the coroutine functions that they create are in order. This section is a little dense, but getting a hold ofasync/await is instrumental, so come back to this if you need to:

• The syntaxasync def introduces either anative coroutine or anasynchronous generator. The expressionsasync with andasync for are also valid, and you’ll see them later on.

• The keywordawait passes function control back to the event loop. (It suspends the execution of the surrounding coroutine.) If Python encounters anawait f() expression in the scope ofg(), this is howawait tells the event loop, “Suspend execution ofg() until whatever I’m waiting on—the result off()—is returned. In the meantime, go let something else run.”

In code, that second bullet point looks roughly like this:

asyncdefg():# Pause here and come back to g() when f() is readyr=awaitf()returnr

There’s also a strict set of rules around when and how you can and cannot useasync/await. These can be handy whether you are still picking up the syntax or already have exposure to usingasync/await:

• A function that you introduce withasync def is a coroutine. It may useawait,return, oryield, but all of these are optional. Declaringasync def noop(): pass is valid:

  • Usingawait and/orreturn creates a coroutine function. To call a coroutine function, you mustawait it to get its results.

  • It is less common (and only recently legal in Python) to useyield in anasync def block. This creates anasynchronous generator, which you iterate over withasync for. Forget about async generators for the time being and focus on getting down the syntax for coroutine functions, which useawait and/orreturn.

  • Anything defined withasync def may not useyield from, which will raise aSyntaxError.

• Just like it’s aSyntaxError to useyield outside of adef function, it is aSyntaxError to useawait outside of anasync def coroutine. You can only useawait in the body of coroutines.

Here are some terse examples meant to summarize the above few rules:

asyncdeff(x):y=awaitz(x)# OK - `await` and `return` allowed in coroutinesreturnyasyncdefg(x):yieldx# OK - this is an async generatorasyncdefm(x):yield fromgen(x)# No - SyntaxErrordefm(x):y=awaitz(x)# Still no - SyntaxError (no `async def` here)returny

Finally, when you useawait f(), it’s required thatf() be an object that isawaitable. Well, that’s not very helpful, is it? For now, just know that an awaitable object is either (1) another coroutine or (2) an object defining an.__await__() dunder method that returns an iterator. If you’re writing a program, for the large majority of purposes, you should only need to worry about case #1.

That brings us to one more technical distinction that you may see pop up: an older way of marking a function as a coroutine is to decorate a normaldef function with@asyncio.coroutine. The result is agenerator-based coroutine. This construction has been outdated since theasync/await syntax was put in place in Python 3.5.

These two coroutines are essentially equivalent (both are awaitable), but the first isgenerator-based, while the second is anative coroutine:

importasyncio@asyncio.coroutinedefpy34_coro():"""Generator-based coroutine, older syntax"""yield fromstuff()asyncdefpy35_coro():"""Native coroutine, modern syntax"""awaitstuff()

If you’re writing any code yourself, prefer native coroutines for the sake of being explicit rather than implicit. Generator-based coroutines will beremoved in Python 3.10.

Towards the latter half of this tutorial, we’ll touch on generator-based coroutines for explanation’s sake only. The reason thatasync/await were introduced is to make coroutines a standalone feature of Python that can be easily differentiated from a normal generator function, thus reducing ambiguity.

Don’t get bogged down in generator-based coroutines, which have beendeliberately outdated byasync/await. They have their own small set of rules (for instance,await cannot be used in a generator-based coroutine) that are largely irrelevant if you stick to theasync/await syntax.

Without further ado, let’s take on a few more involved examples.

Here’s one example of how async IO cuts down on wait time: given a coroutinemakerandom() that keeps producing random integers in the range [0, 10], until one of them exceeds a threshold, you want to let multiple calls of this coroutine not need to wait for each other to complete in succession. You can largely follow the patterns from the two scripts above, with slight changes:

#!/usr/bin/env python3# rand.pyimportasyncioimportrandom# ANSI colorsc=("\033[0m",# End of color"\033[36m",# Cyan"\033[91m",# Red"\033[35m",# Magenta)asyncdefmakerandom(idx:int,threshold:int=6)->int:print(c[idx+1]+f"Initiated makerandom({idx}).")i=random.randint(0,10)whilei<=threshold:print(c[idx+1]+f"makerandom({idx}) =={i} too low; retrying.")awaitasyncio.sleep(idx+1)i=random.randint(0,10)print(c[idx+1]+f"---> Finished: makerandom({idx}) =={i}"+c[0])returniasyncdefmain():res=awaitasyncio.gather(*(makerandom(i,10-i-1)foriinrange(3)))returnresif__name__=="__main__":random.seed(444)r1,r2,r3=asyncio.run(main())print()print(f"r1:{r1}, r2:{r2}, r3:{r3}")

The colorized output says a lot more than I can and gives you a sense for how this script is carried out:

This program uses one main coroutine,makerandom(), and runs it concurrently across 3 different inputs. Most programs will contain small, modular coroutines and one wrapper function that serves to chain each of the smaller coroutines together.main() is then used to gather tasks (futures) by mapping the central coroutine across some iterable or pool.

In this miniature example, the pool isrange(3). In a fuller example presented later, it is a set of URLs that need to be requested, parsed, and processed concurrently, andmain() encapsulates that entire routine for each URL.

While “making random integers” (which is CPU-bound more than anything) is maybe not the greatest choice as a candidate forasyncio, it’s the presence ofasyncio.sleep() in the example that is designed to mimic an IO-bound process where there is uncertain wait time involved. For example, theasyncio.sleep() call might represent sending and receiving not-so-random integers between two clients in a message application.

Chaining Coroutines

A key feature of coroutines is that they can be chained together. (Remember, a coroutine object is awaitable, so another coroutine canawait it.) This allows you to break programs into smaller, manageable, recyclable coroutines:

#!/usr/bin/env python3# chained.pyimportasyncioimportrandomimporttimeasyncdefpart1(n:int)->str:i=random.randint(0,10)print(f"part1({n}) sleeping for{i} seconds.")awaitasyncio.sleep(i)result=f"result{n}-1"print(f"Returning part1({n}) =={result}.")returnresultasyncdefpart2(n:int,arg:str)->str:i=random.randint(0,10)print(f"part2{n,arg} sleeping for{i} seconds.")awaitasyncio.sleep(i)result=f"result{n}-2 derived from{arg}"print(f"Returning part2{n,arg} =={result}.")returnresultasyncdefchain(n:int)->None:start=time.perf_counter()p1=awaitpart1(n)p2=awaitpart2(n,p1)end=time.perf_counter()-startprint(f"-->Chained result{n} =>{p2} (took{end:0.2f} seconds).")asyncdefmain(*args):awaitasyncio.gather(*(chain(n)forninargs))if__name__=="__main__":importsysrandom.seed(444)args=[1,2,3]iflen(sys.argv)==1elsemap(int,sys.argv[1:])start=time.perf_counter()asyncio.run(main(*args))end=time.perf_counter()-startprint(f"Program finished in{end:0.2f} seconds.")

Pay careful attention to the output, wherepart1() sleeps for a variable amount of time, andpart2() begins working with the results as they become available:

$python3chained.py963part1(9) sleeping for 4 seconds.part1(6) sleeping for 4 seconds.part1(3) sleeping for 0 seconds.Returning part1(3) == result3-1.part2(3, 'result3-1') sleeping for 4 seconds.Returning part1(9) == result9-1.part2(9, 'result9-1') sleeping for 7 seconds.Returning part1(6) == result6-1.part2(6, 'result6-1') sleeping for 4 seconds.Returning part2(3, 'result3-1') == result3-2 derived from result3-1.-->Chained result3 => result3-2 derived from result3-1 (took 4.00 seconds).Returning part2(6, 'result6-1') == result6-2 derived from result6-1.-->Chained result6 => result6-2 derived from result6-1 (took 8.01 seconds).Returning part2(9, 'result9-1') == result9-2 derived from result9-1.-->Chained result9 => result9-2 derived from result9-1 (took 11.01 seconds).Program finished in 11.01 seconds.

In this setup, the runtime ofmain() will be equal to the maximum runtime of the tasks that it gathers together and schedules.

Using a Queue

Theasyncio package providesqueue classes that are designed to be similar to classes of thequeue module. In our examples so far, we haven’t really had a need for a queue structure. Inchained.py, each task (future) is composed of a set of coroutines that explicitly await each other and pass through a single input per chain.

There is an alternative structure that can also work with async IO: a number of producers, which are not associated with each other, add items to a queue. Each producer may add multiple items to the queue at staggered, random, unannounced times. A group of consumers pull items from the queue as they show up, greedily and without waiting for any other signal.

In this design, there is no chaining of any individual consumer to a producer. The consumers don’t know the number of producers, or even the cumulative number of items that will be added to the queue, in advance.

It takes an individual producer or consumer a variable amount of time to put and extract items from the queue, respectively. The queue serves as a throughput that can communicate with the producers and consumers without them talking to each other directly.

The synchronous version of this program would look pretty dismal: a group of blocking producers serially add items to the queue, one producer at a time. Only after all producers are done can the queue be processed, by one consumer at a time processing item-by-item. There is a ton of latency in this design. Items may sit idly in the queue rather than be picked up and processed immediately.

An asynchronous version,asyncq.py, is below. The challenging part of this workflow is that there needs to be a signal to the consumers that production is done. Otherwise,await q.get() will hang indefinitely, because the queue will have been fully processed, but consumers won’t have any idea that production is complete.

(Big thanks for some help from a StackOverflowuser for helping to straighten outmain(): the key is toawait q.join(), which blocks until all items in the queue have been received and processed, and then to cancel the consumer tasks, which would otherwise hang up and wait endlessly for additional queue items to appear.)

Here is the full script:

#!/usr/bin/env python3# asyncq.pyimportasyncioimportitertoolsasitimportosimportrandomimporttimeasyncdefmakeitem(size:int=5)->str:returnos.urandom(size).hex()asyncdefrandsleep(caller=None)->None:i=random.randint(0,10)ifcaller:print(f"{caller} sleeping for{i} seconds.")awaitasyncio.sleep(i)asyncdefproduce(name:int,q:asyncio.Queue)->None:n=random.randint(0,10)for_init.repeat(None,n):# Synchronous loop for each single producerawaitrandsleep(caller=f"Producer{name}")i=awaitmakeitem()t=time.perf_counter()awaitq.put((i,t))print(f"Producer{name} added <{i}> to queue.")asyncdefconsume(name:int,q:asyncio.Queue)->None:whileTrue:awaitrandsleep(caller=f"Consumer{name}")i,t=awaitq.get()now=time.perf_counter()print(f"Consumer{name} got element <{i}>"f" in{now-t:0.5f} seconds.")q.task_done()asyncdefmain(nprod:int,ncon:int):q=asyncio.Queue()producers=[asyncio.create_task(produce(n,q))forninrange(nprod)]consumers=[asyncio.create_task(consume(n,q))forninrange(ncon)]awaitasyncio.gather(*producers)awaitq.join()# Implicitly awaits consumers, tooforcinconsumers:c.cancel()if__name__=="__main__":importargparserandom.seed(444)parser=argparse.ArgumentParser()parser.add_argument("-p","--nprod",type=int,default=5)parser.add_argument("-c","--ncon",type=int,default=10)ns=parser.parse_args()start=time.perf_counter()asyncio.run(main(**ns.__dict__))elapsed=time.perf_counter()-startprint(f"Program completed in{elapsed:0.5f} seconds.")

The first few coroutines are helper functions that return a random string, a fractional-second performance counter, and a random integer. A producer puts anywhere from 1 to 5 items into the queue. Each item is a tuple of(i, t) wherei is a random string andt is the time at which the producer attempts to put the tuple into the queue.

When a consumer pulls an item out, it simply calculates the elapsed time that the item sat in the queue using the timestamp that the item was put in with.

Keep in mind thatasyncio.sleep() is used to mimic some other, more complex coroutine that would eat up time and block all other execution if it were a regular blocking function.

Here is a test run with two producers and five consumers:

$python3asyncq.py-p2-c5Producer 0 sleeping for 3 seconds.Producer 1 sleeping for 3 seconds.Consumer 0 sleeping for 4 seconds.Consumer 1 sleeping for 3 seconds.Consumer 2 sleeping for 3 seconds.Consumer 3 sleeping for 5 seconds.Consumer 4 sleeping for 4 seconds.Producer 0 added <377b1e8f82> to queue.Producer 0 sleeping for 5 seconds.Producer 1 added <413b8802f8> to queue.Consumer 1 got element <377b1e8f82> in 0.00013 seconds.Consumer 1 sleeping for 3 seconds.Consumer 2 got element <413b8802f8> in 0.00009 seconds.Consumer 2 sleeping for 4 seconds.Producer 0 added <06c055b3ab> to queue.Producer 0 sleeping for 1 seconds.Consumer 0 got element <06c055b3ab> in 0.00021 seconds.Consumer 0 sleeping for 4 seconds.Producer 0 added <17a8613276> to queue.Consumer 4 got element <17a8613276> in 0.00022 seconds.Consumer 4 sleeping for 5 seconds.Program completed in 9.00954 seconds.

In this case, the items process in fractions of a second. A delay can be due to two reasons:

• Standard, largely unavoidable overhead
• Situations where all consumers are sleeping when an item appears in the queue

With regards to the second reason, luckily, it is perfectly normal to scale to hundreds or thousands of consumers. You should have no problem withpython3 asyncq.py -p 5 -c 100. The point here is that, theoretically, you could have different users on different systems controlling the management of producers and consumers, with the queue serving as the central throughput.

So far, you’ve been thrown right into the fire and seen three related examples ofasyncio calling coroutines defined withasync andawait. If you’re not completely following or just want to get deeper into the mechanics of how modern coroutines came to be in Python, you’ll start from square one with the next section.

Other Features:async for and Async Generators + Comprehensions

Along with plainasync/await, Python also enablesasync for to iterate over anasynchronous iterator. The purpose of an asynchronous iterator is for it to be able to call asynchronous code at each stage when it is iterated over.

A natural extension of this concept is anasynchronous generator. Recall that you can useawait,return, oryield in a native coroutine. Usingyield within a coroutine became possible in Python 3.6 (via PEP 525), which introduced asynchronous generators with the purpose of allowingawait andyield to be used in the same coroutine function body:

>>>asyncdefmygen(u:int=10):..."""Yield powers of 2."""...i=0...whilei<u:...yield2**i...i+=1...awaitasyncio.sleep(0.1)

Last but not least, Python enablesasynchronous comprehension withasync for. Like its synchronous cousin, this is largely syntactic sugar:

>>>asyncdefmain():...# This does *not* introduce concurrent execution...# It is meant to show syntax only...g=[iasyncforiinmygen()]...f=[jasyncforjinmygen()ifnot(j//3%5)]...returng,f...>>>g,f=asyncio.run(main())>>>g[1, 2, 4, 8, 16, 32, 64, 128, 256, 512]>>>f[1, 2, 16, 32, 256, 512]

This is a crucial distinction:neither asynchronous generators nor comprehensions make the iteration concurrent. All that they do is provide the look-and-feel of their synchronous counterparts, but with the ability for the loop in question to give up control to the event loop for some other coroutine to run.

In other words, asynchronous iterators and asynchronous generators are not designed to concurrently map some function over a sequence or iterator. They’re merely designed to let the enclosing coroutine allow other tasks to take their turn. Theasync for andasync with statements are only needed to the extent that using plainfor orwith would “break” the nature ofawait in the coroutine. This distinction between asynchronicity and concurrency is a key one to grasp.

The Event Loop andasyncio.run()

You can think of an event loop as something like awhile True loop that monitors coroutines, taking feedback on what’s idle, and looking around for things that can be executed in the meantime. It is able to wake up an idle coroutine when whatever that coroutine is waiting on becomes available.

Thus far, the entire management of the event loop has been implicitly handled by one function call:

asyncio.run(main())# Python 3.7+

asyncio.run(), introduced in Python 3.7, is responsible for getting the event loop, running tasks until they are marked as complete, and then closing the event loop.

There’s a more long-winded way of managing theasyncio event loop, withget_event_loop(). The typical pattern looks like this:

loop=asyncio.get_event_loop()try:loop.run_until_complete(main())finally:loop.close()

You’ll probably seeloop.get_event_loop() floating around in older examples, but unless you have a specific need to fine-tune control over the event loop management,asyncio.run() should be sufficient for most programs.

If you do need to interact with the event loop within a Python program,loop is a good-old-fashioned Python object that supports introspection withloop.is_running() andloop.is_closed(). You can manipulate it if you need to get more fine-tuned control, such as inscheduling a callback by passing the loop as an argument.

What is more crucial is understanding a bit beneath the surface about the mechanics of the event loop. Here are a few points worth stressing about the event loop.

#1: Coroutines don’t do much on their own until they are tied to the event loop.

You saw this point before in the explanation on generators, but it’s worth restating. If you have a main coroutine that awaits others, simply calling it in isolation has little effect:

>>>importasyncio>>>asyncdefmain():...print("Hello ...")...awaitasyncio.sleep(1)...print("World!")>>>routine=main()>>>routine<coroutine object main at 0x1027a6150>

Remember to useasyncio.run() to actually force execution by scheduling themain() coroutine (future object) for execution on the event loop:

>>>asyncio.run(routine)Hello ...World!

(Other coroutines can be executed withawait. It is typical to wrap justmain() inasyncio.run(), and chained coroutines withawait will be called from there.)

#2: By default, an async IO event loop runs in a single thread and on a single CPU core. Usually, running one single-threaded event loop in one CPU core is more than sufficient. It is also possible to run event loops across multiple cores. Check out thistalk by John Reese for more, and be warned that your laptop may spontaneously combust.

#3. Event loops are pluggable. That is, you could, if you really wanted, write your own event loop implementation and have it run tasks just the same. This is wonderfully demonstrated in theuvloop package, which is an implementation of the event loop in Cython.

That is what is meant by the term “pluggable event loop”: you can use any working implementation of an event loop, unrelated to the structure of the coroutines themselves. Theasyncio package itself ships withtwo different event loop implementations, with the default being based on theselectors module. (The second implementation is built for Windows only.)

When and Why Is Async IO the Right Choice?

This tutorial is no place for an extended treatise on async IO versus threading versus multiprocessing. However, it’s useful to have an idea of when async IO is probably the best candidate of the three.

The battle over async IO versus multiprocessing is not really a battle at all. In fact, they can beused in concert. If you have multiple, fairly uniform CPU-bound tasks (a great example is agrid search in libraries such asscikit-learn orkeras), multiprocessing should be an obvious choice.

Simply puttingasync before every function is a bad idea if all of the functions use blocking calls. (This can actually slow down your code.) But as mentioned previously, there are places where async IO and multiprocessing canlive in harmony.

The contest between async IO and threading is a little bit more direct. I mentioned in the introduction that “threading is hard.” The full story is that, even in cases where threading seems easy to implement, it can still lead to infamous impossible-to-trace bugs due to race conditions and memory usage, among other things.

Threading also tends to scale less elegantly than async IO, because threads are a system resource with a finite availability. Creating thousands of threads will fail on many machines, and I don’t recommend trying it in the first place. Creating thousands of async IO tasks is completely feasible.

Async IO shines when you have multiple IO-bound tasks where the tasks would otherwise be dominated by blocking IO-bound wait time, such as:

• Network IO, whether your program is the server or the client side

• Serverless designs, such as a peer-to-peer, multi-user network like a group chatroom

• Read/write operations where you want to mimic a “fire-and-forget” style but worry less about holding a lock on whatever you’re reading and writing to

The biggest reason not to use it is thatawait only supports a specific set of objects that define a specific set of methods. If you want to do async read operations with a certain DBMS, you’ll need to find not just a Python wrapper for that DBMS, but one that supports theasync/await syntax. Coroutines that contain synchronous calls block other coroutines and tasks from running.

For a shortlist of libraries that work withasync/await, see thelist at the end of this tutorial.

Async IO It Is, but Which One?

This tutorial focuses on async IO, theasync/await syntax, and usingasyncio for event-loop management and specifying tasks.asyncio certainly isn’t the only async IO library out there. This observation from Nathaniel J. Smith says a lot:

[In] a few years,asyncio might find itself relegated to becoming one of those stdlib libraries that savvy developers avoid, likeurllib2.

…

What I’m arguing, in effect, is thatasyncio is a victim of its own success: when it was designed, it used the best approach possible; but since then, work inspired byasyncio – like the addition ofasync/await – has shifted the landscape so that we can do even better, and nowasyncio is hamstrung by its earlier commitments.(Source)

To that end, a few big-name alternatives that do whatasyncio does, albeit with different APIs and different approaches, arecurio andtrio. Personally, I think that if you’re building a moderately sized, straightforward program, just usingasyncio is plenty sufficient and understandable, and lets you avoid adding yet another large dependency outside of Python’s standard library.

But by all means, check outcurio andtrio, and you might find that they get the same thing done in a way that’s more intuitive for you as the user. Many of the package-agnostic concepts presented here should permeate to alternative async IO packages as well.

Other Top-Levelasyncio Functions

In addition toasyncio.run(), you’ve seen a few other package-level functions such asasyncio.create_task() andasyncio.gather().

You can usecreate_task() to schedule the execution of a coroutine object, followed byasyncio.run():

>>>importasyncio>>>asyncdefcoro(seq)->list:..."""'IO' wait time is proportional to the max element."""...awaitasyncio.sleep(max(seq))...returnlist(reversed(seq))...>>>asyncdefmain():...# This is a bit redundant in the case of one task...# We could use `await coro([3, 2, 1])` on its own...t=asyncio.create_task(coro([3,2,1]))# Python 3.7+...awaitt...print(f't: type{type(t)}')...print(f't done:{t.done()}')...>>>t=asyncio.run(main())t: type <class '_asyncio.Task'>t done: True

There’s a subtlety to this pattern: if you don’tawait t withinmain(), it may finish beforemain() itself signals that it is complete. Becauseasyncio.run(main())callsloop.run_until_complete(main()), the event loop is only concerned (withoutawait t present) thatmain() is done, not that the tasks that get created withinmain() are done. Withoutawait t, the loop’s other taskswill be cancelled, possibly before they are completed. If you need to get a list of currently pending tasks, you can useasyncio.Task.all_tasks().

Separately, there’sasyncio.gather(). While it doesn’t do anything tremendously special,gather() is meant to neatly put a collection of coroutines (futures) into a single future. As a result, it returns a single future object, and, if youawait asyncio.gather() and specify multiple tasks or coroutines, you’re waiting for all of them to be completed. (This somewhat parallelsqueue.join() from our earlier example.) The result ofgather() will be a list of the results across the inputs:

>>>importtime>>>asyncdefmain():...t=asyncio.create_task(coro([3,2,1]))...t2=asyncio.create_task(coro([10,5,0]))# Python 3.7+...print('Start:',time.strftime('%X'))...a=awaitasyncio.gather(t,t2)...print('End:',time.strftime('%X'))# Should be 10 seconds...print(f'Both tasks done:{all((t.done(),t2.done()))}')...returna...>>>a=asyncio.run(main())Start: 16:20:11End: 16:20:21Both tasks done: True>>>a[[1, 2, 3], [0, 5, 10]]

You probably noticed thatgather() waits on the entire result set of the Futures or coroutines that you pass it. Alternatively, you can loop overasyncio.as_completed() to get tasks as they are completed, in the order of completion. The function returns an iterator that yields tasks as they finish. Below, the result ofcoro([3, 2, 1]) will be available beforecoro([10, 5, 0]) is complete, which is not the case withgather():

>>>asyncdefmain():...t=asyncio.create_task(coro([3,2,1]))...t2=asyncio.create_task(coro([10,5,0]))...print('Start:',time.strftime('%X'))...forresinasyncio.as_completed((t,t2)):...compl=awaitres...print(f'res:{compl} completed at{time.strftime("%X")}')...print('End:',time.strftime('%X'))...print(f'Both tasks done:{all((t.done(),t2.done()))}')...>>>a=asyncio.run(main())Start: 09:49:07res: [1, 2, 3] completed at 09:49:10res: [0, 5, 10] completed at 09:49:17End: 09:49:17Both tasks done: True

Lastly, you may also seeasyncio.ensure_future(). You should rarely need it, because it’s a lower-level plumbing API and largely replaced bycreate_task(), which was introduced later.

The Precedence ofawait

While they behave somewhat similarly, theawait keyword has significantly higher precedence thanyield. This means that, because it is more tightly bound, there are a number of instances where you’d need parentheses in ayield from statement that are not required in an analogousawait statement. For more information, seeexamples ofawait expressions from PEP 492.

Python Version Specifics

Async IO in Python has evolved swiftly, and it can be hard to keep track of what came when. Here’s a list of Python minor-version changes and introductions related toasyncio:

• 3.3: Theyield from expression allows for generator delegation.

• 3.4:asyncio was introduced in the Python standard library with provisional API status.

• 3.5:async andawait became a part of the Python grammar, used to signify and wait on coroutines. They were not yet reserved keywords. (You could still define functions or variables namedasync andawait.)

• 3.6: Asynchronous generators and asynchronous comprehensions were introduced. The API ofasyncio was declared stable rather than provisional.

• 3.7:async andawait became reserved keywords. (They cannot be used as identifiers.) They are intended to replace theasyncio.coroutine() decorator.asyncio.run() was introduced to theasyncio package, amonga bunch of other features.

If you want to be safe (and be able to useasyncio.run()), go with Python 3.7 or above to get the full set of features.

Articles

Here’s a curated list of additional resources:

• Real Python:Speed up your Python Program with Concurrency
• Real Python:What is the Python Global Interpreter Lock?
• CPython: Theasyncio packagesource
• Python docs:Data model > Coroutines
• TalkPython:Async Techniques and Examples in Python
• Brett Cannon:How the Heck Does Async-Await Work in Python 3.5?
• PYMOTW:asyncio
• A. Jesse Jiryu Davis and Guido van Rossum:A Web Crawler With asyncio Coroutines
• Andy Pearce:The State of Python Coroutines:yield from
• Nathaniel J. Smith:Some Thoughts on Asynchronous API Design in a Post-async/await World
• Armin Ronacher:I don’t understand Python’s Asyncio
• Andy Balaam:series onasyncio (4 posts)
• Stack Overflow:Pythonasyncio.semaphore inasync-await function
• Yeray Diaz:
  • AsyncIO for the Working Python Developer
  • Asyncio Coroutine Patterns: Beyondawait

A few PythonWhat’s New sections explain the motivation behind language changes in more detail:

• What’s New in Python 3.3 (yield from and PEP 380)
• What’s New in Python 3.6 (PEP 525 & 530)

From David Beazley:

• Generator: Tricks for Systems Programmers
• A Curious Course on Coroutines and Concurrency
• Generators: The Final Frontier

YouTube talks:

• John Reese - Thinking Outside the GIL with AsyncIO and Multiprocessing - PyCon 2018
• Keynote David Beazley - Topics of Interest (Python Asyncio)
• David Beazley - Python Concurrency From the Ground Up: LIVE! - PyCon 2015
• Raymond Hettinger, Keynote on Concurrency, PyBay 2017
• Thinking about Concurrency, Raymond Hettinger, Python core developer
• Miguel Grinberg Asynchronous Python for the Complete Beginner PyCon 2017
• Yury Selivanov asyncawait and asyncio in Python 3 6 and beyond PyCon 2017
• Fear and Awaiting in Async: A Savage Journey to the Heart of the Coroutine Dream
• What Is Async, How Does It Work, and When Should I Use It? (PyCon APAC 2014)

Related PEPs

Libraries That Work Withasync/await

Fromaio-libs:

• aiohttp: Asynchronous HTTP client/server framework
• aioredis: Async IO Redis support
• aiopg: Async IO PostgreSQL support
• aiomcache: Async IO memcached client
• aiokafka: Async IO Kafka client
• aiozmq: Async IO ZeroMQ support
• aiojobs: Jobs scheduler for managing background tasks
• async_lru: SimpleLRU cache for async IO

Frommagicstack:

• uvloop: Ultra fast async IO event loop
• asyncpg: (Also very fast) async IO PostgreSQL support

From other hosts:

• trio: Friendlierasyncio intended to showcase a radically simpler design
• aiofiles: Async file IO
• asks: Async requests-like http library
• asyncio-redis: Async IO Redis support
• aioprocessing: Integratesmultiprocessing module withasyncio
• umongo: Async IO MongoDB client
• unsync: Unsynchronizeasyncio
• aiostream: Likeitertools, but async

Take the Quiz: Test your knowledge with our interactive “Async IO in Python: A Complete Walkthrough” quiz. You’ll receive a score upon completion to help you track your learning progress:

---

Interactive Quiz

Async IO in Python: A Complete Walkthrough

In this quiz, you'll test your understanding of async IO in Python. With this knowledge, you'll be able to understand the language-agnostic paradigm of asynchronous IO, use the async/await keywords to define coroutines, and use the asyncio package to run and manage coroutines.