18.5.3.1.Coroutines¶

Coroutines used withasyncio may be implemented using theasyncdef statement, or by usinggenerators.Theasyncdef type of coroutine was added in Python 3.5, andis recommended if there is no need to support older Python versions.

Generator-based coroutines should be decorated with@asyncio.coroutine, although this is not strictly enforced.The decorator enables compatibility withasyncdef coroutines,and also serves as documentation. Generator-basedcoroutines use theyieldfrom syntax introduced inPEP 380,instead of the originalyield syntax.

The word “coroutine”, like the word “generator”, is used for twodifferent (though related) concepts:

• The function that defines a coroutine(a function definition usingasyncdef ordecorated with@asyncio.coroutine). If disambiguation is neededwe will call this acoroutine function (iscoroutinefunction()returnsTrue).

• The object obtained by calling a coroutine function. This objectrepresents a computation or an I/O operation (usually a combination)that will complete eventually. If disambiguation is needed we willcall it acoroutine object (iscoroutine() returnsTrue).

Things a coroutine can do:

• result=awaitfuture orresult=yieldfromfuture –suspends the coroutine until thefuture is done, then returns the future’s result, or raises anexception, which will be propagated. (If the future is cancelled,it will raise aCancelledError exception.) Note that tasks arefutures, and everything said about futures also applies to tasks.

• result=awaitcoroutine orresult=yieldfromcoroutine –wait for another coroutine toproduce a result (or raise an exception, which will be propagated).Thecoroutine expression must be acall to another coroutine.

• returnexpression – produce a result to the coroutine that iswaiting for this one usingawait oryieldfrom.

• raiseexception – raise an exception in the coroutine that iswaiting for this one usingawait oryieldfrom.

Calling a coroutine does not start its code running –the coroutine object returned by the call doesn’t do anything until youschedule its execution. There are two basic ways to start it running:callawaitcoroutine oryieldfromcoroutine from another coroutine(assuming the other coroutine is already running!), or schedule its executionusing theensure_future() function or theAbstractEventLoop.create_task()method.

Coroutines (and tasks) can only run when the event loop is running.

@asyncio.coroutine¶

Decorator to mark generator-based coroutines. This enablesthe generator useyieldfrom to callasyncdef coroutines, and also enables the generator to be called byasyncdef coroutines, for instance using anawait expression.

There is no need to decorateasyncdef coroutines themselves.

If the generator is not yielded from before it is destroyed, an errormessage is logged. SeeDetect coroutines never scheduled.

importasyncioasyncdefhello_world():print("Hello World!")loop=asyncio.get_event_loop()# Blocking call which returns when the hello_world() coroutine is doneloop.run_until_complete(hello_world())loop.close()

importasyncioimportdatetimeasyncdefdisplay_date(loop):end_time=loop.time()+5.0whileTrue:print(datetime.datetime.now())if(loop.time()+1.0)>=end_time:breakawaitasyncio.sleep(1)loop=asyncio.get_event_loop()# Blocking call which returns when the display_date() coroutine is doneloop.run_until_complete(display_date(loop))loop.close()

18.5.3.1.3.Example: Chain coroutines¶

Example chaining coroutines:

importasyncioasyncdefcompute(x,y):print("Compute%s +%s ..."%(x,y))awaitasyncio.sleep(1.0)returnx+yasyncdefprint_sum(x,y):result=awaitcompute(x,y)print("%s +%s =%s"%(x,y,result))loop=asyncio.get_event_loop()loop.run_until_complete(print_sum(1,2))loop.close()

compute() is chained toprint_sum():print_sum() coroutine waitsuntilcompute() is completed before returning its result.

Sequence diagram of the example:

The “Task” is created by theAbstractEventLoop.run_until_complete() methodwhen it gets a coroutine object instead of a task.

The diagram shows the control flow, it does not describe exactly how thingswork internally. For example, the sleep coroutine creates an internal futurewhich usesAbstractEventLoop.call_later() to wake up the task in 1 second.

18.5.3.2.InvalidStateError¶

exceptionasyncio.InvalidStateError¶

The operation is not allowed in this state.

18.5.3.3.TimeoutError¶

exceptionasyncio.TimeoutError¶

The operation exceeded the given deadline.

18.5.3.4.Future¶

classasyncio.Future(*,loop=None)¶

This class isalmost compatible withconcurrent.futures.Future.

Differences:

• result() andexception() do not take a timeout argument andraise an exception when the future isn’t done yet.

• Callbacks registered withadd_done_callback() are always calledvia the event loop’scall_soon().

• This class is not compatible with thewait() andas_completed() functions in theconcurrent.futures package.

This class isnot thread safe.

cancel()¶

Cancel the future and schedule callbacks.

If the future is already done or cancelled, returnFalse. Otherwise,change the future’s state to cancelled, schedule the callbacks and returnTrue.

cancelled()¶

ReturnTrue if the future was cancelled.

done()¶

ReturnTrue if the future is done.

Done means either that a result / exception are available, or that thefuture was cancelled.

result()¶

Return the result this future represents.

If the future has been cancelled, raisesCancelledError. If thefuture’s result isn’t yet available, raisesInvalidStateError. Ifthe future is done and has an exception set, this exception is raised.

exception()¶

Return the exception that was set on this future.

The exception (orNone if no exception was set) is returned only ifthe future is done. If the future has been cancelled, raisesCancelledError. If the future isn’t done yet, raisesInvalidStateError.

add_done_callback(fn)¶

Add a callback to be run when the future becomes done.

The callback is called with a single argument - the future object. If thefuture is already done when this is called, the callback is scheduledwithcall_soon().

Use functools.partial to pass parameters to the callback. For example,fut.add_done_callback(functools.partial(print,"Future:",flush=True)) will callprint("Future:",fut,flush=True).

remove_done_callback(fn)¶

Remove all instances of a callback from the “call when done” list.

Returns the number of callbacks removed.

set_result(result)¶

Mark the future done and set its result.

If the future is already done when this method is called, raisesInvalidStateError.

set_exception(exception)¶

Mark the future done and set an exception.

If the future is already done when this method is called, raisesInvalidStateError.

importasyncioasyncdefslow_operation(future):awaitasyncio.sleep(1)future.set_result('Future is done!')loop=asyncio.get_event_loop()future=asyncio.Future()asyncio.ensure_future(slow_operation(future))loop.run_until_complete(future)print(future.result())loop.close()

importasyncioasyncdefslow_operation(future):awaitasyncio.sleep(1)future.set_result('Future is done!')defgot_result(future):print(future.result())loop.stop()loop=asyncio.get_event_loop()future=asyncio.Future()asyncio.ensure_future(slow_operation(future))future.add_done_callback(got_result)try:loop.run_forever()finally:loop.close()

18.5.3.5.Task¶

classasyncio.Task(coro,*,loop=None)¶

Schedule the execution of acoroutine: wrap it in afuture. A task is a subclass ofFuture.

A task is responsible for executing a coroutine object in an event loop. Ifthe wrapped coroutine yields from a future, the task suspends the executionof the wrapped coroutine and waits for the completion of the future. Whenthe future is done, the execution of the wrapped coroutine restarts with theresult or the exception of the future.

Event loops use cooperative scheduling: an event loop only runs one task ata time. Other tasks may run in parallel if other event loops arerunning in different threads. While a task waits for the completion of afuture, the event loop executes a new task.

The cancellation of a task is different from the cancelation of afuture. Callingcancel() will throw aCancelledError to the wrappedcoroutine.cancelled() only returnsTrue if thewrapped coroutine did not catch theCancelledError exception, or raised aCancelledError exception.

If a pending task is destroyed, the execution of its wrappedcoroutine did not complete. It is probably a bug and a warning islogged: seePending task destroyed.

Don’t directly createTask instances: use theensure_future()function or theAbstractEventLoop.create_task() method.

This class isnot thread safe.

classmethodall_tasks(loop=None)¶

Return a set of all tasks for an event loop.

By default all tasks for the current event loop are returned.

classmethodcurrent_task(loop=None)¶

Return the currently running task in an event loop orNone.

By default the current task for the current event loop is returned.

None is returned when called not in the context of aTask.

cancel()¶

Request that this task cancel itself.

This arranges for aCancelledError to bethrown into the wrapped coroutine on the next cycle through the eventloop. The coroutine then has a chance to clean up or even deny therequest using try/except/finally.

UnlikeFuture.cancel(), this does not guarantee that the taskwill be cancelled: the exception might be caught and acted upon, delayingcancellation of the task or preventing cancellation completely. The taskmay also return a value or raise a different exception.

Immediately after this method is called,cancelled() willnot returnTrue (unless the task was already cancelled). A task willbe marked as cancelled when the wrapped coroutine terminates with aCancelledError exception (even ifcancel() was not called).

get_stack(*,limit=None)¶

Return the list of stack frames for this task’s coroutine.

If the coroutine is not done, this returns the stack where it issuspended. If the coroutine has completed successfully or wascancelled, this returns an empty list. If the coroutine wasterminated by an exception, this returns the list of tracebackframes.

The frames are always ordered from oldest to newest.

The optional limit gives the maximum number of frames to return; bydefault all available frames are returned. Its meaning differs dependingon whether a stack or a traceback is returned: the newest frames of astack are returned, but the oldest frames of a traceback are returned.(This matches the behavior of the traceback module.)

For reasons beyond our control, only one stack frame is returned for asuspended coroutine.

print_stack(*,limit=None,file=None)¶

Print the stack or traceback for this task’s coroutine.

This produces output similar to that of the traceback module, for theframes retrieved by get_stack(). The limit argument is passed toget_stack(). The file argument is an I/O stream to which the outputis written; by default output is written to sys.stderr.

importasyncioasyncdeffactorial(name,number):f=1foriinrange(2,number+1):print("Task%s: Compute factorial(%s)..."%(name,i))awaitasyncio.sleep(1)f*=iprint("Task%s: factorial(%s) =%s"%(name,number,f))loop=asyncio.get_event_loop()loop.run_until_complete(asyncio.gather(factorial("A",2),factorial("B",3),factorial("C",4),))loop.close()

TaskA:Computefactorial(2)...TaskB:Computefactorial(2)...TaskC:Computefactorial(2)...TaskA:factorial(2)=2TaskB:Computefactorial(3)...TaskC:Computefactorial(3)...TaskB:factorial(3)=6TaskC:Computefactorial(4)...TaskC:factorial(4)=24

18.5.3.6.Task functions¶

asyncio.as_completed(fs,*,loop=None,timeout=None)¶

Return an iterator whose values, when waited for, areFutureinstances.

Raisesasyncio.TimeoutError if the timeout occurs before all Futuresare done.

Example:

forfinas_completed(fs):result=yield fromf# The 'yield from' may raise# Use result

Note

The futuresf are not necessarily members of fs.

asyncio.ensure_future(coro_or_future,*,loop=None)¶

Schedule the execution of acoroutine object: wrap it ina future. Return aTask object.

If the argument is aFuture, it is returned directly.

New in version 3.4.4.

Changed in version 3.5.1:The function accepts anyawaitable object.

See also

TheAbstractEventLoop.create_task() method.

asyncio.async(coro_or_future,*,loop=None)¶

A deprecated alias toensure_future().

Deprecated since version 3.4.4.

asyncio.wrap_future(future,*,loop=None)¶

Wrap aconcurrent.futures.Future object in aFutureobject.

asyncio.gather(*coros_or_futures,loop=None,return_exceptions=False)¶

Return a future aggregating results from the given coroutine objects orfutures.

All futures must share the same event loop. If all the tasks are donesuccessfully, the returned future’s result is the list of results (in theorder of the original sequence, not necessarily the order of resultsarrival). Ifreturn_exceptions is true, exceptions in the tasks aretreated the same as successful results, and gathered in the result list;otherwise, the first raised exception will be immediately propagated to thereturned future.

Cancellation: if the outer Future is cancelled, all children (that have notcompleted yet) are also cancelled. If any child is cancelled, this istreated as if it raisedCancelledError – theouter Future isnot cancelled in this case. (This is to prevent thecancellation of one child to cause other children to be cancelled.)

Changed in version 3.6.6:If thegather itself is cancelled, the cancellation is propagatedregardless ofreturn_exceptions.

asyncio.iscoroutine(obj)¶

ReturnTrue ifobj is acoroutine object,which may be based on a generator or anasyncdef coroutine.

asyncio.iscoroutinefunction(func)¶

ReturnTrue iffunc is determined to be acoroutine function, which may be a decorated generator function or anasyncdef function.

asyncio.run_coroutine_threadsafe(coro,loop)¶

Submit acoroutine object to a given event loop.

Return aconcurrent.futures.Future to access the result.

This function is meant to be called from a different thread than the onewhere the event loop is running. Usage:

# Create a coroutinecoro=asyncio.sleep(1,result=3)# Submit the coroutine to a given loopfuture=asyncio.run_coroutine_threadsafe(coro,loop)# Wait for the result with an optional timeout argumentassertfuture.result(timeout)==3

If an exception is raised in the coroutine, the returned future will benotified. It can also be used to cancel the task in the event loop:

try:result=future.result(timeout)exceptasyncio.TimeoutError:print('The coroutine took too long, cancelling the task...')future.cancel()exceptExceptionasexc:print('The coroutine raised an exception:{!r}'.format(exc))else:print('The coroutine returned:{!r}'.format(result))

See theconcurrency and multithreadingsection of the documentation.

Note

Unlike other functions from the module,run_coroutine_threadsafe() requires theloop argument tobe passed explicitly.

New in version 3.5.1.

coroutineasyncio.sleep(delay,result=None,*,loop=None)¶

Create acoroutine that completes after a giventime (in seconds). Ifresult is provided, it is produced to the callerwhen the coroutine completes.

The resolution of the sleep depends on thegranularity of the eventloop.

This function is acoroutine.

coroutineasyncio.shield(arg,*,loop=None)¶

Wait for a future, shielding it from cancellation.

The statement:

res=yield fromshield(something())

is exactly equivalent to the statement:

res=yield fromsomething()

except that if the coroutine containing it is cancelled, the task runninginsomething() is not cancelled. From the point of view ofsomething(), the cancellation did not happen. But its caller is stillcancelled, so the yield-from expression still raisesCancelledError. Note: Ifsomething() iscancelled by other means this will still cancelshield().

If you want to completely ignore cancellation (not recommended) you cancombineshield() with a try/except clause, as follows:

try:res=yield fromshield(something())exceptCancelledError:res=None

coroutineasyncio.wait(futures,*,loop=None,timeout=None,return_when=ALL_COMPLETED)¶

Wait for the Futures and coroutine objects given by the sequencefuturesto complete. Coroutines will be wrapped in Tasks. Returns two sets ofFuture: (done, pending).

The sequencefutures must not be empty.

timeout can be used to control the maximum number of seconds to wait beforereturning.timeout can be an int or float. Iftimeout is not specifiedorNone, there is no limit to the wait time.

return_when indicates when this function should return. It must be one ofthe following constants of theconcurrent.futures module:

Constant	Description
FIRST_COMPLETED	The function will return when anyfuture finishes or is cancelled.
FIRST_EXCEPTION	The function will return when anyfuture finishes by raising anexception. If no future raises anexception then it is equivalent toALL_COMPLETED.
ALL_COMPLETED	The function will return when allfutures finish or are cancelled.

This function is acoroutine.

Usage:

done,pending=yield fromasyncio.wait(fs)

Note

This does not raiseasyncio.TimeoutError! Futures that aren’t donewhen the timeout occurs are returned in the second set.

coroutineasyncio.wait_for(fut,timeout,*,loop=None)¶

Wait for the singleFuture orcoroutine objectto complete with timeout. Iftimeout isNone, block until the futurecompletes.

Coroutine will be wrapped inTask.

Returns result of the Future or coroutine. When a timeout occurs, itcancels the task and raisesasyncio.TimeoutError. To avoid the taskcancellation, wrap it inshield().

If the wait is cancelled, the futurefut is also cancelled.

This function is acoroutine, usage:

result=yield fromasyncio.wait_for(fut,60.0)

Changed in version 3.4.3:If the wait is cancelled, the futurefut is now also cancelled.