18.5.9.1.Debug mode of asyncio¶

The implementation ofasyncio has been written for performance.In order to ease the development of asynchronous code, you may wish toenabledebug mode.

To enable all debug checks for an application:

• Enable the asyncio debug mode globally by setting the environment variablePYTHONASYNCIODEBUG to1, or by callingAbstractEventLoop.set_debug().

• Set the log level of theasyncio logger tologging.DEBUG. For example, calllogging.basicConfig(level=logging.DEBUG) at startup.

• Configure thewarnings module to displayResourceWarningwarnings. For example, use the-Wdefault command line option of Python todisplay them.

Examples debug checks:

• Logcoroutines defined but never “yielded from”

• call_soon() andcall_at() methodsraise an exception if they are called from the wrong thread.

• Log the execution time of the selector

• Log callbacks taking more than 100 ms to be executed. TheAbstractEventLoop.slow_callback_duration attribute is the minimumduration in seconds of “slow” callbacks.

• ResourceWarning warnings are emitted when transports and event loopsarenot closed explicitly.

18.5.9.2.Cancellation¶

Cancellation of tasks is not common in classic programming. In asynchronousprogramming, not only is it something common, but you have to prepare yourcode to handle it.

Futures and tasks can be cancelled explicitly with theirFuture.cancel()method. Thewait_for() function cancels the waited task when the timeoutoccurs. There are many other cases where a task can be cancelled indirectly.

Don’t callset_result() orset_exception() methodofFuture if the future is cancelled: it would fail with an exception.For example, write:

ifnotfut.cancelled():fut.set_result('done')

Don’t schedule directly a call to theset_result() or theset_exception() method of a future withAbstractEventLoop.call_soon(): the future can be cancelled before its methodis called.

If you wait for a future, you should check early if the future was cancelled toavoid useless operations. Example:

@coroutinedefslow_operation(fut):iffut.cancelled():return# ... slow computation ...yield fromfut# ...

Theshield() function can also be used to ignore cancellation.

18.5.9.3.Concurrency and multithreading¶

An event loop runs in a thread and executes all callbacks and tasks in the samethread. While a task is running in the event loop, no other task is running inthe same thread. But when the task usesyieldfrom, the task is suspendedand the event loop executes the next task.

To schedule a callback from a different thread, theAbstractEventLoop.call_soon_threadsafe() method should be used. Example:

loop.call_soon_threadsafe(callback,*args)

Most asyncio objects are not thread safe. You should only worry if you accessobjects outside the event loop. For example, to cancel a future, don’t calldirectly itsFuture.cancel() method, but:

loop.call_soon_threadsafe(fut.cancel)

To handle signals and to execute subprocesses, the event loop must be run inthe main thread.

To schedule a coroutine object from a different thread, therun_coroutine_threadsafe() function should be used. It returns aconcurrent.futures.Future to access the result:

future=asyncio.run_coroutine_threadsafe(coro_func(),loop)result=future.result(timeout)# Wait for the result with a timeout

TheAbstractEventLoop.run_in_executor() method can be used with a thread poolexecutor to execute a callback in different thread to not block the thread ofthe event loop.

18.5.9.4.Handle blocking functions correctly¶

Blocking functions should not be called directly. For example, if a functionblocks for 1 second, other tasks are delayed by 1 second which can have animportant impact on reactivity.

For networking and subprocesses, theasyncio module provides high-levelAPIs likeprotocols.

An executor can be used to run a task in a different thread or even in adifferent process, to not block the thread of the event loop. See theAbstractEventLoop.run_in_executor() method.

18.5.9.5.Logging¶

Theasyncio module logs information with thelogging module inthe logger'asyncio'.

The default log level for theasyncio module islogging.INFO.For those not wanting such verbosity fromasyncio the log level canbe changed. For example, to change the level tologging.WARNING:

logging.getLogger('asyncio').setLevel(logging.WARNING)

18.5.9.6.Detect coroutine objects never scheduled¶

When a coroutine function is called and its result is not passed toensure_future() or to theAbstractEventLoop.create_task() method,the execution of the coroutine object will never be scheduled which isprobably a bug.Enable the debug mode of asynciotolog a warning to detect it.

Example with the bug:

importasyncio@asyncio.coroutinedeftest():print("never scheduled")test()

Output in debug mode:

Coroutinetest()attest.py:3wasneveryieldedfromCoroutineobjectcreatedat(mostrecentcalllast):File"test.py",line7,in<module>test()

The fix is to call theensure_future() function or theAbstractEventLoop.create_task() method with the coroutine object.

18.5.9.7.Detect exceptions never consumed¶

Python usually callssys.excepthook() on unhandled exceptions. IfFuture.set_exception() is called, but the exception is never consumed,sys.excepthook() is not called. Instead,a log is emitted when the future is deleted by the garbage collector, with thetraceback where the exception was raised.

Example of unhandled exception:

importasyncio@asyncio.coroutinedefbug():raiseException("not consumed")loop=asyncio.get_event_loop()asyncio.ensure_future(bug())loop.run_forever()loop.close()

Output:

Taskexceptionwasneverretrievedfuture:<Taskfinishedcoro=<coro()done,definedatasyncio/coroutines.py:139>exception=Exception('not consumed',)>Traceback(mostrecentcalllast):File"asyncio/tasks.py",line237,in_stepresult=next(coro)File"asyncio/coroutines.py",line141,incorores=func(*args,**kw)File"test.py",line5,inbugraiseException("not consumed")Exception:notconsumed

Enable the debug mode of asyncio to get thetraceback where the task was created. Output in debug mode:

Taskexceptionwasneverretrievedfuture:<Taskfinishedcoro=<bug()done,definedattest.py:3>exception=Exception('not consumed',)createdattest.py:8>source_traceback:Objectcreatedat(mostrecentcalllast):File"test.py",line8,in<module>asyncio.ensure_future(bug())Traceback(mostrecentcalllast):File"asyncio/tasks.py",line237,in_stepresult=next(coro)File"asyncio/coroutines.py",line79,in__next__returnnext(self.gen)File"asyncio/coroutines.py",line141,incorores=func(*args,**kw)File"test.py",line5,inbugraiseException("not consumed")Exception:notconsumed

There are different options to fix this issue. The first option is to chain thecoroutine in another coroutine and use classic try/except:

@asyncio.coroutinedefhandle_exception():try:yield frombug()exceptException:print("exception consumed")loop=asyncio.get_event_loop()asyncio.ensure_future(handle_exception())loop.run_forever()loop.close()

Another option is to use theAbstractEventLoop.run_until_complete()function:

task=asyncio.ensure_future(bug())try:loop.run_until_complete(task)exceptException:print("exception consumed")

18.5.9.8.Chain coroutines correctly¶

When a coroutine function calls other coroutine functions and tasks, theyshould be chained explicitly withyieldfrom. Otherwise, the execution isnot guaranteed to be sequential.

Example with different bugs usingasyncio.sleep() to simulate slowoperations:

importasyncio@asyncio.coroutinedefcreate():yield fromasyncio.sleep(3.0)print("(1) create file")@asyncio.coroutinedefwrite():yield fromasyncio.sleep(1.0)print("(2) write into file")@asyncio.coroutinedefclose():print("(3) close file")@asyncio.coroutinedeftest():asyncio.ensure_future(create())asyncio.ensure_future(write())asyncio.ensure_future(close())yield fromasyncio.sleep(2.0)loop.stop()loop=asyncio.get_event_loop()asyncio.ensure_future(test())loop.run_forever()print("Pending tasks at exit:%s"%asyncio.Task.all_tasks(loop))loop.close()

Expected output:

(1) create file(2) write into file(3) close filePending tasks at exit: set()

Actual output:

(3) close file(2) write into filePending tasks at exit: {<Task pending create() at test.py:7 wait_for=<Future pending cb=[Task._wakeup()]>>}Task was destroyed but it is pending!task: <Task pending create() done at test.py:5 wait_for=<Future pending cb=[Task._wakeup()]>>

The loop stopped before thecreate() finished,close() has been calledbeforewrite(), whereas coroutine functions were called in this order:create(),write(),close().

To fix the example, tasks must be marked withyieldfrom:

@asyncio.coroutinedeftest():yield fromasyncio.ensure_future(create())yield fromasyncio.ensure_future(write())yield fromasyncio.ensure_future(close())yield fromasyncio.sleep(2.0)loop.stop()

Or withoutasyncio.ensure_future():

@asyncio.coroutinedeftest():yield fromcreate()yield fromwrite()yield fromclose()yield fromasyncio.sleep(2.0)loop.stop()

18.5.9.9.Pending task destroyed¶

If a pending task is destroyed, the execution of its wrappedcoroutine did not complete. It is probably a bug and so a warning is logged.

Example of log:

Task was destroyed but it is pending!task: <Task pending coro=<kill_me() done, defined at test.py:5> wait_for=<Future pending cb=[Task._wakeup()]>>

Enable the debug mode of asyncio to get thetraceback where the task was created. Example of log in debug mode:

Task was destroyed but it is pending!source_traceback: Object created at (most recent call last):  File "test.py", line 15, in <module>    task = asyncio.ensure_future(coro, loop=loop)task: <Task pending coro=<kill_me() done, defined at test.py:5> wait_for=<Future pending cb=[Task._wakeup()] created at test.py:7> created at test.py:15>

18.5.9.10.Close transports and event loops¶

When a transport is no more needed, call itsclose() method to releaseresources. Event loops must also be closed explicitly.

If a transport or an event loop is not closed explicitly, aResourceWarning warning will be emitted in its destructor. By default,ResourceWarning warnings are ignored. TheDebug mode of asyncio section explains how to display them.