Utilities¶

Functions and classes provided:

classcontextlib.AbstractContextManager¶

Anabstract base class for classes that implementobject.__enter__() andobject.__exit__(). A defaultimplementation forobject.__enter__() is provided which returnsself whileobject.__exit__() is an abstract method which by defaultreturnsNone. See also the definition ofContext Manager Types.

Added in version 3.6.

classcontextlib.AbstractAsyncContextManager¶

Anabstract base class for classes that implementobject.__aenter__() andobject.__aexit__(). A defaultimplementation forobject.__aenter__() is provided which returnsself whileobject.__aexit__() is an abstract method which by defaultreturnsNone. See also the definition ofAsynchronous Context Managers.

Added in version 3.7.

@contextlib.contextmanager¶

This function is adecorator that can be used to define a factoryfunction forwith statement context managers, without needing tocreate a class or separate__enter__() and__exit__() methods.

While many objects natively support use in with statements, sometimes aresource needs to be managed that isn’t a context manager in its own right,and doesn’t implement aclose() method for use withcontextlib.closing.

An abstract example would be the following to ensure correct resourcemanagement:

fromcontextlibimportcontextmanager@contextmanagerdefmanaged_resource(*args,**kwds):# Code to acquire resource, e.g.:resource=acquire_resource(*args,**kwds)try:yieldresourcefinally:# Code to release resource, e.g.:release_resource(resource)

The function can then be used like this:

>>>withmanaged_resource(timeout=3600)asresource:...# Resource is released at the end of this block,...# even if code in the block raises an exception

The function being decorated must return agenerator-iterator whencalled. This iterator must yield exactly one value, which will be bound tothe targets in thewith statement’sas clause, if any.

At the point where the generator yields, the block nested in thewithstatement is executed. The generator is then resumed after the block is exited.If an unhandled exception occurs in the block, it is reraised inside thegenerator at the point where the yield occurred. Thus, you can use atry…except…finally statement to trapthe error (if any), or ensure that some cleanup takes place. If an exception istrapped merely in order to log it or to perform some action (rather than tosuppress it entirely), the generator must reraise that exception. Otherwise thegenerator context manager will indicate to thewith statement thatthe exception has been handled, and execution will resume with the statementimmediately following thewith statement.

contextmanager() usesContextDecorator so the context managersit creates can be used as decorators as well as inwith statements.When used as a decorator, a new generator instance is implicitly created oneach function call (this allows the otherwise “one-shot” context managerscreated bycontextmanager() to meet the requirement that contextmanagers support multiple invocations in order to be used as decorators).

Changed in version 3.2:Use ofContextDecorator.

@contextlib.asynccontextmanager¶

Similar tocontextmanager(), but creates anasynchronous context manager.

This function is adecorator that can be used to define a factoryfunction forasyncwith statement asynchronous context managers,without needing to create a class or separate__aenter__() and__aexit__() methods. It must be applied to anasynchronousgenerator function.

A simple example:

fromcontextlibimportasynccontextmanager@asynccontextmanagerasyncdefget_connection():conn=awaitacquire_db_connection()try:yieldconnfinally:awaitrelease_db_connection(conn)asyncdefget_all_users():asyncwithget_connection()asconn:returnconn.query('SELECT ...')

Added in version 3.7.

Context managers defined withasynccontextmanager() can be usedeither as decorators or withasyncwith statements:

importtimefromcontextlibimportasynccontextmanager@asynccontextmanagerasyncdeftimeit():now=time.monotonic()try:yieldfinally:print(f'it took{time.monotonic()-now}s to run')@timeit()asyncdefmain():# ... async code ...

When used as a decorator, a new generator instance is implicitly created oneach function call. This allows the otherwise “one-shot” context managerscreated byasynccontextmanager() to meet the requirement that contextmanagers support multiple invocations in order to be used as decorators.

Changed in version 3.10:Async context managers created withasynccontextmanager() canbe used as decorators.

contextlib.closing(thing)¶

Return a context manager that closesthing upon completion of the block. Thisis basically equivalent to:

fromcontextlibimportcontextmanager@contextmanagerdefclosing(thing):try:yieldthingfinally:thing.close()

And lets you write code like this:

fromcontextlibimportclosingfromurllib.requestimporturlopenwithclosing(urlopen('https://www.python.org'))aspage:forlineinpage:print(line)

without needing to explicitly closepage. Even if an error occurs,page.close() will be called when thewith block is exited.

Note

Most types managing resources support thecontext manager protocol,which closesthing on leaving thewith statement.As such,closing() is most useful for third party types that don’tsupport context managers.This example is purely for illustration purposes,asurlopen() would normally be used in a context manager.

contextlib.aclosing(thing)¶

Return an async context manager that calls theaclose() method ofthingupon completion of the block. This is basically equivalent to:

fromcontextlibimportasynccontextmanager@asynccontextmanagerasyncdefaclosing(thing):try:yieldthingfinally:awaitthing.aclose()

Significantly,aclosing() supports deterministic cleanup of asyncgenerators when they happen to exit early bybreak or anexception. For example:

fromcontextlibimportaclosingasyncwithaclosing(my_generator())asvalues:asyncforvalueinvalues:ifvalue==42:break

This pattern ensures that the generator’s async exit code is executed inthe same context as its iterations (so that exceptions and contextvariables work as expected, and the exit code isn’t run after thelifetime of some task it depends on).

Added in version 3.10.

contextlib.nullcontext(enter_result=None)¶

Return a context manager that returnsenter_result from__enter__, butotherwise does nothing. It is intended to be used as a stand-in for anoptional context manager, for example:

defmyfunction(arg,ignore_exceptions=False):ifignore_exceptions:# Use suppress to ignore all exceptions.cm=contextlib.suppress(Exception)else:# Do not ignore any exceptions, cm has no effect.cm=contextlib.nullcontext()withcm:# Do something

An example usingenter_result:

defprocess_file(file_or_path):ifisinstance(file_or_path,str):# If string, open filecm=open(file_or_path)else:# Caller is responsible for closing filecm=nullcontext(file_or_path)withcmasfile:# Perform processing on the file

It can also be used as a stand-in forasynchronous context managers:

asyncdefsend_http(session=None):ifnotsession:# If no http session, create it with aiohttpcm=aiohttp.ClientSession()else:# Caller is responsible for closing the sessioncm=nullcontext(session)asyncwithcmassession:# Send http requests with session

Added in version 3.7.

Changed in version 3.10:asynchronous context manager support was added.

contextlib.suppress(*exceptions)¶

Return a context manager that suppresses any of the specified exceptionsif they occur in the body of awith statement and thenresumes execution with the first statement following the end of thewith statement.

As with any other mechanism that completely suppresses exceptions, thiscontext manager should be used only to cover very specific errors wheresilently continuing with program execution is known to be the rightthing to do.

For example:

fromcontextlibimportsuppresswithsuppress(FileNotFoundError):os.remove('somefile.tmp')withsuppress(FileNotFoundError):os.remove('someotherfile.tmp')

This code is equivalent to:

try:os.remove('somefile.tmp')exceptFileNotFoundError:passtry:os.remove('someotherfile.tmp')exceptFileNotFoundError:pass

This context manager isreentrant.

If the code within thewith block raises aBaseExceptionGroup, suppressed exceptions are removed from thegroup. Any exceptions of the group which are not suppressed are re-raised ina new group which is created using the original group’sderive()method.

Added in version 3.4.

Changed in version 3.12:suppress now supports suppressing exceptions raised aspart of aBaseExceptionGroup.

contextlib.redirect_stdout(new_target)¶

Context manager for temporarily redirectingsys.stdout toanother file or file-like object.

This tool adds flexibility to existing functions or classes whose outputis hardwired to stdout.

For example, the output ofhelp() normally is sent tosys.stdout.You can capture that output in a string by redirecting the output to anio.StringIO object. The replacement stream is returned from the__enter__ method and so is available as the target of thewith statement:

withredirect_stdout(io.StringIO())asf:help(pow)s=f.getvalue()

To send the output ofhelp() to a file on disk, redirect the outputto a regular file:

withopen('help.txt','w')asf:withredirect_stdout(f):help(pow)

To send the output ofhelp() tosys.stderr:

withredirect_stdout(sys.stderr):help(pow)

Note that the global side effect onsys.stdout means that thiscontext manager is not suitable for use in library code and most threadedapplications. It also has no effect on the output of subprocesses.However, it is still a useful approach for many utility scripts.

This context manager isreentrant.

Added in version 3.4.

contextlib.redirect_stderr(new_target)¶

Similar toredirect_stdout() but redirectingsys.stderr to another file or file-like object.

This context manager isreentrant.

Added in version 3.5.

contextlib.chdir(path)¶

Non parallel-safe context manager to change the current working directory.As this changes a global state, the working directory, it is not suitablefor use in most threaded or async contexts. It is also not suitable for mostnon-linear code execution, like generators, where the program execution istemporarily relinquished – unless explicitly desired, you should not yieldwhen this context manager is active.

This is a simple wrapper aroundchdir(), it changes the currentworking directory upon entering and restores the old one on exit.

This context manager isreentrant.

Added in version 3.11.

classcontextlib.ContextDecorator¶

A base class that enables a context manager to also be used as a decorator.

Context managers inheriting fromContextDecorator have to implement__enter__ and__exit__ as normal.__exit__ retains its optionalexception handling even when used as a decorator.

ContextDecorator is used bycontextmanager(), so you get thisfunctionality automatically.

Example ofContextDecorator:

fromcontextlibimportContextDecoratorclassmycontext(ContextDecorator):def__enter__(self):print('Starting')returnselfdef__exit__(self,*exc):print('Finishing')returnFalse

The class can then be used like this:

>>>@mycontext()...deffunction():...print('The bit in the middle')...>>>function()StartingThe bit in the middleFinishing>>>withmycontext():...print('The bit in the middle')...StartingThe bit in the middleFinishing

This change is just syntactic sugar for any construct of the following form:

deff():withcm():# Do stuff

ContextDecorator lets you instead write:

@cm()deff():# Do stuff

It makes it clear that thecm applies to the whole function, rather thanjust a piece of it (and saving an indentation level is nice, too).

Existing context managers that already have a base class can be extended byusingContextDecorator as a mixin class:

fromcontextlibimportContextDecoratorclassmycontext(ContextBaseClass,ContextDecorator):def__enter__(self):returnselfdef__exit__(self,*exc):returnFalse

Note

As the decorated function must be able to be called multiple times, theunderlying context manager must support use in multiplewithstatements. If this is not the case, then the original construct with theexplicitwith statement inside the function should be used.

Added in version 3.2.

classcontextlib.AsyncContextDecorator¶

Similar toContextDecorator but only for asynchronous functions.

Example ofAsyncContextDecorator:

fromasyncioimportrunfromcontextlibimportAsyncContextDecoratorclassmycontext(AsyncContextDecorator):asyncdef__aenter__(self):print('Starting')returnselfasyncdef__aexit__(self,*exc):print('Finishing')returnFalse

The class can then be used like this:

>>>@mycontext()...asyncdeffunction():...print('The bit in the middle')...>>>run(function())StartingThe bit in the middleFinishing>>>asyncdeffunction():...asyncwithmycontext():...print('The bit in the middle')...>>>run(function())StartingThe bit in the middleFinishing

Added in version 3.10.

classcontextlib.ExitStack¶

A context manager that is designed to make it easy to programmaticallycombine other context managers and cleanup functions, especially thosethat are optional or otherwise driven by input data.

For example, a set of files may easily be handled in a single withstatement as follows:

withExitStack()asstack:files=[stack.enter_context(open(fname))forfnameinfilenames]# All opened files will automatically be closed at the end of# the with statement, even if attempts to open files later# in the list raise an exception

The__enter__() method returns theExitStack instance, andperforms no additional operations.

Each instance maintains a stack of registered callbacks that are called inreverse order when the instance is closed (either explicitly or implicitlyat the end of awith statement). Note that callbacks arenotinvoked implicitly when the context stack instance is garbage collected.

This stack model is used so that context managers that acquire theirresources in their__init__ method (such as file objects) can behandled correctly.

Since registered callbacks are invoked in the reverse order ofregistration, this ends up behaving as if multiple nestedwithstatements had been used with the registered set of callbacks. This evenextends to exception handling - if an inner callback suppresses or replacesan exception, then outer callbacks will be passed arguments based on thatupdated state.

This is a relatively low level API that takes care of the details ofcorrectly unwinding the stack of exit callbacks. It provides a suitablefoundation for higher level context managers that manipulate the exitstack in application specific ways.

Added in version 3.3.

enter_context(cm)¶

Enters a new context manager and adds its__exit__() method tothe callback stack. The return value is the result of the contextmanager’s own__enter__() method.

These context managers may suppress exceptions just as they normallywould if used directly as part of awith statement.

Changed in version 3.11:RaisesTypeError instead ofAttributeError ifcmis not a context manager.

push(exit)¶

Adds a context manager’s__exit__() method to the callback stack.

As__enter__ isnot invoked, this method can be used to coverpart of an__enter__() implementation with a context manager’s own__exit__() method.

If passed an object that is not a context manager, this method assumesit is a callback with the same signature as a context manager’s__exit__() method and adds it directly to the callback stack.

By returning true values, these callbacks can suppress exceptions thesame way context manager__exit__() methods can.

The passed in object is returned from the function, allowing thismethod to be used as a function decorator.

callback(callback,/,*args,**kwds)¶

Accepts an arbitrary callback function and arguments and adds it tothe callback stack.

Unlike the other methods, callbacks added this way cannot suppressexceptions (as they are never passed the exception details).

The passed in callback is returned from the function, allowing thismethod to be used as a function decorator.

pop_all()¶

Transfers the callback stack to a freshExitStack instanceand returns it. No callbacks are invoked by this operation - instead,they will now be invoked when the new stack is closed (eitherexplicitly or implicitly at the end of awith statement).

For example, a group of files can be opened as an “all or nothing”operation as follows:

withExitStack()asstack:files=[stack.enter_context(open(fname))forfnameinfilenames]# Hold onto the close method, but don't call it yet.close_files=stack.pop_all().close# If opening any file fails, all previously opened files will be# closed automatically. If all files are opened successfully,# they will remain open even after the with statement ends.# close_files() can then be invoked explicitly to close them all.

close()¶

Immediately unwinds the callback stack, invoking callbacks in thereverse order of registration. For any context managers and exitcallbacks registered, the arguments passed in will indicate that noexception occurred.

classcontextlib.AsyncExitStack¶

Anasynchronous context manager, similartoExitStack, that supports combining both synchronous andasynchronous context managers, as well as having coroutines forcleanup logic.

Theclose() method is not implemented;aclose() must be usedinstead.

asyncenter_async_context(cm)¶

Similar toExitStack.enter_context() but expects an asynchronous contextmanager.

Changed in version 3.11:RaisesTypeError instead ofAttributeError ifcmis not an asynchronous context manager.

push_async_exit(exit)¶

Similar toExitStack.push() but expects either an asynchronous context manageror a coroutine function.

push_async_callback(callback,/,*args,**kwds)¶

Similar toExitStack.callback() but expects a coroutine function.

asyncaclose()¶

Similar toExitStack.close() but properly handles awaitables.

Continuing the example forasynccontextmanager():

asyncwithAsyncExitStack()asstack:connections=[awaitstack.enter_async_context(get_connection())foriinrange(5)]# All opened connections will automatically be released at the end of# the async with statement, even if attempts to open a connection# later in the list raise an exception.

Added in version 3.7.

Examples and Recipes¶

This section describes some examples and recipes for making effective use ofthe tools provided bycontextlib.

withExitStack()asstack:forresourceinresources:stack.enter_context(resource)ifneed_special_resource():special=acquire_special_resource()stack.callback(release_special_resource,special)# Perform operations that use the acquired resources

stack=ExitStack()try:x=stack.enter_context(cm)exceptException:# handle __enter__ exceptionelse:withstack:# Handle normal case

fromcontextlibimportcontextmanager,AbstractContextManager,ExitStackclassResourceManager(AbstractContextManager):def__init__(self,acquire_resource,release_resource,check_resource_ok=None):self.acquire_resource=acquire_resourceself.release_resource=release_resourceifcheck_resource_okisNone:defcheck_resource_ok(resource):returnTrueself.check_resource_ok=check_resource_ok@contextmanagerdef_cleanup_on_error(self):withExitStack()asstack:stack.push(self)yield# The validation check passed and didn't raise an exception# Accordingly, we want to keep the resource, and pass it# back to our callerstack.pop_all()def__enter__(self):resource=self.acquire_resource()withself._cleanup_on_error():ifnotself.check_resource_ok(resource):msg="Failed validation for{!r}"raiseRuntimeError(msg.format(resource))returnresourcedef__exit__(self,*exc_details):# We don't need to duplicate any of our resource release logicself.release_resource()

cleanup_needed=Truetry:result=perform_operation()ifresult:cleanup_needed=Falsefinally:ifcleanup_needed:cleanup_resources()

fromcontextlibimportExitStackwithExitStack()asstack:stack.callback(cleanup_resources)result=perform_operation()ifresult:stack.pop_all()

fromcontextlibimportExitStackclassCallback(ExitStack):def__init__(self,callback,/,*args,**kwds):super().__init__()self.callback(callback,*args,**kwds)defcancel(self):self.pop_all()withCallback(cleanup_resources)ascb:result=perform_operation()ifresult:cb.cancel()

fromcontextlibimportExitStackwithExitStack()asstack:@stack.callbackdefcleanup_resources():...result=perform_operation()ifresult:stack.pop_all()

fromcontextlibimportContextDecoratorimportlogginglogging.basicConfig(level=logging.INFO)classtrack_entry_and_exit(ContextDecorator):def__init__(self,name):self.name=namedef__enter__(self):logging.info('Entering:%s',self.name)def__exit__(self,exc_type,exc,exc_tb):logging.info('Exiting:%s',self.name)

withtrack_entry_and_exit('widget loader'):print('Some time consuming activity goes here')load_widget()

@track_entry_and_exit('widget loader')defactivity():print('Some time consuming activity goes here')load_widget()

Single use, reusable and reentrant context managers¶

Most context managers are written in a way that means they can only beused effectively in awith statement once. These single usecontext managers must be created afresh each time they’re used -attempting to use them a second time will trigger an exception orotherwise not work correctly.

This common limitation means that it is generally advisable to createcontext managers directly in the header of thewith statementwhere they are used (as shown in all of the usage examples above).

Files are an example of effectively single use context managers, sincethe firstwith statement will close the file, preventing anyfurther IO operations using that file object.

Context managers created usingcontextmanager() are also single usecontext managers, and will complain about the underlying generator failingto yield if an attempt is made to use them a second time:

>>>fromcontextlibimportcontextmanager>>>@contextmanager...defsingleuse():...print("Before")...yield...print("After")...>>>cm=singleuse()>>>withcm:...pass...BeforeAfter>>>withcm:...pass...Traceback (most recent call last):...RuntimeError:generator didn't yield

>>>fromcontextlibimportredirect_stdout>>>fromioimportStringIO>>>stream=StringIO()>>>write_to_stream=redirect_stdout(stream)>>>withwrite_to_stream:...print("This is written to the stream rather than stdout")...withwrite_to_stream:...print("This is also written to the stream")...>>>print("This is written directly to stdout")This is written directly to stdout>>>print(stream.getvalue())This is written to the stream rather than stdoutThis is also written to the stream

>>>fromcontextlibimportExitStack>>>stack=ExitStack()>>>withstack:...stack.callback(print,"Callback: from first context")...print("Leaving first context")...Leaving first contextCallback: from first context>>>withstack:...stack.callback(print,"Callback: from second context")...print("Leaving second context")...Leaving second contextCallback: from second context>>>withstack:...stack.callback(print,"Callback: from outer context")...withstack:...stack.callback(print,"Callback: from inner context")...print("Leaving inner context")...print("Leaving outer context")...Leaving inner contextCallback: from inner contextCallback: from outer contextLeaving outer context

>>>fromcontextlibimportExitStack>>>withExitStack()asouter_stack:...outer_stack.callback(print,"Callback: from outer context")...withExitStack()asinner_stack:...inner_stack.callback(print,"Callback: from inner context")...print("Leaving inner context")...print("Leaving outer context")...Leaving inner contextCallback: from inner contextLeaving outer contextCallback: from outer context