Source code:Lib/contextlib.py
This module provides utilities for common tasks involving thewithstatement. For more information see alsoContext Manager Types andWith Statement Context Managers.
Functions and classes provided:
This function is adecorator that can be used to define a factoryfunction forwith statement context managers, without needing tocreate a class or separate__enter__() and__exit__() methods.
A simple example (this is not recommended as a real way of generating HTML!):
fromcontextlibimportcontextmanager@contextmanagerdeftag(name):print("<%s>"%name)yieldprint("</%s>"%name)>>>withtag("h1"):...print("foo")...<h1>foo</h1>
The function being decorated must return agenerator-iterator whencalled. This iterator must yield exactly one value, which will be bound tothe targets in thewith statement’sas clause, if any.
At the point where the generator yields, the block nested in thewithstatement is executed. The generator is then resumed after the block is exited.If an unhandled exception occurs in the block, it is reraised inside thegenerator at the point where the yield occurred. Thus, you can use atry...except...finally statement to trapthe error (if any), or ensure that some cleanup takes place. If an exception istrapped merely in order to log it or to perform some action (rather than tosuppress it entirely), the generator must reraise that exception. Otherwise thegenerator context manager will indicate to thewith statement thatthe exception has been handled, and execution will resume with the statementimmediately following thewith statement.
contextmanager() usesContextDecorator so the context managersit creates can be used as decorators as well as inwith statements.When used as a decorator, a new generator instance is implicitly created oneach function call (this allows the otherwise “one-shot” context managerscreated bycontextmanager() to meet the requirement that contextmanagers support multiple invocations in order to be used as decorators).
Changed in version 3.2:Use ofContextDecorator.
Return a context manager that closesthing upon completion of the block. Thisis basically equivalent to:
fromcontextlibimportcontextmanager@contextmanagerdefclosing(thing):try:yieldthingfinally:thing.close()
And lets you write code like this:
fromcontextlibimportclosingfromurllib.requestimporturlopenwithclosing(urlopen('http://www.python.org'))aspage:forlineinpage:print(line)
without needing to explicitly closepage. Even if an error occurs,page.close() will be called when thewith block is exited.
A base class that enables a context manager to also be used as a decorator.
Context managers inheriting fromContextDecorator have to implement__enter__ and__exit__ as normal.__exit__ retains its optionalexception handling even when used as a decorator.
ContextDecorator is used bycontextmanager(), so you get thisfunctionality automatically.
Example ofContextDecorator:
fromcontextlibimportContextDecoratorclassmycontext(ContextDecorator):def__enter__(self):print('Starting')returnselfdef__exit__(self,*exc):print('Finishing')returnFalse>>>@mycontext()...deffunction():...print('The bit in the middle')...>>>function()StartingThebitinthemiddleFinishing>>>withmycontext():...print('The bit in the middle')...StartingThebitinthemiddleFinishing
This change is just syntactic sugar for any construct of the following form:
deff():withcm():# Do stuff
ContextDecorator lets you instead write:
@cm()deff():# Do stuff
It makes it clear that thecm applies to the whole function, rather thanjust a piece of it (and saving an indentation level is nice, too).
Existing context managers that already have a base class can be extended byusingContextDecorator as a mixin class:
fromcontextlibimportContextDecoratorclassmycontext(ContextBaseClass,ContextDecorator):def__enter__(self):returnselfdef__exit__(self,*exc):returnFalse
Note
As the decorated function must be able to be called multiple times, theunderlying context manager must support use in multiplewithstatements. If this is not the case, then the original construct with theexplicitwith statement inside the function should be used.
New in version 3.2.
A context manager that is designed to make it easy to programmaticallycombine other context managers and cleanup functions, especially thosethat are optional or otherwise driven by input data.
For example, a set of files may easily be handled in a single withstatement as follows:
withExitStack()asstack:files=[stack.enter_context(open(fname))forfnameinfilenames]# All opened files will automatically be closed at the end of# the with statement, even if attempts to open files later# in the list raise an exception
Each instance maintains a stack of registered callbacks that are called inreverse order when the instance is closed (either explicitly or implicitlyat the end of awith statement). Note that callbacks arenotinvoked implicitly when the context stack instance is garbage collected.
This stack model is used so that context managers that acquire theirresources in their__init__ method (such as file objects) can behandled correctly.
Since registered callbacks are invoked in the reverse order ofregistration, this ends up behaving as if multiple nestedwithstatements had been used with the registered set of callbacks. This evenextends to exception handling - if an inner callback suppresses or replacesan exception, then outer callbacks will be passed arguments based on thatupdated state.
This is a relatively low level API that takes care of the details ofcorrectly unwinding the stack of exit callbacks. It provides a suitablefoundation for higher level context managers that manipulate the exitstack in application specific ways.
New in version 3.3.
Enters a new context manager and adds its__exit__() method tothe callback stack. The return value is the result of the contextmanager’s own__enter__() method.
These context managers may suppress exceptions just as they normallywould if used directly as part of awith statement.
Adds a context manager’s__exit__() method to the callback stack.
As__enter__ isnot invoked, this method can be used to coverpart of an__enter__() implementation with a context manager’s own__exit__() method.
If passed an object that is not a context manager, this method assumesit is a callback with the same signature as a context manager’s__exit__() method and adds it directly to the callback stack.
By returning true values, these callbacks can suppress exceptions thesame way context manager__exit__() methods can.
The passed in object is returned from the function, allowing thismethod to be used as a function decorator.
Accepts an arbitrary callback function and arguments and adds it tothe callback stack.
Unlike the other methods, callbacks added this way cannot suppressexceptions (as they are never passed the exception details).
The passed in callback is returned from the function, allowing thismethod to be used as a function decorator.
Transfers the callback stack to a freshExitStack instanceand returns it. No callbacks are invoked by this operation - instead,they will now be invoked when the new stack is closed (eitherexplicitly or implicitly at the end of awith statement).
For example, a group of files can be opened as an “all or nothing”operation as follows:
withExitStack()asstack:files=[stack.enter_context(open(fname))forfnameinfilenames]# Hold onto the close method, but don't call it yet.close_files=stack.pop_all().close# If opening any file fails, all previously opened files will be# closed automatically. If all files are opened successfully,# they will remain open even after the with statement ends.# close_files() can then be invoked explicitly to close them all.
Immediately unwinds the callback stack, invoking callbacks in thereverse order of registration. For any context managers and exitcallbacks registered, the arguments passed in will indicate that noexception occurred.
This section describes some examples and recipes for making effective use ofthe tools provided bycontextlib.
The primary use case forExitStack is the one given in the classdocumentation: supporting a variable number of context managers and othercleanup operations in a singlewith statement. The variabilitymay come from the number of context managers needed being driven by userinput (such as opening a user specified collection of files), or fromsome of the context managers being optional:
withExitStack()asstack:forresourceinresources:stack.enter_context(resource)ifneed_specialresource:special=acquire_special_resource()stack.callback(release_special_resource,special)# Perform operations that use the acquired resources
As shown,ExitStack also makes it quite easy to usewithstatements to manage arbitrary resources that don’t natively support thecontext management protocol.
In the specific case of a single optional context manager,ExitStackinstances can be used as a “do nothing” context manager, allowing a contextmanager to easily be omitted without affecting the overall structure ofthe source code:
defdebug_trace(details):if__debug__:returnTraceContext(details)# Don't do anything special with the context in release modereturnExitStack()withdebug_trace():# Suite is traced in debug mode, but runs normally otherwise
It is occasionally desirable to catch exceptions from an__enter__method implementation,without inadvertently catching exceptions fromthewith statement body or the context manager’s__exit__method. By usingExitStack the steps in the context managementprotocol can be separated slightly in order to allow this:
stack=ExitStack()try:x=stack.enter_context(cm)exceptException:# handle __enter__ exceptionelse:withstack:# Handle normal case
Actually needing to do this is likely to indicate that the underlying APIshould be providing a direct resource management interface for use withtry/except/finally statements, but notall APIs are well designed in that regard. When a context manager is theonly resource management API provided, thenExitStack can make iteasier to handle various situations that can’t be handled directly in awith statement.
As noted in the documentation ofExitStack.push(), thismethod can be useful in cleaning up an already allocated resource if latersteps in the__enter__() implementation fail.
Here’s an example of doing this for a context manager that accepts resourceacquisition and release functions, along with an optional validation function,and maps them to the context management protocol:
fromcontextlibimportcontextmanager,ExitStackclassResourceManager:def__init__(self,acquire_resource,release_resource,check_resource_ok=None):self.acquire_resource=acquire_resourceself.release_resource=release_resourceifcheck_resource_okisNone:defcheck_resource_ok(resource):returnTrueself.check_resource_ok=check_resource_ok@contextmanagerdef_cleanup_on_error(self):withExitStack()asstack:stack.push(self)yield# The validation check passed and didn't raise an exception# Accordingly, we want to keep the resource, and pass it# back to our callerstack.pop_all()def__enter__(self):resource=self.acquire_resource()withself._cleanup_on_error():ifnotself.check_resource_ok(resource):msg="Failed validation for {!r}"raiseRuntimeError(msg.format(resource))returnresourcedef__exit__(self,*exc_details):# We don't need to duplicate any of our resource release logicself.release_resource()
A pattern you will sometimes see is atry-finally statement with a flagvariable to indicate whether or not the body of thefinally clause shouldbe executed. In its simplest form (that can’t already be handled just byusing anexcept clause instead), it looks something like this:
cleanup_needed=Truetry:result=perform_operation()ifresult:cleanup_needed=Falsefinally:ifcleanup_needed:cleanup_resources()
As with anytry statement based code, this can cause problems fordevelopment and review, because the setup code and the cleanup code can endup being separated by arbitrarily long sections of code.
ExitStack makes it possible to instead register a callback forexecution at the end of awith statement, and then later decide to skipexecuting that callback:
fromcontextlibimportExitStackwithExitStack()asstack:stack.callback(cleanup_resources)result=perform_operation()ifresult:stack.pop_all()
This allows the intended cleanup up behaviour to be made explicit up front,rather than requiring a separate flag variable.
If a particular application uses this pattern a lot, it can be simplifiedeven further by means of a small helper class:
fromcontextlibimportExitStackclassCallback(ExitStack):def__init__(self,callback,*args,**kwds):super(Callback,self).__init__()self.callback(callback,*args,**kwds)defcancel(self):self.pop_all()withCallback(cleanup_resources)ascb:result=perform_operation()ifresult:cb.cancel()
If the resource cleanup isn’t already neatly bundled into a standalonefunction, then it is still possible to use the decorator form ofExitStack.callback() to declare the resource cleanup inadvance:
fromcontextlibimportExitStackwithExitStack()asstack:@stack.callbackdefcleanup_resources():...result=perform_operation()ifresult:stack.pop_all()
Due to the way the decorator protocol works, a callback functiondeclared this way cannot take any parameters. Instead, any resources tobe released must be accessed as closure variables
ContextDecorator makes it possible to use a context manager inboth an ordinarywith statement and also as a function decorator.
For example, it is sometimes useful to wrap functions or groups of statementswith a logger that can track the time of entry and time of exit. Rather thanwriting both a function decorator and a context manager for the task,inheriting fromContextDecorator provides both capabilities in asingle definition:
fromcontextlibimportContextDecoratorimportlogginglogging.basicConfig(level=logging.INFO)classtrack_entry_and_exit(ContextDecorator):def__init__(self,name):self.name=namedef__enter__(self):logging.info('Entering: {}'.format(name))def__exit__(self,exc_type,exc,exc_tb):logging.info('Exiting: {}'.format(name))
Instances of this class can be used as both a context manager:
withtrack_entry_and_exit('widget loader'):print('Some time consuming activity goes here')load_widget()
And also as a function decorator:
@track_entry_and_exit('widget loader')defactivity():print('Some time consuming activity goes here')load_widget()
Note that there is one additional limitation when using context managersas function decorators: there’s no way to access the return value of__enter__(). If that value is needed, then it is still necessary to usean explicitwith statement.
28.5.warnings — Warning control
28.7.abc — Abstract Base Classes
Enter search terms or a module, class or function name.
