Rejection Notice

Per the SC:

We discussed the PEP and have decided to reject it. Our thinking was themagic and risk of potential breakage didn’t warrant the benefits. We aretotally supportive, though, of exploring a potential context manager v2API or__leave__.

Abstract

This PEP proposes to make the interpreter accept context managers whose__exit__() /__aexit__() methodtakes only a single exception instance,while continuing to also support the current(typ,exc,tb) signaturefor backwards compatibility.

This proposal is part of an ongoing effort to remove the redundancy ofthe 3-item exception representation from the language, a relic of earlierPython versions which now confuses language users while adding complexityand overhead to the interpreter.

The proposed implementation uses introspection, which is tailored to therequirements of this use case. The solution ensures the safety of the newfeature by supporting it only in non-ambiguous cases. In particular, anysignature thatcould accept three arguments is assumed to expect them.

Because reliable introspection of callables is not currently possible inPython, the solution proposed here is limited in that only the common typesof single-arg callables will be identified as such, while some of the moreesoteric ones will continue to be called with three arguments. Thisimperfect solution was chosen among several imperfect alternatives in thespirit of practicality. It is my hope that the discussion about this PEPwill explore the other options and lead us to the best way forward, whichmay well be to remain with our imperfect status quo.

Motivation

In the past, an exception was represented in many parts of Python by atuple of three elements: the type of the exception, its value, and itstraceback. While there were good reasons for this design at the time,they no longer hold because the type and traceback can now be reliablydeduced from the exception instance. Over the last few years we sawseveral efforts to simplify the representation of exceptions.

Since 3.10 inCPython PR #70577,thetraceback module’s functions accept either a 3-tupleas described above, or just an exception instance as a single argument.

Internally, the interpreter no longer represents exceptions as a triplet.This wasremoved for the handled exception in 3.11 andfor the raised exception in 3.12. As a consequence,several APIs that expose the triplet can now be replaced bysimpler alternatives:

The current proposal is a step in this process, and considers the wayforward for one more case in which the 3-tuple representation hasleaked to the language. The motivation for all this work is twofold.

Rationale

The only reason to object today to the removal of the last remainingappearances of the 3-tuple from the language is the concerns aboutdisruption that such changes can bring. The goal of this PEP is to proposea safe, gradual and minimally disruptive way to make this change in thecase of__exit__, and with this to initiate a discussion of our optionsfor evolving its method signature.

In the case of thetraceback module’s API, evolving thefunctions to have a hybrid signature is relatively straightforward andsafe. The functions take one positional and two optional arguments, andinterpret them according to their types. This is safe when sentinelsare used for default values. The signatures of callbacks, which aredefined by the user’s program, are harder to evolve.

The safest option is to make the user explicitly indicate which signaturethe callback is expecting, by marking it with an additional attribute orgiving it a different name. For example, we could make the interpreterlook for a__leave__ method on the context manager, and call it witha single arg if it exists (otherwise, it looks for__exit__ andcontinues as it does now). The introspection-based alternative proposedhere intends to make it more convenient for users to write new code,because they can just use the single-arg version and remain unaware ofthe legacy API. However, if the limitations of introspection are foundto be too severe, we should consider an explicit option. Having both__exit__ and__leave__ around for 5-10 years with similarfunctionality is not ideal, but it is an option.

Let us now examine the limitations of the current proposal. It identifies2-arg python functions andMETH_O C functions as having a single-argsignature, and assumes that anything else is expecting 3 args. Obviouslyit is possible to create false negatives for this heuristic (single-argcallables that it will not identify). Context managers written in thisway won’t work, they will continue to fail as they do now when their__exit__ function will be called with three arguments.

I believe that it will not be a problem in practice. First, all workingcode will continue to work, so this is a limitation on new code ratherthan a problem impacting existing code. Second, exotic callable types arerarely used for__exit__ and if one is needed, it can always be wrappedby a plain vanilla method that delegates to the callable. For example, wecan write this:

classC:__enter__=lambdaself:self__exit__=ExoticCallable()

as follows:

classCM:__enter__=lambdaself:self_exit=ExoticCallable()__exit__=lambdaself,exc:CM._exit(exc)

While discussing the real-world impact of the problem in this PEP, it isworth noting that most__exit__ functions don’t do anything with theirarguments. Typically, a context manager is implemented to ensure that somecleanup actions take place upon exit. It is rarely appropriate for the__exit__ function to handle exceptions raised within the context, andthey are typically allowed to propagate out of__exit__ to the callingfunction. This means that most__exit__ functions do not access theirarguments at all, and we should take this into account when trying toassess the impact of different solutions on Python’s userbase.

Specification

A context manager’s__exit__/__aexit__ method can have a single-argsignature, in which case it is invoked by the interpreter with the argumentequal to an exception instance orNone:

>>>classC:...def__enter__(self):...returnself...def__exit__(self,exc):...print(f'__exit__ called with:{exc!r}')...>>>withC():...pass...__exit__ called with: None>>>withC():...1/0...__exit__ called with: ZeroDivisionError('division by zero')Traceback (most recent call last):  File"<stdin>", line2, in<module>ZeroDivisionError:division by zero

If__exit__/__aexit__ has any other signature, it is invoked withthe 3-tuple(typ,exc,tb) as happens now:

>>>classC:...def__enter__(self):...returnself...def__exit__(self,*exc):...print(f'__exit__ called with:{exc!r}')...>>>withC():...pass...__exit__ called with: (None, None, None)>>>withC():...1/0...__exit__ called with: (<class 'ZeroDivisionError'>, ZeroDivisionError('division by zero'), <traceback object at 0x1039cb570>)Traceback (most recent call last):  File"<stdin>", line2, in<module>ZeroDivisionError:division by zero

These__exit__ methods will also be called with a 3-tuple:

def__exit__(self,typ,*exc):passdef__exit__(self,typ,exc,tb):pass

A reference implementation is provided inCPython PR #101995.

When the interpreter reaches the end of the scope of a context manager,and it is about to call the relevant__exit__ or__aexit__ function,it instrospects this function to determine whether it is the single-argor the legacy 3-arg version. In the draft PR, this introspection is performedby theis_legacy___exit__ function:

staticintis_legacy___exit__(PyObject*exit_func){if(PyMethod_Check(exit_func)){PyObject*func=PyMethod_GET_FUNCTION(exit_func);if(PyFunction_Check(func)){PyCodeObject*code=(PyCodeObject*)PyFunction_GetCode(func);if(code->co_argcount==2&&!(code->co_flags&CO_VARARGS)){/* Python method that expects self + one more arg */returnfalse;}}}elseif(PyCFunction_Check(exit_func)){if(PyCFunction_GET_FLAGS(exit_func)==METH_O){/* C function declared as single-arg */returnfalse;}}returntrue;}

It is important to note that this is not a generic introspection function, butrather one which is specifically designed for our use case. We know thatexit_func is an attribute of the context manager class (taken from thetype of the object that provided__enter__), and it is typically a function.Furthermore, for this to be useful we need to identify enough single-arg forms,but not necessarily all of them. What is critical for backwards compatibility isthat we will never misidentify a legacyexit_func as a single-arg one. So,for example,__exit__(self,*args) and__exit__(self,exc_type,*args)both have the legacy form, even though theycould be invoked with one arg.

In summary, anexit_func will be invoke with a single arg if:

• It is aPyMethod withargcount2 (to countself) and no vararg, or
• it is aPyCFunction with theMETH_O flag.

Note that any performance cost of the introspection can be mitigated viaspecialization, so it won’t be a problem if we need to make it moresophisticated than this for some reason.

Backwards Compatibility

All context managers that previously worked will continue to work in thesame way because the interpreter will call them with three args wheneverthey can accept three args. There may be context managers that previouslydid not work because theirexit_func expected one argument, so the callto__exit__ would have caused aTypeError exception to be raised,and now the call would succeed. This could theoretically change thebehaviour of existing code, but it is unlikely to be a problem in practice.

The backwards compatibility concerns will show up in some cases when librariestry to migrate their context managers from the multi-arg to the single-argsignature. If__exit__ or__aexit__ is called by any code other thanthe interpreter’s eval loop, the introspection does not automatically happen.For example, this will occur where a context manager is subclassed and its__exit__ method is called directly from the derived__exit__. Suchcontext managers will need to migrate to the single-arg version with theirusers, and may choose to offer a parallel API rather than breaking theexisting one. Alternatively, a superclass can stay with the signature__exit__(self,*args), and support both one and three args. Sincemost context managers do not use the value of the arguments to__exit__,and simply allow the exception to propagate onward, this is likely to be thecommon approach.

Security Implications

I am not aware of any.

How to Teach This

The language tutorial will present the single-arg version, and the documentationfor context managers will include a section on the legacy signatures of__exit__ and__aexit__.

Reference Implementation

CPython PR #101995implements the proposal of this PEP.

Rejected Ideas

Copyright

This document is placed in the public domain or under theCC0-1.0-Universal license, whichever is more permissive.