Modify PyErr_*() functions to chain exceptions

Modify C functions raising exceptions of the Python C API toautomatically chain exceptions: modifyPyErr_SetString(),PyErr_Format(),PyErr_SetNone(), etc.

Modify functions to not chain exceptions

Keeping the previous exception is not always interesting when the newexception contains information of the previous exception or even moreinformation, especially when the two exceptions have the same type.

Example of an useless exception chain withint(str):

TypeError:abytes-likeobjectisrequired,not'type'Duringhandlingoftheaboveexception,anotherexceptionoccurred:Traceback(mostrecentcalllast):File"<stdin>",line1,in<module>TypeError:int()argumentmustbeastring,abytes-likeobjectoranumber,not'type'

The newTypeError exception contains more information than theprevious exception. The previous exception should be hidden.

ThePyErr_Clear() function can be called to clear the currentexception before raising a new exception, to not chain the currentexception with a new exception.

Modify functions to chain exceptions

Some functions save and then restore the current exception. If a newexception is raised, the exception is currently displayed intosys.stderr or ignored depending on the function. Some of thesefunctions should be modified to chain exceptions instead.

Examples of function ignoring the new exception(s):

• ptrace_enter_call(): ignore exception
• subprocess_fork_exec(): ignore exception raised by enable_gc()
• t_bootstrap() of the_thread module: ignore exception raisedby trying to display the bootstrap function tosys.stderr
• PyDict_GetItem(),_PyDict_GetItem_KnownHash(): ignoreexception raised by looking for a key in the dictionary
• _PyErr_TrySetFromCause(): ignore exception
• PyFrame_LocalsToFast(): ignore exception raised bydict_to_map()
• _PyObject_Dump(): ignore exception._PyObject_Dump() is usedto debug, to inspect a running process, it should not modify thePython state.
• Py_ReprLeave(): ignore exception “because there is no way toreport them”
• type_dealloc(): ignore exception raised byremove_all_subclasses()
• PyObject_ClearWeakRefs(): ignore exception?
• call_exc_trace(),call_trace_protected(): ignore exception
• remove_importlib_frames(): ignore exception
• do_mktuple(), helper used byPy_BuildValue() for example:ignore exception?
• flush_io(): ignore exception
• sys_write(),sys_format(): ignore exception
• _PyTraceback_Add(): ignore exception
• PyTraceBack_Print(): ignore exception

Examples of function displaying the new exception tosys.stderr:

• atexit_callfuncs(): display exceptions withPyErr_Display() and return the latest exception, the functioncalls multiple callbacks and only returns the latest exception
• sock_dealloc(): log theResourceWarning exception withPyErr_WriteUnraisable()
• slot_tp_del(): display exception withPyErr_WriteUnraisable()
• _PyGen_Finalize(): displaygen_close() exception withPyErr_WriteUnraisable()
• slot_tp_finalize(): display exception raised by the__del__() method withPyErr_WriteUnraisable()
• PyErr_GivenExceptionMatches(): display exception raised byPyType_IsSubtype() withPyErr_WriteUnraisable()

No change

A new private_PyErr_ChainExceptions() function is enough to chainmanually exceptions.

Exceptions will only be chained explicitly where it makes sense.

New helpers to chain exceptions

Functions likePyErr_SetString() don’t chain automaticallyexceptions. To make the usage of_PyErr_ChainExceptions() easier,new private functions are added:

• _PyErr_SetStringChain(exc_type,message)
• _PyErr_FormatChain(exc_type,format,...)
• _PyErr_SetNoneChain(exc_type)
• _PyErr_SetObjectChain(exc_type,exc_value)

Helper functions to raise specific exceptions like_PyErr_SetKeyError(key) orPyErr_SetImportError(message,name,path) don’t chain exceptions. The generic_PyErr_ChainExceptions(exc_type,exc_value,exc_tb) should be usedto chain exceptions with these helper functions.

PEPs

• PEP 3134 – Exception Chaining and Embedded Tracebacks(Python 3.0):new__context__ and__cause__ attributes for exceptions
• PEP 415 – Implement context suppression with exception attributes(Python 3.3):raiseexcfromNone
• PEP 409 – Suppressing exception context(superseded by thePEP 415)

Python C API

The header fileInclude/pyerror.h declares functions related toexceptions.

Functions raising exceptions:

• PyErr_SetNone(exc_type)
• PyErr_SetObject(exc_type,exc_value)
• PyErr_SetString(exc_type,message)
• PyErr_Format(exc,format,...)

Helpers to raise specific exceptions:

• PyErr_BadArgument()
• PyErr_BadInternalCall()
• PyErr_NoMemory()
• PyErr_SetFromErrno(exc)
• PyErr_SetFromWindowsErr(err)
• PyErr_SetImportError(message,name,path)
• _PyErr_SetKeyError(key)
• _PyErr_TrySetFromCause(prefix_format,...)

Manage the current exception:

• PyErr_Clear(): clear the current exception,likeexcept:pass
• PyErr_Fetch(exc_type,exc_value,exc_tb)
• PyErr_Restore(exc_type,exc_value,exc_tb)
• PyErr_GetExcInfo(exc_type,exc_value,exc_tb)
• PyErr_SetExcInfo(exc_type,exc_value,exc_tb)

Others function to handle exceptions:

• PyErr_ExceptionMatches(exc): check to implementexceptexc: ...
• PyErr_GivenExceptionMatches(exc1,exc2)
• PyErr_NormalizeException(exc_type,exc_value,exc_tb)
• _PyErr_ChainExceptions(exc_type,exc_value,exc_tb)

Python Issues

Chain exceptions:

• Issue #23763: Chain exceptions in C
• Issue #23696: zipimport: chain ImportError to OSError
• Issue #21715: Chaining exceptions at C level: added_PyErr_ChainExceptions()
• Issue #18488: sqlite: finalize() method of user function may becalled with an exception set if a call to step() method failed
• Issue #23781: Add private _PyErr_ReplaceException() in 2.7
• Issue #23782: Leak in _PyTraceback_Add

Changes preventing to loose exceptions:

• Issue #23571: Raise SystemError if a function returns a result with anexception set
• Issue #18408: Fixes crashes found by pyfailmalloc