If the error handler is used, a new bytes object is created to set as the object attribute of UnicodeDecodeError, and that bytes object then replaces the original data. A pointer to the decoded data will became invalid after destroying that temporary bytes object. So we need other way to return the first invalid escape from _PyUnicode_DecodeUnicodeEscapeInternal().

_PyBytes_DecodeEscape() does not have such issue, because it does not use the error handlers registry, but it should be changed for compatibility with _PyUnicode_DecodeUnicodeEscapeInternal().

Still I haven’t managed to fix it completely. Still failing on one test:

[  778s] ======================================================================[  778s] FAIL: test_warning (test.test_codeop.CodeopTests.test_warning)[  778s] ----------------------------------------------------------------------[  778s] Traceback (most recent call last):[  778s]   File "/home/abuild/rpmbuild/BUILD/python312-3.12.10-build/Python-3.12.10/Lib/test/test_codeop.py", line 283, in test_warning[  778s]     with warnings_helper.check_warnings([  778s]          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^[  778s]   File "/home/abuild/rpmbuild/BUILD/python312-3.12.10-build/Python-3.12.10/Lib/contextlib.py", line 144, in __exit__[  778s]     next(self.gen)[  778s]   File "/home/abuild/rpmbuild/BUILD/python312-3.12.10-build/Python-3.12.10/Lib/test/support/warnings_helper.py", line 185, in _filterwarnings[  778s]     raise AssertionError("unhandled warning %s" % reraise[0])[  778s] AssertionError: unhandled warning {message : SyntaxWarning("'\\e' is an invalid escape sequence. "), category : 'SyntaxWarning', filename : '<input>', lineno : 1, line : None}[  778s] [  778s] ----------------------------------------------------------------------[  778s] Ran 1 test in 0.001s

Complete build log

Any idea how to fix it?

Issue:Use-after-free inunicode_escape decoder with error handler #133767

Fix use-after-free in the unicode-escape decoder with error handler

0c52510

If the error handler is used, a new bytes object is created to set asthe object attribute of UnicodeDecodeError, and that bytes object thenreplaces the original data. A pointer to the decoded data will became invalidafter destroying that temporary bytes object. So we need other way to returnthe first invalid escape from _PyUnicode_DecodeUnicodeEscapeInternal()._PyBytes_DecodeEscape() does not have such issue, because it does notuse the error handlers registry, but it should be changed for compatibilitywith _PyUnicode_DecodeUnicodeEscapeInternal().