Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork33.7k
gh-125038: Iterator checks are added for some FOR_ITER bytecodes, crash fixed#125051
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
Python/bytecodes.c Outdated
| PyObject*iter_o=PyStackRef_AsPyObjectBorrow(iter); | ||
| PyObject*next_o= (*Py_TYPE(iter_o)->tp_iternext)(iter_o); | ||
| PyObject*next_o=NULL; | ||
| if (PyIter_Check(iter_o)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
CallingPyIter_Check feels a little wasteful. Instead, I'd assign(*Py_TYPE(iter_o)->tp_iternext) to a variable and check if it's NULL. If it is NULL, we should raise a TypeError. Your code instead makes it immediately end the loop, which doesn't feel like the right behavior.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Hmm, I've tried to add a TypeError
Such piece of code is provided:
PyTypeObject *type = Py_TYPE(iter_o); iternextfunc iternext = type->tp_iternext; if (iternext == NULL) { _PyErr_Format(tstate, PyExc_TypeError, "'for' requires an object with " "__iter__ method, got %.100s", type->tp_name); DECREF_INPUTS(); ERROR_IF(true, error); } PyObject *next_o = (*iternext)(iter_o);And there is a temporarily result of my test case:
-> % ./python -m unittest -v test.test_generators.GeneratorTest.test_issue125038test_issue125038 (test.test_generators.GeneratorTest.test_issue125038) ... ERROR======================================================================ERROR: test_issue125038 (test.test_generators.GeneratorTest.test_issue125038)----------------------------------------------------------------------Traceback (most recent call last): File "/home/mikhail.efimov/projects/cpython/Lib/test/test_generators.py", line 274, in test_issue125038 l = list(g) File "/home/mikhail.efimov/projects/cpython/Lib/test/test_generators.py", line 272, in <genexpr> g = (x for x in range(10)) ~~~~~^^^^TypeError: 'for' requires an object with __iter__ method, got range----------------------------------------------------------------------Ran 1 test in 0.001sIt seems like such a message doesn't provide any clarity.
Do you have any better suggestions about error message?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
That seems like a pretty good error message, much better than silently stopping the loop.
Lib/test/test_generators.py Outdated
| exceptTypeError: | ||
| return"TypeError" | ||
| # This should not raise |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I think you wantwith self.assertRaisesRegex.
SIGSEGV on generators in case of gi_frame.f_locals is fixed.This applies to _FOR_ITER bytecode.Similar checks are added to _FOR_ITER_TIER_TWO and INSTRUMENTED_FOR_ITER bytecode implementations.
44a03a1 toc353cf1Comparegaogaotiantian commentedOct 7, 2024 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
We should confirm with@markshannon whether we should fix the interpreter or |
JelleZijlstra commentedOct 7, 2024
I looked at PEP 668 and it seems to assume that mutating
Yes, that's right; we're adding a NULL check and a branch. However, FOR_ITER has a couple of specialized versions, so the overhead should be minimal when iterating over the most common types. Still, it may be worth benchmarking how bad the overhead is. |
JelleZijlstra left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
This looks correct to me, but will leave to Mark to see if there's a better solution.
Misc/NEWS.d/next/Core_and_Builtins/2024-10-07-19-26-50.gh-issue-125038.ffSLCz.rst OutdatedShow resolvedHide resolved
Uh oh!
There was an error while loading.Please reload this page.
…e-125038.ffSLCz.rstNews improvementCo-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
efimov-mikhail commentedOct 8, 2024
Thanks for your advice, it definitely looks better now. But I have a question. |
efimov-mikhail commentedOct 8, 2024 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
It seems that this code is incorrect now: But very similar code is correct: Maybe, some change in framelocalsproxy_setitem function can be provided? I've provided this change, but it breaks two tests: test_proxy_key_stringlikes_overwrite and test_proxy_key_stringlikes_ftrst_write. Those tests are authored by@encukou and@ncoghlan. |
JelleZijlstra commentedOct 8, 2024
Setting I would oppose adding special handling in the FrameLocalsProxy that mutates objects; doing so would complicate the implementation and make it less useful for tools like debuggers. |
efimov-mikhail commentedOct 8, 2024
Actually, there is no real world use case when changing underlying iterator for a generator is needed. Main purpose of my questions is achieving some clarity. |
…cpython into sigsegv_fix_in_iterators
efimov-mikhail commentedOct 9, 2024
I've moved my test from test_generators.py to test_frame.py, renamed it, and added more test cases to emphasize current code behavior and save it explicitly in tests. |
… section for assignment expression topic (python#125074)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>Co-authored-by: Tomas R <tomas.roun8@gmail.com>
…re enabled by default (python#123859)
…ython#124657)*pythongh-124612: Use ghcr.io/python/autoconf instead of public image* Update
…ython#124669)Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: musvaage <musvaage@users.noreply.github.com>
* Replace unicode_compare_eq() with unicode_eq().* Use unicode_eq() in setobject.c.* Replace _PyUnicode_EQ() with _PyUnicode_Equal().* Remove unicode_compare_eq() and _PyUnicode_EQ().
…008 implementation issues (python#125151)Skip test_fma_zero_result on NetBSD due to IEE 754-2008 implementation issues
…124974)Now it returns a tuple of up to 100 strings (an empty tuple on most locales).Previously it returned the first item of that tuple or an empty string.
…pire far in the future by default (pythonGH-107594)This allows testing Y2038 with system time set to after that,so that actual Y2038 issues can be exposed, and not maskedby expired certificate errors.Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
efimov-mikhail commentedOct 9, 2024
See#125178 |
Uh oh!
There was an error while loading.Please reload this page.
SIGSEGV on generators in case of gi_frame.f_locals is fixed.
This applies to _FOR_ITER bytecode implementation.
Similar checks are added to _FOR_ITER_TIER_TWO and INSTRUMENTED_FOR_ITER bytecode implementations.