Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork32k
gh-125400: add async generator return value#125401
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
base:main
Are you sure you want to change the base?
Uh oh!
There was an error while loading.Please reload this page.
Conversation
ghost commentedOct 13, 2024 • edited by ghost
Loading Uh oh!
There was an error while loading.Please reload this page.
edited by ghost
Uh oh!
There was an error while loading.Please reload this page.
All commit authors signed the Contributor License Agreement. |
Most changes to Pythonrequire a NEWS entry. Add one using theblurb_it web app or theblurb command-line tool. If this change has little impact on Python users, wait for a maintainer to apply the |
Thanks for writing a reference implementation for this idea as was requested in the Discourse thread; it makes it much easier to evaluate the proposal! I'm marking the PR as |
Most changes to Pythonrequire a NEWS entry. Add one using theblurb_it web app or theblurb command-line tool. If this change has little impact on Python users, wait for a maintainer to apply the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Some initial comments. I do realize that a lot of this is copied from theStopIteration implementation, so it might be better to ignore some of my comments for consistency (or better yet, apply my comments toStopIteration as well 😄)
Overall, I think this is nice for consistency withStopIteration, but I don't think this is particularly useful without support foryield from in async generators, because you can implement this just fine in current versions--just define an exception with avalue attribute, and raise it.
| return -1; | ||
| } | ||
| if (value == NULL) { | ||
| value = Py_NewRef(Py_None); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Py_None is immortal, I don't think we need to incref it. There's some contention about this, though.
| value=Py_NewRef(Py_None); | |
| value=Py_None; |
| PyObject *value = NULL; | ||
| if (PyErr_ExceptionMatches(PyExc_StopAsyncIteration)) { | ||
| PyObject *exc = PyErr_GetRaisedException(); | ||
| value = Py_NewRef(((PyStopAsyncIterationObject *)exc)->value); | ||
| Py_DECREF(exc); | ||
| } else if (PyErr_Occurred()) { | ||
| return -1; | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
There's going to be a lot of internal reusing of_PyThreadState_GET with this approach. You can use the private API:
| PyObject*value=NULL; | |
| if (PyErr_ExceptionMatches(PyExc_StopAsyncIteration)) { | |
| PyObject*exc=PyErr_GetRaisedException(); | |
| value=Py_NewRef(((PyStopAsyncIterationObject*)exc)->value); | |
| Py_DECREF(exc); | |
| }elseif (PyErr_Occurred()) { | |
| return-1; | |
| } | |
| PyObject*value=NULL; | |
| PyThreadState*tstate=_PyThreadState_GET(); | |
| PyObject*occurred=_PyErr_Occurred(tstate); | |
| if (PyErr_GivenExceptionMatches(occurred,PyExc_StopAsyncIteration)) { | |
| PyObject*exc=_PyErr_GetRaisedException(tstate); | |
| value=Py_NewRef(((PyStopAsyncIterationObject*)exc)->value); | |
| Py_DECREF(exc); | |
| }elseif (occurred) { | |
| return-1; | |
| } |
| if (value == NULL || | ||
| (!PyTuple_Check(value) && !PyExceptionInstance_Check(value))) | ||
| { | ||
| /* Delay exception instantiation if we can */ | ||
| PyErr_SetObject(PyExc_StopAsyncIteration, value); | ||
| return 0; | ||
| } | ||
| /* Construct an exception instance manually with | ||
| * PyObject_CallOneArg and pass it to PyErr_SetObject. | ||
| * | ||
| * We do this to handle a situation when "value" is a tuple, in which | ||
| * case PyErr_SetObject would set the value of StopIteration to | ||
| * the first element of the tuple. | ||
| * | ||
| * (See PyErr_SetObject/_PyErr_CreateException code for details.) | ||
| */ | ||
| e = PyObject_CallOneArg(PyExc_StopAsyncIteration, value); | ||
| if (e == NULL) { | ||
| return -1; | ||
| } | ||
| PyErr_SetObject(PyExc_StopAsyncIteration, e); | ||
| Py_DECREF(e); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Same story here--use the thread state.
| if (value==NULL|| | |
| (!PyTuple_Check(value)&& !PyExceptionInstance_Check(value))) | |
| { | |
| /* Delay exception instantiation if we can */ | |
| PyErr_SetObject(PyExc_StopAsyncIteration,value); | |
| return0; | |
| } | |
| /*Constructanexceptioninstancemanuallywith | |
| *PyObject_CallOneArgandpassittoPyErr_SetObject. | |
| * | |
| *Wedothistohandleasituationwhen"value"isatuple,inwhich | |
| *casePyErr_SetObjectwouldsetthevalueofStopIterationto | |
| *thefirstelementofthetuple. | |
| * | |
| * (SeePyErr_SetObject/_PyErr_CreateExceptioncodefordetails.) | |
| */ | |
| e=PyObject_CallOneArg(PyExc_StopAsyncIteration,value); | |
| if (e==NULL) { | |
| return-1; | |
| } | |
| PyErr_SetObject(PyExc_StopAsyncIteration,e); | |
| Py_DECREF(e); | |
| PyThreadState*tstate=_PyThreadState_GET(); | |
| if (value==NULL|| | |
| (!PyTuple_Check(value)&& !PyExceptionInstance_Check(value))) | |
| { | |
| /* Delay exception instantiation if we can */ | |
| _PyErr_SetObject(tstate,PyExc_StopAsyncIteration,value); | |
| return0; | |
| } | |
| /*Constructanexceptioninstancemanuallywith | |
| *PyObject_CallOneArgandpassittoPyErr_SetObject. | |
| * | |
| *Wedothistohandleasituationwhen"value"isatuple,inwhich | |
| *casePyErr_SetObjectwouldsetthevalueofStopIterationto | |
| *thefirstelementofthetuple. | |
| * | |
| * (SeePyErr_SetObject/_PyErr_CreateExceptioncodefordetails.) | |
| */ | |
| e=PyObject_CallOneArg(PyExc_StopAsyncIteration,value); | |
| if (e==NULL) { | |
| return-1; | |
| } | |
| _PyErr_SetObject(tstate,PyExc_StopAsyncIteration,e); | |
| Py_DECREF(e); |
| Py_ssize_t size = PyTuple_GET_SIZE(args); | ||
| PyObject *value; | ||
| if (BaseException_init((PyBaseExceptionObject *)self, args, kwds) == -1) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Typically, the convention is to use< 0 rather than== -1
| if (BaseException_init((PyBaseExceptionObject*)self,args,kwds)==-1) | |
| if (BaseException_init((PyBaseExceptionObject*)self,args,kwds)<0) |
| if (size > 0) | ||
| value = PyTuple_GET_ITEM(args, 0); | ||
| else | ||
| value = Py_None; | ||
| self->value = Py_NewRef(value); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
This can get simplified a little, asNone is immortal like I mentioned:
| if (size>0) | |
| value=PyTuple_GET_ITEM(args,0); | |
| else | |
| value=Py_None; | |
| self->value=Py_NewRef(value); | |
| if (size>0) { | |
| self->value=Py_NewRef(PyTuple_GET_ITEM(args,0)); | |
| } | |
| else { | |
| self->value=Py_None; | |
| } |
| if (BaseException_init((PyBaseExceptionObject *)self, args, kwds) == -1) | ||
| return -1; | ||
| Py_CLEAR(self->value); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
WhyCLEAR it here? It's an extra operation of setting it toNULL when nothing will touch it in between calls here anyway.
| Py_CLEAR(self->value); | |
| Py_DECREF(self->value); |
| if (result == Py_None) { | ||
| PyErr_SetNone(PyExc_StopAsyncIteration); | ||
| } | ||
| else { | ||
| _PyGen_SetStopAsyncIterationValue(result); | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Consider acquiring the thread state here, and then passing that to_PyGen_SetStopAsyncIterationValue (to omit an extra call there)
Lib/test/test_asyncgen.py Outdated
| async def gen(): | ||
| yield 1 | ||
| yield 2 | ||
| return 3 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
It might be a good idea to add a dummy coroutine (something likeasyncio.sleep(0) should work) here, as that needs to get yielded in a different way--in my experience with dealing with the coroutine implementation, a coroutine that awaits nothing behaves differently than one that does.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Thanks so much for your thorough and prompt review. I've updated this to include awaited calls toasyncio.sleep.
Most changes to Pythonrequire a NEWS entry. Add one using theblurb_it web app or theblurb command-line tool. If this change has little impact on Python users, wait for a maintainer to apply the |
pkch commentedFeb 1, 2025 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
I believe the strongest argument in favor of this feature even without a Separately from the above, IIUC,@MadcowD might have been interested in writing a reference implementation for |
Uh oh!
There was an error while loading.Please reload this page.
📚 Documentation preview 📚:https://cpython-previews--125401.org.readthedocs.build/