Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork33.4k
bpo-31861: Provide aiter and anext builtins#23847
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
...on top of latest master.Also drop now-removed `loop` kwarg from asyncio.sleep call.Ref:https://bugs.python.org/issue42392
the-knights-who-say-ni commentedDec 18, 2020
Hello, and thanks for your contribution! I'm a bot set up to make sure that the project can legally accept this contribution by verifying everyone involved has signed thePSF contributor agreement (CLA). Recognized GitHub usernameWe couldn't find abugs.python.org (b.p.o) account corresponding to the following GitHub usernames: This might be simply due to a missing "GitHub Name" entry in one's b.p.o account settings. This is necessary for legal reasons before we can look at this contribution. Please followthe steps outlined in the CPython devguide to rectify this issue. You cancheck yourself to see if the CLA has been received. Thanks again for the contribution, we look forward to reviewing it! |
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
jab commentedFeb 26, 2021
Shoot, noticed this PR now has conflicts. Will resolve and push a new revision ASAP. |
justin39 commentedFeb 26, 2021
Resolved the conflicts - hopefully we can get this reviewed before a new conflict pops up! |
* master: (129 commits) bpo-43452: Micro-optimizations to PyType_Lookup (pythonGH-24804) bpo-43517: Fix false positive in detection of circular imports (python#24895) bpo-43494: Make some minor changes to lnotab notes (pythonGH-24861) Mention that code.co_lnotab is deprecated in what's new for 3.10. (python#24902) bpo-43244: Remove symtable.h header file (pythonGH-24910) bpo-43466: Add --with-openssl-rpath configure option (pythonGH-24820) Fix a typo in c-analyzer (pythonGH-24468) bpo-41561: Add workaround for Ubuntu's custom security level (pythonGH-24915) bpo-43521: Allow ast.unparse with empty sets and NaN (pythonGH-24897) bpo-43244: Remove the PyAST_Validate() function (pythonGH-24911) bpo-43541: Fix PyEval_EvalCodeEx() regression (pythonGH-24918) bpo-43244: Fix test_peg_generators on Windows (pythonGH-24913) bpo-39342: Expose X509_V_FLAG_ALLOW_PROXY_CERTS in ssl module (pythonGH-18011) bpo-43244: Fix test_peg_generator for PyAST_Validate() (pythonGH-24912) bpo-42128: Add 'missing :' syntax error message to match statements (pythonGH-24733) bpo-43244: Add pycore_ast.h header file (pythonGH-24908) bpo-43244: Rename pycore_ast.h to pycore_ast_state.h (pythonGH-24907) Remove unnecessary imports in the grammar parser (pythonGH-24904) bpo-35883: Py_DecodeLocale() escapes invalid Unicode characters (pythonGH-24843) Add PEP 626 to what's new in 3.10. (python#24892) ...
jab commentedMar 20, 2021 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
Merged in latest master and fixed some minor nits. All checks have passed against the latest revision. UPDATE (ICYMI): There is now ongoing discussion inthis recent python-dev thread about whether to merge this PR (favored by@1st1, and perhaps others who've ❤️'d and participated in this PR), or whether to instead resurrect#8895, my PR from 2018 that added |
gvanrossum left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
The basic code is solid. Some questions and suggestions about how much needs to be public and documented.
Uh oh!
There was an error while loading.Please reload this page.
Doc/library/functions.rst Outdated
| Return an:term:`asynchronous iterator`. This is the async variant | ||
| of the:func:`iter` builtin, and behaves similarly. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Frankly, this doesn't tell me much. The description doesn't even state thataiter(x) is equivalent tox.__aiter__(), which to me is the key point. Certainly it shouldn't start by stating the type of what it returns; it should describe how the return value relates to the input. (Compare the entry for abs(x) above, "Return the absolute value of a number." This clearly references the input and what the function does to that value.)
Also, state explicitly thataiter(aiter(x)) is the same asaiter(x) (IOW thataiter(x) itself has an__aiter__() method that returnsself.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Incorporated in the latest revision, but please let me know if it needs further refinement. Thanks for the great feedback!
Uh oh!
There was an error while loading.Please reload this page.
| /* Takes an AsyncIterable object and returns an AsyncIterator for it. | ||
| This is typically a new iterator but if the argument is an AsyncIterator, | ||
| this returns itself. */ | ||
| PyAPI_FUNC(PyObject*)PyObject_GetAiter(PyObject*); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I don't mind adding this function -- while somewhat trivial, it's something that projects like Cython (and potentially our own modules like_asynciomodule.c) have to reimplement.
Include/iterobject.h Outdated
| PyAPI_DATA(PyTypeObject)PySeqIter_Type; | ||
| PyAPI_DATA(PyTypeObject)PyCallIter_Type; | ||
| PyAPI_DATA(PyTypeObject)PyAsyncCallAwaitable_Type; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Maybe these types and the function below should remain CPython implementation details? Just because they're returned by builtins doesn't mean all the implementation types need to be in the C-level API.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
+1. I'd make them internal.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Makes sense, thanks. Done in the latest revision. Another look?
Uh oh!
There was an error while loading.Please reload this page.
gvanrossum left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Almost there!
Doc/library/functions.rst Outdated
| Equivalent to calling ``x.__aiter__()``. | ||
| *async_iterable* must be an:term:`asynchronous iterable`, | ||
| and:func:`aiter` returnsanasynchronous iterator for it. | ||
| ``aiter(aiter(x))`` is the same as ``aiter(x)``. | ||
| (``aiter(x)`` itself hasan``__aiter__()`` method that returns ``self``.) | ||
| Unlike the:func:`iter` builtin,:func:`aiter` has no 2-argument variant. | ||
| Often, this variant can be replaced with assignment expressions:: | ||
| Formally, given an:term:`asynchronous iterable`, | ||
| return an:term:`asynchronous iterator`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Much better! Here is how I would rearrange this:
Return an :term:`asynchronous iterator`for an :term:`asynchronous iterable`. Equivalent to calling ``x.__aiter__()``.``aiter(x)`` itself has an ``__aiter__()`` method that returns ``x``,so ``aiter(aiter(x))`` is the same as ``aiter(x)``.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Incorporated in the latest revision, thanks!
(((
Come to think of it,iter also has theiter(iter(x)) == iter(x) property, which is not currently mentioned intheiter docs. Theiter docs have a lot more work to do though, to cover the 1- and 2-arg variants (and they're already doing this very well).
And now that I'm looking at those again, I notice the only code example there is for the 2-arg variant:
fromfunctoolsimportpartialwithopen('mydata.db','rb')asf:forblockiniter(partial(f.read,64),b''):process_block(block)
If this pattern is now obsoleted by assignment expressions[1], is it worth updating theiter docs to (1) mention theiter(iter(x)) == iter(x) property, and (2) remove the obsoleted example code? If so, happy to submit a separate PR for that.
[1] as in the following:
withopen('mydata.db','rb')asf:whileblock:=f.read(64):process_block(block)
)))
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I don't want to personally get into the weeds about the iter() docs, sorry. Something for the docs WG perhaps.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I don't think it's worth updating the docs to promiseiter(iter(x)) == iter(x) as it isn't a specific benefit to the user beyond logically doing the right thing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I can see that.
@gvanrossum, do you think@brettcannon's rationale applies equally to theaiter docs too? I did add atest_aiter_idempotent() for this in 6ee8824 to go along with the promise that theaiter docs are now making, but can remove that test along with that part of the docs if that's better.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I think there's a different bar for updating theiter() docs than for the initial version of theaiter() docs. And they don't have to match precisely.
FWIW the main reason the idempotency property is important is because of the implicit[a]iter() call in a for-loop, since in
for i in x: ...the for-loop callsiter(x), so that in
for i in iter(x): ...the for-loop ends up callingiter(iter(x)). It's the same forasync for andaiter().
gvanrossum left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Thanks for your patience. This all looks good to me now! I'll merge after applying my own suggestion.
Uh oh!
There was an error while loading.Please reload this page.
bedevere-bot commentedMar 23, 2021
@gvanrossum: Please replace |
gvanrossum commentedMar 23, 2021
Congrats! |
jab commentedMar 23, 2021
Thanks so much,@gvanrossum! We’re honored to have been able to contribute! |
bedevere-bot commentedMar 23, 2021
|
gvanrossum commentedMar 23, 2021 via email
I believe test_asyncio is unstable or broken on this platform. |
vstinner commentedApr 7, 2021
I proposed to PR#25266 to rename PyAnextAwaitable_Type to _PyAnextAwaitable_Type, and to initialize the type at Python startup: can someone please have a look? |
Uh oh!
There was an error while loading.Please reload this page.
This is the C implementation forbpo-31861 requested as an alternative to the Python implementation I provided in#8895.
For a more direct translation of this into Python (in case it makes reviewing easier), seejab@ce35092.
Patch by@justin39,@lordmauve, and me.
https://bugs.python.org/issue31861