Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

bpo-31861: Provide aiter and anext builtins#23847

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
gvanrossum merged 41 commits intopython:masterfromjab:justin39/aiter-c
Mar 23, 2021

Conversation

@jab
Copy link
Contributor

@jabjab commentedDec 18, 2020
edited
Loading

This is the C implementation forbpo-31861 requested as an alternative to the Python implementation I provided in#8895.

For a more direct translation of this into Python (in case it makes reviewing easier), seejab@ce35092.

Patch by@justin39,@lordmauve, and me.

https://bugs.python.org/issue31861

dimaqq, Fidget-Spinner, belm0, achimnol, asleep-cult, greyli, and lmmx reacted with heart emojidimaqq reacted with rocket emoji
jaband others added20 commitsNovember 30, 2020 09:41
...on top of latest master.Also drop now-removed `loop` kwarg from asyncio.sleep call.Ref:https://bugs.python.org/issue42392
@the-knights-who-say-ni

Hello, and thanks for your contribution!

I'm a bot set up to make sure that the project can legally accept this contribution by verifying everyone involved has signed thePSF contributor agreement (CLA).

Recognized GitHub username

We couldn't find abugs.python.org (b.p.o) account corresponding to the following GitHub usernames:

@justin39

This might be simply due to a missing "GitHub Name" entry in one's b.p.o account settings. This is necessary for legal reasons before we can look at this contribution. Please followthe steps outlined in the CPython devguide to rectify this issue.

You cancheck yourself to see if the CLA has been received.

Thanks again for the contribution, we look forward to reviewing it!

@jab
Copy link
ContributorAuthor

jab commentedFeb 26, 2021

Shoot, noticed this PR now has conflicts. Will resolve and push a new revision ASAP.

@justin39
Copy link
Contributor

Resolved the conflicts - hopefully we can get this reviewed before a new conflict pops up!

* master: (129 commits)  bpo-43452: Micro-optimizations to PyType_Lookup (pythonGH-24804)  bpo-43517: Fix false positive in detection of circular imports (python#24895)  bpo-43494: Make some minor changes to lnotab notes (pythonGH-24861)  Mention that code.co_lnotab is deprecated in what's new for 3.10. (python#24902)  bpo-43244: Remove symtable.h header file (pythonGH-24910)  bpo-43466: Add --with-openssl-rpath configure option (pythonGH-24820)  Fix a typo in c-analyzer (pythonGH-24468)  bpo-41561: Add workaround for Ubuntu's custom security level (pythonGH-24915)  bpo-43521: Allow ast.unparse with empty sets and NaN (pythonGH-24897)  bpo-43244: Remove the PyAST_Validate() function (pythonGH-24911)  bpo-43541: Fix PyEval_EvalCodeEx() regression (pythonGH-24918)  bpo-43244: Fix test_peg_generators on Windows (pythonGH-24913)  bpo-39342: Expose X509_V_FLAG_ALLOW_PROXY_CERTS in ssl module (pythonGH-18011)  bpo-43244: Fix test_peg_generator for PyAST_Validate() (pythonGH-24912)  bpo-42128: Add 'missing :' syntax error message to match statements (pythonGH-24733)  bpo-43244: Add pycore_ast.h header file (pythonGH-24908)  bpo-43244: Rename pycore_ast.h to pycore_ast_state.h (pythonGH-24907)  Remove unnecessary imports in the grammar parser (pythonGH-24904)  bpo-35883: Py_DecodeLocale() escapes invalid Unicode characters (pythonGH-24843)  Add PEP 626 to what's new in 3.10. (python#24892)  ...
@jab
Copy link
ContributorAuthor

jab commentedMar 20, 2021
edited
Loading

Merged in latest master and fixed some minor nits. All checks have passed against the latest revision.

UPDATE (ICYMI): There is now ongoing discussion inthis recent python-dev thread about whether to merge this PR (favored by@1st1, and perhaps others who've ❤️'d and participated in this PR), or whether to instead resurrect#8895, my PR from 2018 that addedoperator.aiter andoperator.anext (favored by@ramalho,@pbryan,@gvanrossum,@brettcannon,@terryjreedy). Looking forward to seeing what the core developers decide, and then hopefully being able to merge one of these two PRs before the upcoming 3.10 feature freeze on May 3. Thank you!

Copy link
Member

@gvanrossumgvanrossum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

The basic code is solid. Some questions and suggestions about how much needs to be public and documented.

Comment on lines 66 to 67
Return an:term:`asynchronous iterator`. This is the async variant
of the:func:`iter` builtin, and behaves similarly.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Frankly, this doesn't tell me much. The description doesn't even state thataiter(x) is equivalent tox.__aiter__(), which to me is the key point. Certainly it shouldn't start by stating the type of what it returns; it should describe how the return value relates to the input. (Compare the entry for abs(x) above, "Return the absolute value of a number." This clearly references the input and what the function does to that value.)

Also, state explicitly thataiter(aiter(x)) is the same asaiter(x) (IOW thataiter(x) itself has an__aiter__() method that returnsself.)

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Incorporated in the latest revision, but please let me know if it needs further refinement. Thanks for the great feedback!

/* Takes an AsyncIterable object and returns an AsyncIterator for it.
This is typically a new iterator but if the argument is an AsyncIterator,
this returns itself. */
PyAPI_FUNC(PyObject*)PyObject_GetAiter(PyObject*);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Is it necessary to add these to the public C API? Just becausePyObject_GetIter() is public I'm not sure that the Aiter variant needs to be.@vstinner tends to push back on adding new things to the C API.@1st1 what do you think?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I don't mind adding this function -- while somewhat trivial, it's something that projects like Cython (and potentially our own modules like_asynciomodule.c) have to reimplement.

gvanrossum reacted with thumbs up emoji

PyAPI_DATA(PyTypeObject)PySeqIter_Type;
PyAPI_DATA(PyTypeObject)PyCallIter_Type;
PyAPI_DATA(PyTypeObject)PyAsyncCallAwaitable_Type;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Maybe these types and the function below should remain CPython implementation details? Just because they're returned by builtins doesn't mean all the implementation types need to be in the C-level API.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

+1. I'd make them internal.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Makes sense, thanks. Done in the latest revision. Another look?

Copy link
Member

@gvanrossumgvanrossum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Almost there!

Comment on lines 66 to 72
Equivalent to calling ``x.__aiter__()``.

*async_iterable* must be an:term:`asynchronous iterable`,
and:func:`aiter` returnsanasynchronous iterator for it.
``aiter(aiter(x))`` is the same as ``aiter(x)``.
(``aiter(x)`` itself hasan``__aiter__()`` method that returns ``self``.)

Unlike the:func:`iter` builtin,:func:`aiter` has no 2-argument variant.
Often, this variant can be replaced with assignment expressions::
Formally, given an:term:`asynchronous iterable`,
return an:term:`asynchronous iterator`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Much better! Here is how I would rearrange this:

Return an :term:`asynchronous iterator`for an :term:`asynchronous iterable`. Equivalent to calling ``x.__aiter__()``.``aiter(x)`` itself has an ``__aiter__()`` method that returns ``x``,so ``aiter(aiter(x))`` is the same as ``aiter(x)``.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Incorporated in the latest revision, thanks!

(((
Come to think of it,iter also has theiter(iter(x)) == iter(x) property, which is not currently mentioned intheiter docs. Theiter docs have a lot more work to do though, to cover the 1- and 2-arg variants (and they're already doing this very well).

And now that I'm looking at those again, I notice the only code example there is for the 2-arg variant:

fromfunctoolsimportpartialwithopen('mydata.db','rb')asf:forblockiniter(partial(f.read,64),b''):process_block(block)

If this pattern is now obsoleted by assignment expressions[1], is it worth updating theiter docs to (1) mention theiter(iter(x)) == iter(x) property, and (2) remove the obsoleted example code? If so, happy to submit a separate PR for that.

[1] as in the following:

withopen('mydata.db','rb')asf:whileblock:=f.read(64):process_block(block)

)))

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I don't want to personally get into the weeds about the iter() docs, sorry. Something for the docs WG perhaps.

jab reacted with thumbs up emoji
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I don't think it's worth updating the docs to promiseiter(iter(x)) == iter(x) as it isn't a specific benefit to the user beyond logically doing the right thing.

Copy link
ContributorAuthor

@jabjabMar 23, 2021
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I can see that.

@gvanrossum, do you think@brettcannon's rationale applies equally to theaiter docs too? I did add atest_aiter_idempotent() for this in 6ee8824 to go along with the promise that theaiter docs are now making, but can remove that test along with that part of the docs if that's better.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I think there's a different bar for updating theiter() docs than for the initial version of theaiter() docs. And they don't have to match precisely.

FWIW the main reason the idempotency property is important is because of the implicit[a]iter() call in a for-loop, since in

for i in x: ...

the for-loop callsiter(x), so that in

for i in iter(x): ...

the for-loop ends up callingiter(iter(x)). It's the same forasync for andaiter().

jab reacted with thumbs up emoji
Copy link
Member

@gvanrossumgvanrossum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thanks for your patience. This all looks good to me now! I'll merge after applying my own suggestion.

jab reacted with heart emoji
@gvanrossumgvanrossum merged commitf0a6fde intopython:masterMar 23, 2021
@bedevere-bot
Copy link

@gvanrossum: Please replace# withGH- in the commit message next time. Thanks!

@gvanrossum
Copy link
Member

Congrats!

justin39 and jab reacted with hooray emoji

@jab
Copy link
ContributorAuthor

jab commentedMar 23, 2021

Thanks so much,@gvanrossum! We’re honored to have been able to contribute!

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure⚠️⚠️⚠️

Hi! The buildbots390x RHEL8 3.x has failed when building commitf0a6fde.

What do you need to do:

  1. Don't panic.
  2. Checkthe buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/509/builds/900) and take a look at the build logs.
  4. Check if the failure is related to this commit (f0a6fde) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/509/builds/900

Summary of the results of the build (if available):

== Tests result: ENV CHANGED ==

412 tests OK.

10 slowest tests:

  • test_concurrent_futures: 3 min 9 sec
  • test_multiprocessing_spawn: 1 min 36 sec
  • test_peg_generator: 1 min 26 sec
  • test_capi: 1 min 17 sec
  • test_gdb: 1 min 14 sec
  • test_unparse: 1 min 12 sec
  • test_multiprocessing_forkserver: 1 min 10 sec
  • test_asyncio: 1 min 5 sec
  • test_tokenize: 1 min 3 sec
  • test_multiprocessing_fork: 1 min

1 test altered the execution environment:
test_asyncio

14 tests skipped:
test_devpoll test_ioctl test_kqueue test_msilib test_nis
test_ossaudiodev test_startfile test_tix test_tk test_ttk_guionly
test_winconsoleio test_winreg test_winsound test_zipfile64

Total duration: 5 min 18 sec

Click to see traceback logs
Traceback (most recent call last):  File"/home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z/build/Lib/asyncio/sslproto.py", line321, in__del__self.close()  File"/home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z/build/Lib/asyncio/sslproto.py", line316, incloseself._ssl_protocol._start_shutdown()  File"/home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z/build/Lib/asyncio/sslproto.py", line590, in_start_shutdownself._abort()  File"/home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z/build/Lib/asyncio/sslproto.py", line731, in_abortself._transport.abort()  File"/home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z/build/Lib/asyncio/selector_events.py", line680, inabortself._force_close(None)  File"/home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z/build/Lib/asyncio/selector_events.py", line731, in_force_closeself._loop.call_soon(self._call_connection_lost, exc)  File"/home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z/build/Lib/asyncio/base_events.py", line745, incall_soonself._check_closed()  File"/home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z/build/Lib/asyncio/base_events.py", line510, in_check_closedraiseRuntimeError('Event loop is closed')RuntimeError:Event loop is closed

@gvanrossum
Copy link
Member

gvanrossum commentedMar 23, 2021 via email

I believe test_asyncio is unstable or broken on this platform.

@vstinner
Copy link
Member

I proposed to PR#25266 to rename PyAnextAwaitable_Type to _PyAnextAwaitable_Type, and to initialize the type at Python startup: can someone please have a look?

@jabjab changed the titlebpo-31861: Add aiter and anext to builtinsbpo-31861: Provide aiter and anext builtinsMay 22, 2024
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@brettcannonbrettcannonbrettcannon left review comments

@nanjekyejoannahnanjekyejoannahnanjekyejoannah left review comments

@gvanrossumgvanrossumgvanrossum approved these changes

@1st11st1Awaiting requested review from 1st1

+4 more reviewers

@eric-wiesereric-wiesereric-wieser left review comments

@lordmauvelordmauvelordmauve left review comments

@justin39justin39justin39 left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

12 participants

@jab@the-knights-who-say-ni@Fidget-Spinner@justin39@bedevere-bot@gvanrossum@vstinner@brettcannon@1st1@eric-wieser@lordmauve@nanjekyejoannah

[8]ページ先頭

©2009-2025 Movatter.jp