Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

gh-77714: Implement as_completed as an asynchronous generator#10251

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Closed
mivade wants to merge5 commits intopython:mainfrommivade:async-as-completed

Conversation

mivade
Copy link
Contributor

@mivademivade commentedOct 31, 2018
edited by bedevere-bot
Loading

This implementsasyncio.as_completed withasync for support while keeping the old iteration method for backwards compatibility.

This obviously needs new tests. I was also hoping for some additional feedback regarding the implementation.

https://bugs.python.org/issue33533

exceptions!) of the original Futures (or coroutines), in the order
in which and as soon as they complete.
This differs from PEP 3148; results are yielded from asynchronous
iteration rather than futures.
Copy link
Contributor

@hniksichniksicOct 31, 2018
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

The new implementation should also return futures, just like the one fromconcurrent.futures does. The difference is that with async iteration we can return theoriginal futures, which are guaranteed to have completed.

Changingyield future.result() toyield future in__aiter__ makes it correct, and compatible with PEP 3148.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

The existing implementation ofas_completed already yields results. I also think yielding results rather than futures is more natural withasync for.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

The existing implementation yields futures which you mustawait.

The idea is to yield futures which are guaranteed to have completed, as they complete. This is howconcurrent.futures.as_completed works, but the benefit goes beyond consistency withconcurrent.futures. It allows code to be robust in the face of exceptions raised by futures, and to associate side data with the futures and retrieve it fromas_completed. For example, when yielding futures,this usage example immediately becomes usable withasyncio.as_completed, just changingfor toasync for.

The sync version doesn't support that usage because it gives you adifferent future than any of those passed toas_completed. Among other things, that's what prompted the BPO issue.

As far as I can tell, your implementation can easily support that, just by removing.result() at theyield point.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

You're right, I misread what was going on before. Thanks for the explanation!

Copy link
Contributor

@willingcwillingc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Hi@mivade, Thanks for the PR. Since this PR has been open for a while, would you prefer to close the PR or resolve the CI errors? Thanks.

@mivade
Copy link
ContributorAuthor

It has indeed been quite some time! I should be able to take a look at resolving conflicts later this week if there is not a competing PR that would be ready to merge.

arhadthedev, willingc, and bvd0 reacted with thumbs up emoji

@gvanrossumgvanrossum changed the titlebpo-33533: Implement as_completed as an asynchronous generatorgh-77714: Implement as_completed as an asynchronous generatorMay 7, 2023
@gvanrossum
Copy link
Member

@kumaraditya303 Can you review this? I just don't have the time, and you expressed interest in the GH issue before.

@kumaraditya303kumaraditya303 added the 3.13bugs and security fixes labelMay 10, 2023
@bvd0
Copy link

What needs to happen for this to get merged?

@gvanrossum
Copy link
Member

I’ve asked@serhiy-storchaka for help.

bvd0 reacted with thumbs up emoji

# This is *not* a @coroutine! It is just an iterator (yielding Futures).
def as_completed(fs, *, timeout=None):
"""Return an iterator whose values are coroutines.
class as_completed(object):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Suggested change
classas_completed(object):
classas_completed:

Inheriting from object isn't necessary

bvd0 reacted with thumbs up emoji
@serhiy-storchaka
Copy link
Member

Thank you for your PR, but#22491 is more complete. It returns an iterator (not simply an iterable) and contains tests. So we focused on it.

mivade reacted with thumbs up emoji

@mivade
Copy link
ContributorAuthor

Glad to see there's movement on addressing this issue!

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@hniksichniksichniksic left review comments

@willingcwillingcwillingc left review comments

@Gobot1234Gobot1234Gobot1234 left review comments

@gvanrossumgvanrossumAwaiting requested review from gvanrossum

@kumaraditya303kumaraditya303Awaiting requested review from kumaraditya303kumaraditya303 is a code owner

@serhiy-storchakaserhiy-storchakaAwaiting requested review from serhiy-storchaka

Assignees
No one assigned
Labels
3.13bugs and security fixesawaiting reviewtopic-asyncio
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

12 participants
@mivade@gvanrossum@bvd0@serhiy-storchaka@hniksic@willingc@Gobot1234@iritkatriel@the-knights-who-say-ni@ezio-melotti@bedevere-bot@kumaraditya303

[8]ページ先頭

©2009-2025 Movatter.jp