Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

gh-132969: Fix error/hang when shutdown(wait=False) and task exited abnormally#133222

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
encukou merged 31 commits intopython:mainfromogbiggles:origin/main
Jun 10, 2025

Conversation

ogbiggles
Copy link
Contributor

@ogbigglesogbiggles commentedApr 30, 2025•
edited by bedevere-appbot
Loading

When shutdown is called with wait=False, the executor thread keeps running even after the ProcessPoolExecutor's state is reset. The executor then tries to replenish the worker processes pool resulting in an error and a potential hang when it comes across a worker that has died. Fixed the issue by having _adjust_process_count() return without doing anything if the ProcessPoolExecutor's state has been reset.

Added unit tests to validate two scenarios:
max_workers < num_tasks (exception)
max_workers > num_tasks (exception + hang)

…ted abnormallyWhen shutdown is called with wait=False, the executor thread keeps runningeven after the ProcessPoolExecutor's state is reset. The executor then triesto replenish the worker processes pool resulting in an error and a potential hangwhen it comes across a worker that has died. Fixed the issue by having_adjust_process_count() return without doing anything if the ProcessPoolExecutor'sstate has been reset.Added unit tests to validate two scenarios:max_workers < num_tasks (exception)max_workers > num_tasks (exception + hang)
@bedevere-app
Copy link

Most changes to Pythonrequire a NEWS entry. Add one using theblurb_it web app or theblurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply theskip news label instead.

@python-cla-bot
Copy link

python-cla-botbot commentedApr 30, 2025•
edited
Loading

All commit authors signed the Contributor License Agreement.

CLA signed

@encukou
Copy link
Member

@hugovk: If this is indeed a blocker for the beta, +1 for me -- it shouldn'tbreak anything.
For a proper review, I'd need more time to familiarize myself with this part of the code. (@gpshead, should I?)

But, as week-old issue that affects 3.12+, I think it's OK to defer it.

@itamaro: Did you intend this to block the beta?

@itamaro
Copy link
Contributor

Did you intend this to block the beta?

Should have said it here, not only in Discord :)
I wasn't sure whether this should block the release, so added the tag to make sure someone at least takes a look, considering beta 1 is imminent!

@hugovk
Copy link
Member

Thanks all, it's been looked at and sounds like it can wait, so let's defer it.

itamaro reacted with thumbs up emoji

@tabrezm
Copy link

Does this fix supersedethis one?

@ogbiggles
Copy link
ContributorAuthor

Does this fix supersedethis one?

The two are addressing different issues, the referenced PR will not prevent the error that I am fixing in this one.

tabrezm reacted with thumbs up emoji

@encukouencukou added the 🔨 test-with-buildbotsTest PR w/ buildbots; report in status section labelJun 9, 2025
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by@encukou for commit4f46875 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F133222%2Fmerge

If you want to schedule another build, you need to add the🔨 test-with-buildbots label again.

@bedevere-botbedevere-bot removed the 🔨 test-with-buildbotsTest PR w/ buildbots; report in status section labelJun 9, 2025
Copy link
Member

@encukouencukou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thank you! I started the buildbots; if they like the PR I'll merge it.

(Don't be alarmed if there are failures; not all buildbots are stable.)

@encukouencukou merged commit598aa7c intopython:mainJun 10, 2025
123 of 128 checks passed
@encukouencukou added needs backport to 3.13bugs and security fixes needs backport to 3.14bugs and security fixes labelsJun 10, 2025
@miss-islington-app
Copy link

Thanks@ogbiggles for the PR, and@encukou for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

@miss-islington-app
Copy link

Thanks@ogbiggles for the PR, and@encukou for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull requestJun 10, 2025
…ited abnormally (pythonGH-133222)When shutdown is called with wait=False, the executor thread keeps runningeven after the ProcessPoolExecutor's state is reset. The executor then triesto replenish the worker processes pool resulting in an error and a potential hangwhen it comes across a worker that has died. Fixed the issue by having_adjust_process_count() return without doing anything if the ProcessPoolExecutor'sstate has been reset.Added unit tests to validate two scenarios:max_workers < num_tasks (exception)max_workers > num_tasks (exception + hang)(cherry picked from commit598aa7c)Co-authored-by: Ajay Kamdar <140011370+ogbiggles@users.noreply.github.com>
@bedevere-app
Copy link

GH-135343 is a backport of this pull request to the3.13 branch.

@bedevere-appbedevere-appbot removed the needs backport to 3.13bugs and security fixes labelJun 10, 2025
miss-islington pushed a commit to miss-islington/cpython that referenced this pull requestJun 10, 2025
…ited abnormally (pythonGH-133222)When shutdown is called with wait=False, the executor thread keeps runningeven after the ProcessPoolExecutor's state is reset. The executor then triesto replenish the worker processes pool resulting in an error and a potential hangwhen it comes across a worker that has died. Fixed the issue by having_adjust_process_count() return without doing anything if the ProcessPoolExecutor'sstate has been reset.Added unit tests to validate two scenarios:max_workers < num_tasks (exception)max_workers > num_tasks (exception + hang)(cherry picked from commit598aa7c)Co-authored-by: Ajay Kamdar <140011370+ogbiggles@users.noreply.github.com>
@bedevere-app
Copy link

GH-135344 is a backport of this pull request to the3.14 branch.

@bedevere-appbedevere-appbot removed the needs backport to 3.14bugs and security fixes labelJun 10, 2025
encukou pushed a commit that referenced this pull requestJun 13, 2025
…xited abnormally (GH-133222) (GH-135344)gh-132969:  Fix error/hang when shutdown(wait=False) and task exited abnormally (GH-133222)When shutdown is called with wait=False, the executor thread keeps runningeven after the ProcessPoolExecutor's state is reset. The executor then triesto replenish the worker processes pool resulting in an error and a potential hangwhen it comes across a worker that has died. Fixed the issue by having_adjust_process_count() return without doing anything if the ProcessPoolExecutor'sstate has been reset.Added unit tests to validate two scenarios:max_workers < num_tasks (exception)max_workers > num_tasks (exception + hang)(cherry picked from commit598aa7c)Co-authored-by: Ajay Kamdar <140011370+ogbiggles@users.noreply.github.com>
encukou pushed a commit that referenced this pull requestJun 16, 2025
…xited abnormally (GH-133222) (GH-135343)gh-132969:  Fix error/hang when shutdown(wait=False) and task exited abnormally (GH-133222)When shutdown is called with wait=False, the executor thread keeps runningeven after the ProcessPoolExecutor's state is reset. The executor then triesto replenish the worker processes pool resulting in an error and a potential hangwhen it comes across a worker that has died. Fixed the issue by having_adjust_process_count() return without doing anything if the ProcessPoolExecutor'sstate has been reset.Added unit tests to validate two scenarios:max_workers < num_tasks (exception)max_workers > num_tasks (exception + hang)(cherry picked from commit598aa7c)Co-authored-by: Ajay Kamdar <140011370+ogbiggles@users.noreply.github.com>
lkollar pushed a commit to lkollar/cpython that referenced this pull requestJun 19, 2025
…ited abnormally (pythonGH-133222)When shutdown is called with wait=False, the executor thread keeps runningeven after the ProcessPoolExecutor's state is reset. The executor then triesto replenish the worker processes pool resulting in an error and a potential hangwhen it comes across a worker that has died. Fixed the issue by having_adjust_process_count() return without doing anything if the ProcessPoolExecutor'sstate has been reset.Added unit tests to validate two scenarios:max_workers < num_tasks (exception)max_workers > num_tasks (exception + hang)
ogbiggles added a commit to ogbiggles/cpython-update-ack-for-gh-132969 that referenced this pull requestJun 30, 2025
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@YvesDupYvesDupYvesDup left review comments

@picnixzpicnixzpicnixz left review comments

@encukouencukouencukou approved these changes

Assignees
No one assigned
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

9 participants
@ogbiggles@encukou@itamaro@hugovk@YvesDup@tabrezm@bedevere-bot@picnixz@ZeroIntensity
[8]ページ先頭
©2009-2025 Movatter.jp