Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

gh-98552: Fix preloading '__main__' with forkserver being broken for a#99515

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Closed
aggieNick02 wants to merge1 commit intopython:mainfromPCPartPicker:fix-98552

Conversation

@aggieNick02
Copy link

@aggieNick02aggieNick02 commentedNov 15, 2022
edited by bedevere-bot
Loading

long time (9 years).

We do this by using spawn.get_preparation_data() and then using the values in that dictionary appropriately. This lets us also fix the setting of sys.path (which never worked) and other minor properties of the forkserver process.

While updating the test to verify these fixes, we also discovered that forkserver was not flushing stdout/stderr before it forked to create a new process. This caused the updated test to fail because unflushed output in the forkserver would then be printed by the forked process as well. This is now fixed too.

…n for along time (9 years).We do this by using spawn.get_preparation_data() and then using thevalues in that dictionary appropriately. This lets us also fix thesetting of sys.path (which never worked) and other minor properties ofthe forkserver process.While updating the test to verify these fixes, we also discovered thatforkserver was not flushing stdout/stderr before it forked to create anew process. This caused the updated test to fail because unflushedoutput in the forkserver would then be printed by the forked process aswell. This is now fixed too.
@aggieNick02
Copy link
Author

One minor thing I noticed incase@pitrou remembers from several years ago. InLib/test/mp_preload.py, the file begins with animport multiprocessing thenmultiprocessing.Lock() . Any idea what purpose the call to multiprocessing.Lock() serves? Some sort of required initialization?

@pitrou
Copy link
Member

@aggieNick02 I suppose it's a simple check thatmultiprocessing is usable from a preloaded module?

@aggieNick02
Copy link
Author

@aggieNick02 I suppose it's a simple check thatmultiprocessing is usable from a preloaded module?

Ah ok, thanks!

@brettcannonbrettcannon removed their request for reviewNovember 16, 2022 21:47
@thesamesam
Copy link
Contributor

Is this one obsolete ("dupe" of#126631 and#135295)?

gpshead reacted with thumbs up emoji

@gpshead
Copy link
Member

this one is obsolete due to the preloading of main being fixed. but the question of the buffering issue remains, lets take that up on the issue; there may be a new, now smaller, PR worth doing.

gpshead pushed a commit to gpshead/cpython that referenced this pull requestNov 22, 2025
This change adds a keyword-only `raise_exceptions` parameter to`multiprocessing.set_forkserver_preload()` that defaults to Falsefor backward compatibility. When set to True, ImportError exceptionsduring module preloading in the forkserver process will be raisedinstead of being silently ignored.This is similar in spirit to the approach attempted inpythonGH-99515,providing developers with the ability to catch import errors duringforkserver module preloading for better debugging and error handling.Changes:- Add _raise_exceptions attribute to ForkServer class- Update set_forkserver_preload() to accept raise_exceptions parameter- Pass raise_exceptions flag through to forkserver main() function- Update main() to conditionally raise ImportError based on flag- Add comprehensive test coverage in test_multiprocessing_forkserver- Update documentation in Doc/library/multiprocessing.rst- Add NEWS entry for Python 3.14
gpshead pushed a commit to gpshead/cpython that referenced this pull requestNov 22, 2025
This change adds a keyword-only `raise_exceptions` parameter to`multiprocessing.set_forkserver_preload()` that defaults to Falsefor backward compatibility. When set to True, ImportError exceptionsduring module preloading in the forkserver process will be raisedinstead of being silently ignored.This is similar in spirit to the approach attempted inpythonGH-99515,providing developers with the ability to catch import errors duringforkserver module preloading for better debugging and error handling.Changes:- Add _raise_exceptions attribute to ForkServer class- Update set_forkserver_preload() to accept raise_exceptions parameter- Pass raise_exceptions flag through to forkserver main() function- Update main() to conditionally raise ImportError based on flag- Add comprehensive test coverage in test_multiprocessing_forkserver- Update documentation in Doc/library/multiprocessing.rst- Add NEWS entry for Python 3.14
gpshead pushed a commit to gpshead/cpython that referenced this pull requestNov 22, 2025
This change adds a keyword-only `raise_exceptions` parameter to`multiprocessing.set_forkserver_preload()` that defaults to Falsefor backward compatibility. When set to True, ImportError exceptionsduring module preloading in the forkserver process will be raisedinstead of being silently ignored.This is similar in spirit to the approach attempted inpythonGH-99515,providing developers with the ability to catch import errors duringforkserver module preloading for better debugging and error handling.Changes:- Add _raise_exceptions attribute to ForkServer class- Update set_forkserver_preload() to accept raise_exceptions parameter- Pass raise_exceptions flag through to forkserver main() function- Update main() to conditionally raise ImportError based on flag- Add comprehensive test coverage in test_multiprocessing_forkserver- Update documentation in Doc/library/multiprocessing.rst- Add NEWS entry for Python 3.14
gpshead pushed a commit to gpshead/cpython that referenced this pull requestNov 22, 2025
This change adds a keyword-only `raise_exceptions` parameter to`multiprocessing.set_forkserver_preload()` that defaults to Falsefor backward compatibility. When set to True, ImportError exceptionsduring module preloading in the forkserver process will be raisedinstead of being silently ignored.This is similar in spirit to the approach attempted inpythonGH-99515,providing developers with the ability to catch import errors duringforkserver module preloading for better debugging and error handling.Changes:- Add _raise_exceptions attribute to ForkServer class- Update set_forkserver_preload() to accept raise_exceptions parameter- Pass raise_exceptions flag through to forkserver main() function- Update main() to conditionally raise ImportError based on flag- Add comprehensive test coverage in test_multiprocessing_forkserver- Update documentation in Doc/library/multiprocessing.rst- Add NEWS entry for Python 3.14
gpshead pushed a commit to gpshead/cpython that referenced this pull requestNov 22, 2025
This change adds a keyword-only `raise_exceptions` parameter to`multiprocessing.set_forkserver_preload()` that defaults to Falsefor backward compatibility. When set to True, ImportError exceptionsduring module preloading in the forkserver process will be raisedinstead of being silently ignored.This is similar in spirit to the approach attempted inpythonGH-99515,providing developers with the ability to catch import errors duringforkserver module preloading for better debugging and error handling.Changes:- Add _raise_exceptions attribute to ForkServer class- Update set_forkserver_preload() to accept raise_exceptions parameter- Pass raise_exceptions flag through to forkserver main() function- Update main() to conditionally raise ImportError based on flag- Add comprehensive test coverage in test_multiprocessing_forkserver- Update documentation in Doc/library/multiprocessing.rst- Add NEWS entry for Python 3.14
gpshead pushed a commit to gpshead/cpython that referenced this pull requestNov 22, 2025
This change adds a keyword-only `raise_exceptions` parameter to`multiprocessing.set_forkserver_preload()` that defaults to Falsefor backward compatibility. When set to True, ImportError exceptionsduring module preloading in the forkserver process will be raisedinstead of being silently ignored.This is similar in spirit to the approach attempted inpythonGH-99515,providing developers with the ability to catch import errors duringforkserver module preloading for better debugging and error handling.Changes:- Add _raise_exceptions attribute to ForkServer class- Update set_forkserver_preload() to accept raise_exceptions parameter- Pass raise_exceptions flag through to forkserver main() function- Update main() to conditionally raise ImportError based on flag- Add comprehensive test coverage in test_multiprocessing_forkserver- Update documentation in Doc/library/multiprocessing.rst- Add NEWS entry for Python 3.14
gpshead pushed a commit to gpshead/cpython that referenced this pull requestNov 22, 2025
This change adds a keyword-only `raise_exceptions` parameter to`multiprocessing.set_forkserver_preload()` that defaults to Falsefor backward compatibility. When set to True, ImportError exceptionsduring module preloading in the forkserver process will be raisedinstead of being silently ignored, causing the forkserver to exitand breaking all use of the forkserver multiprocessing context.This is similar in spirit to the approach attempted inpythonGH-99515,providing developers with the ability to catch import errors duringforkserver module preloading for better debugging and error handling.Implementation:- Add _raise_exceptions attribute to ForkServer class- Update set_forkserver_preload() in both forkserver.py and context.py  to accept raise_exceptions keyword-only parameter- Pass raise_exceptions flag through to forkserver main() function- Update main() to conditionally raise ImportError based on flagTests:- Create new test_multiprocessing_forkserver/test_preload.py with  dedicated tests for raise_exceptions functionality- Remove old test from _test_multiprocessing.py- Add tests for both True/False cases and valid/invalid modules- Ensure proper resource cleanup using try/finallyDocumentation:- Update Doc/library/multiprocessing.rst with detailed description  of raise_exceptions behavior and consequences- Add NEWS entry with contributor attribution- Add Nick Neumann to Misc/ACKSContributed by Nick Neumann.
gpshead pushed a commit to gpshead/cpython that referenced this pull requestNov 22, 2025
Add a keyword-only `raise_exceptions` parameter (default False) to`multiprocessing.set_forkserver_preload()`. When True, ImportErrorexceptions during module preloading cause the forkserver to exit,breaking all use of the forkserver multiprocessing context. Thisallows developers to catch import errors during development ratherthan having them silently ignored.Implementation adds the parameter to both the ForkServer class methodand the BaseContext wrapper, passing it through to the forkservermain() function which conditionally raises ImportError instead ofignoring it.Tests are in new test_multiprocessing_forkserver/test_preload.py withproper resource cleanup using try/finally.Documentation describes the behavior, consequences (forkserver exit,EOFError/ConnectionError on subsequent use), and recommends use duringdevelopment.Based on original work by Nick Neumann inpythonGH-99515.Contributed by Nick Neumann.
gpshead added a commit to gpshead/cpython that referenced this pull requestNov 22, 2025
Add a keyword-only `raise_exceptions` parameter (default False) to`multiprocessing.set_forkserver_preload()`. When True, ImportErrorexceptions during module preloading cause the forkserver to exit,breaking all use of the forkserver multiprocessing context. Thisallows developers to catch import errors during development ratherthan having them silently ignored.Implementation adds the parameter to both the ForkServer class methodand the BaseContext wrapper, passing it through to the forkservermain() function which conditionally raises ImportError instead ofignoring it.Tests are in new test_multiprocessing_forkserver/test_preload.py withproper resource cleanup using try/finally.Documentation describes the behavior, consequences (forkserver exit,EOFError/ConnectionError on subsequent use), and recommends use duringdevelopment.Based on original work by Nick Neumann inpythonGH-99515.Contributed by Nick Neumann.Co-authored-by: Gregory P. Smith <greg@krypto.org>
gpshead added a commit to gpshead/cpython that referenced this pull requestNov 22, 2025
Add a keyword-only `raise_exceptions` parameter (default False) to`multiprocessing.set_forkserver_preload()`. When True, ImportErrorexceptions during module preloading cause the forkserver to exit,breaking all use of the forkserver multiprocessing context. Thisallows developers to catch import errors during development ratherthan having them silently ignored.Implementation adds the parameter to both the ForkServer class methodand the BaseContext wrapper, passing it through to the forkservermain() function which conditionally raises ImportError instead ofignoring it.Tests are in new test_multiprocessing_forkserver/test_preload.py withproper resource cleanup using try/finally.Documentation describes the behavior, consequences (forkserver exit,EOFError/ConnectionError on subsequent use), and recommends use duringdevelopment.Based on original work by Nick Neumann inpythonGH-99515.Contributed by Nick Neumann.Co-authored-by: Gregory P. Smith <greg@krypto.org>
gpshead added a commit to gpshead/cpython that referenced this pull requestNov 23, 2025
Add a keyword-only `raise_exceptions` parameter (default False) to`multiprocessing.set_forkserver_preload()`. When True, ImportErrorexceptions during module preloading cause the forkserver to exit,breaking all use of the forkserver multiprocessing context. Thisallows developers to catch import errors during development ratherthan having them silently ignored.Implementation adds the parameter to both the ForkServer class methodand the BaseContext wrapper, passing it through to the forkservermain() function which conditionally raises ImportError instead ofignoring it.Tests are in new test_multiprocessing_forkserver/test_preload.py withproper resource cleanup using try/finally.Documentation describes the behavior, consequences (forkserver exit,EOFError/ConnectionError on subsequent use), and recommends use duringdevelopment.Based on original work by Nick Neumann inpythonGH-99515.Contributed by Nick Neumann.Co-authored-by: Gregory P. Smith <greg@krypto.org>
gpshead added a commit to gpshead/cpython that referenced this pull requestNov 23, 2025
Add a keyword-only `raise_exceptions` parameter (default False) to`multiprocessing.set_forkserver_preload()`. When True, ImportErrorexceptions during module preloading cause the forkserver to exit,breaking all use of the forkserver multiprocessing context. Thisallows developers to catch import errors during development ratherthan having them silently ignored.Implementation adds the parameter to both the ForkServer class methodand the BaseContext wrapper, passing it through to the forkservermain() function which conditionally raises ImportError instead ofignoring it.Tests are in new test_multiprocessing_forkserver/test_preload.py withproper resource cleanup using try/finally.Documentation describes the behavior, consequences (forkserver exit,EOFError/ConnectionError on subsequent use), and recommends use duringdevelopment.Based on original work by Nick Neumann inpythonGH-99515.Contributed by Nick Neumann.Co-authored-by: aggieNick02 <nick@pcpartpicker.com>Co-authored-by: Claude (Sonnet 4.5) <claude@anthropic.com>Co-authored-by: Gregory P. Smith <greg@krypto.org>
gpshead added a commit to gpshead/cpython that referenced this pull requestNov 23, 2025
Add a keyword-only `raise_exceptions` parameter (default False) to`multiprocessing.set_forkserver_preload()`. When True, ImportErrorexceptions during module preloading cause the forkserver to exit,breaking all use of the forkserver multiprocessing context. Thisallows developers to catch import errors during development ratherthan having them silently ignored.Implementation adds the parameter to both the ForkServer class methodand the BaseContext wrapper, passing it through to the forkservermain() function which conditionally raises ImportError instead ofignoring it.Tests are in new test_multiprocessing_forkserver/test_preload.py withproper resource cleanup using try/finally.Documentation describes the behavior, consequences (forkserver exit,EOFError/ConnectionError on subsequent use), and recommends use duringdevelopment.Based on original work by Nick Neumann inpythonGH-99515.Contributed by Nick Neumann.Co-authored-by: aggieNick02 <nick@pcpartpicker.com>Co-authored-by: Claude (Sonnet 4.5) <noreply@anthropic.com>Co-authored-by: Gregory P. Smith <greg@krypto.org>
gpshead added a commit that referenced this pull requestJan 18, 2026
…er_preload` (GH-141859)Add a keyword-only `on_error` parameter to `multiprocessing.set_forkserver_preload()`. This allows the user to have exceptions during optional `forkserver` start method module preloading cause the forkserver subprocess to warn (generally to stderr) or exit with an error (preventing use of the forkserver) instead of being silently ignored.This _also_ fixes an oversight, errors when preloading a `__main__` module are now treated the similarly. Those would always raise unlike other modules in preload, but that had gone unnoticed as up until bug fix PRGH-135295 in 3.14.1 and 3.13.8, the `__main__` module was never actually preloaded.Based on original work by Nick Neumann@aggieNick02 inGH-99515.
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@encukouencukouAwaiting requested review from encukou

@ericsnowcurrentlyericsnowcurrentlyAwaiting requested review from ericsnowcurrentlyericsnowcurrently is a code owner

@ncoghlanncoghlanAwaiting requested review from ncoghlanncoghlan is a code owner

@warsawwarsawAwaiting requested review from warsawwarsaw is a code owner

@1st11st1Awaiting requested review from 1st1

Assignees

No one assigned

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@aggieNick02@pitrou@thesamesam@gpshead@bedevere-bot

[8]ページ先頭

©2009-2026 Movatter.jp