Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork34k
gh-98552: Fix preloading '__main__' with forkserver being broken for a#99515
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
…n for along time (9 years).We do this by using spawn.get_preparation_data() and then using thevalues in that dictionary appropriately. This lets us also fix thesetting of sys.path (which never worked) and other minor properties ofthe forkserver process.While updating the test to verify these fixes, we also discovered thatforkserver was not flushing stdout/stderr before it forked to create anew process. This caused the updated test to fail because unflushedoutput in the forkserver would then be printed by the forked process aswell. This is now fixed too.
aggieNick02 commentedNov 15, 2022
One minor thing I noticed incase@pitrou remembers from several years ago. In |
pitrou commentedNov 16, 2022
@aggieNick02 I suppose it's a simple check that |
aggieNick02 commentedNov 16, 2022
Ah ok, thanks! |
thesamesam commentedOct 9, 2025
gpshead commentedNov 22, 2025
this one is obsolete due to the preloading of main being fixed. but the question of the buffering issue remains, lets take that up on the issue; there may be a new, now smaller, PR worth doing. |
This change adds a keyword-only `raise_exceptions` parameter to`multiprocessing.set_forkserver_preload()` that defaults to Falsefor backward compatibility. When set to True, ImportError exceptionsduring module preloading in the forkserver process will be raisedinstead of being silently ignored.This is similar in spirit to the approach attempted inpythonGH-99515,providing developers with the ability to catch import errors duringforkserver module preloading for better debugging and error handling.Changes:- Add _raise_exceptions attribute to ForkServer class- Update set_forkserver_preload() to accept raise_exceptions parameter- Pass raise_exceptions flag through to forkserver main() function- Update main() to conditionally raise ImportError based on flag- Add comprehensive test coverage in test_multiprocessing_forkserver- Update documentation in Doc/library/multiprocessing.rst- Add NEWS entry for Python 3.14
This change adds a keyword-only `raise_exceptions` parameter to`multiprocessing.set_forkserver_preload()` that defaults to Falsefor backward compatibility. When set to True, ImportError exceptionsduring module preloading in the forkserver process will be raisedinstead of being silently ignored.This is similar in spirit to the approach attempted inpythonGH-99515,providing developers with the ability to catch import errors duringforkserver module preloading for better debugging and error handling.Changes:- Add _raise_exceptions attribute to ForkServer class- Update set_forkserver_preload() to accept raise_exceptions parameter- Pass raise_exceptions flag through to forkserver main() function- Update main() to conditionally raise ImportError based on flag- Add comprehensive test coverage in test_multiprocessing_forkserver- Update documentation in Doc/library/multiprocessing.rst- Add NEWS entry for Python 3.14
This change adds a keyword-only `raise_exceptions` parameter to`multiprocessing.set_forkserver_preload()` that defaults to Falsefor backward compatibility. When set to True, ImportError exceptionsduring module preloading in the forkserver process will be raisedinstead of being silently ignored.This is similar in spirit to the approach attempted inpythonGH-99515,providing developers with the ability to catch import errors duringforkserver module preloading for better debugging and error handling.Changes:- Add _raise_exceptions attribute to ForkServer class- Update set_forkserver_preload() to accept raise_exceptions parameter- Pass raise_exceptions flag through to forkserver main() function- Update main() to conditionally raise ImportError based on flag- Add comprehensive test coverage in test_multiprocessing_forkserver- Update documentation in Doc/library/multiprocessing.rst- Add NEWS entry for Python 3.14
This change adds a keyword-only `raise_exceptions` parameter to`multiprocessing.set_forkserver_preload()` that defaults to Falsefor backward compatibility. When set to True, ImportError exceptionsduring module preloading in the forkserver process will be raisedinstead of being silently ignored.This is similar in spirit to the approach attempted inpythonGH-99515,providing developers with the ability to catch import errors duringforkserver module preloading for better debugging and error handling.Changes:- Add _raise_exceptions attribute to ForkServer class- Update set_forkserver_preload() to accept raise_exceptions parameter- Pass raise_exceptions flag through to forkserver main() function- Update main() to conditionally raise ImportError based on flag- Add comprehensive test coverage in test_multiprocessing_forkserver- Update documentation in Doc/library/multiprocessing.rst- Add NEWS entry for Python 3.14
This change adds a keyword-only `raise_exceptions` parameter to`multiprocessing.set_forkserver_preload()` that defaults to Falsefor backward compatibility. When set to True, ImportError exceptionsduring module preloading in the forkserver process will be raisedinstead of being silently ignored.This is similar in spirit to the approach attempted inpythonGH-99515,providing developers with the ability to catch import errors duringforkserver module preloading for better debugging and error handling.Changes:- Add _raise_exceptions attribute to ForkServer class- Update set_forkserver_preload() to accept raise_exceptions parameter- Pass raise_exceptions flag through to forkserver main() function- Update main() to conditionally raise ImportError based on flag- Add comprehensive test coverage in test_multiprocessing_forkserver- Update documentation in Doc/library/multiprocessing.rst- Add NEWS entry for Python 3.14
This change adds a keyword-only `raise_exceptions` parameter to`multiprocessing.set_forkserver_preload()` that defaults to Falsefor backward compatibility. When set to True, ImportError exceptionsduring module preloading in the forkserver process will be raisedinstead of being silently ignored.This is similar in spirit to the approach attempted inpythonGH-99515,providing developers with the ability to catch import errors duringforkserver module preloading for better debugging and error handling.Changes:- Add _raise_exceptions attribute to ForkServer class- Update set_forkserver_preload() to accept raise_exceptions parameter- Pass raise_exceptions flag through to forkserver main() function- Update main() to conditionally raise ImportError based on flag- Add comprehensive test coverage in test_multiprocessing_forkserver- Update documentation in Doc/library/multiprocessing.rst- Add NEWS entry for Python 3.14
This change adds a keyword-only `raise_exceptions` parameter to`multiprocessing.set_forkserver_preload()` that defaults to Falsefor backward compatibility. When set to True, ImportError exceptionsduring module preloading in the forkserver process will be raisedinstead of being silently ignored, causing the forkserver to exitand breaking all use of the forkserver multiprocessing context.This is similar in spirit to the approach attempted inpythonGH-99515,providing developers with the ability to catch import errors duringforkserver module preloading for better debugging and error handling.Implementation:- Add _raise_exceptions attribute to ForkServer class- Update set_forkserver_preload() in both forkserver.py and context.py to accept raise_exceptions keyword-only parameter- Pass raise_exceptions flag through to forkserver main() function- Update main() to conditionally raise ImportError based on flagTests:- Create new test_multiprocessing_forkserver/test_preload.py with dedicated tests for raise_exceptions functionality- Remove old test from _test_multiprocessing.py- Add tests for both True/False cases and valid/invalid modules- Ensure proper resource cleanup using try/finallyDocumentation:- Update Doc/library/multiprocessing.rst with detailed description of raise_exceptions behavior and consequences- Add NEWS entry with contributor attribution- Add Nick Neumann to Misc/ACKSContributed by Nick Neumann.
Add a keyword-only `raise_exceptions` parameter (default False) to`multiprocessing.set_forkserver_preload()`. When True, ImportErrorexceptions during module preloading cause the forkserver to exit,breaking all use of the forkserver multiprocessing context. Thisallows developers to catch import errors during development ratherthan having them silently ignored.Implementation adds the parameter to both the ForkServer class methodand the BaseContext wrapper, passing it through to the forkservermain() function which conditionally raises ImportError instead ofignoring it.Tests are in new test_multiprocessing_forkserver/test_preload.py withproper resource cleanup using try/finally.Documentation describes the behavior, consequences (forkserver exit,EOFError/ConnectionError on subsequent use), and recommends use duringdevelopment.Based on original work by Nick Neumann inpythonGH-99515.Contributed by Nick Neumann.
Add a keyword-only `raise_exceptions` parameter (default False) to`multiprocessing.set_forkserver_preload()`. When True, ImportErrorexceptions during module preloading cause the forkserver to exit,breaking all use of the forkserver multiprocessing context. Thisallows developers to catch import errors during development ratherthan having them silently ignored.Implementation adds the parameter to both the ForkServer class methodand the BaseContext wrapper, passing it through to the forkservermain() function which conditionally raises ImportError instead ofignoring it.Tests are in new test_multiprocessing_forkserver/test_preload.py withproper resource cleanup using try/finally.Documentation describes the behavior, consequences (forkserver exit,EOFError/ConnectionError on subsequent use), and recommends use duringdevelopment.Based on original work by Nick Neumann inpythonGH-99515.Contributed by Nick Neumann.Co-authored-by: Gregory P. Smith <greg@krypto.org>
Add a keyword-only `raise_exceptions` parameter (default False) to`multiprocessing.set_forkserver_preload()`. When True, ImportErrorexceptions during module preloading cause the forkserver to exit,breaking all use of the forkserver multiprocessing context. Thisallows developers to catch import errors during development ratherthan having them silently ignored.Implementation adds the parameter to both the ForkServer class methodand the BaseContext wrapper, passing it through to the forkservermain() function which conditionally raises ImportError instead ofignoring it.Tests are in new test_multiprocessing_forkserver/test_preload.py withproper resource cleanup using try/finally.Documentation describes the behavior, consequences (forkserver exit,EOFError/ConnectionError on subsequent use), and recommends use duringdevelopment.Based on original work by Nick Neumann inpythonGH-99515.Contributed by Nick Neumann.Co-authored-by: Gregory P. Smith <greg@krypto.org>
Add a keyword-only `raise_exceptions` parameter (default False) to`multiprocessing.set_forkserver_preload()`. When True, ImportErrorexceptions during module preloading cause the forkserver to exit,breaking all use of the forkserver multiprocessing context. Thisallows developers to catch import errors during development ratherthan having them silently ignored.Implementation adds the parameter to both the ForkServer class methodand the BaseContext wrapper, passing it through to the forkservermain() function which conditionally raises ImportError instead ofignoring it.Tests are in new test_multiprocessing_forkserver/test_preload.py withproper resource cleanup using try/finally.Documentation describes the behavior, consequences (forkserver exit,EOFError/ConnectionError on subsequent use), and recommends use duringdevelopment.Based on original work by Nick Neumann inpythonGH-99515.Contributed by Nick Neumann.Co-authored-by: Gregory P. Smith <greg@krypto.org>
Add a keyword-only `raise_exceptions` parameter (default False) to`multiprocessing.set_forkserver_preload()`. When True, ImportErrorexceptions during module preloading cause the forkserver to exit,breaking all use of the forkserver multiprocessing context. Thisallows developers to catch import errors during development ratherthan having them silently ignored.Implementation adds the parameter to both the ForkServer class methodand the BaseContext wrapper, passing it through to the forkservermain() function which conditionally raises ImportError instead ofignoring it.Tests are in new test_multiprocessing_forkserver/test_preload.py withproper resource cleanup using try/finally.Documentation describes the behavior, consequences (forkserver exit,EOFError/ConnectionError on subsequent use), and recommends use duringdevelopment.Based on original work by Nick Neumann inpythonGH-99515.Contributed by Nick Neumann.Co-authored-by: aggieNick02 <nick@pcpartpicker.com>Co-authored-by: Claude (Sonnet 4.5) <claude@anthropic.com>Co-authored-by: Gregory P. Smith <greg@krypto.org>
Add a keyword-only `raise_exceptions` parameter (default False) to`multiprocessing.set_forkserver_preload()`. When True, ImportErrorexceptions during module preloading cause the forkserver to exit,breaking all use of the forkserver multiprocessing context. Thisallows developers to catch import errors during development ratherthan having them silently ignored.Implementation adds the parameter to both the ForkServer class methodand the BaseContext wrapper, passing it through to the forkservermain() function which conditionally raises ImportError instead ofignoring it.Tests are in new test_multiprocessing_forkserver/test_preload.py withproper resource cleanup using try/finally.Documentation describes the behavior, consequences (forkserver exit,EOFError/ConnectionError on subsequent use), and recommends use duringdevelopment.Based on original work by Nick Neumann inpythonGH-99515.Contributed by Nick Neumann.Co-authored-by: aggieNick02 <nick@pcpartpicker.com>Co-authored-by: Claude (Sonnet 4.5) <noreply@anthropic.com>Co-authored-by: Gregory P. Smith <greg@krypto.org>
…er_preload` (GH-141859)Add a keyword-only `on_error` parameter to `multiprocessing.set_forkserver_preload()`. This allows the user to have exceptions during optional `forkserver` start method module preloading cause the forkserver subprocess to warn (generally to stderr) or exit with an error (preventing use of the forkserver) instead of being silently ignored.This _also_ fixes an oversight, errors when preloading a `__main__` module are now treated the similarly. Those would always raise unlike other modules in preload, but that had gone unnoticed as up until bug fix PRGH-135295 in 3.14.1 and 3.13.8, the `__main__` module was never actually preloaded.Based on original work by Nick Neumann@aggieNick02 inGH-99515.
Uh oh!
There was an error while loading.Please reload this page.
long time (9 years).
We do this by using spawn.get_preparation_data() and then using the values in that dictionary appropriately. This lets us also fix the setting of sys.path (which never worked) and other minor properties of the forkserver process.
While updating the test to verify these fixes, we also discovered that forkserver was not flushing stdout/stderr before it forked to create a new process. This caused the updated test to fail because unflushed output in the forkserver would then be printed by the forked process as well. This is now fixed too.