Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork33.3k
gh-85299: Add note warning about entry point guard for asyncio example#93457
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
Doc/library/asyncio-eventloop.rst Outdated
| asyncio.run(main()) | ||
| Note that on macOS, an entry point guard may be needed as the example would |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
It would be better to just add__name__ == '__main__'. This happens on Windows too not just macOS.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Exactly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
That's a good point, Windows also defaults tospawn so looks like it runs into the same issue as mac. Should this note still remain as an fyi, or just change the example to fix the problem and leave out the note? It's customary to have the guard in general but knowing why it fails without might still be helpful, but I don't feel too strongly either way.
Doc/library/asyncio-eventloop.rst Outdated
| asyncio.run(main()) | ||
| Note that on macOS, an entry point guard may be needed as the example would |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Exactly.
bedevere-bot commentedOct 13, 2022
A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated. Once you have made the requested changes, please leave a comment on this pull request containing the phrase |
gvanrossum commentedOct 14, 2022
Okay, I am beginning to understand the issue. It's due to the process pool example. If I comment out this block of code from the example: # 3. Run in a custom process pool:withconcurrent.futures.ProcessPoolExecutor()aspool:result=awaitloop.run_in_executor(pool,cpu_bound)print('custom process pool',result) everything is fine. If I leave that in the run (on macOS) fails spectacularly with a I presume that's one of the many reasons not to use ProcessPoolExecutor. :-( I propose to rephrase the note to mention the platform and just link tohttps://docs.python.org/3/library/multiprocessing.html#the-spawn-and-forkserver-start-methods, not to the change note. |
Doc/library/asyncio-eventloop.rst Outdated
| Note that on Windows and macOS, the entry point guard may be necessary as the | ||
| example would otherwise cause a:exc:`RuntimeError`. For more details, see the | ||
| `Safe importing of main module | ||
| <https://docs.python.org/3/library/multiprocessing.html#the-spawn-and-forkserver-start-methods>`_. |
gvanrossumOct 15, 2022 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I propose to simplify the note to the following:
Note that the entry point guard (
if __name__ == '__main__') is required for option 3, due to the peculiarities ofmultiprocessing, which is used byProcessPoolExecutor. See (link).
There should be three cross-references:
- multiprocessing (the module)
- ProcessPoolExecutor (the class)
- Safe importing of main module
For the latter I recommend to add a label to the target section in multiprocessing.rst rather than linking to the website. (@CAM-Gerlach can provide details on how to do this.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
My understanding though is that option 3 fails whenspawn/forkserver is the process start method, and only Windows + mac default tospawn, while unix defaults tofork, so saying 'is required' may be a bit too strong. If 'may be required' instead, then the reader might wonder why 'may', in which case I think a link tohttps://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods somewhere would help explain things.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
That's too subtle for the readers. We're not stating the law, we're just explaining the example.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Seemy comment for a suggestion implementing the above and the details Guido mentioned.
Uh oh!
There was an error while loading.Please reload this page.
Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>
gvanrossum left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Thanks!
miss-islington commentedOct 16, 2022
Thanks@slateny for the PR, and@gvanrossum for merging it 🌮🎉.. I'm working now to backport this PR to: 3.10. |
miss-islington commentedOct 16, 2022
Thanks@slateny for the PR, and@gvanrossum for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11. |
…example (pythonGH-93457)(cherry picked from commit79fd6cc)Co-authored-by: Stanley <46876382+slateny@users.noreply.github.com>
bedevere-bot commentedOct 16, 2022
GH-98334 is a backport of this pull request to the3.11 branch. |
gvanrossum commentedOct 16, 2022
Let’s see if we can backport this. |
bedevere-bot commentedOct 16, 2022
GH-98335 is a backport of this pull request to the3.10 branch. |
…example (pythonGH-93457)(cherry picked from commit79fd6cc)Co-authored-by: Stanley <46876382+slateny@users.noreply.github.com>
* main: (31 commits)pythongh-95913: Move subinterpreter exper removal to 3.11 WhatsNew (pythonGH-98345)pythongh-95914: Add What's New item describing PEP 670 changes (python#98315) Remove unused arrange_output_buffer function from zlibmodule.c. (pythonGH-98358)pythongh-98174: Handle EPROTOTYPE under macOS in test_sendfile_fallback_close_peer_in_the_middle_of_receiving (python#98316)pythonGH-98327: Reduce scope of catch_warnings() in _make_subprocess_transport (python#98333)pythongh-93691: Compiler's code-gen passes location around instead of holding it on the global compiler state (pythonGH-98001)pythongh-97669: Create Tools/build/ directory (python#97963)pythongh-95534: Improve gzip reading speed by 10% (python#97664)pythongh-95913: Forward-port int/str security change to 3.11 What's New in main (python#98344)pythonGH-91415: Mention alphabetical sort ordering in the Sorting HOWTO (pythonGH-98336)pythongh-97930: Merge with importlib_resources 5.9 (pythonGH-97929)pythongh-85525: Remove extra row in doc (python#98337)pythongh-85299: Add note warning about entry point guard for asyncio example (python#93457)pythongh-97527: IDLE - fix buggy macosx patch (python#98313)pythongh-98307: Add docstring and documentation for SysLogHandler.createSocket (pythonGH-98319)pythongh-94808: Cover `PyFunction_GetCode`, `PyFunction_GetGlobals`, `PyFunction_GetModule` (python#98158)pythonGH-94597: Deprecate child watcher getters and setters (python#98215)pythongh-98254: Include stdlib module names in error messages for NameErrors (python#98255) Improve speed. Reduce auxiliary memory to 16.6% of the main array. (pythonGH-98294) [doc] Update logging cookbook with an example of custom handling of levels. (pythonGH-98290) ...
#85299
https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.run_in_executor