Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

gh-104690 Disallow thread creation and fork at interpreter finalization#104826

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
gpshead merged 16 commits intopython:mainfromchgnrdv:disallow-thread-creation-and-fork-at-interp-exit
Jun 4, 2023
Merged

gh-104690 Disallow thread creation and fork at interpreter finalization#104826

gpshead merged 16 commits intopython:mainfromchgnrdv:disallow-thread-creation-and-fork-at-interp-exit
Jun 4, 2023

Conversation

chgnrdv
Copy link
Contributor

@chgnrdvchgnrdv commentedMay 24, 2023
edited by gpshead
Loading

Fixes#104690

In the following functions, check if interpreter is finalizing and in this case raiseRuntimeError with appropriate message:

  • _thread.start_new_thread
  • posix.fork
  • posix.fork1
  • posix.forkpty
  • _posixsubprocess.fork_exec when a preexec_fn is used.

in the following functions, check if interpreter is finalizing and raise `RuntimeError` with appropriate message:* `_thread.start_new_thread`* `posix.fork`* `posix.fork1`* `posix.forkpty`* `_posixsubprocess.fork_exec`
@chgnrdvchgnrdv marked this pull request as ready for reviewMay 24, 2023 12:46
@chgnrdvchgnrdv requested a review fromgpshead as acode ownerMay 24, 2023 12:46
@gpsheadgpshead self-assigned thisMay 26, 2023
@bedevere-bot
Copy link

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phraseI have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

And if you don't make the requested changes, you will be put in the comfy chair!

@gpsheadgpshead requested a review fromvstinnerMay 26, 2023 04:23
@gpsheadgpshead added the needs backport to 3.12only security fixes labelMay 26, 2023
…thub.com:chgnrdv/cpython into disallow-thread-creation-and-fork-at-interp-exit
* removed excess test case from `test_threading`* changed NEWS entry to not mention internal function* allowed `subprocess.Popen` at shutdown if `preexec_fn` is `None`
@chgnrdv
Copy link
ContributorAuthor

I have made the requested changes; please review again

@bedevere-bot
Copy link

Thanks for making the requested changes!

@gpshead: please review the changes made to this pull request.

@gpshead
Copy link
Member

FYI - if you want an example of real world code that was spawning threads from atexit (I'm not claiming that this was agood idea... just that it inadvertently happened): See the example given in#86813.

chgnrdv reacted with eyes emoji

@gpsheadgpsheadenabled auto-merge (squash)June 3, 2023 15:45
@gpsheadgpshead merged commitce558e6 intopython:mainJun 4, 2023
@miss-islington
Copy link
Contributor

Thanks@chgnrdv for the PR, and@gpshead for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

@miss-islington
Copy link
Contributor

Sorry@chgnrdv and@gpshead, I had trouble checking out the3.12 backport branch.
Please retry by removing and re-adding the "needs backport to 3.12" label.
Alternatively, you can backport usingcherry_picker on the command line.
cherry_picker ce558e69d4087dd3653207de78345fbb8a2c7835 3.12

@gpsheadgpshead added needs backport to 3.12only security fixes and removed needs backport to 3.12only security fixes labelsJun 4, 2023
@miss-islington
Copy link
Contributor

Thanks@chgnrdv for the PR, and@gpshead for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull requestJun 4, 2023
…lization (pythonGH-104826)Disallow thread creation and fork at interpreter finalization.in the following functions, check if interpreter is finalizing and raise `RuntimeError` with appropriate message:* `_thread.start_new_thread` and thus `threading`* `posix.fork`* `posix.fork1`* `posix.forkpty`* `_posixsubprocess.fork_exec` when a `preexec_fn=` is supplied.---------(cherry picked from commitce558e6)Co-authored-by: chgnrdv <52372310+chgnrdv@users.noreply.github.com>Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>Co-authored-by: Gregory P. Smith <greg@krypto.org>
@bedevere-bot
Copy link

GH-105277 is a backport of this pull request to the3.12 branch.

@bedevere-botbedevere-bot removed the needs backport to 3.12only security fixes labelJun 4, 2023
gpshead added a commit that referenced this pull requestJun 4, 2023
…alization (GH-104826) (#105277)gh-104690 Disallow thread creation and fork at interpreter finalization (GH-104826)Disallow thread creation and fork at interpreter finalization.in the following functions, check if interpreter is finalizing and raise `RuntimeError` with appropriate message:* `_thread.start_new_thread` and thus `threading`* `posix.fork`* `posix.fork1`* `posix.forkpty`* `_posixsubprocess.fork_exec` when a `preexec_fn=` is supplied.---------(cherry picked from commitce558e6)Co-authored-by: chgnrdv <52372310+chgnrdv@users.noreply.github.com>Co-authored-by: Gregory P. Smith <greg@krypto.org>
AdamWill added a commit to AdamWill/anaconda that referenced this pull requestJul 5, 2023
In Python 3.12, you cannot pass a preexec_fn to subprocess.Popenduring interpreter shutdown:python/cpython#104826This avoids the problem by giving startProgram an arg to notuse a preexec_fn, and passing that all the way from anaconda'sexitHandler through execWithRedirect and _run_program.Signed-off-by: Adam Williamson <awilliam@redhat.com>
AdamWill added a commit to AdamWill/anaconda that referenced this pull requestJul 5, 2023
In Python 3.12, you cannot pass a preexec_fn to subprocess.Popenduring interpreter shutdown:python/cpython#104826This avoids the problem by giving startProgram an arg to notuse a preexec_fn, and passing that all the way from anaconda'sexitHandler through execWithRedirect and _run_program.Signed-off-by: Adam Williamson <awilliam@redhat.com>
@vstinner
Copy link
Member

This change introduced a test_concurrent_futures regression: issue#109047. Well, right now I'm not sure about the root cause analysis.

mabdinur added a commit to DataDog/dd-trace-py that referenced this pull requestSep 8, 2023
… python 3.12 (#6859)## Motivation- Make the instrumentation telemetry client compatible with python3.12:python/cpython#104826## Description - Start telemetry worker thread as early as possible. - Delays sending all telemetry events until app-started is queued. - Refactors tests to align with this new logic. ## Risk - Telemetry events (metrics/logs/integrations) are queued as early aspossible but these events are only sent when the trace agent writer isstarted. This **may** result in a memory leak if high cardinalitytelemetry metrics and logs are added in the future. This is not aconcern right now.## Checklist- [x] Change(s) are motivated and described in the PR description.- [x] Testing strategy is described if automated tests are not includedin the PR.- [x] Risk is outlined (performance impact, potential for breakage,maintainability, etc).- [x] Change is maintainable (easy to change, telemetry, documentation).- [x] [Library release noteguidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)are followed. If no release note is required, add label`changelog/no-changelog`.- [x] Documentation is included (in-code, generated user docs, [publiccorp docs](https://github.com/DataDog/documentation/)).- [x] Backport labels are set (if[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))## Reviewer Checklist- [x] Title is accurate.- [x] No unnecessary changes are introduced.- [x] Description motivates each change.- [x] Avoids breaking[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)changes unless absolutely necessary.- [x] Testing strategy adequately addresses listed risk(s).- [x] Change is maintainable (easy to change, telemetry, documentation).- [x] Release note makes sense to a user of the library.- [x] Reviewer has explicitly acknowledged and discussed the performanceimplications of this PR as reported in the benchmarks PR comment.- [x] Backport labels are set in a manner that is consistent with the[release branch maintenancepolicy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)- [x] If this PR touches code that signs or publishes builds orpackages, or handles credentials of any kind, I've requested a reviewfrom `@DataDog/security-design-and-guidance`.- [x] This PR doesn't touch any of that.---------Co-authored-by: Yun Kim <yun.kim@datadoghq.com>Co-authored-by: Tahir H. Butt <tahir.butt@datadoghq.com>Co-authored-by: Gabriele N. Tornetta <gabriele.tornetta@datadoghq.com>Co-authored-by: Yun Kim <35776586+Yun-Kim@users.noreply.github.com>Co-authored-by: Emmett Butler <emmett.butler321@gmail.com>Co-authored-by: ZStriker19 <zach.groves@datadoghq.com>
mabdinur added a commit to DataDog/dd-trace-py that referenced this pull requestSep 11, 2023
… python 3.12 (#6859)- Make the instrumentation telemetry client compatible with python3.12:python/cpython#104826 - Start telemetry worker thread as early as possible. - Delays sending all telemetry events until app-started is queued. - Refactors tests to align with this new logic.- Telemetry events (metrics/logs/integrations) are queued as early aspossible but these events are only sent when the trace agent writer isstarted. This **may** result in a memory leak if high cardinalitytelemetry metrics and logs are added in the future. This is not aconcern right now.- [x] Change(s) are motivated and described in the PR description.- [x] Testing strategy is described if automated tests are not includedin the PR.- [x] Risk is outlined (performance impact, potential for breakage,maintainability, etc).- [x] Change is maintainable (easy to change, telemetry, documentation).- [x] [Library release noteguidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)are followed. If no release note is required, add label`changelog/no-changelog`.- [x] Documentation is included (in-code, generated user docs, [publiccorp docs](https://github.com/DataDog/documentation/)).- [x] Backport labels are set (if[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))- [x] Title is accurate.- [x] No unnecessary changes are introduced.- [x] Description motivates each change.- [x] Avoids breaking[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)changes unless absolutely necessary.- [x] Testing strategy adequately addresses listed risk(s).- [x] Change is maintainable (easy to change, telemetry, documentation).- [x] Release note makes sense to a user of the library.- [x] Reviewer has explicitly acknowledged and discussed the performanceimplications of this PR as reported in the benchmarks PR comment.- [x] Backport labels are set in a manner that is consistent with the[release branch maintenancepolicy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)- [x] If this PR touches code that signs or publishes builds orpackages, or handles credentials of any kind, I've requested a reviewfrom `@DataDog/security-design-and-guidance`.- [x] This PR doesn't touch any of that.---------Co-authored-by: Yun Kim <yun.kim@datadoghq.com>Co-authored-by: Tahir H. Butt <tahir.butt@datadoghq.com>Co-authored-by: Gabriele N. Tornetta <gabriele.tornetta@datadoghq.com>Co-authored-by: Yun Kim <35776586+Yun-Kim@users.noreply.github.com>Co-authored-by: Emmett Butler <emmett.butler321@gmail.com>Co-authored-by: ZStriker19 <zach.groves@datadoghq.com>
majorgreys added a commit to DataDog/dd-trace-py that referenced this pull requestSep 13, 2023
… python 3.12 (#6859)## Motivation- Make the instrumentation telemetry client compatible with python3.12:python/cpython#104826## Description - Start telemetry worker thread as early as possible. - Delays sending all telemetry events until app-started is queued. - Refactors tests to align with this new logic. ## Risk - Telemetry events (metrics/logs/integrations) are queued as early aspossible but these events are only sent when the trace agent writer isstarted. This **may** result in a memory leak if high cardinalitytelemetry metrics and logs are added in the future. This is not aconcern right now.## Checklist- [x] Change(s) are motivated and described in the PR description.- [x] Testing strategy is described if automated tests are not includedin the PR.- [x] Risk is outlined (performance impact, potential for breakage,maintainability, etc).- [x] Change is maintainable (easy to change, telemetry, documentation).- [x] [Library release noteguidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)are followed. If no release note is required, add label`changelog/no-changelog`.- [x] Documentation is included (in-code, generated user docs, [publiccorp docs](https://github.com/DataDog/documentation/)).- [x] Backport labels are set (if[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))## Reviewer Checklist- [x] Title is accurate.- [x] No unnecessary changes are introduced.- [x] Description motivates each change.- [x] Avoids breaking[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)changes unless absolutely necessary.- [x] Testing strategy adequately addresses listed risk(s).- [x] Change is maintainable (easy to change, telemetry, documentation).- [x] Release note makes sense to a user of the library.- [x] Reviewer has explicitly acknowledged and discussed the performanceimplications of this PR as reported in the benchmarks PR comment.- [x] Backport labels are set in a manner that is consistent with the[release branch maintenancepolicy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)- [x] If this PR touches code that signs or publishes builds orpackages, or handles credentials of any kind, I've requested a reviewfrom `@DataDog/security-design-and-guidance`.- [x] This PR doesn't touch any of that.---------Co-authored-by: Yun Kim <yun.kim@datadoghq.com>Co-authored-by: Tahir H. Butt <tahir.butt@datadoghq.com>Co-authored-by: Gabriele N. Tornetta <gabriele.tornetta@datadoghq.com>Co-authored-by: Yun Kim <35776586+Yun-Kim@users.noreply.github.com>Co-authored-by: Emmett Butler <emmett.butler321@gmail.com>Co-authored-by: ZStriker19 <zach.groves@datadoghq.com>
ppigazzini added a commit to ppigazzini/fishtest that referenced this pull requestApr 4, 2024
Python 3.12 introduced a bug during the interpreter shutdown, see:- issuepython/cpython#104826- bugfixpython/cpython#117029With Python 3.12.3, this change could potentially be reverted (after testing).
ppigazzini added a commit to official-stockfish/fishtest that referenced this pull requestApr 4, 2024
Python 3.12 introduced a bug during the interpreter shutdown, see:- issuepython/cpython#104826- bugfixpython/cpython#117029With Python 3.12.3, this change could potentially be reverted (after testing).
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@gpsheadgpsheadgpshead approved these changes

@sunmy2019sunmy2019sunmy2019 approved these changes

@vstinnervstinnerAwaiting requested review from vstinner

Assignees

@gpsheadgpshead

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Threads started in exit handler are still running after their thread states are destroyed
6 participants
@chgnrdv@bedevere-bot@gpshead@miss-islington@vstinner@sunmy2019

[8]ページ先頭

©2009-2025 Movatter.jp