Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

gh-127586: properly restore blocked signals in resource_tracker.py#127587

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged

Conversation

@stephen-hansen
Copy link
Contributor

@stephen-hansenstephen-hansen commentedDec 4, 2024•
edited
Loading

Closes#127586. Instead of using SIG_UNBLOCK to undo the blocked "ignored signals" by resource_manager when creating a child process, we instead will save the blocked signal mask as it was prior to the call to SIG_BLOCK and restore from there using SIG_SETMASK. This has the intended effect of preserving the SIG_BLOCK on either SIGTERM or SIGINT if either was already blocked when entering this method in the parent thread. Existing behavior where either signal was not blocked prior to entering the resource tracker is still preserved since the SIG_SETMASK will undo the SIG_BLOCKs anyway if those signals were previously unblocked.

Please let me know if anything else is needed here, e.g. if a unit test is worthwhile here, I could take a stab tomorrow at writing one.

@ghost
Copy link

ghost commentedDec 4, 2024•
edited by ghost
Loading

All commit authors signed the Contributor License Agreement.
CLA signed

Using SIG_UNBLOCK to remove blocked "ignored signals" may accidentallycause side effects if the calling parent already had said signalsblocked to begin with and did not intend to unblock them whencreating a pool. Use SIG_SETMASK instead with the previous mask ofblocked signals to restore the original blocked set.
@stephen-hansenstephen-hansenforce-pushed thegh-127586-mp-pool-sigblock-fix branch from85370c0 to5e4797cCompareDecember 4, 2024 01:44
@stephen-hansenstephen-hansen changed the titlegh-127586: properly restore blocked signals in resource_manager.pygh-127586: properly restore blocked signals in resource_tracker.pyDec 4, 2024
@ZeroIntensityZeroIntensity added needs backport to 3.12only security fixes needs backport to 3.13bugs and security fixes labelsDec 10, 2024
@ZeroIntensity
Copy link
Member

Please let me know if anything else is needed here, e.g. if a unit test is worthwhile here, I could take a stab tomorrow at writing one.

That would be great. We generally require tests for all bugfixes.

stephen-hansen reacted with thumbs up emoji

@stephen-hansen
Copy link
ContributorAuthor

Please let me know if anything else is needed here, e.g. if a unit test is worthwhile here, I could take a stab tomorrow at writing one.

That would be great. We generally require tests for all bugfixes.

perfect, thanks. I've added a blocked signals test for resource tracker which should cover the issue.

One weird thing I've noticed is that on some of my build pipelines, I'm seeing random test failures for idlelib on macos-13, e.g.https://github.com/python/cpython/actions/runs/12301168597/job/34331141506?pr=127587. Some of the test failures appear similar to what was reported on#76152. Not sure if there is an implicit dependency between resource_tracker and those unit tests, that I broke, or if this is unrelated to my changes. I looked for a while but couldn't find any explanation. I also just don't know idlelib too well.

@ZeroIntensity
Copy link
Member

Yeah, those don't look related. Sometimes we get intermittent failures on some tests, generally not because of a bug in the code but because of a missing platform constraint on that test. I'm not sure if you have permissions to rerun failed CI jobs, so tag me if it happens and I'll do it.

pthread_sigmask is not available on some platformsCo-authored-by: Peter Bierma <zintensitydev@gmail.com>
@gpsheadgpshead self-assigned thisDec 15, 2024
@gpsheadgpshead merged commit46006a1 intopython:mainDec 15, 2024
38 checks passed
@miss-islington-app
Copy link

Thanks@stephen-hansen for the PR, and@gpshead for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12, 3.13.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull requestDec 15, 2024
….py (pythonGH-127587)* Correct pthread_sigmask in resource_tracker to restore old signalsUsing SIG_UNBLOCK to remove blocked "ignored signals" may accidentallycause side effects if the calling parent already had said signalsblocked to begin with and did not intend to unblock them whencreating a pool. Use SIG_SETMASK instead with the previous mask ofblocked signals to restore the original blocked set.* Adding resource_tracker blocked signals test(cherry picked from commit46006a1)Co-authored-by: Stephen Hansen <stephen.paul.hansen@gmail.com>Co-authored-by: Peter Bierma <zintensitydev@gmail.com>Co-authored-by: Gregory P. Smith <greg@krypto.org>
@miss-islington-app
Copy link

Sorry,@stephen-hansen and@gpshead, I could not cleanly backport this to3.12 due to a conflict.
Please backport usingcherry_picker on command line.

cherry_picker 46006a1b355f75d06c10e7b8086912c483b34487 3.12

@bedevere-app
Copy link

GH-127973 is a backport of this pull request to the3.13 branch.

@bedevere-appbedevere-appbot removed the needs backport to 3.13bugs and security fixes labelDec 15, 2024
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure⚠️⚠️⚠️

Hi! The buildbots390x RHEL9 Refleaks 3.x has failed when building commit46006a1.

What do you need to do:

  1. Don't panic.
  2. Checkthe buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1589/builds/630) and take a look at the build logs.
  4. Check if the failure is related to this commit (46006a1) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1589/builds/630

Failed tests:

  • test.test_multiprocessing_fork.test_misc
  • test.test_multiprocessing_spawn.test_misc
  • test.test_multiprocessing_forkserver.test_misc

Failed subtests:

  • test_resource_tracker - test.test_multiprocessing_fork.test_misc.TestResourceTracker.test_resource_tracker
  • test_resource_tracker - test.test_multiprocessing_spawn.test_misc.TestResourceTracker.test_resource_tracker
  • test_resource_tracker - test.test_multiprocessing_forkserver.test_misc.TestResourceTracker.test_resource_tracker

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):  File"/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.refleak/build/Lib/test/_test_multiprocessing.py", line5925, intest_resource_trackerself.assertRegex(err, expected)~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^AssertionError: Regex didn't match: 'resource_tracker: There appear to be 2 leaked semaphore objects' not found in 'Traceback (most recent call last):\n  File "/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/util.py", line 292, in _run_finalizers\n    finalizer()\n    ~~~~~~~~~^^\n  File "/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/util.py", line 216, in __call__\n    res = self._callback(*self._args, **self._kwargs)\n  File "/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/synchronize.py", line 86, in _cleanup\n    sem_unlink(name)\n    ~~~~~~~~~~^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory\n/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/resource_tracker.py:277: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown: {\'/mp-h604h0vv\'}\n  warnings.warn(\n/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/resource_tracker.py:293: UserWarning: resource_tracker: \'/mp-h604h0vv\': [Errno 2] No such file or directory\n  warnings.warn(\'resource_tracker: %r: %s\' % (name, e))\n'Traceback (most recent call last):  File"/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.refleak/build/Lib/test/_test_multiprocessing.py", line5925, intest_resource_trackerself.assertRegex(err, expected)~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^AssertionError: Regex didn't match: 'resource_tracker: There appear to be 2 leaked semaphore objects' not found in 'Traceback (most recent call last):\n  File "/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/util.py", line 292, in _run_finalizers\n    finalizer()\n    ~~~~~~~~~^^\n  File "/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/util.py", line 216, in __call__\n    res = self._callback(*self._args, **self._kwargs)\n  File "/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/synchronize.py", line 86, in _cleanup\n    sem_unlink(name)\n    ~~~~~~~~~~^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory\n/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/resource_tracker.py:277: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown: {\'/mp-4r4kmi3r\'}\n  warnings.warn(\n/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/resource_tracker.py:293: UserWarning: resource_tracker: \'/mp-4r4kmi3r\': [Errno 2] No such file or directory\n  warnings.warn(\'resource_tracker: %r: %s\' % (name, e))\n'Traceback (most recent call last):  File"/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.refleak/build/Lib/test/_test_multiprocessing.py", line5925, intest_resource_trackerself.assertRegex(err, expected)~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^AssertionError: Regex didn't match: 'resource_tracker: There appear to be 2 leaked semaphore objects' not found in 'Traceback (most recent call last):\n  File "/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/util.py", line 292, in _run_finalizers\n    finalizer()\n    ~~~~~~~~~^^\n  File "/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/util.py", line 216, in __call__\n    res = self._callback(*self._args, **self._kwargs)\n  File "/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/synchronize.py", line 86, in _cleanup\n    sem_unlink(name)\n    ~~~~~~~~~~^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory\n/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/resource_tracker.py:277: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown: {\'/mp-cchcz5dh\'}\n  warnings.warn(\n/home/buildbot/buildarea/3.x.cstratak-rhel9-s390x.refleak/build/Lib/multiprocessing/resource_tracker.py:293: UserWarning: resource_tracker: \'/mp-cchcz5dh\': [Errno 2] No such file or directory\n  warnings.warn(\'resource_tracker: %r: %s\' % (name, e))\n'

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure⚠️⚠️⚠️

Hi! The buildbotAMD64 RHEL8 Refleaks 3.x has failed when building commit46006a1.

What do you need to do:

  1. Don't panic.
  2. Checkthe buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/259/builds/1850) and take a look at the build logs.
  4. Check if the failure is related to this commit (46006a1) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/259/builds/1850

Failed tests:

  • test.test_multiprocessing_fork.test_misc
  • test.test_multiprocessing_spawn.test_misc
  • test.test_multiprocessing_forkserver.test_misc

Failed subtests:

  • test_resource_tracker - test.test_multiprocessing_fork.test_misc.TestResourceTracker.test_resource_tracker
  • test_resource_tracker - test.test_multiprocessing_forkserver.test_misc.TestResourceTracker.test_resource_tracker
  • test_resource_tracker - test.test_multiprocessing_spawn.test_misc.TestResourceTracker.test_resource_tracker

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):  File"/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.refleak/build/Lib/test/_test_multiprocessing.py", line5925, intest_resource_trackerself.assertRegex(err, expected)~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^AssertionError: Regex didn't match: 'resource_tracker: There appear to be 2 leaked semaphore objects' not found in 'Traceback (most recent call last):\n  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.refleak/build/Lib/multiprocessing/util.py", line 292, in _run_finalizers\n    finalizer()\n    ~~~~~~~~~^^\n  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.refleak/build/Lib/multiprocessing/util.py", line 216, in __call__\n    res = self._callback(*self._args, **self._kwargs)\n  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.refleak/build/Lib/multiprocessing/synchronize.py", line 86, in _cleanup\n    sem_unlink(name)\n    ~~~~~~~~~~^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory\n/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.refleak/build/Lib/multiprocessing/resource_tracker.py:277: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown: {\'/mp-xh99_f9n\'}\n  warnings.warn(\n/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.refleak/build/Lib/multiprocessing/resource_tracker.py:293: UserWarning: resource_tracker: \'/mp-xh99_f9n\': [Errno 2] No such file or directory\n  warnings.warn(\'resource_tracker: %r: %s\' % (name, e))\n'Traceback (most recent call last):  File"/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.refleak/build/Lib/test/_test_multiprocessing.py", line5925, intest_resource_trackerself.assertRegex(err, expected)~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^AssertionError: Regex didn't match: 'resource_tracker: There appear to be 2 leaked semaphore objects' not found in 'Traceback (most recent call last):\n  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.refleak/build/Lib/multiprocessing/util.py", line 292, in _run_finalizers\n    finalizer()\n    ~~~~~~~~~^^\n  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.refleak/build/Lib/multiprocessing/util.py", line 216, in __call__\n    res = self._callback(*self._args, **self._kwargs)\n  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.refleak/build/Lib/multiprocessing/synchronize.py", line 86, in _cleanup\n    sem_unlink(name)\n    ~~~~~~~~~~^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory\n/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.refleak/build/Lib/multiprocessing/resource_tracker.py:277: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown: {\'/mp-_fx_cbhe\'}\n  warnings.warn(\n/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.refleak/build/Lib/multiprocessing/resource_tracker.py:293: UserWarning: resource_tracker: \'/mp-_fx_cbhe\': [Errno 2] No such file or directory\n  warnings.warn(\'resource_tracker: %r: %s\' % (name, e))\n'Traceback (most recent call last):  File"/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.refleak/build/Lib/test/_test_multiprocessing.py", line5925, intest_resource_trackerself.assertRegex(err, expected)~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^AssertionError: Regex didn't match: 'resource_tracker: There appear to be 2 leaked semaphore objects' not found in 'Traceback (most recent call last):\n  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.refleak/build/Lib/multiprocessing/util.py", line 292, in _run_finalizers\n    finalizer()\n    ~~~~~~~~~^^\n  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.refleak/build/Lib/multiprocessing/util.py", line 216, in __call__\n    res = self._callback(*self._args, **self._kwargs)\n  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.refleak/build/Lib/multiprocessing/synchronize.py", line 86, in _cleanup\n    sem_unlink(name)\n    ~~~~~~~~~~^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory\n/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.refleak/build/Lib/multiprocessing/resource_tracker.py:277: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown: {\'/mp-qy0i0jcq\'}\n  warnings.warn(\n/home/buildbot/buildarea/3.x.cstratak-RHEL8-x86_64.refleak/build/Lib/multiprocessing/resource_tracker.py:293: UserWarning: resource_tracker: \'/mp-qy0i0jcq\': [Errno 2] No such file or directory\n  warnings.warn(\'resource_tracker: %r: %s\' % (name, e))\n'

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure⚠️⚠️⚠️

Hi! The buildbotAMD64 Fedora Stable Refleaks 3.x has failed when building commit46006a1.

What do you need to do:

  1. Don't panic.
  2. Checkthe buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/320/builds/1699) and take a look at the build logs.
  4. Check if the failure is related to this commit (46006a1) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/320/builds/1699

Failed tests:

  • test.test_multiprocessing_fork.test_misc
  • test.test_multiprocessing_spawn.test_misc
  • test.test_multiprocessing_forkserver.test_misc

Failed subtests:

  • test_resource_tracker - test.test_multiprocessing_fork.test_misc.TestResourceTracker.test_resource_tracker
  • test_resource_tracker - test.test_multiprocessing_forkserver.test_misc.TestResourceTracker.test_resource_tracker
  • test_resource_tracker - test.test_multiprocessing_spawn.test_misc.TestResourceTracker.test_resource_tracker

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):  File"/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.refleak/build/Lib/test/_test_multiprocessing.py", line5925, intest_resource_trackerself.assertRegex(err, expected)~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^AssertionError: Regex didn't match: 'resource_tracker: There appear to be 2 leaked semaphore objects' not found in 'Traceback (most recent call last):\n  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.refleak/build/Lib/multiprocessing/util.py", line 292, in _run_finalizers\n    finalizer()\n    ~~~~~~~~~^^\n  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.refleak/build/Lib/multiprocessing/util.py", line 216, in __call__\n    res = self._callback(*self._args, **self._kwargs)\n  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.refleak/build/Lib/multiprocessing/synchronize.py", line 86, in _cleanup\n    sem_unlink(name)\n    ~~~~~~~~~~^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory\n/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.refleak/build/Lib/multiprocessing/resource_tracker.py:277: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown: {\'/mp-diuavsej\'}\n  warnings.warn(\n/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.refleak/build/Lib/multiprocessing/resource_tracker.py:293: UserWarning: resource_tracker: \'/mp-diuavsej\': [Errno 2] No such file or directory\n  warnings.warn(\'resource_tracker: %r: %s\' % (name, e))\n'Traceback (most recent call last):  File"/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.refleak/build/Lib/test/_test_multiprocessing.py", line5925, intest_resource_trackerself.assertRegex(err, expected)~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^AssertionError: Regex didn't match: 'resource_tracker: There appear to be 2 leaked semaphore objects' not found in 'Traceback (most recent call last):\n  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.refleak/build/Lib/multiprocessing/util.py", line 292, in _run_finalizers\n    finalizer()\n    ~~~~~~~~~^^\n  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.refleak/build/Lib/multiprocessing/util.py", line 216, in __call__\n    res = self._callback(*self._args, **self._kwargs)\n  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.refleak/build/Lib/multiprocessing/synchronize.py", line 86, in _cleanup\n    sem_unlink(name)\n    ~~~~~~~~~~^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory\n/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.refleak/build/Lib/multiprocessing/resource_tracker.py:277: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown: {\'/mp-_idjhg2j\'}\n  warnings.warn(\n/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.refleak/build/Lib/multiprocessing/resource_tracker.py:293: UserWarning: resource_tracker: \'/mp-_idjhg2j\': [Errno 2] No such file or directory\n  warnings.warn(\'resource_tracker: %r: %s\' % (name, e))\n'Traceback (most recent call last):  File"/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.refleak/build/Lib/test/_test_multiprocessing.py", line5925, intest_resource_trackerself.assertRegex(err, expected)~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^AssertionError: Regex didn't match: 'resource_tracker: There appear to be 2 leaked semaphore objects' not found in 'Traceback (most recent call last):\n  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.refleak/build/Lib/multiprocessing/util.py", line 292, in _run_finalizers\n    finalizer()\n    ~~~~~~~~~^^\n  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.refleak/build/Lib/multiprocessing/util.py", line 216, in __call__\n    res = self._callback(*self._args, **self._kwargs)\n  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.refleak/build/Lib/multiprocessing/synchronize.py", line 86, in _cleanup\n    sem_unlink(name)\n    ~~~~~~~~~~^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory\n/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.refleak/build/Lib/multiprocessing/resource_tracker.py:277: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown: {\'/mp-wv8gyh8g\'}\n  warnings.warn(\n/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.refleak/build/Lib/multiprocessing/resource_tracker.py:293: UserWarning: resource_tracker: \'/mp-wv8gyh8g\': [Errno 2] No such file or directory\n  warnings.warn(\'resource_tracker: %r: %s\' % (name, e))\n'

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure⚠️⚠️⚠️

Hi! The buildbotaarch64 RHEL8 Refleaks 3.x has failed when building commit46006a1.

What do you need to do:

  1. Don't panic.
  2. Checkthe buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/551/builds/470) and take a look at the build logs.
  4. Check if the failure is related to this commit (46006a1) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/551/builds/470

Failed tests:

  • test.test_multiprocessing_fork.test_misc
  • test.test_multiprocessing_spawn.test_misc
  • test.test_multiprocessing_forkserver.test_misc

Failed subtests:

  • test_resource_tracker - test.test_multiprocessing_fork.test_misc.TestResourceTracker.test_resource_tracker
  • test_resource_tracker - test.test_multiprocessing_forkserver.test_misc.TestResourceTracker.test_resource_tracker

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):  File"/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.refleak/build/Lib/test/_test_multiprocessing.py", line5925, intest_resource_trackerself.assertRegex(err, expected)~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^AssertionError: Regex didn't match: 'resource_tracker: There appear to be 2 leaked semaphore objects' not found in 'Traceback (most recent call last):\n  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.refleak/build/Lib/multiprocessing/util.py", line 292, in _run_finalizers\n    finalizer()\n    ~~~~~~~~~^^\n  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.refleak/build/Lib/multiprocessing/util.py", line 216, in __call__\n    res = self._callback(*self._args, **self._kwargs)\n  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.refleak/build/Lib/multiprocessing/synchronize.py", line 86, in _cleanup\n    sem_unlink(name)\n    ~~~~~~~~~~^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory\n/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.refleak/build/Lib/multiprocessing/resource_tracker.py:277: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown: {\'/mp-5ksl4jwh\'}\n  warnings.warn(\n/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.refleak/build/Lib/multiprocessing/resource_tracker.py:293: UserWarning: resource_tracker: \'/mp-5ksl4jwh\': [Errno 2] No such file or directory\n  warnings.warn(\'resource_tracker: %r: %s\' % (name, e))\n'Traceback (most recent call last):  File"/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.refleak/build/Lib/test/_test_multiprocessing.py", line5925, intest_resource_trackerself.assertRegex(err, expected)~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^AssertionError: Regex didn't match: 'resource_tracker: There appear to be 2 leaked semaphore objects' not found in 'Traceback (most recent call last):\n  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.refleak/build/Lib/multiprocessing/util.py", line 292, in _run_finalizers\n    finalizer()\n    ~~~~~~~~~^^\n  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.refleak/build/Lib/multiprocessing/util.py", line 216, in __call__\n    res = self._callback(*self._args, **self._kwargs)\n  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.refleak/build/Lib/multiprocessing/synchronize.py", line 86, in _cleanup\n    sem_unlink(name)\n    ~~~~~~~~~~^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory\n/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.refleak/build/Lib/multiprocessing/resource_tracker.py:277: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown: {\'/mp-xo0p7x29\'}\n  warnings.warn(\n/home/buildbot/buildarea/3.x.cstratak-RHEL8-aarch64.refleak/build/Lib/multiprocessing/resource_tracker.py:293: UserWarning: resource_tracker: \'/mp-xo0p7x29\': [Errno 2] No such file or directory\n  warnings.warn(\'resource_tracker: %r: %s\' % (name, e))\n'

@hugovk
Copy link
Member

The next 3.14 alpha release is tomorrow and these tier 1 and 2 buildbot failures are blocking it:

https://buildbot.python.org/#/release_status

Do you have an idea how to fix it or shall we revert this for now?

@hugovk
Copy link
Member

Revert PR in case we need it:#127983

gpshead reacted with thumbs up emoji

@ZeroIntensity
Copy link
Member

I swear that I'm a buildbot failure magnet 😄

hugovk and gpshead reacted with laugh emoji

@stephen-hansen
Copy link
ContributorAuthor

Yeah, it looks like the cleanup test behavior might have changed. At first glance it looks like the previous behavior was that both semaphores would leak, guessing terminate() + wait() was immediate? Feels like it is now blocking on the subprocess, as name1 leaks as expected but name2 appears to be cleaned by its finalizer.. this feels correct to me but would appreciate if someone could confirm.

Guessing the fix is to change the 1 expected output line but for now happy to revert this. I’m going to investigate this more tonight.

hugovk and gpshead reacted with thumbs up emoji

hugovk added a commit that referenced this pull requestDec 16, 2024
@gpshead
Copy link
Member

thanks!

srinivasreddy pushed a commit to srinivasreddy/cpython that referenced this pull requestJan 8, 2025
….py (pythonGH-127587)* Correct pthread_sigmask in resource_tracker to restore old signalsUsing SIG_UNBLOCK to remove blocked "ignored signals" may accidentallycause side effects if the calling parent already had said signalsblocked to begin with and did not intend to unblock them whencreating a pool. Use SIG_SETMASK instead with the previous mask ofblocked signals to restore the original blocked set.* Adding resource_tracker blocked signals testCo-authored-by: Peter Bierma <zintensitydev@gmail.com>Co-authored-by: Gregory P. Smith <greg@krypto.org>
srinivasreddy pushed a commit to srinivasreddy/cpython that referenced this pull requestJan 8, 2025
@hugovkhugovk removed the needs backport to 3.12only security fixes labelFeb 27, 2025
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@gpsheadgpsheadgpshead approved these changes

@ZeroIntensityZeroIntensityZeroIntensity left review comments

Assignees

@gpsheadgpshead

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

multiprocessing.Pool does not properly restore blocked signals

5 participants

@stephen-hansen@ZeroIntensity@bedevere-bot@hugovk@gpshead
[8]ページ先頭
©2009-2025 Movatter.jp