Movatterモバイル変換


[0]ホーム

URL:


homepage

Issue38377

This issue trackerhas been migrated toGitHub, and is currentlyread-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title:test_asyncio.test_events.GetEventLoopTestsMixin.test_get_event_loop_new_process mixin breaks in the Unix environment without working /dev/shm
Type:Stage:resolved
Components:asyncioVersions:Python 3.10, Python 3.9, Python 3.8
process
Status:closedResolution:fixed
Dependencies:Superseder:
Assigned To:Nosy List: asvetlov, mcepl, miss-islington, ned.deily, vstinner, yselivanov
Priority:normalKeywords:patch

Created on2019-10-05 13:47 bymcepl, last changed2022-04-11 14:59 byadmin. This issue is nowclosed.

Pull Requests
URLStatusLinkedEdit
PR 16593closedmcepl,2019-10-05 13:48
PR 19073closedmcepl,2020-03-19 15:02
PR 20944mergedvstinner,2020-06-17 16:10
PR 20962mergedvstinner,2020-06-18 13:01
PR 20966mergedvstinner,2020-06-18 15:25
PR 20984mergedvstinner,2020-06-19 14:59
PR 20987mergedmiss-islington,2020-06-19 16:01
PR 20988mergedmiss-islington,2020-06-19 16:01
Messages (12)
msg354010 -(view)Author: Matej Cepl (mcepl)*Date: 2019-10-05 13:47
======================================================================ERROR: test_get_event_loop_new_process (test.test_asyncio.test_events.TestCGetEventLoop)----------------------------------------------------------------------Traceback (most recent call last):  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/test/test_asyncio/test_events.py", line 2647, in test_get_event_loop_new_process    self.loop.run_until_complete(main()),  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/asyncio/base_events.py", line 608, in run_until_complete    return future.result()  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/test/test_asyncio/test_events.py", line 2640, in main    pool = concurrent.futures.ProcessPoolExecutor()  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/concurrent/futures/process.py", line 555, in __init__    self._call_queue = _SafeQueue(  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/concurrent/futures/process.py", line 165, in __init__    super().__init__(max_size, ctx=ctx)  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/multiprocessing/queues.py", line 42, in __init__    self._rlock = ctx.Lock()  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/multiprocessing/context.py", line 68, in Lock    return Lock(ctx=self.get_context())  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/multiprocessing/synchronize.py", line 162, in __init__    SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx)  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/multiprocessing/synchronize.py", line 57, in __init__    sl = self._semlock = _multiprocessing.SemLock(OSError: [Errno 38] Function not implemented======================================================================ERROR: test_get_event_loop_new_process (test.test_asyncio.test_events.TestPyGetEventLoop)----------------------------------------------------------------------Traceback (most recent call last):  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/test/test_asyncio/test_events.py", line 2647, in test_get_event_loop_new_process    self.loop.run_until_complete(main()),  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/asyncio/base_events.py", line 608, in run_until_complete    return future.result()  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/test/test_asyncio/test_events.py", line 2640, in main    pool = concurrent.futures.ProcessPoolExecutor()  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/concurrent/futures/process.py", line 555, in __init__    self._call_queue = _SafeQueue(  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/concurrent/futures/process.py", line 165, in __init__    super().__init__(max_size, ctx=ctx)  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/multiprocessing/queues.py", line 42, in __init__    self._rlock = ctx.Lock()  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/multiprocessing/context.py", line 68, in Lock    return Lock(ctx=self.get_context())  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/multiprocessing/synchronize.py", line 162, in __init__    SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx)  File "/home/abuild/rpmbuild/BUILD/Python-3.8.0rc1/Lib/multiprocessing/synchronize.py", line 57, in __init__    sl = self._semlock = _multiprocessing.SemLock(OSError: [Errno 38] Function not implemented----------------------------------------------------------------------The problem is that in the OpenBuildService (the build system of SUSE distributions) /dev/shm is limited to:abuild@milic:~/rpmbuild/BUILD/Python-3.8.0rc1> ls -ld /dev/shmdrwxr-xr-x 1 root root 0 Oct  5 13:23 /dev/shmabuild@milic:~/rpmbuild/BUILD/Python-3.8.0rc1>Provided PR will skip test if /dev/shm doesn’t exist or it doesn’t sufficient permissions.
msg371809 -(view)Author: STINNER Victor (vstinner)*(Python committer)Date: 2020-06-18 12:53
New changesetddbeb2f3e02a510c5784ffd74c5e09e8c70b5881 by Victor Stinner in branch 'master':bpo-38377: Add support.skip_if_broken_multiprocessing_synchronize() (GH-20944)https://github.com/python/cpython/commit/ddbeb2f3e02a510c5784ffd74c5e09e8c70b5881
msg371819 -(view)Author: STINNER Victor (vstinner)*(Python committer)Date: 2020-06-18 15:20
New changesetb1e736113484c99acb57e4acb417b91a9e58e7ff by Victor Stinner in branch '3.9':bpo-38377: Add support.skip_if_broken_multiprocessing_synchronize() (GH-20944) (GH-20962)https://github.com/python/cpython/commit/b1e736113484c99acb57e4acb417b91a9e58e7ff
msg371826 -(view)Author: STINNER Victor (vstinner)*(Python committer)Date: 2020-06-18 16:56
New changesete8056180a13b6755e4e3e5505b7bf03f79da29fb by Victor Stinner in branch '3.8':bpo-38377: Add support.skip_if_broken_multiprocessing_synchronize() (GH-20944) (GH-20962) (GH-20966)https://github.com/python/cpython/commit/e8056180a13b6755e4e3e5505b7bf03f79da29fb
msg371828 -(view)Author: STINNER Victor (vstinner)*(Python committer)Date: 2020-06-18 16:58
Hum, test_resource_tracker_reused() failed twice on macOS, but it's unclear to me if it's a regression caused by my change or not.x86-64 macOS 3.9:https://buildbot.python.org/all/#builders/696/builds/119FAIL: test_resource_tracker_reused (test.test_multiprocessing_forkserver.TestResourceTracker)----------------------------------------------------------------------Traceback (most recent call last):  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/test/_test_multiprocessing.py", line 5219, in test_resource_tracker_reused    self.assertTrue(is_resource_tracker_reused)AssertionError: False is not true
msg371833 -(view)Author: Ned Deily (ned.deily)*(Python committer)Date: 2020-06-18 19:29
> Hum, test_resource_tracker_reused() failed twice on macOS, but it's unclear to me if it's a regression caused by my change or not.It does appear to be caused by your change. I did a quick check with the 3.8 branch. With that PR, test_multiprocessing_forkserver fails solidly. Backing up to the previous commit, test_multiprocessing_forkserver passes.
msg371836 -(view)Author: STINNER Victor (vstinner)*(Python committer)Date: 2020-06-18 21:01
test_multiprocessing_forkserver.test_resource_tracker_reused() failed on 3.8, 3.9 and master buildbots. It looks like a real reproducible bug. I'm not sure how my change triggered this bug.
msg371886 -(view)Author: STINNER Victor (vstinner)*(Python committer)Date: 2020-06-19 14:55
From what I understood, the problem is that creating a SemLock has an impact on the resource tracker.The failing unit tests checks something about the resource tracker.The test runs a function in a child process. The function is defined in _test_multiprocessing.py. _test_multiprocessing.py calls skip_if_broken_multiprocessing_synchronize() which creates a SemLock(). At the end, the child process modifies the resource tracker because it creates a SemLock().There are different options to fix the issue:* Modify skip_if_broken_multiprocessing_synchronize() only only attempt to create a SemLock on Linux.* Modify skip_if_broken_multiprocessing_synchronize() to only check if the current user is allowed to write into /dev/shm/ directory: that would allow touching the resource tracker.* Modify the test to avoid improting _test_multiprocessing in the child process.I don't know why the test only fails on macOS.Note: I ignored macOS failure when I merged my PR because the macOS job of GitHub Action is failing randomly for at least 3 months and nobody attempted to fix it. So I learnt to ignore this job.
msg371891 -(view)Author: STINNER Victor (vstinner)*(Python committer)Date: 2020-06-19 16:01
New changeset3358da4054b9b0b045eb47dc74dee3d58bfbb1d5 by Victor Stinner in branch 'master':bpo-38377: Fix skip_if_broken_multiprocessing_synchronize() on macOS (GH-20984)https://github.com/python/cpython/commit/3358da4054b9b0b045eb47dc74dee3d58bfbb1d5
msg371892 -(view)Author: miss-islington (miss-islington)Date: 2020-06-19 16:19
New changesetec9bc2da421c456e416d991fd1fe79ac33344d9d by Miss Islington (bot) in branch '3.8':bpo-38377: Fix skip_if_broken_multiprocessing_synchronize() on macOS (GH-20984)https://github.com/python/cpython/commit/ec9bc2da421c456e416d991fd1fe79ac33344d9d
msg371893 -(view)Author: miss-islington (miss-islington)Date: 2020-06-19 16:20
New changeset1529322ca6b7b029af65aceeb04e6d47013eeeb4 by Miss Islington (bot) in branch '3.9':bpo-38377: Fix skip_if_broken_multiprocessing_synchronize() on macOS (GH-20984)https://github.com/python/cpython/commit/1529322ca6b7b029af65aceeb04e6d47013eeeb4
msg372021 -(view)Author: STINNER Victor (vstinner)*(Python committer)Date: 2020-06-21 21:17
Ok, macOS buildbots are back to green, I close the issue.
History
DateUserActionArgs
2022-04-11 14:59:21adminsetgithub: 82558
2020-06-21 21:17:29vstinnersetstatus: open -> closed
resolution: fixed
messages: +msg372021

stage: patch review -> resolved
2020-06-19 16:20:19miss-islingtonsetmessages: +msg371893
2020-06-19 16:19:45miss-islingtonsetmessages: +msg371892
2020-06-19 16:01:47miss-islingtonsetpull_requests: +pull_request20162
2020-06-19 16:01:40miss-islingtonsetnosy: +miss-islington
pull_requests: +pull_request20161
2020-06-19 16:01:25vstinnersetmessages: +msg371891
2020-06-19 14:59:43vstinnersetpull_requests: +pull_request20158
2020-06-19 14:55:19vstinnersetmessages: +msg371886
2020-06-18 21:01:35vstinnersetmessages: +msg371836
versions: + Python 3.10
2020-06-18 19:29:42ned.deilysetnosy: +ned.deily
messages: +msg371833
2020-06-18 16:58:55vstinnersetmessages: +msg371828
2020-06-18 16:56:52vstinnersetmessages: +msg371826
2020-06-18 15:25:02vstinnersetpull_requests: +pull_request20144
2020-06-18 15:20:08vstinnersetmessages: +msg371819
2020-06-18 13:01:02vstinnersetpull_requests: +pull_request20141
2020-06-18 12:53:27vstinnersetmessages: +msg371809
2020-06-17 16:10:44vstinnersetnosy: +vstinner
pull_requests: +pull_request20125
2020-03-19 15:02:18mceplsetpull_requests: +pull_request18428
2019-10-05 13:48:15mceplsetkeywords: +patch
stage: patch review
pull_requests: +pull_request16182
2019-10-05 13:47:18mceplcreate
Supported byThe Python Software Foundation,
Powered byRoundup
Copyright © 1990-2022,Python Software Foundation
Legal Statements

[8]ページ先頭

©2009-2025 Movatter.jp