Movatterモバイル変換


[0]ホーム

URL:


homepage

Issue24421

This issue trackerhas been migrated toGitHub, and is currentlyread-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title:Race condition compiling Modules/_math.c
Type:behaviorStage:resolved
Components:Build, DistutilsVersions:Python 3.6, Python 3.5, Python 2.7
process
Status:closedResolution:fixed
Dependencies:Superseder:
Assigned To:Nosy List: Arfrever, dstufft, eric.araujo, floppymaster, martin.panter, python-dev, taleinat, twouters
Priority:normalKeywords:patch

Created on2015-06-10 08:06 bymartin.panter, last changed2022-04-11 14:58 byadmin. This issue is nowclosed.

Files
File nameUploadedDescriptionEdit
slow-cc.pymartin.panter,2015-06-10 08:06
include-math.c.patchmartin.panter,2015-06-18 05:30review
math-once.patchmartin.panter,2015-12-09 04:46Compile from makefilereview
Messages (11)
msg245121 -(view)Author: Martin Panter (martin.panter)*(Python committer)Date: 2015-06-10 08:06
Once or twice I have had the math or cmath modules fail to build. There seems to be a race condition, because the _math.c file seems to be compiled twice, once for each module. Attached is a GCC wrapper that should help reproduce this fairly consistently. First, build Python normally:$ ./configure$ make -j2Then forceModules/_math.c to be recompiled using my wrapper script to amplify the race condition:$ touchModules/_math.c$ rm build/temp.linux-x86_64-3.6/media/disk/home/proj/python/cpython/Modules/_math.o$ make -j2 CC=./slow-cc.pyrunning buildrunning build_extbuilding 'cmath' extensioncompiling for  cmath./slow-cc.py [. . .]/Modules/cmathmodule.obuilding 'math' extension./slow-cc.py [. . .]/Modules/mathmodule.o./slow-cc.py -fPIC -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -I./Include -I. -IInclude -I/usr/local/include -I/media/disk/home/proj/python/cpython/Include -I/media/disk/home/proj/python/cpython -c /media/disk/home/proj/python/cpython/Modules/_math.c -o build/temp.linux-x86_64-3.6/media/disk/home/proj/python/cpython/Modules/_math.oINITIAL COMPILE./slow-cc.py -fPIC -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -I./Include -I. -IInclude -I/usr/local/include -I/media/disk/home/proj/python/cpython/Include -I/media/disk/home/proj/python/cpython -c /media/disk/home/proj/python/cpython/Modules/_math.c -o build/temp.linux-x86_64-3.6/media/disk/home/proj/python/cpython/Modules/_math.oRECOMPILERECOMPIILE TRUNCATINGINITIAL COMPILE FINISHED./slow-cc.py -shared build/temp.linux-x86_64-3.6/media/disk/home/proj/python/cpython/Modules/mathmodule.o build/temp.linux-x86_64-3.6/media/disk/home/proj/python/cpython/Modules/_math.o -L/usr/local/lib -lm -o build/lib.linux-x86_64-3.6/math.cpython-36m-x86_64-linux-gnu.sobuild/temp.linux-x86_64-3.6/media/disk/home/proj/python/cpython/Modules/_math.o: file not recognized: File truncatedcollect2: error: ld returned 1 exit statusRECOMPILE FINISHED./slow-cc.py -shared build/temp.linux-x86_64-3.6/media/disk/home/proj/python/cpython/Modules/cmathmodule.o build/temp.linux-x86_64-3.6/media/disk/home/proj/python/cpython/Modules/_math.o -L/usr/local/lib -lm -o build/lib.linux-x86_64-3.6/cmath.cpython-36m-x86_64-linux-gnu.soFailed to build these modules:math
msg245298 -(view)Author: Tal Einat (taleinat)*(Python committer)Date: 2015-06-13 06:51
Confirmed on OSX 10.10. Here's my output:running buildrunning build_extbuilding 'cmath' extension./slow-cc.py -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -I./Include -I. -IInclude -I/usr/local/include -I/Users/taleinat/dev/cpython/Include -I/Users/taleinat/dev/cpython -c /Users/taleinat/dev/cpython/Modules/cmathmodule.c -o build/temp.macosx-10.10-x86_64-3.6/Users/taleinat/dev/cpython/Modules/cmathmodule.obuilding 'math' extension./slow-cc.py -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -I./Include -I. -IInclude -I/usr/local/include -I/Users/taleinat/dev/cpython/Include -I/Users/taleinat/dev/cpython -c /Users/taleinat/dev/cpython/Modules/mathmodule.c -o build/temp.macosx-10.10-x86_64-3.6/Users/taleinat/dev/cpython/Modules/mathmodule.o./slow-cc.py -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -I./Include -I. -IInclude -I/usr/local/include -I/Users/taleinat/dev/cpython/Include -I/Users/taleinat/dev/cpython -c /Users/taleinat/dev/cpython/Modules/_math.c -o build/temp.macosx-10.10-x86_64-3.6/Users/taleinat/dev/cpython/Modules/_math.o./slow-cc.py -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -I./Include -I. -IInclude -I/usr/local/include -I/Users/taleinat/dev/cpython/Include -I/Users/taleinat/dev/cpython -c /Users/taleinat/dev/cpython/Modules/_math.c -o build/temp.macosx-10.10-x86_64-3.6/Users/taleinat/dev/cpython/Modules/_math.oINITIAL COMPILERECOMPILERECOMPIILE TRUNCATINGINITIAL COMPILE FINISHED./slow-cc.py -bundle -undefined dynamic_lookup build/temp.macosx-10.10-x86_64-3.6/Users/taleinat/dev/cpython/Modules/mathmodule.o build/temp.macosx-10.10-x86_64-3.6/Users/taleinat/dev/cpython/Modules/_math.o -L/usr/local/lib -o build/lib.macosx-10.10-x86_64-3.6/math.cpython-36m-darwin.sold: file too small (length=0) file 'build/temp.macosx-10.10-x86_64-3.6/Users/taleinat/dev/cpython/Modules/_math.o' for architecture x86_64clang: error: linker command failed with exit code 1 (use -v to see invocation)RECOMPILE FINISHED./slow-cc.py -bundle -undefined dynamic_lookup build/temp.macosx-10.10-x86_64-3.6/Users/taleinat/dev/cpython/Modules/cmathmodule.o build/temp.macosx-10.10-x86_64-3.6/Users/taleinat/dev/cpython/Modules/_math.o -L/usr/local/lib -o build/lib.macosx-10.10-x86_64-3.6/cmath.cpython-36m-darwin.so
msg245429 -(view)Author: Martin Panter (martin.panter)*(Python committer)Date: 2015-06-17 03:46
I think this may have been introduced when _math.c was added as a source file of the “cmath” module inr76978 (Issue 7518). In /setup.py:585 there are two distutils.core.Extension objects, both mentioning _math.c. In the build_ext._build_extensions_parallel() method at /Lib/distutils/command/build_ext.py:449, it appears to be invoking build_extension() in two concurrent threads, once for each module, including math and cmath.I don’t know how this code is meant to work, but it looks like either the “build_ext” class should not be creating the same object file in two different threads, or /setup.py should not be giving it the same _math.c file for two different modules.
msg245453 -(view)Author: Martin Panter (martin.panter)*(Python committer)Date: 2015-06-18 05:30
Here is a hacky patch that adds #include "_math.c" so that it is not compiled as a separate object file. This was suggested by Mark inIssue 7518. It is far from a perfect solution, but I cannot suggest anything better without knowing more about Python’s build system.
msg245459 -(view)Author: Tal Einat (taleinat)*(Python committer)Date: 2015-06-18 10:32
Adding Thomas Wouters to the nosy list, since he's listed on the Experts Index under "autoconf/makefiles" as an interest area. Hopefully he can help move this forward.
msg254560 -(view)Author: Martin Panter (martin.panter)*(Python committer)Date: 2015-11-12 21:46
Ideally, I think the solution would be to compile _math.c once, as a dependency shared between both extension modules (perhaps like a static library if necessary). Does anyone know if the build system supports this, or how to do it?Another semi-favourable solution might be to compile _math.c twice, but send the output to separate directories, e.g. . . ./math-build/_math.o and . . ./cmath-build/_math.o. Again, I don’t know if this is practical.
msg256137 -(view)Author: Martin Panter (martin.panter)*(Python committer)Date: 2015-12-09 04:46
Here is an alternative patch to compile _math.c once from the main makefile, before the extension modules are built. I have only tested this on Linux, building in the source directory, and from a separate directory.
msg258349 -(view)Author: Martin Panter (martin.panter)*(Python committer)Date: 2016-01-16 02:00
I am inclined to try committing my second patch to 3.6, as the race condition keeps bugging me every now and again.My biggest worry is that it may not work on Windows. I assume Windows does not use the Makefile, but does use setup.py. In the worst case, perhaps I could add conditional code to restore the original setup.py parameters for Windows.
msg259362 -(view)Author: Roundup Robot (python-dev)(Python triager)Date: 2016-02-02 05:59
New changeset76624d47ee99 by Martin Panter in branch 'default':Issue#24421: Compile _math.c separately to avoid race conditionhttps://hg.python.org/cpython/rev/76624d47ee99
msg260886 -(view)Author: Roundup Robot (python-dev)(Python triager)Date: 2016-02-26 01:14
New changeseta99580157968 by Martin Panter in branch '3.5':Issue#24421: Compile _math.c separately to avoid race conditionhttps://hg.python.org/cpython/rev/a99580157968
msg260888 -(view)Author: Roundup Robot (python-dev)(Python triager)Date: 2016-02-26 02:15
New changesetd4190ed586a4 by Martin Panter in branch '2.7':Issue#24421: Compile _math.c separately to avoid race conditionhttps://hg.python.org/cpython/rev/d4190ed586a4
History
DateUserActionArgs
2022-04-11 14:58:17adminsetgithub: 68609
2016-02-26 21:46:18martin.pantersetstatus: open -> closed
resolution: fixed
stage: patch review -> resolved
2016-02-26 02:15:01python-devsetmessages: +msg260888
2016-02-26 01:14:07python-devsetmessages: +msg260886
2016-02-07 09:36:29Arfreversetnosy: +Arfrever
2016-02-02 05:59:16python-devsetnosy: +python-dev
messages: +msg259362
2016-01-16 02:00:35martin.pantersetmessages: +msg258349
2015-12-09 04:46:52martin.pantersetfiles: +math-once.patch

messages: +msg256137
versions: - Python 3.4
2015-11-12 21:46:03martin.pantersetmessages: +msg254560
2015-11-12 05:06:22floppymastersetnosy: +floppymaster
2015-06-18 10:32:55taleinatsetnosy: +twouters
messages: +msg245459
2015-06-18 05:30:30martin.pantersetfiles: +include-math.c.patch
keywords: +patch
messages: +msg245453

stage: patch review
2015-06-17 03:46:40martin.pantersetversions: + Python 2.7, Python 3.4, Python 3.5, Python 3.6
nosy: +eric.araujo,dstufft

messages: +msg245429

components: + Distutils
2015-06-13 06:51:15taleinatsetnosy: +taleinat
messages: +msg245298
2015-06-10 08:06:40martin.pantercreate
Supported byThe Python Software Foundation,
Powered byRoundup
Copyright © 1990-2022,Python Software Foundation
Legal Statements

[8]ページ先頭

©2009-2026 Movatter.jp