
This issue trackerhas been migrated toGitHub, and is currentlyread-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.
Created on2015-06-10 08:06 bymartin.panter, last changed2022-04-11 14:58 byadmin. This issue is nowclosed.
| Files | ||||
|---|---|---|---|---|
| File name | Uploaded | Description | Edit | |
| slow-cc.py | martin.panter,2015-06-10 08:06 | |||
| include-math.c.patch | martin.panter,2015-06-18 05:30 | review | ||
| math-once.patch | martin.panter,2015-12-09 04:46 | Compile from makefile | review | |
| Messages (11) | |||
|---|---|---|---|
| msg245121 -(view) | Author: Martin Panter (martin.panter)*![]() | Date: 2015-06-10 08:06 | |
Once or twice I have had the math or cmath modules fail to build. There seems to be a race condition, because the _math.c file seems to be compiled twice, once for each module. Attached is a GCC wrapper that should help reproduce this fairly consistently. First, build Python normally:$ ./configure$ make -j2Then forceModules/_math.c to be recompiled using my wrapper script to amplify the race condition:$ touchModules/_math.c$ rm build/temp.linux-x86_64-3.6/media/disk/home/proj/python/cpython/Modules/_math.o$ make -j2 CC=./slow-cc.pyrunning buildrunning build_extbuilding 'cmath' extensioncompiling for cmath./slow-cc.py [. . .]/Modules/cmathmodule.obuilding 'math' extension./slow-cc.py [. . .]/Modules/mathmodule.o./slow-cc.py -fPIC -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -I./Include -I. -IInclude -I/usr/local/include -I/media/disk/home/proj/python/cpython/Include -I/media/disk/home/proj/python/cpython -c /media/disk/home/proj/python/cpython/Modules/_math.c -o build/temp.linux-x86_64-3.6/media/disk/home/proj/python/cpython/Modules/_math.oINITIAL COMPILE./slow-cc.py -fPIC -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -I./Include -I. -IInclude -I/usr/local/include -I/media/disk/home/proj/python/cpython/Include -I/media/disk/home/proj/python/cpython -c /media/disk/home/proj/python/cpython/Modules/_math.c -o build/temp.linux-x86_64-3.6/media/disk/home/proj/python/cpython/Modules/_math.oRECOMPILERECOMPIILE TRUNCATINGINITIAL COMPILE FINISHED./slow-cc.py -shared build/temp.linux-x86_64-3.6/media/disk/home/proj/python/cpython/Modules/mathmodule.o build/temp.linux-x86_64-3.6/media/disk/home/proj/python/cpython/Modules/_math.o -L/usr/local/lib -lm -o build/lib.linux-x86_64-3.6/math.cpython-36m-x86_64-linux-gnu.sobuild/temp.linux-x86_64-3.6/media/disk/home/proj/python/cpython/Modules/_math.o: file not recognized: File truncatedcollect2: error: ld returned 1 exit statusRECOMPILE FINISHED./slow-cc.py -shared build/temp.linux-x86_64-3.6/media/disk/home/proj/python/cpython/Modules/cmathmodule.o build/temp.linux-x86_64-3.6/media/disk/home/proj/python/cpython/Modules/_math.o -L/usr/local/lib -lm -o build/lib.linux-x86_64-3.6/cmath.cpython-36m-x86_64-linux-gnu.soFailed to build these modules:math | |||
| msg245298 -(view) | Author: Tal Einat (taleinat)*![]() | Date: 2015-06-13 06:51 | |
Confirmed on OSX 10.10. Here's my output:running buildrunning build_extbuilding 'cmath' extension./slow-cc.py -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -I./Include -I. -IInclude -I/usr/local/include -I/Users/taleinat/dev/cpython/Include -I/Users/taleinat/dev/cpython -c /Users/taleinat/dev/cpython/Modules/cmathmodule.c -o build/temp.macosx-10.10-x86_64-3.6/Users/taleinat/dev/cpython/Modules/cmathmodule.obuilding 'math' extension./slow-cc.py -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -I./Include -I. -IInclude -I/usr/local/include -I/Users/taleinat/dev/cpython/Include -I/Users/taleinat/dev/cpython -c /Users/taleinat/dev/cpython/Modules/mathmodule.c -o build/temp.macosx-10.10-x86_64-3.6/Users/taleinat/dev/cpython/Modules/mathmodule.o./slow-cc.py -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -I./Include -I. -IInclude -I/usr/local/include -I/Users/taleinat/dev/cpython/Include -I/Users/taleinat/dev/cpython -c /Users/taleinat/dev/cpython/Modules/_math.c -o build/temp.macosx-10.10-x86_64-3.6/Users/taleinat/dev/cpython/Modules/_math.o./slow-cc.py -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -I./Include -I. -IInclude -I/usr/local/include -I/Users/taleinat/dev/cpython/Include -I/Users/taleinat/dev/cpython -c /Users/taleinat/dev/cpython/Modules/_math.c -o build/temp.macosx-10.10-x86_64-3.6/Users/taleinat/dev/cpython/Modules/_math.oINITIAL COMPILERECOMPILERECOMPIILE TRUNCATINGINITIAL COMPILE FINISHED./slow-cc.py -bundle -undefined dynamic_lookup build/temp.macosx-10.10-x86_64-3.6/Users/taleinat/dev/cpython/Modules/mathmodule.o build/temp.macosx-10.10-x86_64-3.6/Users/taleinat/dev/cpython/Modules/_math.o -L/usr/local/lib -o build/lib.macosx-10.10-x86_64-3.6/math.cpython-36m-darwin.sold: file too small (length=0) file 'build/temp.macosx-10.10-x86_64-3.6/Users/taleinat/dev/cpython/Modules/_math.o' for architecture x86_64clang: error: linker command failed with exit code 1 (use -v to see invocation)RECOMPILE FINISHED./slow-cc.py -bundle -undefined dynamic_lookup build/temp.macosx-10.10-x86_64-3.6/Users/taleinat/dev/cpython/Modules/cmathmodule.o build/temp.macosx-10.10-x86_64-3.6/Users/taleinat/dev/cpython/Modules/_math.o -L/usr/local/lib -o build/lib.macosx-10.10-x86_64-3.6/cmath.cpython-36m-darwin.so | |||
| msg245429 -(view) | Author: Martin Panter (martin.panter)*![]() | Date: 2015-06-17 03:46 | |
I think this may have been introduced when _math.c was added as a source file of the “cmath” module inr76978 (Issue 7518). In /setup.py:585 there are two distutils.core.Extension objects, both mentioning _math.c. In the build_ext._build_extensions_parallel() method at /Lib/distutils/command/build_ext.py:449, it appears to be invoking build_extension() in two concurrent threads, once for each module, including math and cmath.I don’t know how this code is meant to work, but it looks like either the “build_ext” class should not be creating the same object file in two different threads, or /setup.py should not be giving it the same _math.c file for two different modules. | |||
| msg245453 -(view) | Author: Martin Panter (martin.panter)*![]() | Date: 2015-06-18 05:30 | |
Here is a hacky patch that adds #include "_math.c" so that it is not compiled as a separate object file. This was suggested by Mark inIssue 7518. It is far from a perfect solution, but I cannot suggest anything better without knowing more about Python’s build system. | |||
| msg245459 -(view) | Author: Tal Einat (taleinat)*![]() | Date: 2015-06-18 10:32 | |
Adding Thomas Wouters to the nosy list, since he's listed on the Experts Index under "autoconf/makefiles" as an interest area. Hopefully he can help move this forward. | |||
| msg254560 -(view) | Author: Martin Panter (martin.panter)*![]() | Date: 2015-11-12 21:46 | |
Ideally, I think the solution would be to compile _math.c once, as a dependency shared between both extension modules (perhaps like a static library if necessary). Does anyone know if the build system supports this, or how to do it?Another semi-favourable solution might be to compile _math.c twice, but send the output to separate directories, e.g. . . ./math-build/_math.o and . . ./cmath-build/_math.o. Again, I don’t know if this is practical. | |||
| msg256137 -(view) | Author: Martin Panter (martin.panter)*![]() | Date: 2015-12-09 04:46 | |
Here is an alternative patch to compile _math.c once from the main makefile, before the extension modules are built. I have only tested this on Linux, building in the source directory, and from a separate directory. | |||
| msg258349 -(view) | Author: Martin Panter (martin.panter)*![]() | Date: 2016-01-16 02:00 | |
I am inclined to try committing my second patch to 3.6, as the race condition keeps bugging me every now and again.My biggest worry is that it may not work on Windows. I assume Windows does not use the Makefile, but does use setup.py. In the worst case, perhaps I could add conditional code to restore the original setup.py parameters for Windows. | |||
| msg259362 -(view) | Author: Roundup Robot (python-dev)![]() | Date: 2016-02-02 05:59 | |
New changeset76624d47ee99 by Martin Panter in branch 'default':Issue#24421: Compile _math.c separately to avoid race conditionhttps://hg.python.org/cpython/rev/76624d47ee99 | |||
| msg260886 -(view) | Author: Roundup Robot (python-dev)![]() | Date: 2016-02-26 01:14 | |
New changeseta99580157968 by Martin Panter in branch '3.5':Issue#24421: Compile _math.c separately to avoid race conditionhttps://hg.python.org/cpython/rev/a99580157968 | |||
| msg260888 -(view) | Author: Roundup Robot (python-dev)![]() | Date: 2016-02-26 02:15 | |
New changesetd4190ed586a4 by Martin Panter in branch '2.7':Issue#24421: Compile _math.c separately to avoid race conditionhttps://hg.python.org/cpython/rev/d4190ed586a4 | |||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022-04-11 14:58:17 | admin | set | github: 68609 |
| 2016-02-26 21:46:18 | martin.panter | set | status: open -> closed resolution: fixed stage: patch review -> resolved |
| 2016-02-26 02:15:01 | python-dev | set | messages: +msg260888 |
| 2016-02-26 01:14:07 | python-dev | set | messages: +msg260886 |
| 2016-02-07 09:36:29 | Arfrever | set | nosy: +Arfrever |
| 2016-02-02 05:59:16 | python-dev | set | nosy: +python-dev messages: +msg259362 |
| 2016-01-16 02:00:35 | martin.panter | set | messages: +msg258349 |
| 2015-12-09 04:46:52 | martin.panter | set | files: +math-once.patch messages: +msg256137 versions: - Python 3.4 |
| 2015-11-12 21:46:03 | martin.panter | set | messages: +msg254560 |
| 2015-11-12 05:06:22 | floppymaster | set | nosy: +floppymaster |
| 2015-06-18 10:32:55 | taleinat | set | nosy: +twouters messages: +msg245459 |
| 2015-06-18 05:30:30 | martin.panter | set | files: +include-math.c.patch keywords: +patch messages: +msg245453 stage: patch review |
| 2015-06-17 03:46:40 | martin.panter | set | versions: + Python 2.7, Python 3.4, Python 3.5, Python 3.6 nosy: +eric.araujo,dstufft messages: +msg245429 components: + Distutils |
| 2015-06-13 06:51:15 | taleinat | set | nosy: +taleinat messages: +msg245298 |
| 2015-06-10 08:06:40 | martin.panter | create | |