Currently there is only one platform directory for all linux architectures, there are two reasons to have a separate directory for each architecture: - the contents of the files in this directory differ   (most prominent case are the RTLD_* constants on various   architectures). - The _sysconfigdata.py module is installed in the platdir.   A cross build of the extensions in the standard library   is done by running the python interpreter for the build   machine with the platdir for the host machine.  This   currently works for cross builds targeting a different   os, but not cross building from one linux architecture   to another.This would then removeLib/plat-linux and introduce someLib/plat-$(PLATFORM_TRIPLET) directories.

ping? I would like to get this into 3.5.

+1 from me.

+1 from the Python team from Fedora, the patch looks good from downstream standpoint.

I don't know what problems I might have run into previously, but it's working now and seems fine to me.  +1

WhatLib/plat-$(PLATFORM_TRIPLET) directories will be introduced?This patch would make situation worse on rarely used architectures (these withoutLib/plat-$(PLATFORM_TRIPLET) distributed in repository of CPython) if patch for bug#12619 is not applied.

This would likely improve life for folks with cross-compile toolchains.

attaching an updated patch. besides renaming of the platform directory, the following changes are made: - it makes the name of the internal _sysconfigdata module dependent   on the abiflags - it moves the _sysconfigdata module into the platform directory.   This address as well#22724, it should address#18083 as well,   provided that some multiarch tuples for solaris are defined. - it fixes regeneration of the Python modules from C headers for   cross builds. - it introduces a sys.implementation._multiarch attribute to construct   the name of the platform directory without having the   _sysconfigdata module available (which now is located in the platform   dir)One outstanding issue probably is the generation of the platform directory in the source tree, not in the build tree (but this is existing behaviour).This should address#12619 (this patch probably needs an update),

This may be of interest for iOS and Android cross-compilation, so I've added Xavier de Gaye to the nosy list, while Russell Keith-Magee added himself.

New changeset78d2cb7f66b6 by doko in branch 'default':- Issue#23968: Rename the platform directory from plat-$(MACHDEP) tohttps://hg.python.org/cpython/rev/78d2cb7f66b6

now checked in, after 3.6.0a2, to watch behaviour on different platforms. Still needs doc updates.

New changeseta062c60c9f7d by doko in branch 'default':- Issue#23968: Update distutils/sysconfig.py to look for the renamedhttps://hg.python.org/cpython/rev/a062c60c9f7d

New changesete2b09c00ee24 by Ned Deily in branch 'default':Issue#23968: Fix installs of the renamed config directory for OS Xhttps://hg.python.org/cpython/rev/e2b09c00ee24

There is a problem with the current implementation.  When running ./configure using compilers other than a current gcc, for example, when using clang or older versions of gcc, the build variable MULTIARCH may not get set because those compilers do not support --print-multiarch:configure.ac:744MULTIARCH=$($CC --print-multiarch 2>/dev/null)But, a bit later in ./configure, PLATFORM_TRIPLET gets set as a result of running conftest.c.  The end result is that PLATFORM_TRIPET is non-empty but MULTIARCH is empty.  Still later in configure.ac (around 4477:if test x$PLATFORM_TRIPLET = x; then  LIBPL='$(prefix)'"/lib/python${VERSION}/config-${LDVERSION}"else  LIBPL='$(prefix)'"/lib/python${VERSION}/config-${LDVERSION}-${PLATFORM_TRIPLET}"fiSo, the value of LIBPL is extended to include PLATFORM_TRIPLET regardless of whether MULTIARCH has a value.This causes problems at runtime because get_makefile_filename() inLib/sysconfig.py now uses the value of sys.implemntation's _multiarch attribute (whose value is compiled in sysmodule.c from MULTIARCH) to form the path to the config directory (the equivalent of LIBPL).  In these cases, sys.implementation will have no_multiarch attribute so sysconfig looks for the config directory in the pre-3.6 location (e.g. .../lib/python3.6/config-3.6m) even though it was installed in the new location (.../lib/python3.6/config-3.6m-darwin).  Fortunately, both test_sysconfig and test_distutils catch this:======================================================================FAIL: test_srcdir (test.test_sysconfig.TestSysConfig)----------------------------------------------------------------------Traceback (most recent call last):  File "/py/dev/3x/root/uxd/lib/python3.6/test/test_sysconfig.py", line 346, in test_srcdir    self.assertTrue(os.path.isdir(srcdir), srcdir)AssertionError: False is not true : /py/dev/3x/root/uxd/lib/python3.6/config-3.6dm======================================================================FAIL: test_get_makefile_filename (test.test_sysconfig.MakefileTests)----------------------------------------------------------------------Traceback (most recent call last):  File "/py/dev/3x/root/uxd/lib/python3.6/test/test_sysconfig.py", line 414, in test_get_makefile_filename    self.assertTrue(os.path.isfile(makefile), makefile)AssertionError: False is not true : /py/dev/3x/root/uxd/lib/python3.6/config-3.6dm/Makefile======================================================================FAIL: test_srcdir (distutils.tests.test_sysconfig.SysconfigTestCase)----------------------------------------------------------------------Traceback (most recent call last):  File "/py/dev/3x/root/uxd/lib/python3.6/distutils/tests/test_sysconfig.py", line 61, in test_srcdir    self.assertTrue(os.path.isdir(srcdir), srcdir)AssertionError: False is not true : /py/dev/3x/root/uxd/lib/python3.6/config-3.6dm----------------------------------------------------------------------I'm not sure what the best solution is.

The same problem occurs when cross-compiling for Android and running test_sysconfig and test_distutils.Some other points:* The generation of the platform directory breaks the support of read only source trees. The files of this  directory are not tracked by mercurial and printed by 'hg status'.* The new files in the platform directory are regenerated from C headers of the build machine.

> The same problem occurs when cross-compiling for Android and running test_sysconfig and test_distutils.This is misleading. Actually, test_sysconfig and test_distutils do not fail when run from the source tree, but fail on an installed python, whatever the compilation mode: native or cross-compilation.

Regeneration of the platdir files needs to be aware of the value of sysroot [1]. For example on Android cross-builds, sysroot must be set to the path of the headers and libraries for the targeted Android API level corresponding to a version of the Android libc and a version of Android. So, automatically regenerating those files in this case requires to look into the CC, CFLAGS and CPPFLAGS environment variables for sysroot. OTOH RTLD_* constants are exposed in the posix module (issue 13226) and the platdir files usefulness is limited now.Cross building from one linux architecture to another is possible with gnu make VPATH support by building outside the source tree. Android is also a linux architecture and building outside the source tree makes sense, not only to work around the problem described in the second item ofmsg241143, but also to build for different Android API levels and identify the results of those builds.It is not clear that the changes made in this issue fixes correctlyissue 22724, seemsg269359.I think the scope of this isssue should be restricted to multiarch build systems, i.e. when gcc has been configured with '--enable-multiarch' [2].[1] Options for Directory Search:https://gcc.gnu.org/onlinedocs/gcc/Directory-Options.html[2] Installing GCC: Configuration:https://gcc.gnu.org/install/configure.html

To reproduce the problem on any system where Python is built with configure:$ mkdir -p tmp/build tmp/install$ cd tmp$ hg clonehttps://hg.python.org/cpython cpython$ cd build$ $(cd ../cpython && pwd)/configure --prefix=$(cd ../install && pwd) && make install$ ../install/bin/python3 -m test -v test_sysconfig[snip]======================================================================FAIL: test_srcdir (test.test_sysconfig.TestSysConfig)----------------------------------------------------------------------Traceback (most recent call last):  File "/path_to/tmp/install/lib/python3.6/test/test_sysconfig.py", line 346, in test_srcdir    self.assertTrue(os.path.isdir(srcdir), srcdir)AssertionError: False is not true : /path_to/tmp/install/lib/python3.6/config-3.6m======================================================================FAIL: test_get_makefile_filename (test.test_sysconfig.MakefileTests)----------------------------------------------------------------------Traceback (most recent call last):  File "/path_to/tmp/install/lib/python3.6/test/test_sysconfig.py", line 414, in test_get_makefile_filename    self.assertTrue(os.path.isfile(makefile), makefile)AssertionError: False is not true : /path_to/tmp/install/lib/python3.6/config-3.6m/Makefile----------------------------------------------------------------------Ran 21 tests in 0.063sFAILED (failures=2, skipped=2)test test_sysconfig failedtest_sysconfig failed1 test failed:    test_sysconfigTotal duration: 0:00:01

This problem occured for the first time on the "installed Python" buildbot (setup by Zachary) athttp://buildbot.python.org/all/builders/x86%20Gentoo%20Installed%20with%20X%203.x/builds/747on the day changeset78d2cb7f66b6 has been commited (seemsg268528).Those "installed Python" buildbot are very helpful, thanks Zachary.

The reason I would like this problem fixed is that the test suite is run on an installed Python when cross-compiling, obviously you cannot run the test suite with the cross-compiled Python on the source tree of the build system.

It's not just broken for cross-compiling.  As I noted earlier, it's broken when using other than gcc, such as is the case with current OS X systems.  This needs to be fixed before release.

It is also broken for gcc, see the buildbot 'compile' output.It seems broken for all non windows builds whatever the compiler.

New changesetddc4bdae5e41 by Ned Deily in branch 'default':Issue#23968: Make OS X installer build script aware of renamed platformhttps://hg.python.org/cpython/rev/ddc4bdae5e41

Any update on this issue?test_srcdir() of test_sysconfig still fails on "x86 Gentoo Installed with X 3.x" buildbot. I opened the issue#27798 which was closed a duplicate of this issue.First failure of the buildbot:http://buildbot.python.org/all/builders/x86%20Gentoo%20Installed%20with%20X%203.x/builds/747This build contains many changes. I don't know which one introduced the issue?* changea062c60c9f7d60db2b63af88e2d676e617398d01: Update distutils/sysconfig.py to look for the renamed _sysconfigdata module too* change78d2cb7f66b602ec496800fbbe71e590c0f56b00: Rename the platform directory from plat-$(MACHDEP) to plat-$(PLATFORM_TRIPLET)Ned Deily: "This needs to be fixed before release."I agree. I use the release blocker priority as a reminder. Well, if the issue cannot be fixed, I suggest to revert changes related to this issue."It's not just broken for cross-compiling.  As I noted earlier, it's broken when using other than gcc, such as is the case with current OS X systems."test_sysconfig fails on "x86 Gentoo Installed with X 3.x" buildbot which uses GCC. Extract of the configure output:   checking for gcc... gcc

please could somebody test the attached patch (I'm also trying to setup a gentoo env)? The idea is to keep the multiarch and triplet macros in sync.diff -ra7f3678f9509 configure.ac--- a/configure.acSat Aug 20 03:05:13 2016 +0200+++ b/configure.acSat Aug 20 03:54:39 2016 +0200@@ -882,6 +882,8 @@   if test x$PLATFORM_TRIPLET != x$MULTIARCH; then     AC_MSG_ERROR([internal configure error for the platform triplet, please file a bug report])   fi+elif test x$PLATFORM_TRIPLET != x && test x$MULTIARCH = x; then+  MULTIARCH=$PLATFORM_TRIPLET fi if test x$PLATFORM_TRIPLET = x; then   PLATDIR=plat-$MACHDEP

I've just tried doko's patch on Arch Linux. Before this patch there are three test failures in test_distutils and test_sysconfig, just like Gentoo. After the patch, all tests in test_distutils and test_sysconfig pass.On my PC both `gcc -print-multiarch` and `gcc --print-multiarch` print an empty string.By the way, could you attach this patch file to roundup, so that others can download it?Reference: Arch Linux's build script of python-hg:https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=python-hg

New changeset5c77488830bc by doko in branch 'default':- Issue#23968, keep platform_triplet and multiarch macros in synchttps://hg.python.org/cpython/rev/5c77488830bc

now fixed.

Changeset5c77488830bc brings a regression - Android build fails with the following message:export H2PY; H2PY="$PYTHON_FOR_BUILD /home/yen/python3-android/src/cpython/build-target/../Tools/scripts/h2py.py"; \cd ../Lib/plat-aarch64-linux-gnu;  ./regen+ _PYTHON_PROJECT_BASE=/home/yen/python3-android/src/cpython/build-target _PYTHON_HOST_PLATFORM=linux-aarch64 PYTHONPATH=/home/yen/python3-android/src/cpython/build-target/build/lib.linux-aarch64-3.6:../Lib:../Lib/plat-aarch64-linux-gnu python3.6 /home/yen/python3-android/src/cpython/build-target/../Tools/scripts/h2py.py -i '(u_long)' /opt/android-ndk/platforms/android-21/arch-arm64/usr/include/sys/types.h /opt/android-ndk/platforms/android-21/arch-arm64/usr/include/netinet/in.h /opt/android-ndk/platforms/android-21/arch-arm64/usr/include/dlfcn.h./regen: line 33: _PYTHON_PROJECT_BASE=/home/yen/python3-android/src/cpython/build-target: No such file or directorymake[1]: *** [Makefile:1324: ../Lib/plat-aarch64-linux-gnu] Error 127make[1]: Leaving directory '/home/yen/python3-android/src/cpython/build-target'make: *** [Makefile:36: python] Error 2The previous changeset1902e1d79e25 builds fine. Prepending `PYTHON_FOR_BUILD` in configure.ac with `env` makes the whole build work again.From the message seems plat-* directories are re-generated with a different name. Is it expected?

Oh didn't see the title of this issue. Here's the patch for Android.

I don't see how the suggested android fix is related, and what it is supposed to fix. It adds the env command before setting the environment variables, which should be a no-op.However I think we should introduce different platform triplets for android.  Please follow-up inhttp://bugs.python.org/issue27917

Well changes in this issue are not the root cause of my problem - it just exposes a broken implementation done long time ago. I'll examine more and open a new bug.