At some point (probably in 3.2) the support for shared modules built usingModules/Setup was broken, for two reasons: - Python no longer considers 'foomodule.so' when looking for a module called 'foo', butModules/makesetup still appends 'module.so' for a 'foo' module. - Python no longer considers Modules/ to be the exec_prefix (and instead uses what was loaded from the build-info .txt written by setup.py) but the build process still leaves theseModules/Setup-built shared modules in the Modules directory (and it doesn't know about the directory setup.py will make, so it can't really do anything else.)This patch fixes both problems, by makingModules/makesetup name shared library modules 'foo.so' and by adding the Modules subdirectory to sys.path (after the setup.py-provided directory) when running from the build directory. (The existing build process has no problem _installing_ theModules/Setup-built extension modules, it's just Python that doesn't know how to find them.)

Your original patch LGTM.  However, since it was written, Setup *shared* support became even more broken due to changes in Makefile macros.  Attached is an updated patch.  I've never tried customizing Setup before so I don't have a lot of confidence that it now works as expected in all cases, but I tried a few cases (like building _ssl with a non-standard OpenSSL on OS X) and they did work.

New changeset6c468df214dc by Ned Deily in branch '3.4':Issue#17095: FixModules/Setup *shared* support.http://hg.python.org/cpython/rev/6c468df214dcNew changeset227ce85bdbe0 by Ned Deily in branch 'default':Issue#17095: FixModules/Setup *shared* support.http://hg.python.org/cpython/rev/227ce85bdbe0

Committed for release in 3.4.2 and 3.5.0.

I reopen the issue because these changes broke the decimal module when the _decimal module is missing: see the issue#22285.Martin von Loewis proposed to build modules in the build directory, not in the Modules directory directly:http://bugs.python.org/issue22285#msg225961

Clearly this breaks running from the build directory for various modules when their optional C extension module build fails.  I'll either change the Setup shared builds to use a build directory or revert the path changes prior to the upcoming 3.4.2.

New changeset781454f792c4 by Ned Deily in branch '3.4':Issue#17095: Temporarily revert getpath.c change that added the Moduleshttps://hg.python.org/cpython/rev/781454f792c4New changesetd3939f602e1f by Ned Deily in branch 'default':Issue#17095: merge from 3.4https://hg.python.org/cpython/rev/d3939f602e1f

Hi Ned,What are the news for this issue?

Stéphane, as I understand it, this issue covers two problems:(1) Building and installing shared modules usingModules/Setup was broken for current Python 3 releases;(2) Using those shared modules when running Python from a build directory (rather than from a normal installed Python) is broken.Problem (1) is now fixed.  Problem (2) remains to be properly fixed; it's a little tedious to get right as a fix needs to take into account the SO_ABI and ensure built modules get properly cleaned and all the right Makefile dependencies work.  I'd be happy to review a patch if someone gets to it before I do.  But it seems to me that (1) was the serious problem; (2) is more of a minor annoyance for developers with easy workarounds, e.g. use PYTHONPATH or install Python to a temporary location for testing.

Hi Ned,For the first point, maybe close this issue and open a new bug for the second.What do you think?

Sorry, I don't see the point of opening another issue, especially since most of the discussion in this issue has been about the second problem.  I have updated the issue title to be more specific, though.

Thanks Ned