On official Python3.2 beta1 windows binary, I noticed followingcommand fails. (test_tcl alone won't fail)I couldn't reproduce this on binary built from source withoutinstallation.//////////////////////////////////////////////////////////////C:\Python32>.\python.exe -m test.regrtest -v test___all__ test_tcl(snip)[2/2] test_tcltestCall (test.test_tcl.TclTest) ... ERRORtestCallException (test.test_tcl.TclTest) ... ERRORtestCallException2 (test.test_tcl.TclTest) ... ERRORtestEval (test.test_tcl.TclTest) ... ERRORtestEvalException (test.test_tcl.TclTest) ... ERRORtestEvalException2 (test.test_tcl.TclTest) ... ERRORtestEvalFile (test.test_tcl.TclTest) ... ERRORtestEvalFileException (test.test_tcl.TclTest) ... ERRORtestGetVar (test.test_tcl.TclTest) ... ERRORtestGetVarArray (test.test_tcl.TclTest) ... ERRORtestGetVarArrayException (test.test_tcl.TclTest) ... ERRORtestGetVarException (test.test_tcl.TclTest) ... ERRORtestLoadWithUNC (test.test_tcl.TclTest) ... ERRORtestPackageRequireException (test.test_tcl.TclTest) ... ERRORtestSetVar (test.test_tcl.TclTest) ... ERRORtestSetVarArray (test.test_tcl.TclTest) ... ERRORtestUnsetVar (test.test_tcl.TclTest) ... ERRORtestUnsetVarArray (test.test_tcl.TclTest) ... ERRORtestUnsetVarException (test.test_tcl.TclTest) ... ERRORtestFlattenLen (test.test_tcl.TkinterTest) ... ok======================================================================ERROR: testCall (test.test_tcl.TclTest)----------------------------------------------------------------------Traceback (most recent call last):  File "C:\Python32\lib\test\test_tcl.py", line 25, in setUp    self.interp = Tcl()  File "C:\Python32\lib\tkinter\__init__.py", line 1768, in Tcl    return Tk(screenName, baseName, className, useTk)  File "C:\Python32\lib\tkinter\__init__.py", line 1674, in __init__    self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use)_tkinter.TclError: Can't find a usable init.tcl in the following directories:    C:/Python32/lib/tcl8.5 C:/lib/tcl8.5 C:/lib/tcl8.5 C:/library C:/library C:/tcl8.5.2/library C:/tcl8.5.2/libraryThis probably means that Tcl wasn't installed properly.(snip)----------------------------------------------------------------------Ran 20 tests in 1.752sFAILED (errors=19)test test_tcl failed -- multiple errors occurred1 test failed:    test_tcl1 test altered the execution environment:    test___all__

test___all__ imports a lot of modules, but I found one offollowing modules can bring same error.# just import one of these in test_main(test___all__)#    import idlelib.AutoComplete#    import tkinter.scrolledtext#    import tkinter.ttk#    import turtle//////////////////////////////////////////////////////And actually, "import tkinter" is enough.

I think this happens because 1. test___all__.py imports tkinter module, and it   imports tkinter/_fix.py2. _fix.py sets TCL_LIBRARY etc as top level routine3. regrtest.py resets os.environ after test___all__.py ends.   so TCL_LIBRARY gone.4. test_tcl.py tries to import tkinter module, but it won't   be loaded twice, so TCL_LIBRARY won't be set.5. tcl/tk cannot find its libraryI think this can be fixed by save & restore sys.modulesbefore & after each tests, so that tkinter module can beloaded again.

How about this patch? Is this kind of *fix* acceptable?# I hope this works.

I think this test disabling and failure issue should get some attention.Do the failures happen on non-Windows systems too?With installed 3.3.0b1 on Win7, test___all__, test_tcl, test_tk, test_ttk_textonly, and test_ttk_guionly all run if run first (_tk and _ttk_guionly with -ugui). All modify os.environ, resulting inWarning -- os.environ was modified by test_xxxand all disable at least one of the other tests run afterward. (test___all__ disables but is not disabled itself.) In standard alphabetic running order, test___all__ 'wins'.I would like to add test/test_idle,#15392, but I presume it currently would also not run after test___all__ or any of the others.---I patched my installation, but without the unneeded temporary (saved_values = self.saved_values is only an optimization for the repeated access in the loop that follows).        sys.modules.clear()        sys.modules.update(self.saved_modules)        del self.saved_modulesWith this,python -m test -ugui test___all__ test_tcl test_tk test_ttk_guionly test_ttk_textonlyruns and passes all five tests instead of two (test___all__ and test_ttk_textonly).Unfortunately, it causes new failures running the entire test suite:test_concurrent_futures (ok before)test_decimal (ok before)test_email (numberous failures, just one before)I stopped at test_faulthandler, there might be more but this was enough to reject the patch as is. I decided to try the principle of doing the minimum necessary to get the tests to pass. I arrived at the following, which lets all five tests run and pass, at least when run in the given order.         try:            del sys.modules['tkinter']            del sys.modules['tkinter._fix']            del sys.modules['tkinter.ttk']            del sys.modules['tkinter.test']            del sys.modules['tkinter.test.support']            del sys.modules['tkinter.test.runtktests']        except KeyError:            passThe only new failure with python -m test is test_multi-processing, which normally does not even give a warning.[197/368/6] test_multiprocessingWarning -- multiprocessing.process._dangling was modified by test_multiprocessingtest test_multiprocessing failed -- Traceback (most recent call last):  File "C:\Programs\Python33\lib\test\test_multiprocessing.py", line 1909, in test_mymanager_context_prestarted    self.assertEqual(manager._process.exitcode, 0)AssertionError: -15 != 0This might be an unrelated, intermittant failure.Since the exception above will trigger for nearly all tests (costly), and since the above does not guarantee to delete all tkinter.* modules, and since future tkinter tests might expand to test other subpackages and idle tests will import other subpackages, I tried this in __exit__.        if 'tkinter' in sys.modules:            sysmod = sys.modules            tknames = set()            for name in sysmod:                if name.startswith('tkinter'):                    tknames.add(name)            for name in tknames:                del sysmod[name]So aside from the initial test, the extra time is only used when needed. All five tests run and pass and there are no new failures with python -m test. Before:[368/368/7] test_zlib316 tests OK.7 tests failed:    test_asyncore test_datetime test_distutils test_email test_ftplib    test_import test_tcl2 tests altered the execution environment:    test___all__ test_builtin43 tests skipped:    test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp    test_codecmaps_kr test_codecmaps_tw test_crypt test_curses    test_dbm_gnu test_dbm_ndbm test_devpoll test_epoll test_fcntl    test_fork1 test_gdb test_grp test_ioctl test_kqueue test_largefile    test_nis test_openpty test_ossaudiodev test_pipes test_poll    test_posix test_pty test_pwd test_readline test_resource    test_smtpnet test_socketserver test_syslog test_threadsignals    test_timeout test_tk test_tools test_ttk_guionly test_urllib2net    test_urllibnet test_wait3 test_wait4 test_winsound test_xmlrpc_net    test_zipfile645 skips unexpected on win32:    test_gdb test_readline test_tk test_tools test_ttk_guionlyAfter:315 tests OK.6 tests failed:    test_asyncore test_datetime test_distutils test_email test_ftplib    test_import4 tests altered the execution environment:    test___all__ test_builtin test_tcl test_ttk_textonly43 tests skipped:    test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp    test_codecmaps_kr test_codecmaps_tw test_crypt test_curses    test_dbm_gnu test_dbm_ndbm test_devpoll test_epoll test_fcntl    test_fork1 test_gdb test_grp test_ioctl test_kqueue test_largefile    test_nis test_openpty test_ossaudiodev test_pipes test_poll    test_posix test_pty test_pwd test_readline test_resource    test_smtpnet test_socketserver test_syslog test_threadsignals    test_timeout test_tk test_tools test_ttk_guionly test_urllib2net    test_urllibnet test_wait3 test_wait4 test_winsound test_xmlrpc_net    test_zipfile643 skips unexpected on win32:    test_gdb test_readline test_toolsI am puzzled why one less failure and two fewer unexpected skips means one few test ok. Is that because of the two more warnings?Running with -ugui will cause test_tk and test_ttk_guionly to run instead of skip. So the above patch seems to be a pure win on Windows. What about other systems? And do any of you test experts have anything better?

I would like to see my patch (cut and paste from message above) or something like it in the next beta. It fixes the problem (since forever?) of tcl/tk/ttk tests not running properly (at least on windows) as part of the test suite. (I presume the buildbots have extra skips in order to be green.) I am reluctant to apply myself without review simply because I am ignorant enough to miss something that others might see wrong.

It would be much better to have an actual patch to review.  Then, we need someone who can test it on Windows.

I wonder if it would be sensible to have test___all__ restore the state of sys.modules after it runs.

> I wonder if it would be sensible to have test___all__ restore the> state of sys.modules after it runs.Probably not. I don't think we want to debug the consequences of havingduplicate modules lying around by way of old references registered hereand there.

Yeah, we probably do not want to go there, even though technically I think those would be bugs.  But bugs that it would only be "nice" to fix, not necessary to fix.

I tested this on Windows in my installed 3.3.0b1. Copying the inserted lines to my repository copy (where I cannot test it) gives the attached patch. On 3.2.3, the insertion line would be 1056 instead of 1158.

Hirokazu's original patch was to restore sys.modules after *every* test, not just sys.modules. That *did* cause problems because (at minimum) of references in importlib. Restoring only after __all__ would only let test_tcl run but not test_tk. And that is assuming alphabetic, not randomized order.

I was recently bitten by this issue.  I've reviewed Terry's patch on Rietveld, and have tested it myself some.  It seems to get the job done without getting in the way.

Would using import_fresh_module() in test_tcl (and others) work?

It does, in fact.  Here's a patch.

Once I worked around the example.bat problems so repository _tkinter will run for one python version (3.2), Zach's patch seems to solve the test problem as well as mine, and it is better self-contained. In other words,python -m test -ugui test___all__ test_tcl test_tk test_ttk_guionly test_ttk_textonlyruns all five tests with os.environ change the only problem noted.If Ezio or someone else who knows testing better than me agrees that it is the right patch, I am willing to apply it.

LGTM.The changes on os.environ should be unrelated.  I've seen a few other tests (e.g. test_distutils) that change it on Windows, and started writing a patch for it, but I never finished it.

After editing 2.7 files to match Zach's patch, and also after adding the changes in my patch, and also deleting tcltk directory and rerunning external.bat to rebuild tcltk/ for 2.7 with tcl/tk 8.5.2,the tests still do not all work right.--------------F:\Python\dev\py27\PCbuild> python_d -m test.regrtest -ugui test___all__ test_tcl test_tk test_ttk_guionly test_ttk_textonlytest___all__Warning -- os.environ was modified by test___all__test_tcltest test_tcl failed -- multiple errors occurred; run in verbose mode for detailstest_tktest_tk skipped -- tk not available: Can't find a usable init.tcl in the following directories:    F:/Python/dev/py27/lib/tcl8.5 F:/Python/dev/py27/lib/tcl8.5 F:/Python/dev/lib/tcl8.5 F:/Python/dev/py27/library F:/Python/dev/library F:/Python/dev/tcl8.5.2/library F:/Python/tcl8.5.2/libraryThis probably means that Tcl wasn't installed properly.------If test_tcl is run first, it works, test_ttk_guionly gives the same message as test_tk, test_ttk_testonly is OK. So the attached patch, 10652_tkfix_27.diff is an improvement, but not a complete fix.I am going to concentrate on 3.x (which means manually delete tcltk and re-compile) and apply if all goes well.

New changesetb923234b60cb by Terry Jan Reedy in branch '3.2':Issue# 10652: make tcl/tk tests run after __all__ test, patch by Zachary Ware.http://hg.python.org/cpython/rev/b923234b60cbNew changeset596e8855895e by Terry Jan Reedy in branch '3.3':Issue# 10652: make tcl/tk tests run after __all__ test, patch by Zachary Ware.http://hg.python.org/cpython/rev/596e8855895eNew changeset7c76b70075db by Terry Jan Reedy in branch 'default':#10652 null merge from 3.3 which had wrong issue #http://hg.python.org/cpython/rev/7c76b70075db

Terry, I just tested your 2.7 patch, and it does work, but the workaround that (I think) we have both been using on 3.x to find the Tcl/Tk .dlls (copying them into PCbuild) doesn't work on 2.7 for some reason.  However, if you add ..\tcltk\bin to PATH, which the PCbuild\rt.bat script (called by Tools\buildbot\test(-amd64).bat) does, your patch allows all of the test modules to run properly.I wonder if this has anything to do withissue17883's buildbot problems?

Having another chance to look at this one, my previous message was incorrect; Terry's patch does not fix the issue for an installed Python 2.7.  The only change it needs to work, though, is to replace 'Tkinter' with 'FixTk' in the import_fresh_module call.

An alternative that works and also removes repeated "Warning -- os.environ was modified by test_*" is to import FixTk at the top of test_support, allowing the environment to be set up and to persist throughout all of the tests.  I'm not sure if this is the right way to go about the problem, though.

Zach, do you have any further thoughts in light of patches pushed since? What do you think is the exact remaining issue?

As for what's actually wrong here, Hirokazu Yamamoto's diagnosis inmsg123615 (adjusted for 2.7) is correct.Either of the last two patches I posted should work to fix this issue, but they're both just band-aids rather than a real, once-and-for-all fix.#20035 (which I need to rewrite again) will be a once-and-for-all fix for 3.5 by getting rid of tkinter._fix, but I'm not sure if such an invasive fix is appropriate for 2.7 and 3.4.  I prefer the second band-aid (import FixTk at the top of test_support) just because it's simpler and also prevents the 'os.environ has been changed' warnings.A workaround that doesn't require a patch is to just set TCL_LIBRARY manually in your environment before running the tests, which is how the 3.x buildbots are currently working (seeTools/buildbot/test.bat:4).For the record, I'm not sure why the 3.x fix we came up with earlier in this issue worked, though I suspect it has something to do with _fix being part of the tkinter package.  The same patch fails on 2.7 because Tkinter is not a package; FixTk is a standalone module and is thus completely unaffected by support.import_fresh_module('Tkinter').  Fresh-importing FixTk itself works, since it's what we actually need to run.

The root of this issue was fixed by#20035 for 3.5+.  Terry, do you still have issues with this with 2.7, and is it still worth trying to fix?

I believe the following means 'No' and that you can close this.f:\dev\27>python -m test.regrtest test___all__ test_tclRunning Debug|Win32 interpreter...Run tests sequentially0:00:00 [1/2] test___all__0:00:24 [2/2] test_tclAll 2 tests OK.

Excellent.  Thanks, Terry!