Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

gh-109276: libregrtest: add RunTests.work_dir#109290

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Closed
vstinner wants to merge3 commits intopython:mainfromvstinner:regrtest_work_dir

Conversation

vstinner
Copy link
Member

@vstinnervstinner commentedSep 11, 2023•
edited by bedevere-appbot
Loading

  • WorkerThread now always creates a temporary directory, even on Emscripten and WASI: it's used as the working directory of the test worker process.
  • Fix Emscripten and WASI: start the test worker process in the Python source code directory, where 'python.js' and 'python.wasm' can be found. Then worker_process() goes to the temporary directory created to run tests.

@vstinner
Copy link
MemberAuthor

This PR fix how libregrtest creates temporary directories for worker processes and which directory is used as the working directory of worker processes. It seems like two tests fail with this change since they leak temporary files/directories!

  • test.test_asyncio.test_events
  • test_compileall

@vstinner
Copy link
MemberAuthor

It seems like two tests fail with this change since they leak temporary files/directories!
test.test_asyncio.test_events
test_compileall

I created issue#109295 to track this issue and PR#109298 to fix it.

@vstinner
Copy link
MemberAuthor

Oh, and Windows two tests also failed, but other tests: test_generated_cases and test_os.

ERROR: test_access_denied (test.test_os.StatAttributeTests.test_access_denied)----------------------------------------------------------------------Traceback (most recent call last):  File "D:\a\cpython\cpython\Lib\test\test_os.py", line 742, in test_access_denied    create_file(fname, b'ABC')  File "D:\a\cpython\cpython\Lib\test\test_os.py", line 94, in create_file    with open(filename, "xb", 0) as fp:         ^^^^^^^^^^^^^^^^^^^^^^^FileExistsError: [Errno 17] File exists: 'C:\\Users\\runneradmin\\AppData\\Local\\Temp\\test_python_worker_a5djsgg5\\@test_5340_tmp�'

and

0:19:14 load avg: 6.45 [2/3/2] test_generated_cases failed (uncaught exception)Re-running test_generated_cases in verbose modetest test_generated_cases crashed -- Traceback (most recent call last):  File "D:\a\cpython\cpython\Lib\test\libregrtest\single.py", line 139, in _runtest_env_changed_exc    _load_run_test(result, runtests)  File "D:\a\cpython\cpython\Lib\test\libregrtest\single.py", line 87, in _load_run_test    test_mod = importlib.import_module(module_name)               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "D:\a\cpython\cpython\Lib\importlib\__init__.py", line 88, in import_module    return _bootstrap._gcd_import(name[level:], package, level)           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "<frozen importlib._bootstrap>", line 1381, in _gcd_import  File "<frozen importlib._bootstrap>", line 1354, in _find_and_load  File "<frozen importlib._bootstrap>", line 1325, in _find_and_load_unlocked  File "<frozen importlib._bootstrap>", line 929, in _load_unlocked  File "<frozen importlib._bootstrap_external>", line 1006, in exec_module  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed  File "D:\a\cpython\cpython\Lib\test\test_generated_cases.py", line 11, in <module>    import generate_cases  File "D:\a\cpython\cpython\Tools\cases_generator\generate_cases.py", line 41, in <module>    DEFAULT_INPUT = os.path.relpath(os.path.join(ROOT, "Python/bytecodes.c"))                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File "D:\a\cpython\cpython\Lib\ntpath.py", line 791, in relpath    raise ValueError("path is on mount %r, start on mount %r" % (ValueError: path is on mount 'D:', start on mount 'C:'

@vstinnervstinnerforce-pushed theregrtest_work_dir branch 2 times, most recently fromb7dec3c to0507687CompareSeptember 12, 2023 02:12
* Enable collect_sysconfig() on Windows.* Add sysconfig 'abs_builddir' and 'srcdir'* Add sysconfig.is_python_build()* Add tempfile.gettempdir()* Remove compatiblity with Python 2.7 (print_function).
* WorkerThread now always creates a temporary directory, even on  Emscripten and WASI: it's used as the working directory of the test  worker process.* Fix Emscripten and WASI: start the test worker process in the  Python source code directory, where 'python.js' and 'python.wasm'  can be found. Then worker_process() goes to the temporary directory  created to run tests.* --cleanup now also removes "temporary" directories of test worker  processes (in /tmp).
@vstinner
Copy link
MemberAuthor

Many tests fail on the macOS CI because the AF_UNIX domain name is too longer (more than ~100 bytes):

15 tests failed:    test.test_asyncio.test_unix_events    test.test_concurrent_futures.test_as_completed    test.test_concurrent_futures.test_deadlock    test.test_concurrent_futures.test_init    test.test_concurrent_futures.test_process_pool    test.test_concurrent_futures.test_shutdown    test.test_concurrent_futures.test_wait    test.test_multiprocessing_forkserver.test_manager    test.test_multiprocessing_forkserver.test_misc    test.test_multiprocessing_forkserver.test_processes    test.test_multiprocessing_forkserver.test_threads    test.test_multiprocessing_spawn.test_manager    test.test_multiprocessing_spawn.test_misc    test.test_multiprocessing_spawn.test_processes    test_multiprocessing_main_handling

The main test process:

tempfile.gettempdir: /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/Tself.tmp_dir: /Users/runner/work/cpython/cpython/buildruntests.work_dir: /Users/runner/work/cpython/cpython/build/test_python_4592æcwd: /Users/runner/work/cpython/cpython/build/test_python_4592æ

A worker test process:

worker test_turtle (pid 4597): /private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/test_python_worker_2eq4te_m

/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/test_python_worker_2eq4te_m is 84 (ASCII) characters long.

multiprocessing.util.get_temp_dir() creates a temporary directory withpymp-XXXXXXXX (13 characters) name.

multiprocessing.connection.arbitrary_connection(AF_UNIX) creates alistener-XXXXXXXX (17 characters) temporary file inpymp-XXXXXXXX directory.

A UNIX domain name liike/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/test_python_worker_2eq4te_m/pymp-XXXXXXXX/listener-XXXXXXXX is already 117 characters long.

@vstinner
Copy link
MemberAuthor

Windows x64:

  • Python source code:D:\a\cpython\cpython (driveD:)
  • Temporary directory:C:\Users\RUNNER~1\AppData\Local\Temp (driveC:)

test.pythoninfo:

sysconfig.is_python_build: Truesysconfig[prefix]: D:\a\cpython\cpythonsysconfig[srcdir]: D:\a\cpython\cpythontempfile.gettempdir: C:\Users\RUNNER~1\AppData\Local\Temp

Main test process:

tempfile.gettempdir: C:\Users\RUNNER~1\AppData\Local\Tempself.tmp_dir: D:\a\cpython\cpython\buildruntests.work_dir: D:\a\cpython\cpython\build\test_python_3212�cwd: D:\a\cpython\cpython\build\test_python_3212�

A test worker process:

C:\Users\RUNNER~1\AppData\Local\Temp\test_python_worker_xa_vmpox => C:\Users\runneradmin\AppData\Local\Temp\test_python_worker_xa_vmpox

test_generated_cases was skipped:

0:16:54 load avg: 11.61 [393/463/2] test_generated_cases skipped -- running (1): test_subprocess (1 min 1 sec)test_generated_cases skipped -- the current working directory and the Python source code directory have different mount drives (C: and D:)

@vstinner
Copy link
MemberAuthor

PR#106219 is awaiting for an Emscripten/WASI fix for libregrtest.

@vstinner
Copy link
MemberAuthor

This change seems to be too disruptive, so I wrote a more conservative change (come back to old behavior): PR#109313.

@vstinner
Copy link
MemberAuthor

This change seems to be too disruptive, so I wrote a more conservative change (come back to old behavior): PR#109313.

Merged. I close this PR.

@vstinnervstinner deleted the regrtest_work_dir branchSeptember 13, 2023 01:46
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

1 participant
@vstinner
[8]ページ先頭
©2009-2025 Movatter.jp