Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

gh-109276: libregrtest: WASM use filename for JSON#109326

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Closed
vstinner wants to merge5 commits intopython:mainfromvstinner:regrtest_json_file2

Conversation

vstinner
Copy link
Member

@vstinnervstinner commentedSep 12, 2023•
edited by bedevere-appbot
Loading

On Emscripten and WASI platforms, libregrtest now uses a filename for the JSON file. Passing a file descriptor to a child process doesn't work on these platforms.

@vstinner
Copy link
MemberAuthor

!buildbot wasm32

@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by@vstinner for commitc45529b 🤖

The command will test the builders whose names match following regular expression:wasm32

The builders matched are:

  • wasm32-emscripten node (dynamic linking) PR
  • wasm32-wasi PR
  • wasm32-emscripten node (pthreads) PR
  • wasm32-emscripten browser (dynamic linking, no tests) PR

@vstinner
Copy link
MemberAuthor

Emscripten/WASI still use a file descriptor. It makes no sense to me :-( I'm not sure if buildbots got the right commit.

On Emscripten and WASI platforms, libregrtest now uses a filename forthe JSON file. Passing a file descriptor to a child process doesn'twork on these platforms.
@vstinner
Copy link
MemberAuthor

!buildbot wasm32

@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by@vstinner for commit9c052ba 🤖

The command will test the builders whose names match following regular expression:wasm32

The builders matched are:

  • wasm32-emscripten node (dynamic linking) PR
  • wasm32-wasi PR
  • wasm32-emscripten node (pthreads) PR
  • wasm32-emscripten browser (dynamic linking, no tests) PR

@vstinnervstinner marked this pull request as draftSeptember 12, 2023 14:08
@vstinner
Copy link
MemberAuthor

!buildbot wasm32

@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by@vstinner for commit9c052ba 🤖

The command will test the builders whose names match following regular expression:wasm32

The builders matched are:

  • wasm32-emscripten node (dynamic linking) PR
  • wasm32-wasi PR
  • wasm32-emscripten node (pthreads) PR
  • wasm32-emscripten browser (dynamic linking, no tests) PR

@vstinner
Copy link
MemberAuthor

wasm32-emscripten node (dynamic linking) PR failed with:

_PYTHON_HOSTRUNNER='/opt/emsdk/node/current/bin/node --experimental-wasm-bigint' _PYTHON_PROJECT_BASE=/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.emscripten-node-dl/build/build_oot/host _PYTHON_HOST_PLATFORM=emscripten-wasm32 PYTHONPATH=/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.emscripten-node-dl/build/build_oot/host/build/lib.emscripten-wasm32-3.13:../../Lib _PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata__emscripten_wasm32-emscripten ../build/python  ../../Tools/scripts/run_tests.py -j 1 -u all -W --slowest --fail-env-changed --timeout=900 -j2 --junit-xml test-results.xml -j2 == CPython 3.13.0a0 (heads/refs/pull/109326/merge-dirty:a70b894cdf, Sep 12 2023, 15:11:12) [GCC 9.4.0]== Linux-5.15.0-1019-azure-x86_64-with-glibc2.31 little-endian== Python build: release== cwd: /opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.emscripten-node-dl/build/build_oot/host/build/test_python_worker_432395æ== CPU count: 8== encodings: locale=UTF-8, FS=utf-8Using random seed 967649210:00:00 load avg: 2.54 Run 463 tests in parallel using 2 worker processes (timeout: 15 min, worker timeout: 20 min)main process is_emscripten: Falsemain process is_wasi: Falsemain process JSON_FILE_USE_FILENAME: Falsemain process is_emscripten: Falsemain process is_wasi: Falsemain process JSON_FILE_USE_FILENAME: Falsemain process: create TemporaryFilemain process json_type file: <class 'int'>main process json_type: 5main process: create TemporaryFilemain process json_type file: <class 'int'>main process json_type: 60:00:03 load avg: 2.54 [  1/463/1] test_optparse process crashed (Exit code 1)worker: json_file type: <class 'int'>worker: json_file: 5Traceback (most recent call last):  File "<frozen runpy>", line 198, in _run_module_as_main  File "<frozen runpy>", line 88, in _run_code  File "/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.emscripten-node-dl/build/Lib/test/libregrtest/worker.py", line 131, in <module>    main()  File "/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.emscripten-node-dl/build/Lib/test/libregrtest/worker.py", line 127, in main    worker_process(worker_json)  File "/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.emscripten-node-dl/build/Lib/test/libregrtest/worker.py", line 110, in worker_process    with open(json_file, 'w', encoding='utf-8') as fp:         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^OSError: [Errno 8] Bad file descriptor

wasm32-wasi PR failed with:

_PYTHON_HOSTRUNNER='wasmtime run --env PYTHONPATH=/build_oot/host/build/lib.wasi-wasm32-3.13:/Lib --mapdir /::../.. --' _PYTHON_PROJECT_BASE=/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.wasi/build/build_oot/host _PYTHON_HOST_PLATFORM=wasi-wasm32 PYTHONPATH=/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.wasi/build/build_oot/host/build/lib.wasi-wasm32-3.13:../../Lib _PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata__wasi_wasm32-wasi ../build/python  ../../Tools/scripts/run_tests.py -j 1 -u all -W --slowest --fail-env-changed --timeout=900 -j2 --junit-xml test-results.xml -j2 == CPython 3.13.0a0 (heads/refs/pull/109326/merge-dirty:a70b894cdf, Sep 12 2023, 15:14:49) [GCC 9.4.0]== Linux-5.15.0-1019-azure-x86_64-with-glibc2.31 little-endian== Python build: release== cwd: /opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.wasi/build/build_oot/host/build/test_python_worker_458151æ== CPU count: 8== encodings: locale=UTF-8, FS=utf-8Using random seed 768792180:00:00 load avg: 2.51 Run 463 tests in parallel using 2 worker processes (timeout: 15 min, worker timeout: 20 min)main process is_emscripten: Falsemain process is_wasi: Falsemain process JSON_FILE_USE_FILENAME: Falsemain process is_emscripten: Falsemain process is_wasi: Falsemain process JSON_FILE_USE_FILENAME: Falsemain process: create TemporaryFilemain process json_type file: <class 'int'>main process json_type: 5main process: create TemporaryFilemain process json_type file: <class 'int'>main process json_type: 60:00:00 load avg: 2.47 [  1/463/1] test_embed process crashed (Exit code 1)/Lib/test/support/os_helper.py:531: RuntimeWarning: tests may fail, unable to create temporary directory '/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.wasi/build/build_oot/host/build/test_python_48104æ': [Errno 44] No such file or directory: '/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.wasi/build/build_oot/host/build/test_python_48104æ'  with temp_dir(path=name, quiet=quiet) as temp_path:/Lib/test/support/os_helper.py:532: RuntimeWarning: tests may fail, unable to change the current working directory to '/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.wasi/build/build_oot/host/build/test_python_48104æ': [Errno 44] No such file or directory: '/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.wasi/build/build_oot/host/build/test_python_48104æ'  with change_cwd(temp_path, quiet=quiet) as cwd_dir:worker: json_file type: <class 'int'>worker: json_file: 6test_embed skipped -- test module requires subprocessTraceback (most recent call last):  File "<frozen runpy>", line 198, in _run_module_as_main  File "<frozen runpy>", line 88, in _run_code  File "/Lib/test/libregrtest/worker.py", line 131, in <module>    main()  File "/Lib/test/libregrtest/worker.py", line 127, in main    worker_process(worker_json)  File "/Lib/test/libregrtest/worker.py", line 110, in worker_process    with open(json_file, 'w', encoding='utf-8') as fp:         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^OSError: [Errno 8] Bad file descriptor

@vstinner
Copy link
MemberAuthor

I just merged PR#109337 which collect more data in test.support and test.support sub-modules.

@vstinner
Copy link
MemberAuthor

I just merged PR#109337 which collect more data in test.support and test.support sub-modules.

wasm32-emscripten node (dynamic linking) 3.x: test.pythoninfo,support.is_emscripten: True andsys.platform: emscripten as expected.

support.has_fork_support: Falsesupport.has_socket_support: Falsesupport.has_strftime_extensions: Falsesupport.has_subprocess_support: Falsesupport.is_android: Falsesupport.is_emscripten: Truesupport.is_jython: Falsesupport.is_wasi: Falsesupport_threading_helper.can_start_thread: Falsesys.platform: emscripten

wasm32-emscripten node (pthreads) 3.x: test.pythoninfo,support.is_emscripten: True andsys.platform: emscripten as expected

support.has_fork_support: Falsesupport.has_socket_support: Falsesupport.has_strftime_extensions: Falsesupport.has_subprocess_support: Falsesupport.is_android: Falsesupport.is_emscripten: Truesupport.is_jython: Falsesupport.is_wasi: Falsesupport_threading_helper.can_start_thread: Truesys.platform: emscripten

wasm32-wasi 3.x: test.pythoninfo,support.is_wasi: True andsys.platform: wasi as expected

support.has_fork_support: Falsesupport.has_socket_support: Falsesupport.has_strftime_extensions: Truesupport.has_subprocess_support: Falsesupport.is_android: Falsesupport.is_emscripten: Falsesupport.is_jython: Falsesupport.is_wasi: Truesupport_threading_helper.can_start_thread: Falsesys.platform: wasi

@vstinner
Copy link
MemberAuthor

!buildbot wasm32

@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by@vstinner for commitc436370 🤖

The command will test the builders whose names match following regular expression:wasm32

The builders matched are:

  • wasm32-emscripten node (dynamic linking) PR
  • wasm32-wasi PR
  • wasm32-emscripten node (pthreads) PR
  • wasm32-emscripten browser (dynamic linking, no tests) PR

@vstinner
Copy link
MemberAuthor

!buildbot wasm32

@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by@vstinner for commit4c8d5c8 🤖

The command will test the builders whose names match following regular expression:wasm32

The builders matched are:

  • wasm32-emscripten node (dynamic linking) PR
  • wasm32-wasi PR
  • wasm32-emscripten node (pthreads) PR
  • wasm32-emscripten browser (dynamic linking, no tests) PR

@vstinner
Copy link
MemberAuthor

!buildbot wasm32

@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by@vstinner for commit4e2cfab 🤖

The command will test the builders whose names match following regular expression:wasm32

The builders matched are:

  • wasm32-emscripten node (dynamic linking) PR
  • wasm32-wasi PR
  • wasm32-emscripten node (pthreads) PR
  • wasm32-emscripten browser (dynamic linking, no tests) PR

@vstinner
Copy link
MemberAuthor

Ok, I wrote a clean fix from this debug PR: PR#109340.

@vstinner
Copy link
MemberAuthor

wasm32-wasi PR failed: the WASI test worker process doesn't seem to have access to/tmp/test_python_json_7sbyggem of the Linuxparent process :-(

_PYTHON_HOSTRUNNER='wasmtime run --env PYTHONPATH=/build_oot/host/build/lib.wasi-wasm32-3.13:/Lib --mapdir /::../.. --' _PYTHON_PROJECT_BASE=/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.wasi/build/build_oot/host _PYTHON_HOST_PLATFORM=wasi-wasm32 PYTHONPATH=/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.wasi/build/build_oot/host/build/lib.wasi-wasm32-3.13:../../Lib _PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata__wasi_wasm32-wasi ../build/python  ../../Tools/scripts/run_tests.py -j 1 -u all -W --slowest --fail-env-changed --timeout=900 -j2 --junit-xml test-results.xml -j2 1118966 JsonFileType: int | strmain process start sys.platform: linuxmain process start is_emscripten: Falsemain process start is_wasi: Falsemain process python_cmd: ('wasmtime', 'run', '--env', 'PYTHONPATH=/build_oot/host/build/lib.wasi-wasm32-3.13:/Lib', '--mapdir', '/::../..', '--', 'python.wasm')== CPython 3.13.0a0 (heads/refs/pull/109326/merge-dirty:0540fe293c, Sep 12 2023, 17:39:37) [GCC 9.4.0]== Linux-5.15.0-1019-azure-x86_64-with-glibc2.31 little-endian== Python build: release== cwd: /opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.wasi/build/build_oot/host/build/test_python_worker_1118966æ== CPU count: 8== encodings: locale=UTF-8, FS=utf-8Using random seed 301805380:00:00 load avg: 1.66 Run 463 tests in parallel using 2 worker processes (timeout: 15 min, worker timeout: 20 min)main process: json_file_use_filename? ('wasmtime', 'run', '--env', 'PYTHONPATH=/build_oot/host/build/lib.wasi-wasm32-3.13:/Lib', '--mapdir', '/::../..', '--', 'python.wasm')main process: json_file_use_filename? ('wasmtime', 'run', '--env', 'PYTHONPATH=/build_oot/host/build/lib.wasi-wasm32-3.13:/Lib', '--mapdir', '/::../..', '--', 'python.wasm')main process: create NamedTemporaryFile /tmp/test_python_json_7sbyggemmain process json_file type: <class 'str'>main process json_file: /tmp/test_python_json_7sbyggemmain process: create NamedTemporaryFile /tmp/test_python_json_yq7jvz76main process json_file type: <class 'str'>main process json_file: /tmp/test_python_json_yq7jvz76main process executable: ('wasmtime', 'run', '--env', 'PYTHONPATH=/build_oot/host/build/lib.wasi-wasm32-3.13:/Lib', '--mapdir', '/::../..', '--', 'python.wasm')main process worker cmd: ['wasmtime', 'run', '--env', 'PYTHONPATH=/build_oot/host/build/lib.wasi-wasm32-3.13:/Lib', '--mapdir', '/::../..', '--', 'python.wasm', '-bb', '-Wdefault', '-u', '-m', 'test.libregrtest.worker', '{"tests": ["test_bigaddrspace"], "fail_fast": false, "fail_env_changed": true, "match_tests": null, "ignore_tests": null, "match_tests_dict": null, "rerun": null, "forever": false, "pgo": false, "pgo_extended": false, "output_on_failure": true, "timeout": 900.0, "verbose": 0, "quiet": false, "hunt_refleak": null, "test_dir": null, "use_junit": true, "memory_limit": null, "gc_threshold": null, "use_resources": ["audio", "curses", "largefile", "network", "decimal", "cpu", "subprocess", "urlfetch", "gui", "walltime"], "python_cmd": ["wasmtime", "run", "--env", "PYTHONPATH=/build_oot/host/build/lib.wasi-wasm32-3.13:/Lib", "--mapdir", "/::../..", "--", "python.wasm"], "randomize": true, "random_seed": 30180538, "json_file": "/tmp/test_python_json_7sbyggem", "__runtests__": true}']create_worker_process() json_file: filenamemain process executable: ('wasmtime', 'run', '--env', 'PYTHONPATH=/build_oot/host/build/lib.wasi-wasm32-3.13:/Lib', '--mapdir', '/::../..', '--', 'python.wasm')main process worker cmd: ['wasmtime', 'run', '--env', 'PYTHONPATH=/build_oot/host/build/lib.wasi-wasm32-3.13:/Lib', '--mapdir', '/::../..', '--', 'python.wasm', '-bb', '-Wdefault', '-u', '-m', 'test.libregrtest.worker', '{"tests": ["test_venv"], "fail_fast": false, "fail_env_changed": true, "match_tests": null, "ignore_tests": null, "match_tests_dict": null, "rerun": null, "forever": false, "pgo": false, "pgo_extended": false, "output_on_failure": true, "timeout": 900.0, "verbose": 0, "quiet": false, "hunt_refleak": null, "test_dir": null, "use_junit": true, "memory_limit": null, "gc_threshold": null, "use_resources": ["audio", "curses", "largefile", "network", "decimal", "cpu", "subprocess", "urlfetch", "gui", "walltime"], "python_cmd": ["wasmtime", "run", "--env", "PYTHONPATH=/build_oot/host/build/lib.wasi-wasm32-3.13:/Lib", "--mapdir", "/::../..", "--", "python.wasm"], "randomize": true, "random_seed": 30180538, "json_file": "/tmp/test_python_json_yq7jvz76", "__runtests__": true}']create_worker_process() json_file: filename0:00:00 load avg: 1.66 [  1/463/1] test_bigaddrspace process crashed (Exit code 1)42 JsonFileType: int | strworker process sys.platform: wasiworker process is_emscripten: Falseworker process is_wasi: True/Lib/test/support/os_helper.py:531: RuntimeWarning: tests may fail, unable to create temporary directory '/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.wasi/build/build_oot/host/build/test_python_230432æ': [Errno 44] No such file or directory: '/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.wasi/build/build_oot/host/build/test_python_230432æ'  with temp_dir(path=name, quiet=quiet) as temp_path:/Lib/test/support/os_helper.py:532: RuntimeWarning: tests may fail, unable to change the current working directory to '/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.wasi/build/build_oot/host/build/test_python_230432æ': [Errno 44] No such file or directory: '/opt/buildbot/bcannon-wasm/pull_request.bcannon-wasm.wasi/build/build_oot/host/build/test_python_230432æ'  with change_cwd(temp_path, quiet=quiet) as cwd_dir:worker: json_file type: <class 'str'>worker: json_file: /tmp/test_python_json_7sbyggemTraceback (most recent call last):  File "<frozen runpy>", line 198, in _run_module_as_main  File "<frozen runpy>", line 88, in _run_code  File "/Lib/test/libregrtest/worker.py", line 143, in <module>    main()  File "/Lib/test/libregrtest/worker.py", line 139, in main    worker_process(worker_json)  File "/Lib/test/libregrtest/worker.py", line 118, in worker_process    with open(json_file, 'w', encoding='utf-8') as fp:         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^FileNotFoundError: [Errno 44] No such file or directory: '/tmp/test_python_json_7sbyggem'Kill <WorkerThread #1 running test=test_venv pid=1118986 time=423 ms> process group

@vstinner
Copy link
MemberAuthor

Ok, I wrote a clean fix from this debug PR: PR#109340.

I merged this PR instead of this DEBUG PR.

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

2 participants
@vstinner@bedevere-bot
[8]ページ先頭
©2009-2025 Movatter.jp