Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

gh-109700: Improve stress tests for interpreter creation#109946

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Conversation

serhiy-storchaka
Copy link
Member

@serhiy-storchakaserhiy-storchaka commentedSep 27, 2023•
edited
Loading

  • Ensure that destructors are called in the test that created interpreters, not after finishing it.
  • Try to create interpreters simultaneously in test_create_many_threaded.
  • Mark test_create_many_threaded that requires over 6GB of memory as bigmemtest.

@bedevere-appbedevere-appbot added the testsTests in the Lib/test dir labelSep 27, 2023
@serhiy-storchakaserhiy-storchaka changed the titleWIP: gh-109700: Make stress tests on interpreter creation more stressfulgh-109700: Improve stress tests on interpreter creationNov 8, 2023
@serhiy-storchakaserhiy-storchaka added skip news needs backport to 3.11only security fixes needs backport to 3.12only security fixes 🔨 test-with-buildbotsTest PR w/ buildbots; report in status section and removed needs backport to 3.11only security fixes labelsNov 8, 2023
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by@serhiy-storchaka for commit6aecbf9 🤖

If you want to schedule another build, you need to add the🔨 test-with-buildbots label again.

@bedevere-botbedevere-bot removed the 🔨 test-with-buildbotsTest PR w/ buildbots; report in status section labelNov 8, 2023
@serhiy-storchakaserhiy-storchaka marked this pull request as ready for reviewNovember 8, 2023 21:35
@serhiy-storchakaserhiy-storchaka changed the titlegh-109700: Improve stress tests on interpreter creationgh-109700: Improve stress tests for interpreter creationNov 8, 2023
@serhiy-storchaka
Copy link
MemberAuthor

While leaks are not yet fixed, we want to minimize effect of these tests on other tests.

@serhiy-storchakaserhiy-storchaka added the needs backport to 3.13bugs and security fixes labelMay 9, 2024
@Yhg1sYhg1s removed the needs backport to 3.12only security fixes labelApr 8, 2025
@serhiy-storchaka
Copy link
MemberAuthor

  • Ensure that destructors are called in the test that created interpreters, not after finishing it.
  • Try to create/run interpreters in threads simultaneously.
  • Mark tests that requires over 6GB of memory with bigmemtest.

@serhiy-storchakaserhiy-storchaka merged commit61b50a9 intopython:mainMay 4, 2025
38 checks passed
@miss-islington-app
Copy link

Thanks@serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

@serhiy-storchakaserhiy-storchaka deleted the test_interpreters-test_create_many_threaded branchMay 4, 2025 18:53
miss-islington pushed a commit to miss-islington/cpython that referenced this pull requestMay 4, 2025
…nGH-109946)* Ensure that destructors are called in the test that created interpreters, not after finishing it.* Try to create/run interpreters in threads simultaneously.* Mark tests that requires over 6GB of memory with bigmemtest.(cherry picked from commit61b50a9)Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
@bedevere-app
Copy link

GH-133391 is a backport of this pull request to the3.13 branch.

@bedevere-appbedevere-appbot removed the needs backport to 3.13bugs and security fixes labelMay 4, 2025
serhiy-storchaka added a commit that referenced this pull requestMay 4, 2025
…09946) (GH-133391)* Ensure that destructors are called in the test that created interpreters, not after finishing it.* Try to create/run interpreters in threads simultaneously.* Mark tests that requires over 6GB of memory with bigmemtest.(cherry picked from commit61b50a9)Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
diegorusso added a commit to diegorusso/cpython that referenced this pull requestMay 4, 2025
* origin/main: (111 commits)pythongh-91048: Add filename and line number to external inspection routines (pythonGH-133385)pythongh-131178: Add tests for `ast` command-line interface (python#133329)  Regenerate pcbuild.sln in Visual Studio 2022 (python#133394)pythongh-133042: disable HACL* HMAC on Emscripten (python#133064)pythongh-133351: Fix remote PDB's multi-line block tab completion (python#133387)pythongh-109700: Improve stress tests for interpreter creation (pythonGH-109946)pythongh-81793: Skip tests for os.link() to symlink on Android (pythonGH-133388)pythongh-126835: Rename `ast_opt.c` to `ast_preprocess.c` and related stuff after moving const folding to the peephole optimizier (python#131830)pythongh-91048: Relax test_async_global_awaited_by to fix flakyness (python#133368)pythongh-132457: make staticmethod and classmethod generic (python#132460)pythongh-132805: annotationlib: Fix handling of non-constant values in FORWARDREF (python#132812)pythongh-132426: Add get_annotate_from_class_namespace replacing get_annotate_function (python#132490)pythongh-81793: Always call linkat() from os.link(), if available (pythonGH-132517)pythongh-122559: Synchronize C and Python implementation of the io module about pickling (pythonGH-122628)pythongh-69605: Add PyREPL import autocomplete feature to 'What's New' (python#133358)  bpo-44172: Keep reference to original window in curses subwindow objects (pythonGH-26226)pythonGH-133231: Changes to executor management to support proposed `sys._jit` module (pythonGH-133287)pythongh-133363: Fix Cmd completion for lines beginning with `! ` (python#133364)pythongh-132983: Introduce `_zstd` bindings module (pythonGH-133027)pythonGH-91048: Add utils for printing the call stack for asyncio tasks (python#133284)  ...
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure⚠️⚠️⚠️

Hi! The buildbotAMD64 Windows10 3.13 (tier-1) has failed when building commit5daeebb.

What do you need to do:

  1. Don't panic.
  2. Checkthe buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1501/builds/1058) and take a look at the build logs.
  4. Check if the failure is related to this commit (5daeebb) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1501/builds/1058

Failed tests:

  • test_venv

Failed subtests:

  • test_paths_depend_on_site_initialization - test.test_sysconfig.TestSysConfig.test_paths_depend_on_site_initialization

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):  File"D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\__init__.py", line203, in_force_runreturn func(*args)PermissionError:[WinError 5] Access is denied: 'D:\\temp\\test_python_6l6mytan\\tmp2458yp30\\Scripts\\python_d.exe'Traceback (most recent call last):  File"D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\__init__.py", line203, in_force_runreturn func(*args)PermissionError:[WinError 5] Access is denied: 'D:\\temp\\test_python_s9fayd3i\\tmphcwb_78x\\Scripts\\python_d.exe'Traceback (most recent call last):  File"D:\buildarea\3.13.bolen-windows10\build\Lib\shutil.py", line625, in_rmtree_unsafe    os.unlink(fullname)~~~~~~~~~^^^^^^^^^^PermissionError:[WinError 5] Access is denied: 'd:\\temp\\test_python_5wmfyhpx\\test.test_sysconfig.TestSysConfig.test_paths_depend_on_site_initialization-venv-hp6qk7yr\\Scripts\\python_d.exe'Traceback (most recent call last):  File"D:\buildarea\3.13.bolen-windows10\build\Lib\test\test_venv.py", line92, intearDown    rmtree(self.env_dir)~~~~~~^^^^^^^^^^^^^^  File"D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line470, inrmtree    _rmtree(path)~~~~~~~^^^^^^  File"D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line413, in_rmtree    _waitfor(_rmtree_inner, path,waitall=True)~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File"D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line354, in_waitfor    func(pathname)~~~~^^^^^^^^^^  File"D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line409, in_rmtree_inner    _waitfor(_rmtree_inner, fullname,waitall=True)~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File"D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line354, in_waitfor    func(pathname)~~~~^^^^^^^^^^  File"D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line412, in_rmtree_inner    _force_run(fullname, os.unlink, fullname)~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File"D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\__init__.py", line214, in_force_runreturn func(*args)PermissionError:[WinError 5] Access is denied: 'D:\\temp\\test_python_s9fayd3i\\tmphcwb_78x\\Scripts\\python_d.exe'Traceback (most recent call last):  File"D:\buildarea\3.13.bolen-windows10\build\Lib\test\test_sysconfig.py", line653, intest_paths_depend_on_site_initializationwithself.venv()as venv:~~~~~~~~~^^  File"D:\buildarea\3.13.bolen-windows10\build\Lib\contextlib.py", line148, in__exit__next(self.gen)~~~~^^^^^^^^^^  File"D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\venv.py", line27, infrom_tmpdirwith tempfile.TemporaryDirectory(prefix=prefix,dir=dir,delete=delete)as tmpdir:~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File"D:\buildarea\3.13.bolen-windows10\build\Lib\tempfile.py", line950, in__exit__self.cleanup()~~~~~~~~~~~~^^  File"D:\buildarea\3.13.bolen-windows10\build\Lib\tempfile.py", line954, incleanupself._rmtree(self.name,ignore_errors=self._ignore_cleanup_errors)~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File"D:\buildarea\3.13.bolen-windows10\build\Lib\tempfile.py", line934, in_rmtree    _shutil.rmtree(name,onexc=onexc)~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^  File"D:\buildarea\3.13.bolen-windows10\build\Lib\shutil.py", line790, inrmtreereturn _rmtree_unsafe(path, onexc)  File"D:\buildarea\3.13.bolen-windows10\build\Lib\shutil.py", line629, in_rmtree_unsafe    onexc(os.unlink, fullname, err)~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^  File"D:\buildarea\3.13.bolen-windows10\build\Lib\tempfile.py", line909, inonexc    _os.unlink(path)~~~~~~~~~~^^^^^^PermissionError:[WinError 5] Access is denied: 'd:\\temp\\test_python_5wmfyhpx\\test.test_sysconfig.TestSysConfig.test_paths_depend_on_site_initialization-venv-hp6qk7yr\\Scripts\\python_d.exe'Traceback (most recent call last):  File"D:\buildarea\3.13.bolen-windows10\build\Lib\test\test_venv.py", line92, intearDown    rmtree(self.env_dir)~~~~~~^^^^^^^^^^^^^^  File"D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line470, inrmtree    _rmtree(path)~~~~~~~^^^^^^  File"D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line413, in_rmtree    _waitfor(_rmtree_inner, path,waitall=True)~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File"D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line354, in_waitfor    func(pathname)~~~~^^^^^^^^^^  File"D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line409, in_rmtree_inner    _waitfor(_rmtree_inner, fullname,waitall=True)~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File"D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line354, in_waitfor    func(pathname)~~~~^^^^^^^^^^  File"D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line412, in_rmtree_inner    _force_run(fullname, os.unlink, fullname)~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  File"D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\__init__.py", line214, in_force_runreturn func(*args)PermissionError:[WinError 5] Access is denied: 'D:\\temp\\test_python_6l6mytan\\tmp2458yp30\\Scripts\\python_d.exe'

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@ericsnowcurrentlyericsnowcurrentlyAwaiting requested review from ericsnowcurrentlyericsnowcurrently is a code owner

Assignees
No one assigned
Labels
skip newstestsTests in the Lib/test dir
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

3 participants
@serhiy-storchaka@bedevere-bot@Yhg1s
[8]ページ先頭
©2009-2025 Movatter.jp