Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

gh-109649: Use os.process_cpu_count()#110165

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
vstinner merged 4 commits intopython:mainfromvstinner:use_process_count
Oct 1, 2023

Conversation

vstinner
Copy link
Member

@vstinnervstinner commentedSep 30, 2023•
edited by github-actionsbot
Loading

Replace os.cpu_count() with os.process_cpu_count() in modules:

  • compileall
  • concurrent.futures
  • multiprocessing

Replace os.cpu_count() with os.process_cpu_count() in programs:

  • _decimal deccheck.py test
  • freeze.py
  • multissltests.py
  • python -m test (regrtest)
  • wasm_build.py

Other changes:

  • test.pythoninfo logs os.process_cpu_count().
  • regrtest gets os.process_cpu_count() / os.cpu_count() in headers.

📚 Documentation preview 📚:https://cpython-previews--110165.org.readthedocs.build/

Replace os.cpu_count() with os.process_cpu_count() in modules:* compileall* concurrent.futures* multiprocessingReplace os.cpu_count() with os.process_cpu_count() in programs:* _decimal deccheck.py test* freeze.py* multissltests.py* python -m test (regrtest)* wasm_build.pyOther changes:* test.pythoninfo logs os.process_cpu_count().* regrtest gets os.process_cpu_count() / os.cpu_count() in headers.
@vstinner
Copy link
MemberAuthor

Example:CPU count: 2 (process) / 12 (system)

vstinner@mona$ taskset -c 0-1 ./python -m test == CPython 3.13.0a0 (heads/use_process_count:a8514200b8, Oct 1 2023, 01:31:10) [GCC 13.2.1 20230728 (Red Hat 13.2.1-1)]== Linux-6.4.15-200.fc38.x86_64-x86_64-with-glibc2.37 little-endian== Python build: debug== cwd: /home/vstinner/python/main/build/test_python_worker_2132029æ== CPU count: 2 (process) / 12 (system)(...)

@vstinner
Copy link
MemberAuthor

Replace os.cpu_count() with os.process_cpu_count() in programs:

  • _decimal deccheck.py test
  • freeze.py
  • multissltests.py
  • python -m test (regrtest)
  • wasm_build.py

multissltests.py and wasm_build.py use the code below to keep support for old Python versions:

ifhasattr(os,'process_cpu_count'):jobs=os.process_cpu_count()else:jobs=os.cpu_count()
stonebig reacted with heart emoji

@vstinnervstinner merged commita46e960 intopython:mainOct 1, 2023
@vstinnervstinner deleted the use_process_count branchOctober 1, 2023 01:14
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure⚠️⚠️⚠️

Hi! The buildbots390x SLES 3.x has failed when building commita46e960.

What do you need to do:

  1. Don't panic.
  2. Checkthe buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/540/builds/6704) and take a look at the build logs.
  4. Check if the failure is related to this commit (a46e960) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/540/builds/6704

Failed tests:

  • test.test_concurrent_futures.test_thread_pool

Failed subtests:

  • test_idle_thread_reuse - test.test_concurrent_futures.test_thread_pool.ThreadPoolExecutorTest.test_idle_thread_reuse

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):  File"/home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/Lib/test/test_concurrent_futures/test_thread_pool.py", line49, intest_idle_thread_reuseself.assertEqual(len(executor._threads),1)AssertionError:2 != 1

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure⚠️⚠️⚠️

Hi! The buildbots390x Debian 3.x has failed when building commita46e960.

What do you need to do:

  1. Don't panic.
  2. Checkthe buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/49/builds/6739) and take a look at the build logs.
  4. Check if the failure is related to this commit (a46e960) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/49/builds/6739

Failed tests:

  • test.test_multiprocessing_forkserver.test_processes

Summary of the results of the build (if available):

==

Click to see traceback logs
remote:Enumerating objects: 40, done.remote:Counting objects:   2% (1/40)remote:Counting objects:   5% (2/40)remote:Counting objects:   7% (3/40)remote:Counting objects:  10% (4/40)remote:Counting objects:  12% (5/40)remote:Counting objects:  15% (6/40)remote:Counting objects:  17% (7/40)remote:Counting objects:  20% (8/40)remote:Counting objects:  22% (9/40)remote:Counting objects:  25% (10/40)remote:Counting objects:  27% (11/40)remote:Counting objects:  30% (12/40)remote:Counting objects:  32% (13/40)remote:Counting objects:  35% (14/40)remote:Counting objects:  37% (15/40)remote:Counting objects:  40% (16/40)remote:Counting objects:  42% (17/40)remote:Counting objects:  45% (18/40)remote:Counting objects:  47% (19/40)remote:Counting objects:  50% (20/40)remote:Counting objects:  52% (21/40)remote:Counting objects:  55% (22/40)remote:Counting objects:  57% (23/40)remote:Counting objects:  60% (24/40)remote:Counting objects:  62% (25/40)remote:Counting objects:  65% (26/40)remote:Counting objects:  67% (27/40)remote:Counting objects:  70% (28/40)remote:Counting objects:  72% (29/40)remote:Counting objects:  75% (30/40)remote:Counting objects:  77% (31/40)remote:Counting objects:  80% (32/40)remote:Counting objects:  82% (33/40)remote:Counting objects:  85% (34/40)remote:Counting objects:  87% (35/40)remote:Counting objects:  90% (36/40)remote:Counting objects:  92% (37/40)remote:Counting objects:  95% (38/40)remote:Counting objects:  97% (39/40)remote:Counting objects: 100% (40/40)remote:Counting objects: 100% (40/40), done.remote:Compressing objects:  16% (1/6)remote:Compressing objects:  33% (2/6)remote:Compressing objects:  50% (3/6)remote:Compressing objects:  66% (4/6)remote:Compressing objects:  83% (5/6)remote:Compressing objects: 100% (6/6)remote:Compressing objects: 100% (6/6), done.remote:Total 40 (delta 34), reused 40 (delta 34), pack-reused 0From https://github.com/python/cpython * branch                  main       -> FETCH_HEADNote:switching to 'a46e96076898d126c9f276aef1934195aac34b4e'.You are in 'detached HEAD' state. You can look around, make experimentalchanges and commit them, and you can discard any commits you make in thisstate without impacting any branches by switching back to a branch.If you want to create a new branch to retain commits you create, you maydo so (now or later) by using -c with the switch command. Example:  git switch -c <new-branch-name>Or undo this operation with:  git switch -Turn off this advice by setting config variable advice.detachedHead to falseHEAD is now at a46e960768 gh-109649: Use os.process_cpu_count() (#110165)Switched to and reset branch 'main'make:*** [Makefile:2018: buildbottest] Error 5

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure⚠️⚠️⚠️

Hi! The buildbots390x Fedora 3.x has failed when building commita46e960.

What do you need to do:

  1. Don't panic.
  2. Checkthe buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/223/builds/4583) and take a look at the build logs.
  4. Check if the failure is related to this commit (a46e960) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/223/builds/4583

Failed tests:

  • test.test_concurrent_futures.test_shutdown

Failed subtests:

  • test_interpreter_shutdown - test.test_concurrent_futures.test_shutdown.ProcessPoolForkserverProcessPoolShutdownTest.test_interpreter_shutdown

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):  File"/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z/build/Lib/test/test_concurrent_futures/test_shutdown.py", line50, intest_interpreter_shutdownself.assertEqual(out.strip(),b"apple")AssertionError:b'' != b'apple'

Glyphack pushed a commit to Glyphack/cpython that referenced this pull requestSep 2, 2024
Replace os.cpu_count() with os.process_cpu_count() in modules:* compileall* concurrent.futures* multiprocessingReplace os.cpu_count() with os.process_cpu_count() in programs:* _decimal deccheck.py test* freeze.py* multissltests.py* python -m test (regrtest)* wasm_build.pyOther changes:* test.pythoninfo logs os.process_cpu_count().* regrtest gets os.process_cpu_count() / os.cpu_count() in headers.
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@corona10corona10corona10 approved these changes

@brettcannonbrettcannonAwaiting requested review from brettcannon

@rhettingerrhettingerAwaiting requested review from rhettinger

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

3 participants
@vstinner@bedevere-bot@corona10
[8]ページ先頭
©2009-2025 Movatter.jp