Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

gh-103977: compile re expressions in platform.py only if required#103981

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
JelleZijlstra merged 12 commits intopython:mainfromeendebakpt:platform_re_compile
Apr 30, 2023

Conversation

@eendebakpt
Copy link
Contributor

@eendebakpteendebakpt commentedApr 28, 2023•
edited
Loading

Inplatform.py there are several regular expressions compiled during import that might not be used at all. The regular expressions are only called once at most, since they are called from within cached methods. We can eliminate the cost of compilation by moving the regular expressions into the methods.

The import time ofplatform.py (measured withhttps://github.com/sympy/sympy/blob/master/bin/test_import, adapted for platform.py):

Main: The speed of "import platform" is: 7.567131 +- 0.208541 [ms]PR: The speed of "import platform" is: 6.513338 +- 0.404813 [ms]

Notes:

  • All regular expressions modified are within cached methods. The only exceptions is are_libc_search andcomponent_re which are used in the methodlibc_ver (not cached). Thelibc_ver is called fromplatform which is cached.
    If we expectlibc_ver to be called many times, it should be cached as well, or the_libc_search regular expression moved back as a module variable.
  • A couple of ms of the import time is spend in importing other modules (such ascollections)

@hauntsaninja This is a followup of#102492

@JelleZijlstra
Copy link
Member

Can you runmake patchcheck to fix the whitespace issue?

@TeamSpen210
Copy link

Since they’re now all local variables, it might be nice to remove the underscore prefixes.

Eclips4 reacted with thumbs up emoji

…sF70A.rstCo-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure⚠️⚠️⚠️

Hi! The buildbotAMD64 Debian root 3.x has failed when building commitf186557.

What do you need to do:

  1. Don't panic.
  2. Checkthe buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/345/builds/4626) and take a look at the build logs.
  4. Check if the failure is related to this commit (f186557) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/345/builds/4626

Failed tests:

  • test_threading

Summary of the results of the build (if available):

==

Click to see traceback logs
remote:Enumerating objects: 9, done.remote:Counting objects:  11% (1/9)remote:Counting objects:  22% (2/9)remote:Counting objects:  33% (3/9)remote:Counting objects:  44% (4/9)remote:Counting objects:  55% (5/9)remote:Counting objects:  66% (6/9)remote:Counting objects:  77% (7/9)remote:Counting objects:  88% (8/9)remote:Counting objects: 100% (9/9)remote:Counting objects: 100% (9/9), done.remote:Compressing objects:  14% (1/7)remote:Compressing objects:  28% (2/7)remote:Compressing objects:  42% (3/7)remote:Compressing objects:  57% (4/7)remote:Compressing objects:  71% (5/7)remote:Compressing objects:  85% (6/7)remote:Compressing objects: 100% (7/7)remote:Compressing objects: 100% (7/7), done.remote:Total 9 (delta 1), reused 2 (delta 1), pack-reused 0From https://github.com/python/cpython * branch                  main       -> FETCH_HEADNote:switching to 'f186557dc3e77495ba65b9b7e492ab6ddb7cecc3'.You are in 'detached HEAD' state. You can look around, make experimentalchanges and commit them, and you can discard any commits you make in thisstate without impacting any branches by switching back to a branch.If you want to create a new branch to retain commits you create, you maydo so (now or later) by using -c with the switch command. Example:  git switch -c <new-branch-name>Or undo this operation with:  git switch -Turn off this advice by setting config variable advice.detachedHead to falseHEAD is now at f186557dc3 gh-103977: compile re expressions in platform.py only if required (#103981)Switched to and reset branch 'main'configure:WARNING: pkg-config is missing. Some dependencies may not be detected correctly.Fatal Python error: Segmentation faultThread 0x00007fb2ea5bb280 (most recent call first):  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/runner.py", line 74 in _write_status  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/runner.py", line 95 in addSuccess  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/support/testresult.py", line 122 in addSuccess  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/case.py", line 648 in run  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/case.py", line 690 in __call__  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/suite.py", line 122 in run  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/suite.py", line 84 in __call__  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/suite.py", line 122 in run  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/suite.py", line 84 in __call__  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/suite.py", line 122 in run  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/suite.py", line 84 in __call__  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/unittest/runner.py", line 240 in run  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/support/__init__.py", line 1106 in _run_suite  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/support/__init__.py", line 1232 in run_unittest  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/libregrtest/runtest.py", line 294 in _test_module  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/libregrtest/runtest.py", line 330 in _runtest_inner2  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/libregrtest/runtest.py", line 373 in _runtest_inner  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/libregrtest/runtest.py", line 248 in _runtest  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/libregrtest/runtest.py", line 278 in runtest  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/libregrtest/main.py", line 365 in rerun_failed_tests  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/libregrtest/main.py", line 783 in _main  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/libregrtest/main.py", line 738 in main  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/libregrtest/main.py", line 802 in main  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/__main__.py", line 2 in <module>  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/runpy.py", line 88 in _run_code  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/runpy.py", line 198 in _run_module_as_mainExtension modules: _testcapi (total: 1)Segmentation faultmake:*** [Makefile:1954: buildbottest] Error 139Cannot open file '/root/buildarea/3.x.angelico-debian-amd64/build/test-results.xml' for upload

@JelleZijlstra
Copy link
Member

Seems unlikely that this would cause a segfault in test_threading.

@hugovk
Copy link
Member

And backport to 3.11?

@JelleZijlstra
Copy link
Member

I don't think so; we don't backport perf improvements.

hugovk reacted with thumbs up emoji

carljm added a commit to carljm/cpython that referenced this pull requestMay 1, 2023
* main: (26 commits)pythongh-104028: Reduce object creation while calling callback function from gc (pythongh-104030)pythongh-104036: Fix direct invocation of test_typing (python#104037)pythongh-102213: Optimize the performance of `__getattr__` (pythonGH-103761)pythongh-103895: Improve how invalid `Exception.__notes__` are displayed (python#103897)  Adjust expression from `==` to `!=` in alignment with the meaning of the paragraph. (pythonGH-104021)pythongh-88496: Fix IDLE test hang on macOS (python#104025)  Improve int test coverage (python#104024)pythongh-88773: Added teleport method to Turtle library (python#103974)pythongh-104015: Fix direct invocation of `test_dataclasses` (python#104017)pythongh-104012: Ensure test_calendar.CalendarTestCase.test_deprecation_warning consistently passes (python#104014)pythongh-103977: compile re expressions in platform.py only if required (python#103981)pythongh-98003: Inline call frames for CALL_FUNCTION_EX (pythonGH-98004)  Replace Netlify with Read the Docs build previews (python#103843)  Update name in acknowledgements and add mailmap (python#103696)pythongh-82054: allow test runner to split test_asyncio to execute in parallel by sharding. (python#103927)  Remove non-existing tools from Sundry skiplist (python#103991)pythongh-103793: Defer formatting task name (python#103767)pythongh-87092: change assembler to use instruction sequence instead of CFG (python#103933)pythongh-103636: issue warning for deprecated calendar constants (python#103833)  Various small fixes to dis docs (python#103923)  ...
@eendebakpteendebakpt deleted the platform_re_compile branchJune 26, 2025 13:45
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@JelleZijlstraJelleZijlstraJelleZijlstra approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@eendebakpt@JelleZijlstra@TeamSpen210@bedevere-bot@hugovk
[8]ページ先頭
©2009-2025 Movatter.jp