Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

gh-109653: Fix py312 regression in the import time ofrandom#110221

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
AlexWaygood merged 2 commits intopython:mainfromAlexWaygood:random-import-time
Oct 2, 2023

Conversation

AlexWaygood
Copy link
Member

@AlexWaygoodAlexWaygood commentedOct 2, 2023•
edited by bedevere-appbot
Loading

As an optimisation to reduce the import time of the module,random first tries to importsha512 from the internal_sha512 module before falling back tohashlib. The problem, however, is that Python no longer has a_sha512 module! It was removed in0b13575, by@gpshead. That means we're currently always falling back to the slow path inrandom.py, leading to the import time ofrandom being far slower than it should be.

Importingsha512 from the correct module in the fast path cuts 60% off the import time ofrandom.

gpshead reacted with thumbs up emoji
@AlexWaygoodAlexWaygood added the performancePerformance or resource usage labelOct 2, 2023
@AlexWaygoodAlexWaygood changed the titleReduce the import time ofrandom by 60%gh-109653: Reduce the import time ofrandom by 60%Oct 2, 2023
@JelleZijlstra
Copy link
Member

Given that this is a regression, should we backport it into 3.12?

@AlexWaygood
Copy link
MemberAuthor

AlexWaygood commentedOct 2, 2023•
edited
Loading

Given that this is a regression, should we backport it into 3.12?

I was wondering that. I'd vote in favour of doing so, since it doesn't seem particularly high-risk to me. But I'd like to hear Raymond's and/or Greg's thoughts.

JelleZijlstra and gpshead reacted with thumbs up emoji

@AlexWaygoodAlexWaygood added the stdlibPython modules in the Lib dir labelOct 2, 2023
@rhettinger
Copy link
Contributor

A backport to 3.12 would be reasonable.

AlexWaygood reacted with thumbs up emoji

@AlexWaygoodAlexWaygood added the needs backport to 3.12only security fixes labelOct 2, 2023
@AlexWaygoodAlexWaygoodenabled auto-merge (squash)October 2, 2023 22:30
@AlexWaygood
Copy link
MemberAuthor

A backport to 3.12 would be reasonable.

Great, I've scheduled the backport. Thanks for the review!

@AlexWaygoodAlexWaygood changed the titlegh-109653: Reduce the import time ofrandom by 60%gh-109653: Fix py312 regression in the import time ofrandomOct 2, 2023
@AlexWaygoodAlexWaygood merged commit21a6263 intopython:mainOct 2, 2023
@AlexWaygoodAlexWaygood deleted the random-import-time branchOctober 2, 2023 22:56
@miss-islington
Copy link
Contributor

Thanks@AlexWaygood for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull requestOct 2, 2023
…110221)(cherry picked from commit21a6263)Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
@bedevere-app
Copy link

GH-110247 is a backport of this pull request to the3.12 branch.

gpshead reacted with thumbs up emoji

@bedevere-appbedevere-appbot removed the needs backport to 3.12only security fixes labelOct 2, 2023
@bedevere-app

This comment was marked as duplicate.

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure⚠️⚠️⚠️

Hi! The buildbotaarch64 Debian Clang LTO + PGO 3.x has failed when building commit21a6263.

What do you need to do:

  1. Don't panic.
  2. Checkthe buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/1084/builds/2183) and take a look at the build logs.
  4. Check if the failure is related to this commit (21a6263) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/1084/builds/2183

Failed tests:

  • test.test_concurrent_futures.test_shutdown

Failed subtests:

  • test_interpreter_shutdown - test.test_concurrent_futures.test_shutdown.ProcessPoolSpawnProcessPoolShutdownTest.test_interpreter_shutdown

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):  File"/var/lib/buildbot/workers/arm64-clang/3.x.gps-arm64-debian.clang.lto-pgo/build/Lib/test/test_concurrent_futures/test_shutdown.py", line50, intest_interpreter_shutdownself.assertEqual(out.strip(),b"apple")AssertionError:b'' != b'apple'

AlexWaygood added a commit that referenced this pull requestOct 2, 2023
… (#110247)gh-109653: Fix regression in the import time of `random` in Python 3.12 (GH-110221)(cherry picked from commit21a6263)Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
@@ -65,7 +65,7 @@

try:
# hashlib is pretty heavy to load, try lean internal module first
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This code is technically awkward... It tried to speed up import time and did so by circumventing hashlib which means that it is loading and using the slowest possible sha512 implementation by default (hashlib will pick up openssl 3's accelerated sha512 support by default on most platforms and not use our builtin). so faster startup time for a slower runtime computation? thankfully this is only ever used by seed() which is a single/constant number of calls for most programs on tiny data so there is zero reason to care about sha512 performance for its purposes. The slower implementation may still be faster on small seed data anyways due to less setup overhead.

Nothing to do here. This works for random's purposes & thanks for the fixup. But this being a regression in the first place demonstrates how fragile direct use of internal details can be.(and indirectly how much in need of an overhaul hashlib.py could use)

I'd file an issue rather than leaving this comment in the merged PR void if there were anything concrete to describe and tackle, there isn't. :)

AlexWaygood reacted with heart emoji
Glyphack pushed a commit to Glyphack/cpython that referenced this pull requestSep 2, 2024
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@gpsheadgpsheadgpshead left review comments

@rhettingerrhettingerrhettinger approved these changes

@AA-TurnerAA-TurnerAA-Turner approved these changes

Assignees
No one assigned
Labels
performancePerformance or resource usagestdlibPython modules in the Lib dir
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

7 participants
@AlexWaygood@JelleZijlstra@rhettinger@miss-islington@bedevere-bot@gpshead@AA-Turner
[8]ページ先頭
©2009-2025 Movatter.jp