Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork33.4k
bpo-5885: fix slow uuid initialization#3684
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
the-knights-who-say-ni commentedSep 21, 2017
Hello, and thanks for your contribution! I'm a bot set up to make sure that the project can legally accept your contribution by verifying you have signed thePSF contributor agreement (CLA). Unfortunately we couldn't find an account corresponding to your GitHub username onbugs.python.org (b.p.o) to verify you have signed the CLA (this might be simply due to a missing "GitHub Name" entry in your b.p.o account settings). This is necessary for legal reasons before we can look at your contribution. Please followthe steps outlined in the CPython devguide to rectify this issue. Thanks again to your contribution and we look forward to looking at it! |
gronke commentedSep 21, 2017 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
|
gronke commentedSep 21, 2017
@the-knights-who-say-ni the CLA is signed by@igalic and me. |
igalic commentedSep 21, 2017
added a NEWS entry now too |
Lib/uuid.py Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
This is redundant:(true expression) is True
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I wanted to be explicit that this is a boolean value. Think that's much easier to read.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
especially considering that nothing in this module is explicitly typed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
But it’s not the python way._dont_do_something = _detect_thing() is legible.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I'm happy to adopt suggestions for better naming. After thinking back and forward (pairing with@igalic) we ended with this solution. Personally I'd rather make an exception to the naming pattern for boolean values than complicating the logic here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
i didn't notice that the function is actually called_is_this_thing()… fixed according to@merwok's comment.
igalic commentedSep 22, 2017 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
can someone please explain why the news check is failing? |
gronke commentedSep 22, 2017 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
I wanted to compare the performance impact duplicating the ctypes imports: importtimeimportctypesimportctypes.utildefmeasure(cb,iterations):start=time.time()foriinrange(iterations):cb()end=time.time()return (end-start)defdo():importctypesimportctypes.utildefdont():passdefrun(iterations):print("Iterations:",iterations)print("A",measure(do,iterations))print("B",measure(dont,iterations))run(1000000)run(100000000) # python3.6 measure-import-time.pyIterations: 1000000A 0.5856528282165527B 0.08484673500061035Iterations: 100000000A 58.5674729347229B 8.484308242797852Is this something we should optimize for? |
merwok left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Not sure why news check is failing. Maybe next push will cause a new check.
Lib/uuid.py Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Minor: parens not needed
Lib/uuid.py Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
If I read correctly, this line is not needed. Declaring global is only needed to assign to a name, not to read from it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
This one is actually required. I've tried to remove it, but then the function call fails.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Ah! The function sets the global at line 508, that’s why.
Lib/uuid.py Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
That’s a big except that could hide developer errors, not just catch the exact AttributeError (or other) that you’re interested in here.
bedevere-bot commentedSep 22, 2017
A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated. Once you have made the requested changes, please leave a comment on this pull request containing the phrase And if you don't make the requested changes,you will be put in the comfy chair! |
gronke commentedSep 22, 2017
@merwok your requested changes were addressed / commented. Thanks for the review! |
gronke commentedSep 23, 2017
I didn't expect the Spanish Inquisition! |
bedevere-bot commentedSep 23, 2017
Nobody expects the Spanish Inquisition! @merwok: please review the changes made to this pull request. |
merwok commentedSep 23, 2017
Alright I started a build on custom buildbots:http://buildbot.python.org/all/waterfall?category=custom.stable&category=custom.unstable I don’t feel qualified to merge this; maybe@methane would like to take it? |
serhiy-storchaka commentedSep 23, 2017
Tests are failed on Windows:http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%20custom/builds/20/steps/test/logs/stdio Why |
merwok commentedSep 23, 2017
That’s why I pinged Inada-san: there was a recent python-dev discussion about improving startup times, with the pros and cons of function-level imports. |
gronke commentedSep 23, 2017 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
The import was in a try/except block and I thought it was more elegant to let theexisting exception handling do the job. I didtest the performance impact of importing ctypes multiple times on a system that successfully imported it once before - results may vary when no ctypes package is available. After thinking about it again, I'd now rather load it once when importing uuid and directly set
Working on it. Downloading Visual Studio |
gronke commentedSep 24, 2017
@serhiy-storchaka I've fixed the Windows problem in the commit that caused it and moved the imports to the module level in a separate one. I had to move the code a bit so that the import works for the whole module. |
optimizes uuid ctypes initialization for BSD and Darwin
_uuid_generate_time is lazy-loaded on first use to speed upinitialization of applications depending on other uuid features
gronke commentedSep 27, 2017
@serhiy-storchaka are there any changes required or can the |
methane commentedSep 28, 2017
gronke commentedSep 28, 2017
@methane so you're suggesting to break the code path for So this PR aims to fix a bug. Since the uuid code is used across various versions of Python, the choice was to fix it with small changes. |
methane commentedSep 28, 2017
Would you discuss onhttps://bugs.python.org/issue11063 ? |
vstinner commentedSep 28, 2017
igalic commentedSep 28, 2017
👍 |
Uh oh!
There was an error while loading.Please reload this page.
Initializing
_uuid_generate_timein Lib/uuid.py was found to be slow on FreeBSD systems with many CPU cores. This changes optimizes code paths for Darwin and FreeBSD and delays the initialization of the_uuid_generate_timectypes implementation to time of it's first use (for instance when callinguuid.uuid1()).https://bugs.python.org/issue5885