Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

gh-109653: Speedup import of threading module#114509

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged

Conversation

danielhollas
Copy link
Contributor

@danielhollasdanielhollas commentedJan 23, 2024
edited
Loading

Delayed import offunctools speeds up theimport threading by ~50% (2ms -> 1ms) in my testing.

Since thefunctools module is only used in the internal_register_atexit function that is called byconcurrent.futures, this seems like a worthwhile win for users ofthreading module who do not useasyncio.

Part of#109653

CC@AlexWaygood

Delayed import of functools leads to 50% speedupof import time.
@bedevere-app
Copy link

Most changes to Pythonrequire a NEWS entry. Add one using theblurb_it web app or theblurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply theskip news label instead.

@danielhollas
Copy link
ContributorAuthor

danielhollas commentedJan 24, 2024
edited by hugovk
Loading

To be precise, compiling python with./configure --enable-optimizations and measuring withpython -Ximporttime -c "import threading", I am getting 2.44ms on main and 1.17ms on this PR.

@Eclips4Eclips4 added the performancePerformance or resource usage labelJan 24, 2024
@ajoino
Copy link

Just a thought, is it even necessary to usefunctools.partial here? Could this not be replaced withlambda: f(*args, **kwargs), that would avoid importing functools at all. Is there something I'm missing here?

AlexWaygood reacted with thumbs up emoji

@AlexWaygood
Copy link
Member

Just a thought, is it even necessary to usefunctools.partial here? Could this not be replaced withlambda: f(*args, **kwargs), that would avoid importing functools at all. Is there something I'm missing here?

This was my thought as well on first seeing the patch.functools.partial can be faster than alambda function, but here I doubt it makes a significant difference. (If we wanted to check whether using alambda here slowed things down, we'd need to do a benchmark usingconcurrent.futures, since theconcurrent.futures module is the only public API that makes use of this private API. It might be possible to write such a benchmark, but it also might be difficult -- not sure.)

@danielhollas
Copy link
ContributorAuthor

functools.partial can be faster than a lambda function, but here I doubt it makes a significant difference. (If we wanted to check whether using a lambda here slowed things down, we'd need to do a benchmark using concurrent.futures, since the concurrent.futures module is the only public API that makes use of this private API. It might be possible to write such a benchmark, but it also might be difficult -- not sure.)

Looking at the code, thethreading.register_atexit() is only ever called duringconcurrent.futures import, so I would assume any performance difference here would be marginal?

@AlexWaygood
Copy link
Member

Looking at the code, thethreading.register_atexit() is only ever called duringconcurrent.futures import, so I would assume any performance difference here would be marginal?

Oh, great point 😄

In that case, let's just go with alambda here -- it seems simpler :)

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Copy link
Member

@AlexWaygoodAlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

LGTM, thanks! I'd love to check with a core dev more familiar with subinterpreters before merging, though (since this feature wasspecifically added to help with subinterpreter support).

@ericsnowcurrently, there's no reason why switching to alambda rather thanfunctools.partial could be problematic for subinterpreter support, is there?

danielhollas reacted with heart emoji
@danielhollas
Copy link
ContributorAuthor

@AlexWaygood thanks!

@ericsnowcurrently, there's no reason why switching to a lambda rather than functools.partial could be problematic for subinterpreter support, is there?

Just a note, if this was a problem, we could still get away with it by simply not doing either: the function is (at least currently) being called without any extra*args or**args arguments so we could make_register_atexit less general and simply pass the callback function directly to_threading_atexits list.

AlexWaygood reacted with thumbs up emoji

@danielhollasdanielhollas changed the titlegh-109653: Speedup import ofthreading modulegh-109653: Speedup import of threading moduleJan 24, 2024
@AlexWaygood
Copy link
Member

I can't see a way in which this would cause problems — I'll go ahead and merge, since it's been a few days :)

Thanks@danielhollas!

danielhollas reacted with heart emojidanielhollas reacted with rocket emoji

@AlexWaygoodAlexWaygood merged commit5e390a0 intopython:mainJan 31, 2024
@danielhollasdanielhollas deleted the import-threading-speedup branchJanuary 31, 2024 10:59
@ericsnowcurrently
Copy link
Member

@ericsnowcurrently, there's no reason why switching to a lambda rather than functools.partial could be problematic for subinterpreter support, is there?

I'm not aware of any such reason.

AlexWaygood reacted with heart emoji

aisk pushed a commit to aisk/cpython that referenced this pull requestFeb 11, 2024
Avoiding an import of functools leads to 50% speedup of import time.Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@ericsnowcurrentlyericsnowcurrentlyericsnowcurrently left review comments

@AlexWaygoodAlexWaygoodAlexWaygood approved these changes

Assignees
No one assigned
Labels
performancePerformance or resource usage
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

5 participants
@danielhollas@ajoino@AlexWaygood@ericsnowcurrently@Eclips4

[8]ページ先頭

©2009-2025 Movatter.jp