Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

gh-143658: importlib.metadata: Usestr.translate to improve performance ofimportlib.metadata.Prepared.normalized#143660

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged

Conversation

@hugovk
Copy link
Member

@hugovkhugovk commentedJan 10, 2026•
edited by bedevere-appbot
Loading

We can apply@henryiii's improvement topackaging inpypa/packaging#1030 (see alsohttps://iscinumpy.dev/post/packaging-faster/) to improve the performance ofcanonicalize_name and make it ~3.7 times faster.

Benchmark

RunPrepared.normalize(n) on every name in PyPI:

# benchmark_names_stdlib.pyimportsqlite3importtimeitfromimportlib.metadataimportPrepared# Get data with:# curl -L https://github.com/pypi-data/pypi-json-data/releases/download/latest/pypi-data.sqlite.gz | gzip -d > pypi-data.sqlite# Or ues pre-cached files from:# https://gist.github.com/hugovk/efdbee0620cc64df7b405b52cf0b6e42CACHE_FILE="/tmp/bench/names.txt"DB_FILE="/tmp/bench/pypi-data.sqlite"try:withopen(CACHE_FILE)asf:TEST_ALL_NAMES= [line.rstrip("\n")forlineinf]exceptFileNotFoundError:TEST_ALL_NAMES= []withsqlite3.connect(DB_FILE)asconn:withopen(CACHE_FILE,"w")ascache:for (name,)inconn.execute("SELECT name FROM projects"):ifname:TEST_ALL_NAMES.append(name)cache.write(name+"\n")defbench():forninTEST_ALL_NAMES:Prepared.normalize(n)if__name__=="__main__":print(f"Loaded{len(TEST_ALL_NAMES):,} names")t=timeit.timeit("bench()",globals=globals(),number=1)print(f"Time:{t:.4f} seconds")

Benchmark data can be found athttps://gist.github.com/hugovk/efdbee0620cc64df7b405b52cf0b6e42

Before

With optimisations:

❯./python.exe benchmark_names_stdlib.pyLoaded 8,344,947 namesTime: 5.1483 seconds

After

❯./python.exe benchmark_names_stdlib.pyLoaded 8,344,947 namesTime: 1.3754 seconds

3.7 times faster.

hugovkand others added2 commitsJanuary 10, 2026 14:29
Co-Authored-By: Henry Schreiner <henryschreineriii@gmail.com>
@hugovkhugovk changed the titleimportlib.metadata: Usetranslate to improve performance ofcanonicalize_namegh-143658: importlib.metadata: Usetranslate to improve performance ofcanonicalize_nameJan 10, 2026
@hugovkhugovk added performancePerformance or resource usage topic-importlib labelsJan 10, 2026
@picnixzpicnixz changed the titlegh-143658: importlib.metadata: Usetranslate to improve performance ofcanonicalize_namegh-143658: importlib.metadata: Usestr.translate to improve performance ofimportlib.metadata.Prepared.normalizedJan 10, 2026
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Copy link
Member

@picnixzpicnixz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Do we have tests actually? if not, maybe it'd be good to add some.

hugovk reacted with thumbs up emoji
hugovkand others added3 commitsJanuary 10, 2026 17:16
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Copy link
Member

@johnslavikjohnslavik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Small ideas

Co-authored-by: Bartosz Sławecki <bartosz@ilikepython.com>
@hugovkhugovk merged commitcbf9b8c intopython:mainJan 13, 2026
50 checks passed
@hugovkhugovk deleted the 3.15-importlib.metadata-canonicalize_name branchJanuary 13, 2026 06:54
@hugovk
Copy link
MemberAuthor

See#144083 for a followup.

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@picnixzpicnixzpicnixz left review comments

@johnslavikjohnslavikjohnslavik left review comments

@hauntsaninjahauntsaninjahauntsaninja approved these changes

@jaracojaracoAwaiting requested review from jaracojaraco is a code owner

@warsawwarsawAwaiting requested review from warsawwarsaw is a code owner

+2 more reviewers

@henryiiihenryiiihenryiii left review comments

@a12ka12ka12k left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

performancePerformance or resource usagetopic-importlib

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@hugovk@henryiii@a12k@picnixz@hauntsaninja@johnslavik
[8]ページ先頭
©2009-2026 Movatter.jp