Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

gh-130167: Improve speed ofinspect.formatannotation by replacingre#130242

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Open
donBarbos wants to merge8 commits intopython:main
base:main
Choose a base branch
Loading
fromdonBarbos:improve-inspect-speed

Conversation

donBarbos
Copy link
Contributor

@donBarbosdonBarbos commentedFeb 18, 2025
edited
Loading

timeit benchmark with my script:

inspect_bench.py:

importreimporttimeitfromtypingimportList,Dict,Any,Uniondefold_format_annotation(annotation):ifgetattr(annotation,"__module__",None)=="typing":defrepl(match):text=match.group()returntext.removeprefix("typing.")returnre.sub(r"[\w\.]+",repl,repr(annotation))returnrepr(annotation)defnew_format_annotation(annotation):ifgetattr(annotation,"__module__",None)=="typing":return (repr(annotation)                .replace(".typing.","@TYPING@")                .replace("typing.","")                .replace("@TYPING@",".typing."))returnrepr(annotation)old_time=timeit.timeit(lambda:old_format_annotation(Union[List[str],Dict[str,Any]]),number=100_000)new_time=timeit.timeit(lambda:new_format_annotation(Union[List[str],Dict[str,Any]]),number=100_000)print(f"Old version (re.sub):{old_time:.6f}s")print(f"New version (.replace()):{new_time:.6f}s")print(f"Difference:{old_time/new_time:.2f}x")

Result: 2.0s -> 1.18s = x1.74 as fast

$ ./python -B inspect_bench.pyOld version (re.sub): 2.043900sNew version (.replace()): 1.175867sDifference: 1.74x

✔️ This was also the only use ofre in module so i got rid of the import

@donBarbosdonBarbos changed the titleImprove speed ofinspect.formatannotation by replacingregh-130167: Improve speed ofinspect.formatannotation by replacingreFeb 18, 2025
donBarbosand others added2 commitsMarch 28, 2025 01:58
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Copy link
Member

@AA-TurnerAA-Turner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I'm less sure this is worth it, the replacement isn't clearly better maintenence-wise.

What do the benchmarks look like if you extractrepl to a module-level_formatannotation_repl and use.sub() on a pre-compiled pattern?

A

@donBarbos
Copy link
ContributorAuthor

donBarbos commentedMay 1, 2025
edited
Loading

sorry, I forgot how python was compiled then :( but I'm pretty sure I compiled it with gcc last time

so I reran the test with the following config args'--enable-optimizations' '--with-lto' 'CC=/usr/bin/clang-18' and got a new result for my bench:

Old version (re.sub): 0.982520sNew version (.replace()): 0.701049sDifference: 1.40x# sometimes it's 1.35x

and if we rewrite old version of benchmark as you suggested:

_pattern=re.compile(r"[\w\.]+")def_repl(match):text=match.group()returntext.removeprefix("typing.")defold_format_annotation(annotation):ifgetattr(annotation,"__module__",None)=="typing":returnre.sub(_pattern,_repl,repr(annotation))returnrepr(annotation)

we will get a bigger difference:

Old version (re.sub): 1.027774sNew version (.replace()): 0.698902sDifference: 1.47x# the result can drop to 1.40

P.S.: another reason i like the PR version is because we get rid of there import that is used once here

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@hugovkhugovkhugovk left review comments

@AA-TurnerAA-TurnerAA-Turner left review comments

Assignees
No one assigned
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

3 participants
@donBarbos@hugovk@AA-Turner

[8]ページ先頭

©2009-2025 Movatter.jp