Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

gh-146385: Switch back tore to detect shlex.quote slow path#146408

Open
bonzini wants to merge 1 commit intopython:mainfrom
bonzini:fix146385
Open

gh-146385: Switch back tore to detect shlex.quote slow path#146408
bonzini wants to merge 1 commit intopython:mainfrom
bonzini:fix146385

Conversation

@bonzini
Copy link

@bonzinibonzini commentedMar 25, 2026
edited by bedevere-appbot
Loading

Commit06a26fd ("gh-118761: Optimise import time forshlex (#132036)") when the input has to be quoted. This is because the regular expression search was able to short-circuit at the first unsafe character.

Go back to the same algorithm as 3.13, but make the "import re" and compilation of the regular expression lazy.

Testings.isascii() makesshlex.quote() twice as fast in the non-ASCII case, but costs up to 25% of the full run time (because it necessitates an earlierisinstance check) if the stringis ASCII. The latter is probably the common case, so drop the check.

@bedevere-app
Copy link

Most changes to Pythonrequire a NEWS entry. Add one using theblurb_it web app or theblurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply theskip news label instead.

bonzini reacted with thumbs up emoji

@python-cla-bot
Copy link

python-cla-botbot commentedMar 25, 2026
edited
Loading

All commit authors signed the Contributor License Agreement.

CLA signed

@picnixz
Copy link
Member

@AA-Turner Can you check if the results are consistent on Windows as well please?

@picnixz
Copy link
Member

Here is what I suggest:

  • For 3.14, we can fix the regression (I will consider it a bug) if this reproduces on multiple systems. We might have other things at play here depending on where Python is built.
  • For 3.15, we can uselazy imports to make it cleaner if necessary.

I will also try to check what happens on older systems (openSUSE + older GCC/clang).

Commit06a26fd ("pythongh-118761: Optimise import time for ``shlex`` (python#132036)")when the input has to be quoted. This is because the regular expressionsearch was able to short-circuit at the first unsafe character.Go back to the same algorithm as 3.13, but make the "import re" and compilationof the regular expression lazy.Testing s.isascii() makes shlex.quote() twice as fast in the non-ASCIIcase, but costs up to 25% of the full run time (because it necessitatesan earlier isinstance check) if the string *is* ASCII.  The latter isprobably the common case, so drop the check.
@bedevere-app
Copy link

Most changes to Pythonrequire a NEWS entry. Add one using theblurb_it web app or theblurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply theskip news label instead.

@bonzini
Copy link
Author

Yeah I didn't uselazy import to cater for backports. Would you prefer thelazy import version to be in this fix PR, or in a follow-up?

@picnixz
Copy link
Member

For now, let's check whether the regression is consistent across platforms.

@bonzini
Copy link
Author

For now, let's check whether the regression is consistent across platforms.

It is, and also it's really a complexity change.shlex.quote('a ' + 'b' * N) will always be slower in 3.14+ if you make N high enough. (And in practice, 10 is enough to have a substantial effect).

@picnixz
Copy link
Member

What I want to know is why Adam reported numbers that went in the other direction. Ideally I also want to know the stdev of the numbers usingpyperf (which I will do this w-e)

@picnixz
Copy link
Member

picnixz commentedMar 25, 2026
edited
Loading

I suggested in#132036 (comment) to replace there callif we had something more efficient, and Adam found out that it was faster usingbytes.translate so I'd like to know which data it was tested on:

With this, we go from ~26 microseconds for quote to ~2 (with only safe characters) and ~6 (with characters needing quotation).

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@AA-TurnerAA-TurnerAwaiting requested review from AA-Turner

@picnixzpicnixzAwaiting requested review from picnixz

Assignees

No one assigned

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@bonzini@picnixz

[8]ページ先頭

©2009-2026 Movatter.jp