Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

GH-130328: pasting in new REPL is slow on Windows#132884

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
ambv merged 7 commits intopython:mainfromchris-eibl:fix_wait
Apr 29, 2025

Conversation

chris-eibl
Copy link
Member

@chris-eiblchris-eibl commentedApr 24, 2025•
edited by bedevere-appbot
Loading

The reason why pasting is so slow on Windows (especially in the legacy console case where virtual terminal mode - and thus bracketed paste - is disabled):

whileTrue:
# We use the same timeout as in readline.c: 100ms
self.run_hooks()
self.console.wait(100)
event=self.console.get_event(block=False)

and

defwait(self,timeout:float|None)->bool:
"""Wait for an event."""
# Poor man's Windows select loop
start_time=time.time()
whileTrue:
ifmsvcrt.kbhit():# type: ignore[attr-defined]
returnTrue
iftimeoutandtime.time()-start_time>timeout/1000:
returnFalse
time.sleep(0.01)

It is interesting, thatmsvcrt.kbhit() returns True in case of pasting, but if that weren't the case, we'd hit the 100ms timeout and would be even way slower. However,msvcrt.kbhit() is very slow in case of the legacy console, and it is called at least twice as often, because we get a key up and a key down event - but only a key down event in the virtual terminal case. If the pasted input contains upper case letters, we additionally get a "shift key pressed" event - so in the worst case,msvcrt.kbhit() gets called 3 times more often (and each call is slower).

Output ofpython.bat -m cProfile -m _pyrepl when pasting the "test value" given in the OP for a legacy console:

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)     50/1    0.010    0.000   17.202   17.202 {built-in method builtins.exec}        1    0.000    0.000   17.202   17.202 <frozen runpy>:201(run_module)        1    0.000    0.000   17.199   17.199 <frozen runpy>:65(_run_code)        1    0.000    0.000   17.199   17.199 __main__.py:1(<module>)        1    0.000    0.000   17.112   17.112 main.py:24(interactive_console)        1    0.000    0.000   17.112   17.112 simple_interact.py:99(run_multiline_interactive_console)        5    0.000    0.000   17.111    3.422 readline.py:375(multiline_input)        5    0.004    0.001   17.111    3.422 reader.py:741(readline)     3090    0.036    0.000   17.103    0.006 reader.py:694(handle1)     6210    0.013    0.000   13.714    0.002 windows_console.py:523(wait)     6236   13.433    0.002   13.433    0.002 {built-in method msvcrt.kbhit}

Here the relevant line in case of virtual terminal:

  3270    0.554    0.000    0.554    0.000 {built-in method msvcrt.kbhit}

The fix is to useWaitForSingleObject

defwait(self,timeout:float|None)->bool:"""Wait for an event."""ret=WaitForSingleObject(InHandle,int(timeout))ifret==WAIT_FAILED:raiseWinError(ctypes.get_last_error())

which speeds up especially the legacy console (times in seconds):

legacy terminalvirtual terminal
before15.60.89
after1.80.26

Note, see also#132440 (comment):

  • legacy terminal: manually startcmd.exe. Timings gotten by pasting
import timet1 = time.time()<"test value" given in the OP>print(time.time() - t1)
  • virtual terminal: power shell via Windows terminal. Timings gotton by temporarily patchingcommands.py:
importtimeclassenable_bracketed_paste(Command):defdo(self)->None:self.reader.bp_begin=time.perf_counter()self.reader.paste_mode=Trueself.reader.in_bracketed_paste=Trueclassdisable_bracketed_paste(Command):defdo(self)->None:print("bracketed_paste took %5.2f s"% (time.perf_counter()-self.reader.bp_begin, ))self.reader.paste_mode=Falseself.reader.in_bracketed_paste=Falseself.reader.dirty=True

@sergey-miryanov
Copy link
Contributor

Good analysis and speedup! 👍

chris-eibl reacted with heart emoji

if not events.value:
return None
if not block and not self.wait(timeout=0):
return None
Copy link
MemberAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Now we are nicely in sync with unix_console.py

ifnotblockandnotself.wait(timeout=0):
returnNone

@Fidget-Spinner
Copy link
Member

We generally don't backport perf fixes unless they lead to a denial-of-service problem (so usually like an exponential-time algorithm), or result in a serious bug. So I'm removing the backport label.

@Fidget-SpinnerFidget-Spinner removed the needs backport to 3.13bugs and security fixes labelApr 24, 2025
@Fidget-Spinner
Copy link
Member

On second thought, "slow" is perhaps an understatement. So once this is merged, let's ping Thomas to ask for his permission to backport to 3.13

chris-eibl reacted with thumbs up emoji

@chris-eibl
Copy link
MemberAuthor

chris-eibl commentedApr 24, 2025•
edited
Loading

Yeah, in case of the legacy console pasting of the 30 + 3 lines given by the OP takes 15 seconds, and this brings it down to 1.8. Most people will hopefully use a virtual terminal, where the situation is much less severe. After all, the new REPL can be disabled usinghttps://docs.python.org/3/using/cmdline.html#envvar-PYTHON_BASIC_REPL.

3 more lines added by me for easier benchmarking#130328 (comment)

import timet1 = time.time()
"test value" given in the OP

"""1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890"""

print(time.time() - t1)

@pablogsal
Copy link
Member

If@ambv has not checked this this in a couple of days ping me and I will do a pass and land it. Thanks a lot for looking into this@chris-eibl ❤️

chris-eibl reacted with heart emoji

@ambvambv merged commitacb222c intopython:mainApr 29, 2025
47 checks passed
@ambv
Copy link
Contributor

Thanks for the fix, Chris!

@Fidget-Spinner
Copy link
Member

@Yhg1s seeking permission to backport this to 3.13.

According to Chris, it takes 15 seconds to paste 58 lines in the new repl in 3.13. Do you think that's slow enough to be worth backporting?

@chris-eiblchris-eibl deleted the fix_wait branchMay 2, 2025 14:01
@chris-eibl
Copy link
MemberAuthor

@Yhg1s friendly ping and also asking@ambv whether this is worth backporting to 3.13?

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@tomasr8tomasr8tomasr8 left review comments

@pablogsalpablogsalAwaiting requested review from pablogsalpablogsal is a code owner

@lysnikolaoulysnikolaouAwaiting requested review from lysnikolaoulysnikolaou is a code owner

@ambvambvAwaiting requested review from ambvambv is a code owner

Assignees
No one assigned
Labels
OS-windowstopic-replRelated to the interactive shell
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

6 participants
@chris-eibl@sergey-miryanov@Fidget-Spinner@pablogsal@ambv@tomasr8
[8]ページ先頭
©2009-2025 Movatter.jp