Abstract

One of Python’s core strengths is its interactive mode, also known as theRead-Eval-Print Loop (REPL), or the Python console, or the Python shell. ThisPEP describes the new implementation of this functionality written in Python.The new REPL released in Python 3.13 aims to provide modern features expected bytoday’s users, such as multi-line editing, syntax highlighting, custom commands,and an overall improved interactive experience.

Motivation

Up to Python 3.12, the interactive shell of CPython was written in C as aspecial mode of the parser. It was therefore difficult to maintain and extend.It relied on the existence of GNU readline (or an equivalent) for basicfunctionality like cursor movement and history tracking. Python compiled withoutthis library provided an interactive mode of very limited capability. On theother hand, Python compiled with readline outsourced decisions and configurationaround user input in ways that made extending it difficult.

This complexity has deterred contributions and has made it challenging toimplement new features. As a result, the CPython interactive shell has seenminimal changes, falling behind user expectations for modern equivalents.

Many features that users have come to expect from modern REPLs were absent inthe previous version. Some examples of these features include multi-line editingand history, custom commands, syntax highlighting, or ergonomic handling of copyand paste. The lack of these features greatly impacts the user experience ofmany groups of users of CPython, in particular in environments where users don’tcontrol dependencies and cannot install their own packages. This is especiallycommon for users learning the language and educators.

Addressing such issues with the C implementation would require complexworkarounds, such as AST matching of commands, which would add prohibitivecomplexity to the codebase.

With the new REPL written in Python, we are addressing these limitations whilealso bringing CPython’s interactive experience more in line with modernexpectations and capabilities.

Rationale

Implementing the new REPL in Python, rather than C, has significantly loweredthe barrier to entry for contributors. This change has made it easier to test,validate, and modify the REPL, leading to increased community involvement andfaster feature development. The improved accessibility of the codebase isexpected to result in a more rapidly evolving and user-responsive REPL.

Instead of writing a Python REPL from scratch, we decided to base theimplementation of the new REPL onPyREPL.This decision was driven by several key factors. First and foremost,developing a terminal application that works consistently across differentoperating systems and terminal emulators is a complex undertaking.By adopting PyREPL, which has been battle-tested in the PyPy project,we can leverage existing, proven code rather than starting from scratch.

Sharing a codebase with PyPy for the REPL implementation offers mutual benefitsto both projects. It allows for shared maintenance efforts, faster bug fixes,and feature improvements that can benefit users of both CPython and PyPy. Thiscollaboration can lead to a more robust and feature-rich REPL for the entirePython ecosystem.

The previous REPL written in C leveraged the “readline” or “editline” librariesas a backend to allow certain features such as navigation, history preservationand recall, autocompletion, and configurable keyboard behavior. PyREPL does notuse those libraries, implementing most of the other functionality directly aspart of the shell. The main missing functionality (configurability of input) isoutweighed by the benefits of the new architecture. The configuration files forthese libraries (e.g. inputrc) are complex and include features that PyREPLdoesn’t plan to implement, making it infeasible to transparently add support forthem in the new shell. Using “readline” or “editline” in PyREPL would beprohibitively complex due to multi-line editing handling and multiplatformsupport.

Although this means that existing readline/editline configurations will not becompatible with the PyREPL, we believe the enhanced features and improvedextensibility are an overall win. See “Backwards Compatibility” for discussionof continued support for customized workflows.

The previous REPL made it challenging to properly implement custom commands,which is a very common feature of interactive shells. For instance, theexitcommand was implemented as a method call of a custom object injected in theglobal namespace, leading to unintuitive behavior that often confuses users whenthey simply typeexit, as the interpreter prompts them to the supposedlycorrect usageexit().

Specification

PyREPL is implemented as a new private Python module called_pyrepl, existingalongside the current C implementation. In its first implementation, itintroduces the following key features:

1. Multi-line History and Editing: Users are able to navigate and edit theircommand history across multiple lines, improving the ability to refine and reusecomplex blocks of code.

   Editing multi-line blocks provides automatic indentation using four spaces, whichis consistent withPEP 8 recommendations. When a line ending with a colon isencountered, the following line is automatically indented utilizing theindentation pattern that is inferred from the first line that containsindentation. Lines are indented with four spaces, and tabs are converted intospaces.

   Users can access history of commands by usingup anddown arrows. Withina multi-line entry, the arrow keys navigate line-by-line within the block beforemoving to the next history entry. Thedown arrow works in reverse, navigatingfrom older entries to the most recent.

   History can be searched forward (usingCtrl+S) and in reverse (usingCtrl+R)using a custom-specified substring query. It can also be searched with a prefixquery by entering the prefix into a shell line and using PageUp and PageDownkeys.

2. Copying and Pasting: in supported terminal emulators, bracketed pastingcapability is discovered and used by PyREPL. This allows for transparent pastingof blocks of code without immediate execution or invalid automatic indentation.

   For terminal emulators that don’t support this mode, a dedicated paste mode isimplemented to allow for easy insertion of multi-line code snippets withouttriggering immediate execution or indentation issues.

   Users enter manual paste mode by hitting theF3 key. The prompt changes from>>> to(paste) where users can paste contents from their clipboard ormanually type as desired. Once the content is ready, hittingF3 exits pastemode. Then, pressing Enter executes the block.

   Users can enter multiple commands on a single input when using paste mode, whichwill help paste code from other sources.

   To copy blocks of code without the leading command prompts and without theoutput of the commands, users can enter the history view via theF2 key. Thismode uses a pager to display history of executed commands without the promptsand output.

3. Help viaF1.

   Access to the standard Help module is accessible via a Custom Commandhelp(see below) or via theF1 key. HitF1 to enter help mode. When you’re done, hitF1 or a standard command (q,quit orexit) to exit.

   Browsing interactive help does not retain command history.

4. Custom Commands: The REPL supports the implementation of custom commands,such asexit, in a more natural and user-friendly manner, avoiding the currentfunction call workaround.

   The initial list of custom commands includes:

   • exit
   • quit
   • copyright
   • help
   • clear

   Commands are available as long as there is no name conflict with a variable in areachable scope. For example, after assigningexit=1, the variable willtake precedence over PyREPL commands.delexit in this case will remove theconflict and the command will function again.

5. Colors: the prompts as well as certain elements of the output, like exceptiontracebacks, are now colored. Colors can be disabled using the standardNO_COLOR environment variable, or forced by using the standardFORCE_COLOR environment variable. A Python-specific environment variable isalso available calledPYTHON_COLORS. The initial implementation in Python3.13 does not offer customization of the color theme.

These features are significantly enhancing the interactive Python experience,bringing it more in line with modern development environments and userexpectations. The implementation is in Python, offering several advantages:

1. Easier Testing and Validation: Writing tests for Python code is dramaticallysimpler and more straightforward than for C code, allowing for morecomprehensive test coverage of all existing and old features.
2. Lower Contribution Barrier: Python’s accessibility compared to C has beenencouraging more community contributions, leading to faster feature developmentand bug fixes.
3. Flexibility: A Python implementation is easier to extend and modify,improving developer velocity on new features and improvements by core developersand contributors alike.

Backwards Compatibility

The PyREPL implementation is designed to maintain full backward compatibilitywith existing Python code as the old basic REPL will be preserved as a fallbackand is available on demand, in case custom workflows require it. It will also beused in cases where the new REPL cannot be used due to environmental constraintsor other issues. Users have the option to explicitly choose the old basic REPLby setting the environment variablePYTHON_BASIC_REPL to 1. This ensuresthat users can continue using the familiar interface and capabilities if theyprefer, or if they encounter any issues with the new implementation.

It’s important to emphasize that the introduction of PyREPL does not remove anyexisting functionality. Any functionality of the old basic REPL unavailable inPyREPL is preserved and maintained in the old basic REPL that can be used byusers as a fallback.

In particular, users wanting to continue using their custom input configurationininputrc oreditrc files can continue using the old basic REPL.

The authors do not expect any PyREPL functionality to be ported to the old basicREPL. Similarly,inputrc andeditrc support is explicitly not planned inPyREPL. Those configuration files are provided by and parsed by “readline” and“editline” libraries, and their functional scope does not match thefunctionality PyREPL is targeting.

To facilitate a smooth transition,clear documentationis provided on how to switch between PyREPL and the old basic REPL.

This approach ensures that while we’re introducing significant improvements withthe new REPL, we’re not forcing any immediate changes on users who rely on thecurrent implementation. The fallback mechanism and user choice option provide asafety net that allows for gradual adoption of the new REPL while maintainingall existing functionality.

Security Implications

There are no security implications derived from this proposal.

How to Teach This

The introduction of PyREPL is accompanied by documentation and tutorials. Keyareas of focus for education will include:

1. Detailed explanations on using multi-line editing, paste mode, and other newfeatures.
2. Custom commands (existing and new).
3. How to switch to the new REPL, including anydifferences from the previous readline/editline-based configuration.

Rejected Ideas

Several alternative approaches were considered and ultimately rejected:

1. Extending the current C implementation: While this would maintain maximumbackwards compatibility, it was deemed too complex and would not address thefundamental limitations described ut supra.
2. Developing a new REPL from scratch: This approach was rejected due to thecomplexity of creating a cross-platform terminal application and the desire toleverage existing, proven code.
3. Using other existing REPL implementations: The authors looked at severalalternatives likeIPython,bpython,ptpython, andxonsh. While all the above are impressive projects,in the end PyREPL was chosen for its combination of maturity, feature set,and lack of additional dependencies. Another key factor was the alignmentwith PyPy’s implementation.

Acknowledgments

Thanks to Diego Russo for providing feedback on drafts of this PEP.

Copyright

This document is placed in the public domain or under the CC0-1.0-Universallicense, whichever is more permissive.