Abstract

One of Python’s core strengths is its interactive mode, also known as the Read-Eval-Print Loop (REPL), or the Python console, or the Python shell. This PEP describes the new implementation of this functionality written in Python. The new REPL released in Python 3.13 aims to provide modern features expected by today’s users, such as multi-line editing, syntax highlighting, custom commands, and an overall improved interactive experience.

Motivation

Up to Python 3.12, the interactive shell of CPython was written in C as a special mode of the parser. It was therefore difficult to maintain and extend. It relied on the existence of GNU readline (or an equivalent) for basic functionality like cursor movement and history tracking. Python compiled without this library provided an interactive mode of very limited capability. On the other hand, Python compiled with readline outsourced decisions and configuration around user input in ways that made extending it difficult.

This complexity has deterred contributions and has made it challenging to implement new features. As a result, the CPython interactive shell has seen minimal changes, falling behind user expectations for modern equivalents.

Many features that users have come to expect from modern REPLs were absent in the previous version. Some examples of these features include multi-line editing and history, custom commands, syntax highlighting, or ergonomic handling of copy and paste. The lack of these features greatly impacts the user experience of many groups of users of CPython, in particular in environments where users don’t control dependencies and cannot install their own packages. This is especially common for users learning the language and educators.

Addressing such issues with the C implementation would require complex workarounds, such as AST matching of commands, which would add prohibitive complexity to the codebase.

With the new REPL written in Python, we are addressing these limitations while also bringing CPython’s interactive experience more in line with modern expectations and capabilities.

Rationale

Implementing the new REPL in Python, rather than C, has significantly lowered the barrier to entry for contributors. This change has made it easier to test, validate, and modify the REPL, leading to increased community involvement and faster feature development. The improved accessibility of the codebase is expected to result in a more rapidly evolving and user-responsive REPL.

Instead of writing a Python REPL from scratch, we decided to base the implementation of the new REPL on PyREPL. This decision was driven by several key factors. First and foremost, developing a terminal application that works consistently across different operating systems and terminal emulators is a complex undertaking. By adopting PyREPL, which has been battle-tested in the PyPy project, we can leverage existing, proven code rather than starting from scratch.

Sharing a codebase with PyPy for the REPL implementation offers mutual benefits to both projects. It allows for shared maintenance efforts, faster bug fixes, and feature improvements that can benefit users of both CPython and PyPy. This collaboration can lead to a more robust and feature-rich REPL for the entire Python ecosystem.

The previous REPL written in C leveraged the “readline” or “editline” libraries as a backend to allow certain features such as navigation, history preservation and recall, autocompletion, and configurable keyboard behavior. PyREPL does not use those libraries, implementing most of the other functionality directly as part of the shell. The main missing functionality (configurability of input) is outweighed by the benefits of the new architecture. The configuration files for these libraries (e.g. inputrc) are complex and include features that PyREPL doesn’t plan to implement, making it infeasible to transparently add support for them in the new shell. Using “readline” or “editline” in PyREPL would be prohibitively complex due to multi-line editing handling and multiplatform support.

Although this means that existing readline/editline configurations will not be compatible with the PyREPL, we believe the enhanced features and improved extensibility are an overall win. See “Backwards Compatibility” for discussion of continued support for customized workflows.

The previous REPL made it challenging to properly implement custom commands, which is a very common feature of interactive shells. For instance, the exit command was implemented as a method call of a custom object injected in the global namespace, leading to unintuitive behavior that often confuses users when they simply type exit, as the interpreter prompts them to the supposedly correct usage exit().

Specification

PyREPL is implemented as a new private Python module called _pyrepl, existing alongside the current C implementation. In its first implementation, it introduces the following key features:

1. Multi-line History and Editing: Users are able to navigate and edit their command history across multiple lines, improving the ability to refine and reuse complex blocks of code.

   Editing multi-line blocks provides automatic indentation using four spaces, which is consistent with PEP 8 recommendations. When a line ending with a colon is encountered, the following line is automatically indented utilizing the indentation pattern that is inferred from the first line that contains indentation. Lines are indented with four spaces, and tabs are converted into spaces.

   Users can access history of commands by using up and down arrows. Within a multi-line entry, the arrow keys navigate line-by-line within the block before moving to the next history entry. The down arrow works in reverse, navigating from older entries to the most recent.

   History can be searched forward (using Ctrl+S) and in reverse (using Ctrl+R) using a custom-specified substring query. It can also be searched with a prefix query by entering the prefix into a shell line and using PageUp and PageDown keys.

2. Copying and Pasting: in supported terminal emulators, bracketed pasting capability is discovered and used by PyREPL. This allows for transparent pasting of blocks of code without immediate execution or invalid automatic indentation.

   For terminal emulators that don’t support this mode, a dedicated paste mode is implemented to allow for easy insertion of multi-line code snippets without triggering immediate execution or indentation issues.

   Users enter manual paste mode by hitting the F3 key. The prompt changes from >>> to (paste) where users can paste contents from their clipboard or manually type as desired. Once the content is ready, hitting F3 exits paste mode. Then, pressing Enter executes the block.

   Users can enter multiple commands on a single input when using paste mode, which will help paste code from other sources.

   To copy blocks of code without the leading command prompts and without the output of the commands, users can enter the history view via the F2 key. This mode uses a pager to display history of executed commands without the prompts and output.

3. Help via F1.

   Access to the standard Help module is accessible via a Custom Command help (see below) or via the F1 key. Hit F1 to enter help mode. When you’re done, hit F1 or a standard command (q, quit or exit) to exit.

   Browsing interactive help does not retain command history.

4. Custom Commands: The REPL supports the implementation of custom commands, such as exit, in a more natural and user-friendly manner, avoiding the current function call workaround.

   The initial list of custom commands includes:

   • exit
   • quit
   • copyright
   • help
   • clear

   Commands are available as long as there is no name conflict with a variable in a reachable scope. For example, after assigning exit = 1, the variable will take precedence over PyREPL commands. del exit in this case will remove the conflict and the command will function again.

5. Colors: the prompts as well as certain elements of the output, like exception tracebacks, are now colored. Colors can be disabled using the standard NO_COLOR environment variable, or forced by using the standard FORCE_COLOR environment variable. A Python-specific environment variable is also available called PYTHON_COLORS. The initial implementation in Python 3.13 does not offer customization of the color theme.

These features are significantly enhancing the interactive Python experience, bringing it more in line with modern development environments and user expectations. The implementation is in Python, offering several advantages:

1. Easier Testing and Validation: Writing tests for Python code is dramatically simpler and more straightforward than for C code, allowing for more comprehensive test coverage of all existing and old features.
2. Lower Contribution Barrier: Python’s accessibility compared to C has been encouraging more community contributions, leading to faster feature development and bug fixes.
3. Flexibility: A Python implementation is easier to extend and modify, improving developer velocity on new features and improvements by core developers and contributors alike.

Backwards Compatibility

The PyREPL implementation is designed to maintain full backward compatibility with existing Python code as the old basic REPL will be preserved as a fallback and is available on demand, in case custom workflows require it. It will also be used in cases where the new REPL cannot be used due to environmental constraints or other issues. Users have the option to explicitly choose the old basic REPL by setting the environment variable PYTHON_BASIC_REPL to 1. This ensures that users can continue using the familiar interface and capabilities if they prefer, or if they encounter any issues with the new implementation.

It’s important to emphasize that the introduction of PyREPL does not remove any existing functionality. Any functionality of the old basic REPL unavailable in PyREPL is preserved and maintained in the old basic REPL that can be used by users as a fallback.

In particular, users wanting to continue using their custom input configuration in inputrc or editrc files can continue using the old basic REPL.

The authors do not expect any PyREPL functionality to be ported to the old basic REPL. Similarly, inputrc and editrc support is explicitly not planned in PyREPL. Those configuration files are provided by and parsed by “readline” and “editline” libraries, and their functional scope does not match the functionality PyREPL is targeting.

To facilitate a smooth transition, clear documentation is provided on how to switch between PyREPL and the old basic REPL.

This approach ensures that while we’re introducing significant improvements with the new REPL, we’re not forcing any immediate changes on users who rely on the current implementation. The fallback mechanism and user choice option provide a safety net that allows for gradual adoption of the new REPL while maintaining all existing functionality.

Security Implications

There are no security implications derived from this proposal.

How to Teach This

The introduction of PyREPL is accompanied by documentation and tutorials. Key areas of focus for education will include:

1. Detailed explanations on using multi-line editing, paste mode, and other new features.
2. Custom commands (existing and new).
3. How to switch to the new REPL, including any differences from the previous readline/editline-based configuration.

Rejected Ideas

Several alternative approaches were considered and ultimately rejected:

1. Extending the current C implementation: While this would maintain maximum backwards compatibility, it was deemed too complex and would not address the fundamental limitations described ut supra.
2. Developing a new REPL from scratch: This approach was rejected due to the complexity of creating a cross-platform terminal application and the desire to leverage existing, proven code.
3. Using other existing REPL implementations: The authors looked at several alternatives like IPython, bpython, ptpython, and xonsh. While all the above are impressive projects, in the end PyREPL was chosen for its combination of maturity, feature set, and lack of additional dependencies. Another key factor was the alignment with PyPy’s implementation.

Acknowledgments

Thanks to Diego Russo for providing feedback on drafts of this PEP.

Copyright

This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.