Abstract

PEP 750 introduced template strings (t-strings) as a generalization of f-strings, providing a way to safely handle string interpolation in various contexts. This PEP proposes extending the subprocess and shlex modules to natively support t-strings, enabling safer and more ergonomic shell command execution with interpolated values, as well as serving as a reference implementation for the t-string feature to improve API ergonomics.

Motivation

Despite the safety benefits and flexibility that template strings offer in PEP 750, they lack a concrete consumer implementation in the standard library that demonstrates their practical application. One of the most compelling use cases for t-strings is safer shell command execution, as noted in the withdrawn PEP 501:

# Unsafe with f-strings:
os.system(f"echo {message_from_user}")

# Also unsafe with f-strings
subprocess.run(f"echo {message_from_user}", shell=True)

# Fails with f-strings
subprocess.run(f"echo {message_from_user}")

# Safe with t-strings and POSIX-compliant shell quoting:
subprocess.run(t"echo {message_from_user}", shell=True)

# Safe on all platforms with t-strings:
subprocess.run(t"echo {message_from_user}")

# Safe on all platforms without t-strings:
subprocess.run(["echo", str(message_from_user)])

Currently, developers must choose between convenience (using f-strings with potential security risks) and safety (using more verbose, list-based APIs). By adding native t-string support to the subprocess module, we provide a consumer reference implementation that demonstrates the value of t-strings while addressing a common security concern.

Rationale

The subprocess module is an ideal candidate for t-string support for several reasons:

• Command injection vulnerabilities in shell commands are a well-known security risk.
• The subprocess module already supports both string and list-based command specifications.
• There’s a natural mapping between t-strings and proper shell escaping that provides both convenience and safety.
• It serves as a practical showcase for t-strings that developers can immediately understand and appreciate.

By extending subprocess to handle t-strings natively, we make it easier to write secure code without sacrificing the convenience that led many developers to use potentially unsafe f-strings.

Specification

This PEP proposes two main additions to the standard library:

1. A new sh() renderer function in the shlex module for safe shell command construction

2. Adding t-string support to the subprocess module’s core functions,

   particularly subprocess.Popen, subprocess.run(), and other related functions that accept a command argument

os.system(shlex.sh(t"cat {myfile}"))

os.system("cat " + shlex.quote(myfile)))

from string.templatelib import Template, Interpolation

def sh(template: Template) -> str:
    parts: list[str] = []
    for item in template:
        if isinstance(item, Interpolation):
            # shlex.sh implementation, so shlex.quote can be used directly
            parts.append(quote(str(item.value)))
        else:
            parts.append(item)

    # shlex.sh implementation, so `join` references shlex.join
    return join(parts)

import shlex
# Safe POSIX-compliant shell command construction
command = shlex.sh(t"cat {filename}")
os.system(command)

subprocess.run(t"cat {myfile}", shell=True)

subprocess.run(shlex.sh(t"cat {myfile}"), shell=True)

subprocess.run(t"cat {myfile} --flag {value}")

subprocess.run(["cat", myfile, "--flag", value])

subprocess.run(shlex.split(f"cat {shlex.quote(myfile)} --flag {shlex.quote(value)}"))

from string.templatelib import Template

if isinstance(args, Template):
    import shlex
    if shell:
        args = shlex.sh(args)
    else:
        args = shlex.split(shlex.sh(args))

Backwards Compatibility

This change is fully backwards compatible as it only adds new functionality without altering existing behavior. The subprocess module will continue to handle strings and lists in the same way it currently does.

Security Implications

This PEP is explicitly designed to improve security by providing a safer alternative to using f-strings with shell commands. By automatically applying appropriate escaping based on context (shell or non-shell), it helps prevent command injection vulnerabilities.

However, it’s worth noting that when shell=True is used, the safety is limited to POSIX-compliant shells. On Windows systems where cmd.exe or PowerShell may be used as the shell, the escaping mechanism provided by shlex.quote() is not sufficient to prevent all forms of command injection.

How to Teach This

This feature can be taught as a natural extension of t-strings that demonstrates their practical value:

1. Introduce the problem of command injection and why f-strings are dangerous with shell commands
2. Show the traditional solutions (list-based commands, manual escaping)
3. Introduce the shlex.sh renderer for explicit shell escaping:

   # Unsafe:
   os.system(f"cat {filename}")  # Potential command injection!
   
   # Safe using shlex.sh:
   os.system(shlex.sh(t"cat {filename}"))  # Explicitly escaping for shell
   

4. Introduce the subprocess module’s native t-string support:

   # Unsafe:
   subprocess.run(f"cat {filename}", shell=True)  # Potential command injection!
   
   # Safe but verbose:
   subprocess.run(["cat", filename])
   
   # Safe and readable with t-strings:
   subprocess.run(t"cat {filename}", shell=True)  # Automatically escapes filename
   subprocess.run(t"cat {filename}")  # Automatically converts to list form
   

The implementation should be added to both the shlex and subprocess module documentation with clear examples and security advisories.

$ cat > run_quoted.py
import sys, shlex, subprocess
subprocess.run(f"echo {shlex.quote(sys.argv[1])}", shell=True)
$ python3 run_quoted.py pwd
pwd
$ python3 run_quoted.py '; pwd'
; pwd
$ python3 run_quoted.py "'pwd'"
'pwd'

S:\> echo import sys, shlex, subprocess > run_quoted.py
S:\> echo subprocess.run(f"echo {shlex.quote(sys.argv[1])}", shell=True) >> run_quoted.py
S:\> type run_quoted.py
import sys, shlex, subprocess
subprocess.run(f"echo {shlex.quote(sys.argv[1])}", shell=True)
S:\> python3 run_quoted.py "echo OK"
'echo OK'
S:\> python3 run_quoted.py "'& echo Oh no!"
''"'"'
Oh no!'

Copyright

This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.