Following system colour scheme Selected dark colour scheme Selected light colour scheme

Python Enhancement Proposals

PEP 750 – Template Strings

Author:
Jim Baker <jim.baker at python.org>, Guido van Rossum <guido at python.org>, Paul Everitt <pauleveritt at me.com>, Koudai Aono <koxudaxi at gmail.com>, Lysandros Nikolaou <lisandrosnik at gmail.com>, Dave Peck <davepeck at davepeck.org>
Discussions-To:
Discourse thread
Status:
Draft
Type:
Standards Track
Created:
08-Jul-2024
Python-Version:
3.14
Post-History:
09-Aug-2024, 17-Oct-2024, 21-Oct-2024, 18-Nov-2024

Table of Contents

Abstract

This PEP introduces template strings for custom string processing.

Template strings are a generalization of f-strings, using a t in place of the f prefix. Instead of evaluating to str, t-strings evaluate to a new type, Template:

template: Template = t"Hello {name}"

Templates provide developers with access to the string and its interpolated values before they are combined. This brings native flexible string processing to the Python language and enables safety checks, web templating, domain-specific languages, and more.

Relationship With Other PEPs

Python introduced f-strings in Python 3.6 with PEP 498. The grammar was then formalized in PEP 701 which also lifted some restrictions. This PEP is based on PEP 701.

At nearly the same time PEP 498 arrived, PEP 501 was written to provide “i-strings” – that is, “interpolation template strings”. The PEP was deferred pending further experience with f-strings. Work on this PEP was resumed by a different author in March 2023, introducing “t-strings” as template literal strings, and built atop PEP 701.

The authors of this PEP consider it to be a generalization and simplification of the updated work in PEP 501. (That PEP has also recently been updated to reflect the new ideas in this PEP.)

Motivation

Python f-strings are easy to use and very popular. Over time, however, developers have encountered limitations that make them unsuitable for certain use cases. In particular, f-strings provide no way to intercept and transform interpolated values before they are combined into a final string.

As a result, incautious use of f-strings can lead to security vulnerabilities. For example, a user executing a SQL query with sqlite3 may be tempted to use an f-string to embed values into their SQL expression, which could lead to a SQL injection attack. Or, a developer building HTML may include unescaped user input in the string, leading to a cross-site scripting (XSS) vulnerability.

More broadly, the inability to transform interpolated values before they are combined into a final string limits the utility of f-strings in more complex string processing tasks.

Template strings address these problems by providing developers with access to the string and its interpolated values.

For example, imagine we want to generate some HTML. Using template strings, we can define an html() function that allows us to automatically sanitize content:

evil = "<script>alert('evil')</script>"
template = t"<p>{evil}</p>"
assert html(template) == "<p>&lt;script&gt;alert('evil')&lt;/script&gt;</p>"

Likewise, our hypothetical html() function can make it easy for developers to add attributes to HTML elements using a dictionary:

attributes = {"src": "shrubbery.jpg", "alt": "looks nice"}
template = t"<img {attributes} />"
assert html(template) == '<img src="shrubbery.jpg" alt="looks nice" />'

Neither of these examples is possible with f-strings. By providing a mechanism to intercept and transform interpolated values, template strings enable a wide range of string processing use cases.

Specification

Template String Literals

This PEP introduces a new string prefix, t, to define template string literals. These literals resolve to a new type, Template, found in the standard library module <<TBD>>.

The following code creates a Template instance:

from TBD import Template
template = t"This is a template string."
assert isinstance(template, Template)

Template string literals support the full syntax of PEP 701. This includes the ability to nest template strings within interpolations, as well as the ability to use all valid quote marks (', ", ''', and """). Like other string prefixes, the t prefix must immediately precede the quote. Like f-strings, both lowercase t and uppercase T prefixes are supported. Like f-strings, t-strings may not be combined with u or the b prefix.

Additionally, f-strings and t-strings cannot be combined, so the ft prefix is invalid. t-strings may be combined with the r prefix; see the Raw Template Strings section below for more information.

The Template Type

Template strings evaluate to an instance of a new immutable type, <<TBD>>.Template:

class Template:
    strings: tuple[str, ...]
    """
    A non-empty tuple of the string parts of the template,
    with N+1 items, where N is the number of interpolations
    in the template.
    """

    interpolations: tuple[Interpolation, ...]
    """
    A tuple of the interpolation parts of the template.
    This will be an empty tuple if there are no interpolations.
    """

    def __new__(cls, *args: str | Interpolation):
        """
        Create a new Template instance.

        Arguments can be provided in any order.
        """
        ...

    @property
    def values(self) -> tuple[object, ...]:
        """
        Return a tuple of the `value` attributes of each Interpolation
        in the template.
        This will be an empty tuple if there are no interpolations.
        """
        ...

    def __iter__(self) -> Iterator[str | Interpolation]:
        """
        Iterate over the string parts and interpolations in the template.

        These may appear in any order. Empty strings will not be included.
        """
        ...

The strings and interpolations attributes provide access to the string parts and any interpolations in the literal:

name = "World"
template = t"Hello {name}"
assert template.strings[0] == "Hello "
assert template.interpolations[0].value == "World"

The Interpolation Type

The Interpolation type represents an expression inside a template string. Like Template, it is a new class found in the <<TBD>> module:

class Interpolation:
    value: object
    expr: str
    conv: Literal["a", "r", "s"] | None
    format_spec: str

    __match_args__ = ("value", "expr", "conv", "format_spec")

    def __new__(
        cls,
        value: object,
        expr: str,
        conv: Literal["a", "r", "s"] | None = None,
        format_spec: str = "",
    ):
        ...

The Interpolation type is shallow immutable. Its attributes cannot be reassigned.

The value attribute is the evaluated result of the interpolation:

name = "World"
template = t"Hello {name}"
assert template.interpolations[0].value == "World"

The expr attribute is the original text of the interpolation:

name = "World"
template = t"Hello {name}"
assert template.interpolations[0].expr == "name"

We expect that the expr attribute will not be used in most template processing code. It is provided for completeness and for use in debugging and introspection. See both the Common Patterns Seen in Processing Templates section and the Examples section for more information on how to process template strings.

The conv attribute is the optional conversion to be used, one of r, s, and a, corresponding to repr(), str(), and ascii() conversions. As with f-strings, no other conversions are supported:

name = "World"
template = t"Hello {name!r}"
assert template.interpolations[0].conv == "r"

If no conversion is provided, conv is None.

The format_spec attribute is the format specification. As with f-strings, this is an arbitrary string that defines how to present the value:

value = 42
template = t"Value: {value:.2f}"
assert template.interpolations[0].format_spec == ".2f"

Format specifications in f-strings can themselves contain interpolations. This is permitted in template strings as well; format_spec is set to the eagerly evaluated result:

value = 42
precision = 2
template = t"Value: {value:.{precision}f}"
assert template.interpolations[0].format_spec == ".2f"

If no format specification is provided, format_spec defaults to an empty string (""). This matches the format_spec parameter of Python’s format() built-in.

Unlike f-strings, it is up to code that processes the template to determine how to interpret the conv and format_spec attributes. Such code is not required to use these attributes, but when present they should be respected, and to the extent possible match the behavior of f-strings. It would be surprising if, for example, a template string that uses {value:.2f} did not round the value to two decimal places when processed.

Convenience Accessors in Template

The Template.values property is equivalent to:

@property
def values(self) -> tuple[object, ...]:
    return tuple(i.value for i in self.interpolations)

The Template.__iter__() method is equivalent to:

def __iter__(self) -> Iterator[str | Interpolation]:
    for s, i in zip_longest(self.strings, self.interpolations):
        if s:
            yield s
        if i:
            yield i

Processing Template Strings

Developers can write arbitrary code to process template strings. For example, the following function renders static parts of the template in lowercase and interpolations in uppercase:

from TBD import Template, Interpolation

def lower_upper(template: Template) -> str:
    """Render static parts lowercased and interpolations uppercased."""
    parts: list[str] = []
    for item in template:
        if isinstance(item, Interpolation):
            parts.append(str(item.value).upper())
        else:
            parts.append(item.lower())
    return "".join(parts)

name = "world"
assert lower_upper(t"HELLO {name}") == "hello WORLD"

There is no requirement that template strings are processed in any particular way. Code that processes templates has no obligation to return a string. Template strings are a flexible, general-purpose feature.

See the Common Patterns Seen in Processing Templates section for more information on how to process template strings. See the Examples section for detailed working examples.

Template String Concatenation

Template strings support explicit concatenation using +. Concatenation is supported for two Template instances as well as for a Template instance and a str:

name = "World"
template = t"{name}"

assert isinstance(t"Hello " + template, Template)
assert (t"Hello " + template).strings == ("Hello ", "")
assert (t"Hello " + template).interpolations[0].value == "World"

assert isinstance("Hello " + template, Template)
assert ("Hello " + template).strings == ("Hello ", "")
assert ("Hello " + template).interpolations[0].value == "World"

Concatenation of templates is “viral”: the concatenation of a Template and a str always results in a Template instance.

Python’s implicit concatenation syntax is also supported. The following code will work as expected:

name = "World"
assert (t"Hello " t"World").strings == ("Hello World",)
assert ("Hello " t"World").strings == ("Hello World",)

The Template type supports the __add__() and __radd__() methods between two Template instances and between a Template instance and a str.

Template and Interpolation Equality

Template and Interpolation instances compare with object identity (is).

Template instances are intended to be used by template processing code, which may return a string or any other type. Those types can provide their own equality semantics as needed.

No Support for Ordering

The Template and Interpolation types do not support ordering. This is unlike all other string literal types in Python, which support lexicographic ordering. Because interpolations can contain arbitrary values, there is no natural ordering for them. As a result, neither the Template nor the Interpolation type implements the standard comparison methods.

Support for the debug specifier (=)

The debug specifier, =, is supported in template strings and behaves similarly to how it behaves in f-strings, though due to limitations of the implementation there is a slight difference.

In particular, t'{expr=}' is treated as t'expr={expr!r}':

name = "World"
template = t"Hello {name=}"
assert template.strings[0] == "Hello name="
assert template.interpolations[0].value == "World"
assert template.interpolations[0].conv == "r"

If a separate format string is also provided, t'{expr=:fmt} is treated instead as t'expr={expr!s:fmt}'.

Whitespace is preserved in the debug specifier, so t'{expr = }' is treated as t'expr = {expr!r}'.

Raw Template Strings

Raw template strings are supported using the rt (or tr) prefix:

trade = 'shrubberies'
template = rt'Did you say "{trade}"?\n'
assert template.strings[0] == r'Did you say "'
assert template.strings[1] == r'"?\n'

In this example, the \n is treated as two separate characters (a backslash followed by ‘n’) rather than a newline character. This is consistent with Python’s raw string behavior.

As with regular template strings, interpolations in raw template strings are processed normally, allowing for the combination of raw string behavior and dynamic content.

Interpolation Expression Evaluation

Expression evaluation for interpolations is the same as in PEP 498:

The expressions that are extracted from the string are evaluated in the context where the template string appeared. This means the expression has full access to its lexical scope, including local and global variables. Any valid Python expression can be used, including function and method calls.

Template strings are evaluated eagerly from left to right, just like f-strings. This means that interpolations are evaluated immediately when the template string is processed, not deferred or wrapped in lambdas.

Exceptions

Exceptions raised in t-string literals are the same as those raised in f-string literals.

No Template.__str__() Implementation

The Template type does not provide a specialized __str__() implementation.

This is because Template instances are intended to be used by template processing code, which may return a string or any other type. There is no canonical way to convert a Template to a string.

The Template and Interpolation types both provide useful __repr__() implementations.

Examples

All examples in this section of the PEP have fully tested reference implementations available in the public pep750-examples git repository.

Example: Implementing f-strings with t-strings

It is easy to “implement” f-strings using t-strings. That is, we can write a function f(template: Template) -> str that processes a Template in much the same way as an f-string literal, returning the same result:

name = "World"
value = 42
templated = t"Hello {name!r}, value: {value:.2f}"
formatted = f"Hello {name!r}, value: {value:.2f}"
assert f(templated) == formatted

The f() function supports both conversion specifiers like !r and format specifiers like :.2f. The full code is fairly simple:

from TBD import Template, Interpolation

def convert(value: object, conv: Literal["a", "r", "s"] | None) -> object:
    if conv == "a":
        return ascii(value)
    elif conv == "r":
        return repr(value)
    elif conv == "s":
        return str(value)
    return value

def f(template: Template) -> str:
    parts = []
    for item in template:
        match item:
            case str() as s:
                parts.append(s)
            case Interpolation(value, _, conv, format_spec):
                value = convert(value, conv)
                value = format(value, format_spec)
                parts.append(value)
    return "".join(parts)

Note

Example code

See fstring.py and test_fstring.py.

Example: Structured Logging

Structured logging allows developers to log data in machine-readable formats like JSON. With t-strings, developers can easily log structured data alongside human-readable messages using just a single log statement.

We present two different approaches to implementing structured logging with template strings.

Approach 1: Custom Log Messages

The Python Logging Cookbook has a short section on how to implement structured logging.

The logging cookbook suggests creating a new “message” class, StructuredMessage, that is constructed with a simple text message and a separate dictionary of values:

message = StructuredMessage("user action", {
    "action": "traded",
    "amount": 42,
    "item": "shrubs"
})
logging.info(message)

# Outputs:
# user action >>> {"action": "traded", "amount": 42, "item": "shrubs"}

The StructuredMessage.__str__() method formats both the human-readable message and the values, combining them into a final string. (See the logging cookbook for its full example.)

We can implement an improved version of StructuredMessage using template strings:

import json
from TBD import Interpolation, Template
from typing import Mapping

class TemplateMessage:
    def __init__(self, template: Template) -> None:
        self.template = template

    @property
    def message(self) -> str:
        # Use the f() function from the previous example
        return f(self.template)

    @property
    def values(self) -> Mapping[str, object]:
        return {
            item.expr: item.value
            for item in self.template
            if isinstance(item, Interpolation)
        }

    def __str__(self) -> str:
        return f"{self.message} >>> {json.dumps(self.values)}"

_ = TemplateMessage  # optional, to improve readability
action, amount, item = "traded", 42, "shrubs"
logging.info(_(t"User {action}: {amount:.2f} {item}"))

# Outputs:
# User traded: 42.00 shrubs >>> {"action": "traded", "amount": 42, "item": "shrubs"}

Template strings give us a more elegant way to define the custom message class. With template strings it is no longer necessary for developers to make sure that their format string and values dictionary are kept in sync; a single template string literal is all that is needed. The TemplateMessage implementation can automatically extract structured keys and values from the Interpolation.expr and Interpolation.value attributes, respectively.

Approach 2: Custom Formatters

Custom messages are a reasonable approach to structured logging but can be a little awkward. To use them, developers must wrap every log message they write in a custom class. This can be easy to forget.

An alternative approach is to define custom logging.Formatter classes. This approach is more flexible and allows for more control over the final output. In particular, it’s possible to take a single template string and output it in multiple formats (human-readable and JSON) to separate log streams.

We define two simple formatters, a MessageFormatter for human-readable output and a ValuesFormatter for JSON output:

import json
from logging import Formatter, LogRecord
from TBD import Interpolation, Template
from typing import Any, Mapping


class MessageFormatter(Formatter):
    def message(self, template: Template) -> str:
        # Use the f() function from the previous example
        return f(template)

    def format(self, record: LogRecord) -> str:
        msg = record.msg
        if not isinstance(msg, Template):
            return super().format(record)
        return self.message(msg)


class ValuesFormatter(Formatter):
    def values(self, template: Template) -> Mapping[str, Any]:
        return {
            item.expr: item.value
            for item in template
            if isinstance(item, Interpolation)
        }

    def format(self, record: LogRecord) -> str:
        msg = record.msg
        if not isinstance(msg, Template):
            return super().format(record)
        return json.dumps(self.values(msg))

We can then use these formatters when configuring our logger:

import logging
import sys

logger = logging.getLogger(__name__)
message_handler = logging.StreamHandler(sys.stdout)
message_handler.setFormatter(MessageFormatter())
logger.addHandler(message_handler)

values_handler = logging.StreamHandler(sys.stderr)
values_handler.setFormatter(ValuesFormatter())
logger.addHandler(values_handler)

action, amount, item = "traded", 42, "shrubs"
logger.info(t"User {action}: {amount:.2f} {item}")

# Outputs to sys.stdout:
# User traded: 42.00 shrubs

# At the same time, outputs to sys.stderr:
# {"action": "traded", "amount": 42, "item": "shrubs"}

This approach has a couple advantages over the custom message approach to structured logging:

  • Developers can log a t-string directly without wrapping it in a custom class.
  • Human-readable and structured output can be sent to separate log streams. This is useful for log aggregation systems that process structured data independently from human-readable data.

Note

Example code

See logging.py and test_logging.py.

Example: HTML Templating

This PEP contains several short HTML templating examples. It turns out that the “hypothetical” html() function mentioned in the Motivation section (and a few other places in this PEP) exists and is available in the pep750-examples repository. If you’re thinking about parsing a complex grammar with template strings, we hope you’ll find it useful.

Backwards Compatibility

Like f-strings, use of template strings will be a syntactic backwards incompatibility with previous versions.

Security Implications

The security implications of working with template strings, with respect to interpolations, are as follows:

  1. Scope lookup is the same as f-strings (lexical scope). This model has been shown to work well in practice.
  2. Code that processes Template instances can ensure that any interpolations are processed in a safe fashion, including respecting the context in which they appear.

How To Teach This

Template strings have several audiences:

  • Developers using template strings and processing functions
  • Authors of template processing code
  • Framework authors who build interesting machinery with template strings

We hope that teaching developers will be straightforward. At a glance, template strings look just like f-strings. Their syntax is familiar and the scoping rules remain the same.

The first thing developers must learn is that template string literals don’t evaluate to strings; instead, they evaluate to a new type, Template. This is a simple type intended to be used by template processing code. It’s not until developers call a processing function that they get the result they want: typically, a string, although processing code can of course return any arbitrary type.

Developers will also want to understand how template strings relate to other string formatting methods like f-strings and str.format(). They will need to decide when to use each method. If a simple string is all that is needed, and there are no security implications, f-strings are likely the best choice. For most cases where a format string is used, it can be replaced with a function wrapping the creation of a template string. In cases where the format string is obtained from user input, the filesystem, or databases, it is possible to write code to convert it into a Template instance if desired.

Because developers will learn that t-strings are nearly always used in tandem with processing functions, they don’t necessarily need to understand the details of the Template type. As with descriptors and decorators, we expect many more developers will use t-strings than write t-string processing functions.

Over time, a small number of more advanced developers will wish to author their own template processing code. Writing processing code often requires thinking in terms of formal grammars. Developers will need to learn how to work with the strings and interpolation attributes of a Template instance and how to process interpolations in a context-sensitive fashion. More sophisticated grammars will likely require parsing to intermediate representations like an AST. Great template processing code will handle format specifiers and conversions when appropriate. Writing production-grade template processing code – for instance, to support HTML templates – can be a large undertaking.

We expect that template strings will provide framework authors with a powerful new tool in their toolbox. While the functionality of template strings overlaps with existing tools like template engines, t-strings move that logic into the language itself. Bringing the full power and generality of Python to bear on string processing tasks opens new possibilities for framework authors.

Why another templating approach?

The world of Python already has mature templating languages with wide adoption, such as Jinja. Why build support for creating new templating systems?

Projects such as Jinja are still needed in cases where the template is less part of the software by the developers, and more part of customization by designers or even content created by users, for example in a CMS.

The trends in frontend development have treated templating as part of the software and written by developers. They want modern language features and a good tooling experience. PEP 750 envisions DSLs where the non-static parts are Python: same scope rules, typing, expression syntax, and the like.

Common Patterns Seen in Processing Templates

Structural Pattern Matching

Iterating over the Template with structural pattern matching is the expected best practice for many template function implementations:

from TBD import Template, Interpolation

def process(template: Template) -> Any:
    for item in template:
        match item:
            case str() as s:
                ... # handle each string part
            case Interpolation() as interpolation:
                ... # handle each interpolation

Processing code may also commonly sub-match on attributes of the Interpolation type:

match arg:
    case Interpolation(int()):
        ... # handle interpolations with integer values
    case Interpolation(value=str() as s):
        ... # handle interpolations with string values
    # etc.

Memoizing

Template functions can efficiently process both static and dynamic parts of templates. The structure of Template objects allows for effective memoization:

strings = template.strings  # Static string parts
values = template.values  # Dynamic interpolated values

This separation enables caching of processed static parts while dynamic parts can be inserted as needed. Authors of template processing code can use the static strings as cache keys, leading to significant performance improvements when similar templates are used repeatedly.

Parsing to Intermediate Representations

Code that processes templates can parse the template string into intermediate representations, like an AST. We expect that many template processing libraries will use this approach.

For instance, rather than returning a str, our theoretical html() function (see the Motivation section) could return an HTML Element defined in the same package:

@dataclass(frozen=True)
class Element:
    tag: str
    attributes: Mapping[str, str | bool]
    children: Sequence[str | Element]

    def __str__(self) -> str:
        ...


def html(template: Template) -> Element:
    ...

Calling str(element) would then render the HTML but, in the meantime, the Element could be manipulated in a variety of ways.

Context-sensitive Processing of Interpolations

Continuing with our hypothetical html() function, it could be made context-sensitive. Interpolations could be processed differently depending on where they appear in the template.

For example, our html() function could support multiple kinds of interpolations:

attributes = {"id": "main"}
attribute_value = "shrubbery"
content = "hello"
template = t"<div {attributes} data-value={attribute_value}>{content}</div>"
element = html(template)
assert str(element) == '<div id="main" data-value="shrubbery">hello</div>'

Because the {attributes} interpolation occurs in the context of an HTML tag, and because there is no corresponding attribute name, it is treated as a dictionary of attributes. The {attribute_value} interpolation is treated as a simple string value and is quoted before inclusion in the final string. The {content} interpolation is treated as potentially unsafe content and is escaped before inclusion in the final string.

Nested Template Strings

Going a step further with our html() function, we could support nested template strings. This would allow for more complex HTML structures to be built up from simpler templates:

name = "World"
content = html(t"<p>Hello {name}</p>")
template = t"<div>{content}</div>"
element = html(template)
assert str(element) == '<div><p>Hello World</p></div>'

Because the {content} interpolation is an Element instance, it does not need to be escaped before inclusion in the final string.

One could imagine a nice simplification: if the html() function is passed a Template instance, it could automatically convert it to an Element by recursively calling itself on the nested template.

We expect that nesting and composition of templates will be a common pattern in template processing code and, where appropriate, used in preference to simple string concatenation.

Approaches to Lazy Evaluation

Like f-strings, interpolations in t-string literals are eagerly evaluated. However, there are cases where lazy evaluation may be desirable.

If a single interpolation is expensive to evaluate, it can be explicitly wrapped in a lambda in the template string literal:

name = "World"
template = t"Hello {(lambda: name)}"
assert callable(template.interpolations[0].value)
assert template.interpolations[0].value() == "World"

This assumes, of course, that template processing code anticipates and handles callable interpolation values. (One could imagine also supporting iterators, awaitables, etc.) This is not a requirement of the PEP, but it is a common pattern in template processing code.

In general, we hope that the community will develop best practices for lazy evaluation of interpolations in template strings and that, when it makes sense, common libraries will provide support for callable or awaitable values in their template processing code.

Approaches to Asynchronous Evaluation

Closely related to lazy evaluation is asynchronous evaluation.

As with f-strings, the await keyword is allowed in interpolations:

async def example():
    async def get_name() -> str:
        await asyncio.sleep(1)
        return "Sleepy"

    template = t"Hello {await get_name()}"
    # Use the f() function from the f-string example, above
    assert f(template) == "Hello Sleepy"

More sophisticated template processing code can take advantage of this to perform asynchronous operations in interpolations. For example, a “smart” processing function could anticipate that an interpolation is an awaitable and await it before processing the template string:

async def example():
    async def get_name() -> str:
        await asyncio.sleep(1)
        return "Sleepy"

    template = t"Hello {get_name}"
    assert await async_f(template) == "Hello Sleepy"

This assumes that the template processing code in async_f() is asynchronous and is able to await an interpolation’s value.

Note

Example code

See afstring.py and test_afstring.py.

Approaches to Template Reuse

If developers wish to reuse template strings multiple times with different values, they can write a function to return a Template instance:

def reusable(name: str, question: str) -> Template:
    return t"Hello {name}, {question}?"

template = reusable("friend", "how are you")
template = reusable("King Arthur", "what is your quest")

This is, of course, no different from how f-strings can be reused.

Relation to Format Strings

The venerable str.format() method accepts format strings that can later be used to format values:

alas_fmt = "We're all out of {cheese}."
assert alas_fmt.format(cheese="Red Leicester") == "We're all out of Red Leicester."

If one squints, one can think of format strings as a kind of function definition. The call to str.format() can be seen as a kind of function call. The t-string equivalent is to simply define a standard Python function that returns a Template instance:

def make_template(*, cheese: str) -> Template:
    return t"We're all out of {cheese}."

template = make_template(cheese="Red Leicester")
# Using the f() function from the f-string example, above
assert f(template) == "We're all out of Red Leicester."

The make_template() function itself can be thought of as analogous to the format string. The call to make_template() is analogous to the call to str.format().

Of course, it is common to load format strings from external sources like a filesystem or database. Thankfully, because Template and Interpolation are simple Python types, it is possible to write a function that takes an old-style format string and returns an equivalent Template instance:

def from_format(fmt: str, /, *args: object, **kwargs: object) -> Template:
     """Parse `fmt` and return a `Template` instance."""
     ...

 # Load this from a file, database, etc.
 fmt = "We're all out of {cheese}."
 template = from_format(fmt, cheese="Red Leicester")
 # Using the f() function from the f-string example, above
 assert f(template) == "We're all out of Red Leicester."

This is a powerful pattern that allows developers to use template strings in places where they might have previously used format strings. A full implementation of from_format() is available in the examples repository, which supports the full grammar of format strings.

Note

Example code

See format.py and test_format.py.

Reference Implementation

A CPython implementation of PEP 750 is available.

There is also a public repository of examples and tests built around the reference implementation. If you’re interested in playing with template strings, this repository is a great place to start.

Rejected Ideas

This PEP has been through several significant revisions. In addition, quite a few interesting ideas were considered both in revisions of PEP 501 and in the Discourse discussion.

We attempt to document the most significant ideas that were considered and rejected.

Arbitrary String Literal Prefixes

Inspired by JavaScript tagged template literals, an earlier version of this PEP allowed for arbitrary “tag” prefixes in front of literal strings:

my_tag'Hello {name}'

The prefix was a special callable called a “tag function”. Tag functions received the parts of the template string in an argument list. They could then process the string and return an arbitrary value:

def my_tag(*args: str | Interpolation) -> Any:
    ...

This approach was rejected for several reasons:

  • It was deemed too complex to build in full generality. JavaScript allows for arbitrary expressions to precede a template string, which is a significant challenge to implement in Python.
  • It precluded future introduction of new string prefixes.
  • It seemed to needlessly pollute the namespace.

Use of a single t prefix was chosen as a simpler, more Pythonic approach and more in keeping with template strings’ role as a generalization of f-strings.

Delayed Evaluation of Interpolations

An early version of this PEP proposed that interpolations should be lazily evaluated. All interpolations were “wrapped” in implicit lambdas. Instead of having an eagerly evaluated value attribute, interpolations had a getvalue() method that would resolve the value of the interpolation:

class Interpolation:
    ...
    _value: Callable[[], object]

    def getvalue(self) -> object:
        return self._value()

This was rejected for several reasons:

  • The overwhelming majority of use cases for template strings naturally call for immediate evaluation.
  • Delayed evaluation would be a significant departure from the behavior of f-strings.
  • Implicit lambda wrapping leads to difficulties with type hints and static analysis.

Most importantly, there are viable (if imperfect) alternatives to implicit lambda wrapping in many cases where lazy evaluation is desired. See the section on Approaches to Lazy Evaluation, above, for more information.

While delayed evaluation was rejected for this PEP, we hope that the community continues to explore the idea.

Making Template and Interpolation Into Protocols

An early version of this PEP proposed that the Template and Interpolation types be runtime checkable protocols rather than classes.

In the end, we felt that using classes was more straightforward.

Overridden __eq__ and __hash__ for Template and Interpolation

Earlier versions of this PEP proposed that the Template and Interpolation types should have their own implementations of __eq__ and __hash__.

Templates were considered equal if their strings and interpolations were equal; Interpolations were considered equal if their value, expr, conv, and format_spec were equal. Interpolation hashing was similar to tuple hashing: an Interpolation was hashable if and only if its value was hashable.

This was rejected because Template.__hash__ so defined was not useful as a cache key in template processing code; we were concerned that it would be confusing to developers.

By dropping these implementations of __eq__ and __hash__, we lose the ability to write asserts such as:

name = "World"
assert t"Hello " + t"{name}" == t"Hello {name}"

Because Template instances are intended to be quickly processed by further code, we felt that the utility of these asserts was limited.

An Additional Decoded Type

An early version of this PEP proposed an additional type, Decoded, to represent the “static string” parts of a template string. This type derived from str and had a single extra raw attribute that provided the original text of the string. We rejected this in favor of the simpler approach of using plain str and allowing combination of r and t prefixes.

The Final Home for Template and Interpolation

Previous versions of this PEP proposed placing the Template and Interpolation types in: types, collections, collections.abc, and even in a new top-level module, templatelib. As of this writing, no core team consensus has emerged on the final location for these types. The current PEP leaves this open for a final decision.

One argument in favor of a new top-level templatelib module is that it would allow for future addition of related methods (like convert()) and for potential future template processing code to be added to submodules (templatelib.shell, etc.).

Enable Full Reconstruction of Original Template Literal

Earlier versions of this PEP attempted to make it possible to fully reconstruct the text of the original template string from a Template instance. This was rejected as being overly complex.

There are several limitations with respect to round-tripping to the original source text:

  • Interpolation.format_spec defaults to "" if not provided. It is therefore impossible to distinguish t"{expr}" from t"{expr:}".
  • The debug specifier, =, is treated as a special case. It is therefore not possible to distinguish t"{expr=}" from t"expr={expr}".
  • Finally, format specifiers in f-strings allow arbitrary nesting. In this PEP and in the reference implementation, the specifier is eagerly evaluated to set the format_spec in the Interpolation, thereby losing the original expressions. For example:
value = 42
precision = 2
template = t"Value: {value:.{precision}f}"
assert template.interpolations[0].format_spec == ".2f"

We do not anticipate that these limitations will be a significant issue in practice. Developers who need to obtain the original template string literal can always use inspect.getsource() or similar tools.

Disallowing String Concatenation

Earlier versions of this PEP proposed that template strings should not support concatenation. This was rejected in favor of allowing concatenation.

There are reasonable arguments in favor of rejecting one or all forms of concatenation: namely, that it cuts off a class of potential bugs, particularly when one takes the view that template strings will often contain complex grammars for which concatenation doesn’t always have the same meaning (or any meaning).

Moreover, the earliest versions of this PEP proposed a syntax closer to JavaScript’s tagged template literals, where an arbitrary callable could be used as a prefix to a string literal. There was no guarantee that the callable would return a type that supported concatenation.

In the end, we decided that the surprise to developers of a new string type not supporting concatenation was likely to be greater than the theoretical harm caused by supporting it. (Developers concatenate f-strings all the time, after all, and while we are sure there are cases where this introduces bugs, it’s not clear that those bugs outweigh the benefits of supporting concatenation.)

While concatenation is supported, we expect that code that uses template strings will more commonly build up larger templates through nesting and composition rather than concatenation.

Arbitrary Conversion Values

Python allows only r, s, or a as possible conversion type values. Trying to assign a different value results in SyntaxError.

In theory, template functions could choose to handle other conversion types. But this PEP adheres closely to PEP 701. Any changes to allowed values should be in a separate PEP.

Removing conv From Interpolation

During the authoring of this PEP, we considered removing the conv attribute from Interpolation and specifying that the conversion should be performed eagerly, before Interpolation.value is set.

This was done to simplify the work of writing template processing code. The conv attribute is of limited extensibility (it is typed as Literal["r", "s", "a"] | None). It is not clear that it adds significant value or flexibility to template strings that couldn’t better be achieved with custom format specifiers. Unlike with format specifiers, there is no equivalent to Python’s format() built-in. (Instead, we include an sample implementation of convert() in the Examples section.)

Ultimately we decided to keep the conv attribute in the Interpolation type to maintain compatibility with f-strings and to allow for future extensibility.

Alternate Interpolation Symbols

In the early stages of this PEP, we considered allowing alternate symbols for interpolations in template strings. For example, we considered allowing ${name} as an alternative to {name} with the idea that it might be useful for i18n or other purposes. See the Discourse thread for more information.

This was rejected in favor of keeping t-string syntax as close to f-string syntax as possible.

Alternate Layouts for Template

During the development of this PEP, we considered several alternate layouts for the Template type. Many focused on a single args tuple that contained both strings and interpolations. Variants included:

  • args was a tuple[str | Interpolation, ...]` with the promise that its first and last items were strings and that strings and interpolations always alternated. This implied that args was always non-empty and that empty strings would be inserted between neighboring interpolations. This was rejected because alternation could not be captured by the type system and was not a guarantee we wished to make.
  • args remained a tuple[str | Interpolation, ...] but did not support interleaving. As a result, empty strings were not added to the sequence. It was no longer possible to obtain static strings with args[::2]; instead, instance checks or structural pattern matching had to be used to distinguish between strings and interpolations. This approach was rejected as offering less future opportunity for performance optimization.
  • args was typed as a Sequence[tuple[str, Interpolation | None]]. Each static string was paired with is neighboring interpolation. The final string part had no corresponding interpolation. This was rejected as being overly complex.

Mechanism to Describe the “Kind” of Template

If t-strings prove popular, it may be useful to have a way to describe the “kind” of content found in a template string: “sql”, “html”, “css”, etc. This could enable powerful new features in tools such as linters, formatters, type checkers, and IDEs. (Imagine, for example, black formatting HTML in t-strings, or mypy checking whether a given attribute is valid for an HTML tag.) While exciting, this PEP does not propose any specific mechanism. It is our hope that, over time, the community will develop conventions for this purpose.

Binary Template Strings

The combination of t-strings and bytes (tb) is considered out of scope for this PEP. However, unlike f-strings, there is no fundamental reason why t-strings and bytes cannot be combined. Support could be considered in a future PEP.

Acknowledgements

Thanks to Ryan Morshead for contributions during development of the ideas leading to template strings. Special mention also to Dropbox’s pyxl for tackling similar ideas years ago. Andrea Giammarchi provided thoughtful feedback on the early drafts of this PEP. Finally, thanks to Joachim Viide for his pioneering work on the tagged library. Tagged was not just the precursor to template strings, but the place where the whole effort started via a GitHub issue comment!


Source: https://github.com/python/peps/blob/main/peps/pep-0750.rst

Last modified: 2025-02-01 07:28:42 GMT