Abstract

This PEP proposes to add a feature that automatically removes indentation from multiline string literals.

Dedented multiline strings use a new prefix “d” (shorthand for “dedent”) before the opening quote of a multiline string literal.

Example (spaces are visualized as .):

def hello_paragraph() -> str:
....return d"""
........<p>
..........Hello, World!
........</p>
...."""

Unlike textwrap.dedent(), indentation before closing quotes is also considered when determining the amount of indentation to be removed. Therefore, the string returned in the example above consists of the following three lines.

• "....<p>\n"
• "......Hello, World!\n"
• "....</p>\n"

Motivation

When writing multiline string literals within deeply indented Python code, users are faced with the following choices:

• Write the contents of the string without indentation.
• Use multiple single-line string literals concatenated together instead of a multiline string literal.
• Use textwrap.dedent() to remove indentation.

All of these options have drawbacks in terms of code readability and maintainability.

• Writing multiline strings without indentation in deeply indented code looks awkward and tends to be avoided. In practice, many places including Python’s own test code choose other methods.
• Concatenated single-line string literals are more verbose and harder to maintain. Writing "\n" at the end of each line is tedious. It’s easy to miss the semicolons between many string concatenations.
• textwrap.dedent() is implemented in Python so it requires some runtime overhead. Moreover, it cannot be used to dedent t-strings.

This PEP aims to provide a built-in syntax for dedented multiline strings that is both easy to read and write, while also being efficient at runtime.

Rationale

The main alternative to this idea is to implement textwrap.dedent() in C and provide it as a str.dedent() method. This idea reduces the runtime overhead of textwrap.dedent(). By making it a built-in method, it also allows for compile-time dedentation when called directly on string literals.

However, this approach has several drawbacks:

• To support cases where users want to include some indentation in the string, the dedent() method would need to accept an argument specifying the amount of indentation to remove. This would be cumbersome and error-prone for users.
• When continuation lines (lines after line ends with a backslash) are used, they cannot be dedented.
• f-strings may interpolate expressions as multiline string without indent. In such case, f-string + str.dedent() cannot dedent the whole string.
• t-strings do not create str objects, so they cannot use the str.dedent() method. While adding a dedent() method to string.templatelib.Template is an option, it would lead to inconsistency since t-strings and f-strings are very similar but would have different behaviors regarding dedentation.

The str.dedent() method can still be useful for non-literal strings, so this PEP does not preclude that idea. However, for ease of use with multiline string literals, providing dedicated syntax is superior.

Specification

Add a new string literal prefix “d” for dedented multiline strings. This prefix can be combined with “f”, “t”, “r”, and “b” prefixes.

This prefix is only for multiline string literals. So it can only be used with triple quotes (""" or ''').

Opening triple quotes needs to be followed by a newline character. This newline is not included in the resulting string. The content of the d-string starts from the next line.

Indentation is leading whitespace characters (spaces and tabs) of each line.

The amount of indentation to be removed is determined by the longest common indentation of lines in the string. Lines consisting entirely of whitespace characters are ignored when determining the common indentation, except for the line containing the closing triple quotes.

Spaces and tabs are treated as different characters. For example, " hello" and "\thello" have no common indentation.

The dedentation process removes the determined indentation from every line in the string.

• Lines that are longer than or equal in length to the determined indentation must start with the determined indentation. Othrerwise, Python raises an IndentationError. The determined indentation is removed from these lines.
• Lines that are shorter than the determined indentation (including empty lines) must be a prefix of the determined indentation. Otherwise, Python raises an IndentationError. These lines become empty lines.

Unless combined with the “r” prefix, backslash escapes are processed after the dedentation process. So you cannot use \\t in indentations. And you can use line continuation (backslash at the end of line) and remove indentation from the continued line.

Examples:

# d-string must starts with a newline.
s = d""           # SyntaxError: d-string must be triple-quoted
s = d""""""       # SyntaxError: d-string must start with a newline
s = d"""Hello"""  # SyntaxError: d-string must start with a newline
s = d"""Hello
..World!
"""  # SyntaxError: d-string must start with a newline

# d-string removes the longest common indentation from each line.
# Empty lines are ignored, but closing quotes line is always considered.
s = d"""
..Hello
..World!
.."""
print(repr(s))  # 'Hello\nWorld!\n'

s = d"""
..Hello
..World!
."""
print(repr(s))  # '.Hello\n.World!\n'

s = d"""
..Hello
..World!
"""
print(repr(s))  # '..Hello\n..World!\n'

s = d"""
..Hello
.

..World!
..."""  # Longest common indentation is '..'.
print(repr(s))  # 'Hello\n\n\nWorld!\n.'

# Closing qutotes can be on the same line as the last content line.
# In this case, the string does not end with a newline.
s = d"""
..Hello
..World!"""
print(repr(s))  # 'Hello\nWorld!'

# Tabs are allowed as indentation.
# But tabs and spaces are treated as different characters.
s = d"""
--->..Hello
--->..World!
--->"""
print(repr(s))  # '..Hello\n..World!\n'

s = d"""
--->Hello
..World!
.."""  # There is no common indentation.
print(repr(s))  # '\tHello\n..World!\n..'

# Line continuation with backslash works as usual.
# But you cannot put a backslash right after the opening quotes.
s = d"""
..Hello \
..World!\
.."""
print(repr(s))  # 'Hello World!'

s = d"""\
..Hello
..World
.."""  # SyntaxError: d-string must starts with a newline.

# d-string can be combined with r-string, b-string, f-string, and t-string.
s = dr"""
..Hello\
..World!\
.."""
print(repr(s))  # 'Hello\\\nWorld!\\\n'

s = db"""
..Hello
..World!
.."""
print(repr(s))  # b'Hello\nWorld!\n'

s = df"""
....Hello, {"world".title()}!
...."""
print(repr(s))  # 'Hello,.World!\n'

s = dt"""
....Hello, {"world".title()}!
...."""
print(type(s))   # <class 'string.templatelib.Template'>
print(s.strings) # ('Hello, ', '!\n')
print(s.values)  # ('World',)

How to Teach This

The main difference between textwrap.dedent("""...""") and d-string can be explained as follows:

• textwrap.dedent() is a regular function, but d-string is part of the language syntax. d-string has no runtime overhead, and it can remove indentation from t-strings.
• When using textwrap.dedent(), you need to start with """\ to avoid including the first newline character, but with d-string, the string content starts from the line after d""", so no backslash is needed.

  import textwrap
  
  s1 = textwrap.dedent("""\
      Hello
      World!
      """)
  s2 = d"""
      Hello
      World!
      """
  assert s1 == s2
  

• textwrap.dedent() ignores all blank lines when determining the common indentation, but d-string also considers the indentation of the closing quotes. This allows d-string to preserve some indentation in the result when needed.

  import textwrap
  
  s1 = textwrap.dedent("""\
        Hello
        World!
      """)
  s2 = d"""
        Hello
        World!
      """
  assert s1 != s2
  assert s1 == 'Hello\nWorld!\n'
  assert s2 == '  Hello\n  World!\n'
  

• Since d-string removes indentation before processing escape sequences, when using line continuation (backslash at the end of a line), the next line can also be dedented.

  import textwrap
  
  s1 = textwrap.dedent("""\
      Lorem ipsum dolor sit amet, consectetur adipiscing elit, \
  sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.
      Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris \
  nisi ut aliquip ex ea commodo consequat.
      """)
  s2 = d"""
      Lorem ipsum dolor sit amet, consectetur adipiscing elit, \
      sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.
      Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris \
      nisi ut aliquip ex ea commodo consequat.
      """
  assert s1 == s2
  

Other Languages having Similar Features

Java 15 introduced a feature called text blocks. Since Java had not used triple qutes before, they introduced triple quotes for multiline string literals with automatic indent removal.

C# 11 also introduced a similar feature called raw string literals.

Julia and Swift also support triple-quoted string literals that automatically remove indentation.

PHP 7.3 introduced Flexible Heredoc and Nowdoc Syntaxes Although it uses closing marker (e.g. <<<END ... END) instead of triple quote, it removes indent from text too.

Ruby also has “squiggly” heredoc that removes indent from lines in heredoc.

Java, Julia, and Ruby uses the least-indented line to determine the amount of indentation to be removed. Swift, C#, and PHP uses the indentation of the closing triple quotes or closing marker.

Reference Implementation

A CPython implementation of PEP 822 is available at methane/cpython#108.

Rejected Ideas

s = d"""
    Hello
    World!
  """  # "  Hello\n  World!" (no trailing newline)

s = d"""
    Hello
    World!

  """  # "  Hello\n  World!\n" (has a trailing newline)

s = d"""
    Hello
    World!"""
assert s == "Hello\nWorld!"

s = d"""
    Hello
    World!\
  """
assert s == "  Hello\n  World!"

s = dr"""
    Hello
    World!
  """.rstrip()
assert s == "  Hello\n  World!"

Copyright

This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.