Following system colour scheme Selected dark colour scheme Selected light colour scheme

Python Enhancement Proposals

PEP 767 – Annotating Read-Only Attributes

Author:
Eneg <eneg at discuss.python.org>
Sponsor:
Carl Meyer <carl at oddbird.net>
Discussions-To:
Discourse thread
Status:
Draft
Type:
Standards Track
Topic:
Typing
Created:
18-Nov-2024
Python-Version:
3.14
Post-History:
09-Oct-2024

Table of Contents

Abstract

PEP 705 introduced the typing.ReadOnly type qualifier to allow defining read-only typing.TypedDict items.

This PEP proposes using ReadOnly in annotations of class and protocol attributes, as a single concise way to mark them read-only.

Akin to PEP 705, it makes no changes to setting attributes at runtime. Correct usage of read-only attributes is intended to be enforced only by static type checkers.

Motivation

The Python type system lacks a single concise way to mark an attribute read-only. This feature is present in other statically and gradually typed languages (such as C# or TypeScript), and is useful for removing the ability to reassign or delete an attribute at a type checker level, as well as defining a broad interface for structural subtyping.

Classes

Today, there are three major ways of achieving read-only attributes, honored by type checkers:

  • annotating the attribute with typing.Final:
    class Foo:
        number: Final[int]
    
        def __init__(self, number: int) -> None:
            self.number = number
    
    
    class Bar:
        def __init__(self, number: int) -> None:
            self.number: Final = number
    
    • Supported by dataclasses (and type checkers since typing#1669).
    • Overriding number is not possible - the specification of Final imposes that the name cannot be overridden in subclasses.
  • read-only proxy via @property:
    class Foo:
        _number: int
    
        def __init__(self, number: int) -> None:
            self._number = number
    
        @property
        def number(self) -> int:
            return self._number
    
    • Overriding number is possible. Type checkers disagree about the specific rules. [1]
    • Read-only at runtime. [2]
    • Requires extra boilerplate.
    • Supported by dataclasses, but does not compose well - the synthesized __init__ and __repr__ will use _number as the parameter/attribute name.
  • using a “freezing” mechanism, such as dataclasses.dataclass() or typing.NamedTuple:
    @dataclass(frozen=True)
    class Foo:
        number: int  # implicitly read-only
    
    
    class Bar(NamedTuple):
        number: int  # implicitly read-only
    
    • Overriding number is possible in the @dataclass case.
    • Read-only at runtime. [2]
    • No per-attribute control - these mechanisms apply to the whole class.
    • Frozen dataclasses incur some runtime overhead.
    • NamedTuple is still a tuple. Most classes do not need to inherit indexing, iteration, or concatenation.

Protocols

A read-only attribute name: T on a Protocol in principle defines two requirements:

  1. hasattr(obj, "name")
  2. isinstance(obj.name, T)

Those requirements are satisfiable at runtime by all of the following:

  • an object with an attribute name: T,
  • a class with a class variable name: ClassVar[T],
  • an instance of the class above,
  • an object with a @property def name(self) -> T,
  • an object with a custom descriptor, such as functools.cached_property().

The current typing spec allows creation of such protocol members using (abstract) properties:

class HasName(Protocol):
    @property
    def name(self) -> T: ...

This syntax has several drawbacks:

  • It is somewhat verbose.
  • It is not obvious that the quality conveyed here is the read-only character of a property.
  • It is not composable with type qualifiers.
  • Not all type checkers agree [3] that all of the above five objects are assignable to this structural type.

Rationale

These problems can be resolved by an attribute-level type qualifier. ReadOnly has been chosen for this role, as its name conveys the intent well, and the newly proposed changes complement its semantics defined in PEP 705.

A class with a read-only instance attribute can now be defined as:

from typing import ReadOnly


class Member:
    def __init__(self, id: int) -> None:
        self.id: ReadOnly[int] = id

…and the protocol described in Protocols is now just:

from typing import Protocol, ReadOnly


class HasName(Protocol):
    name: ReadOnly[str]


def greet(obj: HasName, /) -> str:
    return f"Hello, {obj.name}!"
  • A subclass of Member can redefine .id as a writable attribute or a descriptor. It can also narrow the type.
  • The HasName protocol has a more succinct definition, and is agnostic to the writability of the attribute.
  • The greet function can now accept a wide variety of compatible objects, while being explicit about no modifications being done to the input.

Specification

The typing.ReadOnly type qualifier becomes a valid annotation for attributes of classes and protocols. It can be used at class-level or within __init__ to mark individual attributes read-only:

class Book:
    id: ReadOnly[int]

    def __init__(self, id: int, name: str) -> None:
        self.id = id
        self.name: ReadOnly[str] = name

Type checkers should error on any attempt to reassign or delete an attribute annotated with ReadOnly. Type checkers should also error on any attempt to delete an attribute annotated as Final. (This is not currently specified.)

Use of ReadOnly in annotations at other sites where it currently has no meaning (such as local/global variables or function parameters) is considered out of scope for this PEP.

Akin to Final [4], ReadOnly does not influence how type checkers perceive the mutability of the assigned object. Immutable ABCs and containers may be used in combination with ReadOnly to forbid mutation of such values at a type checker level:

from collections import abc
from dataclasses import dataclass
from typing import Protocol, ReadOnly


@dataclass
class Game:
    name: str


class HasGames[T: abc.Collection[Game]](Protocol):
    games: ReadOnly[T]


def add_games(shelf: HasGames[list[Game]]) -> None:
    shelf.games.append(Game("Half-Life"))  # ok: list is mutable
    shelf.games[-1].name = "Black Mesa"    # ok: "name" is not read-only
    shelf.games = []                       # error: "games" is read-only
    del shelf.games                        # error: "games" is read-only and cannot be deleted


def read_games(shelf: HasGames[abc.Sequence[Game]]) -> None:
    shelf.games.append(...)             # error: "Sequence" has no attribute "append"
    shelf.games[0].name = "Blue Shift"  # ok: "name" is not read-only
    shelf.games = []                    # error: "games" is read-only

All instance attributes of frozen dataclasses and NamedTuple should be implied to be read-only. Type checkers may inform that annotating such attributes with ReadOnly is redundant, but it should not be seen as an error:

from dataclasses import dataclass
from typing import NewType, ReadOnly


@dataclass(frozen=True)
class Point:
    x: int            # implicit read-only
    y: ReadOnly[int]  # ok, redundant


uint = NewType("uint", int)


@dataclass(frozen=True)
class UnsignedPoint(Point):
    x: ReadOnly[uint]  # ok, redundant; narrower type
    y: Final[uint]     # not redundant, Final imposes extra restrictions; narrower type

Initialization

Assignment to a read-only attribute can only occur in the class declaring the attribute. There is no restriction to how many times the attribute can be assigned to. The assignment must be allowed in the following contexts:

  • In __init__, on the instance received as the first parameter (likely, self).
  • In __new__, on instances of the declaring class created via a call to a super-class’ __new__ method.
  • At declaration in the body of the class.

Additionally, a type checker may choose to allow the assignment:

  • In __new__, on instances of the declaring class, without regard to the origin of the instance. (This choice trades soundness, as the instance may already be initialized, for the simplicity of implementation.)
  • In @classmethods, on instances of the declaring class created via a call to the class’ or super-class’ __new__ method.

Note that a child class cannot assign to any read-only attributes of a parent class in any of the aforementioned contexts, unless the attribute is redeclared.

from collections import abc
from typing import ReadOnly


class Band:
    name: str
    songs: ReadOnly[list[str]]

    def __init__(self, name: str, songs: abc.Iterable[str] | None = None) -> None:
        self.name = name
        self.songs = []

        if songs is not None:
            self.songs = list(songs)  # multiple assignments are fine

    def clear(self) -> None:
        # error: assignment to read-only "songs" outside initialization
        self.songs = []


band = Band(name="Bôa", songs=["Duvet"])
band.name = "Python"           # ok: "name" is not read-only
band.songs = []                # error: "songs" is read-only
band.songs.append("Twilight")  # ok: list is mutable


class SubBand(Band):
    def __init__(self) -> None:
        self.songs = []  # error: cannot assign to a read-only attribute of a base class
# a simplified immutable Fraction class
class Fraction:
    numerator: ReadOnly[int]
    denominator: ReadOnly[int]

    def __new__(
        cls,
        numerator: str | int | float | Decimal | Rational = 0,
        denominator: int | Rational | None = None
    ) -> Self:
        self = super().__new__(cls)

        if denominator is None:
            if type(numerator) is int:
                self.numerator = numerator
                self.denominator = 1
                return self

            elif isinstance(numerator, Rational): ...

        else: ...

    @classmethod
    def from_float(cls, f: float, /) -> Self:
        self = super().__new__(cls)
        self.numerator, self.denominator = f.as_integer_ratio()
        return self

When a class-level declaration has an initializing value, it can serve as a flyweight default for instances:

class Patient:
    number: ReadOnly[int] = 0

    def __init__(self, number: int | None = None) -> None:
        if number is not None:
            self.number = number

Note

This feature conflicts with __slots__. An attribute with a class-level value cannot be included in slots, effectively making it a class variable.

Type checkers may choose to warn on read-only attributes which could be left uninitialized after an instance is created (except in stubs, protocols or ABCs):

class Patient:
    id: ReadOnly[int]    # error: "id" is not initialized on all code paths
    name: ReadOnly[str]  # error: "name" is never initialized

    def __init__(self) -> None:
        if random.random() > 0.5:
            self.id = 123


class HasName(Protocol):
    name: ReadOnly[str]  # ok

Subtyping

Read-only attributes are covariant. This has a few subtyping implications. Borrowing from PEP 705:

  • Read-only attributes can be redeclared as writable attributes, descriptors or class variables:
    @dataclass
    class HasTitle:
        title: ReadOnly[str]
    
    
    @dataclass
    class Game(HasTitle):
        title: str
        year: int
    
    
    game = Game(title="DOOM", year=1993)
    game.year = 1994
    game.title = "DOOM II"  # ok: attribute is not read-only
    
    
    class TitleProxy(HasTitle):
        @functools.cached_property
        def title(self) -> str: ...
    
    
    class SharedTitle(HasTitle):
        title: ClassVar[str] = "Still Grey"
    
  • If a read-only attribute is not redeclared, it remains read-only:
    class Game(HasTitle):
        year: int
    
        def __init__(self, title: str, year: int) -> None:
            super().__init__(title)
            self.title = title  # error: cannot assign to a read-only attribute of base class
            self.year = year
    
    
    game = Game(title="Robot Wants Kitty", year=2010)
    game.title = "Robot Wants Puppy"  # error: "title" is read-only
    
  • Subtypes can narrow the type of read-only attributes:
    class GameCollection(Protocol):
        games: ReadOnly[abc.Collection[Game]]
    
    
    @dataclass
    class GameSeries(GameCollection):
        name: str
        games: ReadOnly[list[Game]]  # ok: list[Game] is assignable to Collection[Game]
    
  • Nominal subclasses of protocols and ABCs should redeclare read-only attributes in order to implement them, unless the base class initializes them in some way:
    class MyBase(abc.ABC):
        foo: ReadOnly[int]
        bar: ReadOnly[str] = "abc"
        baz: ReadOnly[float]
    
        def __init__(self, baz: float) -> None:
            self.baz = baz
    
        @abstractmethod
        def pprint(self) -> None: ...
    
    
    @final
    class MySubclass(MyBase):
        # error: MySubclass does not override "foo"
    
        def pprint(self) -> None:
            print(self.foo, self.bar, self.baz)
    
  • In a protocol attribute declaration, name: ReadOnly[T] indicates that a structural subtype must support .name access, and the returned value is assignable to T:
    class HasName(Protocol):
        name: ReadOnly[str]
    
    
    class NamedAttr:
        name: str
    
    class NamedProp:
        @property
        def name(self) -> str: ...
    
    class NamedClassVar:
        name: ClassVar[str]
    
    class NamedDescriptor:
        @cached_property
        def name(self) -> str: ...
    
    # all of the following are ok
    has_name: HasName
    has_name = NamedAttr()
    has_name = NamedProp()
    has_name = NamedClassVar
    has_name = NamedClassVar()
    has_name = NamedDescriptor()
    

Interaction with Other Type Qualifiers

ReadOnly can be used with ClassVar and Annotated in any nesting order:

class Foo:
    foo: ClassVar[ReadOnly[str]] = "foo"
    bar: Annotated[ReadOnly[int], Gt(0)]
class Foo:
    foo: ReadOnly[ClassVar[str]] = "foo"
    bar: ReadOnly[Annotated[int, Gt(0)]]

This is consistent with the interaction of ReadOnly and typing.TypedDict defined in PEP 705.

An attribute annotated as both ReadOnly and ClassVar can only be assigned to at declaration in the class body.

An attribute cannot be annotated as both ReadOnly and Final, as the two qualifiers differ in semantics, and Final is generally more restrictive. Final remains allowed as an annotation of attributes that are only implied to be read-only. It can be also used to redeclare a ReadOnly attribute of a base class.

Backwards Compatibility

This PEP introduces new contexts where ReadOnly is valid. Programs inspecting those places will have to change to support it. This is expected to mainly affect type checkers.

However, caution is advised while using the backported typing_extensions.ReadOnly in older versions of Python. Mechanisms inspecting annotations may behave incorrectly when encountering ReadOnly; in particular, the @dataclass decorator which looks for ClassVar may mistakenly treat ReadOnly[ClassVar[...]] as an instance attribute.

To avoid issues with introspection, use ClassVar[ReadOnly[...]] instead of ReadOnly[ClassVar[...]].

Security Implications

There are no known security consequences arising from this PEP.

How to Teach This

Suggested changes to the typing module documentation, following the footsteps of PEP 705:

  • Add this PEP to the others listed.
  • Link typing.ReadOnly to this PEP.
  • Update the description of typing.ReadOnly:
    A special typing construct to mark an attribute of a class or an item of a TypedDict as read-only.
  • Add a standalone entry for ReadOnly under the type qualifiers section:
    The ReadOnly type qualifier in class attribute annotations indicates that the attribute of the class may be read, but not reassigned or deleted. For usage in TypedDict, see ReadOnly.

Rejected Ideas

Clarifying Interaction of @property and Protocols

The Protocols section mentions an inconsistency between type checkers in the interpretation of properties in protocols. The problem could be fixed by amending the typing specification, clarifying what implements the read-only quality of such properties.

This PEP makes ReadOnly a better alternative for defining read-only attributes in protocols, superseding the use of properties for this purpose.

Assignment Only in __init__ and Class Body

An earlier version of this PEP proposed that read-only attributes could only be assigned to in __init__ and the class’ body. A later discussion revealed that this restriction would severely limit the usability of ReadOnly within immutable classes, which typically do not define __init__.

fractions.Fraction is one example of an immutable class, where the initialization of its attributes happens within __new__ and classmethods. However, unlike in __init__, the assignment in __new__ and classmethods is potentially unsound, as the instance they work on can be sourced from an arbitrary place, including an already finalized instance.

We find it imperative that this type checking feature is useful to the foremost use site of read-only attributes - immutable classes. Thus, the PEP has changed since to allow assignment in __new__ and classmethods under a set of rules described in the Initialization section.

Open Issues

Extending Initialization

Mechanisms such as dataclasses.__post_init__() or attrs’ initialization hooks augment object creation by providing a set of special hooks which are called during initialization.

The current initialization rules defined in this PEP disallow assignment to read-only attributes in such methods. It is unclear whether the rules could be satisfyingly shaped in a way that is inclusive of those 3rd party hooks, while upkeeping the invariants associated with the read-only-ness of those attributes.

The Python type system has a long and detailed specification regarding the behavior of __new__ and __init__. It is rather unfeasible to expect the same level of detail from 3rd party hooks.

A potential solution would involve type checkers providing configuration in this regard, requiring end users to manually specify a set of methods they wish to allow initialization in. This however could easily result in users mistakenly or purposefully breaking the aforementioned invariants. It is also a fairly big ask for a relatively niche feature.

ReadOnly[ClassVar[...]] and __init_subclass__

Should read-only class variables be assignable to within the declaring class’ __init_subclass__?

class URI:
    protocol: ReadOnly[ClassVar[str]] = ""

    def __init_subclass__(cls, protocol: str = "") -> None:
        cls.foo = protocol

class File(URI, protocol="file"): ...

Footnotes


Source: https://github.com/python/peps/blob/main/peps/pep-0767.rst

Last modified: 2024-12-05 19:37:44 GMT