PEP 746 – Type checking Annotated metadata
- Author:
- Adrian Garcia Badaracco <adrian at adriangb.com>
- Sponsor:
- Jelle Zijlstra <jelle.zijlstra at gmail.com>
- Discussions-To:
- Discourse thread
- Status:
- Draft
- Type:
- Standards Track
- Topic:
- Typing
- Created:
- 20-May-2024
- Python-Version:
- 3.14
- Post-History:
- 20-May-2024
Table of Contents
Abstract
This PEP proposes a mechanism for type checking metadata that uses
the typing.Annotated
type. Metadata objects that implement
the new __supports_type__
protocol will be type checked by static
type checkers to ensure that the metadata is valid for the given type.
Motivation
PEP 593 introduced Annotated
as a way to attach runtime metadata to types.
In general, the metadata is not meant for static type checkers, but even so,
it is often useful to be able to check that the metadata makes sense for the given
type.
Take the first example in PEP 593, which uses Annotated
to attach
serialization information to a field:
class Student(struct2.Packed):
name: Annotated[str, struct2.ctype("<10s")]
Here, the struct2.ctype("<10s")
metadata is meant to be used by a serialization
library to serialize the field. Such libraries can only serialize a subset of types:
it would not make sense to write, for example, Annotated[list[str], struct2.ctype("<10s")]
.
Yet the type system provides no way to enforce this. The metadata are completely
ignored by type checkers.
This use case comes up in libraries like pydantic, which use
Annotated
to attach validation and conversion information to fields.
Specification
This PEP introduces a protocol that can be used by static and runtime type checkers to validate
the consistency between Annotated
metadata and a given type.
Objects that implement this protocol have a method named __supports_type__
that takes a single positional argument and returns bool
:
class Int64:
def __supports_type__(self, obj: int) -> bool:
return isinstance(obj, int)
The protocol being introduced would be defined as follows if it were to be defined in code form:
from typing import Protocol
class SupportsType[T](Protocol):
def __supports_type__(self, obj: T, /) -> bool:
...
When a static type checker encounters a type expression of the form Annotated[T, M1, M2, ...]
,
it should enforce that for each metadata element in M1, M2, ...
, one of the following is true:
- The metadata element evaluates to an object that does not have a
__supports_type__
attribute; or - The metadata element evaluates to an object
M
that implements theSupportsType
protocol; and, withT
instantiated to a valuev
, a call toM.__supports_type__(v)
type checks without errors; and that call does not evaluate toLiteral[False]
.
The body of the __supports_type__
method is not used to check the validity of the metadata
and static type checkers can ignore it. However, tools that use the annotation at
runtime may call the method to check that a particular value is valid.
For example, to support a generic Gt
metadata, one might write:
from typing import Protocol
class SupportsGt[T](Protocol):
def __gt__(self, __other: T) -> bool:
...
class Gt[T]:
def __init__(self, value: T) -> None:
self.value = value
def __supports_type__(self, obj: SupportsGt[T], /) -> bool:
return obj > self.value
x1: Annotated[int, Gt(0)] = 1 # OK
x2: Annotated[str, Gt(0)] = 0 # type checker error: str is not assignable to SupportsGt[int]
x3: Annotated[int, Gt(1)] = 0 # OK for static type checkers; runtime type checkers may flag this
Implementations may be generic and may use overloads that return Literal[True]
or Literal[False]
to indicate if the metadata is valid for the given type.
Implementations may raise a NotImplementedError if they cannot determine if the metadata is valid for the given type.
Tools calling __supports_type__
at runtime should catch this exception and treat it as if __supports_type__
was not present; they should not take this as an indication that the metadata is invalid for the type.
Tools that use the metadata at runtime may choose to ignore the implementation of __supports_type__
; this PEP does not
specify how the method should be used at runtime, only that it may be available for use.
Backwards Compatibility
Metadata that does not implement the protocol will be considered valid for all types, so no breaking changes are introduced for existing code. The new checks only apply to metadata objects that explicitly implement the protocol specified by this PEP.
Security Implications
None.
How to Teach This
This protocol is intended mostly for libraries that provide Annotated
metadata;
end users of those libraries are unlikely to need to implement the protocol themselves.
The protocol should be mentioned in the documentation for typing.Annotated
and
in the typing specification.
Reference Implementation
None yet.
Rejected ideas
Introducing a type variable instead of a generic class
We considered using a special type variable, AnnotatedT = TypeVar("AnnotatedT")
,
to represent the type T
of the inner type in Annotated
; metadata would be
type checked against this type variable. However, this would require using the old
type variable syntax (before PEP 695), which is now a discouraged feature.
In addition, this would use type variables in an unusual way that does not fit well
with the rest of the type system.
Introducing a new type to typing.py
that all metadata objects should subclass
A previous version of this PEP suggested adding a new generic base class, TypedMetadata[U]
,
that metadata objects would subclass. If a metadata object is a subclass of TypedMetadata[U]
,
then type checkers would check that the annotation’s base type is assignable to U
.
However, this mechanism does not integrate as well with the rest of the language; Python
does not generally use marker base classes. In addition, it provides less flexibility than
the current proposal: it would not allow overloads, and it would require metadata objects
to add a new base class, which may make their runtime implementation more complex.
Acknowledgments
We thank Eric Traut for suggesting the idea of using a protocol.
Copyright
This document has been placed in the public domain.
Source: https://github.com/python/peps/blob/main/peps/pep-0746.rst
Last modified: 2024-06-12 06:19:05 GMT