Why can’t type[] be used?

One might think you could define the example functions above to take a type[C] - which is syntax that already exists - rather than a TypeExpr[T]. However if you were to do that then certain type expressions like str | None - which are not class objects and therefore not types at runtime - would be rejected:

# NOTE: Uses a type[C] parameter rather than a TypeExpr[T]
def trycast_type[C](typ: type[C], value: object) -> T | None: ...

trycast_type(str, 'hi')  # ok; str is a type
trycast_type(Optional[str], 'hi')  # ERROR; Optional[str] is not a type
trycast_type(str | int, 'hi')  # ERROR; (str | int) is not a type
trycast_type(MyTypedDict, dict(value='hi'))  # questionable; accepted by mypy 1.9.0

To solve that problem, type[] could be widened to include the additional values allowed by TypeExpr. However doing so would lose type[]’s current ability to spell a class object which always supports instantiation and isinstance checks, unlike arbitrary type expression objects. Therefore TypeExpr is proposed as new notation instead.

For a longer explanation of why we don’t just widen type[T] to accept all type expressions, see Widen type[C] to support all type expressions.

Common kinds of functions that would benefit from TypeExpr

A survey of various Python libraries revealed a few kinds of commonly defined functions which would benefit from TypeExpr[]:

• Assignability checkers:
  • Returns whether a value is assignable to a type expression. If so then also narrows the type of the value to match the type expression.
  • Pattern 1: def isassignable[T](value: object, typx: TypeExpr[T]) -> TypeIs[T]
  • Pattern 2: def ismatch[T](value: object, typx: TypeExpr[T]) -> TypeGuard[T]
  • Examples: beartype.is_bearable, trycast.isassignable, typeguard.check_type, xdsl.isa

• Converters:
  • If a value is assignable to (or coercible to) a type expression, a converter returns the value narrowed to (or coerced to) that type expression. Otherwise, it raises an exception.
  • Pattern 1: def convert[T](value: object, typx: TypeExpr[T]) -> T
    • Examples: cattrs.BaseConverter.structure, trycast.checkcast, typedload.load
  • Pattern 2:

    class Converter[T]:
        def __init__(self, typx: TypeExpr[T]) -> None: ...
        def convert(self, value: object) -> T: ...
    

    • Examples: pydantic.TypeAdapter(T).validate_python, mashumaro.JSONDecoder(T).decode

• Typed field definitions:
  • Pattern:

    class Field[T]:
        value_type: TypeExpr[T]
    

  • Examples: attrs.make_class, dataclasses.make_dataclass [3], openapify

The survey also identified some introspection functions that take annotation expressions as input using plain objects which would not gain functionality by marking those inputs as TypeExpr[]:

• General introspection operations:
  • Pattern: def get_annotation_info(maybe_annx: object) -> object
  • Examples: typing.{get_origin, get_args}, typing_inspect.{is_*_type, get_origin, get_parameters}

Using TypeExprs

A TypeExpr is a new kind of type expression, usable in any context where a type expression is valid, as a function parameter type, a return type, or a variable type:

def is_union_type(typx: TypeExpr) -> bool: ...  # parameter type

def union_of[S, T](s: TypeExpr[S], t: TypeExpr[T]) \
    -> TypeExpr[S | T]: ...  # return type

STR_TYPE: TypeExpr = str  # variable type
assert_type(STR_TYPE, TypeExpr[str])

Note however that an unannotated variable assigned a type expression literal will not be inferred to be of TypeExpr type by type checkers because PEP 484 reserves that syntax for defining type aliases:

• No:

  STR_TYPE = str  # OOPS; treated as a type alias!
  

If you want a type checker to recognize a type expression literal in a bare assignment you’ll need to explicitly declare the assignment-target as having TypeExpr type:

• Yes:

  STR_TYPE: TypeExpr = str
  

• Yes:

  STR_TYPE: TypeExpr
  STR_TYPE = str
  

• Okay, but discouraged:

  STR_TYPE = str  # type: TypeExpr  # the type comment is significant
  

TypeExpr values can be passed around and assigned just like normal values:

def swap1[S, T](t1: TypeExpr[S], t2: TypeExpr[T]) -> tuple[TypeExpr[T], TypeExpr[S]]:
    t1_new: TypeExpr[T] = t2  # assigns a TypeExpr value to a new annotated variable
    t2_new: TypeExpr[S] = t1
    return (t1_new, t2_new)

def swap2[S, T](t1: TypeExpr[S], t2: TypeExpr[T]) -> tuple[TypeExpr[T], TypeExpr[S]]:
    t1_new = t2  # assigns a TypeExpr value to a new unannotated variable
    t2_new = t1
    assert_type(t1_new, TypeExpr[T])
    assert_type(t2_new, TypeExpr[S])
    return (t1_new, t2_new)

# NOTE: A more straightforward implementation would use isinstance()
def ensure_int(value: object) -> None:
    value_type: TypeExpr = type(value)  # assigns a type (a subtype of TypeExpr)
    assert value_type == int

TypeExpr Values

A variable of type TypeExpr[T] where T is a type, can hold any type expression object - the result of evaluating a type expression at runtime - which is a subtype of T.

Incomplete expressions like a bare Optional or Union which do not spell a type are not TypeExpr values.

TypeExpr[...] is itself a TypeExpr value:

OPTIONAL_INT_TYPE: TypeExpr = TypeExpr[int | None]  # OK
assert isassignable(Optional[int], OPTIONAL_INT_TYPE)

TypeExpr[] values include all type expressions including some non-universal type expressions which are not valid in all annotation contexts. In particular:

• Self (valid only in some contexts)
• TypeGuard[...] (valid only in some contexts)
• TypeIs[...] (valid only in some contexts)

NONE = TypeExpr(None)
INT1 = TypeExpr('int')  # stringified type expression
INT2 = TypeExpr(int)

STRS_TYPE_NAME: Literal['str', 'list[str]'] = 'str'
STRS_TYPE: TypeExpr = STRS_TYPE_NAME  # ERROR: Literal[] value is not a TypeExpr

>>> IntTree = list[typing.Union[int, 'IntTree']]
>>> IntTree
list[typing.Union[int, ForwardRef('IntTree')]]

INT_TREE: TypeExpr = ForwardRef('IntTree')  # ERROR: Runtime-only form

INT_TREE = cast(TypeExpr, ForwardRef('IntTree'))  # OK

Subtyping

Whether a TypeExpr value can be assigned from one variable to another is determined by the following rules:

TypeExpr[] is covariant in its argument type, just like type[]:

• TypeExpr[T1] is a subtype of TypeExpr[T2] iff T1 is a subtype of T2.
• type[C1] is a subtype of TypeExpr[C2] iff C1 is a subtype of C2.

A plain type can be assigned to a plain TypeExpr but not the other way around:

• type[Any] is assignable to TypeExpr[Any]. (But not the other way around.)

TypeExpr[] is a kind of object, just like type[]:

• TypeExpr[T] for any T is a subtype of object.

TypeExpr[T], where T is a type variable, is assumed to have all the attributes and methods of object and is not callable.

Interactions with isinstance() and issubclass()

The TypeExpr special form cannot be used as the type argument to isinstance:

>>> isinstance(str, TypeExpr)
TypeError: typing.TypeExpr cannot be used with isinstance()

>>> isinstance(str, TypeExpr[str])
TypeError: isinstance() argument 2 cannot be a parameterized generic

The TypeExpr special form cannot be used as any argument to issubclass:

>>> issubclass(TypeExpr, object)
TypeError: issubclass() arg 1 must be a class

>>> issubclass(object, TypeExpr)
TypeError: typing.TypeExpr cannot be used with issubclass()

Affected signatures in the standard library

Introspecting TypeExpr Values

A TypeExpr is very similar to an object at runtime, with no additional attributes or methods defined.

You can use existing introspection functions like typing.get_origin and typing.get_args to extract the components of a type expression that looks like Origin[Arg1, Arg2, ..., ArgN]:

import typing

def strip_annotated_metadata(typx: TypeExpr[T]) -> TypeExpr[T]:
    if typing.get_origin(typx) is typing.Annotated:
        typx = cast(TypeExpr[T], typing.get_args(typx)[0])
    return typx

You can also use isinstance and is to distinguish one kind of type expression from another:

import types
import typing

def split_union(typx: TypeExpr) -> tuple[TypeExpr, ...]:
    if isinstance(typx, types.UnionType):  # X | Y
        return cast(tuple[TypeExpr, ...], typing.get_args(typx))
    if typing.get_origin(typx) is typing.Union:  # Union[X, Y]
        return cast(tuple[TypeExpr, ...], typing.get_args(typx))
    if typx in (typing.Never, typing.NoReturn,):
        return ()
    return (typx,)

Combining with a type variable

TypeExpr[] can be parameterized by a type variable that is used elsewhere within the same function definition:

def as_instance[T](typx: TypeExpr[T]) -> T | None:
    return typx() if isinstance(typx, type) else None

Combining with type[]

Both TypeExpr[] and type[] can be parameterized by the same type variable within the same function definition:

def as_type[T](typx: TypeExpr[T]) -> type[T] | None:
    return typx if isinstance(typx, type) else None

Combining with TypeIs[] and TypeGuard[]

A type variable parameterizing a TypeExpr[] can also be used by a TypeIs[] within the same function definition:

def isassignable[T](value: object, typx: TypeExpr[T]) -> TypeIs[T]: ...

count: int | str = ...
if isassignable(count, int):
    assert_type(count, int)
else:
    assert_type(count, str)

or by a TypeGuard[] within the same function definition:

def isdefault[T](value: object, typx: TypeExpr[T]) -> TypeGuard[T]:
    return (value == typx()) if isinstance(typx, type) else False

value: int | str = ''
if isdefault(value, int):
    assert_type(value, int)
    assert 0 == value
elif isdefault(value, str):
    assert_type(value, str)
    assert '' == value
else:
    assert_type(value, int | str)

Challenges When Accepting All TypeExprs

A function that takes an arbitrary TypeExpr as input must support a large variety of possible type expressions and is not easy to write. Some challenges faced by such a function include:

• An ever-increasing number of typing special forms are introduced with each new Python version which must be recognized, with special handling required for each one.
• Stringified type annotations [5] (like 'list[str]') must be parsed (to something like typing.List[str]) to be introspected.
  • In practice it is extremely difficult for stringified type annotations to be handled reliably at runtime, so runtime type checkers may opt to not support them at all.
• Resolving string-based forward references inside type expressions to actual values must typically be done using eval(), which is difficult/impossible to use in a safe way.
• Recursive types like IntTree = list[typing.Union[int, 'IntTree']] are not possible to fully resolve.
• Supporting user-defined generic types (like Django’s QuerySet[User]) requires user-defined functions to recognize/parse, which a runtime type checker should provide a registration API for.

Widen type[C] to support all type expressions

type was designed to only be used to describe class objects. A class object can always be used as the second argument of isinstance() and can usually be instantiated by calling it.

TypeExpr on the other hand is typically introspected by the user in some way, is not necessarily directly instantiable, and is not necessarily directly usable in a regular isinstance() check.

It would be possible to widen type to include the additional values allowed by TypeExpr but it would reduce clarity about the user’s intentions when working with a type. Different concepts and usage patterns; different spellings.

Accept arbitrary annotation expressions

Certain typing special forms can be used in some but not all annotation contexts:

For example Final[] can be used as a variable type but not as a parameter type or a return type:

some_const: Final[str] = ...  # OK

def foo(not_reassignable: Final[object]): ...  # ERROR: Final[] not allowed here

def nonsense() -> Final[object]: ...  # ERROR: Final[] not meaningful here

TypeExpr[T] does not allow matching such annotation expressions because it is not clear what it would mean for such an expression to parameterized by a type variable in position T:

def ismatch[T](value: object, typx: TypeExpr[T]) -> TypeGuard[T]: ...

def foo(some_arg):
    if ismatch(some_arg, Final[int]):  # ERROR: Final[int] is not a TypeExpr
        reveal_type(some_arg)  # ? NOT Final[int], because invalid for a parameter

Functions that wish to operate on all kinds of annotation expressions, including those that are not TypeExprs, can continue to accept such inputs as object parameters, as they must do so today.

Accept only universal type expressions

Earlier drafts of this PEP only allowed TypeExpr[] to match the subset of type expressions which are valid in all contexts, excluding non-universal type expressions. However doing that would effectively create a new subset of annotation expressions that Python typing users would have to understand, on top of all the existing distinctions between “class objects”, “type expressions”, and “annotation expressions”.

To avoid introducing yet another concept that everyone has to learn, this proposal just rounds TypeExpr[] to exactly match the existing definition of a “type expression”.

Support pattern matching on type expressions

It was asserted that some functions may wish to pattern match on the interior of type expressions in their signatures.

One use case is to allow a function to explicitly enumerate all the specific kinds of type expressions it supports as input. Consider the following possible pattern matching syntax:

@overload
def checkcast(typx: TypeExpr[AT=Annotated[T, *Anns]], value: str) -> T: ...
@overload
def checkcast(typx: TypeExpr[UT=Union[*Ts]], value: str) -> Union[*Ts]: ...
@overload
def checkcast(typx: type[C], value: str) -> C: ...
# ... (more)

All functions observed in the wild that conceptually take a TypeExpr[] generally try to support all kinds of type expressions, so it doesn’t seem valuable to enumerate a particular subset.

Additionally the above syntax isn’t precise enough to fully describe the actual input constraints for a typical function in the wild. For example many functions recognize un-stringified type expressions like list[Movie] but may not recognize type expressions with stringified subcomponents like list['Movie'].

A second use case for pattern matching on the interior of type expressions is to explicitly match an Annotated[] form to pull out the interior type argument and strip away the metadata:

def checkcast(
    typx: TypeExpr[T] | TypeExpr[AT=Annotated[T, *Anns]],
    value: object
) -> T:

However Annotated[T, metadata] is already treated equivalent to T anyway. There’s no additional value in being explicit about this behavior. The example above could be more-straightforwardly written as the equivalent:

def checkcast(typx: TypeExpr[T], value: object) -> T: