PEP 718 – Subscriptable functions
- James Hilton-Balfe <gobot1234yt at gmail.com>
- Guido van Rossum <guido at python.org>
- Discourse thread
- Standards Track
This PEP proposes making function objects subscriptable for typing purposes. Doing so gives developers explicit control over the types produced by the type checker where bi-directional inference (which allows for the types of parameters of anonymous functions to be inferred) and other methods than specialisation are insufficient.
Currently, it is not possible to infer the type parameters to generic functions in certain situations:
def make_list[T](*args: T) -> list[T]: ... reveal_type(make_list()) # type checker cannot infer a meaningful type for T
Making instances of
FunctionType subscriptable would allow for this constructor to
reveal_type(make_list[int]()) # type is list[int]
Currently you have to use an assignment to provide a precise type:
x: list[int] = make_list() reveal_type(x) # type is list[int]
but this code is unnecessarily verbose taking up multiple lines for a simple function call.
T in this example cannot currently be meaningfully inferred, so
untyped without an extra assignment:
def factory[T](func: Callable[[T], Any]) -> Foo[T]: ... reveal_type(factory(lambda x: "Hello World" * x))
If function objects were subscriptable, however, a more specific type could be given:
reveal_type(factory[int](lambda x: "Hello World" * x)) # type is Foo[int]
Currently, with unspecialised literals, it is not possible to determine a type for situations similar to:
def foo[T](x: list[T]) -> T: ... reveal_type(foo()) # type checker cannot infer T (yet again)
reveal_type(foo[int]()) # type is int
It is also useful to be able to specify in cases in which a certain type must be passed to a function beforehand:
words = ["hello", "world"] foo[int](words) # Invalid: list[str] is incompatible with list[int]
Allowing subscription makes functions and methods consistent with generic classes where they weren’t already. Whilst all of the proposed changes can be implemented using callable generic classes, syntactic sugar would be highly welcome.
Due to this, specialising the function and using it as a new factory is fine
make_int_list = make_list[int] reveal_type(make_int_list()) # type is list[int]
This proposal also opens the door to monomorphisation and reified types
Function objects in this PEP is used to refer to
MethodType you should be able to write:
class Foo: def make_list[T](self, *args: T) -> list[T]: ... Foo().make_list[int]()
and have it work similarly to a
BuiltinFunctionType, so builtin generic functions (e.g.
work like ones defined in Python. Built-in functions should behave as much like
functions implemented in Python as possible.
BuiltinMethodType is the same type as
MethodWrapperType (e.g. the type of
object().__str__) is useful for
generic magic methods.
Function objects should implement
__getitem__ to allow for subscription at runtime
and return an instance of
__origin__ set as the
__args__ as the types passed.
Type checkers should support subscripting functions and understand that the parameters passed to the function subscription should follow the same rules as a generic callable class.
__orig_class__ is an attribute set in
GenericAlias.__call__ to the
instance of the
GenericAlias that created the called class e.g.
class Foo[T]: ... assert Foo[int]().__orig_class__ == Foo[int]
__orig_class__ is unconditionally set; however, to avoid potential
erasure on any created instances, this attribute should not be set if
an instance of any function object.
The following code snippet would fail at runtime without this change as
__orig_class__ would be
bar[str] and not
def bar[U](): return Foo[int]() assert bar[str]().__orig_class__ is Foo[int]
Currently these classes are not subclassable and so there are no backwards
compatibility concerns with regards to classes already implementing
The runtime changes proposed can be found here https://github.com/Gobot1234/cpython/tree/function-subscript
Thank you to Alex Waygood and Jelle Zijlstra for their feedback on this PEP and Guido for some motivating examples.
This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.
Last modified: 2023-09-09 17:39:29 GMT