Following system colour scheme Selected dark colour scheme Selected light colour scheme

Python Enhancement Proposals

PEP 718 – Subscriptable functions

Author:
James Hilton-Balfe <gobot1234yt at gmail.com>
Sponsor:
Guido van Rossum <guido at python.org>
Discussions-To:
Discourse thread
Status:
Draft
Type:
Standards Track
Topic:
Typing
Created:
23-Jun-2023
Python-Version:
3.13
Post-History:
24-Jun-2023

Table of Contents

Abstract

This PEP proposes making function objects subscriptable for typing purposes. Doing so gives developers explicit control over the types produced by the type checker where bi-directional inference (which allows for the types of parameters of anonymous functions to be inferred) and other methods than specialisation are insufficient. It also brings functions in line with regular classes in their ability to be subscriptable.

Motivation

Unknown Types

Currently, it is not possible to infer the type parameters to generic functions in certain situations:

def make_list[T](*args: T) -> list[T]: ...
reveal_type(make_list())  # type checker cannot infer a meaningful type for T

Making instances of FunctionType subscriptable would allow for this constructor to be typed:

reveal_type(make_list[int]())  # type is list[int]

Currently you have to use an assignment to provide a precise type:

x: list[int] = make_list()
reveal_type(x)  # type is list[int]

but this code is unnecessarily verbose taking up multiple lines for a simple function call.

Similarly, T in this example cannot currently be meaningfully inferred, so x is untyped without an extra assignment:

def factory[T](func: Callable[[T], Any]) -> Foo[T]: ...

reveal_type(factory(lambda x: "Hello World" * x))

If function objects were subscriptable, however, a more specific type could be given:

reveal_type(factory[int](lambda x: "Hello World" * x))  # type is Foo[int]

Undecidable Inference

There are even cases where subclass relations make type inference impossible. However, if you can specialise the function type checkers can infer a meaningful type.

def foo[T](x: Sequence[T] | T) -> list[T]: ...

reveal_type(foo[bytes](b"hello"))

Currently, type checkers do not consistently synthesise a type here.

Unsolvable Type Parameters

Currently, with unspecialised literals, it is not possible to determine a type for situations similar to:

def foo[T](x: list[T]) -> T: ...
reveal_type(foo([]))  # type checker cannot infer T (yet again)
reveal_type(foo[int]([]))  # type is int

It is also useful to be able to specify in cases in which a certain type must be passed to a function beforehand:

words = ["hello", "world"]
foo[int](words)  # Invalid: list[str] is incompatible with list[int]

Allowing subscription makes functions and methods consistent with generic classes where they weren’t already. Whilst all of the proposed changes can be implemented using callable generic classes, syntactic sugar would be highly welcome.

Due to this, specialising the function and using it as a new factory is fine

make_int_list = make_list[int]
reveal_type(make_int_list())  # type is list[int]

Monomorphisation and Reification

This proposal also opens the door to monomorphisation and reified types.

This would allow for a functionality which anecdotally has been requested many times.

Please note this feature is not being proposed by the PEP, but may be implemented in the future.

The syntax for such a feature may look something like:

def foo[T]():
   return T.__value__

assert foo[int]() is int

Rationale

Function objects in this PEP is used to refer to FunctionType, MethodType, BuiltinFunctionType, BuiltinMethodType and MethodWrapperType.

For MethodType you should be able to write:

class Foo:
    def make_list[T](self, *args: T) -> list[T]: ...

Foo().make_list[int]()

and have it work similarly to a FunctionType.

For BuiltinFunctionType, so builtin generic functions (e.g. max and min) work like ones defined in Python. Built-in functions should behave as much like functions implemented in Python as possible.

BuiltinMethodType is the same type as BuiltinFunctionType.

MethodWrapperType (e.g. the type of object().__str__) is useful for generic magic methods.

Specification

Function objects should implement __getitem__ to allow for subscription at runtime and return an instance of types.GenericAlias with __origin__ set as the callable and __args__ as the types passed.

Type checkers should support subscripting functions and understand that the parameters passed to the function subscription should follow the same rules as a generic callable class.

Setting __orig_class__

Currently, __orig_class__ is an attribute set in GenericAlias.__call__ to the instance of the GenericAlias that created the called class e.g.

class Foo[T]: ...

assert Foo[int]().__orig_class__ == Foo[int]

Currently, __orig_class__ is unconditionally set; however, to avoid potential erasure on any created instances, this attribute should not be set if __origin__ is an instance of any function object.

The following code snippet would fail at runtime without this change as __orig_class__ would be bar[str] and not Foo[int].

def bar[U]():
    return Foo[int]()

assert bar[str]().__orig_class__  == Foo[int]

Interactions with @typing.overload

Overloaded functions should work much the same as already, since they have no effect on the runtime type. The only change is that more situations will be decidable and the behaviour/overload can be specified by the developer rather than leaving it to ordering of overloads/unions.

Backwards Compatibility

Currently these classes are not subclassable and so there are no backwards compatibility concerns with regards to classes already implementing __getitem__.

Reference Implementation

The runtime changes proposed can be found here https://github.com/Gobot1234/cpython/tree/function-subscript

Acknowledgements

Thank you to Alex Waygood and Jelle Zijlstra for their feedback on this PEP and Guido for some motivating examples.


Source: https://github.com/python/peps/blob/main/peps/pep-0718.rst

Last modified: 2024-02-17 03:29:08 GMT