PEP 702 – Marking deprecations using the type system
- Author:
- Jelle Zijlstra <jelle.zijlstra at gmail.com>
- Discussions-To:
- Discourse thread
- Status:
- Final
- Type:
- Standards Track
- Topic:
- Typing
- Created:
- 30-Dec-2022
- Python-Version:
- 3.13
- Post-History:
- 01-Jan-2023, 22-Jan-2023
- Resolution:
- 07-Nov-2023
Table of Contents
Abstract
This PEP adds an @warnings.deprecated()
decorator that marks a class or function
as deprecated, enabling static checkers to warn when it is used. By default, this
decorator will also raise a runtime DeprecationWarning
.
Motivation
As software evolves, new functionality is added and old functionality becomes
obsolete. Library developers want to work towards removing obsolete code while
giving their users time to migrate to new APIs. Python provides a mechanism for
achieving these goals: the DeprecationWarning
warning class, which is
used to show warnings when deprecated functionality is used. This mechanism is
widely used: as of the writing of this PEP, the CPython main branch contains
about 150 distinct code paths that raise DeprecationWarning
. Many
third-party libraries also use DeprecationWarning
to mark deprecations.
In the top 5000 PyPI packages,
there are:
- 1911 matches for the regex
warnings\.warn.*\bDeprecationWarning\b
, indicating use ofDeprecationWarning
(not including cases where the warning is split over multiple lines); - 1661 matches for the regex
^\s*@deprecated
, indicating use of some sort of deprecation decorator.
However, the current mechanism is often insufficient to ensure that users
of deprecated functionality update their code in time. For example, the
removal of various long-deprecated unittest
features had to be
reverted
from Python 3.11 to give users more time to update their code.
Users may run their test suite with warnings disabled for practical reasons,
or deprecations may be triggered in code paths that are not covered by tests.
Providing more ways for users to find out about deprecated functionality
can speed up the migration process. This PEP proposes to leverage static type
checkers to communicate deprecations to users. Such checkers have a thorough
semantic understanding of user code, enabling them to detect and report
deprecations that a single grep
invocation could not find. In addition, many type
checkers integrate with IDEs, enabling users to see deprecation warnings
right in their editors.
Rationale
At first glance, deprecations may not seem like a topic that type checkers should touch. After all, type checkers are concerned with checking whether code will work as is, not with potential future changes. However, the analysis that type checkers perform on code to find type errors is very similar to the analysis that would be needed to detect usage of many deprecations. Therefore, type checkers are well placed to find and report deprecations.
Other languages already have similar functionality:
- GCC supports a
deprecated
attribute on function declarations. This powers CPython’sPy_DEPRECATED
macro. - GraphQL supports
marking fields as
@deprecated
. - Kotlin supports
a
Deprecated
annotation. - Scala supports
an
@deprecated
annotation. - Swift supports
using the
@available
attribute to mark APIs as deprecated. - TypeScript uses
the
@deprecated
JSDoc tag to issue a hint marking use of deprecated functionality.
Several users have requested support for such a feature:
There are similar existing third-party tools:
- Deprecated provides a decorator to mark classes, functions, or methods as deprecated. Access to decorated objects raises a runtime warning, but is not detected by type checkers.
- flake8-deprecated is a linter plugin that warns about use of deprecated features. However, it is limited to a short, hard-coded list of deprecations.
Specification
A new decorator @deprecated()
is added to the warnings
module. This
decorator can be used on a class, function or method to mark it as deprecated.
This includes typing.TypedDict
and typing.NamedTuple
definitions.
With overloaded functions, the decorator may be applied to individual overloads,
indicating that the particular overload is deprecated. The decorator may also be
applied to the overload implementation function, indicating that the entire function
is deprecated.
The decorator takes the following arguments:
- A required positional-only argument representing the deprecation message.
- Two keyword-only arguments,
category
andstacklevel
, controlling runtime behavior (see under “Runtime behavior” below).
The positional-only argument is of type str
and contains a message that should
be shown by the type checker when it encounters a usage of the decorated object.
Tools may clean up the deprecation message for display, for example
by using inspect.cleandoc()
or equivalent logic.
The message must be a string literal.
The content of deprecation messages is up to the user, but it may include the version
in which the deprecated object is to be removed, and information about suggested
replacement APIs.
Type checkers should produce a diagnostic whenever they encounter a usage of an object marked as deprecated. For deprecated overloads, this includes all calls that resolve to the deprecated overload. For deprecated classes and functions, this includes:
- References through module, class, or instance attributes (
module.deprecated_object
,module.SomeClass.deprecated_method
,module.SomeClass().deprecated_method
) - Any usage of deprecated objects in their defining module
(
x = deprecated_object()
inmodule.py
) - If
import *
is used, usage of deprecated objects from the module (from module import *; x = deprecated_object()
) from
imports (from module import deprecated_object
)- Any syntax that indirectly triggers a call to the function. For example,
if the
__add__
method of a classC
is deprecated, then the codeC() + C()
should trigger a diagnostic. Similarly, if the setter of a property is marked deprecated, attempts to set the property should trigger a diagnostic.
If a method is marked with the typing.override()
decorator from PEP 698
and the base class method it overrides is deprecated, the type checker should
produce a diagnostic.
There are additional scenarios where deprecations could come into play.
For example, an object may implement a typing.Protocol
, but one
of the methods required for protocol compliance is deprecated.
As scenarios such as this one appear complex and relatively unlikely to come up in practice,
this PEP does not mandate that type checkers detect them.
Example
As an example, consider this library stub named library.pyi
:
from warnings import deprecated
@deprecated("Use Spam instead")
class Ham: ...
@deprecated("It is pining for the fiords")
def norwegian_blue(x: int) -> int: ...
@overload
@deprecated("Only str will be allowed")
def foo(x: int) -> str: ...
@overload
def foo(x: str) -> str: ...
class Spam:
@deprecated("There is enough spam in the world")
def __add__(self, other: object) -> object: ...
@property
@deprecated("All spam will be equally greasy")
def greasy(self) -> float: ...
@property
def shape(self) -> str: ...
@shape.setter
@deprecated("Shapes are becoming immutable")
def shape(self, value: str) -> None: ...
Here is how type checkers should handle usage of this library:
from library import Ham # error: Use of deprecated class Ham. Use Spam instead.
import library
library.norwegian_blue(1) # error: Use of deprecated function norwegian_blue. It is pining for the fiords.
map(library.norwegian_blue, [1, 2, 3]) # error: Use of deprecated function norwegian_blue. It is pining for the fiords.
library.foo(1) # error: Use of deprecated overload for foo. Only str will be allowed.
library.foo("x") # no error
ham = Ham() # no error (already reported above)
spam = library.Spam()
spam + 1 # error: Use of deprecated method Spam.__add__. There is enough spam in the world.
spam.greasy # error: Use of deprecated property Spam.greasy. All spam will be equally greasy.
spam.shape # no error
spam.shape = "cube" # error: Use of deprecated property setter Spam.shape. Shapes are becoming immutable.
The exact wording of the diagnostics is up to the type checker and is not part of the specification.
Runtime behavior
In addition to the positional-only message
argument,
the @deprecated
decorator takes two keyword-only arguments:
category
: A warning class. Defaults toDeprecationWarning
. If this is set toNone
, no warning is issued at runtime and the decorator returns the original object, except for setting the__deprecated__
attribute (see below).stacklevel
: The number of stack frames to skip when issuing the warning. Defaults to 1, indicating that the warning should be issued at the site where the deprecated object is called. Internally, the implementation will add the number of stack frames it uses in wrapper code.
If the decorated object is a class, the decorator wraps the __new__
method
such that instantiating the class issues a warning. If the decorated object is a
callable, the decorator returns a new callable that wraps the original callable but
raises a warning when called. Otherwise, the decorator raises a TypeError
(unless category=None
is passed).
There are several scenarios where use of the decorated object cannot issue a warning,
including overloads, Protocol
classes, and abstract methods. Type checkers may show a
warning if @deprecated
is used without category=None
in these cases.
To accommodate runtime introspection, the decorator sets an attribute
__deprecated__
on the object it is passed, as well as on the wrapper
callables it generates for deprecated classes and functions.
The value of the attribute is the message passed to the decorator.
Decorating objects that do not allow setting this attribute is not supported.
If a Protocol
with the @runtime_checkable
decorator is marked as deprecated,
the __deprecated__
attribute should not be considered a member of the protocol,
so its presence should not affect isinstance
checks.
For compatibility with typing.get_overloads()
, the @deprecated
decorator should be placed after the @overload
decorator.
Type checker behavior
This PEP does not specify exactly how type checkers should present deprecation
diagnostics to their users. However, some users (e.g., application developers
targeting only a specific version of Python) may not care about deprecations,
while others (e.g., library developers who want their library to remain
compatible with future versions of Python) would want to catch any use of
deprecated functionality in their CI pipeline. Therefore, it is recommended
that type checkers provide configuration options that cover both use cases.
As with any other type checker error, it is also possible to ignore deprecations
using # type: ignore
comments.
Deprecation policy
We propose that CPython’s deprecation policy (PEP 387) is updated to require that new deprecations
use the functionality in this PEP to alert users
about the deprecation, if possible. Concretely, this means that new
deprecations should be accompanied by a change to the typeshed
repo to
add the @deprecated
decorator in the appropriate place.
This requirement does not apply to deprecations that cannot be expressed
using this PEP’s functionality.
Backwards compatibility
Creating a new decorator poses no backwards compatibility concerns.
As with all new typing functionality, the @deprecated
decorator
will be added to the typing_extensions
module, enabling its use
in older versions of Python.
How to teach this
For users who encounter deprecation warnings in their IDE or type
checker output, the messages they receive should be clear and self-explanatory.
Usage of the @deprecated
decorator will be an advanced feature
mostly relevant to library authors. The decorator should be mentioned
in relevant documentation (e.g., PEP 387 and the DeprecationWarning
documentation) as an additional way to mark deprecated functionality.
Reference implementation
A runtime implementation of the @deprecated
decorator is
available in the typing-extensions
library since version 4.5.0.
The pyanalyze
type checker has
prototype support
for emitting deprecation errors, as does
Pyright.
Rejected ideas
Deprecation of modules and attributes
This PEP covers deprecations of classes, functions and overloads. This allows type checkers to detect many but not all possible deprecations. To evaluate whether additional functionality would be worthwhile, I examined all current deprecations in the CPython standard library.
I found:
- 74 deprecations of functions, methods and classes (supported by this PEP)
- 28 deprecations of whole modules (largely due to PEP 594)
- 9 deprecations of function parameters (supported by this PEP through decorating overloads)
- 1 deprecation of a constant
- 38 deprecations that are not easily detectable in the type system (for
example, for calling
asyncio.get_event_loop()
without an active event loop)
Modules could be marked as deprecated by adding a __deprecated__
module-level constant. However, the need for this is limited, and it
is relatively easy to detect usage of deprecated modules simply by
grepping. Therefore, this PEP omits support for whole-module deprecations.
As a workaround, users could mark all module-level classes and functions
with @deprecated
.
For deprecating module-level constants, object attributes, and function
parameters, a Deprecated[type, message]
type modifier, similar to
Annotated
could be added. However, this would create a new place
in the type system where strings are just strings, not forward references,
complicating the implementation of type checkers. In addition, my data
show that this feature is not commonly needed.
Features for deprecating more kinds of objects could be added in a future PEP.
Placing the decorator in the typing
module
An earlier version of this PEP proposed placing the @deprecated
decorator in the typing
module. However, there was feedback
that it would be unexpected for a decorator in the typing
module
to have runtime behavior. Therefore, the PEP now proposes adding the
decorator the warnings
module instead.
Acknowledgments
A call with the typing-sig meetup group led to useful feedback on this proposal.
Copyright
This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.
Source: https://github.com/python/peps/blob/main/peps/pep-0702.rst
Last modified: 2024-10-16 16:05:18 GMT