PEP 773 – A Python Installation Manager for Windows
- Author:
- Steve Dower
- Discussions-To:
- Discourse thread
- Status:
- Draft
- Type:
- Standards Track
- Topic:
- Release
- Created:
- 21-Jan-2025
- Post-History:
- 18-Dec-2024, 21-Jan-2025
- Replaces:
- 397, 486
Table of Contents
- Abstract
- Background
- Overview of PyManager
- Specification
- Rationale
- Backwards Compatibility
- Security Implications
- Impact on Existing PEPs
- How to Teach This
- Reference Implementation
- Rejected Ideas
- Make PyManager available on all platforms
- Include a runtime pre-installed with the manager
- Include a runtime INSTEAD OF the manager
- Use a built-in module rather than subcommands
- Use a new command-line option rather than subcommands
- Improving the current traditional installer instead
- Delete the Store package completely
- Rely on WinGet or equivalents
- Make every version a Windows Store package
- Just publish the plain ZIP file
- Only publish PyManager to one place
- Inline Script Metadata
- Open Issues
- Footnotes
- Copyright
Abstract
Installation of the python.org Python distribution on Windows is complex. There are three main approaches with roughly equivalent levels of user experience, and yet all of these suffer from different limitations, including failing to satisfy modern usage scenarios. This PEP proposes a design for a single Windows install workflow tool that satisfies all the needs of the existing installers for the platform, while avoiding most of their limitations, and provides the core team with the ability to manage releases for many years to come.
With the new installer, the recommended command to launch the default Python on
Windows will be python
, while the command for managing multiple versions
(including launching a specific one) will be py
. Users can also opt-in to
having global python3.x
commands by adding an extra entry to their PATH
environment variable. We recommend starting with the “How to teach this” section
for the most concise high-level overview of the features.
Reader’s Note
This document is a detailed proposal, intended to assist in an “approve” or “reject” decision on the replacement of the existing install formats for Python on Windows with a new format. It is not intended to be a binding specification, nor is it user documentation or an interoperability specification. The actual implementation may vary over time as needs change, and that is not to be considered a “violation” of this PEP. Any interfaces and protocols meant for public use will be independently documented and maintained in accordance with standard deprecation timelines.
Documentation on installing Python on Windows can be found at docs.python.org/using/windows.html.
Background
There are a large range of needs users may have that lead to them wanting to install a Python runtime. Many, likely most, are interested in running (perhaps writing) short scripts, such as those that perform a simple task, or help teach someone a concept. Some users are looking for a specific version to integrate with existing code or another application. Some are after a full set of different interpreter versions to perform testing.
In this section, we discuss the expectations that users have of “installing Python”, provide an overview of the existing installers for Windows, and identify some of the gaps and challenges inherent in these offerings.
Expectations
Based on significant anecdotal experience and analysis of quantitative data available (though not necessarily public), we make the following assertions about the majority of Python users on Windows:
- most users just want the latest stable version
- most users want a “one-click” (or fewer) install
- most users do not want to use administrator privileges
- most users will benefit from installing maintenance updates
- most users expect the
python
command to work after installation
The primary support for these assertions is that the most popular installers actively chosen by users are the latest stable release on python.org, and the latest stable release on the Windows Store, both of which meet these requirements.
We make the following assumptions about other significant sets of users. These may have some overlap between groups, and at least some users expect all of them.
- some users want to install Python programmatically
- some users want to install a particular version
- some users want to install many versions
- some users want to install for all users of their machine
- some users do not want Start Menu shortcuts
- some users want to install as part of their project’s build process
- some users want to install as part of their project’s install process
- some users intend to never update their install
- some users expect the
py
command to work after installation
These assumptions have all been demonstrated over time to exist, though the relative importance has not been quantified. The NuGet packages and the embeddable distro can meet most of these needs.
Traditional Installer
The traditional installer is an executable downloadable directly from
python.org that installs the entire development kit for Python. This includes
the CPython interpreter, the standard library, Python headers and import
libraries, builds of Tcl and Tk, the documentation as HTML files, the runtime
and standard library test suite, Start Menu shortcuts for Python and IDLE,
debugging symbols and debug builds of the binaries, the py.exe
launcher
and its file associations, and functionality to modify the user’s PATH
environment variable, enable long-path support on their system, pre-generate
.pyc
files for the standard library, and install pip.
As of 3.13, it also includes a set of experimental free-threaded binaries.
Many of these components are optional.
After downloading the executable, users are presented with a “quick install” option, which installs into their user directory with most options enabled. We believe that most users select this option.
A second option alongside the quick install takes the user to two pages worth of options, listing the components that they need not install, as well as other options such as the install directory and whether to install for all users.
All of these options may be specified on the command line, and there is also an option to proceed with the install without displaying any UI. Based on feedback and bug reports, all of these options are used by at least some users. However, as we do not track install telemetry, we have no way to know which options are more important than others.
Behind the scenes, the traditional installer is a Burn bundle, generated using the Wix Toolset installer framework, containing one or more MSI files for each feature. This framework is used extensively by Microsoft themselves, and provides the most direct method of using Windows Installer. The bundle is a custom C++ application, based on their template, which allows us to customise the overall behaviour of the installer to determine precisely which MSI files should actually be installed. The process of copying files, updating the registry, and generating shortcuts is handled entirely by Windows Installer.
As well as the intended uses, it is understood that many users will (attempt to) use the traditional installer for other scenarios, such as unregistered installs and automated CI system installs. While better alternatives are available, they are not as obvious, and the hope is that a future design would make these scenarios easier.
Windows Store
The Windows Store packages for CPython are produced as part of our normal
release process using almost all identical binaries to the other packages.
Due to being in an app store package, the primary python.exe
is enhanced
to be able to determine its location properly, and alternative pip.exe
and
other shortcuts are included to make up for the lack of PATH
environment
variable settings. These are implemented in PC\python_uwp.cpp
in our repo.
These packages are installed by searching for Python in the Microsoft Store
app, which will find results for each major version since 3.8. Users then have
to select a version and install it. These packages include the CPython
interpreter, standard library, Tcl/Tk, IDLE, and pip, and create file
associations, Start Menu shortcuts, and global commands for python.exe
,
python3.exe
, python3.X.exe
, pip[3[.X]].exe
and idle[3[.X]].exe
.
No PATH
modification is possible or required, though users may need to
manage their global shortcuts through the “Manage App Execution Alias” settings
page.
In addition, Microsoft has added to a clean Windows install a default
python.exe
command. This captures attempts by users to launch Python on
a machine that has not yet installed it. When launched directly, the command
will open the Microsoft Store app to the page containing the recommended
Python app, typically the latest version. This app is entirely controlled by
Microsoft. Based on telemetry associated with the Python app (which is
controlled by the upstream Python project), approximately 300,000 installs
per month come through this redirector, making up about 90% of the total
installs of that version.
Behind the scenes, the Store package is based on Microsoft’s new installer technology for apps known as APPX or MSIX. These are essentially plain ZIP files with a small amount of metadata, except that installation is handled by the operating system. They are always extracted to a fixed location, accessible to all users but not modifiable by any, and automatically updated to the latest release. The user’s own data is stored in an OS-managed location in their user profile, and is able to be reset, backed-up and restored using regular OS functionality.
NuGet Package
The NuGet packages for CPython are produced and published as part of our normal release process. The contents are identical to the traditional installer. A NuGet package is published to nuget.org, which is a package manager typically associated with .NET languages, but highly integrated with any project supported by Visual Studio. This makes it a nice format for users who want a lightweight install of Python as part of their regular build process, and can simplify embedding scenarios.
The packages are installed using any tool capable of using the NuGet API, or
may be downloaded directly once the URL of the package is known. The package is
a plain ZIP file with some metadata. It contains the CPython interpreter, the
standard library, development headers and import libraries, and pip. It does
not execute any code at install time, and users must locate the package
themselves in order to launch the python.exe
contained within.
Embeddable Package
The embeddable package for CPython is produced and published as part of our
normal release process. It is published to python.org alongside the
traditional installer. The contents are identical, however, the layout is
changed to store all binaries at the top level, with the standard library
packed into a ZIP file. A ._pth
file is included to override sys.path
so that only the files that are part of the distro are used, and environment
variables or registry entries are ignored.
This package does not include pip, as the intention is for it to be embedded into a broader application. Other libraries should be installed at build time, since after distribution the runtime is meant to be an internal implementation detail of the app it is a part of.
As well as its intended use, some users attempt to use this package as a development kit rather than a runtime package. This is believed to be due to those users preferring to avoid “heavyweight” installers, and believing that this package is intended to be a “portable” install (extract and run), likely because it is the only ZIP file option listed on the python.org download pages (speaking to the importance of clarity and limiting options on those pages). It is hoped that a future installer design will avoid or limit this confusion.
Alternate Distributions
While outside of our purview as the core team, alternate distributions of Python for Windows often use a project, workflow or environment-centric model for installation of the runtime. By this, we mean that the tool is installed first, and is used to create a working space that includes a runtime, as well as other dependencies. Examples of these tools include conda and uv.
Two observations are worth making about these tools. Firstly, they are often praised for being low impact, in that they usually don’t install additional entry points or files for the runtime, making the install fast and also isolated to a single project. Secondly, their users often appreciate the ease of selecting a particular version of a runtime, or alternatively, not having to select at all because existing specifications (or constraints) can choose for them.
These tools tend to meet many of the second set of expectations described above, usually combining multiple tasks in a single command to reduce the cognitive overhead of learning how to use and combine multiple commands.
It’s also worth pointing out that the core team does not view these alternate distributions as competitors to any upstream distribution. They are a fundamental part of how the open source ecosystem is intended to work. Our own distributions are a convenience for those who choose to use them, as not all scenarios are well served by a workflow tool or even a pre-built package.
Challenges
There are numerous challenges we face with the current set of installers, which largely break down into two categories: mismatched or unachievable user expectations, and general unreliability.
The traditional installer has the highest level of unreliability. The Windows Installer technology is very old, and effectively no longer under development. While its basic functionality is okay, interference may come from many sources, such as virus scanners, other installers, system configuration, admin policies, and even other files in the same directory as the installer. On top of this, most of its advanced and beneficial functionality such as update patches, incremental updates, and automatic rollback are unimportant for Python users.
Most user expectations are defined by the traditional installer, and so by definition, it meets them. One primary gap is that it is not able to create an “unmanaged” install - that is, the equivalent of only copying files onto the user’s system without registration. If you have installed it once, and you try to install it again, you will only even be able to manage (or upgrade) the existing install. This can lead to installs moving on update, which will break users.
Additionally, the PATH
environment variable cannot be intelligently
modified - at best, we can prepend or append the install path. This usually
results in the most recent install of Python being the highest priority. For
example, if the user has Python 3.14 installed and then installs (or updates)
3.13, the python
command will switch from the later version to the earlier
version.
The py.exe
launcher, defined in PEP 397 and implicitly updated by PEP 514,
is an attempt to avoid this particular issue. It uses its own logic for finding
installed versions without relying on PATH
. However, the PEP 514 logic does
not allow for prerelease or experimental builds to be treated specially, and so
py.exe
often prefers these builds by default over the non-experimental
version expected by the user.
The Windows Store package is very reliable, with the exception of the global
shortcuts. Rather than modifying PATH
to add its own directory, these
shortcuts are created in a single OS managed directory that has all the
shortcuts defined by any app. Users are able to modify their PATH
to exclude
or de-prioritise this directory, leading to unreliable or inconsistent
behaviour, and historically we have also seen this caused by installers.
For example, installing Python from the Store followed by Python from the
traditional installer with its PATH
modification enabled will almost always
shadow the Store package’s Python with the later install.
User expectations that are un-met by the Store package tend to be performance
and technical. Due to the overhead of launching an app, Python starts up slower.
Because apps are designed to be isolated from each other, it is more difficult
to use hidden directories (such as AppData
or TEMP
) to communicate
between different versions of Python, as each version has its own space. Apps
are subject to stricter security requirements that legacy applications usually
have disabled, such as DLL hijacking protection, which causes some libraries to fail.
The python3
and python
shortcuts are managed through system settings,
and the user interface is not very good (and not going to be improved, according
to Microsoft). Without managing these, it is relatively easy for an undesired
version to be launched, though in general the targets can only be changed
manually by the user, and not by merely installing another app.
Both the NuGet package and the embeddable distro are as simple and reliable to install as extracting an archive file, though it’s worth noting that for many Python users this is not a common task. They provide no install management at all, and cannot be reliably updated other than by deleting and re-extracting. User expectations that are un-met are almost always due to users selecting the wrong installer. Both these packages are for specialised cases, and while they are documented as such, the attraction of a plain ZIP file leads some users into failure.
Overview of PyManager
PyManager is the internal name of our proposed replacement installer tool. It will be distributed both in the Windows Store and on python.org as an MSIX package. Downloading from either source will get an identical package, and both will support automatic updates (through the Store) for new releases.
The user visible name will be “Python Install Manager”, published by the
Python Software Foundation. After publishing, we will request that Microsoft
adjust their python.exe
stub to open to this new app.
This app does not directly provide a version of Python, but it does provide the global commands that users expect to work, as well as file associations and Start menu shortcuts. The OS will prompt users to launch the app after install, which will trigger an automatic install of the current release of CPython and then launch it. From the user’s perspective, they have the same initial experience as today, with one added progress bar on first launch.
The global commands provided by the app must be static and bundled into the app
itself. They can only change their behaviour at runtime, and cannot be
redirected to different executables except by the user (and then only to another
installed app). So the commands to be provided by PyManager are python.exe
,
python3.exe
, py.exe
, pymanager.exe
. Each of these must have the
ability to inspect the user’s system and choose the correct runtime to launch.
Additionally, py
and pymanager
will have management subcommands to allow
adding and removing runtimes.
In line with PEP 394 and the default behaviour of Windows, the recommended
command for launching Python is python.exe
. As provided by PyManager, this
will locate an existing install, either among those that PyManager manages or
using PEP 514, or it will install the latest available version of CPython and
select that. The python3.exe
command behaves similarly, but is only allowed
to find 3.x installs from python.org.
The py.exe
command provided by PyManager will be recommended for most
management use, due to its brevity. py install ...
, py list ...
and so
on. The proposed commands are detailed later. The existing behaviour of the
PEP 397 launcher is preserved, however, launching through py
will not
automatically install runtimes (by default). If one is requested but is not
installed, users will just get an error. The py exec ...
subcommand,
however, will install automatically, and supports the same options as bare
py
.
These commands are added at very low priority in the user’s PATH
by the OS.
Every existing configuration we may have created on a user’s machine will take
precedence over these commands, and so these are a last resort in place of an
error message. As a result, we can generally assume that a user is launching
these commands because they haven’t configured a stronger preference (for
example, a user who has activated an environment will never launch our
python.exe
, because activation will put a different one ahead of it, and a
user who wants precisely the behaviour of the existing py.exe
can just
install it and will never launch our new one).
The pymanager.exe
command is to allow for handling ambiguous situations.
Existing installs of Python and the launcher may shadow python.exe
and
py.exe
, but in an automated environment, this can make administrative
scripts unreliable, and so the pymanager
command is unlikely to refer to
something other than PyManager. It has all the subcommands, and launching it
with no command specified will print help for the user.
Replacing other installers
Our intent is to immediately stop publishing individual versions to the Windows Store, and to deprecate and phase out the traditional installer by Python 3.16. The embeddable distro will remain, but its listing on python.org download pages will be phased out and it will be available only through PyManager. No changes will be made to the NuGet packages.
PyManager will be made available as an app package downloadable manually from python.org, and the double-click install experience is generally smooth. This provides an equivalent to the current approach of downloading from our site. It will bundle a recent (unspecified) version of CPython so that the download can be moved to a non-internet connected machine and still provide a Python runtime after install.
Some automated deployment scenarios do not work with the newer MSIX format, and so a simple MSI will also be provided on python.org. This will have no options or user interface, and require administrative privileges, which are typically available for these kinds of scenarios. Additionally, the MSI will be required by users on non-standard systems, such as Wine, that do not support the MSIX format. While not officially supported, the MSI will enable these platforms to continue using upstream distributions of CPython. The MSI would be discouraged for most other users, and the MSIX is considered the default.
It’s worth noting that there is no way to make the MSI install fully compatible with the MSIX, and users with both will likely encounter confusion or problems. It is anticipated that only users without Store app support will use the MSI.
Our release processes will start publishing plain ZIP packages to python.org. These will be available from the FTP pages, but will not be listed directly on regular download pages.
Third-party tools that currently distribute their own builds of CPython will be welcome to use ours, though will be expected to be the initial point of contact for their users requiring support.
Project ownership and development
PyManager will be developed and maintained in its own repository adjacent to the CPython repository, and under the same terms. The CPython CLA will apply, and all (and only) core developers will have commit rights.
PyManager releases are independent from CPython releases. There is no need for versions to match, or releases to be simultaneous. Unless otherwise arranged, the PyManager release manager is whoever is the build manager for Windows.
Specification
Note
In this document, all command line options will be shown with one or two hyphens. In implementation, all options will support one or two hyphens or a forward slash, to be permissive of both Windows and UNIX conventions.
Exec subcommand
py [-V:<TAG>] [interpreter opts] [script.py|-m module|-c code] [script args]
py [-3.*] [interpreter opts] [script.py|-m module|-c code] [script args]
python ...
python3 ...
pymanager exec -V:tag ...
pymanager exec -3.* ...
This subcommand is used to select and launch a runtime. It is the default action
for the py
command, and the only action supported by the python
and
python3
commands. The default options are subtly different in each case for
consistency with existing use of these commands.
This subcommand is available on both py
and pymanager
. However, since
py
offers it by default, we would not expect users to use it there. The
intent is that the py
, python
and python3
commands are the default
ways to launch a runtime, and pymanager exec
is for advanced scenarios.
The -V:tag
command is used to request a specific runtime from the command
line. The tag is a Company\\Tag
pair, or just Tag
if no slash is
included, and is used as defined by PEP 514. The -3.*
option is interpreted
as -V:PythonCore\\3.*
. This option is only available for py
and
py exec
variants.
If no tag is specified on the command line, and a script file is specified,
the script will be inspected for a shebang. If one is found that matches a
recognised pattern, it will either provide the tag to be used for search, or it
will override all other processing and its specified executable will be launched
without further effort being made. This is to handle the (unfortunate) legacy
support of arbitrary Windows-specific paths being allowed in what was meant to
be a portability feature. In general, shebangs including simple patterns like
/usr/bin/python3.13
are intended, while those that use /usr/bin/env
python
are unlikely to be of benefit since the environment tends to be less
reliable than our search.
If no tag is yet requested, the VIRTUAL_ENV
environment variable will be
consulted to see if an environment has been activated. If so, that will become
the request.
If a tag has been requested at this stage, the python3
command will verify
that it matches PythonCore\\3.*
and exit with an error if not. This allows
allows the python3
command to be used in an active environment consistent
with other platforms, but not if the environment would not have included the
command. This applies to most existing versions of Python on Windows. (The
alternative to this behaviour is to make python3
always error when an
environment is active, as anything else would behave inconsistently for the
user.)
If no tag is requested, the default will be consulted. For python3
, this is
PythonCore\\3
, but for all other commands it is read from configuration
(which might involve an environment variable). If it’s still empty, any tag will
be allowed.
The best installed runtime matching the tag is then selected and launched with the remaining command line.
If no matching runtime is found, the py exec
and pymanager exec
commands
will automatically install and launch one (user configuration permitting). The
py
, python
and python3
commands will report and error and exit.
However, in the case where no runtimes are available at all, including none
detected from other installers, the first runtime will be automatically
installed. This provides a useful first-launch experience, where a new user who
has just installed PyManager will directly launch the latest (or requested)
version of CPython. After this first time, errors will again be reported when a
requested runtime is not found.
Install subcommand
py install [-s|--source <URL>] [-f|--force] [-u|--upgrade] tag [...]
py install [-s|--source <URL>] [-t|--target <DIR>] tag
py install [-s|--source <URL>] [-d|--download <DIR>] tag [...]
py install --refresh
Note
This and all later subcommands are also available under pymanager
.
However, as we intend for py
to be the usual command, we only show that
one.
This subcommand will install one or more runtimes onto the current machine.
The tags are Company\\Tag
pairs (or just Tag
if no slash is included),
and are used to search the index file. Company names match as case-insensitive
prefixes, preferring a full match over a prefix, and tags use case-insensitive,
number-aware matches, with dotted numbers treated as versions. Tags must match
one of the listed “install for” tags, and entries list multiple such tags to
handle abbreviated requests. The special tag default
resolves to the user’s
configured default (typically 3
).
For example, the 3.10.5
entry would list all of 3
, 3.10
and
3.10.5
as tags to be installed for. A request for 3.10
would match one
of these and so the entry is selected. Due to the number-aware matches, a
request for 03.0010
would also match, and 3.10.50
would not.
Tags may also be specified as a constraint, using >
, >=
, <
, <=
or !=
followed by the Company\\Tag
or Tag
value. When matching a
constraint only the primary tag metadata is used for comparisons. Since the
comparisons are version-aware, constraints such as >3.10
will select
3.11 as a minimum, while >3.10.0
may select 3.10.1.
The behaviour of constraints against arbitrary tags is likely to be unintuitive
in some circumstances. It is anticipated that constraints will mainly be used
with upstream releases, which typically use version-shaped tags, and primarily
for cases where other metadata such as Requires-Python
are being handled.
Users are expected to use shorter tags for convenience, rather than ranges.
The default index file is hosted on python.org, and contains install information
including package URLs and hashes for all installable versions. An alternate
index may be specified by the user or their administrator (see Configuration
below). Entries in the index file list the full set of tags they should be
installed for, and if an exact match is found the package will be selected. In
the case of no exact match, a prefix match will be used. In both cases, numbers
in the tag are treated logically - that is, 3.1
is a prefix of 3.1.2
but
not of 3.10
.
If a tag is already satisfied by an existing install, nothing will be installed.
The user must pass an --upgrade
or --force
option to replace the
existing install; the former will only replace it with a newer version, while
the latter will remove and replace even with the same version.
Calling the command without providing any tags will not install anything, but
will display the help text and an error (as for any invalid option). However,
passing --upgrade
with no tags will attempt to upgrade all installs.
Passing --refresh
will regenerate all metadata and shortcuts for all
installs. This is intentionally applied to all installs at once, as shortcut
prioritisation relies on all installs being consistent (for example, the latest
3.x version should get the python3.exe
shortcut, which gets complicated if
users can choose to only refresh an older install).
If a --target <DIR>
option is passed with only a single tag, that runtime
will be extracted to the specified directory without being registered as an
install (or generating aliases or shortcuts). This is intended to cover
embedding cases, or downloading the files for incompatible platforms. Passing
multiple tags with --target
is an error.
If the --download <DIR>
option is passed, runtimes will be downloaded to the
specified directory as their source packages, and an index.json
will be
created referencing these files. This index can be referenced later to perform
offline installs with python install --source <index.json> [tag ...]
.
Uninstall subcommand
py uninstall [-y|--yes] [--purge] [tag ...]
This subcommand will uninstall one or more runtimes on the current machine. Tags
are exactly as for the install command, including prefix matching, but only
inspect existing installs. Unless the --yes
option is passed, the user will
be prompted before uninstalling each runtime.
If the --purge
option is passed with no tags, then (after confirmation) all
runtimes will be removed, along with shortcuts and any cached files.
Uninstalling PyManager does not uninstall any runtimes that were installed. For
technical reasons, this would not be reliably possible (we cannot run arbitrary
code at uninstall time), and so instead we deliberately ensure that anything
that has been installed will continue to work. Reinstalling PyManager allows
management of these installs to resume. Running py uninstall --purge
before uninstalling
PyManager will perform a complete uninstall.
List subcommand
py list [-f|--format <FMT>] [-1|--one] [--only-managed] [tag ...]
py list [-f|--format <FMT>] [-1|--one] [--online] [--source <URL>] [tag ...]
py [--list|--list-paths|-0|-0p]
This subcommand will list any or all installs matching the specified tags or ranges. If no tags are provided, lists all installs. Runtimes not managed by PyManager (including an active virtual environment) may be listed separately.
The default format is user-friendly. Other formats will include machine-readable
and single string formats (e.g. --format=prefix
simply prints sys.prefix
on a line by itself). The exact list of formats is left to implementation.
If --one
is provided, only the best result is listed. This is to assist
shell scripts that want to locate the default (or a suitable) runtime without
launching it. (Note that “best” is loosely defined, but will always be the
user’s preferred default environment if it is included in the results.)
The --only-managed
option omits runtimes that were discovered but are not
managed by PyManager, for example, those found using a regular PEP 514 lookup.
Passing --source
(or --online
to implicitly pass the default source)
will search an online index rather than currently installed runtimes. The option
is here rather than on the install
subcommand because the filtering and
formatting options are already available on list
.
The legacy --list
, --list-paths
, -0
and -0p
arguments from the
py.exe
launcher will also be provided. However, they will not support the
new options listed here, and are limited to reproducing the output from the
existing launcher. Unmanaged installs are not distinguishable in this listing.
Help subcommand
py help <COMMAND>
This subcommand will display the help text for each specified command, or if
none specified, will show the list of commands. Specifying one command is the
equivalent of py <COMMAND> --help
. Showing the list of subcommands is the
default action for the pymanager
command.
The command is added primarily to offer a simple way to tell users how to find
more information: they can be told to run py help
. This avoids having to
override or extend the python -?
output, which otherwise forwards to the
selected runtime and already prints at least one screen’s worth of text.
After an automatic install (e.g. running python
with nothing installed), a
message will be displayed telling users that they can run py help
for more
information on how to manage their installs.
Environment Variables
No environment variables can be updated automatically when installing a Store app, and so no updates will be done automatically. The core commands should already be available on a correctly functioning machine.
One directory within the user’s PyManager data directory is set aside for
generated aliases. If desired, the user can add this directory to their PATH
themselves. The contents of this directory will be managed by PyManager, and
will contain executables to directly launch installed runtimes (for example,
python3.exe
and python3.13.exe
for an install of Python 3.13). Whenever
aliases are added to this directory, PATH
will be checked and if it is
missing, the user will be presented a message containing the path to add.
Scripts installed by packages installed into a runtime will be in yet another directory. Due to the current design, we do not believe it is safe to have them all install into a single directory, or a directory shared by multiple runtimes. However, a future development may include a command for PyManager to generate its own entry points based on metadata in installed packages.
File Associations
Standard file associations will be created when installing PyManager, and will
launch scripts and packaged apps with PyManager’s global python.exe
alias.
This provides sensible behaviour for users who are double-clicking on scripts or
.pyz
files.
Windowed Executables
For each of the global aliases described earlier, a *w.exe
also exists.
These launch without creating or attaching a console window, which typically
means they will only display UI created by the script. For example, IDLE always
launches using pythonw.exe
, as this avoids an unnecessary native console.
These commands otherwise behave identically to their console counterparts.
Configuration
PyManager is configured using a hierarchy of JSON-based configuration files. Command-line options always override configuration file options. Configuration files in user editable locations may be disabled by a configuration or command-line option.
In ascending order of priority, these will be located:
- within the app package
- specified by admin-only configuration (see below)
- in the
base_config
setting (default: none) - in the
user_config
setting (default:%AppData%\\Python\\PyManager.json
) - in the
additional_config
setting (default:%PYTHON_MANAGER_CONFIG%
) - specified with the
-c
command line option
The specific behaviour of each configuration option is left to implementation. However, a number of intended options are discussed in other sections.
App package configuration is provided to allow PyManager to be embedded in other applications or packages. For example, an alternative distribution may want to include PyManager but have it locate installs from their own index. The app package configuration allows reusing our build and overriding the default settings.
The user_config
and additional_config
settings are pre-configured in
earlier configuration files, allowing them to be overridden by admin-only
configuration or an alternate root configuration. If a configuration file
overwrites the setting that caused the file to be loaded, it is ignored.
The base_config
setting is similar, but starts empty and is intended for
easy overriding through admin configuration.
Admin-only configuration is provided to allow administrators to manage systems under their control using existing tools, such as group policy or registry updates. By design, these controls cannot be overridden, such that it is possible for administrators to deploy policy that prevents or limits the use of PyManager. These controls are essential to allow PyManager to be deployed safely into certain environments, and without them, it would simply be disallowed and those users would have no access to Python.
The intent is for the main admin-only configuration to be a path to a new
base_config
configuration file that an administrator can deploy to any
controlled location. This allows a network administrator to control the source
of their users’ default Python runtimes, without forcibly restricting them, or
to override the other sources for configuration files (apart from the command
line option).
Index Schema
The index file is made available either online or locally, and provides PyManager with all the information needed to find, select, install, and manage any Python runtime.
The index is stored as JSON. The main top level key is versions
, which
contains a list of objects. Each version object has its own schema version, and
there is no overall file schema version. Future changes may add additional
top-level keys to provide functionality that cannot be safely integrated into
an existing one.
Version objects may be split between the index file and a __install__.json
stored in the root of each package archive. The entries in the bundled file will
fill in any gaps from the index file. This is intended to allow the typically
large shortcuts
key to be removed from the index file, but may also extend
to alias
, executable
and executable_args
. Omitting other keys from
the index may result in problems installing the package. Ensuring correct
behaviour is left to the implementation - this is not an interoperability point,
and so we do not intend to specify the details here (beyond the normal compatibility
requirements that apply).
A second top-level key next
contains an optional URL to another index. This
may be used if PyManager cannot find a suitable package in the included
versions. The intent is to allow for older indexes to be archived and only
accessed when required, reducing the size of the initial download without
cutting off users from older versions. When searching for a suitable install,
later indexes will not be searched if a viable candidate is found (in other
words, the first index consulted should have the latest versions in it).
The initial schema is shown below:
SCHEMA = {
"versions": [
{
# Should be 1.
"schema": int,
# Unique ID used for install detection/side-by-side.
# Must be valid as a filename.
"id": str,
# Name to display in the UI
"displayName": str,
# Version used to sort packages. Also determines prerelease status.
# Should follow Python's format, but is only compared among releases
# with the same Company.
"sort-version": Version,
# Specifies platforms to consider this package for.
# Initially, 'win32' is the only supported value. Others may be
# defined in the future. This condition is evaluated silently, and
# is not intended to replace platform requests in "install-for".
"platform": [str],
# Company field, used for filtering and displayed as the publisher.
"company": str,
# Default tag, mainly for UI purposes.
# It should also be specified in 'install-for' and 'run-for'.
"tag": str,
# List of tags to install this package for. This does not have to be
# unique across all installs; the first match will be selected.
"install-for": [str],
# List of tags to run this package for. Does not have to be unique
# across all installs; the first match will be selected. The target
# is the executable path relative to the root of the archive.
# Explicit args (optional) are inserted before user args.
"run-for": [{"tag": str, "target": str, "args": [str], "windowed": int}, ...],
# List of global CLI aliases to create for this package. Does not
# have to be unique across all installs; the first match will be
# created.
"alias": [{"name": str, "target": str, "windowed": int}, ...],
# List of shortcuts to create for this package. Additional keys on
# each instance are allowed based on the value of 'kind'.
# Initially, 'kind' supports the following values:
# * 'pep514' - other keys define registry values to set
# * 'start' - generate shortcuts in the user's Start Menu
# * 'uninstall' - generate an Add/Remove Programs entry
"shortcuts": [{"kind": str, ...}, ...]
# Default executable path, relative to the root of the archive.
# Usually the values from 'run-for' will be used instead, and this
# is mainly for display purposes.
"executable": str,
# Default executable args
"executable_args": [str],
# URL to download the package archive from
"url": str,
# Optional set of hashes to validate the download. Hashes are stored
# as hex digests. Any hash supported by hashlib without OpenSSL is
# permitted.
"hash": {
"<hash_name>": str,
}
}
],
# Full or partial URL to the next index file
"next": str,
}
Shebang Processing
For limited compatibility with scripts designed for sh-like shells, PyManager will check scripts for a shebang line. A shebang line specifying a Python command will be used (when not overridden on the command line) to select a suitable runtime for the script.
Unlike the support currently in the py.exe
launcher, we propose to reduce
this functionality to only support Python commands where the command matches
a global alias listed for an install, with anything not matching being treated
as an arbitrary executable path. In practice, this is expected to produce the
same (or better) results for non-contrived cases, but may result in subtle
changes where users are relying on unspecified edge case behaviours of the
existing launcher.
The specific patterns to be detected are left to the implementation, but should be largely compatible with the existing launcher.
Rationale
“Changing” the python.exe command
It may be argued that the global python.exe
alias provided by PyManager is
“not real Python” and so should use a different name. While this is strictly
true, there are three reasons we argue that it should be used.
Firstly, thousands of users daily install through the Store page after being
led there by having typed python
at the terminal of a clean machine. Due to
how this redirection is implemented, if the app they install does not provide a
python
command, then the redirection will remain in place. In order to
ensure that users do not get stuck always going back to the Store, we need to
provide this command. (The same applies to python3
.)
Second, the alternative to the “not real” alias is not “the real” one. It’s
nothing. We don’t have the ability to replace the global static alias with one
that follows the user’s preference or installs, and so the alternative would be
to provide nothing and have python
be an error in all cases. This is worse,
and in our opinion, actively harmful to Python’s reputation.
Third, although the underlying implementation of the python
alias is more
complex than the default Programs/python.c
, the experience of using it is
identical. The alias is only launched in the absence of another expressed
preference (that is, there’s nothing else on PATH
), it respects any
indirect preferences (such as through configuration or shebangs), and it
launches the most appropriate version of Python available on the user’s machine.
This is much closer to the desired behaviour of the global python
command
than any alternative.
It has been noted that Gentoo also distributes an intelligent python
command, in order to serve their users better than a simple symlink.
Replacing py.exe
The py.exe
launcher exists to provide some of the functionality that will be
replicated by PyManager - specifically, the ability to launch an already
installed runtime. Despite its long history, the launcher does not seem to have
become the preferred method for most users, with many preferring the global
modifications to the PATH
environment variable. However, the command itself
has come to be relied upon, and should be preserved as long as possible. This is
achieved in two ways.
Firstly, we install our own py.exe
alias with PyManager that provides the
same functionality, along with PyManager specific functionality. This is
intended to become the default/preferred install of py.exe
over time.
Second, we generate PEP 514 metadata (when requested) for each install, which
allows a legacy py.exe
to continue to work normally with installs managed by
PyManager.
Due to how the existing py.exe
launcher configures itself, and how the MSIX
package for PyManager is constrained, it is not possible for PyManager’s py
alias to override the launcher. As a result, users who install the launcher will
always find py
resolving to the launcher. Ultimately, the only way to
resolve this in favour of PyManager is to uninstall the launcher, which can be
done through the standard Installed Apps control panel.
We propose to update the existing launcher, which will continue to be released with the traditional installer, to detect attempted use of the new subcommands and provide a useful message for the user rather than the current error. These warnings can also detect the unlikely case where a user is intentionally launching extensionless files by those names and retain that behaviour, allowing a viable alternative for users who cannot make other changes to their setup.
Interaction with venv
An activated virtual environment, as implemented by the standard library
venv
module, will modify the user’s PATH
environment variable to ensure
that the venv launcher will take precedence over other executables. As a result,
when a venv has been activated, PyManager can only be launched by its aliases
other than python
. When an active virtual environment is detected, it will
be treated as the user’s default runtime (except for uninstall), which ensures
that other commands will also behave as expected.
This means that working virtual environments will behave as they do today with no additional support from PyManager.
Backwards Compatibility
In general, there are no compatibility guarantees to the install process between
minor versions (3.x
to 3.y
), and so “having to use a different
installer” is not considered compatibility breakage. The versions of Python
installed are only impacted by this change to the extent that the install method
modified their behaviour. In general, most installs will be closer to the
behaviour of having been built from source by the user themselves.
That said, there are a number of changes that will impact certain users when they do move to a new install process. This section outlines as many of these changes as we are aware of, in no particular order, and will likely form the basis of a migration guide.
Scripted downloads
Users who wrote scripts to generate the download filename of our old installer will find those scripts are broken. These URLs were never guaranteed stable or predictable, and so we have no recourse to do anything other than apologise and suggest users use our own tooling for downloads.
The deprecation period for the traditional installer allows time for these users to learn about the upcoming change. Where possible, we will add deprecation warnings to the traditional installer.
Scripted installs
Users who wrote scripts to execute our installer with particular options will have to change their script. Most options have been removed, to begin with, and those that remain have new spelling. Since it is not possible to reach a state where options for the old installer are being passed to the new without manual intervention (that is, someone has to change the command already), this is considered an acceptable change.
The deprecation period for the traditional installer allows time for these users to learn about the upcoming change. Where possible, we will add deprecation warnings to the traditional installer.
Old runtime installed
Users with existing runtimes installed will find them selected by PyManager and its aliases, provided the registration is not corrupt.
The priority order among installed runtimes has changed to only include
prerelease versions when specifically requested (for example, -V:3
will
match 3.14.0 rather than 3.15.0a1, but -V:3.15
will match 3.15.0a1), and to
correctly sort text suffixes on tags (for example, 3.14t is now lower
priority than 3.14).
While it is possible to provide warnings in cases where this may be impacting a
user, such warnings would be considered very noisy (e.g. a message every time
you launch python
because you have a prerelease installed that wasn’t
selected) and require complicating the selection logic unnecessarily. This
change will be documented only.
Old py.exe
launcher installed
Users who do not manually uninstall an old py.exe
launcher will find that
both their existing and new installs of Python are found, though where versions
match the existing install will take priority over the new install (whereas the
new py
would select the new install).
They will also find that commands such as py list
do not work. The solution
here is to use Windows Settings to uninstall the launcher.
There is no way to detect that a user has accidentally left an old py
installed, or to remove it for them. This change will be documented only.
Misconfigured venv activated
Users who activate a corrupt or misconfigured virtual environment that is either
missing its python.exe
or has it not on PATH
may receive a different
error from before.
PyManager’s global python
alias will be found and executed instead,
suppressing any system “not found” error. As it fails to find the environment’s
actual runtime, it will then fail, though the code and message may be different.
As this scenario requires an already corrupt system, this change will be documented only.
Old version availability
Python versions prior to the first release of PyManager can be backfilled into the python.org index, either based on newly repackaged archives or using the almost equivalent packages from NuGet (the latter does not include Tcl/Tk, making them significantly incompatible for some users, but this is likely okay for especially old versions).
Administrator installs
Installing a copy of Python for all users is no longer possible, as PyManager will only install into the user’s own directory. No scenario has been presented to show that per-machine installs are in line with our intent for the upstream distribution, and so we will simply not provide an option for them. Third parties who desire this functionality are encouraged to provide their own distributions.
PyManager can only be installed for all users, though an MSIX does not require administrative privileges to install, and can be extensively configured by an administrator, including to constrain the actual runtimes which users may install. Additionally, PyManager supports local extraction for bundling, and so embedding apps can easily generate their own layout, which can be installed for all users if they so desire.
As this scenario requires administrator intervention with or without any changes, this will be documented only.
Build-time installs
Users using the embeddable distro may have to change to a new method for discovering the URL to the packages, though the recommendation would be to use PyManager to discover and install. No differences are anticipated due to the change of installer, and the embeddable distro package would be identical to today.
No changes to the NuGet packages are proposed.
Single architecture installer
The current proposal only makes an Intel 64-bit build of PyManager available. This will prevent users on 32-bit only operating systems or CPUs from being able to install PyManager. As this is a vanishingly small proportion of machines today, and an even smaller proportion of developer machines (the target audience for PyManager), we are not concerned about excluding users.
Windows ARM64 machines support running binaries built for Intel 64-bit through efficient emulation. CPython 32-bit builds will still be available as they have an important role in integrating with 32-bit executables, which does not allow substitution of 64-bit binaries. As PyManager runs as a standalone executable, this is not a necessary feature for the manager.
Security Implications
In this section we compare the security implications of the installer itself to the existing installers. The implications of Python being installed on a machine are out of scope, and the ability of a malicious user to execute the installer is also out of scope.
The typical risk introduced by an installer is that an elevated install may make changes to a system that allow a low-privileged user to later affect a high-privilege user, for example, by inappropriately setting access control on shared folders. PyManager only operates as the same privilege level as the user, and therefore cannot introduce any escalation path.
An install using the MSI described earlier may introduce additional risks, due to using older installer technology. Typical users are directed towards the MSIX or Windows Store install paths, which are safe, and it is assumed that users of the MSI are capable of ensuring the security of their install process (for example, by correctly quoting their commands to launch the installer and ensuring the initial system configuration is suitable).
Once PyManager is installed on a machine, it is likely that malicious users will use it to install Python. The admin-only configuration described earlier in “Configuration” is intended to control these scenarios. Ultimately though, an attacker who can run PyManager is able to do whatever the user can do, and only a complete application whitelisting approach can prevent the use of Python.
Acquisition of packages over HTTPS is protected by the connection security. We use native Windows download mechanisms that support public and enterprise certificates, as well as authenticated proxy servers. The feed may include hashes for downloadable packages, which will be verified, but there is no third-party hosted verification built into PyManager. This is consistent with today’s model.
Runtime installs by PyManager are fully accessible by, and modifiable by, the current user. This is equivalent to typical installs using the traditional installer or a NuGet package, but is more vulnerable to tampering than a Store install or a per-machine install with the traditional installer. It is not possible to fully protect an install from the user who installed it. Reinstalling or updating an install performs a clean install, which will revert any tampering that may have occurred since the original install.
The aliases generated by PyManager when installing a runtime are designed to use a signed, unmodified executable that uses an adjacent data file to launch the correct target. This can be easily abused to direct the launcher to launch an alternative, however, the only way to resolve this would sacrifice the trust in the executable itself, making it trivial to replace it instead of the data. Such risk already exists, and is equivalent to replacing the script that a user may launch, or any part of the standard library. Importantly, since aliases are not shared between users, there is no escalation of privilege along this route.
PyManager has no mechanism to perform a per-machine install. This may be useful functionality to some users, as it would allow an install to be completely unmodifiable by the regular user (excluding virtual environments and the user site folders). Such functionality may be manually imitated by an administrator using PyManager and other OS commands, but it is not considered a critical workflow. The recommended alternative is for an administrator to provide PyManager and override its configuration.
Impact on Existing PEPs
This proposal would effectively replace PEP 397 (“Python launcher for Windows”) and PEP 486 (“Make the Python Launcher aware of virtual environments”) by defining the same functionality as part of a new tool with the same name. Both are already considered final, and the launcher is defined by its documentation and normal compatibility processes. New functionality is based on the current implementation, and not the original PEP text.
This proposal has no impact on PEP 394 (“The “python” Command on Unix-Like Systems”), and is believed to be consistent with it in devising an approach for Windows that allows similar guidance to be given to users on all platforms.
This proposal has no impact on PEP 514 (“Python registration in the Windows registry”), and in fact improves our ability to follow it with a more flexible system for registering our own runtimes. Tools that follow PEP 514 will find any runtimes that choose to use the registration, regardless of how they were installed.
How to Teach This
Basic Use
A central goal of this proposal is that “type ‘python’ in your terminal” will be sufficient instruction for the most basic cases. Thanks to the redirector added by Microsoft, following this instruction will at least result in something useful happening, and with PyManager we can ensure that “something useful” means that the user is running the latest version.
To explain what is actually happening, we propose the following as introductory text:
Python installs on Windows are managed using an installer tool. After it has
been installed, you can run ``python`` to launch the interpreter, and it will
choose the best version already installed, available online, or referenced by
the script you are launching (if any). If you have a preference for a
particular version, you can specify it with ``py -V:<version>`` followed
by the rest of your command.
To install a version of Python without running any command, use ``py install
<version>``. You can see all of your installs with ``py list`` and remove them
with ``py uninstall <version>``. Run ``py help`` to see all the options that
are available.
Because each version of Python will be shared by all your projects, we
recommend using virtual environments. This will usually be created for a
particular Python version by running ``py -V:<version> -m venv .venv``, and
activated with ``.venv\Scripts\Activate``. Now, rather than the install
manager, ``python`` or ``py`` will always launch your virtual environment, and
any packages you install are only available while this environment is active.
To get access to the manager again, you can ``deactivate`` the environment, or
use ``py <command>``.
Many Python projects include information about how to launch their projects as part of their own README files. Historically, such information has been complicated due to the range of options available to users. We propose that, after the install manager is published, such guidance could be written along these lines:
To install and use our application, first install Python following the
guidance for your operating system at https://docs.python.org/using/. Then,
create a virtual environment and use 'pip' to install.
``python3 -m venv .venv``
``source .venv/bin/activate`` or ``.venv\Scripts\Activate`` (on Windows)
``python -m pip install OurAwesomePackage``
...
If instructions will not include information about virtual environments, then
the python
or python3
command can be shown, and on Windows both will
operate as intended for users with the install manager.
Instructions currently referring to py
for Windows can continue to do so, as
the install manager provides a practically equivalent command. Projects that
wish to provide Windows-specific instructions, such as by using the -V:
or --install
options to install the correct version, should also link to the
documentation as guidance for
ensuring that the install manager is installed.
Uninstallation
Complete uninstallation is an important topic to cover before a user is likely
to consider removing the install manager. Since not all parts of installs can be
automatically cleaned up when removing the manager, we choose not to remove most
of them. So while the default python
and py
commands will go away, all
the runtimes that were installed are still present and usable.
We suggest an explanation like this:
Before you uninstall the Python install manager, you'll want to uninstall any
runtimes that you added. This can be done easily with the "purge" option:
``py uninstall --purge``
This will remove all installs and any shortcuts that would otherwise be left
behind. If you already removed the manager, you can reinstall it and run the
above uninstall command again to clean up. Individual runtimes can be
uninstalled by replacing the ``--purge`` option with the tag, found by looking
at ``python list``.
Configuration
Configuration files are a common feature that will be documented, but do not need to be taught to regular users. Similarly, advanced deployment options, such as those that might be used by system administrators or organisations wanting their users to use a preferred index, are best covered in reference material.
Custom Index
We suggest that indexes only need to be introduced when instructing users to install a specialised runtime or distribution. Administrators seeking to provide an index are anticipated to actively seek out the relevant information in the documentation.
To explain how and when to use an alternate index, we propose text along these lines:
Our distribution can be installed on Windows using the Python Install Manager
(include link) by referencing our index:
``py install --source <your index URL here> latest``
This index contains all our versions. Use ``py list --source <URL>`` to
see everything that is available.
Reference Implementation
The reference implementation is available at the author’s repository with a precompiled MSIX package under Releases. This sample includes a bundled index, rather than a hosted one, and references a range of existing NuGet packages to allow install testing.
Reference documentation can also be found in the same repository.
Rejected Ideas
Make PyManager available on all platforms
While we are not inherently opposed to this idea, it relies on many more components being aligned before it can become possible.
Firstly, as it stands, the reference implementation has a lot of Windows-specific knowledge. Equivalent knowledge for other platforms would need to be collated and implemented, as well as any additional behaviours specific to non-Windows platforms.
Second, we need a source of pre-built, relocatable binaries that can be extracted onto the system. While such sources do exist, due to our position in the supply chain, we cannot justifiably use them (they should be using us). For Windows, our own binaries already meet these criteria, so we can repackage them without modification.
Third, the current implementation relies on a bundled Python runtime, which must be isolated from any user interference for obvious reasons. This would also require the relocatable binaries mentioned above, which we currently only have for Windows.
Due to the additional steps needed to make this functional on other platforms, and the fact that there isn’t a need to replace existing installers for those platforms, we consider this idea out of scope for this PEP. It may be pursued in the future (and the contributors most likely to do so have indicated that they are looking into it and would be able to use a consistent interface).
Include a runtime pre-installed with the manager
The proposal is to have a full Python runtime included with PyManager, so that
its python.exe
alias can refer directly to it rather than resolving to the
best available version dynamically.
It is very important for stability and updates that runtime releases are fully independent of the manager. Updating the manager should be possible without affecting any existing runtime installs, and likewise there should be no requirement to update the manager to get a newer runtime.
Hypothetically, if we were to include Python 3.14.0 with the manager such that it did not need to be installed, it would be a breaking change to later replace that with 3.15.0. As we only have a single install for the manager, this would result in the newest installs getting the oldest runtime.
This would also ignore the status of PyManager’s python
alias as being of an
unspecified version - when the user is launching this alias, it’s because they
didn’t care what version they get enough to specify one. In that situation, we
ought to select the best available, and allow them the options to stabilise it
as is appropriate (whether through a shebang or an active environment).
Include a runtime INSTEAD OF the manager
This is the current situation, which we are trying to change. If you read this far and still chose to make this argument, please go back and start again.
Use a built-in module rather than subcommands
Two alternatives to using commands like py list
or py install
that have
been proposed are to use either dedicated modules, invoked like py -m list
and py -m install
, or a single dedicated module invoked like py -m manage
list
. This idea is rejected on the basis that it attempts to reuse existing
semantics for a scenario that cannot be reliably implemented by those semantics,
and so will require a special case that is harder to explain, understand, and
maintain.
The main reason this idea is rejected is due to the interaction of two otherwise
desirable semantics: first, that the default py
command should launch the
latest available runtime as if it were launched directly; and second, that the
behaviour of -m
should not be treated as a special case in some
circumstances. If the first part were dropped, we would freely modify the
command to behave as users expect - nobody would be raising compatibility
concerns at all if we were agreed to completely break compatibility. However, if
the second constraint were dropped, users would bear the burden of the ensuring
confusion. (We aren’t proposing dropping either - this is a rejected idea, after
all - but it helps to illustrate what the options are.)
First, since one of the subcommands is intended to install your first runtime,
we cannot treat python -m [manage] install
as if it is running through the
default runtime - there isn’t one! It inherently requires special case handling
in order to read the command and execute it through a different program.
Additionally, Python allows other options to precede or mingle with the -m
,
which would have to be supported by this special case.
Finally, the semantics of the -m
option include searching the initial
sys.path
for matching module names. This is a considerably more broad search
than a bare name. py -m install
would gladly execute install.py
,
install.pyc
, install.pyd
, install\\__init__.py
, and more after
searching a number of directories found by inspecting the file system, the
environment, the registry, as well as any transitively included paths found in
those. Compared to py install
, which would only look for a file called
precisely install
in the current working directory, the -m
behaviour is
far more likely to be already relied upon by real scenarios. (For example,
Django projects typically have a manage.py
script, meaning that py -m
manage
would always behave incorrectly.)
Changing py -m install
to not behave like -m
, but instead to execute
an internal command, is vastly more likely to break users than changing
py install
. As such, this idea is rejected.
Use a new command-line option rather than subcommands
A reasonable alternative to subcommands is to specify their names with leading
punctuation, like an option rather than a subcommand. For example, this may look
like py /install ...
rather than py install
, or py --list
. Because
some of these are currently errors for a normal CPython interpreter, they
could be added without any backwards compatibility concern.
Notably, however, the typical Windows format of a leading slash is not an error in CPython. Windows users therefore cannot directly transfer existing knowledge and must learn a new way to specify options. As we are proposing a Windows specific tool, this is a terrible start. Additionally, those users familiar with Unix-style command lines will recognise the misuse of options as commands.
We desire to create a clean interface, and starting with a design that includes obvious warts or learning challenges is counter to that goal. Modern tools universally use subcommands for these purposes, and so the idea to use something different is rejected.
Improving the current traditional installer instead
Rather than creating a new install mechanism, we could invest in maintaining the current installer. At this stage, however, our current installer is based entirely on retired technology. Windows is no longer developing the Windows Installer service, and Wix are no longer improving the version of their toolset that we use. Migrating to a newer Wix Toolset is a significant amount of work, and ultimately still leaves us tied to old technologies.
As mentioned earlier, the most beneficial functionality provided by Windows Installer is not used for CPython, and generally has caused more issues than it has ever solved (for example, accidental downgrades due to automatically collected file version information).
The implementation of the Burn bundle, which is our primary source of installer logic, is in C++ and integrated into a framework that few core developers are familiar with. This makes maintenance challenging, and is not a good long term position to take. Migrating desired features such as registration-free installs into the Burn bundle is not possible (without writing the end-to-end reimplementation and integrating it as an afterthought).
Our view is that maintaining the current traditional installer is at least as much effort as implementing a new installer, and would not provide meaningful benefits for the core team or for our users. As such, this idea is rejected.
Delete the Store package completely
Removing the Store packages would reduce the number of options users face when choosing a Python runtime. By all measures apart from reliability and security, the traditional installer is entirely sufficient as a substitute. The effort to migrate parts of the ecosystem to more secure settings (such as not relying on DLL hijacking) has largely occurred, but some packages remain that still only work with less secure configurations, and moving all users back to these configurations would ensure that users of these packages would not face the issues they face today.
However, the majority of users of the Store packages appear to have no complaints. Anecdotally, they are often fully satisfied by the Store install, and particularly appreciate the ease and reliability of installation. (And on a personal note, this author has been using Store packages exclusively since Python 3.8 with no blocking issues.)
The greatest number of issues have been caused by misconfigured PATH
variables and the default python.exe
redirector installed by Microsoft. In
other words, entirely unrelated to our own package (though sometimes related to
unresolvable issues in our other installer). For the sake of the high number of
successful installs through this path, we consider the burden of diagnosing and
assisting impacted users to be worthwhile, and consider the idea to simply drop
the Store package rejected.
That said, when PyManager is published to the Store, we would plan to delist all existing runtimes on the Store to ensure users find the manager. This only impacts new installs, and anyone who has previously installed a particular version (even on another machine, if they were logged in) will be able to continue to use and install those versions.
Rely on WinGet or equivalents
WinGet, Chocolatey, and other similar tools are not installers in the sense that we require. They use their own repository of metadata to download, validate, and run installers. Without our own installer, they have nothing to run, and so cannot be used.
It is possible that their metadata will not support installing PyManager and then running it to install a particular runtime. If this is the case, they may need to investigate using our binary packages directly.
Currently, none of these install tools are officially supported by CPython, and so we have no obligation to make them work.
Make every version a Windows Store package
It is possible to release each version to the Windows Store as we currently do, but make them unlisted and rely on an installer (potentially PyManager, WinGet, or another tool that can install Store packages). This would avoid the risk of overwhelming the user, while greatly simplifying our own reponsibilities for package management.
This approach would leave a significant burden on whichever contributor has access to the Store publishing interface, as updating packages is a manual operation. Additionally, it would leave every Python runtime with the technical limitations outlined earlier. As such, this idea is rejected.
Making every version a MSIX package rather than a ZIP, even though this avoids the Store publishing interface, would still impose technical limitations on users. It is also rejected.
Just publish the plain ZIP file
Publishing the plain ZIP file is part of the plan, however, it will not be visibly listed (for example, on the python.org download pages, though they will be visible in the FTP view). An alternative would be to publish and list these packages, and expect users to download and manually extract and configure them.
Given the workflows we see, we believe that most users do not want to configure a Python install at all. Not only do they not want to choose the install location, they do not want to choose a version, or even have to search for a download provider or instructions. However, they do want to be able to find an install later, launch, update or remove it, or list all known installs.
It is also worth recognising that there will be more ZIP files than are currently listed on the Download pages, and so the list of files will become longer. Choosing the correct download is already challenging for users (those who bypass the primary “Download” button and view the list of all available versions and then files), and we have no desire to make it more challenging.
The index protocol and download list will be available for tools that wish to
use it, or for users who are willing to navigate JSON in order to find the URL.
The --target
option on the install command also provides a trivial download
and extract operation, allowing users to have the same experience as a ZIP file.
And the --download
option can give users the ZIP file still zipped.
Only publish PyManager to one place
Whether the Windows Store or python.org, it would be viable to publish to only one location.
However, users strongly expect to be able to download something from python.org. If we were to remove any option at all, we would inevitably hurt our users. Without an MSIX available on python.org, users have no way to transfer the package to another machine, or to fully script the initial install of the manager.
Many users rely on the Windows Store app to install packages, and the built-in redirector in Windows can only open to a Store page. As such, removing the Store app is equivalent to denying hundreds of thousands of installs each month.
The two builds are practically identical. The only difference between the MSIX we provide to the Store and the one that goes to python.org is package signing: we sign the python.org package ourselves, while the Store package is signed as part of the publish process. Otherwise, there is no additional cost to producing and publishing both packages.
Inline Script Metadata
PEP 723 introduced inline script metadata, a structured comment intended for third-party tools to interpret and then launch a Python script in the correct environment. An example taken from that PEP:
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "requests<3",
# "rich",
# ]
# ///
PyManager has no integrated support for installing dependencies, and does not propose adding any. As a result, we could not fully implement handling of this metadata, and as we consider partial handling to be worse than nothing, we choose not to implement any.
It is possible for a user to specify the constraint directly as an option, for
example, py -V:>=3.11 my-script.py
to get the selection behaviour.
We could also detect the metadata and warn if the selected runtime does not match its requirement, but this is not part of the initial proposal.
Open Issues
Generate packages for old versions
At present, we can install versions of Python back to 3.5 from the NuGet feed. These are identical to the traditional installer releases of the same versions, but do not include Tkinter, IDLE, documentation, or the test suite. However, it is very little work (and no additional storage space) to use them for older releases.
New binaries releases will generate new packages, even for already released versions (currently 3.12 and 3.13). So these will exist. The open issue is whether we go back through old releases and generate full PyManager packages for them all, and how far back to go. This is merely repackaging, and does not require rebuilding or modifying the release.
Include test suite in PyManager packages
The test suite consumes the majority of the size of our current distribution, but is not intended to be used by most of our users. By omitting it from the standard PyManager package, all downloads become smaller and faster, at the cost of some users having to obtain the test suite in another way (such as building from source).
It seems obviously preferable to have some way to access the exact test suite intended for a particular build. So the question is whether to include it by default (there are no optional components anymore), to publish it independently (for manual download as a ZIP file), or to publish a second feed of packages that include both the runtime and the test suite.
Footnotes
TODO: Collate references
Copyright
This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.
Source: https://github.com/python/peps/blob/main/peps/pep-0773.rst
Last modified: 2025-01-30 14:49:25 GMT