Organizations

Any package repository that allows for the creation of projects (e.g. non-mirrors) MAY offer the concept of organizations. Organizations are entities that own projects and have various users associated with them.

Organizations MAY reserve one or more namespaces. Such reservations neither confer ownership nor grant special privileges to existing projects.

Naming

A namespace MUST be a valid project name and normalized internally e.g. foo.bar would become foo-bar.

Semantics

A namespace grant bestows ownership over the following:

1. A project matching the namespace itself such as the placeholder package microsoft.
2. Projects that start with the namespace followed by a hyphen. For example, the namespace foo would match the normalized project name foo-bar but not the project name foobar.

Package name matching acts upon the normalized namespace.

Namespaces are per-package repository and SHALL NOT be shared between repositories. For example, if PyPI has a namespace microsoft that is owned by the company Microsoft, packages starting with microsoft- that come from other non-PyPI mirror repositories do not confer the same level of trust.

Grants MUST NOT overlap. For example, if there is an existing grant for foo-bar then a new grant for foo would be forbidden. An overlap is determined by comparing the normalized proposed namespace with the normalized namespace of every existing root grant. Every comparison must append a hyphen to the end of the proposed and existing namespace. An overlap is detected when any existing namespace starts with the proposed namespace.

Uploads

If the following criteria are all true for a given upload:

1. The project does not yet exist.
2. The name matches a reserved namespace.
3. The project is not owned by an organization with an active grant for the namespace.

Then the upload MUST fail with a 403 HTTP status code.

Open Namespaces

The owner of a grant may choose to allow others the ability to release new projects with the associated namespace. Doing so MUST allow uploads for new projects matching the namespace from any user.

It is possible for the owner of a namespace to both make it open and allow other organizations to use the grant. In this case, the authorized organizations have no special permissions and are equivalent to an open grant without ownership.

Hidden Grants

Repositories MAY create hidden grants that are not visible to the public which prevent their namespaces from being claimed by others. Such grants MUST NOT be open and SHOULD NOT be exposed in the API.

Hidden grants are useful for repositories that wish to enforce upload restrictions without the need to expose the namespace to the public.

Repository Metadata

The JSON API version will be incremented from 1.2 to 1.3. The following API changes MUST be implemented by repositories that support this PEP. Repositories that do not support this PEP MUST NOT implement these changes so that consumers of the API are able to determine whether the repository supports this PEP.

Grant Removal

When a reserved namespace becomes unclaimed, repositories MUST set the namespace key to null in the API.

Namespaces that were previously claimed but are now not SHOULD be eligible for claiming again by any organization.

Artifact-level Namespace Association

An earlier version of this PEP proposed that metadata be associated with individual artifacts at the point of release. This was rejected because it had the potential to cause confusion for users who would expect the namespace authorization guarantee to be at the project level based on current grants rather than the time at which a given release occurred.

Organization Scoping

The primary motivation for this PEP is to reduce dependency confusion attacks and NPM-style scoping with an allowance of the legacy flat namespace would increase the risk. If documentation instructed a user to install bar in the namespace foo then the user must be careful to install @foo/bar and not foo-bar, or vice versa. The Python packaging ecosystem has normalization rules for names in order to maximize the ease of communication and this would be a regression.

The runtime environment of Python is also not conducive to scoping. Whereas multiple versions of the same JavaScript package may coexist, Python only allows a single global namespace. Barring major changes to the language itself, this is nearly impossible to change. Additionally, users have come to expect that the package name is usually the same as what they would import and eliminating the flat namespace would do away with that convention.

Scoping would be particularly affected by organization changes which are bound to happen over time. An organization may change their name due to internal shuffling, an acquisition, or any other reason. Whenever this happens every project they own would in effect be renamed which would cause unnecessary confusion for users, frequently.

Finally, the disruption to the community would be massive because it would require an update from every package manager, security scanner, IDE, etc. New packages released with the scoping would be incompatible with older tools and would cause confusion for users along with frustration from maintainers having to triage such complaints.

Encourage Dedicated Package Repositories

Critically, this imposes a burden on projects to maintain their own infra. This is an unrealistic expectation for the vast majority of companies and a complete non-starter for community projects.

This does not help in most cases because the default behavior of most package managers is to use PyPI so users attempting to perform a simple pip install would already be vulnerable to malicious packages.

In this theoretical future every project must document how to add their repository to dependency resolution, which would be different for each package manager. Few package managers are able to download specific dependencies from specific repositories and would require users to use verbose configuration in the common case.

The ones that do not support this would instead find a given package using an ordered enumeration of repositories, leading to dependency confusion. For example, say a user wants two packages from two custom repositories X and Y. If each repository has both packages but one is malicious on X and the other is malicious on Y then the user would be unable to satisfy their requirements without encountering a malicious package.

Exclusive Reliance on Provenance Assertions

The idea here [5] would be to design a general purpose way for clients to make provenance assertions to verify certain properties of dependencies, each with custom syntax. Some examples:

• The package was uploaded by a specific organization or user name e.g. pip install "azure-loganalytics from microsoft"
• The package was uploaded by an owner of a specific domain name e.g. pip install "google-cloud-compute from cloud.google.com"
• The package was uploaded by a user with a specific email address e.g. pip install "aws-cdk-lib from contact@amazon.com"
• The package matching a namespace was uploaded by an authorized party (this PEP)

A fundamental downside is that it doesn’t play well with multiple repositories. For example, say a user wants the azure-loganalytics package and wants to ensure it comes from the organization named microsoft. If Microsoft’s organization name on PyPI is microsoft then a package manager that defaults to PyPI could accept azure-loganalytics from microsoft. However, if multiple repositories are used for dependency resolution then the user would have to specify the repository as part of the definition which is unrealistic for reasons outlined in the dedicated section on asserting package owner names.

Another general weakness with this approach is that a user attempting to perform a simple pip install without special syntax, which is the most common scenario, would already be vulnerable to malicious packages. In order to overcome this there would have to be some default trust mechanism, which in all cases would impose certain UX or resolver logic upon every tool.

For example, package managers could be changed such that the first time a package is installed the user would receive a confirmation prompt displaying the provenance details. This would be very confusing and noisy, especially for new users, and would be a breaking UX change for existing users. Many methods of installation wouldn’t work for this scenario such as running in CI or installing from a requirements file where the user would potentially be getting hundreds of prompts.

One solution to make this less disruptive for users would be to manually maintain a list of trustworthy details (organization/user names, domain names, email addresses, etc.). This could be discoverable by packages providing entry points which package managers could learn to detect and which corporate environments could install by default. This has the major downside of not providing automatic guarantees which would limit the usefulness for the average user who is more likely to be affected.

There are two ideas that could be used to provide automatic protection, which could be based on PEP 740 attestations or a new mechanism for utilizing third-party APIs that host the metadata.

First, each repository could offer a service that verifies the owner of a package using whatever criteria they deem appropriate. After verification, the repository would add the details to a dedicated package that would be installed by default.

This would require dedicated maintenance which is unrealistic for most repositories, even PyPI currently. It’s unclear how community projects without the resources for something like a domain name would be supported. Critically, this solution would cause extra confusion for users in the case of multiple repositories as each might have their own verification processes, attestation criteria and default package containing the verified details. It would be challenging to get community buy-in of every package manager to be aware of each repositories’ chosen verification package and install that by default before dependency resolution.

Should digital attestations become the chosen mechanism, a downside is that implementing this in custom package repositories would require a significant amount of work. In the case of PyPI, the prerequisite work on Trusted Publishing and then the PEP 740 implementation itself took the equivalent of a full-time engineer one year whose time was paid for by a corporate sponsor. Other organizations are unlikely to implement similar work because simpler mechanisms make it possible to implement reproducible builds. When everything is internally managed, attestations are also not very useful. Community projects are unlikely to undertake this effort because they would likely lack the resources to maintain the necessary infrastructure themselves and moreover there are significant downsides to encouraging dedicated package repositories.

The other idea would be to host provenance assertions externally and push more logic client-side. A possible implementation might be to specify a provenance API that could be hosted at a designated relative path like /provenance. Projects on each repository could then be configured to point to a particular domain and this information would be passed on to clients during installation.

While this distributed approach does impose less of an infrastructure burden on repositories, it has the potential to be a security risk. If an external provenance API is compromised, it could lead to malicious packages being installed. If an external API is down, it could lead to package installation failing or package managers might only emit warnings in which case there is no security benefit.

Additionally, this disadvantages community projects that do not have the resources to maintain such an API. They could use free hosting solutions such as what many do for documentation but they do not technically own the infrastructure and they would be compromised should the generous offerings be restricted.

Finally, while both of these theoretical approaches are not yet prescriptive, they imply assertions at the artifact level which was already a rejected idea.

Asserting Package Owner Names

This is about asserting that the package came from a specific organization or user name. It’s quite similar to the organization scoping idea except that a flat namespace is the base assumption.

This would require modifications to the JSON API of each supported repository and could be implemented by exposing extra metadata or as proper provenance assertions.

As with the organization scoping idea, a new syntax would be required like microsoft::azure-loganalytics where microsoft is the organization and azure-loganalytics is the package. Although this plays well with the existing flat namespace in comparison, it retains the critical downside of being a disruption for the community with the number of changes required.

A unique downside is that names are an implementation detail of repositories. On PyPI, the names of organizations are separate from user names so there is potential for conflicts. In the case of multiple repositories, users might run into cases of dependency confusion similar to the one at the end of the Encourage Dedicated Package Repositories rejected idea.

To ameliorate this, it was suggested that the syntax be expanded to also include the expected repository URL like microsoft@pypi.org::azure-loganalytics. This syntax or something like it is so verbose that it could lead to user confusion, and even worse, frustration should it gain increased adoption among those able to maintain dedicated infrastructure (community projects would not benefit).

The expanded syntax is an attempt to standardize resolver behavior and configuration within dependency specifiers. Not only would this be mandating the UX of tools, it lacks precedent in package managers for language ecosystems with or without the concept of package repositories. In such cases, the resolver configuration is separate from the dependency definition.

Use Fixed Prefixes

The idea here would be to have one or more top-level fixed prefixes that are used for namespace reservations:

• com-: Reserved for corporate organizations.
• org-: Reserved for community organizations.

Organizations would then apply for a namespace prefixed by the type of their organization.

This would cause perpetual disruption because when projects begin it is unknown whether a user base will be large enough to warrant a namespace reservation. Whenever that happens the project would have to be renamed which would put a high maintenance burden on the project maintainers and would cause confusion for users who have to learn a new way to reference the project’s packages. The potential for this deterring projects from reserving namespaces at all is high.

Another issue with this approach is that projects often have branding in mind (example) and would be reluctant to change their package names.

It’s unrealistic to expect every company and project to voluntarily change their existing and future package names.

Use DNS

The idea here is to add a new metadata field to projects in the API called domain-authority. Repositories would support a new endpoint for verifying the domain via HTTPS. Clients would then support options to allow certain domains.

This does not solve the problem for the target audience who do not check where their packages are coming from and is more about checking for the integrity of uploads which is already supported in a more secure way by PEP 740.

Most projects do not have a domain and could not benefit from this, unfairly favoring organizations that have the financial means to acquire one.