Organizations

Any package repository that allows for the creation of projects (e.g.non-mirrors) MAY offer the concept of organizations[6]. Organizations areentities that own projects and have various users associated with them.

Organizations MAY reserve one or more namespaces. Such reservations neitherconfer ownership nor grant special privileges to existing projects.

Naming

A namespace MUST be avalid project name andnormalized internally e.g.foo.bar would becomefoo-bar.

Semantics

A namespace grant bestows ownership over the following:

1. A project matching the namespace itself such as the placeholder packagemicrosoft.
2. Projects that start with the namespace followed by a hyphen. For example,the namespacefoo would match the normalized project namefoo-barbut not the project namefoobar.

Package name matching acts upon thenormalized namespace.

Namespaces are per-package repository and SHALL NOT be shared betweenrepositories. For example, if PyPI has a namespacemicrosoft that is ownedby the company Microsoft, packages starting withmicrosoft- that come fromother non-PyPI mirror repositories do not confer the same level of trust.

Grants MUST NOT overlap. For example, if there is an existing grantforfoo-bar then a new grant forfoo would be forbidden. An overlap isdetermined by comparing thenormalized proposed namespace with thenormalized namespace of every existing root grant. Every comparison must appenda hyphen to the end of the proposed and existing namespace. An overlap isdetected when any existing namespace starts with the proposed namespace.

Uploads

If the name of a package being uploaded matches a reserved namespace and eitherof the following criteria are true:

• The project does not yet exist.
• The project is not owned by an organization with an active grant for thenamespace.

Then the upload MUST fail with a 403 HTTP status code.

Open Namespaces

The owner of a grant may choose to allow others the ability to release newprojects with the associated namespace. Doing so MUST allowuploads for new projects matching the namespace from any user.

It is possible for the owner of a namespace to both make it open and allowother organizations to use the grant. In this case, the authorizedorganizations have no special permissions and are equivalent to an open grantwithout ownership.

Hidden Grants

Repositories MAY create hidden grants that are not visible to the public whichprevent their namespaces from being claimed by others. Such grants MUST NOT beopen and SHOULD NOT be exposed in theAPI.

Hidden grants are useful for repositories that wish to enforce uploadrestrictions without the need to expose the namespace to the public.

Repository Metadata

TheJSON API version will be incremented from1.2 to1.3.The following API changes MUST be implemented by repositories that supportthis PEP. Repositories that do not support this PEP MUST NOT implement thesechanges so that consumers of the API are able to determine whether therepository supports this PEP.

Grant Removal

When a reserved namespace becomes unclaimed, repositories MUST set thenamespace key tonull in theAPI.

Namespaces that were previously claimed but are now not SHOULD be eligible forclaiming again by any organization.

Granting Reservations to Users

As package repositories have a flat namespace, allowing any user to reserve anamespace would be untenable not just because there would becontention for a finite resource, but also because no repository has enoughhuman operators to manage the vetting of an arbitrary number of users.

Artifact-level Namespace Association

An earlier version of this PEP proposed that metadata be associated withindividual artifacts at the point of release. This was rejected because ithad the potential to cause confusion for users who would expect the namespaceauthorization guarantee to be at the project level based on current grantsrather than the time at which a given release occurred.

Organization Scoping

The primary motivation for this PEP is to reduce dependency confusion attacksand NPM-style scoping with an allowance of the legacy flat namespace wouldincrease the risk. If documentation instructed a user to installbar in thenamespacefoo then the user must be careful to install@foo/bar and notfoo-bar, or vice versa. The Python packaging ecosystem has normalizationrules for names in order to maximize the ease of communication and this wouldbe a regression.

The runtime environment of Python is also not conducive to scoping. Whereasmultiple versions of the same JavaScript package may coexist, Python onlyallows a single global namespace. Barring major changes to the language itself,this is nearly impossible to change. Additionally, users have come to expectthat the package name is usually the same as what they would import andeliminating the flat namespace would do away with that convention.

Scoping would be particularly affected by organization changes which are boundto happen over time. An organization may change their name due to internalshuffling, an acquisition, or any other reason. Whenever this happens everyproject they own would in effect be renamed which would cause unnecessaryconfusion for users, frequently.

Finally, the disruption to the community would be massive because it wouldrequire an update from every package manager, security scanner, IDE, etc. Newpackages released with the scoping would be incompatible with older tools andwould cause confusion for users along with frustration from maintainers havingto triage such complaints.

Encourage Dedicated Package Repositories

Critically, this imposes a burden on projects to maintain their own infra. Thisis an unrealistic expectation for the vast majority of companies and a completenon-starter for community projects.

This does not help in most cases because the default behavior of most packagemanagers is to use PyPI so users attempting to perform a simplepipinstallwould already be vulnerable to malicious packages.

In this theoretical future every project must document how to add theirrepository to dependency resolution, which would be different for each packagemanager. Few package managers are able to download specific dependencies fromspecific repositories and would require users to use verbose configuration inthe common case.

The ones that do not support this would instead find a given package using anordered enumeration of repositories, leading to dependency confusion.For example, say a user wants two packages from two custom repositoriesXandY. If each repository has both packages but one is malicious onXand the other is malicious onY then the user would be unable to satisfytheir requirements without encountering a malicious package.

Exclusive Reliance on Provenance Assertions

The idea here[5] would be to design a general purpose way for clients to makeprovenance assertions to verify certain properties of dependencies, each withcustom syntax. Some examples:

• The package was uploaded by a specific organization or user name e.g.pipinstall"azure-loganalyticsfrommicrosoft"
• The package was uploaded by an owner of a specific domain name e.g.pipinstall"google-cloud-computefromcloud.google.com"
• The package was uploaded by a user with a specific email address e.g.pipinstall"aws-cdk-libfromcontact@amazon.com"
• The package matching a namespace was uploaded by an authorized party (thisPEP)

A fundamental downside is that it doesn’t play well with multiplerepositories. For example, say a user wants theazure-loganalytics packageand wants to ensure it comes from the organization namedmicrosoft. IfMicrosoft’s organization name on PyPI ismicrosoft then a package managerthat defaults to PyPI could acceptazure-loganalyticsfrommicrosoft.However, if multiple repositories are used for dependency resolution then theuser would have to specify the repository as part of the definition which isunrealistic for reasons outlined in the dedicated section onasserting package owner names.

Another general weakness with this approach is that a user attempting toperform a simplepipinstall without special syntax, which is the mostcommon scenario, would already be vulnerable to malicious packages. In order toovercome this there would have to be some default trust mechanism, which in allcases would impose certain UX or resolver logic upon every tool.

For example, package managers could be changed such that the first time apackage is installed the user would receive a confirmation prompt displayingthe provenance details. This would be very confusing and noisy, especially fornew users, and would be a breaking UX change for existing users. Many methodsof installation wouldn’t work for this scenario such as running in CI orinstalling from a requirements file where the user would potentially be gettinghundreds of prompts.

One solution to make this less disruptive for users would be to manuallymaintain a list of trustworthy details (organization/user names, domain names,email addresses, etc.). This could be discoverable by packages providingentry points which package managers could learn to detect and whichcorporate environments could install by default. This has the major downside ofnot providing automatic guarantees which would limit the usefulness for theaverage user who is more likely to be affected.

There are two ideas that could be used to provide automatic protection, whichcould be based onPEP 740 attestations or a new mechanism for utilizingthird-party APIs that host the metadata.

First, each repository could offer a service that verifies the owner of apackage using whatever criteria they deem appropriate. After verification, therepository would add the details to a dedicated package that would be installedby default.

This would require dedicated maintenance which is unrealistic for mostrepositories, even PyPI currently. It’s unclear how community projects withoutthe resources for something like a domain name would be supported. Critically,this solution would cause extra confusion for users in the case of multiplerepositories as each might have their own verification processes, attestationcriteria and default package containing the verified details. It would bechallenging to get community buy-in of every package manager to be aware ofeach repositories’ chosen verification package and install that by defaultbefore dependency resolution.

Should digital attestations become the chosen mechanism, a downside is thatimplementing this in custom package repositories would require a significantamount of work. In the case of PyPI, the prerequisite work onTrusted Publishing and then thePEP 740 implementation itself took theequivalent of a full-time engineer one year whose time was paid for by acorporate sponsor. Other organizations are unlikely to implement similar workbecause simpler mechanisms make it possible to implement reproducible builds.When everything is internally managed, attestations are also not very useful.Community projects are unlikely to undertake this effort because they wouldlikely lack the resources to maintain the necessary infrastructure themselvesand moreover there are significant downsides toencouraging dedicated package repositories.

The other idea would be to host provenance assertions externally and push morelogic client-side. A possible implementation might be to specify a provenanceAPI that could be hosted at a designated relative path like/provenance. Projects on each repository could then be configured to pointto a particular domain and this information would be passed on to clientsduring installation.

While this distributed approach does impose less of an infrastructure burden onrepositories, it has the potential to be a security risk. If an externalprovenance API is compromised, it could lead to malicious packages beinginstalled. If an external API is down, it could lead to package installationfailing or package managers might only emit warnings in which case there is nosecurity benefit.

Additionally, this disadvantages community projects that do not have theresources to maintain such an API. They could use free hosting solutions suchas what many do for documentation but they do not technically own theinfrastructure and they would be compromised should the generous offerings berestricted.

Finally, while both of these theoretical approaches are not yet prescriptive,they imply assertions at the artifact level which was already arejected idea.

Asserting Package Owner Names

This is about asserting that the package came from a specific organization oruser name. It’s quite similar to theorganization scoping idea except that a flatnamespace is the base assumption.

This would require modifications to theJSON API of each supportedrepository and could be implemented by exposing extra metadata or as properprovenance assertions.

As with the organization scoping idea, a newsyntax would be required likemicrosoft::azure-loganalytics wheremicrosoft is the organization andazure-loganalytics is the package. Although this plays well with theexisting flat namespace in comparison, it retains the critical downside ofbeing a disruption for the community with the number of changes required.

A unique downside is that names are an implementation detail of repositories.On PyPI, the names of organizations are separate from user names so there ispotential for conflicts. In the case of multiple repositories, users might runinto cases of dependency confusion similar to the one at the end of theEncourage Dedicated Package Repositoriesrejected idea.

To ameliorate this, it was suggested that the syntax be expanded to alsoinclude the expected repository URL likemicrosoft@pypi.org::azure-loganalytics. This syntax or something like itis so verbose that it could lead to user confusion, and even worse, frustrationshould it gain increased adoption among those able to maintain dedicatedinfrastructure (community projects would not benefit).

The expanded syntax is an attempt to standardize resolver behavior andconfiguration within dependency specifiers. Not only would this be mandatingthe UX of tools, it lacks precedent in package managers for language ecosystemswith or without the concept of package repositories. In such cases, theresolver configuration is separate from the dependency definition.

Use Fixed Prefixes

The idea here would be to have one or more top-level fixed prefixes that areused for namespace reservations:

• com-: Reserved for corporate organizations.
• org-: Reserved for community organizations.

Organizations would then apply for a namespace prefixed by the type of theirorganization.

This would cause perpetual disruption because when projects begin it is unknownwhether a user base will be large enough to warrant a namespace reservation.Whenever that happens the project would have to be renamed which would put ahigh maintenance burden on the project maintainers and would cause confusionfor users who have to learn a new way to reference the project’s packages.The potential for this deterring projects from reserving namespaces at all ishigh.

Another issue with this approach is that projects often have branding in mind(example) and would be reluctant to change their package names.

It’s unrealistic to expect every company and project to voluntarily changetheir existing and future package names.

Use DNS

Theidea here is to add a newmetadata field to projects in the API calleddomain-authority. Repositorieswould support a new endpoint for verifying the domain via HTTPS. Clients wouldthen support options to allow certain domains.

This does not solve the problem for the target audience who do not check wheretheir packages are coming from and is more about checking for the integrity ofuploads which is already supported in a more secure way byPEP 740.

Most projects do not have a domain and could not benefit from this, unfairlyfavoring organizations that have the financial means to acquire one.