Rationale¶

Historically pip has not had any dependencies except forsetuptools itself,choosing instead to implement any functionality it needed to prevent needinga dependency. However, starting with pip 1.5, we began to replace code that wasimplemented inside of pip with reusable libraries from PyPI. This brought thetypical benefits of reusing libraries instead of reinventing the wheel likehigher quality and more battle tested code, centralization of bug fixes(particularly security sensitive ones), and better/more features for less work.

However, there are several issues with having dependencies in the traditionalway (viainstall_requires) for pip. These issues are:

Fragility

When pip depends on another library to function then if for whatever reasonthat library either isn’t installed or an incompatible version is installedthen pip ceases to function. This is of course true for all Pythonapplications, however for every applicationexcept for pip the way you fixit is by re-running pip. Obviously, when pip can’t run, you can’t use pip tofix pip, so you’re left having to manually resolve dependencies andinstalling them by hand.

Making other libraries uninstallable

One of pip’s current dependencies is therequests library, for which piprequires a fairly recent version to run. If pip depended onrequests inthe traditional manner, then we’d either have to maintain compatibility witheveryrequests version that has ever existed (and ever will), OR allowpip to render certain versions ofrequests uninstallable. (The secondissue, although technically true for any Python application, is magnified bypip’s ubiquity; pip is installed by default in Python, inpyvenv, and invirtualenv.)

Security

This might seem puzzling at first glance, since vendoring has a tendency tocomplicate updating dependencies for security updates, and that holds truefor pip. However, given theother reasons for avoiding dependencies, thealternative is for pip to reinvent the wheel itself. This is what pip didhistorically. It forced pip to re-implement its own HTTPS verificationroutines as a workaround for the Python standard library’s lack of SSLvalidation, which resulted in similar bugs in the validation routine inrequests andurllib3, except that they had to be discovered andfixed independently. Even though we’re vendoring, reusing libraries keepspip more secure by relying on the great work of our dependencies,andallowing for faster, easier security fixes by simply pulling in newerversions of dependencies.

Bootstrapping

Currently most popular methods of installing pip rely on pip’sself-contained nature to install pip itself. These tools work by bundling acopy of pip, adding it tosys.path, and then executing that copy of pip.This is done instead of implementing a “mini installer” (to reduceduplication); pip already knows how to install a Python package, and is farmore battle-tested than any “mini installer” could ever possibly be.

Many downstream redistributors have policies against this kind of bundling, andinstead opt to patch the software they distribute to debundle it and make itrely on the global versions of the software that they already have packaged(which may have its own patches applied to it). We (the pip team) would preferit if pip wasnot debundled in this manner due to the above reasons andinstead we would prefer it if pip would be left intact as it is now.

In the longer term, if someone has aportable solution to the above problems,other than the bundling method we currently use, that doesn’t add additionalproblems that are unreasonable then we would be happy to consider, and possiblyswitch to said method. This solution must function correctly across all of thesituation that we expect pip to be used and not mandate some external mechanismsuch as OS packages.

Modifications¶

• setuptools is completely stripped to only keeppkg_resources.

• pkg_resources has been modified to import its dependencies frompip._vendor, and to use the vendored copy ofplatformdirsrather thanappdirs.

• packaging has been modified to import its dependencies frompip._vendor.

• CacheControl has been modified to import its dependencies frompip._vendor.

• requests has been modified to import its other dependencies frompip._vendor and tonot loadsimplejson (all platforms) andpyopenssl (Windows).

• platformdirs has been modified to import its submodules frompip._vendor.platformdirs.

Automatic Vendoring¶

Vendoring is automated via thevendoring tool from the content ofpip/_vendor/vendor.txt and the different patches intools/vendoring/patches.Launch it viavendoringsync.-v (requiresvendoring>=0.2.2).Tool configuration is done viapyproject.toml.

To update the vendored library versions, we have a session defined innox.The command to upgrade everything is:

nox-svendoring----upgrade-all--skipurllib3--skipsetuptools

At the time of writing (April 2025) we do not upgradeurllib3 because thenext version is a major upgrade and will be handled as an independent PR. We alsodo not upgradesetuptools, because we only rely onpkg_resources, andtracking everysetuptools change is unnecessary for our needs.

Managing Local Patches¶

Thevendoring tool automatically applies our local patches, but updating,the patches sometimes no longer apply cleanly. In that case, the update willfail. To resolve this, take the following steps:

1. Revert any incomplete changes in the revendoring branch, to ensure you havea clean starting point.

2. Run the revendoring of the library with a problem again:nox-svendoring----upgrade<library_name>.

3. This will fail again, but you will have the original source in your workingdirectory. Review the existing patch against the source, and modify the patchto reflect the new version of the source. If yougitadd the changes thevendoring made, you can modify the source to reflect the patch file and thengenerate a new patch withgitdiff.

4. Now, revert everythingexcept the patch file changes. Leave the modifiedpatch file unstaged but saved in the working tree.

5. Re-run the vendoring. This time, it should pick up the changed patch fileand apply it cleanly. The patch file changes will be committed along with therevendoring, so the new commit should be ready to test and publish as a PR.

Debundling¶

As mentioned in the rationale, we, the pip team, would prefer it if pip was notdebundled (other than optionallypip/_vendor/requests/cacert.pem) and thatpip was left intact. However, if you insist on doing so, we have asemi-supported method (that we don’t test in our CI) and requires a bit ofextra work on your end in order to solve the problems described above.

1. Delete everything inpip/_vendor/except forpip/_vendor/__init__.py andpip/_vendor/vendor.txt.

2. Generate wheels for each of pip’s dependencies (and any of theirdependencies) using your patched copies of these libraries. These must beplaced somewhere on the filesystem that pip can access (pip/_vendor isthe default assumption).

3. Modifypip/_vendor/__init__.py so that theDEBUNDLED variable isTrue.

4. Upon installation, theINSTALLER file in pip’s owndist-infodirectory should be set to something other thanpip, so that pipcan detect that it wasn’t installed using itself.

5. (optional) If you’ve placed the wheels in a location other thanpip/_vendor/, then modifypip/_vendor/__init__.py so that theWHEEL_DIR variable points to the location you’ve placed them.

6. (optional) Update thepip_self_version_check logic to use theappropriate logic for determining the latest available version of pip andprompt the user with the correct upgrade message.

Note that partial debundling isNOT supported. You need to prepare wheelsfor all dependencies for successful debundling.