Project
pip is capable of determining and installing the dependencies of packages. Theprocess of determining which version of a dependency to install is known asdependency resolution. This behaviour can be disabled by passing--no-deps
topip install.
When a user does apipinstall
(e.g.pipinstalltea
), pip needs to workout the package’s dependencies (e.g.spoon
,hot-water
,tea-leaves
etc.)and what the versions of each of those dependencies it should install.
At the start of apipinstall
run, pip does not have all the dependencyinformation of the requested packages. It needs to work out the dependenciesof the requested packages, the dependencies of those dependencies, and so on.Over the course of the dependency resolution process, pip will need to downloaddistribution files of the packages which are used to get the dependencies of apackage.
Changed in version 20.3:pip’s dependency resolver is now capable of backtracking.
During dependency resolution, pip needs to make assumptions about the packageversions it needs to install and, later, check these assumptions were notincorrect. When pip finds that an assumption it made earlier is incorrect, ithas to backtrack, which means also discarding some of the work that has alreadybeen done, and going back to choose another path.
This can look like pip downloading multiple versions of the same package,since pip explicitly presents each download to the user. The backtracking ofchoices made during this step is not unexpected behaviour or a bug. It is partof how dependency resolution for Python packages works.
Example
The user requestspipinstalltea
. The packagetea
declares a dependency onhot-water
,spoon
,cup
, amongst others.
pip starts by picking the most recent version oftea
and gets the list ofdependencies of that version oftea
. It will then repeat the process forthose packages, picking the most recent version ofspoon
and thencup
. Now,pip notices that the version ofcup
it has chosen is not compatible with theversion ofspoon
it has chosen. Thus, pip will “go back” (backtrack) and tryto use another version ofcup
. If it is successful, it will continue onto thenext package (likesugar
). Otherwise, it will continue to backtrack oncup
until it finds a version ofcup
that is compatible with all the otherpackages.
This can look like:
$pipinstallteaCollecting tea Downloading tea-1.9.8-py2.py3-none-any.whl (346 kB) |████████████████████████████████| 346 kB 10.4 MB/sCollecting spoon==2.27.0 Downloading spoon-2.27.0-py2.py3-none-any.whl (312 kB) |████████████████████████████████| 312 kB 19.2 MB/sCollecting cup>=1.6.0 Downloading cup-3.22.0-py2.py3-none-any.whl (397 kB) |████████████████████████████████| 397 kB 28.2 MB/sINFO: pip is looking at multiple versions of this package to determinewhich version is compatible with other requirements.This could take a while. Downloading cup-3.21.0-py2.py3-none-any.whl (395 kB) |████████████████████████████████| 395 kB 27.0 MB/s Downloading cup-3.20.0-py2.py3-none-any.whl (394 kB) |████████████████████████████████| 394 kB 24.4 MB/s Downloading cup-3.19.1-py2.py3-none-any.whl (394 kB) |████████████████████████████████| 394 kB 21.3 MB/s Downloading cup-3.19.0-py2.py3-none-any.whl (394 kB) |████████████████████████████████| 394 kB 26.2 MB/s Downloading cup-3.18.0-py2.py3-none-any.whl (393 kB) |████████████████████████████████| 393 kB 22.1 MB/s Downloading cup-3.17.0-py2.py3-none-any.whl (382 kB) |████████████████████████████████| 382 kB 23.8 MB/s Downloading cup-3.16.0-py2.py3-none-any.whl (376 kB) |████████████████████████████████| 376 kB 27.5 MB/s Downloading cup-3.15.1-py2.py3-none-any.whl (385 kB) |████████████████████████████████| 385 kB 30.4 MB/sINFO: pip is looking at multiple versions of this package to determinewhich version is compatible with other requirements.This could take a while. Downloading cup-3.15.0-py2.py3-none-any.whl (378 kB) |████████████████████████████████| 378 kB 21.4 MB/s Downloading cup-3.14.0-py2.py3-none-any.whl (372 kB) |████████████████████████████████| 372 kB 21.1 MB/s
These multipleDownloadingcup-{version}
lines show that pip is backtrackingchoices it is making during dependency resolution.
If pip starts backtracking during dependency resolution, it does not know howmany choices it will reconsider, and how much computation would be needed.
For the user, this means it can take a long time to complete when pip startsbacktracking. In the case where a package has a lot of versions, arriving at agood candidate can take a lot of time. The amount of time depends on thepackage size, the number of versions pip must try, and various other factors.
Backtracking reduces the risk that installing a new package will accidentallybreak an existing installed package, and so reduces the risk that yourenvironment gets messed up. To do this, pip has to do more work, to find outwhich version of a package is a good candidate to install.
There is no one-size-fits-all answer to situations where pip is backtrackingexcessively during dependency resolution. There are ways to reduce thedegree to which pip might backtrack though. Nearly all of these approachesrequire some amount of trial and error.
In most cases, pip will complete the backtracking process successfully.This could take a very long time to complete, so this may not be yourpreferred option.
However, it is a possible that pip will not be able to find a set ofcompatible versions. For this, pip will try every possible combination thatit needs to and determine that there is no compatible set.
If you’d prefer not to wait, you can interrupt pip (Ctrl+c) and try thestrategies listed below.
It is usually a good idea to add constraints the package(s) that pip is backtracking on (e.g. in the above example -cup
).
You could try something like:
$python-mpipinstalltea"cup >= 3.13"
$python-mpipinstalltea"cup >= 3.13"
C:> py -m pip install tea"cup >= 3.13"
This will reduce the number of versions ofcup
it tries, andpossibly reduce the time pip takes to install.
There is a possibility that the addition constraint is incorrect. When thishappens, the reduced search space makes it easier for pip to more quicklydetermine what caused the conflict and present that to the user. It could alsoresult in pip backtracking on a different package due to some other conflict.
This option is a progression of the previous section. It requires users to knowhow to inspect:
the packages they’re trying to install
the package release frequency and compatibility policies
their release notes and changelogs from past versions
During deployment, you can create a lockfile stating the exact package andversion number for each dependency of that package. You can create thiswithpip-tools.
This means the “work” is done once during development process, and thuswill avoid performing dependency resolution during deployment.
This section provides practical suggestions to pip users who encounteraResolutionImpossible
error, where pip cannot install their specifiedpackages due to conflicting dependencies.
When you get aResolutionImpossible
error, you might see somethinglike this:
$python-mpipinstallpackage_coffee==0.44.1package_tea==4.3.0[regular pip output]ERROR: Cannot install package_coffee==0.44.1 and package_tea==4.3.0 because these package versions have conflicting dependencies.The conflict is caused by: package_coffee 0.44.1 depends on package_water<3.0.0,>=2.4.2 package_tea 4.3.0 depends on package_water==2.3.1
$python-mpipinstallpackage_coffee==0.44.1package_tea==4.3.0[regular pip output]ERROR: Cannot install package_coffee==0.44.1 and package_tea==4.3.0 because these package versions have conflicting dependencies.The conflict is caused by: package_coffee 0.44.1 depends on package_water<3.0.0,>=2.4.2 package_tea 4.3.0 depends on package_water==2.3.1
C:> py -m pip install package_coffee==0.44.1 package_tea==4.3.0[regular pip output]ERROR: Cannot install package_coffee==0.44.1 and package_tea==4.3.0 because these package versions have conflicting dependencies.The conflict is caused by: package_coffee 0.44.1 depends on package_water<3.0.0,>=2.4.2 package_tea 4.3.0 depends on package_water==2.3.1
In this example, pip cannot install the packages you have requested,because they each depend on different versions of the same package(package_water
):
package_coffee
version0.44.1
depends on a version ofpackage_water
that is less than3.0.0
but greater than or equal to2.4.2
package_tea
version4.3.0
depends on version2.3.1
ofpackage_water
Sometimes these messages are straightforward to read, because they usecommonly understood comparison operators to specify the required version(e.g.<
or>
).
However, Python packaging also supports some more complex ways forspecifying package versions (e.g.~=
or*
):
Operator | Description | Example |
---|---|---|
| Any version greater than the specified version. |
|
| Any version less than the specified version. |
|
| Any version less than or equal to the specified version. |
|
| Any version greater than or equal to the specified version. |
|
| Exactly the specified version. |
|
| Any version not equal to the specified version. |
|
| Any compatible1 version. |
|
| Can be used at the end of a version number to representall. |
|
1 Compatible versions are higher versions that only differ in the final segment.~=3.1.2
is equivalent to>=3.1.2,==3.1.*
.~=3.1
is equivalent to>=3.1,==3.*
.
The detailed specification of supported comparison operators can befound inPEP 440.
The solution to your error will depend on your individual use case. Hereare some things to try:
As a first step, it is useful to audit your project and remove anyunnecessary or out of date requirements (e.g. from yoursetup.py
orrequirements.txt
files). Removing these can significantly reduce thecomplexity of your dependency tree, thereby reducing opportunities forconflicts to occur.
Sometimes the packages that you have asked pip to install areincompatible because you have been too strict when you specified thepackage version.
In our first example bothpackage_coffee
andpackage_tea
have beenpinned to use specific versions(package_coffee==0.44.1package_tea==4.3.0
).
To find a version of bothpackage_coffee
andpackage_tea
that depend onthe same version ofpackage_water
, you might consider:
Loosening the range of packages that you are prepared to install(e.g.pipinstall"package_coffee>0.44""package_tea>4.0.0"
)
Asking pip to installany version ofpackage_coffee
andpackage_tea
by removing the version specifiers altogether (e.g.pipinstallpackage_coffeepackage_tea
)
In the second case, pip will automatically find a version of bothpackage_coffee
andpackage_tea
that depend on the same version ofpackage_water
, installing:
package_coffee0.44.1
, which depends onpackage_water2.6.1
package_tea4.4.3
whichalso depends onpackage_water2.6.1
If you want to prioritize one package over another, you can add versionspecifiers toonly the more important package:
$python-mpipinstallpackage_coffee==0.44.1package_tea
$python-mpipinstallpackage_coffee==0.44.1package_tea
C:> py -m pip install package_coffee==0.44.1 package_tea
This will result in:
package_coffee0.44.1
, which depends onpackage_water2.6.1
package_tea4.4.3
whichalso depends onpackage_water2.6.1
Now that you have resolved the issue, you can repin the compatiblepackage versions as required.
Assuming that you cannot resolve the conflict by loosening the versionof the package you require (as above), you can try to fix the issue onyourdependency by:
Requesting that the package maintainers loosentheir dependencies
Forking the package and loosening the dependencies yourself
Warning
If you choose to fork the package yourself, you areopting out ofany support provided by the package maintainers. Proceed at your own risk!
Sometimes it’s simply impossible to find a combination of packageversions that do not conflict. Welcome todependency hell.
In this situation, you could consider:
Using an alternative package, if that is acceptable for your project.SeeAwesome Python for similar packages.
Refactoring your project to reduce the number of dependencies (forexample, by breaking up a monolithic code base into smaller pieces).
Sometimes pip’s dependency resolver may exceed its search depth and terminatewith aResolutionTooDeepError
exception. This typically occurs when thedependency graph is extremely complex or when there are too many packageversions to evaluate.
To address this error, consider the following strategies:
By setting a higher lower bound for your dependencies, you narrow the searchspace. This excludes older versions that might trigger excessive backtracking.For example:
$python-mpipinstall"package_coffee>=0.44.0""package_tea>=4.0.0"
$python-mpipinstall"package_coffee>=0.44.0""package_tea>=4.0.0"
C:> py -m pip install"package_coffee>=0.44.0""package_tea>=4.0.0"
--upgrade
Flag¶The--upgrade
flag directs pip to ignore already installed versions andsearch for the latest versions that meet your requirements. This can helpavoid unnecessary resolution paths:
$python-mpipinstall--upgradepackage_coffeepackage_tea
$python-mpipinstall--upgradepackage_coffeepackage_tea
C:> py -m pip install --upgrade package_coffee package_tea
If you need to impose additional version restrictions on transitivedependencies (dependencies of dependencies), consider using a constraintfile. A constraint file specifies version limits for packages that areindirectly required. For example:
# constraints.txtindirect_dependency>=2.0.0
Then install your packages with:
$python-mpipinstall--constraintconstraints.txtpackage_coffeepackage_tea
$python-mpipinstall--constraintconstraints.txtpackage_coffeepackage_tea
C:> py -m pip install --constraint constraints.txt package_coffee package_tea
Although upper bounds are generally discouraged because they can complicatedependency management, they may be necessary when certain versions are knownto cause conflicts. Use them cautiously—for example:
$python-mpipinstall"package_coffee>=0.44.0,<1.0.0""package_tea>=4.0.0"
$python-mpipinstall"package_coffee>=0.44.0,<1.0.0""package_tea>=4.0.0"
C:> py -m pip install"package_coffee>=0.44.0,<1.0.0""package_tea>=4.0.0"
If you encounter aResolutionTooDeep
error consider reporting it, to helpthe pip team have real world examples to test against, at the dedicatedpip issue.
If none of the suggestions above work for you, we recommend that you askfor help on:
See“How do I ask a good question?” for tips on asking for help.
Unfortunately,the pip team cannot provide support for individualdependency conflict errors. Pleaseonly open a ticket onpip’s issue tracker if you believethat your problem has exposed a bug in pip.