33
44Add note here to explain that this does_ not_ prevent_ nor_ discourage library
55author to release 2 version of their software one Python 3 only and the other
6- python 2.
6+ python 2.
77
8- This actually made the above easier and less-likely to break.
98
9+ This page gather information and links to resources allowing to release a
10+ library that stop supporting an older version of Python without causing too
11+ much disruption for users who haven't upgraded to this new version.
1012
11- Too long, did not read:
13+ Wether you are a user, or a developer, being aware of the issue listed here, at
14+ least the main points should ease lots of the pain.
15+
16+ #Too long, did not read:
1217
1318- Help and encourage users to install** pip 9.0+**
1419- Help and encourage users to install** setuptools 24.3+**
15- - Use** ` setup(..., python_requires='>=3.3') ` ** new option.
20+ - As maintainer use` setup(..., python_requires='>=3.4') ` new option.
21+ - do use` pip install [-e] . ` and do** not** invoke` setup.py ` directly.
1622- ** Fail** early at** install time** if on Python 2.
1723
24+ ##The problem
25+
26+ Up until December 2016 it was hard to publish a new major version of library
27+ that changed requirements in Python version and mark it as such so that user
28+ system will not try to upgrade said library.
29+
30+ With the recent changes in Python packaging this is now possible.
31+
32+ As an example let's look at the example of the` fictious ` library.
33+
34+ - ` fictious ` 1.1, 1.2, 1.3, 1.4 are compatible Python 2.7 and 3.3+
35+ - ` fictious ` 2.0 has been released and is python 3.4+ only.
1836
19- #As a user
37+ As a Python 2.7 user, if I don't pay attention, or if the library is not
38+ correctly tagged, if I issue the following:
2039
21- ##Install Pip 9.0
40+ $ python -c 'import fictious; print(fictious.__version__)'
41+ 1.3.2
42+ $ pip install fiction --upgrade
43+
44+ Either my system will install 2.0, which will not work, on the worst case
45+ scenario, or fail to install, in which case I will not get the critical 1.4
46+ upgrade.
47+
48+ ##As a user
49+
50+ ###Install Pip 9.0
2251
2352If you are already a Python 3 user, you should not encounter a lot of
2453disruption. Please still check that the libraries you use follow best practices
2554not to break for Python 2 users. Python is a community regardless of which
26- python version you have toa decide to run, making sure thatthings works make
27- the community strong.
55+ python version you have to(or decided to) run, making sure thateverything
56+ works make the community strong.
2857
2958Make sure you have Pip >= 9.0, this is especially important if you have Python
30- 2 installations. Having pip 9.0+ will not insure that you install will not
31- break, but they are more likely not to. Having a version off pip < 9.0 can lead
32- your system to try to upgrade to non-compatible versions of Python packages
33- even if these are marked as non-compatible.
59+ 2 installations. Having pip 9.0+ is not a guaranty to flawless upgrade. But pip
60+ 9.0+ does have a number of safety check not available on previous versions.
61+
62+ Having a version of pip < 9.0 can lead your system to try to upgrade to
63+ non-compatible versions of Python packages even if these are marked as
64+ non-compatible.
3465
3566Help as many other_ users_ as possible to install pip >=9.0, for the
3667transition, it is the slowest part of the ecosystem to update, and is the only
37- piece thatconcerns allinstallations .
68+ piece thatrequires action of allPython users .
3869
3970The simplest way to make sure all is up to date is to run the following for
4071each installation of Python:
4172
42- pip install --upgrade setuptools pip
73+ $ pip install --upgrade setuptools pip
4374
4475This will install the latest version of pip and setuptools.
4576
4677You can issue the following to see the version of pip:
4778
48- pip --version
79+ $ pip --version
80+ 9.0.0
81+
82+ All good.
4983
5084
5185
5286##Setuptools
5387
54- If you are on a system that will not install python wheel and use` setuptools ` ,
55- make sure you have setuptools >=24.2.0, or building Python 3 only libraries
56- might fail. In particular if authors have taken time to mark their library as
57- Python 3 only, the` python_requires ` argument to` setup() ` will not be
58- recognize and installation will fail.
88+ If you are on a system for which no wheel is available, pip will try to
89+ install a source distribution (aka` sdist ` ).
90+
91+ Installing an` sdist ` will require setuptools make sure you have setuptools
92+ ` >=24.2.0 ` (mnemonic: 2-42,[ why
93+ 42] ( https://en.wikipedia.org/wiki/The_answer_to_life_the_universe_and_everything )
94+ ?) or building Python 3 only libraries is likely to fail. In particular if
95+ library authors have taken time to mark their library as Python 3 only, the
96+ ` python_requires ` argument to` setup() ` may not be recognized and installation
97+ will fail.
5998
6099Use the following to check setuptools version :
61100
62- python -c 'import setuptools; print(setuptools.__version__)
101+ $ python -c 'import setuptools; print(setuptools.__version__)
102+ 24.2.0
63103
64- Again make suretu upgrade pip and setuptools to make sure you have an up to
104+ Again make sureto upgrade pip and setuptools to make sure you have an up to
65105date system:
66106
67- pip install --upgrade setuptools pip
107+ $ pip install --upgrade setuptools pip
68108
69109##Local package index
70110
71111If you are using a custom local package index, for example if you are working
72112at a company with private packages, make sure it implement correctly
73- [ pep-503] (https://www.gg python.org/dev/peps/pep-0503/) and let pip knows about
74- the` python_requires ` field.
113+ [ pep-503] ( https://www.python.org/dev/peps/pep-0503/ ) and let pip knows about
114+ the` python_requires ` field. This_ mostly_ mean that the html you are exposing
115+ should get a` data-python-requires ` data attribute with the (html escaped)
116+ version specifier.
75117
76118##The state of PyPI
77119
78120Note that at the time of this writing the patches to` pypi.python.org ` are not
79121deployed yet but should hopefully be deployed soon.
80122
123+ [ Warehouse] ( https://github.com/pypi/warehouse ) and[ Legacy
124+ PyPI] ( https://github.com/pypa/legacy-pypi ) have received various patches to
125+ insure they support this new functionality.
126+
127+ You can publish a package with the` requires_python ` metadata** now** , it will
128+ be correctly exposed once pypi is deployed.
81129
82130#Preparing your library
83131
@@ -109,65 +157,99 @@ setup(
109157```
110158
111159Changes` >=3.3 ` accordingly depending on what version your library decides to
112- support.
160+ support. In particular you can use` >=2.6 ` or` >=3.5 ` ! Note that this also
161+ support the_ compable with_ syntax:` ~=2.5 ` (meaning,` >=2.5 ` and` <3 ` .
113162
114163This will make[ PyPI aware] ( https://github.com/pypa/warehouse/pull/1448 ) that
115- your package is Python 3 only, and[ allow
164+ your package is Python 3.3+ only, and[ allow
116165pip] ( https://github.com/pypa/pip/pull/3877 ) to be[ made aware of
117166this] ( https://github.com/pypa/pypi-legacy/pull/506 ) .
118167
168+ Thus as long as your user have a recent enough version of pip, and setuptools
169+ they will get the right version of your library.
170+
171+ #Unit Testing and documentation
172+
173+ It is recommended** not** to invoke` setup.py ` directly either with` install ` or
174+ ` develop ` subcommands. These may not correctly resolve dependencies, and can
175+ install incompatible versions of dependencies. Please recommend and use `pip
176+ install .` and ` pip install -e .` for regular and developer install.
177+
178+ Check in scripts, and documentation that the correct installation command is
179+ used.
180+
181+ #Recommended Mitigations
182+
183+ These are not mandatory but should make the transition seamless by warning your
184+ user early enough_ and_ providing useful error messages.
119185
120- - Add a warning at_ runtime_ early on master (before switching to Python 3
186+ ##Runtime warning on master
187+
188+ Add a warning at_ runtime_ early on master (before switching to Python 3
121189 only)
122190
123191```
124192import warnings
125193import sys
126194if sys.version_info < (3,):
127- warnings.warn('You are using master of `Frobulator` with Python 2. Frobulator will soon be Python 3 only. See this issue to know more.', UserWarning)
128- else:
129-
195+ warnings.warn('You are using master of `Frobulator` with Python 2. '
196+ 'Frobulator will soon be Python 3 only. '
197+ 'See this issue to know more.',
198+ UserWarning)
130199```
131200
132- - Add an error early at import at runtime with a clear error message, leave the
133- early import compatible Python 2 for users to not be welcomed with a useless` SyntaxError ` .
134- You are_ allowed_ to use multi-line strings in error messages.
201+ Your Python 2 user have a chance to upgrade, or get off master, (for example on
202+ the LTS branch).
203+
204+ ##Fail early at import time
205+
206+ Add an error early at import at runtime with a clear error message, leave the
207+ early import compatible Python 2 for users to not be welcomed with a useless
208+ ` SyntaxError ` . You are_ allowed_ to use multi-line strings in error messages.
209+
210+ Error at import time_ will_ happen on system with old version of pip and
211+ setuptools. Keep in mind that saying the package is Python 3 only is not a lot
212+ more helpful than a Syntax error. The most reasonable reason would be out of
213+ data pip and setuptools:
135214
136215
137216```
138217import sys
139218
140219if sys.version_info < (3,):
141- Raise ValueError (
220+ raise ImportError (
142221 """You are running Frobulator 6.0 on Python 2
143222
144- Unfortunately Frobulator 6.0 and abovere not compatible with Python 2 anymore,
145- and you still ended up with this version installed on your system. That's a
146- bummer sorry about that it should not have happen. Make sure you have pip >=
147- 9.0 to avoid this kind of issues:
223+ Unfortunately Frobulator 6.0 and aboveare not compatible with Python 2
224+ anymore, and you still ended up with this version installed on your system.
225+ That's a bummer. Sorry about that. It should not have happen. Make sure you
226+ have pip >= 9.0 to avoid this kind of issues, as well as setuptools >= 24.2 :
148227
149- $ pip install pip --upgrade
228+ $ pip install pipsetuptools --upgrade
150229
151- Use the following to check pip version
230+ You have various other choices
152231
153- You have various choices :
232+ - install an older version of Frobulator :
154233
155- - You can still install an older version of Frobulator:
156- $ pip install frobulator<6.0
234+ $ pip install 'frobulator<6.0'
157235
158236- Upgrade your system to use Python 3.
159237
160238It would be great if you can figure out how this version ended up being
161239installed, and try to check how to prevent that for future users.
162240
163- This this page for more information : url to here for example.
241+ See the following url for more up to date informations:
242+
243+ https://i.am.an/url
244+
164245""")
165246
166247```
167248
168-
249+ ## Watch out for beta releases
169250
170- Make sure your version number match pep440 or you will get surprises during
251+
252+ Make sure your version number match pep 440 or you will get surprises during
171253beta in particular as the` sdist ` and` wheel ` will appear as being different
172254versions, in particular sdist (during beta/rc/post) can appear with a greater
173255version number than wheels. Pip thus try to install the sdist instead of the
@@ -178,38 +260,35 @@ The regular expression to check for validity of pep440 can be find below:
178260
179261` ^([1-9]\\d*!)?(0|[1-9]\\d*)(\\.(0|[1-9]\\d*))*((a|b|rc)(0|[1-9]\\d*))?(\\.post(0|[1-9]\\d*))?(\\.dev(0|[1-9]\\d*))? `
180262
181-
263+ ## Depend on setuptools
182264
183265You can mark your library as dependent on setuptools greater than 24.3 starting
184266now, this will insure that during the next upgrade (when the packages drop
185267python 2 support) will have the right version of setuptools.
186268
187-
188-
189- #Recommended Mitigations
190-
191269Of course regardless of all the care you will take for your library to no break
192270and to install only on python 2, you will likely have cases where it still end
193271up being installed on incompatible versions of Python. Simply because users
194272upgrades rarely and only an old version of pip or setuptools is enough to make
195273the all update process broken.
196274
197- - Leave` setup.py ` python 2 compatible and fail early. If you detect Python 2
198- raise a clear error message and ask user to make sure they have pip >9.0 (or
199- migrate to Python 3). You can (try to) conditionally import pip and check for
200- its version but this might not be the same pip. Failing early is important to
201- make sure the Python installation does not install and incompatible version.
202- Otherwise user code can fail at runtime arbitrary later in the future, which
203- can be a difficult to debug and fix.
275+ ##fail early in setup.py
204276
205- - If you control dependant packages, Make sure to include conditional
206- dependencies depending on the version of Python.
277+ Leave` setup.py ` python 2 compatible and fail early. If you detect Python 2
278+ raise a clear error message and ask user to make sure they have pip >9.0 (or
279+ migrate to Python 3). You can (try to) conditionally import pip and check for
280+ its version but this might not be the same pip. Failing early is important to
281+ make sure the Python installation does not install and incompatible version.
282+ Otherwise user code can fail at runtime arbitrary later in the future, which can
283+ be a difficult to debug and fix. Get inspiration from the message of failure at
284+ runtime, and adapt for installation time.
207285
208- - Regardless of whether the installation step fails on Python 2, implement a
209- similar check in the top level import of your package.
286+ ##Fix dependant libraries
210287
288+ If you control dependant packages, Make sure to include conditional dependencies
289+ depending on the version of Python.
211290
212- #Alternative mitigation
291+ #Incorrect mitigations
213292
214293This is a collection of "mitigation" or "solutions" you will find on the web
215294and that you will hear about. This is an attempt to acknowledge them, and
@@ -222,26 +301,25 @@ It is possible to release a meta-package that has _virtually_ no code and rely
222301on conditional dependency to install its actual core code on the user system.
223302For example, Frob-6.0 could be a meta-package which depends on
224303Frob-real-py2 on Python <3.0, and Frob-real-py3 on Python >= 3.4. While
225- this approach is_ doable_ this can makeimport confusing.
304+ this approach is_ doable_ this can makeimports confusing.
226305
227306Moreover, upgrading your package may need the user to explicitly tell pip to
228- upgrade dependencies as` pip install frob ` will only upgrade the meta-package.
307+ upgrade dependencies as` pip install-U frob ` will only upgrade the meta-package.
229308
230309###Multiple Sdist.
231310
232311Pip (used to) support a "feature" where a sdist ending in` -pyX.Y.tar.gz ` would
233312only be seen as compatible on Python X.Y, thus it used to be possible to
234- publish multiple sdist of a package targeting various python version.
313+ publish multiple sdist of a package targeting various python version.
235314
236315Though it is not possible anymore to upload multiple sdist on PyPI. This
237316solution is thus not possible.
238317
239318###Wheel only ?
240319
241- Break downstream packages.
242-
243-
244-
320+ Releasing a package only using wheel for a given python version is doable, but
321+ this will break downstream packages that may require the original source to
322+ reproduce the build.
245323
246324#Why all that ?
247325
@@ -254,19 +332,17 @@ Many libraries have transitioned from Python 2-only to Python 2 + 3. And the
254332issue of transitioning to Python 3 only is relatively recent. Technically it
255333can also apply to libraries that are only stopping support for 2.6, or even are
256334already Python 3 only, but are starting to stop support for earlier versions of
257- Python. For example a library releasing a Python 3.4+ only version.
335+ Python. For example a library releasing a Python 3.4+ only version.
258336
259337Python 3.3 was release end of 2012, and was the first version to support
260338(again)` u ` as a prefix for Unicode string. It was one of the first minor
261339version of Python 3 that saw a majority of single-source project working both
262340on Python 2 and Python 3. These are the Project that will likely be affected by
263- this issue.
341+ this issue.
264342
265343The introduction of Python 3 was chaotic, there are still strong argument both
266344in Python 2 and Python 3 camps. In the one suffering the most from this are
267345users. Starting with the fact that inevitably some libraries will stop support
268346for Python 2 and release Python 3 only library. And that inevitably some system
269347will will not be upgraded to Python 3 how can we_ ensure_ that users get the
270348_ least_ breakage as possible ? And what are the best practices to follow.
271-
272-