Replacement Note

Although the size of the to-do list in this PEP is much less scarythan that inPEP 101, it turns out not to be enough justificationfor the duplication of information, and with it, the danger of oneof the copies to become out of date. Therefore, this PEP is notmaintained anymore, and micro releases are fully covered byPEP 101.

Abstract

Making a Python release is an arduous process that takes aminimum of half a day’s work even for an experienced releaser.Until recently, most – if not all – of that burden was borne byGuido himself. But several recent releases have been performed byother folks, so this PEP attempts to collect, in one place, allthe steps needed to make a Python bugfix release.

The major Python release process is covered inPEP 101 - this PEPis justPEP 101, trimmed down to only include the bits that arerelevant for micro releases, a.k.a. patch, or bug fix releases.

It is organized as a recipe and you can actually print this out andcheck items off as you complete them.

How to Make A Release

Here are the steps taken to make a Python release. Some steps aremore fuzzy than others because there’s little that can beautomated (e.g. writing the NEWS entries). Where a step isusually performed by An Expert, the name of that expert is given.Otherwise, assume the step is done by the Release Manager (RM),the designated person performing the release. Almost every placethe RM is mentioned below, this step can also be done by the BDFLof course!

XXX: We should include a dependency graph to illustrate the stepsthat can be taken in parallel, or those that depend on othersteps.

We use the following conventions in the examples below. Where arelease number is given, it is of the form X.Y.MaA, e.g. 2.1.2c1for Python 2.1.2 release candidate 1, where “a” == alpha, “b” ==beta, “c” == release candidate. Final releases are tagged with“releaseXYZ” in CVS. The micro releases are made from themaintenance branch of the major release, e.g. Python 2.1.2 is madefrom the release21-maint branch.

1. Send an email topython-dev@python.org indicating the release isabout to start.
2. Put a freeze on check ins into the maintenance branch. At thispoint, nobody except the RM should make any commits to the branch(or his duly assigned agents, i.e. Guido the BDFL, Fred Drake fordocumentation, or Thomas Heller for Windows). If the RM screwed upand some desperate last minute change to the branch isnecessary, it can mean extra work for Fred and Thomas. So try toavoid this!
3. On the branch, change Include/patchlevel.h in two places, toreflect the new version number you’ve just created. You’ll wantto change the PY_VERSION macro, and one or several of theversion subpart macros just above PY_VERSION, as appropriate.
4. Change the “%define version” line of Misc/RPM/python-2.3.spec to thesame string asPY_VERSION was changed to above. E.g:

   %defineversion2.3.1

   You also probably want to reset the %define release lineto ‘1pydotorg’ if it’s not already that.

5. If you’re changing the version number for Python (e.g. fromPython 2.1.1 to Python 2.1.2), you also need to update theREADME file, which has a big banner at the top proclaiming itsidentity. Don’t do this if you’re just releasing a new alpha orbeta release, but /do/ do this if you’re release a new micro,minor or major release.
6. The LICENSE file also needs to be changed, due to severalreferences to the release number. As for the README file, changingthese are necessary for a new micro, minor or major release.

   The LICENSE file contains a table that describes the legalheritage of Python; you should add an entry for the X.Y.Zrelease you are now making. You should update this table in theLICENSE file on the CVS trunk too.

7. When the year changes, copyright legends need to be updated inmany places, including the README and LICENSE files.
8. For the Windows build, additional files have to be updated.

   PCbuild/BUILDno.txt contains the Windows build number, see theinstructions in this file how to change it. Saving the projectfile PCbuild/pythoncore.dsp results in a change toPCbuild/pythoncore.dsp as well.

   PCbuild/python20.wse sets up the Windows installer versionresource (displayed when you right-click on the installer .exeand select Properties), and also contains the Python versionnumber.

   (Before version 2.3.2, it was required to manually editPC/python_nt.rc, this step is now automated by the buildprocess.)

9. After starting the process, the most important thing to do nextis to update the Misc/NEWS file. Thomas will need this in order todo the Windows release and he likes to stay up late. This stepcan be pretty tedious, so it’s best to get to it immediatelyafter making the branch, or even before you’ve made the branch.The sooner the better (but again, watch for new checkins upuntil the release is made!)

   Add high level items new to this release. E.g. if we’rereleasing 2.2a3, there must be a section at the top of the fileexplaining “What’s new in Python 2.2a3”. It will be followed bya section entitled “What’s new in Python 2.2a2”.

   Note that you /hope/ that as developers add new features to thetrunk, they’ve updated the NEWS file accordingly. You can’t bepositive, so double check. If you’re a Unix weenie, it helps toverify with Thomas about changes on Windows, and Jack Jansenabout changes on the Mac.

   This command should help you (but substitute the correct -r tag!):

   %cvslog-rr22a1:|pythonTools/scripts/logmerge.py>/tmp/news.txt

   IOW, you’re printing out all the cvs log entries from theprevious release until now. You can then troll through thenews.txt file looking for interesting things to add to NEWS.

10. Check your NEWS changes into the maintenance branch. It’s easyto forget to update the release date in this file!
11. Check in any changes to IDLE’s NEWS.txt. Update the header inLib/idlelib/NEWS.txt to reflect its release version and date.Update the IDLE version in Lib/idlelib/idlever.py to match.

11. Once the release process has started, the documentation needs tobe built and posted on python.org according to the instructionsinPEP 101.

    Note that Fred is responsible both for merging doc changes fromthe trunk to the branch AND for merging any branch changes fromthe branch to the trunk during the cleaning up phase.Basically, if it’s in Doc/ Fred will take care of it.

12. Thomas compiles everything with MSVC 6.0 SP5, and moves thepython23.chm file into the src/chm directory. The installerexecutable is then generated with Wise Installation System.

    The installer includes the MSVC 6.0 runtime in the filesMSVCRT.DLL and MSVCIRT.DLL. It leads to disaster if these filesare taken from the system directory of the machine where theinstaller is built, instead it must be absolutely made sure thatthese files come from the VCREDIST.EXE redistributable packagecontained in the MSVC SP5 CD. VCREDIST.EXE must be unpackedwith winzip, and the Wise Installation System prompts for thedirectory.

    After building the installer, it should be opened with winzip,and the MS dlls extracted again and check for the same versionnumber as those unpacked from VCREDIST.EXE.

    Thomas uploads this file to the starship. He then sends the RMa notice which includes the location and MD5 checksum of theWindows executable.

    Note that Thomas’s creation of the Windows executable may generatea few more commits on the branch. Thomas will be responsible formerging Windows-specific changes from trunk to branch, and frombranch to trunk.

13. Sean performs his Red Hat magic, generating a set of RPMs. Heuploads these files to python.org. He then sends the RM a noticewhich includes the location and MD5 checksum of the RPMs.
14. It’s Build Time!

    Now, you’re ready to build the source tarball. First cd to yourworking directory for the branch. E.g.% cd …/python-22a3

15. Do a “cvs update” in this directory. Do NOT include the -A flag!

    You should not see any “M” files, but you may see several “P”and/or “U” files. I.e. you better not have any uncommittedchanges in your working directory, but you may pick up some ofFred’s or Thomas’s last minute changes.

16. Now tag the branch using a symbolic name like “rXYMaZ”,e.g. r212

    %cvstagr212

    Be sure to tag only the python/dist/src subdirectory of thePython CVS tree!

17. Change to a neutral directory, i.e. one in which you can do afresh, virgin, cvs export of the branch. You will be creating anew directory at this location, to be named “Python-X.Y.M”. Doa CVS export of the tagged branch.

    %cd~%cvs-dcvs.sf.net:/cvsroot/pythonexport-rr212 \-dPython-2.1.2python/dist/src

18. Generate the tarball. Note that we’re not using the ‘z’ optionon the tar command because 1) that’s only supported by GNU taras far as we know, and 2) we’re going to max out the compressionlevel, which isn’t a supported option. We generate both tar.gztar.bz2 formats, as the latter is about 1/6th smaller.

    %tar-cf-Python-2.1.2|gzip-9>Python-2.1.2.tgz%tar-cf-Python-2.1.2|bzip2-9>Python-2.1.2.tar.bz2

19. Calculate the MD5 checksum of the tgz and tar.bz2 files youjust created

    %md5sumPython-2.1.2.tgz

    Note that if you don’t have the md5sum program, there is aPython replacement in the Tools/scripts/md5sum.py file.

20. Create GPG keys for each of the files.

    %gpg-baPython-2.1.2.tgz%gpg-baPython-2.1.2.tar.bz2%gpg-baPython-2.1.2.exe

21. Now you want to perform the very important step of checking thetarball you just created, to make sure a completely clean,virgin build passes the regression test. Here are the beststeps to take:

    % cd /tmp% tar zxvf ~/Python-2.1.2.tgz% cd Python-2.1.2% ls(Do things look reasonable?)% ./configure(Loads of configure output)% make test(Do all the expected tests pass?)

    If the tests pass, then you can feel good that the tarball isfine. If some of the tests fail, or anything else about thefreshly unpacked directory looks weird, you better stop now andfigure out what the problem is.

22. You need to upload the tgz and the exe file to creosote.python.org.This step can take a long time depending on your networkbandwidth. scp both files from your own machine to creosote.
23. While you’re waiting, you can start twiddling the web pages toinclude the announcement.
    1. In the top of the python.org web site CVS tree, create asubdirectory for the X.Y.Z release. You can actually copy anearlier patch release’s subdirectory, but be sure to deletethe X.Y.Z/CVS directory and “cvs add X.Y.Z”, for example:

       %cd.../pydotorg%cp-r2.2.22.2.3%rm-rf2.2.3/CVS%cvsadd2.2.3%cd2.2.3

    2. Edit the files for content: usually you can globally replaceX.Ya(Z-1) with X.YaZ. However, you’ll need to think about the“What’s New?” section.
    3. Copy the Misc/NEWS file to NEWS.txt in the X.Y.Z directory forpython.org; this contains the “full scoop” of changes toPython since the previous release for this version of Python.
    4. Copy the .asc GPG signatures you created earlier here as well.
    5. Also, update the MD5 checksums.
    6. Preview the web page by doing a “make” or “make install” (aslong as you’ve created a new directory for this release!)
    7. Similarly, edit the ../index.ht file, i.e. the python.org homepage. In the Big Blue Announcement Block, move the paragraphfor the new version up to the top and boldify the phrase“Python X.YaZ is out”. Edit for content, and preview locally,but do NOT do a “make install” yet!
24. Now we’re waiting for the scp to creosote to finish. Da de da,da de dum, hmm, hmm, dum de dum.
25. Once that’s done you need to go to creosote.python.org and moveall the files in place over there. Our policy is that everyPython version gets its own directory, but each directory maycontain several releases. We keep all old releases, moving theminto a “prev” subdirectory when we have a new release.

    So, there’s a directory called “2.2” which containsPython-2.2a2.exe and Python-2.2a2.tgz, along with a “prev”subdirectory containing Python-2.2a1.exe and Python-2.2a1.tgz.

    So…

    1. On creosote, cd to ~ftp/pub/python/X.Y creating it ifnecessary.
    2. Move the previous release files to a directory called “prev”creating the directory if necessary (make sure the directoryhas g+ws bits on). If this is the first alpha release of anew Python version, skip this step.
    3. Move the .tgz file and the .exe file to this directory. Makesure they are world readable. They should also be groupwritable, and group-owned by webmaster.
    4. md5sum the files and make sure they got uploaded intact.
26. the X.Y/bugs.ht file if necessary. It is best to getBDFL input for this step.
27. Go up to the parent directory (i.e. the root of the web pagehierarchy) and do a “make install” there. You’re release is nowlive!
28. Now it’s time to write the announcement for the mailing lists.This is the fuzzy bit because not much can be automated. Youcan use one of Guido’s earlier announcements as a template, butplease edit it for content!

    Once the announcement is ready, send it to the followingaddresses:

    python-list@python.orgpython-announce@python.orgpython-dev@python.org

29. Send a SourceForge News Item about the release. From theproject’s “menu bar”, select the “News” link; once in News,select the “Submit” link. Type a suitable subject (e.g. “Python2.2c1 released” :-) in the Subject box, add some text to theDetails box (at the very least including the release URL atwww.python.org and the fact that you’re happy with the release)and click the SUBMIT button.

    Feel free to remove any old news items.

Now it’s time to do some cleanup. These steps are very important!

1. Edit the file Include/patchlevel.h so that the PY_VERSIONstring says something like “X.YaZ+”. Note the trailing ‘+’indicating that the trunk is going to be moving forward withdevelopment. E.g. the line should look like:

   #define PY_VERSION              "2.1.2+"

   Make sure that the otherPY_ version macros contain thecorrect values. Commit this change.

2. For the extra paranoid, do a completely clean test of therelease. This includes downloading the tarball fromwww.python.org.
3. Make sure the md5 checksums match. Then unpack the tarball,and do a clean make test.

   %makedistclean%./configure%maketest

   To ensure that the regression test suite passes. If not, youscrewed up somewhere!

Step 5 …

Verify! This can be interleaved with Step 4. Pretend you’re auser: download the files from python.org, and make Python fromit. This step is too easy to overlook, and on several occasionswe’ve had useless release files. Once a general server problemcaused mysterious corruption of all files; once the source tarballgot built incorrectly; more than once the file upload process onSF truncated files; and so on.

What Next?

Rejoice. Drink. Be Merry. Write a PEP like this one. Or belike unto Guido and take A Vacation.

You’ve just made a Python release!

Actually, there is one more step. You should turn over ownershipof the branch to Jack Jansen. All this means is that now he willbe responsible for making commits to the branch. He’s going touse this to build the MacOS versions. He may send you informationabout the Mac release that should be merged into the informationalpages on www.python.org. When he’s done, he’ll tag the branchsomething like “rX.YaZ-mac”. He’ll also be responsible formerging any Mac-related changes back into the trunk.

Final Release Notes

The Final release of any major release, e.g. Python 2.2 final, hasspecial requirements, specifically because it will be one of thelongest lived releases (i.e. betas don’t last more than a coupleof weeks, but final releases can last for years!).

For this reason we want to have a higher coordination between thethree major releases: Windows, Mac, and source. The Windows andsource releases benefit from the close proximity of the respectiverelease-bots. But the Mac-bot, Jack Jansen, is 6 hours away. Sowe add this extra step to the release process for a finalrelease:

1. Hold up the final release until Jack approves, or until welose patience <wink>.

The python.org site also needs some tweaking when a new bugfix releaseis issued.

2. The documentation should be installed at doc/<version>/.
3. Add a link from doc/<previous-minor-release>/index.ht to thedocumentation for the new version.
4. All older doc/<old-release>/index.ht files should be updated topoint to the documentation for the new version.
5. /robots.txt should be modified to prevent the old version’sdocumentation from being crawled by search engines.

Windows Notes

Windows has a GUI installer, various flavors of Windows have“special limitations”, and the Windows installer also packsprecompiled “foreign” binaries (Tcl/Tk, expat, etc). So Windowstesting is tiresome but very necessary.

Concurrent with uploading the installer, Thomas installs Pythonfrom it twice: once into the default directory suggested by theinstaller, and later into a directory with embedded spaces in itsname. For each installation, he runs the full regression suitefrom a DOS box, and both with and without -0.

He also triesevery shortcut created under Start -> Menu -> thePython group. When trying IDLE this way, you need to verify thatHelp -> Python Documentation works. When trying pydoc this way(the “Module Docs” Start menu entry), make sure the “StartBrowser” button works, and make sure you can search for a randommodule (Thomas uses “random” <wink>) and then that the “go toselected” button works.

It’s amazing how much can go wrong here – and even more amazinghow often last-second checkins break one of these things. Ifyou’re “the Windows geek”, keep in mind that you’re likely theonly person routinely testing on Windows, and that Windows issimply a mess.

Repeat all of the above on at least one flavor of Win9x, and oneof NT/2000/XP. On NT/2000/XP, try both an Admin and a plain User(not Power User) account.

WRT Step 5 above (verify the release media), since by the timerelease files are ready to download Thomas has generally run manyWindows tests on the installer he uploaded, he usually doesn’t doanything for Step 5 except a full byte-comparison (“fc /b” ifusing a Windows shell) of the downloaded file against the file heuploaded.

Copyright

This document has been placed in the public domain.