Guide for IPython core Developers
This guide documents the development of IPython itself. Alternatively,developers of third party tools and libraries that use IPython should see theDeveloper’s guide for third party tools and libraries.
For instructions on how to make a developer install seeInstalling the development version.
Backporting Pull requests
All pull requests should usually be made againstmain, if a Pull Requestneed to be backported to an earlier release; then it should be tagged with thecorrectmilestone.
If you tag a pull request with a milestonebefore merging the pull request,and the base ref ismain, then our backport bot should automatically createa corresponding pull-request that backport on the correct branch.
If you have write access to the IPython repository you can also just mention thebackport bot to do the work for you. The bot is evolving so instructions maybe different. At the time of this writing you can use:
@meeseeksdev[bot]backport[to]<branchname>
The bot will attempt to backport the current pull-request and issue a PR ifpossible.
Note
The@ and[bot] when mentioning the bot should be optional and canbe omitted.
If the pull request cannot be automatically backported, the bot should tell youso on the PR and apply a “Need manual backport” tag to the origin PR.
IPython release process
This document contains the process that is used to create an IPython release.
Conveniently, therelease script in thetools directory of theIPythonrepository automates most of the release process. This document serves as ahandy reminder and checklist for the release manager.
During the release process, you might need the extra following dependencies:
keyringto access your GitHub authentication tokens
graphvizto generate some graphs in the documentation
ghproto generate the stats
Make sure you have all the required dependencies to run the tests as well.
You can try tosourcetools/release_helper.sh when releasing via bash, itshould guide you through most of the process.
1. Set Environment variables
Set environment variables to document previous release tag, currentrelease milestone, current release version, and git tag.
These variables may be used later to copy/paste as answers to the scriptquestions instead of typing the appropriate command when the time comes. Thesevariables are not used by the scripts directly; therefore, there is no need toexport them. The format for bash is as follows, but note that these valuesare just an example valid only for the 5.0 release; you’ll need to update themfor the release you are actually making:
PREV_RELEASE=4.2.1MILESTONE=5.0VERSION=5.0.0BRANCH=main
Forreproducibility of builds,we recommend settingSOURCE_DATE_EPOCH prior to running the build; record the used valueofSOURCE_DATE_EPOCH as it may not be available from build artifact. Youshould be able to usedate+%s to get a formatted timestamp:
SOURCE_DATE_EPOCH=$(date +%s)
2. Create GitHub stats and finish release note
Note
This step is optional if making a Beta or RC release.
Note
Before generating the GitHub stats, verify that all closed issues and pullrequests haveappropriate milestones.This searchshould return no results before creating the GitHub stats.
If a major release:
merge any pull request notes into what’s new:
pythontools/update_whatsnew.pyupdate
docs/source/whatsnew/development.rst, to ensure it coversthe major release featuresmove the contents of
development.rsttoversionX.rstwhereXisthe numerical release versionYou do not need to temporarily remove the first entry called
development, nor re-add it after the release, it will automatically behidden when releasing a stable version of IPython (if_version_extrainrelease.pyis an empty string.Make sure that the stats file has a header or it won’t be rendered inthe final documentation.
To find duplicates and update.mailmap, use:
git log --format="%aN <%aE>" $PREV_RELEASE... | sort -u -f
If a minor release you might need to do some of the above points manually, andforward port the changes.
3. Make sure the repository is clean
- of any file that could be problematic.
Remove all non-tracked files with:
gitclean-xfdi
This will ask for confirmation before removing all untracked files. Makesure the
dist/folder is clean to avoid any stale builds fromprevious build attempts.
4. Update the release version number
EditIPython/core/release.py to have the current version.
in particular, update version number and_version_extra content inIPython/core/release.py.
Step 5 will validate your changes automatically, but you might still want tomake sure the version number matches pep440.
In particular,rc andbeta are not separated by. or thesdistandbdist will appear as different releases. For example, a valid versionnumber for a release candidate (rc) release is:1.3rc1. Notice that thereis no separator between the ‘3’ and the ‘r’. Check the environment variable$VERSION as well.
You will likely just have to modify/comment/uncomment one of the lines setting_version_extra
5. Run thetools/build_release script
Runningtools/build_release does all the file checking and building thatthe real release script will do. This makes test installations, checks thatthe build procedure runs OK, and tests other steps in the release process.
Thebuild_release script will in particular verify that the version numbermatch PEP 440, in order to avoid surprise at the time of build upload.
We encourage creating a test build of the docs as well.
6. Create and push the new tag
Commit the changes to release.py:
git commit -am "release $VERSION" -Sgit push origin $BRANCH
(omit the-S if you are no signing the package)
Create and push the tag:
git tag -am "release $VERSION" "$VERSION" -sgit push origin $VERSION
(omit the-s if you are no signing the package)
Update release.py back tox.y-dev orx.y-maint commit and push:
git commit -am "back to development" -Sgit push origin $BRANCH
(omit the-S if you are no signing the package)
Now checkout the tag we just made:
git checkout $VERSION
7. Run the release script
Run therelease script, this step requires having a current wheel, Python>=3.4 and Python 2.7.:
./tools/release
This makes the tarballs and wheels, and puts them under thedist/folder. Be sure to test thewheels and thesdist locally beforeuploading them to PyPI. We do not use an universal wheel as each wheelinstalls anipython2 oripython3 script, depending on the version ofPython it is built for. Using an universal wheel would prevent this.
Check the shasum of files with:
shasum-a256dist/*
and takes notes of them you might need them to update the conda-forge recipes.Rerun the command and check the hash have not changed:
./tools/releaseshasum-a256dist/*
Use the following to actually upload the result of the build:
./tools/releaseupload
It should posts them toarchive.ipython.org and to PyPI.
PyPI/Warehouse will automatically hide previous releases. If you are uploadinga non-stable version, make sure to log-in to PyPI and un-hide previous version.
8. Draft a short release announcement
The announcement should include:
release highlights
a link to the html version of theWhat’s new section of the documentation
a link to upgrade or installation tips (if necessary)
Post the announcement to the mailing list and or blog, and link from Twitter.
Note
If you are doing a RC or Beta, you can likely skip the next steps.
9. Update milestones on GitHub
These steps will bring milestones up to date:
close the just released milestone
open a new milestone for the next release (x, y+1), if the milestone doesn’texist already
10. Update the IPython website
The IPython website should document the new release:
add release announcement (news, announcements)
update current version and download links
update links on the documentation page (especially if a major release)
11. Update readthedocs
Make sure to update readthedocs and set the latest tag as stable, as well aschecking that previous release is still building under its own tag.
12. Update the Conda-Forge feedstock
Follow the instructions onthe repository
13. Celebrate!
Celebrate the release and please thank the contributors for their work. Greatjob!
Old Documentation
Out of date documentation is still available and have been kept for archival purposes.
Note
Developers documentation used to be on the IPython wiki, but are now out ofdate. The wiki is though still available for historical reasons:Old IPythonGitHub Wiki.