pyproject.tomlsetup.py based project?Summaries and links for the most relevant projects in the space of Pythoninstallation and packaging.
bandersnatch is a PyPI mirroring client designed to efficientlycreate a complete mirror of the contents of PyPI. Organizations thussave bandwidth and latency on package downloads (especially in thecontext of automated tests) and to prevent heavily loading PyPI’sContent Delivery Network (CDN).Files can be served from a local directory orAWS S3.
build is aPEP 517 compatible Python package builder. It provides a CLI tobuild packages, as well as a Python API.
Docs |Issues |GitHub |PyPI |Discussions |Discord #cibuildwheel
cibuildwheel is a Python package that buildswheels for all commonplatforms and Python versions on most CI systems. Also seemultibuild.
distlib is a library which implements low-level functions thatrelate to packaging and distribution of Python software.distlibimplements several relevant PEPs (Python Enhancement Proposalstandards) and is useful for developers of third-party packaging toolsto make and upload binary and sourcedistributions, achieve interoperability, resolvedependencies, manage package resources, and do other similarfunctions.
Unlike the stricterpackaging project (below), whichspecifically implements modern Python packaging interoperabilitystandards,distlib also attempts to provide reasonable fallbackbehaviours when asked to handle legacy packages and metadata thatpredate the modern interoperability standards and fall into the subsetof packages that are incompatible with those standards.
The original Python packaging system, added to the standard library inPython 2.0 and removed in 3.12.
Due to the challenges of maintaining a packaging systemwhere feature updates are tightly coupled to language runtime updates,direct usage ofdistutils has been actively discouraged, withSetuptools being the preferred replacement.Setuptoolsnot only provides features that plaindistutils doesn’t offer(such as dependency declarations and entry point declarations), italso provides a consistent build interface and feature set across allsupported Python versions.
Consequently,distutils was deprecated in Python 3.10 byPEP 632 andhas beenremoved from the standard library inPython 3.12. Setuptools bundles the standalone copy of distutils, and it isinjected even on Python < 3.12 if you import setuptools first or use pip.
Flit provides a simple way to create and upload pure Python packages andmodules to PyPI. It focuses onmaking the easy things easyfor packaging. Flit can generate a configuration file to quickly set up asimple project, build source distributions and wheels, and upload them to PyPI.
Flit usespyproject.toml to configure a project. Flit does not rely on toolssuch asSetuptools to build distributions, ortwine to upload themto PyPI. Flit requires Python 3, but you can use it to distribute modules forPython 2, so long as they can be imported on Python 3.
The flit package is lifted byMatthias Bussonnier since October 2023 on thetidelift platform, and funds sent to the PSF andearmarked for PyPA usage.
Hatch is a unified command-line tool meant to conveniently managedependencies and environment isolation for Python developers. Pythonpackage developers use Hatch and itsbuild backend Hatchling toconfigure, version, specify dependencies for, and publish packagesto PyPI. Its plugin system allows for easily extending functionality.
Core utilities for Python packaging used bypip andSetuptools.
The core utilities in the packaging library handle version handling,specifiers, markers, requirements, tags, and similar attributes andtasks for Python packages. Most Python users rely on this librarywithout needing to explicitly call it; developers of the other Pythonpackaging, distribution, and installation tools listed here often useits functionality to parse, discover, and otherwise handle dependencyattributes.
This project specifically focuses on implementing the modern Pythonpackaging interoperability standards defined atPyPA specifications, and will report errors forsufficiently old legacy packages that are incompatible with thosestandards. In contrast, thedistlib project is a morepermissive library that attempts to provide a plausible reading ofambiguous metadata in cases wherepackaging will instead reporton error.
The most popular tool for installing Python packages, and the oneincluded with modern versions of Python.
It provides the essential core features for finding, downloading, andinstalling packages from PyPI and other Python package indexes, and can beincorporated into a wide range of development workflows via itscommand-line interface (CLI).
Pipenv is a project that aims to bring the best of all packaging worlds to thePython world. It harnessesPipfile,pip, andvirtualenvinto one single toolchain. It can autoimportrequirements.txt and alsocheck for CVEs inPipfile usingsafety.
Pipenv aims to help users manage environments, dependencies, andimported packages on the command line. It also works well on Windows(which other tools often underserve), makes and checks file hashes,to ensure compliance with hash-locked dependency specifiers, and easesuninstallation of packages and dependencies.
Pipfile and its sisterPipfile.lock are a higher-levelapplication-centric alternative topip’s lower-levelrequirements.txt file.
pipx is a tool to install and run Python command-line applications withoutcausing dependency conflicts with other packages installed on the system.
This guide!
readme_renderer is a library that package developers use to rendertheir user documentation (README) files into HTML from markuplanguages such as Markdown or reStructuredText. Developers call it onits own or viatwine, as part of their release managementprocess, to check that their package descriptions will properlydisplay on PyPI.
Setuptools (which includeseasy_install) is a collection ofenhancements to the Python distutils that allow you to more easilybuild and distribute Pythondistributions, especially ones that have dependencies on other packages.
trove-classifiers is the canonical source forclassifiers on PyPI, which project maintainers use tosystematically describe their projectsso that users can better find projects that match their needs on the PyPI.
The trove-classifiers package contains a list of valid classifiers anddeprecated classifiers (which are paired with the classifiers that replacethem). Use this package to validate classifiers used in packages intended foruploading to PyPI. As this list of classifiers is published as code, youcan install and import it, giving you a more convenient workflow compared toreferring to thelist published on PyPI. Theissue tracker for theproject hosts discussions on proposed classifiers and requests for newclassifiers.
Twine is the primary tool developers use to upload packages to thePython Package Index or other Python package indexes. It is acommand-line program that passes program files and metadata to a webAPI. Developers use it because it’s the official PyPI upload tool,it’s fast and secure, it’s maintained, and it reliably works.
virtualenv is a tool for creating isolated PythonVirtual Environments, likevenv. Unlikevenv, virtualenv cancreate virtual environments for other versions of Python, which it locatesusing the PATH environment variable. It also provides convenient features forconfiguring, maintaining, duplicating, and troubleshooting virtual environments.For more information, see the section onCreating Virtual Environments.
The current codebase powering thePython Package Index(PyPI). It is hosted atpypi.org. The defaultsource forpip downloads.
Primarily, the wheel project offers thebdist_wheelSetuptools extension forcreatingwheel distributions. Additionally, it offers its owncommand line utility for creating and installing wheels.
See alsoauditwheel, a toolthat package developers use to check and fix Python packages they aremaking in the binary wheel format. It provides functionality todiscover dependencies, check metadata for compliance, and repair thewheel and metadata to properly link and include external sharedlibraries in a package.
Buildout is a Python-based build system for creating, assembling and deployingapplications from multiple parts, some of which may be non-Python-based. Itlets you create a buildout configuration and reproduce the same software later.
Conda is a package, dependency, and environment management system for any language — Python, R,Ruby, C/C++, Fortran, and more. It is written in Python andwidely used in the Python scientific computing community, due to its support for non-Pythoncompiled libraries and extensions. It is used as the basis of theAnaconda Python distribution from Anaconda, Inc. It was originallyaimed at the scientific community, but can also be used on its own, or with theminiconda,miniforge orpixi systems. It is available for Windows, Mac and Linux systems.
Conda is a completely separate tool frompip, virtualenv and wheel, but providesmany of their combined features, such as package management, virtual environmentmanagement and deployment of binary extensions and other binary code.
Conda does not install packages from PyPI – it can only manage packages built specificallyfor conda, which can be made available on a “conda channel”, such as those hosted onanaconda.org, or a local (e.g. intranet) package server.In addition to the “default” channels managed byAnaconda, Inc., there are a wide variety of packages from the community supportedconda-forge project
Note thatpip can be installed into, and work side-by-side with condafor managingdistributions from PyPI. It is also possibleto build conda packages from Python source packages using tools such asconda skeleton: a tool to automatically make conda packages from Python packages available on PyPI.
devpi features a powerful PyPI-compatible server and PyPI proxy cachewith a complementary command line tool to drive packaging, testing andrelease activities with Python. devpi also provides a browsable andsearchable web interface.devpi supports mirroring PyPI, multiplepackage indexes with inheritance, syncing betweenthese indexes, index replication and fail-over, and package upload.
dumb-pypi is a simplepackage index static file sitegenerator, which then must be hosted by a static file webserver to become thepackage index. It supports serving the hash, core-metadata, and yank-status.
Enscons is a Python packaging tool based onSCons. It buildspip-compatible source distributions and wheels without usingdistutils or setuptools, including distributions with Cextensions. Enscons has a different architecture and philosophy thandistutils. Rather than adding build features to a Pythonpackaging system, enscons adds Python packaging to a general purposebuild system. Enscons helps you to build sdists that can beautomatically built bypip, and wheels that are independent ofenscons.
Warning
Not maintained, project archived
Flask-Pypi-Proxy is apackage index as a cachedproxy for PyPI.
Hashdist is a library for building non-root softwaredistributions. Hashdist is trying to be “the Debian of choice forcases where Debian technology doesn’t work”. The best way forPythonistas to think about Hashdist may be a more powerful hybrid ofvirtualenv andbuildout. It is aimed at solving theproblem of installing scientific software, and making packagedistribution stateless, cached, and branchable. It is used by someresearchers but has been lacking in maintenance since 2016.
Maturin is a build backend for Rust extension modules, also written inRust. It supports building wheels for python 3.7+ on Windows, Linux, macOS andFreeBSD, can upload them to PyPI and has basic PyPy and GraalPy support.
meson-python is a build backend that uses theMeson build system. It enablesPython package authors to useMeson as the build system for their package. Itsupports a wide variety of languages, including C, and is able to fill the needsof most complex build configurations.
Multibuild is a set of CI scripts for building and testing Pythonwheels forLinux, macOS, and (less flexibly) Windows. Also seecibuildwheel.
nginx_pypi_cache is apackage index caching proxyusingnginx.
PDM is a modern Python package manager. It usespyproject.toml to storeproject metadata as defined inPEP 621.
Pex is a tool for generating.pex (Python EXecutable)files, standalone Python environments in the spirit ofvirtualenv.PEX files arezipapps thatmake deployment of Python applications as simple ascp. A single PEXfile can support multiple target platforms and can be created from standardpip-resolvable requirements, a lockfile generated withpex3lock...or even another PEX. PEX files can optionally have tools embedded that supportturning the PEX file into a standard venv, graphing dependencies and more.
pip-tools is a suite of tools meant for Python system administratorsand release managers who particularly want to keep their buildsdeterministic yet stay up to date with new versions of theirdependencies. Users can specify particular release of theirdependencies via hash, conveniently make a properly formatted list ofrequirements from information in other parts of their program, updateall dependencies (a featurepip currently does not provide), andcreate layers of constraints for the program to obey.
pip2pi is apackage index server where specificpackages are manually synchronised.
piwheels is a website, and software underpinning it, that fetchessource code distribution packages from PyPI and compiles them intobinary wheels that are optimized for installation onto Raspberry Picomputers. Raspberry Pi OS pre-configures pip to use piwheels.org asan additional index to PyPI.
poetry is a command-line tool to handle dependency installation andisolation as well as building and packaging of Python packages. Itusespyproject.toml and, instead of depending on the resolverfunctionality withinpip, provides its own dependency resolver.It attempts to speed users’ experience of installation and dependencyresolution by locally caching metadata about dependencies.
proxpi is a simplepackage index which proxies PyPIand other indexes with caching.
Pulp-python is the Pythonpackage index plugin forPulp. Pulp-python supports mirrors backed bylocal orAWS S3, package upload, and proxying to multiple packageindexes.
Warning
Not maintained, project archived
PyPI Cloud is apackage index server, backed byAWS S3 or another cloud storage service, or local files. PyPI Cloudsupports redirect/cached proxying for PyPI, as well as authentication andauthorisation.
pypiprivate serves a local (orAWS S3-hosted) directory of packages as apackage index.
pypiserver is a minimalist application that serves as a private Pythonpackage index (from a local directory) withinorganizations, implementing a simple API andbrowser interface. You can upload private packages using standardupload tools, and users can download and install them withpip,without publishing them publicly. Organizations who use pypiserverusually download packages both from pypiserver and from PyPI.
PyScaffold is a project generator for bootstrapping Python packages,ready to be shared on PyPI and installable viapip.It relies on a set of sane default configurations for established tools(such asSetuptools,pytest andSphinx) to provide a productiveenvironment so developers can start coding right away.PyScaffold can also be used with existing projects to make packagingeasier.
Warning
Not maintained, project archived
pywharf is apackage index server, serving fileslocally or fromGitHub.
Scikit-build is aSetuptools wrapper for CPython that buildsC/C++/Fortran/Cython extensions It usescmake (available on PyPI) to providebetter support for additional compilers, build systems, cross compilation, andlocating dependencies and their associated build requirements. To speed up andparallelize the build of large projects, the user can installninja (also available on PyPI).
Scikit-build-core is a build backend for CPython C/C++/Fortran/Cythonextensions. It enables users to write extensions withcmake (available on PyPI) to provide bettersupport for additional compilers, build systems, cross compilation, andlocating dependencies and their associated build requirements. CMake/Ninjaare automatically downloaded from PyPI if not available on the system.
shiv is a command line utility for building fully self containedPython zipapps as outlined inPEP 441, but with all theirdependencies included. Its primary goal is making distributing Pythonapplications and command line tools fast & easy.
simpleindex is apackage index which routes URLs tomultiple package indexes (including PyPI), serves local (or cloud-hosted,for exampleAWS S3, with a custom plugin) directories of packages, andsupports custom plugins.
A flexible package manager designed to support multiple versions,configurations, platforms, and compilers. Spack is like Homebrew, butpackages are written in Python and parameterized to allow easyswapping of compilers, library versions, build options,etc. Arbitrarily many versions of packages can coexist on the samesystem. Spack was designed for rapidly building high performancescientific applications on clusters and supercomputers.
Spack is not in PyPI (yet), but it requires no installation and can beused immediately after cloning from GitHub.
zest.releaser is a Python package release tool providing anabstraction layer on top oftwine. Python developers usezest.releaser to automate incrementing package version numbers,updating changelogs, tagging releases in source control, and uploadingnew packages to PyPI.
A package in the Python Standard Library that provides support for bootstrappingpip into an existing Python installation or virtual environment. In mostcases, end users won’t use this module, but rather it will be used during thebuild of the Python distribution.
A package and command-line interface which can host a directory as awebsite, for example as apackage index (seeHosting your own simple repository).
A package in the Python Standard Library (starting with Python 3.3) forcreatingVirtual Environments. For moreinformation, see the section onCreating Virtual Environments.