This PEP specifies how Python software packages should specify whatbuild dependencies they have in order to execute their chosen buildsystem. As part of this specification, a new configuration file isintroduced for software packages to use to specify their builddependencies (with the expectation that the same configuration filewill be used for future configuration details).
When Python first developed its tooling for building distributions ofsoftware for projects, distutils[1] was the chosensolution. As time went on, setuptools[2] gained popularityto add some features on top of distutils. Both used the concept of asetup.py file that project maintainers executed to builddistributions of their software (as well as users to install saiddistribution).
Using an executable file to specify build requirements under distutilsisn’t an issue as distutils is part of Python’s standard library.Having the build tool as part of Python means that asetup.py hasno external dependency that a project maintainer needs to worry aboutto build a distribution of their project. There was no need to specifyany dependency information as the only dependency is Python.
But when a project chooses to use setuptools, the use of an executablefile likesetup.py becomes an issue. You can’t execute asetup.py file without knowing its dependencies, but currentlythere is no standard way to know what those dependencies are in anautomated fashion without executing thesetup.py file where thatinformation is stored. It’s a catch-22 of a file not being runnablewithout knowing its own contents which can’t be known programmaticallyunless you run the file.
Setuptools tried to solve this with asetup_requires argument toitssetup() function[3]. This solution has a numberof issues, such as:
setup.py, butsetup.py can’t beexecuted without having these items installed.setup()function, which means that the only way to actually use anythingadded here is through increasingly complex machinations that delaythe import and usage of these modules until later on in theexecution of thesetup() function.setuptools itself nor can it include areplacement tosetuptools, which means that projects such asnumpy.distutils are largely incapable of utilizing it andprojects cannot take advantage of newer setuptools features untiltheir users naturally upgrade the version of setuptools to a newerone.setup_requires get implicitly installedwhenever you execute thesetup.py but one of the common waysthat thesetup.py is executed is via another tool, such aspip, who is already managing dependencies. This means thata command likepipinstallspam might end up having bothpip and setuptools downloading and installing packages and endusers needing to configureboth tools (and forsetuptoolswithout being in control of the invocation) to change settingslike which repository it installs from. It also means that usersneed to be aware of the discovery rules for both tools, as onemay support different package formats or determine the latestversion differently.This has culminated in a situation where use ofsetup_requiresis rare, where projects tend to either simply copy and paste snippetsbetweensetup.py files or they eschew it all together in favorof simply documenting elsewhere what they expect the user to havemanually installed prior to attempting to build or install theirproject.
All of this has led pip[4] to simply assume that setuptools isnecessary when executing asetup.py file. The problem with this,though, is it doesn’t scale if another project began to gain tractionin the community as setuptools has. It also prevents other projectsfrom gaining traction due to the friction required to use it with aproject when pip can’t infer the fact that something other thansetuptools is required.
This PEP attempts to rectify the situation by specifying a way to listthe minimal dependencies of the build system of a project in adeclarative fashion in a specific file. This allows a project to listwhat build dependencies it has to go from e.g. source checkout towheel, while not falling into the catch-22 trap that asetup.pyhas where tooling can’t infer what a project needs to build itself.Implementing this PEP will allow projects to specify what build systemthey depend on upfront so that tools like pip can make sure that theyare installed in order to run the build system to build the project.
To provide more context and motivation for this PEP, think of the(rough) steps required to produce a built artifact for a project:
This PEP covers step #2.PEP 517 covers step #3, including how to havethe build system dynamically specify more dependencies that the buildsystem requires to perform its job. The purpose of this PEP though, isto specify the minimal set of requirements for the build system tosimply begin execution.
The build system dependencies will be stored in a file namedpyproject.toml that is written in the TOML format[6].
This format was chosen as it is human-usable (unlike JSON[7]),it is flexible enough (unlike configparser[9]), stemsfrom a standard (also unlike configparser[9]), and itis not overly complex (unlike YAML[8]). The TOML format isalready in use by the Rust community as part of theirCargo package manager[14] and in private email stated they havebeen quite happy with their choice of TOML. A more thoroughdiscussion as to why various alternatives were not chosen can be readin theOther file formats section. The authors do realize, though,that choice of configuration file format is ultimately subjective anda choice had to be made and the authors prefer TOML for this situation.
Below we list the tables that tools are expected to recognize/respect.Tables not specified in this PEP are reserved for future use by otherPEPs.
The[build-system] table is used to store build-related data.Initially only one key of the table will be valid and is mandatoryfor the table:requires. This key must have a value of a listof strings representingPEP 508 dependencies required to execute thebuild system (currently that means what dependencies are required toexecute asetup.py file).
For the vast majority of Python projects that rely upon setuptools,thepyproject.toml file will be:
[build-system]# Minimum requirements for the build system to execute.requires=["setuptools"]# PEP 508 specifications.
Because the use of setuptools is so expansive in thecommunity at the moment, build tools are expected to use the exampleconfiguration file above as their default semantics when apyproject.toml file is not present.
Tools should not require the existence of the[build-system] table.Apyproject.toml file may be used to store configuration detailsother than build-related data and thus lack a[build-system] tablelegitimately. If the file exists but is lacking the[build-system]table then the default values as specified above should be used.If the table is specified but is missing required fields then the toolshould consider it an error.
The[tool] table is where any tool related to your Pythonproject, not just build tools, can have users specify configurationdata as long as they use a sub-table within[tool], e.g. theflit tool would store itsconfiguration in[tool.flit].
We need some mechanism to allocate names within thetool.*namespace, to make sure that different projects don’t attempt to usethe same sub-table and collide. Our rule is that a project can usethe subtabletool.$NAME if, and only if, they own the entry for$NAME in the Cheeseshop/PyPI.
To provide a type-specific representation of the resulting data fromthe TOML file for illustrative purposes only, the following JSONSchema[15] would match the data format:
{"$schema":"http://json-schema.org/schema#","type":"object","additionalProperties":false,"properties":{"build-system":{"type":"object","additionalProperties":false,"properties":{"requires":{"type":"array","items":{"type":"string"}}},"required":["requires"]},"tool":{"type":"object"}}}
For future-proofing the structure of the configuration file, asemantics-version key was initially proposed. Defaulting to1,the idea was that if any semantics changes to previously defined keysor tables occurred which were not backwards-compatible, then thesemantics-version would be incremented to a new number.
In the end, though, it was decided that this was a prematureoptimization. The expectation is that changes to what is pre-definedsemantically in the configuration file will be rather conservative.And in the instances where a backwards-incompatible change would haveoccurred, different names can be used for the new semantics to avoidbreaking older tools.
An earlier draft of this PEP had a top-level[package] table. Theidea was to impose some scoping for a semantics versioning scheme(seeA semantic version key for why that idea was rejected).With the need for scoping removed, the point of having a top-leveltable became superfluous.
Another name proposed for the[build-system] table was[build]. The alternative name is shorter, but doesn’t convey asmuch of the intention of what information is stored in the table. Aftera vote on the distutils-sig mailing list, the current name won out.
Several other file formats were put forward for consideration, allrejected for various reasons. Key requirements were that the formatbe editable by human beings and have an implementation that can bevendored easily by projects. This outright excluded certain formatslike XML which are not friendly towards human beings and were neverseriously discussed.
The key reasons for rejecting the other alternatives considered aresummarised in the following sections, while the full review (includingpositive arguments in favour of TOML) can be found at[16].
TOML was ultimately selected as it provided all the features wewere interested in, while avoiding the downsides introduced bythe alternatives.
| Feature | TOML | YAML | JSON | CFG/INI |
|---|---|---|---|---|
| Well-defined | yes | yes | yes | |
| Real data types | yes | yes | yes | |
| Reliable Unicode | yes | yes | yes | |
| Reliable comments | yes | yes | ||
| Easy for humans to edit | yes | ?? | ?? | |
| Easy for tools to edit | yes | ?? | yes | ?? |
| In standard library | yes | yes | ||
| Easy for pip to vendor | yes | n/a | n/a |
(“??” in the table indicates items where most folks would beinclined to answer “yes”, but there turn out to be a lot ofquirks and edge cases that arise in practice due to eitherthe lack of a clear specification, or else the underlyingfile format specification being surprisingly complicated)
Thepytoml TOML parser is ~300 lines of pure Python code,so being outside the standard library didn’t count heavilyagainst it.
Python literals were also discussed as a potential format, butweren’t considered in the file format review (since they’re nota common pre-existing file format).
The JSON format[7] was initially considered but quicklyrejected. While great as a human-readable, string-based data exchangeformat, the syntax does not lend itself to easy editing by a humanbeing (e.g. the syntax is more verbose than necessary while notallowing for comments).
An example JSON file for the proposed data would be:
{"build":{"requires":["setuptools","wheel>=0.27"]}}
The YAML format[8] was designed to be a superset of JSON[7] while being easier to work with by hand. There are three mainissues with YAML.
One is that the specification is large: 86 pages if printed onletter-sized paper. That leaves the possibility that someone may use afeature of YAML that works with one parser but not another. It hasbeen suggested to standardize on a subset, but that basically meanscreating a new standard specific to this file which is not tractablelong-term.
Two is that YAML itself is not safe by default. The specificationallows for the arbitrary execution of code which is best avoided whendealing with configuration data. It is of course possible to avoidthis behavior – for example, PyYAML provides asafe_load operation– but if any tool carelessly usesload instead then they openthemselves up to arbitrary code execution. While this PEP is focused onthe building of projects which inherently involves code execution,other configuration data such as project name and version number mayend up in the same file someday where arbitrary code execution is notdesired.
And finally, the most popular Python implementation of YAML isPyYAML[10] which is a large project of a few thousand lines ofcode and an optional C extension module. While in and of itself thisisn’t necessarily an issue, this becomes more of a problem forprojects like pip where they would most likely need to vendor PyYAMLas a dependency so as to be fully self-contained (otherwise you endup with your install tool needing an install tool to work). Aproof-of-concept re-working of PyYAML has been done to see how easyit would be to potentially vendor a simpler version of the librarywhich shows it is a possibility.
An example YAML file is:
build:requires:-setuptools-wheel>=0.27
An INI-style configuration file based on whatconfigparser[9] accepts was considered. Unfortunatelythere is no specification of what configparser accepts, leading tosupport skew between versions. For instance, what ConfigParser inPython 2.7 accepts is not the same as what configparser in Python 3accepts. While one could standardize on what Python 3 accepts andsimply vendor the backport of the configparser module, that does meanthis PEP would have to codify that the backport of configparser mustbe used by all project wishes to consume the metadata specified bythis PEP. This is overly restrictive and could lead to confusion ifsomeone is not aware of that a specific version of configparser isexpected.
An example INI file is:
[build]requires=setuptoolswheel>=0.27
Someone proposed using Python literals as the configuration format.The file would contain one dict at the top level, with the data allinside that dict, with sections defined by the keys. All Pythonprogrammers would be used to the format, there would implicitly be nothird-party dependency to read the configuration data, and it can besafe if parsed byast.literal_eval()[13].Python literals can be identical to JSON, with the added benefit ofsupporting trailing commas and comments. In addition, Python’s richerdata model may be useful for some future configuration needs (e.g. non-stringdict keys, floating point vs. integer values).
On the other hand, python literals are a Python-specific format, andit is anticipated that these data may need to be read by packagingtools, etc. that are not written in Python.
An example Python literal file for the proposed data would be:
# The build configuration{"build":{"requires":["setuptools","wheel>=0.27",# note the trailing comma# "numpy>=1.10" # a commented out data line]# and here is an arbitrary comment.}}
setup.cfgThere are two issues withsetup.cfg used by setuptools as a generalformat. One is that they are.ini files which have issues as mentionedin theconfigparser discussion above. The other is that the schema forthat file has never been rigorously defined and thus it’s unknown whichformat would be safe to use going forward without potentially confusingsetuptools installations.
Several other file names were considered and rejected (although thisis very much a bikeshedding topic, and so the decision comes down tomostly taste).
setup.py, it does notnecessarily match what the file may contain in the future (e.g. isknowing the name of a project inherently part of its setup?).ast.literal_eval()(https://docs.python.org/3/library/ast.html#ast.literal_eval)This document has been placed in the public domain.
Source:https://github.com/python/peps/blob/main/peps/pep-0518.rst
Last modified:2025-09-22 12:21:58 GMT