Abstract

Python package metadata is stored in the distribution file in a standardformat, defined in theCore Metadata Specification. However, forsource distributions, while the format of the data is defined, there hastraditionally been a lot of inconsistency in what data is recorded inthe source distribution. Seeherefor a discussion of this issue.

As a result, metadata consumers are unable to rely on the data availablefrom source distributions, and need to use the (costly)PEP 517 buildmechanisms to extract medatata.

This PEP defines a standard that allows build backends to reliably storepackage metadata in the source distribution, while still retaining thenecessary flexibility to handle metadata fields that have to be calculatedat build time.

Motivation

There are a number of issues with the way that metadata is currentlystored in source distributions:

• The details of how to store metadata, while standardised, are noteasy to find.
• The specification requires an old metadata version, and has not beenupdated in line with changes to the core metadata spec.
• There is no way in the spec to distinguish between “this field has beenomitted because its value will not be known until build time” and “thisfield does not have a value”.
• The core metadata specification allows most fields to be optional,meaning that the previous issue affects nearly every metadata field.

This PEP proposes an update to the metadata specification to allowrecording of fields which are expected to be “filled in later”, andupdates the source distribution specification to clarify that backendsshould record sdist metadata using that version of the spec (or later).

Rationale

This PEP allows projects to define source distribution metadata valuesas being “dynamic”. In this context, saying that a field is “dynamic”means that the value has not been fixed at the time that the sourcedistribution was generated. Dynamic values will be supplied by the buildbackend at the time when the wheel is generated, and could depend ondetails of the build environment.

PEP 621 has a similar concept, of “dynamic” values that will be“filled in later”, and so we choose to use the same term here byanalogy.

Specification

This PEP defines the relationship between metadata values specified in asource distribution, and the corresponding values in wheels built fromit. It requires build backends to clearly mark any fields which willnot simply be copied unchanged from the sdist to the wheel.

In addition, this PEP makes thePyPA Specifications document thecanonical location for the specification of the source distributionformat (collecting the information inPEP 517 and in this PEP).

A new field,Dynamic, will be added to theCore Metadata Specification.This field will be multiple use, and will be allowed to contain the nameof another core metadata field.

When found in the metadata of a source distribution, the followingrules apply:

1. If a field isnot marked asDynamic, then the value of the fieldin any wheel built from the sdist MUST match the value in the sdist.If the field is not in the sdist, and not marked asDynamic, thenit MUST NOT be present in the wheel.
2. If a field is marked asDynamic, it may contain any valid value ina wheel built from the sdist (including not being present at all).
3. Backends MUST NOT mark a field asDynamic if they can determine thatit was generated from data that will not change at build time.

Backends MAY record the value they calculated for a field they mark asDynamic in a source distribution. Consumers, however, MUST NOT treatthis value as canonical, but MAY use it as an hint about what the finalvalue in a wheel could be.

In any context other than a source distribution, if a field is marked asDynamic, that indicates that the value was generated at wheel buildtime and may not match the value in the sdist (or in other builds of theproject). Backends are not required to record this information, though,and consumers MUST NOT assume that the lack of aDynamic marking hasany significance, except in a source distribution.

The fieldsName andVersion MUST NOT be marked asDynamic.

As it adds a new metadata field, this PEP updates the core metadataformat to version 2.2.

Source distributions SHOULD use the latest version of the core metadataspecification that was available when they were created.

Build backends are strongly encouraged to only mark fields asDynamic when absolutely necessary, and to encourage projects toavoid backend features that require the use ofDynamic. Projectsshould prefer to use environment markers on static values to adapt todetails of the install location.

Backwards Compatibility

As this proposal increments the core metadata version, it is compatiblewith existing source distributions, which will use an older metadataversion. Tools can determine whether a source distribution conforms tothis PEP by checking the metadata version.

Security Implications

As this specification is purely for the storage of data that is intendedto be publicly available, there are no security implications.

How to Teach This

This is a data storage format for project metadata, and so will nottypically be visible to end users. There is therefore no need to teachusers how to use the format. Developers wanting to reference themetadata will be able to find the details in thePyPA Specifications.

Rejected Ideas

1. Rather than marking fields asDynamic, fields should be assumedto be dynamic unless explicitly marked asStatic.

   This is logically equivalent to the current proposal, but it impliesthat fields being dynamic is the norm. Packaging tools can be muchmore efficient in the presence of metadata that is known to be static,so the PEP chooses to make dynamic fields the exception, and requirebackends to “opt in” to making a field dynamic.

   In addition, if dynamic is the default, then in future, as moreand more metadata becomes static, metadata files will include anincreasing number ofStatic declarations.

2. Rather than having aDynamic field, add a special value thatindicates that a field is “not yet defined”.

   Again, this is logically equivalent to the current proposal. It makes“being dynamic” an explicit choice, but requires a special value. Assome fields can contain arbitrary text, choosing a such a value issomewhat awkward (although likely not a problem in practice). Theredoes not seem to be enough benefit to this approach to make it worthusing instead of the proposed mechanism.

3. Special handling ofRequires-Python.

   Early drafts of the PEP needed special discussion ofRequires-Python,because the lack of environment markers for this field meant that it mightbe difficult to require it to be static. The final form of the PEP no longerneeds this, as the idea of a whitelist of fields allowed to be dynamic wasdropped.

4. Restrict the use ofDynamic to a minimal “white list” ofpermitted fields.

   This approach was likely to prove extremely difficult for setuptoolsto implement in a backward compatible way, due to the dynamic natureof the setuptools interface. Instead, the proposal now allows mostfields to be dynamic, but encourages backends to avoid dynamic valuesunless essential.

Open Issues

None

Copyright

This document is placed in the public domain or under theCC0-1.0-Universal license, whichever is more permissive.