Stability of TOML

The release of TOML 1.0.0 in January 2021 indicates the TOML format shouldnow be officially considered stable. Empirically, TOML has proven to be astable format even prior to the release of TOML 1.0.0. From thechangelog, wecan see that TOML has had no major changes since April 2020, and has hadtwo releases in the past five years (2017-2021).

In the event of changes to the TOML specification, we can treat minorrevisions as bug fixes and update the implementation in place. In the event ofmajor breaking changes, we should preserve support for TOML 1.x.

Maintainability of proposed implementation

The proposed implementation (tomli) is pure Python, well tested andweighs in at under 1000 lines of code. It is minimalist, offering a smaller APIsurface area than other TOML implementations.

The author oftomli is willing to help integratetomli into the standardlibrary and help maintain it,as per this post.Furthermore, Python core developer Petr Viktorin has indicated a willingnessto maintain a read API,as per this post.

Rewriting the parser in C is not deemed necessary at this time. It is rare forTOML parsing to be a bottleneck in applications, and users with higher performanceneeds can use a third-party library (as is already often the case with JSON,despite Python offering a standard library C-extension module).

TOML support a slippery slope for other things

As discussed in theMotivation section, TOML holds a special place in thePython ecosystem, for readingPEP 518pyproject.toml packagingand tool configuration files.This chief reason to include TOML in the standard library does not apply toother formats, such as YAML or MessagePack.

In addition, the simplicity of TOML distinguishes it from other formats likeYAML, which are highly complicated to construct and parse.

An API for writing TOML may, however, be added in a future PEP.

Basing on another TOML implementation

Several potential alternative implementations exist:

• tomlkit is well established, actively maintained and supports TOML 1.0.0.An important difference is thattomlkit supports style roundtripping. As aresult, it has a more complex API and implementation (about 5x as much code astomli). Its author does not believe thattomlkit is a good choice forthe standard library.
• toml is a very widely used library. However, it is not activelymaintained, does not support TOML 1.0.0 and has a number of known bugs. ItsAPI is more complex than that oftomli. It allows customising output stylethrough a complicated encoder API, and some very limited and mostly unusedfunctionality to preserve input style through an undocumented decoder API.For more details on its API differences from this PEP, refer toAppendix A.
• pytomlpp is a Python wrapper for the C++ projecttoml++. Pure Pythonlibraries are easier to maintain than extension modules.
• rtoml is a Python wrapper for the Rust projecttoml-rs and hence hassimilar shortcomings topytomlpp.In addition, it does not support TOML 1.0.0.
• Writing an implementation from scratch. It’s unclear what we would get fromthis;tomli meets our needs and the author is willing to help with itsinclusion in the standard library.

Including an API for writing TOML

There are several reasons to not include an API for writing TOML.

The ability to write TOML is not needed for the use cases that motivate thisPEP: core Python packaging tools, and projects that need to read TOMLconfiguration files.

Use cases that involve editing an existing TOML file (as opposed to writing abrand new one) are better served by a style preserving library. TOML isintended as a human-readable and -editable configuration format, so it’simportant to preserve comments, formatting and other markup. This requiresa parser whose output includes style-related metadata, making it impracticalto output plain Python types likestr anddict. Furthermore, itsubstantially complicates the design of the API.

Even without considering style preservation, there are too many degrees offreedom in how to design a write API. For example, what default style(indentation, vertical and horizontal spacing, quotes, etc) should the libraryuse for the output, and how much control should users be given over it?How should the library handle input and output validation? Should it supportserialization of custom types, and if so, how? While there are reasonableoptions for resolving these issues, the nature of the standard library is suchthat we only get “one chance to get it right”.

Currently, no CPython core developers have expressed willingness to maintain awrite API, or sponsor a PEP that includes one. Since it is hard to changeor remove something in the standard library, it is safer to err on the side ofexclusion for now, and potentially revisit this later.

Therefore, writing TOML is left to third-party libraries. If a good API andrelevant use cases for it are found later, write support can be added in afuture PEP.

Assorted API details

Controlling the type of mappings returned bytomllib.load[s]

Thetoml library on PyPI accepts a_dict argument in itsload[s]functions, which works similarly to theobject_hook argument injson.load[s]. There are several uses of_dict found onhttps://grep.app; however, almost all of them are passing_dict=OrderedDict, which should be unnecessary as of Python 3.7.We found two instances of relevant use: in one case, a custom class was passedfor friendlier KeyErrors; in the other, the custom class had severaladditional lookup and mutation methods (e.g. to help resolve dotted keys).

Such a parameter is not necessary for the core use cases outlined in theMotivation section. The absence of this can be pretty easily worked aroundusing a wrapper class, transformer function, or a third-party library. Finally,support could be added later in a backward-compatible way.

Removing support forparse_float intomllib.load[s]

This option is not strictly necessary, since TOML floats should be implementedas “IEEE 754 binary64 values”, which is equivalent to a Pythonfloat on mostarchitectures.

The TOML specification uses the word “SHOULD”, however, implying arecommendation that can be ignored for valid reasons. Parsing floatsdifferently, such as todecimal.Decimal, allows users extra precision beyondthat promised by the TOML format. In the author oftomli’s experience, thisis particularly useful in scientific and financial applications. This is alsouseful for other cases that need greater precision, or where end-users includenon-developers who may not be aware of the limits of binary64 floats.

There are also niche architectures where the Pythonfloat is not a IEEE 754binary64 value. Theparse_float argument allows users to achieve correctTOML semantics even on such architectures.

Alternative names for the module

Ideally, we would be able to use thetoml module name.

However, thetoml package on PyPI is widely used, so there are backwardcompatibility concerns. Since the standard library takes precedence over thirdparty packages, libraries and applications who current depend on thetomlpackage would likely break when upgrading Python versions due to the manyAPI incompatibilities listed inAppendix A,even if they pin their dependency versions.

To further clarify, applications with pinned dependencies are of greatestconcern here. Even if we were able to obtain control of thetoml PyPIpackage name and repurpose it for a backport of the proposed new module,we would still break users on new Python versions that included it in thestandard library, regardless of whether they have pinned an older version ofthe existingtoml package. This is unfortunate, since pinningwould likely be a common response to breaking changes introduced by repurposingthetoml package as a backport (that is incompatible with today’stoml).

Finally, thetoml package on PyPI is not actively maintained, but as ofyet, efforts to request that the author add other maintainershave been unsuccessful,so action here would likely have to be taken without the author’s consent.

Instead, this PEP proposes the nametomllib. This mirrorsplistlibandxdrlib, two other file format modules in the standard library, as wellas other modules, such aspathlib,contextlib andgraphlib.

Other names considered but rejected include:

• tomlparser. This mirrorsconfigparser, but is perhaps somewhat lessappropriate if we include a write API in the future.
• tomli. This assumes we usetomli as the basis for implementation.
• toml under some namespace, such asparser.toml. However, this isawkward, especially so since existing parsing libraries likejson,pickle,xml,html etc. would not be included in the namespace.