@@ -12,6 +12,7 @@ Created: 04-Aug-2023
1212Post-History: `04-Aug-2023 <https://discuss.python.org/t/30979 >`__,
1313 `06-Aug-2023 <https://discuss.python.org/t/31151 >`__,
1414 `23-Aug-2023 <https://discuss.python.org/t/32149 >`__,
15+ `06-Dec-2023 <https://discuss.python.org/t/40418 >`__,
1516Replaces: 722
1617Resolution: https://discuss.python.org/t/36763
1718
@@ -55,23 +56,12 @@ Rationale
5556This PEP defines a mechanism for embedding metadata *within the script itself *,
5657and not in an external file.
5758
58- We choose to follow the latest developments of other modern packaging
59- ecosystems (namely `Go `__ and provisionally `Rust `__) by embedding the existing
60- file used to describe projects, in our case ``pyproject.toml ``.
61-
62- __ https://github.com/erning/gorun
63- __ https://rust-lang.github.io/rfcs/3424-cargo-script.html
64-
65- The format is intended to bridge the gap between different types of users
66- of Python. Users will benefit from seamless interoperability with tools that
67- already work with TOML.
68-
69- One of the central themes we discovered from the recent
70- `packaging survey <https://discuss.python.org/t/22420 >`__ is that users have
71- begun getting frustrated with the lack of unification regarding both tooling
72- and specs. Adding yet another metadata format (like:pep: `722 ` syntax for a
73- list of dependencies), even for a currently unsatisfied use case, would
74- further fragment the community.
59+ The metadata format is designed to be similar to the layout of data in the
60+ ``pyproject.toml `` file of a Python project directory, to provide a familiar
61+ experience for users who have experience writing Python projects. By using a
62+ similar format, we avoid unnecessary inconsistency between packaging tools,
63+ a common frustration expressed by users in the recent
64+ `packaging survey <https://discuss.python.org/t/22420 >`__.
7565
7666The following are some of the use cases that this PEP wishes to support:
7767
@@ -83,11 +73,8 @@ The following are some of the use cases that this PEP wishes to support:
8373* A script that desires to transition to a directory-type project. A user may
8474 be rapidly prototyping locally or in a remote REPL environment and then
8575 decide to transition to a more formal project layout if their idea works
86- out. This intermediate script stage would be very useful to have fully
87- reproducible bug reports. By using the same format, the user can simply copy
88- and paste the metadata into a ``pyproject.toml `` file and continue working
89- without having to learn a new format. More likely, even, is that tooling will
90- eventually support this transformation with a single command.
76+ out. Being able to define dependencies in the script would be very useful to
77+ have fully reproducible bug reports.
9178* Users that wish to avoid manual dependency management. For example, package
9279 managers that have commands to add/remove dependencies or dependency update
9380 automation in CI that triggers based on new versions or in response to
@@ -155,43 +142,39 @@ and the regular expression, the text specification takes precedence.
155142Tools MUST NOT read from metadata blocks with types that have not been
156143standardized by this PEP or future ones.
157144
158- pyproject type
159- --------------
145+ script type
146+ -----------
160147
161- The first type of metadata block is named ``pyproject `` whichrepresents
162- content similar to [ 3 ]_ what one would see in a `` pyproject.toml `` file .
148+ The first type of metadata block is named ``script `` whichcontains script
149+ metadata (dependency data and tool configuration) .
163150
164- This document MAY include the ``[run] `` and ``[tool] `` tables.
151+ This document MAY include top-level fields ``dependencies `` and ``requires-python ``,
152+ and MAY optionally include a ``[tool] `` table.
165153
166- The:pep: `tool table <518#tool-table >` MAY be used by any tool, script runner
167- or otherwise, to configure behavior.
154+ The ``[tool] `` table MAY be used by any tool, script runner or otherwise, to configure
155+ behavior. It has the same semantics as the:pep: `tool table <518#tool-table >` in
156+ ``pyproject.toml ``.
168157
169- The`` [run] `` table MAY include the following optional fields:
158+ Thetop-level fields are :
170159
171160* ``dependencies ``: A list of strings that specifies the runtime dependencies
172161 of the script. Each entry MUST be a valid:pep: `508 ` dependency.
173162* ``requires-python ``: A string that specifies the Python version(s) with which
174163 the script is compatible. The value of this field MUST be a valid
175164:pep: `version specifier <440#version-specifiers >`.
176165
177- Any future PEPs that define additional fields for the ``[run] `` table when used
178- in a ``pyproject.toml `` file MUST include the aforementioned fields exactly as
179- specified. The fields defined by this PEP are equally as applicable to
180- full-fledged projects as they are to single-file scripts.
181-
182166Script runners MUST error if the specified ``dependencies `` cannot be provided.
183167Script runners SHOULD error if no version of Python that satisfies the specified
184168``requires-python `` can be provided.
185169
186170Example
187171-------
188172
189- The following is an example of a script withan embedded`` pyproject.toml `` :
173+ The following is an example of a script with embeddedmetadata :
190174
191175..code ::python
192176
193- # /// pyproject
194- # [run]
177+ # /// script
195178# requires-python = ">=3.11"
196179# dependencies = [
197180# "requests<3",
@@ -206,24 +189,6 @@ The following is an example of a script with an embedded ``pyproject.toml``:
206189 data= resp.json()
207190 pprint([(k, v[" title" ])for k, vin data.items()][:10 ])
208191
209- The following [4 ]_ is an example of a proposed syntax for single-file Rust
210- projects that embeds their equivalent of ``pyproject.toml ``, which is called
211- ``Cargo.toml ``:
212-
213- ..code ::rust
214-
215- #!/usr/bin/env cargo
216-
217- //! ```cargo
218- //! [dependencies]
219- //! regex = "1.8.0"
220- //! ```
221-
222- fn main() {
223- let re = Regex::new(r"^\d{4}-\d{2}-\d{2}$").unwrap();
224- println!("Did our date match? {}", re.is_match("2014-01-01"));
225- }
226-
227192 Reference Implementation
228193========================
229194
@@ -238,7 +203,7 @@ higher.
238203REGEX = r ' (?m) ^ # ///( ?P<type> [a-zA-Z0-9- ]+ ) $ \s ( ?P<content> ( ^ #( | . * ) $ \s ) + ) ^ # ///$ '
239204
240205def read (script :str ) ->dict | None :
241- name= ' pyproject '
206+ name= ' script '
242207 matches= list (
243208filter (lambda m : m.group(' type' )== name, re.finditer(REGEX , script))
244209 )
@@ -275,7 +240,7 @@ __ https://tomlkit.readthedocs.io/en/latest/
275240 )
276241
277242 config= tomlkit.parse(content)
278- config[' project ' ][ ' dependencies' ].append(dependency)
243+ config[' dependencies' ].append(dependency)
279244 new_content= ' ' .join(
280245f ' # { line} ' if line.strip()else f ' # { line} '
281246for linein tomlkit.dumps(config).splitlines(keepends = True )
@@ -309,24 +274,24 @@ blocks.
309274 Backwards Compatibility
310275=======================
311276
312- At the time of writing, the ``# ///pyproject `` block comment starter does not
313- appear `on GitHub `__. Therefore, there is little risk of existing scripts being
314- broken by this PEP.
277+ At the time of writing, the ``# ///script `` block comment starter does not
278+ appearin any Python files `on GitHub `__. Therefore, there is little risk of existing
279+ scripts being broken by this PEP.
315280
316- __ https://github.com/search?q=%22%23+%2F%2F%2F+pyproject %22&type=code
281+ __ https://github.com/search?q=%22%23+%2F%2F%2F+script %22&type=code
317282
318283
319284Security Implications
320285=====================
321286
322- If a script containing embedded metadata isran using a tool that automatically
287+ If a script containing embedded metadata isrun using a tool that automatically
323288installs dependencies, this could cause arbitrary code to be downloaded and
324289installed in the user's environment.
325290
326291The risk here is part of the functionality of the tool being used to run the
327292script, and as such should already be addressed by the tool itself. The only
328- additional risk introduced by this PEP is if an untrusted script witha
329- embedded metadata is run, when a potentially malicious dependency or transitive
293+ additional risk introduced by this PEP is if an untrusted script withembedded
294+ metadata is run, when a potentially malicious dependency or transitive
330295dependency might be installed.
331296
332297This risk is addressed by the normal good practice of reviewing code
@@ -338,24 +303,21 @@ How to Teach This
338303=================
339304
340305To embed metadata in a script, define a comment block that starts with the
341- line ``# ///pyproject `` and ends with the line ``# /// ``. Every line between
306+ line ``# ///script `` and ends with the line ``# /// ``. Every line between
342307those two lines must be a comment and the full content is derived by removing
343- the first two characters. The ``pyproject `` type indicates that the content
344- is TOML and resembles a ``pyproject.toml `` file.
308+ the first two characters.
345309
346310..code ::python
347311
348- # /// pyproject
349- # [run]
312+ # /// script
350313# dependencies = [
351314# "requests<3",
352315# "rich",
353316# ]
354317# requires-python = ">=3.11"
355318# ///
356319
357- The two allowed tables are ``[run] `` and ``[tool] ``. The ``[run] `` table may
358- contain the following fields:
320+ The allowed fields are described in the following table:
359321
360322..list-table ::
361323
@@ -376,6 +338,10 @@ contain the following fields:
376338 - Tools might error if no version of Python that satisfies
377339 the constraint can be executed.
378340
341+ In addition, a ``[tool] `` table is allowed. Details of what is permitted are similar
342+ to what is permitted in ``pyproject.toml ``, but precise information must be included
343+ in the documentation of the relevant tool.
344+
379345It is up to individual tools whether or not their behavior is altered based on
380346the embedded metadata. For example, every script runner may not be able to
381347provide an environment for specific Python versions as defined by the
@@ -455,7 +421,7 @@ would live as single-file scripts:
455421 up each Python script with a shebang line pointing to the desired Python
456422 executable or script runner.
457423
458- This PEP argues thatreusing our TOML-based metadata format is the best for
424+ This PEP argues thatthe proposed TOML-based metadata format is the best for
459425each category of user and that the requirements-like block comment is only
460426approachable for those who have familiarity with ``requirements.txt ``, which
461427represents a small subset of users.
@@ -464,12 +430,11 @@ represents a small subset of users.
464430 already starting with zero context and are unlikely to be familiar with
465431 TOML nor ``requirements.txt ``. These users will very likely rely on
466432 snippets found online via a search engine or utilize AI in the form
467- of a chat bot or direct code completion software. Searching for Python
468- metadata formatting will lead them to the TOML-based format that already
469- exists which they can reuse. The author tested GitHub Copilot with this
470- PEP and it already supports auto-completion of ``dependencies ``. In contrast,
471- a new format may take years of being trained on the Internet for models to
472- learn.
433+ of a chat bot or direct code completion software. The similarity with
434+ dependency information stored in ``pyproject.toml `` will provide useful
435+ search results relatively quickly, and while the ``pyproject.toml `` format
436+ and the script metadata format are not the same, any resulting discrepancies
437+ are unlikely to be difficult for the intended users to resolve.
473438
474439 Additionally, these users are most susceptible to formatting quirks and
475440 syntax errors. TOML is a well-defined format with existing online
@@ -484,7 +449,7 @@ represents a small subset of users.
484449 with TOML since they are used to structured data formats and there would be
485450 less perceived magic in their systems.
486451
487- Additionally, for maintenance of their systems ``///pyproject `` would be
452+ Additionally, for maintenance of their systems ``///script `` would be
488453 much easier to search for from a shell than a block comment with potentially
489454 numerous extensions over time.
490455* For the SRE types, they are likely to be familiar with TOML already from
@@ -817,15 +782,7 @@ Footnotes
817782
818783__ https://github.com/facelessuser/pymdown-extensions/discussions/1973
819784__ https://github.com/Python-Markdown/markdown
820- .. [3 ]A future PEP that officially introduces the ``[run] `` table to
821- ``pyproject.toml `` files will make this PEP not just similar but a strict
822- subset.
823- .. [4 ]One important thing to note is that the metadata is embedded in a
824- `doc-comment `__ (their equivalent of docstrings). `Other syntaxes `__ are
825- under consideration within the Rust project.
826-
827- __ https://doc.rust-lang.org/stable/book/ch14-02-publishing-to-crates-io.html#making-useful-documentation-comments
828- __ https://github.com/epage/cargo-script-mvs/blob/main/0000-cargo-script.md#embedded-manifest-format
785+
829786
830787
831788 Copyright