Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Python: Add support for template string literals#20708

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Open
tausbn wants to merge10 commits intomain
base:main
Choose a base branch
Loading
fromtausbn/python-add-support-for-template-string-literals

Conversation

@tausbn
Copy link
Contributor

@tausbntausbn commentedOct 28, 2025
edited
Loading

Extends the parser and libraries to support the new t-string syntax introduced in Python 3.14 (cf.PEP-750)

Due to the complexity of our current handling of f-strings, I optednot to extend the existing f-string support to also handle t-strings. Instead, t-strings are a completely separate (and much simpler) construction.

@tausbntausbnforce-pushed thetausbn/python-add-support-for-template-string-literals branch from691a54f tob6c5b53CompareDecember 4, 2025 13:48
@tausbntausbnforce-pushed thetausbn/python-add-support-for-template-string-literals branch 6 times, most recently from6d2d0eb to98279f7CompareDecember 4, 2025 21:49
@tausbntausbn marked this pull request as ready for reviewDecember 4, 2025 22:38
@tausbntausbn requested review froma team ascode ownersDecember 4, 2025 22:38
CopilotAI review requested due to automatic review settingsDecember 4, 2025 22:38
Copy link
Contributor

CopilotAI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Pull request overview

This pull request adds support for Python 3.14's template string literals (t-strings) as defined in PEP-750. The implementation introduces new AST nodes (TemplateString,JoinedTemplateString,TemplateStringPart) to represent template strings, taking a simpler approach than the existing f-string support.

Key Changes:

  • Extended the Tree-sitter scanner and grammar to recognize the 't'/'T' prefix for template strings
  • Added new database schema entities for template string AST nodes with proper upgrade/downgrade paths
  • Created new QL classes to represent template strings in the library

Reviewed changes

Copilot reviewed 25 out of 26 changed files in this pull request and generated 1 comment.

Show a summary per file
FileDescription
python/ql/lib/upgrades/acf8d3b08ae3cfac8833d16efbfa5a10fef86819/*Database upgrade files for adding template string support
python/ql/lib/semmlecode.python.dbscheme*Schema updates defining new template string AST nodes
python/ql/lib/semmle/python/AstGenerated.qllGenerated QL classes for template string nodes
python/ql/lib/semmle/python/AstExtended.qllExtended classes for template string functionality
python/extractor/tsg-python/tsp/src/scanner.ccScanner logic to recognize t-string prefix and handle interpolation
python/extractor/tsg-python/tsp/grammar.jsGrammar rules for template string syntax
python/extractor/tsg-python/src/main.rsRust extractor updates for handling t-string prefixes
python/extractor/semmle/python/*.pyPython extractor updates for template string AST nodes
python/extractor/tests/parser/template_strings_new.*Test cases for template string parsing
python/downgrades/8d257a4a9bc78e39856d6cd33499389fc5148d4f/*Downgrade path for removing template string support
python/ql/lib/change-notes/2025-12-04-support-template-string-literals.mdRelease note documenting the new feature

💡Add Copilot custom instructions for smarter, more guided reviews.Learn how to get started.

Comment on lines +7 to +17
t""
if2:
t"Hello, {name}!"
if3:
t"Value: {value:.2f}, Hex: {value:#x}"
if4:
t"Just a regular string."
if5:
t"Multiple {first} and {second} placeholders."
if6:
t"Implicit concatenation: "t"Hello, {name}!"t" How are you?"
Copy link

CopilotAIDec 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Syntax Error (in Python 3).

Suggested change
t""
if2:
t"Hello, {name}!"
if3:
t"Value: {value:.2f}, Hex: {value:#x}"
if4:
t"Just a regular string."
if5:
t"Multiple {first} and {second} placeholders."
if6:
t"Implicit concatenation: "t"Hello, {name}!"t" How are you?"
""
if2:
f"Hello,{name}!"
if3:
f"Value:{value:.2f}, Hex:{value:#x}"
if4:
"Just a regular string."
if5:
f"Multiple{first} and{second} placeholders."
if6:
"Implicit concatenation: "+f"Hello,{name}!"+" How are you?"

Copilot uses AI. Check for mistakes.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This is a pretty interesting edgecase from Copilot. I would have thought it could figure it out based on the PR description. I wonder if there was some overly verbose comments in this file if it would still have said this?

But it's literally in the filename...

HeeMa6 reacted with thumbs up emoji
yoff
yoff previously approved these changesDec 16, 2025
Copy link
Contributor

@yoffyoff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I think f-strings had their own delimiter at some point, but then we could get away without it. It seems like it would have been more future-proof to keep it :-)
I am actually somewhat OK with template strings being singled out since theyare special: They are more complex objects with accessible fields. But I do feel a little bit that we have with this PR classified the existing code as tech-debt, and we should make the grammar more symmetric at some point...

- Extends the scanner with a new token kind representing the start of atemplate string. This is used to distinguish template strings fromregular strings (because only a template string will start with a`_template_string_start` external token).- Cleans up the logic surrounding interpolations (and the method names)so that format strings and template strings behave the same in thiscase.Finally, we add two new node types in the tree-sitter grammar:- `template_string` behaves like format strings, but is a distinct type(mainly so that an implicit concatenation between template strings andregular strings becomes a syntax error).- `concatenated_template_string` is the counterpart of`concatenated_string`.However, internally, the string parts of a template strings are just thesame `string_content` nodes that are used in regular format strings. Wewill disambiguate these inside `tsg-python`.
Adds three new AST nodes to the mix:- `TemplateString` represents a t-string in Python 3.14- `TemplateStringPart` represents one of the string constituents of at-string. (The interpolated expressions are represented as `Expr` nodes,just like f-strings.)- `JoinedTemplateString` represents an implicit concatenation oftemplate strings.Importantly, we _completely avoid_ the complicated construction wecurrently do for format strings (as well as the confusing nomenclature).No extra injection of empty strings (so that a template string is astrict alternation of strings and expressions). A `JoinedTemplateString`simply has a list of template string children, and a `TemplateString`has a list of "values" which may be either `Expr` or`TemplateStringPart` nodes.If we ever find that we actually want the more complicated interface forthese strings, then I would much rather we reconstruct this inside of QLrather than in the parser.
We do the usual thing. Downgrade scripts remove the relevant relations;upgrade scripts do nothing.
Not actually based on any measurements, just the usual 100/1000 stuff.
@tausbntausbnforce-pushed thetausbn/python-add-support-for-template-string-literals branch froma35fba1 to4d45b58CompareDecember 16, 2025 22:58
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@yoffyoffyoff left review comments

@miketspraguemiketspraguemiketsprague left review comments

Copilot code reviewCopilotCopilot left review comments

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@tausbn@yoff@miketsprague

[8]ページ先頭

©2009-2025 Movatter.jp