Should we add a test for this?

Hmm, not sure what the test would look like. If it's verifying the list of token, we would have to update every time a new token is added, which is as easy to miss as just changing theifs like in this PR.

And I don't think we have the ability to run scripts with different python versions to verify they work, right?

Maybe add a reminder comment to e.g.Grammar/Tokens?

And I don't think we have the ability to run scripts with different python versions to verify they work, right?

We could add something to theCheck if generated files are up to date CI check perhaps? Though just having a comment may be simpler.

A

All commit authors signed the Contributor License Agreement.

After committing@AA-Turner suggestion directly from the GitHub UI, the CLA bot now require signing for the noreply mail used. Should I edit the commit manually to update the mail address, or is there a another way to do this?

If you click the 'CLA not signed' button, you should be able to sign it. If you have problems, can you paste a screenshot of what you see?

A

All good now, thanks! I was not sure if we were supposed to sign those noreply mail addresses as well (my primary gmail addresses was already signed).

@alexprengere Could you add a comment toGrammar/Tokens that the peg generator (ideally mentioning specific files and lines) needs to be udpated as well when adding new tokens?

Done.
Is there a fundamental reason whypython -m pegen python does not takeGrammar/Tokens as input, the same way thatpython -m pegen c does? This would also solve the issue of updatingparser_generator.py when new tokens are added.

Thanks@alexprengere for the PR, and@AA-Turner for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

GH-133630 is a backport of this pull request to the3.14 branch.