Documentation

Thetoken docs use a generated list of tokens, with short blurbs for the OP tokens but nothing for the other (more interesting) ones.
Some tokens (COMMENT,NL, etc.) have secondary, hand-written definitions. However, these are excluded from the index;links go to the empty entries.

I propose adding hand-written prose for the non-OP tokens, and changing the automation tocheck the entries rather than generate them.
For the OP tokens, generating the docs is the correct thing to do, but the list can be more compact since they're all the same.

Linked PRs

• gh-130587: Add hand-written docs for non-OP tokens #130588
• [3.13] gh-130587: Add hand-written docs for non-OP tokens (GH-130588) #131465
• gh-130587: Invoke regen-token rst with rstfile as an argument #132304
• [3.13] gh-130587: Invoke regen-token rst with rstfile as an argument (GH-132304) #132327