* main: (24 commits)pythongh-98040: Move the Single-Phase Init Tests Out of test_imp (pythongh-102561)pythongh-83861: Fix datetime.astimezone() method (pythonGH-101545)pythongh-102856: Clean some of the PEP 701 tokenizer implementation (python#103634)pythongh-102856: Skip test_mismatched_parens in WASI builds (python#103633)pythongh-102856: Initial implementation of PEP 701 (python#102855)pythongh-103583: Add ref. dependency between multibytecodec modules (python#103589)pythongh-83004: Harden msvcrt further (python#103420)pythonGH-88342: clarify that `asyncio.as_completed` accepts generators yielding tasks (python#103626)pythongh-102778: IDLE - make sys.last_exc available in Shell after traceback (python#103314)pythongh-103582: Remove last references to `argparse.REMAINDER` from docs (python#103586)pythongh-103583: Always pass multibyte codec structs as const (python#103588)pythongh-103617: Fix compiler warning in _iomodule.c (python#103618)pythongh-103596: [Enum] do not shadow mixed-in methods/attributes (pythonGH-103600)pythonGH-100530: Change the error message for non-class class patterns (pythonGH-103576)pythongh-95299: Remove lingering setuptools reference in installer scripts (pythonGH-103613)  [Doc] Fix a typo in optparse.rst (python#103504)pythongh-101100: Fix broken reference `__format__` in `string.rst` (python#103531)pythongh-95299: Stop installing setuptools as a part of ensurepip and venv (python#101039)pythonGH-103484: Docs: add linkcheck allowed redirects entries for most cases (python#103569)pythongh-67230: update whatsnew note for csv changes (python#103598)  ...

## SummaryThis PR adds support for PEP 701 in the parser to use the new tokensemitted by the lexer to construct the f-string node.### GrammarWithout an official grammar, the f-strings were parsed manually. Nowthat we've the specification, that is being used in the LALRPOP to parsethe f-strings.### `string.rs`This file includes the logic for parsing string literals and joining theimplicit string concatenation. Now that we don't require parsingf-strings manually a lot of code involving the same is removed.Earlier, there were 2 entry points to this module:* `parse_string`: Used to parse a single string literal* `parse_strings`: Used to parse strings which were implicitlyconcatenatedNow, there are 3 entry points:* `parse_string_literal`: Renamed from `parse_string`* `parse_fstring_middle`: Used to parse a `FStringMiddle` token which isbasically a string literal without the quotes* `concatenate_strings`: Renamed from `parse_strings` but now it takesthe parsed nodes instead. So, we just need to concatenate them into asingle node.> A short primer on `FStringMiddle` token: This includes the portion oftext inside the f-string that's not part of the expression and isn't anopening or closing brace. For example, in `f"foo {bar:.3f{x}} bar"`, the`foo `, `.3f` and ` bar` are `FStringMiddle` token content.### `Constant::kind` changed in the AST***Discussion in the official implementation:python/cpython#102855 (comment)This change in the AST is when unicode strings (prefixed with `u`) andf-strings are used in an implicitly concatenated string value. Forexample,```pythonu"foo" f"{bar}" "baz" " some"```Pre Python 3.12, the kind field would be assigned only if the prefix wason the first string. So, taking the above example, both `"foo"` and`"baz some"` (implicit concatenation) would be given the `u` kind:<details><summary>Pre 3.12 AST:</summary><p>```pythonConstant(value='foo', kind='u'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='baz some', kind='u')```</p></details> But, post Python 3.12, only the string with the `u` prefix will beassigned the value:<details><summary>Pre 3.12 AST:</summary><p>```pythonConstant(value='foo', kind='u'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='baz some')```</p></details>Here are some more iterations around the change:1. `"foo" f"{bar}" u"baz" "no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno', kind='u')```</p></details> 2. `"foo" f"{bar}" "baz" u"no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details>3. `u"foo" f"bar {baz} realy" u"bar" "no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foobar ', kind='u'),FormattedValue(  value=Name(id='baz', ctx=Load()),  conversion=-1),Constant(value=' realybarno', kind='u')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foobar ', kind='u'),FormattedValue(  value=Name(id='baz', ctx=Load()),  conversion=-1),Constant(value=' realybarno')```</p></details> ### ErrorsWith the hand written parser, we were able to provide better errormessages in case of any errors such as the following but now they allare removed and in those cases an "unexpected token" error will bethrown by lalrpop:* A closing delimiter was not opened properly* An opening delimiter was not closed properly* Empty expression not allowedThe "Too many nested expressions in an f-string" was removed and insteadwe can create a lint rule for that.And, "The f-string expression cannot include the given character" wasremoved because f-strings now support those characters which are mainlysame quotes as the outer ones, escape sequences, comments, etc.## Test Plan1. Refactor existing test cases to use `parse_suite` instead of`parse_fstrings` (doesn't exists anymore)2. Additional test cases are added as requiredUpdated the snapshots. The change from `parse_fstrings` to `parse_suite`means that the snapshot would produce the module node instead of just alist of f-string parts. I've manually verified that the parts are stillthe same along with the node ranges.## Benchmarks#7263 (comment)fixes:#7043fixes:#6835

## SummaryThis PR adds support for PEP 701 in the parser to use the new tokensemitted by the lexer to construct the f-string node.### GrammarWithout an official grammar, the f-strings were parsed manually. Nowthat we've the specification, that is being used in the LALRPOP to parsethe f-strings.### `string.rs`This file includes the logic for parsing string literals and joining theimplicit string concatenation. Now that we don't require parsingf-strings manually a lot of code involving the same is removed.Earlier, there were 2 entry points to this module:* `parse_string`: Used to parse a single string literal* `parse_strings`: Used to parse strings which were implicitlyconcatenatedNow, there are 3 entry points:* `parse_string_literal`: Renamed from `parse_string`* `parse_fstring_middle`: Used to parse a `FStringMiddle` token which isbasically a string literal without the quotes* `concatenate_strings`: Renamed from `parse_strings` but now it takesthe parsed nodes instead. So, we just need to concatenate them into asingle node.> A short primer on `FStringMiddle` token: This includes the portion oftext inside the f-string that's not part of the expression and isn't anopening or closing brace. For example, in `f"foo {bar:.3f{x}} bar"`, the`foo `, `.3f` and ` bar` are `FStringMiddle` token content.### `Constant::kind` changed in the AST***Discussion in the official implementation:python/cpython#102855 (comment)This change in the AST is when unicode strings (prefixed with `u`) andf-strings are used in an implicitly concatenated string value. Forexample,```pythonu"foo" f"{bar}" "baz" " some"```Pre Python 3.12, the kind field would be assigned only if the prefix wason the first string. So, taking the above example, both `"foo"` and`"baz some"` (implicit concatenation) would be given the `u` kind:<details><summary>Pre 3.12 AST:</summary><p>```pythonConstant(value='foo', kind='u'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='baz some', kind='u')```</p></details> But, post Python 3.12, only the string with the `u` prefix will beassigned the value:<details><summary>Pre 3.12 AST:</summary><p>```pythonConstant(value='foo', kind='u'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='baz some')```</p></details>Here are some more iterations around the change:1. `"foo" f"{bar}" u"baz" "no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno', kind='u')```</p></details> 2. `"foo" f"{bar}" "baz" u"no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details>3. `u"foo" f"bar {baz} realy" u"bar" "no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foobar ', kind='u'),FormattedValue(  value=Name(id='baz', ctx=Load()),  conversion=-1),Constant(value=' realybarno', kind='u')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foobar ', kind='u'),FormattedValue(  value=Name(id='baz', ctx=Load()),  conversion=-1),Constant(value=' realybarno')```</p></details> ### ErrorsWith the hand written parser, we were able to provide better errormessages in case of any errors such as the following but now they allare removed and in those cases an "unexpected token" error will bethrown by lalrpop:* A closing delimiter was not opened properly* An opening delimiter was not closed properly* Empty expression not allowedThe "Too many nested expressions in an f-string" was removed and insteadwe can create a lint rule for that.And, "The f-string expression cannot include the given character" wasremoved because f-strings now support those characters which are mainlysame quotes as the outer ones, escape sequences, comments, etc.## Test Plan1. Refactor existing test cases to use `parse_suite` instead of`parse_fstrings` (doesn't exists anymore)2. Additional test cases are added as requiredUpdated the snapshots. The change from `parse_fstrings` to `parse_suite`means that the snapshot would produce the module node instead of just alist of f-string parts. I've manually verified that the parts are stillthe same along with the node ranges.## Benchmarks#7263 (comment)fixes:#7043fixes:#6835

## SummaryThis PR adds support for PEP 701 in the parser to use the new tokensemitted by the lexer to construct the f-string node.### GrammarWithout an official grammar, the f-strings were parsed manually. Nowthat we've the specification, that is being used in the LALRPOP to parsethe f-strings.### `string.rs`This file includes the logic for parsing string literals and joining theimplicit string concatenation. Now that we don't require parsingf-strings manually a lot of code involving the same is removed.Earlier, there were 2 entry points to this module:* `parse_string`: Used to parse a single string literal* `parse_strings`: Used to parse strings which were implicitlyconcatenatedNow, there are 3 entry points:* `parse_string_literal`: Renamed from `parse_string`* `parse_fstring_middle`: Used to parse a `FStringMiddle` token which isbasically a string literal without the quotes* `concatenate_strings`: Renamed from `parse_strings` but now it takesthe parsed nodes instead. So, we just need to concatenate them into asingle node.> A short primer on `FStringMiddle` token: This includes the portion oftext inside the f-string that's not part of the expression and isn't anopening or closing brace. For example, in `f"foo {bar:.3f{x}} bar"`, the`foo `, `.3f` and ` bar` are `FStringMiddle` token content.### `Constant::kind` changed in the AST***Discussion in the official implementation:python/cpython#102855 (comment)This change in the AST is when unicode strings (prefixed with `u`) andf-strings are used in an implicitly concatenated string value. Forexample,```pythonu"foo" f"{bar}" "baz" " some"```Pre Python 3.12, the kind field would be assigned only if the prefix wason the first string. So, taking the above example, both `"foo"` and`"baz some"` (implicit concatenation) would be given the `u` kind:<details><summary>Pre 3.12 AST:</summary><p>```pythonConstant(value='foo', kind='u'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='baz some', kind='u')```</p></details> But, post Python 3.12, only the string with the `u` prefix will beassigned the value:<details><summary>Pre 3.12 AST:</summary><p>```pythonConstant(value='foo', kind='u'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='baz some')```</p></details>Here are some more iterations around the change:1. `"foo" f"{bar}" u"baz" "no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno', kind='u')```</p></details> 2. `"foo" f"{bar}" "baz" u"no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details>3. `u"foo" f"bar {baz} realy" u"bar" "no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foobar ', kind='u'),FormattedValue(  value=Name(id='baz', ctx=Load()),  conversion=-1),Constant(value=' realybarno', kind='u')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foobar ', kind='u'),FormattedValue(  value=Name(id='baz', ctx=Load()),  conversion=-1),Constant(value=' realybarno')```</p></details> ### ErrorsWith the hand written parser, we were able to provide better errormessages in case of any errors such as the following but now they allare removed and in those cases an "unexpected token" error will bethrown by lalrpop:* A closing delimiter was not opened properly* An opening delimiter was not closed properly* Empty expression not allowedThe "Too many nested expressions in an f-string" was removed and insteadwe can create a lint rule for that.And, "The f-string expression cannot include the given character" wasremoved because f-strings now support those characters which are mainlysame quotes as the outer ones, escape sequences, comments, etc.## Test Plan1. Refactor existing test cases to use `parse_suite` instead of`parse_fstrings` (doesn't exists anymore)2. Additional test cases are added as requiredUpdated the snapshots. The change from `parse_fstrings` to `parse_suite`means that the snapshot would produce the module node instead of just alist of f-string parts. I've manually verified that the parts are stillthe same along with the node ranges.## Benchmarks#7263 (comment)fixes:#7043fixes:#6835

## SummaryThis PR adds support for PEP 701 in the parser to use the new tokensemitted by the lexer to construct the f-string node.### GrammarWithout an official grammar, the f-strings were parsed manually. Nowthat we've the specification, that is being used in the LALRPOP to parsethe f-strings.### `string.rs`This file includes the logic for parsing string literals and joining theimplicit string concatenation. Now that we don't require parsingf-strings manually a lot of code involving the same is removed.Earlier, there were 2 entry points to this module:* `parse_string`: Used to parse a single string literal* `parse_strings`: Used to parse strings which were implicitlyconcatenatedNow, there are 3 entry points:* `parse_string_literal`: Renamed from `parse_string`* `parse_fstring_middle`: Used to parse a `FStringMiddle` token which isbasically a string literal without the quotes* `concatenate_strings`: Renamed from `parse_strings` but now it takesthe parsed nodes instead. So, we just need to concatenate them into asingle node.> A short primer on `FStringMiddle` token: This includes the portion oftext inside the f-string that's not part of the expression and isn't anopening or closing brace. For example, in `f"foo {bar:.3f{x}} bar"`, the`foo `, `.3f` and ` bar` are `FStringMiddle` token content.### `Constant::kind` changed in the AST***Discussion in the official implementation:python/cpython#102855 (comment)This change in the AST is when unicode strings (prefixed with `u`) andf-strings are used in an implicitly concatenated string value. Forexample,```pythonu"foo" f"{bar}" "baz" " some"```Pre Python 3.12, the kind field would be assigned only if the prefix wason the first string. So, taking the above example, both `"foo"` and`"baz some"` (implicit concatenation) would be given the `u` kind:<details><summary>Pre 3.12 AST:</summary><p>```pythonConstant(value='foo', kind='u'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='baz some', kind='u')```</p></details> But, post Python 3.12, only the string with the `u` prefix will beassigned the value:<details><summary>Pre 3.12 AST:</summary><p>```pythonConstant(value='foo', kind='u'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='baz some')```</p></details>Here are some more iterations around the change:1. `"foo" f"{bar}" u"baz" "no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno', kind='u')```</p></details> 2. `"foo" f"{bar}" "baz" u"no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details>3. `u"foo" f"bar {baz} realy" u"bar" "no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foobar ', kind='u'),FormattedValue(  value=Name(id='baz', ctx=Load()),  conversion=-1),Constant(value=' realybarno', kind='u')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foobar ', kind='u'),FormattedValue(  value=Name(id='baz', ctx=Load()),  conversion=-1),Constant(value=' realybarno')```</p></details> ### ErrorsWith the hand written parser, we were able to provide better errormessages in case of any errors such as the following but now they allare removed and in those cases an "unexpected token" error will bethrown by lalrpop:* A closing delimiter was not opened properly* An opening delimiter was not closed properly* Empty expression not allowedThe "Too many nested expressions in an f-string" was removed and insteadwe can create a lint rule for that.And, "The f-string expression cannot include the given character" wasremoved because f-strings now support those characters which are mainlysame quotes as the outer ones, escape sequences, comments, etc.## Test Plan1. Refactor existing test cases to use `parse_suite` instead of`parse_fstrings` (doesn't exists anymore)2. Additional test cases are added as requiredUpdated the snapshots. The change from `parse_fstrings` to `parse_suite`means that the snapshot would produce the module node instead of just alist of f-string parts. I've manually verified that the parts are stillthe same along with the node ranges.## Benchmarks#7263 (comment)fixes:#7043fixes:#6835

This PR adds support for PEP 701 in the parser to use the new tokensemitted by the lexer to construct the f-string node.Without an official grammar, the f-strings were parsed manually. Nowthat we've the specification, that is being used in the LALRPOP to parsethe f-strings.This file includes the logic for parsing string literals and joining theimplicit string concatenation. Now that we don't require parsingf-strings manually a lot of code involving the same is removed.Earlier, there were 2 entry points to this module:* `parse_string`: Used to parse a single string literal* `parse_strings`: Used to parse strings which were implicitlyconcatenatedNow, there are 3 entry points:* `parse_string_literal`: Renamed from `parse_string`* `parse_fstring_middle`: Used to parse a `FStringMiddle` token which isbasically a string literal without the quotes* `concatenate_strings`: Renamed from `parse_strings` but now it takesthe parsed nodes instead. So, we just need to concatenate them into asingle node.> A short primer on `FStringMiddle` token: This includes the portion oftext inside the f-string that's not part of the expression and isn't anopening or closing brace. For example, in `f"foo {bar:.3f{x}} bar"`, the`foo `, `.3f` and ` bar` are `FStringMiddle` token content.***Discussion in the official implementation:python/cpython#102855 (comment)This change in the AST is when unicode strings (prefixed with `u`) andf-strings are used in an implicitly concatenated string value. Forexample,```pythonu"foo" f"{bar}" "baz" " some"```Pre Python 3.12, the kind field would be assigned only if the prefix wason the first string. So, taking the above example, both `"foo"` and`"baz some"` (implicit concatenation) would be given the `u` kind:<details><summary>Pre 3.12 AST:</summary><p>```pythonConstant(value='foo', kind='u'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='baz some', kind='u')```</p></details>But, post Python 3.12, only the string with the `u` prefix will beassigned the value:<details><summary>Pre 3.12 AST:</summary><p>```pythonConstant(value='foo', kind='u'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='baz some')```</p></details>Here are some more iterations around the change:1. `"foo" f"{bar}" u"baz" "no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno', kind='u')```</p></details>2. `"foo" f"{bar}" "baz" u"no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details>3. `u"foo" f"bar {baz} realy" u"bar" "no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foobar ', kind='u'),FormattedValue(  value=Name(id='baz', ctx=Load()),  conversion=-1),Constant(value=' realybarno', kind='u')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foobar ', kind='u'),FormattedValue(  value=Name(id='baz', ctx=Load()),  conversion=-1),Constant(value=' realybarno')```</p></details>With the hand written parser, we were able to provide better errormessages in case of any errors such as the following but now they allare removed and in those cases an "unexpected token" error will bethrown by lalrpop:* A closing delimiter was not opened properly* An opening delimiter was not closed properly* Empty expression not allowedThe "Too many nested expressions in an f-string" was removed and insteadwe can create a lint rule for that.And, "The f-string expression cannot include the given character" wasremoved because f-strings now support those characters which are mainlysame quotes as the outer ones, escape sequences, comments, etc.1. Refactor existing test cases to use `parse_suite` instead of`parse_fstrings` (doesn't exists anymore)2. Additional test cases are added as requiredUpdated the snapshots. The change from `parse_fstrings` to `parse_suite`means that the snapshot would produce the module node instead of just alist of f-string parts. I've manually verified that the parts are stillthe same along with the node ranges.#7263 (comment)fixes:#7043fixes:#6835

This PR adds support for PEP 701 in the parser to use the new tokensemitted by the lexer to construct the f-string node.Without an official grammar, the f-strings were parsed manually. Nowthat we've the specification, that is being used in the LALRPOP to parsethe f-strings.This file includes the logic for parsing string literals and joining theimplicit string concatenation. Now that we don't require parsingf-strings manually a lot of code involving the same is removed.Earlier, there were 2 entry points to this module:* `parse_string`: Used to parse a single string literal* `parse_strings`: Used to parse strings which were implicitlyconcatenatedNow, there are 3 entry points:* `parse_string_literal`: Renamed from `parse_string`* `parse_fstring_middle`: Used to parse a `FStringMiddle` token which isbasically a string literal without the quotes* `concatenate_strings`: Renamed from `parse_strings` but now it takesthe parsed nodes instead. So, we just need to concatenate them into asingle node.> A short primer on `FStringMiddle` token: This includes the portion oftext inside the f-string that's not part of the expression and isn't anopening or closing brace. For example, in `f"foo {bar:.3f{x}} bar"`, the`foo `, `.3f` and ` bar` are `FStringMiddle` token content.***Discussion in the official implementation:python/cpython#102855 (comment)This change in the AST is when unicode strings (prefixed with `u`) andf-strings are used in an implicitly concatenated string value. Forexample,```pythonu"foo" f"{bar}" "baz" " some"```Pre Python 3.12, the kind field would be assigned only if the prefix wason the first string. So, taking the above example, both `"foo"` and`"baz some"` (implicit concatenation) would be given the `u` kind:<details><summary>Pre 3.12 AST:</summary><p>```pythonConstant(value='foo', kind='u'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='baz some', kind='u')```</p></details>But, post Python 3.12, only the string with the `u` prefix will beassigned the value:<details><summary>Pre 3.12 AST:</summary><p>```pythonConstant(value='foo', kind='u'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='baz some')```</p></details>Here are some more iterations around the change:1. `"foo" f"{bar}" u"baz" "no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno', kind='u')```</p></details>2. `"foo" f"{bar}" "baz" u"no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details>3. `u"foo" f"bar {baz} realy" u"bar" "no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foobar ', kind='u'),FormattedValue(  value=Name(id='baz', ctx=Load()),  conversion=-1),Constant(value=' realybarno', kind='u')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foobar ', kind='u'),FormattedValue(  value=Name(id='baz', ctx=Load()),  conversion=-1),Constant(value=' realybarno')```</p></details>With the hand written parser, we were able to provide better errormessages in case of any errors such as the following but now they allare removed and in those cases an "unexpected token" error will bethrown by lalrpop:* A closing delimiter was not opened properly* An opening delimiter was not closed properly* Empty expression not allowedThe "Too many nested expressions in an f-string" was removed and insteadwe can create a lint rule for that.And, "The f-string expression cannot include the given character" wasremoved because f-strings now support those characters which are mainlysame quotes as the outer ones, escape sequences, comments, etc.1. Refactor existing test cases to use `parse_suite` instead of`parse_fstrings` (doesn't exists anymore)2. Additional test cases are added as requiredUpdated the snapshots. The change from `parse_fstrings` to `parse_suite`means that the snapshot would produce the module node instead of just alist of f-string parts. I've manually verified that the parts are stillthe same along with the node ranges.#7263 (comment)fixes:#7043fixes:#6835

This PR adds support for PEP 701 in the parser to use the new tokensemitted by the lexer to construct the f-string node.Without an official grammar, the f-strings were parsed manually. Nowthat we've the specification, that is being used in the LALRPOP to parsethe f-strings.This file includes the logic for parsing string literals and joining theimplicit string concatenation. Now that we don't require parsingf-strings manually a lot of code involving the same is removed.Earlier, there were 2 entry points to this module:* `parse_string`: Used to parse a single string literal* `parse_strings`: Used to parse strings which were implicitlyconcatenatedNow, there are 3 entry points:* `parse_string_literal`: Renamed from `parse_string`* `parse_fstring_middle`: Used to parse a `FStringMiddle` token which isbasically a string literal without the quotes* `concatenate_strings`: Renamed from `parse_strings` but now it takesthe parsed nodes instead. So, we just need to concatenate them into asingle node.> A short primer on `FStringMiddle` token: This includes the portion oftext inside the f-string that's not part of the expression and isn't anopening or closing brace. For example, in `f"foo {bar:.3f{x}} bar"`, the`foo `, `.3f` and ` bar` are `FStringMiddle` token content.***Discussion in the official implementation:python/cpython#102855 (comment)This change in the AST is when unicode strings (prefixed with `u`) andf-strings are used in an implicitly concatenated string value. Forexample,```pythonu"foo" f"{bar}" "baz" " some"```Pre Python 3.12, the kind field would be assigned only if the prefix wason the first string. So, taking the above example, both `"foo"` and`"baz some"` (implicit concatenation) would be given the `u` kind:<details><summary>Pre 3.12 AST:</summary><p>```pythonConstant(value='foo', kind='u'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='baz some', kind='u')```</p></details>But, post Python 3.12, only the string with the `u` prefix will beassigned the value:<details><summary>Pre 3.12 AST:</summary><p>```pythonConstant(value='foo', kind='u'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='baz some')```</p></details>Here are some more iterations around the change:1. `"foo" f"{bar}" u"baz" "no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno', kind='u')```</p></details>2. `"foo" f"{bar}" "baz" u"no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details>3. `u"foo" f"bar {baz} realy" u"bar" "no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foobar ', kind='u'),FormattedValue(  value=Name(id='baz', ctx=Load()),  conversion=-1),Constant(value=' realybarno', kind='u')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foobar ', kind='u'),FormattedValue(  value=Name(id='baz', ctx=Load()),  conversion=-1),Constant(value=' realybarno')```</p></details>With the hand written parser, we were able to provide better errormessages in case of any errors such as the following but now they allare removed and in those cases an "unexpected token" error will bethrown by lalrpop:* A closing delimiter was not opened properly* An opening delimiter was not closed properly* Empty expression not allowedThe "Too many nested expressions in an f-string" was removed and insteadwe can create a lint rule for that.And, "The f-string expression cannot include the given character" wasremoved because f-strings now support those characters which are mainlysame quotes as the outer ones, escape sequences, comments, etc.1. Refactor existing test cases to use `parse_suite` instead of`parse_fstrings` (doesn't exists anymore)2. Additional test cases are added as requiredUpdated the snapshots. The change from `parse_fstrings` to `parse_suite`means that the snapshot would produce the module node instead of just alist of f-string parts. I've manually verified that the parts are stillthe same along with the node ranges.#7263 (comment)fixes:#7043fixes:#6835

This PR adds support for PEP 701 in the parser to use the new tokensemitted by the lexer to construct the f-string node.Without an official grammar, the f-strings were parsed manually. Nowthat we've the specification, that is being used in the LALRPOP to parsethe f-strings.This file includes the logic for parsing string literals and joining theimplicit string concatenation. Now that we don't require parsingf-strings manually a lot of code involving the same is removed.Earlier, there were 2 entry points to this module:* `parse_string`: Used to parse a single string literal* `parse_strings`: Used to parse strings which were implicitlyconcatenatedNow, there are 3 entry points:* `parse_string_literal`: Renamed from `parse_string`* `parse_fstring_middle`: Used to parse a `FStringMiddle` token which isbasically a string literal without the quotes* `concatenate_strings`: Renamed from `parse_strings` but now it takesthe parsed nodes instead. So, we just need to concatenate them into asingle node.> A short primer on `FStringMiddle` token: This includes the portion oftext inside the f-string that's not part of the expression and isn't anopening or closing brace. For example, in `f"foo {bar:.3f{x}} bar"`, the`foo `, `.3f` and ` bar` are `FStringMiddle` token content.***Discussion in the official implementation:python/cpython#102855 (comment)This change in the AST is when unicode strings (prefixed with `u`) andf-strings are used in an implicitly concatenated string value. Forexample,```pythonu"foo" f"{bar}" "baz" " some"```Pre Python 3.12, the kind field would be assigned only if the prefix wason the first string. So, taking the above example, both `"foo"` and`"baz some"` (implicit concatenation) would be given the `u` kind:<details><summary>Pre 3.12 AST:</summary><p>```pythonConstant(value='foo', kind='u'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='baz some', kind='u')```</p></details>But, post Python 3.12, only the string with the `u` prefix will beassigned the value:<details><summary>Pre 3.12 AST:</summary><p>```pythonConstant(value='foo', kind='u'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='baz some')```</p></details>Here are some more iterations around the change:1. `"foo" f"{bar}" u"baz" "no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno', kind='u')```</p></details>2. `"foo" f"{bar}" "baz" u"no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details>3. `u"foo" f"bar {baz} realy" u"bar" "no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foobar ', kind='u'),FormattedValue(  value=Name(id='baz', ctx=Load()),  conversion=-1),Constant(value=' realybarno', kind='u')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foobar ', kind='u'),FormattedValue(  value=Name(id='baz', ctx=Load()),  conversion=-1),Constant(value=' realybarno')```</p></details>With the hand written parser, we were able to provide better errormessages in case of any errors such as the following but now they allare removed and in those cases an "unexpected token" error will bethrown by lalrpop:* A closing delimiter was not opened properly* An opening delimiter was not closed properly* Empty expression not allowedThe "Too many nested expressions in an f-string" was removed and insteadwe can create a lint rule for that.And, "The f-string expression cannot include the given character" wasremoved because f-strings now support those characters which are mainlysame quotes as the outer ones, escape sequences, comments, etc.1. Refactor existing test cases to use `parse_suite` instead of`parse_fstrings` (doesn't exists anymore)2. Additional test cases are added as requiredUpdated the snapshots. The change from `parse_fstrings` to `parse_suite`means that the snapshot would produce the module node instead of just alist of f-string parts. I've manually verified that the parts are stillthe same along with the node ranges.#7263 (comment)fixes:#7043fixes:#6835

This PR adds support for PEP 701 in the parser to use the new tokensemitted by the lexer to construct the f-string node.Without an official grammar, the f-strings were parsed manually. Nowthat we've the specification, that is being used in the LALRPOP to parsethe f-strings.This file includes the logic for parsing string literals and joining theimplicit string concatenation. Now that we don't require parsingf-strings manually a lot of code involving the same is removed.Earlier, there were 2 entry points to this module:* `parse_string`: Used to parse a single string literal* `parse_strings`: Used to parse strings which were implicitlyconcatenatedNow, there are 3 entry points:* `parse_string_literal`: Renamed from `parse_string`* `parse_fstring_middle`: Used to parse a `FStringMiddle` token which isbasically a string literal without the quotes* `concatenate_strings`: Renamed from `parse_strings` but now it takesthe parsed nodes instead. So, we just need to concatenate them into asingle node.> A short primer on `FStringMiddle` token: This includes the portion oftext inside the f-string that's not part of the expression and isn't anopening or closing brace. For example, in `f"foo {bar:.3f{x}} bar"`, the`foo `, `.3f` and ` bar` are `FStringMiddle` token content.***Discussion in the official implementation:python/cpython#102855 (comment)This change in the AST is when unicode strings (prefixed with `u`) andf-strings are used in an implicitly concatenated string value. Forexample,```pythonu"foo" f"{bar}" "baz" " some"```Pre Python 3.12, the kind field would be assigned only if the prefix wason the first string. So, taking the above example, both `"foo"` and`"baz some"` (implicit concatenation) would be given the `u` kind:<details><summary>Pre 3.12 AST:</summary><p>```pythonConstant(value='foo', kind='u'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='baz some', kind='u')```</p></details>But, post Python 3.12, only the string with the `u` prefix will beassigned the value:<details><summary>Pre 3.12 AST:</summary><p>```pythonConstant(value='foo', kind='u'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='baz some')```</p></details>Here are some more iterations around the change:1. `"foo" f"{bar}" u"baz" "no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno', kind='u')```</p></details>2. `"foo" f"{bar}" "baz" u"no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details>3. `u"foo" f"bar {baz} realy" u"bar" "no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foobar ', kind='u'),FormattedValue(  value=Name(id='baz', ctx=Load()),  conversion=-1),Constant(value=' realybarno', kind='u')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foobar ', kind='u'),FormattedValue(  value=Name(id='baz', ctx=Load()),  conversion=-1),Constant(value=' realybarno')```</p></details>With the hand written parser, we were able to provide better errormessages in case of any errors such as the following but now they allare removed and in those cases an "unexpected token" error will bethrown by lalrpop:* A closing delimiter was not opened properly* An opening delimiter was not closed properly* Empty expression not allowedThe "Too many nested expressions in an f-string" was removed and insteadwe can create a lint rule for that.And, "The f-string expression cannot include the given character" wasremoved because f-strings now support those characters which are mainlysame quotes as the outer ones, escape sequences, comments, etc.1. Refactor existing test cases to use `parse_suite` instead of`parse_fstrings` (doesn't exists anymore)2. Additional test cases are added as requiredUpdated the snapshots. The change from `parse_fstrings` to `parse_suite`means that the snapshot would produce the module node instead of just alist of f-string parts. I've manually verified that the parts are stillthe same along with the node ranges.#7263 (comment)fixes:#7043fixes:#6835

This PR adds support for PEP 701 in the parser to use the new tokensemitted by the lexer to construct the f-string node.Without an official grammar, the f-strings were parsed manually. Nowthat we've the specification, that is being used in the LALRPOP to parsethe f-strings.This file includes the logic for parsing string literals and joining theimplicit string concatenation. Now that we don't require parsingf-strings manually a lot of code involving the same is removed.Earlier, there were 2 entry points to this module:* `parse_string`: Used to parse a single string literal* `parse_strings`: Used to parse strings which were implicitlyconcatenatedNow, there are 3 entry points:* `parse_string_literal`: Renamed from `parse_string`* `parse_fstring_middle`: Used to parse a `FStringMiddle` token which isbasically a string literal without the quotes* `concatenate_strings`: Renamed from `parse_strings` but now it takesthe parsed nodes instead. So, we just need to concatenate them into asingle node.> A short primer on `FStringMiddle` token: This includes the portion oftext inside the f-string that's not part of the expression and isn't anopening or closing brace. For example, in `f"foo {bar:.3f{x}} bar"`, the`foo `, `.3f` and ` bar` are `FStringMiddle` token content.***Discussion in the official implementation:python/cpython#102855 (comment)This change in the AST is when unicode strings (prefixed with `u`) andf-strings are used in an implicitly concatenated string value. Forexample,```pythonu"foo" f"{bar}" "baz" " some"```Pre Python 3.12, the kind field would be assigned only if the prefix wason the first string. So, taking the above example, both `"foo"` and`"baz some"` (implicit concatenation) would be given the `u` kind:<details><summary>Pre 3.12 AST:</summary><p>```pythonConstant(value='foo', kind='u'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='baz some', kind='u')```</p></details>But, post Python 3.12, only the string with the `u` prefix will beassigned the value:<details><summary>Pre 3.12 AST:</summary><p>```pythonConstant(value='foo', kind='u'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='baz some')```</p></details>Here are some more iterations around the change:1. `"foo" f"{bar}" u"baz" "no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno', kind='u')```</p></details>2. `"foo" f"{bar}" "baz" u"no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details>3. `u"foo" f"bar {baz} realy" u"bar" "no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foobar ', kind='u'),FormattedValue(  value=Name(id='baz', ctx=Load()),  conversion=-1),Constant(value=' realybarno', kind='u')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foobar ', kind='u'),FormattedValue(  value=Name(id='baz', ctx=Load()),  conversion=-1),Constant(value=' realybarno')```</p></details>With the hand written parser, we were able to provide better errormessages in case of any errors such as the following but now they allare removed and in those cases an "unexpected token" error will bethrown by lalrpop:* A closing delimiter was not opened properly* An opening delimiter was not closed properly* Empty expression not allowedThe "Too many nested expressions in an f-string" was removed and insteadwe can create a lint rule for that.And, "The f-string expression cannot include the given character" wasremoved because f-strings now support those characters which are mainlysame quotes as the outer ones, escape sequences, comments, etc.1. Refactor existing test cases to use `parse_suite` instead of`parse_fstrings` (doesn't exists anymore)2. Additional test cases are added as requiredUpdated the snapshots. The change from `parse_fstrings` to `parse_suite`means that the snapshot would produce the module node instead of just alist of f-string parts. I've manually verified that the parts are stillthe same along with the node ranges.#7263 (comment)fixes:#7043fixes:#6835

This PR adds support for PEP 701 in the parser to use the new tokensemitted by the lexer to construct the f-string node.Without an official grammar, the f-strings were parsed manually. Nowthat we've the specification, that is being used in the LALRPOP to parsethe f-strings.This file includes the logic for parsing string literals and joining theimplicit string concatenation. Now that we don't require parsingf-strings manually a lot of code involving the same is removed.Earlier, there were 2 entry points to this module:* `parse_string`: Used to parse a single string literal* `parse_strings`: Used to parse strings which were implicitlyconcatenatedNow, there are 3 entry points:* `parse_string_literal`: Renamed from `parse_string`* `parse_fstring_middle`: Used to parse a `FStringMiddle` token which isbasically a string literal without the quotes* `concatenate_strings`: Renamed from `parse_strings` but now it takesthe parsed nodes instead. So, we just need to concatenate them into asingle node.> A short primer on `FStringMiddle` token: This includes the portion oftext inside the f-string that's not part of the expression and isn't anopening or closing brace. For example, in `f"foo {bar:.3f{x}} bar"`, the`foo `, `.3f` and ` bar` are `FStringMiddle` token content.***Discussion in the official implementation:python/cpython#102855 (comment)This change in the AST is when unicode strings (prefixed with `u`) andf-strings are used in an implicitly concatenated string value. Forexample,```pythonu"foo" f"{bar}" "baz" " some"```Pre Python 3.12, the kind field would be assigned only if the prefix wason the first string. So, taking the above example, both `"foo"` and`"baz some"` (implicit concatenation) would be given the `u` kind:<details><summary>Pre 3.12 AST:</summary><p>```pythonConstant(value='foo', kind='u'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='baz some', kind='u')```</p></details>But, post Python 3.12, only the string with the `u` prefix will beassigned the value:<details><summary>Pre 3.12 AST:</summary><p>```pythonConstant(value='foo', kind='u'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='baz some')```</p></details>Here are some more iterations around the change:1. `"foo" f"{bar}" u"baz" "no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno', kind='u')```</p></details>2. `"foo" f"{bar}" "baz" u"no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details>3. `u"foo" f"bar {baz} realy" u"bar" "no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foobar ', kind='u'),FormattedValue(  value=Name(id='baz', ctx=Load()),  conversion=-1),Constant(value=' realybarno', kind='u')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foobar ', kind='u'),FormattedValue(  value=Name(id='baz', ctx=Load()),  conversion=-1),Constant(value=' realybarno')```</p></details>With the hand written parser, we were able to provide better errormessages in case of any errors such as the following but now they allare removed and in those cases an "unexpected token" error will bethrown by lalrpop:* A closing delimiter was not opened properly* An opening delimiter was not closed properly* Empty expression not allowedThe "Too many nested expressions in an f-string" was removed and insteadwe can create a lint rule for that.And, "The f-string expression cannot include the given character" wasremoved because f-strings now support those characters which are mainlysame quotes as the outer ones, escape sequences, comments, etc.1. Refactor existing test cases to use `parse_suite` instead of`parse_fstrings` (doesn't exists anymore)2. Additional test cases are added as requiredUpdated the snapshots. The change from `parse_fstrings` to `parse_suite`means that the snapshot would produce the module node instead of just alist of f-string parts. I've manually verified that the parts are stillthe same along with the node ranges.#7263 (comment)fixes:#7043fixes:#6835

This PR adds support for PEP 701 in the parser to use the new tokensemitted by the lexer to construct the f-string node.Without an official grammar, the f-strings were parsed manually. Nowthat we've the specification, that is being used in the LALRPOP to parsethe f-strings.This file includes the logic for parsing string literals and joining theimplicit string concatenation. Now that we don't require parsingf-strings manually a lot of code involving the same is removed.Earlier, there were 2 entry points to this module:* `parse_string`: Used to parse a single string literal* `parse_strings`: Used to parse strings which were implicitlyconcatenatedNow, there are 3 entry points:* `parse_string_literal`: Renamed from `parse_string`* `parse_fstring_middle`: Used to parse a `FStringMiddle` token which isbasically a string literal without the quotes* `concatenate_strings`: Renamed from `parse_strings` but now it takesthe parsed nodes instead. So, we just need to concatenate them into asingle node.> A short primer on `FStringMiddle` token: This includes the portion oftext inside the f-string that's not part of the expression and isn't anopening or closing brace. For example, in `f"foo {bar:.3f{x}} bar"`, the`foo `, `.3f` and ` bar` are `FStringMiddle` token content.***Discussion in the official implementation:python/cpython#102855 (comment)This change in the AST is when unicode strings (prefixed with `u`) andf-strings are used in an implicitly concatenated string value. Forexample,```pythonu"foo" f"{bar}" "baz" " some"```Pre Python 3.12, the kind field would be assigned only if the prefix wason the first string. So, taking the above example, both `"foo"` and`"baz some"` (implicit concatenation) would be given the `u` kind:<details><summary>Pre 3.12 AST:</summary><p>```pythonConstant(value='foo', kind='u'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='baz some', kind='u')```</p></details>But, post Python 3.12, only the string with the `u` prefix will beassigned the value:<details><summary>Pre 3.12 AST:</summary><p>```pythonConstant(value='foo', kind='u'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='baz some')```</p></details>Here are some more iterations around the change:1. `"foo" f"{bar}" u"baz" "no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno', kind='u')```</p></details>2. `"foo" f"{bar}" "baz" u"no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foo'),FormattedValue(  value=Name(id='bar', ctx=Load()),  conversion=-1),Constant(value='bazno')```</p></details>3. `u"foo" f"bar {baz} realy" u"bar" "no"`<details><summary>Pre 3.12</summary><p>```pythonConstant(value='foobar ', kind='u'),FormattedValue(  value=Name(id='baz', ctx=Load()),  conversion=-1),Constant(value=' realybarno', kind='u')```</p></details><details><summary>3.12</summary><p>```pythonConstant(value='foobar ', kind='u'),FormattedValue(  value=Name(id='baz', ctx=Load()),  conversion=-1),Constant(value=' realybarno')```</p></details>With the hand written parser, we were able to provide better errormessages in case of any errors such as the following but now they allare removed and in those cases an "unexpected token" error will bethrown by lalrpop:* A closing delimiter was not opened properly* An opening delimiter was not closed properly* Empty expression not allowedThe "Too many nested expressions in an f-string" was removed and insteadwe can create a lint rule for that.And, "The f-string expression cannot include the given character" wasremoved because f-strings now support those characters which are mainlysame quotes as the outer ones, escape sequences, comments, etc.1. Refactor existing test cases to use `parse_suite` instead of`parse_fstrings` (doesn't exists anymore)2. Additional test cases are added as requiredUpdated the snapshots. The change from `parse_fstrings` to `parse_suite`means that the snapshot would produce the module node instead of just alist of f-string parts. I've manually verified that the parts are stillthe same along with the node ranges.#7263 (comment)fixes:#7043fixes:#6835