(added in 3.3.0b1)

• Implemented a TINY language parser/interpreter using pyparsing, in theexamples/tiny directory. This is a little tutorial language that I used to demonstrate how to use pyparsing to build a simple interpreter, following a recommended parser+AST+engine+run structure. The docs sub-directory also includes transcripts of the AI session used to create the parser and the interpreter.

• Fixed minor formatting bugs in pyparsing.testing.with_line_numbers, found during development of the TINY language example.

• Added test inDelimitedList andnested_expr which auto-suppress delimiting commas to avoid wrapping in aSuppress if delimiter is already aSuppress.

• Added performance benchmarking tools and documentation:

  • tests/perf_pyparsing.py runs a series of benchmark parsing tests, exercising different aspects of the pyparsing package. For cross-version analysis, this script can export results as CSV and append to a consolidated data file.
  • Runner scriptsrun_perf_all_tags.bat (Windows) andrun_perf_all_tags.sh (Ubuntu/bash) execute the benchmark across multiple Python versions (3.9–3.14) and pyparsing versions (3.1.1 through 3.3.0), aggregating results intoperf_pyparsing.csv at the repo root.
  • Seetests/README.md for usage instructions.

• Supports Python 3.14 and free-threaded 3.14t. (Note: this does not mean that pyparsing is thread-safe, simply that pyparsing can safely run using the free-threaded build of Python.)
• The version 3.3.0 release will begin emittingDeprecationWarnings for pyparsing methods that have been renamed to PEP8-compliant names (introduced in pyparsing 3.0.0, in August, 2021, with legacy names retained as aliases).
• AI instructions have been included for AI guidance when generating parsers using pyparsing. They can be accessed usingpython -m pyparsing.ai.show_best_practices.
• Thecvt_pyparsing_pep8_names utility (released in pyparsing 3.2.2) will raise UserWarnings when convertinglocatedExpr andindentedBlock calls, since the new methods have slightly different call signatures and return types.
• Source code examples have all been upgraded to use new PEP8-compliant names.

Barring any catastrophic bugs inthis release, this will be the last release in the 3.2.x line. The next release, 3.3.0, will begin emittingDeprecationWarnings when the pre-PEP8 methods are used (see header notes in CHANGES file for more information, including available automation for converting any existing code using pyparsing using the old names).

• JINX! Well, 3.2.4 had a bug forWord expressions that include a space character, if that expression was then copied, either directly with .copy() or by adding a results name, or including in another construct (likeDelimitedList) that makes a copy internally. Issue#618, reported by mstinberg, among others - thanks, and sorry for the inconvenience.

Full Changelog:3.2.4...3.2.5

Barring any catastrophic bugs in this release, this will be the last release in the 3.2.x line. The next release, 3.3.0, will begin emittingDeprecationWarnings when the pre-PEP8 methods are used (see header notes above for more information, including available automation for converting any existing code using pyparsing with the old names).

• Fixed bug when using a copy of aWord expression (either by using the explicitcopy() method, or attaching a results name), and setting a new expression name, a raisedParseException still used the original expression name. Also affectedRegex expressions withas_match oras_group_list = True. Reported by Waqas Ilyas, in Issue#612 - good catch!

• Fixed type annotation forreplace_with, to acceptAny type. Fixes Issue#602, reported by esquonk.

• Added locking around potential race condition inParserElement.reset_cache, as well as other cache-related methods. Fixes Issue#604, reported by CarlosDescalziIM.

• Substantial update to docstrings and doc generation in preparation for 3.3.0, great effort by FeRD, thanks!

• Notable addition by FeRD to convert docstring examples to work with doctest! This was long overdue, thanks so much!

• Fixed bug released in 3.2.2 in whichnested_expr could overwrite parse actions for defined content, and could truncate list of items within a nested list. Fixes Issue#600, reported by hoxbro and luisglft, with helpful diag logs and repro code.

The upcoming version 3.3.0 release will begin emittingDeprecationWarnings for pyparsing methods that have been renamed to PEP8-compliant names (introduced in pyparsing 3.0.0, in August, 2021, with legacy names retained as aliases). In preparation, I have added in pyparsing 3.2.2 a utility for finding and replacing the legacy method names with the new names. This utility is located atpyparsing/tools/cvt_pep8_names.py. This script will scan all Python files specified on the command line, and if the-u option is selected, will replace all occurrences of the old method names with the new PEP8-compliant names, updating the files in place.

Here is an example that converts all the files in the pyparsing/examples directory:

  python -m pyparsing.tools.cvt_pyparsing_pep8_names -u examples/*.py

The new names are compatible with pyparsing versions 3.0.0 and later.

• Releasedcvt_pyparsing_pep8_names.py conversion utility to upgrade pyparsing-based programs and libraries that use legacy camelCase names to use the new PEP8-compliant snake_case method names. The converter can also be imported into other scripts as

    from pyparsing.tools.cvt_pyparsing_pep8_names import pep8_converter

• Fixed bug innested_expr where nested contents were stripped of whitespace when the default whitespace characters were cleared (raised in this StackOverflow questionhttps://stackoverflow.com/questions/79327649 by Ben Alan). Also addressed bug in resolving PEP8 compliant argument name and legacy argument name.

• Fixed bug inrest_of_line and the underlyingRegex class, in which matching a pattern that could match an empty string (such as".*" or"[A-Z]*" would not raise aParseException at or beyond the end of the input string. This could cause an infinite parsing loop when parsingrest_of_line at the end of the input string. Reported by user Kylotan, thanks! (Issue#593)

• Enhancements and extra input validation forpyparsing.util.make_compressed_re - see usage inexamples/complex_chemical_formulas.py and result in the generated railroad diagramexamples/complex_chemical_formulas_diagram.html. Properly escapes characters like "." and "*" that have special meaning in regular expressions.

• Fixed bug inone_of() to properly escape characters that are regular expression markers (such as '*', '+', '?', etc.) before building the internal regex.

• Better exception message forMatchFirst andOr expressions, showing all alternatives rather than just the first one. Fixes Issue#592, reported by Focke, thanks!

• Added return type annotation of "-> None" for all__init__() methods, to satisfymypy --strict type checking. PR submitted by FeRD, thank you!

• Added optional argumentshow_hidden tocreate_diagram to show elements that are used internally by pyparsing, but are not part of the actual parser grammar. For instance, theTag class can insert values into the parsed results but it does not actually parse any input, so by default it is not included in a railroad diagram. By callingcreate_diagram withshow_hidden =True, these internal elements will be included. (You can see this in the tag_metadata.py script in the examples directory.)

• Fixed bug innumber_words.py example. Also addedebnf_number_words.py to demonstrate using theebnf.py EBNF parser generator to build a similar parser directly from EBNF.

• Fixed syntax warning raised inbigquery_view_parser.py, invalid escape sequence "\s". Reported by sameer-google, nice catch! (Issue#598)

• Added support for Python 3.14.

• Updated generated railroad diagrams to make non-terminal elements links to their related sub-diagrams. Thisgreatly improves navigation of the diagram, especially for large, complex parsers.

• Simplified railroad diagrams emitted for parsers usinginfix_notation, by hiding lookahead terms. Renamed internally generated expressions for clarity, and improved diagramming.

• Improved performance ofcpp_style_comment,c_style_comment,common.fnumber andcommon.ieee_float Regex expressions. PRs submitted by Gabriel Gerlero,
  nice work, thanks!

• Add missing type annotations tomatch_only_at_col,replace_with,remove_quotes,with_attribute, andwith_class. Issue#585 reported by rafrafrek.

• Added generated diagrams for many of the examples.

• Replaced old examples/0README.html file with examples/README.md file.

Version 3.2.0 - October, 2024

• Discontinued support for Python 3.6, 3.7, and 3.8. Adopted new Python features from Python versions 3.7-3.9:

  • Updated type annotations to use built-in container types instead of names imported from thetyping module (e.g.,list[str] vsList[str]).
  • Reworked portions of the packrat cache to leverage insertion-preserving ordering in dicts (including removal of uses ofOrderedDict).
  • Changedpdb.set_trace() call inParserElement.set_break() tobreakpoint().
  • Convertedtyping.NamedTuple todataclasses.dataclass in railroad diagramming code.
  • Addedfrom __future__ import annotations to clean up some type annotations. (with assistance from ISyncWithFoo, issue#535, thanks for the help!)

• POSSIBLE BREAKING CHANGES

  The following bugfixes may result in subtle changes in the results returned or exceptions raised by pyparsing.

  • Fixed code inParseElementEnhance subclasses that replaced detailed exception messages raised in contained expressions with a less-specific and less-informative generic exception message and location.

    If your code has conditional logic based on the message content in raisedParseExceptions, this bugfix may require changes in your code.

  • Fixed bug intransform_string() where whitespace in the input string was not properly preserved in the output string.

    If your code usestransform_string, this bugfix may require changes in your code.

  • Fixed bug where anIndexError raised in a parse action was incorrectly handled as anIndexError raised as part of theParserElement parsing methods, and reraised as aParseException. Now anIndexError that raises inside a parse action will properly propagate out as anIndexError. (Issue#573, reported by August Karlstedt, thanks!)

    If your code raisesIndexErrors in parse actions, this bugfix may require changes in your code.

• FIXES AND NEW FEATURES

  • Added type annotations to remainder ofpyparsing package, and addedmypy run totox.ini, so that type annotations are now run as part of pyparsing's CI. Addresses Issue#373, raised by Iwan Aucamp, thanks!

  • Exception message format can now be customized, by overridingParseBaseException.format_message:

    def custom_exception_message(exc) -> str:    found_phrase = f", found {exc.found}" if exc.found else ""    return f"{exc.lineno}:{exc.column} {exc.msg}{found_phrase}"ParseBaseException.formatted_message = custom_exception_message

    (PR#571 submitted by Odysseyas Krystalakos, nice work!)

  • run_tests now detects if an exception is raised in a parse action, and will report it with an enhanced error message, with the exception type, string, and parse action name.

  • QuotedString now handles translation of escaped integer, hex, octal, and Unicode sequences to their corresponding characters.

  • Fixed the displayed output ofRegex terms to deduplicate repeated backslashes, for easier reading in debugging, printing, and railroad diagrams.

  • Fixed (or at least reduced) elusive bug when generating railroad diagrams, where some diagram elements were just empty blocks. Fix submitted by RoDuth, thanks a ton!

  • Fixed railroad diagrams that get generated with a parser containing a Regex element defined using a verbose pattern - the pattern gets flattened and comments removed before creating the corresponding diagram element.

  • Defined a more performant regular expression used internally bycommon_html_entity.

  • Regex instances can now be created using a callable that takes no arguments and just returns a string or a compiled regular expression, so that creating complex regular expression patterns can be deferred until they are actually used for the first time in the parser.

  • Added optionalflatten Boolean argument toParseResults.as_list(), to return the parsed values in a flattened list.

  • Addedindent andbase_1 arguments topyparsing.testing.with_line_numbers. When usingwith_line_numbers inside a parse action, setbase_1=False, since the reportedloc value is 0-based.indent can be a leading string (typically of spaces or tabs) to indent the numbered string passed towith_line_numbers. Added while working on#557, reported by Bernd Wechner.

• NEW/ENHANCED EXAMPLES

  • Added query syntax tomongodb_query_expression.py with:

    • better support for array fields ("contains", "contains all", "contains any", and "contains none")
    • "like" and "not like" operators to support SQL "%" wildcard matching and "=~" operator to support regex matching
    • text search using "search for"
    • dates and datetimes as query values
    • a[0] style array referencing

  • Addedlox_parser.py example, a parser for the Lox language used as a tutorial in Robert Nystrom's "Crafting Interpreters" (http://craftinginterpreters.com/). With helpful corrections from RoDuth.

  • Addedcomplex_chemical_formulas.py example, to add parsing capability for formulas such as "3(C₆H₅OH)₂".

  • Updatedtag_emitter.py to use newTag class, introduced in pyparsing 3.1.3.

Changes since 3.2.0b3:

• Fixed handling ofIndexError raised in a parse action.
• QuotedString parser now handles\xnn,\ooo, and\unnnn characters whenconvert_whitespace_escapes is True.
• Reformatted CHANGES file for final release.

All changes in 3.2.0:

Version 3.2.0 - October, 2024

• Discontinued support for Python 3.6, 3.7, and 3.8. Adopted new Python features from Python versions 3.7-3.9:

  • Updated type annotations to use built-in container types instead of names imported from thetyping module (e.g.,list[str] vsList[str]).
  • Reworked portions of the packrat cache to leverage insertion-preserving ordering in dicts (including removal of uses ofOrderedDict).
  • Changedpdb.set_trace() call inParserElement.set_break() tobreakpoint().
  • Convertedtyping.NamedTuple todataclasses.dataclass in railroad diagramming code.
  • Addedfrom __future__ import annotations to clean up some type annotations. (with assistance from ISyncWithFoo, issue#535, thanks for the help!)

• POSSIBLE BREAKING CHANGES

  The following bugfixes may result in subtle changes in the results returned or exceptions raised by pyparsing.

  • Fixed code inParseElementEnhance subclasses that replaced detailed exception messages raised in contained expressions with a less-specific and less-informative generic exception message and location.

    If your code has conditional logic based on the message content in raisedParseExceptions, this bugfix may require changes in your code.

  • Fixed bug intransform_string() where whitespace in the input string was not properly preserved in the output string.

    If your code usestransform_string, this bugfix may require changes in your code.

  • Fixed bug where anIndexError raised in a parse action was incorrectly handled as anIndexError raised as part of theParserElement parsing methods, and reraised as aParseException. Now anIndexError that raises inside a parse action will properly propagate out as anIndexError. (Issue#573, reported by August Karlstedt, thanks!)

    If your code raisesIndexErrors in parse actions, this bugfix may require changes in your code.

• FIXES AND NEW FEATURES

  • Added type annotations to remainder ofpyparsing package, and addedmypy run totox.ini, so that type annotations are now run as part of pyparsing's CI. Addresses Issue#373, raised by Iwan Aucamp, thanks!

  • Exception message format can now be customized, by overridingParseBaseException.format_message:

    def custom_exception_message(exc) -> str:    found_phrase = f", found {exc.found}" if exc.found else ""    return f"{exc.lineno}:{exc.column} {exc.msg}{found_phrase}"ParseBaseException.formatted_message = custom_exception_message

    (PR#571 submitted by Odysseyas Krystalakos, nice work!)

  • run_tests now detects if an exception is raised in a parse action, and will report it with an enhanced error message, with the exception type, string, and parse action name.

  • QuotedString now handles translation of escaped integer, hex, octal, and Unicode sequences to their corresponding characters.

  • Fixed the displayed output ofRegex terms to deduplicate repeated backslashes, for easier reading in debugging, printing, and railroad diagrams.

  • Fixed (or at least reduced) elusive bug when generating railroad diagrams, where some diagram elements were just empty blocks. Fix submitted by RoDuth, thanks a ton!

  • Fixed railroad diagrams that get generated with a parser containing a Regex element defined using a verbose pattern - the pattern gets flattened and comments removed before creating the corresponding diagram element.

  • Defined a more performant regular expression used internally bycommon_html_entity.

  • Regex instances can now be created using a callable that takes no arguments and just returns a string or a compiled regular expression, so that creating complex regular expression patterns can be deferred until they are actually used for the first time in the parser.

  • Added optionalflatten Boolean argument toParseResults.as_list(), to return the parsed values in a flattened list.

  • Addedindent andbase_1 arguments topyparsing.testing.with_line_numbers. When usingwith_line_numbers inside a parse action, setbase_1=False, since the reportedloc value is 0-based.indent can be a leading string (typically of spaces or tabs) to indent the numbered string passed towith_line_numbers. Added while working on#557, reported by Bernd Wechner.

• NEW/ENHANCED EXAMPLES

  • Added query syntax tomongodb_query_expression.py with:

    • better support for array fields ("contains", "contains all", "contains any", and "contains none")
    • "like" and "not like" operators to support SQL "%" wildcard matching and "=~" operator to support regex matching
    • text search using "search for"
    • dates and datetimes as query values
    • a[0] style array referencing

  • Addedlox_parser.py example, a parser for the Lox language used as a tutorial in Robert Nystrom's "Crafting Interpreters" (http://craftinginterpreters.com/). With helpful corrections from RoDuth.

  • Addedcomplex_chemical_formulas.py example, to add parsing capability for formulas such as "3(C₆H₅OH)₂".

  • Updatedtag_emitter.py to use newTag class, introduced in pyparsing 3.1.3.

(This is the final beta release before 3.2.0.)

• QuotedString now handles translation of escaped integer, hex, octal, and Unicode sequences to their corresponding characters.