Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

gh-115133: test_xml_etree.py: Fix for Expat >=2.6.0 with reparse deferral (fixes #115133)#115138

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Conversation

hartwork
Copy link
Contributor

@hartworkhartwork commentedFeb 7, 2024•
edited by bedevere-appbot
Loading

hardfalcon reacted with thumbs up emoji
@hartworkhartworkforce-pushed theissue-115133-fix-etree-xml-pull-parser-tests-for-expat-2-6-0 branch from3c711d7 toa147a3eCompareFebruary 7, 2024 14:50
@serhiy-storchaka
Copy link
Member

It rather defeats the purpose of the test.

The purpose of the incremental parsing is that you can process the XML file without reading all data and building the full tree. This test tests that you get the parsed data as fast as the corresponding input data is available. The modified test only tests that it is available after feeding all input data and closing the parser. But you do not need the incremental parsing for this.

@mgorny
Copy link
Contributor

I wonder how this affects stuff like XMPP where you actually need incremental parsing for things to work at all.

@Snild-Sony
Copy link

This test tests that you get the parsed data as fast as the corresponding input data is available. The modified test only tests that it is available after feeding all input data and closing the parser. But you do not need the incremental parsing for this.

I'm not sure how to guarantee that one gets the data "as fast as the input", but it should at least be possible to assert thatsome parsing happens beforeclose(). Would that be close enough?

I wonder how this affects stuff like XMPP where you actually need incremental parsing for things to work at all.

Probably not well, at least in general. With XMPP specifically, you may get lucky and always send a whole packet for parsing at once (or just generally have small tokens), in which case there will be no incomplete tokens and therefore no need for reparse deferral. But it will be very implementation-dependent.

For such a special usecase, this should work as a "flush" operation:

XML_SetReparseDeferralEnabled(parser, XML_FALSE);XML_Parse(parser, NULL, 0, XML_FALSE); // or the last data of your "batch", if you know that ahead of timeXML_SetReparseDeferralEnabled(parser, XML_TRUE);

@hardfalcon
Copy link

hardfalcon commentedFeb 7, 2024•
edited
Loading

In case it's relevant to anyone: Python 3.11.8, built with the patch from this pull request against expat 2.6.0, and also using expat 2.6.0 after installing said Python package, is working just fine for me, both for running Gajim (an XMPP client written in Python) and for running a matrix-synapse server.

@mweineltmweinelt mentioned this pull requestFeb 10, 2024
13 tasks
@serhiy-storchaka
Copy link
Member

Thank you for your PR, but it was fixed by#115164 which keeps some of more strict testing.

@hartworkhartwork deleted the issue-115133-fix-etree-xml-pull-parser-tests-for-expat-2-6-0 branchFebruary 17, 2024 13:54
gpshead pushed a commit that referenced this pull requestFeb 29, 2024
…GH-115623)Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods:- `xml.etree.ElementTree.XMLParser.flush`- `xml.etree.ElementTree.XMLPullParser.flush`- `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled`- `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled`- `xml.sax.expatreader.ExpatParser.flush`Based on the "flush" idea from#115138 (comment) .### Notes- Please treat as a security fix related toCVE-2023-52425.Includes code suggested-by: Snild Dolkow <snild@sony.com>and by core dev Serhiy Storchaka.
hartwork added a commit to hartwork/cpython that referenced this pull requestMar 2, 2024
…52425) (pythonGH-115623)Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods:- `xml.etree.ElementTree.XMLParser.flush`- `xml.etree.ElementTree.XMLPullParser.flush`- `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled`- `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled`- `xml.sax.expatreader.ExpatParser.flush`Based on the "flush" idea frompython#115138 (comment) .- Please treat as a security fix related toCVE-2023-52425.Includes code suggested-by: Snild Dolkow <snild@sony.com>and by core dev Serhiy Storchaka.(cherry picked from commit6a95676)
hartwork added a commit to hartwork/cpython that referenced this pull requestMar 3, 2024
…52425) (pythonGH-115623)Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods:- `xml.etree.ElementTree.XMLParser.flush`- `xml.etree.ElementTree.XMLPullParser.flush`- `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled`- `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled`- `xml.sax.expatreader.ExpatParser.flush`Based on the "flush" idea frompython#115138 (comment) .- Please treat as a security fix related toCVE-2023-52425.Includes code suggested-by: Snild Dolkow <snild@sony.com>and by core dev Serhiy Storchaka.(cherry picked from commit6a95676)
hartwork added a commit to hartwork/cpython that referenced this pull requestMar 3, 2024
…52425) (pythonGH-115623)Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods:- `xml.etree.ElementTree.XMLParser.flush`- `xml.etree.ElementTree.XMLPullParser.flush`- `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled`- `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled`- `xml.sax.expatreader.ExpatParser.flush`Based on the "flush" idea frompython#115138 (comment) .- Please treat as a security fix related toCVE-2023-52425.Includes code suggested-by: Snild Dolkow <snild@sony.com>and by core dev Serhiy Storchaka.(cherry picked from commit6a95676)
hartwork added a commit to hartwork/cpython that referenced this pull requestMar 3, 2024
…52425) (pythonGH-115623)Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods:- `xml.etree.ElementTree.XMLParser.flush`- `xml.etree.ElementTree.XMLPullParser.flush`- `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled`- `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled`- `xml.sax.expatreader.ExpatParser.flush`Based on the "flush" idea frompython#115138 (comment) .- Please treat as a security fix related toCVE-2023-52425.Includes code suggested-by: Snild Dolkow <snild@sony.com>and by core dev Serhiy Storchaka.(cherry picked from commit6a95676)
hartwork added a commit to hartwork/cpython that referenced this pull requestMar 3, 2024
…52425) (pythonGH-115623)Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods:- `xml.etree.ElementTree.XMLParser.flush`- `xml.etree.ElementTree.XMLPullParser.flush`- `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled`- `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled`- `xml.sax.expatreader.ExpatParser.flush`Based on the "flush" idea frompython#115138 (comment) .- Please treat as a security fix related toCVE-2023-52425.Includes code suggested-by: Snild Dolkow <snild@sony.com>and by core dev Serhiy Storchaka.(cherry picked from commit6a95676)
woodruffw pushed a commit to woodruffw-forks/cpython that referenced this pull requestMar 4, 2024
…52425) (pythonGH-115623)Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods:- `xml.etree.ElementTree.XMLParser.flush`- `xml.etree.ElementTree.XMLPullParser.flush`- `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled`- `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled`- `xml.sax.expatreader.ExpatParser.flush`Based on the "flush" idea frompython#115138 (comment) .- Please treat as a security fix related toCVE-2023-52425.Includes code suggested-by: Snild Dolkow <snild@sony.com>and by core dev Serhiy Storchaka.
gpshead added a commit that referenced this pull requestMar 6, 2024
…-52425) (GH-115623) (GH-116248)Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods:- `xml.etree.ElementTree.XMLParser.flush`- `xml.etree.ElementTree.XMLPullParser.flush`- `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled`- `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled`- `xml.sax.expatreader.ExpatParser.flush`Based on the "flush" idea from#115138 (comment) .- Please treat as a security fix related toCVE-2023-52425.(cherry picked from commit6a95676)(cherry picked from commit73807eb)(cherry picked from commiteda2963)---------Includes code suggested-by: Snild Dolkow <snild@sony.com>and by core dev Serhiy Storchaka.Co-authored-by: Gregory P. Smith <greg@krypto.org>
hartwork added a commit to hartwork/cpython that referenced this pull requestMar 6, 2024
…52425) (pythonGH-115623)Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods:- `xml.etree.ElementTree.XMLParser.flush`- `xml.etree.ElementTree.XMLPullParser.flush`- `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled`- `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled`- `xml.sax.expatreader.ExpatParser.flush`Based on the "flush" idea frompython#115138 (comment) .- Please treat as a security fix related toCVE-2023-52425.Includes code suggested-by: Snild Dolkow <snild@sony.com>and by core dev Serhiy Storchaka.(cherry picked from commit6a95676)
github-actionsbot pushed a commit to m-aciek/python-docs-weblate that referenced this pull requestMar 6, 2024
…-52425) (GH-115623) (GH-116248)Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods:- `xml.etree.ElementTree.XMLParser.flush`- `xml.etree.ElementTree.XMLPullParser.flush`- `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled`- `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled`- `xml.sax.expatreader.ExpatParser.flush`Based on the "flush" idea frompython/cpython#115138 (comment) .- Please treat as a security fix related toCVE-2023-52425.(cherry picked from commit 6a95676bb526261434dd068d6c49927c44d24a9b)(cherry picked from commit 73807eb634315f70a464a18feaae33d9e065de09)(cherry picked from commit eda2963378a3c292cf6bb202bb00e94e46ee6d90)---------Includes code suggested-by: Snild Dolkow <snild@sony.com>and by core dev Serhiy Storchaka.Co-authored-by: Gregory P. Smith <greg@krypto.org>CPython-sync-commit-latest: 0a01ed6c2a116bd3e174fce33c21d84d650de569
gpshead added a commit that referenced this pull requestMar 6, 2024
…-52425) (GH-115623) (#116268)Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods:- `xml.etree.ElementTree.XMLParser.flush`- `xml.etree.ElementTree.XMLPullParser.flush`- `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled`- `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled`- `xml.sax.expatreader.ExpatParser.flush`Based on the "flush" idea from#115138 (comment) .- Please treat as a security fix related toCVE-2023-52425.(cherry picked from commit 6a95676)(cherry picked from commit73807eb)(cherry picked from commiteda2963)---------Includes code suggested-by: Snild Dolkow <snild@sony.com>and by core dev Serhiy Storchaka.Co-authored-by: Gregory P. Smith <greg@krypto.org>
ambv pushed a commit that referenced this pull requestMar 6, 2024
…-52425) (GH-115623) (GH-116270)Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods:- `xml.etree.ElementTree.XMLParser.flush`- `xml.etree.ElementTree.XMLPullParser.flush`- `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled`- `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled`- `xml.sax.expatreader.ExpatParser.flush`Based on the "flush" idea from#115138 (comment) .Includes code suggested-by: Snild Dolkow <snild@sony.com>and by core dev Serhiy Storchaka.Co-authored-by: Gregory P. Smith <greg@krypto.org>
ambv pushed a commit that referenced this pull requestMar 6, 2024
…52425) (GH-115623) (GH-116272)Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods:- `xml.etree.ElementTree.XMLParser.flush`- `xml.etree.ElementTree.XMLPullParser.flush`- `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled`- `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled`- `xml.sax.expatreader.ExpatParser.flush`Based on the "flush" idea from#115138 (comment) .Includes code suggested-by: Snild Dolkow <snild@sony.com>and by core dev Serhiy Storchaka.Co-authored-by: Gregory P. Smith <greg@krypto.org>
ambv pushed a commit that referenced this pull requestMar 6, 2024
…52425) (GH-115623) (GH-116275)Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods:- `xml.etree.ElementTree.XMLParser.flush`- `xml.etree.ElementTree.XMLPullParser.flush`- `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled`- `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled`- `xml.sax.expatreader.ExpatParser.flush`Based on the "flush" idea from#115138 (comment) .Includes code suggested-by: Snild Dolkow <snild@sony.com>and by core dev Serhiy Storchaka.Co-authored-by: Gregory P. Smith <greg@krypto.org>
adorilson pushed a commit to adorilson/cpython that referenced this pull requestMar 25, 2024
…52425) (pythonGH-115623)Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods:- `xml.etree.ElementTree.XMLParser.flush`- `xml.etree.ElementTree.XMLPullParser.flush`- `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled`- `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled`- `xml.sax.expatreader.ExpatParser.flush`Based on the "flush" idea frompython#115138 (comment) .### Notes- Please treat as a security fix related toCVE-2023-52425.Includes code suggested-by: Snild Dolkow <snild@sony.com>and by core dev Serhiy Storchaka.
diegorusso pushed a commit to diegorusso/cpython that referenced this pull requestApr 17, 2024
…52425) (pythonGH-115623)Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods:- `xml.etree.ElementTree.XMLParser.flush`- `xml.etree.ElementTree.XMLPullParser.flush`- `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled`- `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled`- `xml.sax.expatreader.ExpatParser.flush`Based on the "flush" idea frompython#115138 (comment) .### Notes- Please treat as a security fix related toCVE-2023-52425.Includes code suggested-by: Snild Dolkow <snild@sony.com>and by core dev Serhiy Storchaka.
jessecomeau87 pushed a commit to jessecomeau87/Python that referenced this pull requestMay 20, 2024
… (GH-115623)Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods:- `xml.etree.ElementTree.XMLParser.flush`- `xml.etree.ElementTree.XMLPullParser.flush`- `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled`- `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled`- `xml.sax.expatreader.ExpatParser.flush`Based on the "flush" idea frompython/cpython#115138 (comment) .- Please treat as a security fix related toCVE-2023-52425.Includes code suggested-by: Snild Dolkow <snild@sony.com>and by core dev Serhiy Storchaka.(cherry picked from commit6a95676)
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
awaiting reviewtestsTests in the Lib/test dir
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

test.test_xml_etree*.XMLPullParserTest.test_simple_xml fails with (system) expat 2.6.0
5 participants
@hartwork@serhiy-storchaka@mgorny@Snild-Sony@hardfalcon
[8]ページ先頭
©2009-2025 Movatter.jp