I'm using the Expat python interface to parse multiple XML files in anapplication and have found that it throws a "Memory Error" exception ifmultiple calls are made to xmlparser.ParseFile(file) on the samexmlparser object. This occurs even with a vanilla xmlparser objectcreated with xml.parsers.expat.ParserCreate().Python Version: 2.6.2Operating System: Ubuntu

This also occurs with Python 2.5.1 on OS X

I am also seeing this with Python 2.5.2 on Ubuntu.

Just in case it wasn't obvious - the workaround is to create a newparser (with xml.parsers.expat.ParserCreate()) for every XML file youwant to parse.

I'm not familiar with expat, but we can see what is happening moreclearly with attached adhok patch.Traceback (most recent call last):  File "expat-error.py", line 14, in <module>    p.ParseFile(file)xml.parsers.expat.ExpatError: parsing finished: line 2, column 482It seems ParseFile() doesn't support second call. I'm not sure this isintended behavior or not.

The patch is good; a test would be appreciated.The difference now is that in case of true low-memory conditions,ExpatError("no memory") is raised instead of MemoryError.This is acceptable IMO.> It seems ParseFile() doesn't support second callThis is correct; the C expat library has a function XML_ParserReset()which could be called before calling ParseFile() again, but pyexpat doesnot expose it yet (seeissue1208730).

Well, I tried to write test like this.1. Check if xml.parsers.expat.error is raised.2. Compare *code* attribute of error object withxml.parsers.expat.errors.XML_ERROR_FINISHEDBut I noticed XML_ERROR_FINISHED is not integer but string. (!)According tohttp://docs.python.org/library/pyexpat.html#expaterror-objects> ExpatError.code>>    Expat’s internal error number for the specific error. This will>    match one of the constants defined in the errors object from>    this module.Is this document bug or implementation bug? Personally, I think string'parsing finished' as error constant might be useless...

Looks like an implementation bug to me; far too late to change it, though.In your test, you could use  pyexpat.ErrorString(e.code) == pyexpat.errors.XML_ERROR_FINISHEDAnd the docs could mention this trick.

Here is the patch. I'm not confident with my English comment though.

Do you know the new "context manager" feature of assertRaises? it makesit easier to check for exceptions.I join a new patch that uses it.

I knew existence of that new feature, but didn't know how to use it.

Hmm, looks useful. I think your patch is good. Only one problem is thatwe cannot use this new feature in python2.6. If we use my patch in thatbranch, I think there is no problem.

I don't think this is a python specific problem. I have just seen the same error when working with the expat library from C, and the causeis using the same parser to read multiple files.

The documentation should definitely be updated to clarify that a parser instance is not reusable with more than one file.  I had a look at the equivalent documentation for Perl and TCL, and Perl's implementation explicitly does not allow attempts to reuse the parser instance (which is clearly noted in the documentation), and TCL's implementation (or one of them, anyway) offers a reset call that explicitly resets the parser in preparation for another file to be submitted.

I agree that, at a minimum, the documentation should be updated to include a warning about not reusing a parser instance.  Whether it's worth trying to plug all the holes in the expat library is another issue (see, for instance,issue12829).  David, would you be willing to propose a wording for a documentation change?

Also, noteissue1208730 proposes a feature to expose a binding for XML_ParserReset and has the start of a patch.

Ned: My proposed wording is: "Note that only one document can be parsed by a given instance; it is not possible to reuse an instance to parse multiple files."  To provide more detail, one could also add something like: "The isfinal argument of the Parse() method is intended to allow the parsing of a single file in fragments, not the submission of multiple files."

Updating to reflect the Python 3.4 documentation is now also relevant to this discussion. Perhaps someone could commit a change something like my suggestion inmsg143295?

Thanks for the reminder, David.  Here are patches for 3.x and 2.7 that include updated versions of the proposed pyexpat.c and test_pyexpat.py patches along with a doc update along the lines suggested by David.

New changeset74faca1ac59c by Ned Deily in branch '2.7':Issue#6676: Ensure a meaningful exception is raised when attemptinghttp://hg.python.org/cpython/rev/74faca1ac59cNew changeset9e3fc66ee0b8 by Ned Deily in branch '3.4':Issue#6676: Ensure a meaningful exception is raised when attemptinghttp://hg.python.org/cpython/rev/9e3fc66ee0b8New changesetee0034434e65 by Ned Deily in branch 'default':Issue#6676: merge from 3.4http://hg.python.org/cpython/rev/ee0034434e65

Applied for release in 3.5.0, 3.4.1 and 2.7.7.  Thanks, everyone!