gsnedders@49f37d2,gsnedders@64e8b0b, andgsnedders@cc9f28a are all things I'm against landing upstream (they should just be special-cased in the downstream tree builder rather than requiring invoking separate handling for all tree builders).
gsnedders@7702d80 is something I find very inelegant, and goes against the recommendations laid out in PEP 8. Also, with CPython, in theTrue/False cases it's likely slower, therefore failing at its stated goal, as it results in more byte code andPOP_JUMP_IF_FALSE andPOP_JUMP_IF_TRUE special-case the condition beingTrue orFalse (oddly, they don't specialiseNone, though it is inPyObject_IsTrue; if that makes any notable performance difference then I'd suggest fixing that in CPython).
gsnedders@a2d2e05 is unlikely to land because the input stream isn't public API (not that we document what is anywhere…).
gsnedders@1576515 in principle is something we want, but I think we want for all tokens and not just elements. (See alsoAdd line/column number in tokens from HTMLTokenizer #87 andAdd element.sourceline (lxml) #97.) That or something similar will likely land in future.
gsnedders@124e975 is included here, after9dca7d8 which fixes the DOM tree builder to follow the tree builder API correctly. (It exposed bugs in the DOM tree builder, yay!)
gsnedders@db0b5a0 is what most of the commits on this PR deal with, fixing up small issues with the Calibre code.

Copy link

MemberAuthor

gsnedders commentedMay 7, 2016

This seems to have broken 2.6 badly. Huh.

Copy link

codecov-io commentedMay 7, 2016•
edited
Loading

Current coverage is89.15%

Merging#245 intomaster will decrease coverage by1.39%

@@             master       #245   diff @@==========================================  Files            51         50     -1     Lines          6817       6726    -91     Methods           0          0            Messages          0          0            Branches       1316       1307     -9   ==========================================- Hits           6172       5996   -176- Misses          485        559    +74- Partials        160        171    +11

Powered byCodecov. Last updated by143b0d4...82b971c

Copy link

MemberAuthor

gsnedders commentedMay 7, 2016

Oh, right, this isviewkeys (and dictionary views in general) not existing on 2.6.

gsnedders modified the milestone:0.99999999

May 8, 2016

Copy link

Contributor

kovidgoyal commentedMay 8, 2016•
edited
Loading

How do you suggest I override the application of attributes for html and body tags in my builder? Since without those patches, it would require overriding the entire getPhases() method in html5parser.py Remember that the problem those patches is solving is that there can be multiple<html> and<body> tags in malformed documents that I need to handle by merging the attributes from those tags into the first / tag. Doing that after parsing would require an whole extra tree traversal which can be slow for large trees.

If you dont want to mergegsnedders/html5lib-python@a2d2e05 then how do you suggest I replace the the stream input class? The one in html5lib is too slow. The only alternative I can see is monkey patching -- which is less than optimal for obvious reasons.

Copy link

MemberAuthor

gsnedders commentedMay 8, 2016

@kovidgoyal I'll take a look at dealing with html/body attributes later (I'm literally amount to board a plane).

When it comes to the input stream, if it yields good perf increases when given a byte/unicode object we should just specialise them in html5lib.

Copy link

Contributor

kovidgoyal commentedMay 8, 2016

Sure you are welcome to take the input stream class from calibre for dealing with unicode objects. It is faster because it avoids wrapping the unicode in StringIO. And it actually implements tracking of positions. For my use case, that is important, since I need line and col numbers.