Do not directly use isolated surrogates in unicode literals#185

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Jump to bottom

Closed

gsnedders wants to merge14 commits intohtml5lib:masterfromgsnedders:isolated-surrogates

Closed

Do not directly use isolated surrogates in unicode literals#185

gsnedders wants to merge14 commits intohtml5lib:masterfromgsnedders:isolated-surrogates

Conversation

Copy link

Member

gsnedders commentedApr 26, 2015

This again. Replacement of#150 and#168 .

jimbakerand others added10 commits

April 26, 2015 00:43

Do not directly use isolated surrogates in unicode literals for platf…

c488227

…orms besides Jython

Use six.unichr for Python 3.x

89c732f

Ignore compiled Python classes for Jython

b182d1d

Pass on constructed tests in test_tokenizer that attempt to build

f8d50aa

HTMLUnicodeInputStream objects from unicode strings that containisolated surrogates. Such tests are not meaningful on Jython whichdoes not allow for invalid unicode strings to be decoded in the firstplace.

Use utils.supports_lone_surrogates in place of Jython-specific tlogic

ce7ad47

Fix flake8 trailing whitespace

75e970b

Use eval instead of unichr to deal with lone surrogates.

a11981a

IMO this is a lot more readable.

Check whether what we want is available, not interlinking things.

f4ee9d3

unicode-escape is undocumented as a decoder, so implement our own.

625303f

This also makes it easier to see what's going wrong if somethingdoes under Jython.

Rewrite support_lone_surrogates to feature-sniff.

e04fff8

Copy link

hoppipolla-critic-bot commentedApr 26, 2015

Critic review:https://critic.hoppipolla.co.uk/r/4798

This is an external review system which you may optionally use for the code review of your pull request.

In order to help critic track your changes, please donot make in-place history rewrites (e.g. viagit rebase -i orgit commit --amend) when updating this pull request.