- Notifications
You must be signed in to change notification settings - Fork32
Treat emoji presentation sequences as fullwidth#35
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Treat emoji presentation sequences as fullwidth#35
Uh oh!
There was an error while loading.Please reload this page.
Conversation
Jules-Bertholet commentedFeb 11, 2024
In terms ofUTS 51 conformance, with this PR, this crate will give the correct widths for:
However, it may overestimate (though never underestimate) the rendered widths of: |
Jules-Bertholet commentedFeb 14, 2024 • edited
Loading Uh oh!
There was an error while loading.Please reload this page.
edited
Uh oh!
There was an error while loading.Please reload this page.
I've replaced the binary search with a better datastructure, and also added a section in the rustdoc documenting the full width rules. |
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Jules-Bertholet commentedMar 12, 2024
The not-yet-released Unicode 16 adds 8 new non-emoji standardized variation sequences that affect width:https://unicode.org/alloc/Pipeline.html#variation_sequences,https://www.unicode.org/L2/L2023/23212r-quotes-svs-proposal.pdf. In time, we'll need to support those as well. |
Faster and smaller!
Ensure rows don't cross cache lines, makes a small difference in the benchmarks
Jules-Bertholet commentedApr 23, 2024
This PR is bloated, I'm splitting it up. |
Uh oh!
There was an error while loading.Please reload this page.
UAX11 says:
Lookup is done with a 2-level trie.