- Notifications
You must be signed in to change notification settings - Fork936
Pull requests: huggingface/tokenizers
Author
Uh oh!
There was an error while loading.Please reload this page.
Label
Uh oh!
There was an error while loading.Please reload this page.
Projects
Uh oh!
There was an error while loading.Please reload this page.
Milestones
Uh oh!
There was an error while loading.Please reload this page.
Reviews
Assignee
Assigned to nobodyLoading
Uh oh!
There was an error while loading.Please reload this page.
Sort
Pull requests list
Bump on-headers and compression in /tokenizers/examples/unstable_wasm/www dependenciesPull requests that update a dependency file javascriptPull requests that update Javascript code
#1827 openedJul 17, 2025 bydependabotbotLoading…
Expose
Encoding
attributes via the buffer protocol interface #1789 openedJun 4, 2025 bymariosaskoLoading…
Add benchmark for deserializing large added vocab + optimizations
#1782 openedMay 27, 2025 byArthurZucker • Draft
Implement
from_bytes
andread_bytes
Methods in WordPiece Tokenizer for WebAssembly Compatibility #1758 openedMar 31, 2025 bysondalexLoading…
Pre-tokenizers that support multi-word/non-whitespace BPE in single pass
#1753 openedMar 22, 2025 bymjbommarLoading…
[Feature] support Assign token to update the content of a token Stale
#1570 openedJul 12, 2024 byArthurZucker • Draft
1 task
ProTip! Filter pull requests by the default branch withbase:main.