ngram

Touch typing trainer using N-grams as data source, with options to customize the auto-generated lessons and specify the minimum typing performance needed. There are sound/color effects as well.

vue qwerty colemak dvorak ngram touch-typing keybr norman monkeytype lesson-generator amphetype

UpdatedAug 12, 2024
JavaScript

lonePatient /daguan_2019_rank9

Star130

datagrand 2019 information extraction competition rank9

crf pytorch information-extraction lstm dropout ie ngram span ner bert lookahead train-bert

UpdatedDec 29, 2019
Python

Colibri core is an NLP tool as well as a C++ and Python library for working with basic linguistic constructions such as n-grams and skipgrams (i.e patterns with one or more gaps, either of fixed or dynamic size) in a quick and memory-efficient way. At the core is the tool ``colibri-patternmodeller`` whi ch allows you to build, view, manipulate a…

python nlp c-plus-plus library corpus linguistics pattern-recognition computational-linguistics text-processing ngram ngrams skipgram

UpdatedDec 17, 2024
C++

ChrisMuir /refinr

Star104

Cluster and merge similar string values: an R implementation of Open Refine clustering algorithms

cran r openrefine clustering fuzzy-matching rstats ngram approximate-string-matching data-clustering data-cleaning

UpdatedMar 14, 2024
C++

joshualoehr /ngram-language-model

Star88

Python implementation of an N-gram language model with Laplace smoothing and sentence generation.

python nlp ngram ngrams language-models language-model ngram-language-model laplace-smoothing perplexity smoothing-methods

UpdatedFeb 9, 2018
Python

words /n-gram

Star83

Get n-grams from text

ngram unigram n-gram pentagram trigram hexagram bigram dugram tetragram heptagram octogram enneagram

UpdatedNov 1, 2022
JavaScript

vickumar1981 /stringdistance

Star81

A fuzzy matching string distance library for Scala and Java that includes Levenshtein distance, Jaro distance, Jaro-Winkler distance, Dice coefficient, N-Gram similarity, Cosine similarity, Jaccard similarity, Longest common subsequence, Hamming distance, and more..

fuzzy-matching levenshtein jaro-winkler levenshtein-distance cosine-similarity ngram soundex jaccard-similarity longest-common-subsequence hacktoberfest jaccard jaro-winkler-distance string-similarity hamming-distance jaro jaro-distance cosine-similarity-scores sorensen-dice-distance dice-coefficient soundex-algorithm

UpdatedApr 25, 2022
Scala

wrathematics /ngram

Star71

Fast n-Gram Tokenization

text-mining r text ngram

UpdatedDec 10, 2023
C

jiangnanboy /llm_corpus_quality

Star70

大模型预训练中文语料清洗及质量评估 Large model pre-training corpus cleaning

java ngram llm corpus-cleaning

UpdatedJul 25, 2024
Java

suggest-go /suggest

Star68

Top-k Approximate String Matching.

search-engine autocomplete fuzzy-search golang-library ngram language-model spellchecker fuzzy-string-matching top-k-approximate-string-matching

UpdatedDec 16, 2021
Go

BitSpeech /SRILM

Star57

Mirror of SRILM

ngram language-model srilm

UpdatedAug 11, 2020
Roff

myazi /NLP

Star37

natural language processing

crf ngram bayes mm memm

UpdatedJul 3, 2018
C++

0xVavaldi /gramify

Star35

Create n-grams of wordlists based on words, characters, or charsets to use in offline password attacks and data analysis

password ngram jtr hashcat password-cracking password-analysis mdxfind

UpdatedJun 27, 2024
Python

JackHCC /Chinese-Tokenization

Star35

利用传统方法（N-gram，HMM等）、神经网络方法（CNN，LSTM等）和预训练方法（Bert等）的中文分词任务实现【The word segmentation task is realized by using traditional methods (n-gram, HMM, etc.), neural network methods (CNN, LSTM, etc.) and pre training methods (Bert, etc.)】

nlp ngram tokenization hmm-viterbi-algorithm bilstm-crf bert-crf