Ann-gram is a sequence ofn adjacent symbols in particular order.[1] The symbols may ben adjacentletters (includingpunctuation marks and blanks),syllables, or rarely wholewords found in a language dataset; or adjacentphonemes extracted from a speech-recording dataset, or adjacent base pairs extracted from a genome. They are collected from atext corpus orspeech corpus.
N-gram is actually theparent of a family of names term, wherefamily members can be (depending onn numeral) 1-gram, 2-gram etc., or the same using spoken numeral prefixes.
IfLatin numerical prefixes are used, thenn-gram of size 1 is called a "unigram", size 2 a "bigram" (or, less commonly, a "digram") etc. If, instead of the Latin ones, theEnglish cardinal numbers are furtherly used, then they are called "four-gram", "five-gram", etc. Similarly, usingGreek numerical prefixes such as "monomer", "dimer", "trimer", "tetramer", "pentamer", etc., or English cardinal numbers, "one-mer", "two-mer", "three-mer", etc. are used in computational biology, forpolymers oroligomers of a known size, calledk-mers. When the items are words,n-grams may also be calledshingles.[2]
In the context ofnatural language processing (NLP), the use ofn-grams allowsbag-of-words models to capture information such as word order, which would not be possible in the traditional bag of words setting.
(Shannon 1951)[3] discussedn-gram models of English. For example:
| Field | Unit | Sample sequence | 1-gram sequence | 2-gram sequence | 3-gram sequence |
|---|---|---|---|---|---|
| Vernacular name | unigram | bigram | trigram | ||
| Order of resultingMarkov model | 0 | 1 | 2 | ||
| Protein sequencing | amino acid | ... Cys-Gly-Leu-Ser-Trp ... | ..., Cys, Gly, Leu, Ser, Trp, ... | ..., Cys-Gly, Gly-Leu, Leu-Ser, Ser-Trp, ... | ..., Cys-Gly-Leu, Gly-Leu-Ser, Leu-Ser-Trp, ... |
| DNA sequencing | base pair | ...AGCTTCGA... | ..., A, G, C, T, T, C, G, A, ... | ..., AG, GC, CT, TT, TC, CG, GA, ... | ..., AGC, GCT, CTT, TTC, TCG, CGA, ... |
| Language model | character | ...to_be_or_not_to_be... | ..., t, o, _, b, e, _, o, r, _, n, o, t, _, t, o, _, b, e, ... | ..., to, o_, _b, be, e_, _o, or, r_, _n, no, ot, t_, _t, to, o_, _b, be, ... | ..., to_, o_b, _be, be_, e_o, _or, or_, r_n, _no, not, ot_, t_t, _to, to_, o_b, _be, ... |
| Wordn-gram language model | word | ... to be or not to be ... | ..., to, be, or, not, to, be, ... | ..., to be, be or, or not, not to, to be, ... | ..., to be or, be or not, or not to, not to be, ... |
Figure 1 shows several example sequences and the corresponding 1-gram, 2-gram and 3-gram sequences.
Here are further examples; these are word-level 3-grams and 4-grams (and counts of the number of times they appeared) from the Googlen-gram corpus.[4]
3-grams
4-grams