Annotated, they have been used incorpus linguistics for statisticalhypothesis testing, checking occurrences or validating linguistic rules within a specific language territory.
A corpus may contain texts in a single language (monolingual corpus) or text data in multiple languages (multilingual corpus).
In order to make the corpora more useful for doing linguistic research, they are often subjected to a process known asannotation. An example of annotating a corpus ispart-of-speech tagging, orPOS-tagging, in which information about each word's part of speech (verb, noun, adjective, etc.) is added to the corpus in the form oftags. Another example is indicating thelemma (base) form of each word. When the language of the corpus is not a working language of the researchers who use it,interlinear glossing is used to make the annotation bilingual.
Some corpora have furtherstructured levels of analysis applied. In particular, smaller corpora may be fullyparsed. Such corpora are usually calledTreebanks orParsed Corpora. The difficulty of ensuring that the entire corpus is completely and consistently annotated means that these corpora are usually smaller, containing around one to three million words. Other levels of linguistic structured analysis are possible, including annotations formorphology,semantics andpragmatics.
The analysis and processing of various types of corpora are also the subject of much work incomputational linguistics,speech recognition andmachine translation, where they are often used to createhidden Markov models for part of speech tagging and other purposes. Corpora andfrequency lists derived from them are useful forlanguage teaching. Corpora can be considered as a type offoreign language writing aid as the contextualised grammatical knowledge acquired by non-native language users through exposure to authentic texts in corpora allows learners to grasp the manner of sentence formation in the target language, enabling effective writing.[1]
Multilingual corpora that have been specially formatted for side-by-side comparison are calledaligned parallel corpora. There are two main types ofparallel corpora which contain texts in two languages. In atranslation corpus, the texts in one language are translations of texts in the other language. In acomparable corpus, the texts are of the same kind and cover the same content, but they are not translations of each other.[2] To exploit a parallel text, some kind of text alignment identifying equivalent text segments (phrases or sentences) is a prerequisite for analysis.Machine translation algorithms for translating between two languages are often trained using parallel fragments comprising a first-language corpus and a second-language corpus, which is an element-for-element translation of the first-language corpus.[3]
Text corpora are also used in the study ofhistorical documents, for example in attempts todecipher ancient scripts, or inBiblical scholarship. Some archaeological corpora can be of such short duration that they provide a snapshot in time. One of the shortest corpora in time may be the 15–30 yearAmarna letters texts (1350 BC). Thecorpus of an ancient city, (for example the "Kültepe Texts" of Turkey), may go through a series of corpora, determined by their find site dates.
^Wolk, Krzysztof; Marasek, Krzysztof (2015). "Tuned and GPU-accelerated parallel data mining from comparable corpora". In Král, Pavel; Matoušek, Václav (eds.).Text, Speech, and Dialogue – 18th International Conference, TSD 2015, Plzeň, Czech Republic, September 14–17, 2015, Proceedings. Lecture Notes in Computer Science. Vol. 9302. Springer. pp. 32–40.arXiv:1509.08639.doi:10.1007/978-3-319-24033-6_4.ISBN978-3-319-24032-9.
