 | This article has multiple issues. Please helpimprove it or discuss these issues on thetalk page.(Learn how and when to remove these messages) (Learn how and when to remove this message)
|
| BLOOM | |
|---|---|
| Original author | BigScience research workshop |
| Initial release | July 12, 2022; 3 years ago (2022-07-12) |
| Repository | huggingface |
| Written in | Python |
| License | BigScience Responsible AI License (RAIL) v1.0 |
| Website | bigscience |
TheBigScience Large Open-science Open-access Multilingual Language Model (BLOOM) is an open-accesslarge language model (LLM).[1] It was created by a volunteer-driven research effort to provide a transparently-created alternative to proprietary AI models.[2]
With 176 billion parameters, BLOOM is atransformer-basedautoregressive model designed to generate text in 46 natural languages and 13 programming languages. The model, source code, and the data used to train it are all distributed under free licences, allowing for public research and use.[3][4]
BLOOM is the main outcome of the BigScience initiative, a one-year-long research workshop that took place from May 2021 to May 2022.[5] The project was led byHuggingFace and involved several hundred volunteer researchers and engineers from academia and the private sector. The model was trained between March and July 2022 on the Jean Zay public supercomputer in France, managed byGENCI and IDRIS (CNRS).[6]
BLOOM's training corpus, named ROOTS, combines data extracted from the then-latest version of the web-based OSCAR corpus (38% of ROOTS) and newly collected data extracted from a manually selected and documented list of language data sources. In total, the model was trained on approximately 366 billion (1.6TB) tokens.[7][8]
 | Thislarge language model-related article is astub. You can help Wikipedia byexpanding it. |
