I'm currently working on the awesomeFlairlibrary and love contributing to various open source projects.

Latest news of new language models, PRs and many more!

• 03.03.2025:: BERT5urk, a new T5 language model for Turkish isout now!

• 21.12.2024: TheTurkish Model Zoo got new evaluations - performed with the awesome Flair library - seehere.

• 09.12.2024: Public announcement of theTensorFlow Model Garden LMs, including first FineWeb-LM releases on theModel Hub.

• 02.10.2024: Zeitungs-LM, a new language model trained on Historical German Newspapers isout now!

• 04.07.2024: Flair fine-tuned NER models on the awesomeCleanCoNLL dataset are now available on theModel Hub.

• 28.03.2024: New project: NER models on the recently releasedCO-Fun NER dataset. Repo ishere with a lot of fine-tuned models on theModel Hub.

• 23.12.2023: New project: NER Datasets for Historical German (HisGermaNER) is out and available on the Model Hubhere.

• 11.10.2023: New launch of hmBench project: it benchmarks Historical Multilingual Language Models such ashmBERT,hmTEAMS andhmByT5, seehere.

• 25.05.2023: New project: Historical Multilingual and Monolingual ELECTRA Models is releasedhere.

• 25.05.2023: Several ByT5 Historical Language Models are released underhmByT5 Preliminary andhmByT5 are released on the Hugging Face Model Hub.More information can be foundin this repository.

• 06.03.2023: Updated Ukrainian ELECTRA repository, seehere.

• 05.02.2023: New repository on experiments for XLM-V 🤗 Transformers Integeration, seehere.

• 03.02.2023: New repository for on-going evaluation of German T5 models on the GermEval 2014 NER task is up now! Seehere.

• 28.01.2023: Start of new language models trained on the British Library corpus (model size ranges from 110M to 1B!), repository ishere.

• 23.01.2023: New German T5 models are released (trained on the the head and middle of GC4 corpus) and are availablehere.

• 09.06.2022: Preprint of our upcoming HIPE-2022 Working Notes paper is now available here:hmBERT: Historical Multilingual Language Models for Named Entity Recognition.

• 20.02.2022: Check out our newGermanT5 organization - expect new T5 models for German soon!

• 14.12.2021: New badge: Member ofHugging Face Supporter org now 🎉

• 13.12.2021: Release of Historical Language Model for Dutch (trained on Delpher corpus) - see repohere.

• 06.12.2021: Release of smaller multilingual Historical Language Models (ranging from 2-8 layers) - see repohere.

• 18.11.2021: Release of new multilingual and monolingual Historical Language Models - as preparation for upcoming CLEF-HIPE 2022 - see repohere.

• 23.09.2021: Release of ConvBERTurk (cased and uncased) and ELECTRA (uncased) trained on Turkish part of mC4 corpus - see repohere.

• 07.09.2021: Release of new larger German GPT-2 model - see model hub cardhere.

• 17.08.2021: Release of new re-trained German GPT-2 model - see repohere.

• 05.07.2021: Preprint of the ICDAR 2021 paper"Data Centric Domain Adaptation for Historical Text with OCR Errors" together with Luisa März, Nina Poerner, Benjamin Roth and Hinrich Schütze is out now!

• 24.06.2021: Turkish Language Model Zoo repo got a new logo fromMerve Noyan, please follow her! Additionally, a new Turkish ELECTRA model was released, that was trained on the Turkish part of multilingual C4 dataset. More detailshere.

• 03.05.2021: GC4LM: A Colossal (Biased) language model for German was released. Repo with more detailshere.

• 27.04.2021: Our paper "Data Centric Domain Adaptation for Historical Text with OCR Errors" was accepted at ICDAR 2021. More details soon!

• 16.03.2021: Turkish model zoo is still growing! Public release of ConvBERTurk - see repohere.

• 07.02.2021: Public release of German Europeana DistilBERT and ConvBERT models. Repo with more information ishere.

• 28.01.2021: Expect a new German Europeana ELECTRA Large model incl. a distilled German Europeana BERT model soon 🤗

• 16.11.2020: Public release of French Europeana BERT and ELECTRA models - see repositoryhere.

• 16.11:2020: Public release of a German GPT-2 model (incl. fine-tuned model on Faust I and II). Repo with more information is availablehere.

• 11.11.2020: Public release of Ukrainian ELECTRA model. Repo is now availablehere.

• 11.11.2020: New workstation build (RTX 3090 and Ryzen 9 5900X) has completed! Expect a lot of new Flair/Transformers models in near future!

• 02.11.2020: Public release of Italian XXL ELECTRA model. New repo for Italian BERT and ELECTRA models is now availablehere 🎉

• 22.10.2020: Preprint of "German's Next Language Model" is now availablehere. Models are also available on theHugging Face model hub 🎉

• 22.10.2020: Our shared task paperTriple E - Effective Ensembling of Embeddings and Language Models for NER of Historical German together withLuisa März is released 🎉

• 30.09.2020: "German's Next Language Model" together withBranden Chan andTimo Möller was accepted atCOLING 2020!Expect new language models for German on the Hugging Face model hub soon 🤗

• 23.09.2020: Flair in version 0.6.1 isout now!

• 02.09.2020: Slow response time - I'm currently focussing on EACL 2021. Expect great new things 😎

• 18.08.2020: French BERT model, trained on Historical newspapers from Europeana:find the modelhereand the corresponding repositoryhere.

• Lukas Thoma, Ivonne Weyers, Erion Çano, Stefan Schweter, Jutta L Mueller and Benjamin Roth.CogMemLM: Human-Like Memory Mechanisms Improve Performance and Cognitive Plausibility of LLMs. In Proceedings of the BabyLM Challenge at the 27th Conference on Computational Natural Language Learning (CoNLL 2023).

• Stefan Schweter, Luisa März, Katharina Schmid and Erion Çano.hmBERT: Historical Multilingual Language Models for Named Entity Recognition. In Experimental IR Meets Multilinguality, Multimodality, and Interaction - Proceedings of the Eleventh International Conference of the CLEF Association (CLEF 2022).

• Francesco De Toni, Christopher Akiki, Javier de la Rosa, Clémentine Fourrier, Enrique Manjavacas, Stefan Schweter and Daniel Van Strien.Entities, Dates, and Languages: Zero-Shot on Historical Texts with T0. Accepted at "Challenges & Perspectives in Creating Large Language Models" Workshop at ACL 2022.

• Luisa März, Stefan Schweter, Nina Poerner, Benjamin Roth and Hinrich Schütze.Data Centric Domain Adaptation for Historical Text with OCR Errors. In International Conference on Document Analysis and Recognition, ICDAR 2021.

• Branden Chan, Stefan Schweter and Timo Möller.German's Next Language Model. In Proceedings of the 28th International Conference on Computational Linguistics.

• Stefan Schweter and Luisa März.Triple E - Effective Ensembling of Embeddings and Language Models for NER of Historical German. In Experimental IR Meets Multilinguality, Multimodality, and Interaction - Proceedings of the Eleventh International Conference of the CLEF Association (CLEF 2020).

• Stefan Schweter and Sajawel Ahmed.Deep-EOS: General-Purpose Neural Networks for Sentence Boundary Detection. In Proceedings of the 15th Conference on Natural Language Processing (KONVENS 2019).

• Alan Akbik, Tanja Bergmann, Duncan Blythe, Kashif Rasul, Stefan Schweter and Roland Vollgraf.FLAIR: An Easy-to-Use Framework for State-of-the-Art NLP. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations).

• Stefan Schweter and Johannes Baiter.Towards Robust Named Entity Recognition for Historic German. In Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019).

• Part ofBLOOM: A 176B-Parameter Open-Access Multilingual Language Model.
• Stefan Schweter and Alan Akbik.FLERT: Document-Level Features for Named Entity Recognition.

Please open an issue in the corresponding repository, tag me (@stefan-it) inissues/prs/commits on GitHub or connect with me onLinkedIn :)

PinnedLoading

1. turkish-bertturkish-bertPublic

   Turkish BERT/DistilBERT, ELECTRA, ConvBERT and T5 models

   Python 519 43

2. hmByT5hmByT5Public

   Upcoming Historical Multilingual and Monolingual ByT5 Models

   Python 6

3. europeana-berteuropeana-bertPublic

   BERT and ELECTRA models trained on Europeana Newspapers

   Python 37 2

4. ukrainian-electraukrainian-electraPublic

   Ukrainian ELECTRA model

   Python 12

5. gc4lmgc4lmPublic

   GC4LM: A Colossal (Biased) language model for German

   13

6. xlm-v-experimentsxlm-v-experimentsPublic

   Experiments for XLM-V Transformers Integeration

   Python 13 3