Happy to announce Slovenian LLM eval and the preliminary SlovenianGPT results! 🇸🇮 TL;DR it's currently the strongest Slovenian LLM in the world outperforming Mistral 7B, LLaMA 2 7B, Google's Gemma and various Slovenian baselines.HuggingFace:https://lnkd.in/dGsQCRx2This is just the start, version 0 of the eval, and I'm actively looking for sponsors to further refine it using GPT-4. If you are willing to sponsor the project that will benefit Slovenian LLM ecosystem - dm me!On the chart below are the preliminary results of the SlovenianGPT 7B LLM I recently finished training. Note that winogrande is missing for now and will become available in the v1 of the eval.As I said it outperforms Mistral 7B, LLaMA 7B, Google's Gemma as well as some smaller Slovenian baselines (gpt-sl-base, t5-sl-large) and thus receives the title of the best (soon to be open-sourced) Slovenian LLM in the world!I'll also kick off a new iteration probably later during the day and I strongly suspect it'll be even stronger. :)Gemma seems to be the weakest baseline, Mistral is a bit stronger on triviaqa & nq_open but as I said that's going to change soon (I suspect it has to do with lower quality of evals right now as well).Over the next few weeks the model will become available through yugochat (https://www.yugochat.com/) and Runa AI API (https://dev.runaai.com/).As always thank you to theHyperstack folks for H100 compute sponsorship! Check them out if you need to make some GPUs go brrrr. :)) Also a big thank you to my Discord community who helped with the eval effort.