- Notifications
You must be signed in to change notification settings - Fork0
UBC-NLP/SPARROW
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
Chiyu Zhang, Khai Duy Doan, Qisheng Liao,Muhammad Abdul-Mageed
Publish at Main Conference of EMNLP 2023
Comparison of SM benchmarks with leaderboards. SPARROW is an evaluation benchmark for sociopragmatic meaning understanding. SPARROW comprises 169 datasets covering 13 task types across six primary categories (e.g., anti-social language detection, emotion recognition). SPARROW datasets encompass 64 different languages originating from 12 language families representing 16 writing scripts.- You can access our SPRROW benchmark and leaderboardhere.
- You can find SPARROW benchmark onhuggingface datasets.
- If you want to get the labels of Test set, please contact Chiyu via (zcy94@outlook.com)
- More guidance for submitting your system is providedhere.
- InfoDCL-XLM-RoBERTa Base trained with multilingual TweetEmoji-multi:https://huggingface.co/UBC-NLP/InfoDCL-Emoji-XLMR-Base
Please cite us if you use our data or models.
@inproceedings{zhang-etal-2023-skipped,title ="The Skipped Beat: A Study of Sociopragmatic Understanding in LLMs for 64 Languages",author ="Zhang, Chiyu and Khai Duy Doan and, Qisheng Liao and, Abdul-Mageed, Muhammad",booktitle ="Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP)",year ="2023",publisher ="Association for Computational Linguistics",}