As an important research issue in the natural language processing community, multi-label emotion detection has been drawing more and more attention in the last few years. However, almost all existing studies focus on one modality (e.g., textual modality). In this paper, we focus on multi-label emotion detection in a multi-modal scenario. In this scenario, we need to consider both the dependence among different labels (label dependence) and the dependence between each predicting label and different modalities (modality dependence). Particularly, we propose a multi-modal sequence-to-set approach to effectively model both kinds of dependence in multi-modal multi-label emotion detection. The detailed evaluation demonstrates the effectiveness of our approach.
Dong Zhang, Xincheng Ju, Junhui Li, Shoushan Li, Qiaoming Zhu, and Guodong Zhou. 2020.Multi-modal Multi-label Emotion Detection with Modality and Label Dependence. InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3584–3593, Online. Association for Computational Linguistics.
@inproceedings{zhang-etal-2020-multi, title = "Multi-modal Multi-label Emotion Detection with Modality and Label Dependence", author = "Zhang, Dong and Ju, Xincheng and Li, Junhui and Li, Shoushan and Zhu, Qiaoming and Zhou, Guodong", editor = "Webber, Bonnie and Cohn, Trevor and He, Yulan and Liu, Yang", booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)", month = nov, year = "2020", address = "Online", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2020.emnlp-main.291/", doi = "10.18653/v1/2020.emnlp-main.291", pages = "3584--3593", abstract = "As an important research issue in the natural language processing community, multi-label emotion detection has been drawing more and more attention in the last few years. However, almost all existing studies focus on one modality (e.g., textual modality). In this paper, we focus on multi-label emotion detection in a multi-modal scenario. In this scenario, we need to consider both the dependence among different labels (label dependence) and the dependence between each predicting label and different modalities (modality dependence). Particularly, we propose a multi-modal sequence-to-set approach to effectively model both kinds of dependence in multi-modal multi-label emotion detection. The detailed evaluation demonstrates the effectiveness of our approach."}
<?xml version="1.0" encoding="UTF-8"?><modsCollection xmlns="http://www.loc.gov/mods/v3"><mods ID="zhang-etal-2020-multi"> <titleInfo> <title>Multi-modal Multi-label Emotion Detection with Modality and Label Dependence</title> </titleInfo> <name type="personal"> <namePart type="given">Dong</namePart> <namePart type="family">Zhang</namePart> <role> <roleTerm authority="marcrelator" type="text">author</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Xincheng</namePart> <namePart type="family">Ju</namePart> <role> <roleTerm authority="marcrelator" type="text">author</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Junhui</namePart> <namePart type="family">Li</namePart> <role> <roleTerm authority="marcrelator" type="text">author</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Shoushan</namePart> <namePart type="family">Li</namePart> <role> <roleTerm authority="marcrelator" type="text">author</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Qiaoming</namePart> <namePart type="family">Zhu</namePart> <role> <roleTerm authority="marcrelator" type="text">author</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Guodong</namePart> <namePart type="family">Zhou</namePart> <role> <roleTerm authority="marcrelator" type="text">author</roleTerm> </role> </name> <originInfo> <dateIssued>2020-11</dateIssued> </originInfo> <typeOfResource>text</typeOfResource> <relatedItem type="host"> <titleInfo> <title>Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)</title> </titleInfo> <name type="personal"> <namePart type="given">Bonnie</namePart> <namePart type="family">Webber</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Trevor</namePart> <namePart type="family">Cohn</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Yulan</namePart> <namePart type="family">He</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Yang</namePart> <namePart type="family">Liu</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <originInfo> <publisher>Association for Computational Linguistics</publisher> <place> <placeTerm type="text">Online</placeTerm> </place> </originInfo> <genre authority="marcgt">conference publication</genre> </relatedItem> <abstract>As an important research issue in the natural language processing community, multi-label emotion detection has been drawing more and more attention in the last few years. However, almost all existing studies focus on one modality (e.g., textual modality). In this paper, we focus on multi-label emotion detection in a multi-modal scenario. In this scenario, we need to consider both the dependence among different labels (label dependence) and the dependence between each predicting label and different modalities (modality dependence). Particularly, we propose a multi-modal sequence-to-set approach to effectively model both kinds of dependence in multi-modal multi-label emotion detection. The detailed evaluation demonstrates the effectiveness of our approach.</abstract> <identifier type="citekey">zhang-etal-2020-multi</identifier> <identifier type="doi">10.18653/v1/2020.emnlp-main.291</identifier> <location> <url>https://aclanthology.org/2020.emnlp-main.291/</url> </location> <part> <date>2020-11</date> <extent unit="page"> <start>3584</start> <end>3593</end> </extent> </part></mods></modsCollection>
%0 Conference Proceedings%T Multi-modal Multi-label Emotion Detection with Modality and Label Dependence%A Zhang, Dong%A Ju, Xincheng%A Li, Junhui%A Li, Shoushan%A Zhu, Qiaoming%A Zhou, Guodong%Y Webber, Bonnie%Y Cohn, Trevor%Y He, Yulan%Y Liu, Yang%S Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)%D 2020%8 November%I Association for Computational Linguistics%C Online%F zhang-etal-2020-multi%X As an important research issue in the natural language processing community, multi-label emotion detection has been drawing more and more attention in the last few years. However, almost all existing studies focus on one modality (e.g., textual modality). In this paper, we focus on multi-label emotion detection in a multi-modal scenario. In this scenario, we need to consider both the dependence among different labels (label dependence) and the dependence between each predicting label and different modalities (modality dependence). Particularly, we propose a multi-modal sequence-to-set approach to effectively model both kinds of dependence in multi-modal multi-label emotion detection. The detailed evaluation demonstrates the effectiveness of our approach.%R 10.18653/v1/2020.emnlp-main.291%U https://aclanthology.org/2020.emnlp-main.291/%U https://doi.org/10.18653/v1/2020.emnlp-main.291%P 3584-3593
[Multi-modal Multi-label Emotion Detection with Modality and Label Dependence](https://aclanthology.org/2020.emnlp-main.291/) (Zhang et al., EMNLP 2020)
Dong Zhang, Xincheng Ju, Junhui Li, Shoushan Li, Qiaoming Zhu, and Guodong Zhou. 2020.Multi-modal Multi-label Emotion Detection with Modality and Label Dependence. InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3584–3593, Online. Association for Computational Linguistics.