In this paper, we propose Minimalist Transfer Learning (MinTL) to simplify the system design process of task-oriented dialogue systems and alleviate the over-dependency on annotated data. MinTL is a simple yet effective transfer learning framework, which allows us to plug-and-play pre-trained seq2seq models, and jointly learn dialogue state tracking and dialogue response generation. Unlike previous approaches, which use a copy mechanism to “carryover” the old dialogue states to the new one, we introduce Levenshtein belief spans (Lev), that allows efficient dialogue state tracking with a minimal generation length. We instantiate our learning framework with two pre-trained backbones: T5 and BART, and evaluate them on MultiWOZ. Extensive experiments demonstrate that: 1) our systems establish new state-of-the-art results on end-to-end response generation, 2) MinTL-based systems are more robust than baseline methods in the low resource setting, and they achieve competitive results with only 20% training data, and 3) Lev greatly improves the inference efficiency.
Zhaojiang Lin, Andrea Madotto, Genta Indra Winata, and Pascale Fung. 2020.MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems. InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3391–3405, Online. Association for Computational Linguistics.
@inproceedings{lin-etal-2020-mintl, title = "{M}in{TL}: Minimalist Transfer Learning for Task-Oriented Dialogue Systems", author = "Lin, Zhaojiang and Madotto, Andrea and Winata, Genta Indra and Fung, Pascale", editor = "Webber, Bonnie and Cohn, Trevor and He, Yulan and Liu, Yang", booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)", month = nov, year = "2020", address = "Online", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2020.emnlp-main.273/", doi = "10.18653/v1/2020.emnlp-main.273", pages = "3391--3405", abstract = "In this paper, we propose Minimalist Transfer Learning (MinTL) to simplify the system design process of task-oriented dialogue systems and alleviate the over-dependency on annotated data. MinTL is a simple yet effective transfer learning framework, which allows us to plug-and-play pre-trained seq2seq models, and jointly learn dialogue state tracking and dialogue response generation. Unlike previous approaches, which use a copy mechanism to {\textquotedblleft}carryover{\textquotedblright} the old dialogue states to the new one, we introduce Levenshtein belief spans (Lev), that allows efficient dialogue state tracking with a minimal generation length. We instantiate our learning framework with two pre-trained backbones: T5 and BART, and evaluate them on MultiWOZ. Extensive experiments demonstrate that: 1) our systems establish new state-of-the-art results on end-to-end response generation, 2) MinTL-based systems are more robust than baseline methods in the low resource setting, and they achieve competitive results with only 20{\%} training data, and 3) Lev greatly improves the inference efficiency."}
<?xml version="1.0" encoding="UTF-8"?><modsCollection xmlns="http://www.loc.gov/mods/v3"><mods ID="lin-etal-2020-mintl"> <titleInfo> <title>MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems</title> </titleInfo> <name type="personal"> <namePart type="given">Zhaojiang</namePart> <namePart type="family">Lin</namePart> <role> <roleTerm authority="marcrelator" type="text">author</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Andrea</namePart> <namePart type="family">Madotto</namePart> <role> <roleTerm authority="marcrelator" type="text">author</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Genta</namePart> <namePart type="given">Indra</namePart> <namePart type="family">Winata</namePart> <role> <roleTerm authority="marcrelator" type="text">author</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Pascale</namePart> <namePart type="family">Fung</namePart> <role> <roleTerm authority="marcrelator" type="text">author</roleTerm> </role> </name> <originInfo> <dateIssued>2020-11</dateIssued> </originInfo> <typeOfResource>text</typeOfResource> <relatedItem type="host"> <titleInfo> <title>Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)</title> </titleInfo> <name type="personal"> <namePart type="given">Bonnie</namePart> <namePart type="family">Webber</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Trevor</namePart> <namePart type="family">Cohn</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Yulan</namePart> <namePart type="family">He</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Yang</namePart> <namePart type="family">Liu</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <originInfo> <publisher>Association for Computational Linguistics</publisher> <place> <placeTerm type="text">Online</placeTerm> </place> </originInfo> <genre authority="marcgt">conference publication</genre> </relatedItem> <abstract>In this paper, we propose Minimalist Transfer Learning (MinTL) to simplify the system design process of task-oriented dialogue systems and alleviate the over-dependency on annotated data. MinTL is a simple yet effective transfer learning framework, which allows us to plug-and-play pre-trained seq2seq models, and jointly learn dialogue state tracking and dialogue response generation. Unlike previous approaches, which use a copy mechanism to “carryover” the old dialogue states to the new one, we introduce Levenshtein belief spans (Lev), that allows efficient dialogue state tracking with a minimal generation length. We instantiate our learning framework with two pre-trained backbones: T5 and BART, and evaluate them on MultiWOZ. Extensive experiments demonstrate that: 1) our systems establish new state-of-the-art results on end-to-end response generation, 2) MinTL-based systems are more robust than baseline methods in the low resource setting, and they achieve competitive results with only 20% training data, and 3) Lev greatly improves the inference efficiency.</abstract> <identifier type="citekey">lin-etal-2020-mintl</identifier> <identifier type="doi">10.18653/v1/2020.emnlp-main.273</identifier> <location> <url>https://aclanthology.org/2020.emnlp-main.273/</url> </location> <part> <date>2020-11</date> <extent unit="page"> <start>3391</start> <end>3405</end> </extent> </part></mods></modsCollection>
%0 Conference Proceedings%T MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems%A Lin, Zhaojiang%A Madotto, Andrea%A Winata, Genta Indra%A Fung, Pascale%Y Webber, Bonnie%Y Cohn, Trevor%Y He, Yulan%Y Liu, Yang%S Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)%D 2020%8 November%I Association for Computational Linguistics%C Online%F lin-etal-2020-mintl%X In this paper, we propose Minimalist Transfer Learning (MinTL) to simplify the system design process of task-oriented dialogue systems and alleviate the over-dependency on annotated data. MinTL is a simple yet effective transfer learning framework, which allows us to plug-and-play pre-trained seq2seq models, and jointly learn dialogue state tracking and dialogue response generation. Unlike previous approaches, which use a copy mechanism to “carryover” the old dialogue states to the new one, we introduce Levenshtein belief spans (Lev), that allows efficient dialogue state tracking with a minimal generation length. We instantiate our learning framework with two pre-trained backbones: T5 and BART, and evaluate them on MultiWOZ. Extensive experiments demonstrate that: 1) our systems establish new state-of-the-art results on end-to-end response generation, 2) MinTL-based systems are more robust than baseline methods in the low resource setting, and they achieve competitive results with only 20% training data, and 3) Lev greatly improves the inference efficiency.%R 10.18653/v1/2020.emnlp-main.273%U https://aclanthology.org/2020.emnlp-main.273/%U https://doi.org/10.18653/v1/2020.emnlp-main.273%P 3391-3405
Zhaojiang Lin, Andrea Madotto, Genta Indra Winata, and Pascale Fung. 2020.MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems. InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3391–3405, Online. Association for Computational Linguistics.