In this paper, we attempt to link the inner workings of a neural language model to linguistic theory, focusing on a complex phenomenon well discussed in formal linguistics: (negative) polarity items. We briefly discuss the leading hypotheses about the licensing contexts that allow negative polarity items and evaluate to what extent a neural language model has the ability to correctly process a subset of such constructions. We show that the model finds a relation between the licensing context and the negative polarity item and appears to be aware of thescope of this context, which we extract from a parse tree of the sentence. With this research, we hope to pave the way for other studies linking formal linguistics to deep learning.
@inproceedings{jumelet-hupkes-2018-language, title = "Do Language Models Understand Anything? On the Ability of {LSTM}s to Understand Negative Polarity Items", author = "Jumelet, Jaap and Hupkes, Dieuwke", editor = "Linzen, Tal and Chrupa{\l}a, Grzegorz and Alishahi, Afra", booktitle = "Proceedings of the 2018 {EMNLP} Workshop {B}lackbox{NLP}: Analyzing and Interpreting Neural Networks for {NLP}", month = nov, year = "2018", address = "Brussels, Belgium", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/W18-5424/", doi = "10.18653/v1/W18-5424", pages = "222--231", abstract = "In this paper, we attempt to link the inner workings of a neural language model to linguistic theory, focusing on a complex phenomenon well discussed in formal linguistics: (negative) polarity items. We briefly discuss the leading hypotheses about the licensing contexts that allow negative polarity items and evaluate to what extent a neural language model has the ability to correctly process a subset of such constructions. We show that the model finds a relation between the licensing context and the negative polarity item and appears to be aware of the \textit{scope} of this context, which we extract from a parse tree of the sentence. With this research, we hope to pave the way for other studies linking formal linguistics to deep learning."}
<?xml version="1.0" encoding="UTF-8"?><modsCollection xmlns="http://www.loc.gov/mods/v3"><mods ID="jumelet-hupkes-2018-language"> <titleInfo> <title>Do Language Models Understand Anything? On the Ability of LSTMs to Understand Negative Polarity Items</title> </titleInfo> <name type="personal"> <namePart type="given">Jaap</namePart> <namePart type="family">Jumelet</namePart> <role> <roleTerm authority="marcrelator" type="text">author</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Dieuwke</namePart> <namePart type="family">Hupkes</namePart> <role> <roleTerm authority="marcrelator" type="text">author</roleTerm> </role> </name> <originInfo> <dateIssued>2018-11</dateIssued> </originInfo> <typeOfResource>text</typeOfResource> <relatedItem type="host"> <titleInfo> <title>Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP</title> </titleInfo> <name type="personal"> <namePart type="given">Tal</namePart> <namePart type="family">Linzen</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Grzegorz</namePart> <namePart type="family">Chrupała</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <name type="personal"> <namePart type="given">Afra</namePart> <namePart type="family">Alishahi</namePart> <role> <roleTerm authority="marcrelator" type="text">editor</roleTerm> </role> </name> <originInfo> <publisher>Association for Computational Linguistics</publisher> <place> <placeTerm type="text">Brussels, Belgium</placeTerm> </place> </originInfo> <genre authority="marcgt">conference publication</genre> </relatedItem> <abstract>In this paper, we attempt to link the inner workings of a neural language model to linguistic theory, focusing on a complex phenomenon well discussed in formal linguistics: (negative) polarity items. We briefly discuss the leading hypotheses about the licensing contexts that allow negative polarity items and evaluate to what extent a neural language model has the ability to correctly process a subset of such constructions. We show that the model finds a relation between the licensing context and the negative polarity item and appears to be aware of the scope of this context, which we extract from a parse tree of the sentence. With this research, we hope to pave the way for other studies linking formal linguistics to deep learning.</abstract> <identifier type="citekey">jumelet-hupkes-2018-language</identifier> <identifier type="doi">10.18653/v1/W18-5424</identifier> <location> <url>https://aclanthology.org/W18-5424/</url> </location> <part> <date>2018-11</date> <extent unit="page"> <start>222</start> <end>231</end> </extent> </part></mods></modsCollection>
%0 Conference Proceedings%T Do Language Models Understand Anything? On the Ability of LSTMs to Understand Negative Polarity Items%A Jumelet, Jaap%A Hupkes, Dieuwke%Y Linzen, Tal%Y Chrupała, Grzegorz%Y Alishahi, Afra%S Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP%D 2018%8 November%I Association for Computational Linguistics%C Brussels, Belgium%F jumelet-hupkes-2018-language%X In this paper, we attempt to link the inner workings of a neural language model to linguistic theory, focusing on a complex phenomenon well discussed in formal linguistics: (negative) polarity items. We briefly discuss the leading hypotheses about the licensing contexts that allow negative polarity items and evaluate to what extent a neural language model has the ability to correctly process a subset of such constructions. We show that the model finds a relation between the licensing context and the negative polarity item and appears to be aware of the scope of this context, which we extract from a parse tree of the sentence. With this research, we hope to pave the way for other studies linking formal linguistics to deep learning.%R 10.18653/v1/W18-5424%U https://aclanthology.org/W18-5424/%U https://doi.org/10.18653/v1/W18-5424%P 222-231
[Do Language Models Understand Anything? On the Ability of LSTMs to Understand Negative Polarity Items](https://aclanthology.org/W18-5424/) (Jumelet & Hupkes, EMNLP 2018)