Analyzing and interpreting neural networks for NLP: A report on the first BlackboxNLP workshop. (July 2019)

• Record Type: Journal Article
• Authors: Alishahi, Afra; Chrupała, Grzegorz; Linzen, Tal
• Is Part Of: Natural language engineering. Volume 25:Part 4(2019)
• Page Start: 543
• Page End: 557
• Publication Date: 2019-07
• Subjects: neural networks, -- interpretability, -- natural language processing; Natural language processing (Computer science) -- Periodicals; Software engineering -- Periodicals; 006.35
• DOI: 10.1017/S135132491900024X
• Languages: English
• ISSNs: 1351-3249

Abstract

Abstract: The Empirical Methods in Natural Language Processing (EMNLP) 2018 workshop BlackboxNLP was dedicated to resources and techniques specifically developed for analyzing and understanding the inner-workings and representations acquired by neural models of language. Approaches included: systematic manipulation of input to neural networks and investigating the impact on their performance, testing whether interpretable knowledge can be decoded from intermediate representations acquired by neural networks, proposing modifications to neural network architectures to make their knowledge state or generated output more explainable, and examining the performance of networks on simplified or formal languages. Here we review a number of representative studies in each category.