Sequential multi‐headed attention for entity‐based relational neural networks. Issue 5 (10th February 2022)

• Record Type: Journal Article
• Authors: Moges, Dagmawi; Zhao, Jiaxu; Qu, Hong
• Other Names: Sharma Rohit guestEditor.; Gupta Deepak guestEditor.; Maseleno Andino guestEditor.; Peng Sheng‐Lung guestEditor.; Menon Varun G. guestEditor.; Khosravi Reza guestEditor.; Jolfaei Alireza guestEditor.; Kumar Akshi guestEditor.; P Vinod guestEditor.
• Is Part Of: Expert systems. Volume 39:Issue 5(2022)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2022-02-10
• Subjects: artificial intelligence -- computational models for AI -- natural language processing -- relational learning over aggregated representations; Expert systems (Computer science); 006.33
• DOI: 10.1111/exsy.12945
• Languages: English
• ISSNs: 0266-4720

Abstract

Abstract: The capacity of relational interaction between high‐level information and reason is the defining characteristic of human intelligence. Regardless of the remarkable progress in artificial intelligence, recent machine reading comprehension models still heavily rely on high‐dimensional word‐based distributed representations. Since these models employ statistical means to answer questions of complex textual corpus and employ an accuracy‐based metric system, their learning capacity of the required skills is not guaranteed. To ensure the capacity of MRC models to learn the desired skills, explainability has become an emerging requirement. In this paper, we propose an end‐to‐end natural language reasoning model that is based on sets of high‐level aggregated representations which promote operational explainability. To this end, sequential multi‐head attention, and a loss regularization function is proposed. We show analysis of the proposed approach on two natural language reasoning oriented question and answering datasets (bAbI and NewsQA).