A Bayesian end-to-end model with estimated uncertainties for simple question answering over knowledge bases. (March 2021)

• Record Type: Journal Article
• Authors: Zhang, Linhai; Lin, Chao; Zhou, Deyu; He, Yulan; Zhang, Meng
• Is Part Of: Computer speech & language. Volume 66(2021)
• Publication Date: 2021-03
• Subjects: Question answering over knowledge bases -- Bayesian neural network -- Uncertainty estimation; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2020.101167
• Languages: English
• ISSNs: 0885-2308

Abstract

Highlights: To our best knowledge, our proposed Bayesian end-to-end model is the first attempt to explore Bayesian neural network (BNN) in the knowledge base-based question answering (KBQA) and uncertainty estimation for KBQA task is firstly considered. Neural network weights are transformed into distributions and uncertainties could be further quantified by sampling weights and forward inputs through the network multiple times. The proposed end-to-end framework could avoid uncertainty propagation problem in the multi-staged approaches and we propose a novel uncertainty estimation measure. The empirical results show that the proposed model achieves better performance for KBQA compared to the existing state-of-the-art end-to-end approaches and demonstrate the effectiveness of the proposed uncertainty estimation measure. Abstract: Existing methods for question answering over knowledge bases (KBQA) ignore the consideration of the model prediction uncertainties. We argue that estimating such uncertainties is crucial for the reliability and interpretability of KBQA systems. Therefore, we propose a novel end-to-end KBQA model based on Bayesian Neural Network (BNN) to estimate uncertainties arose from both model and data. To our best knowledge, we are the first to consider the uncertainty estimation problem for the KBQA task using BNN. The proposed end-to-end model integrates entity detection and relation prediction into a unified framework, and employs BNN to model entity and relation under the given question semantics, transforming network weights into distributions. Therefore, predictive distributions can be estimated by sampling weights and forward inputs through the network multiple times. Uncertainties can be further quantified by calculating the variances of predictive distributions. The experimental results demonstrate the effectiveness of uncertainties in both the misclassification detection task and cause of error detection task. Furthermore, the proposed model also achieves comparable performance compared to the existing state-of-the-art approaches on SimpleQuestions dataset.