Boosting question answering over knowledge graph with reward integration and policy evaluation under weak supervision. Issue 2 (March 2023)

• Record Type: Journal Article
• Authors: Bi, Xin; Nie, Haojie; Zhang, Guoliang; Hu, Lei; Ma, Yuliang; Zhao, Xiangguo; Yuan, Ye; Wang, Guoren
• Is Part Of: Information processing & management. Volume 60:Issue 2(2023)
• Publication Date: 2023-03
• Subjects: Knowledge graph-based question answering -- Multi-hop reasoning -- Weak supervision -- Augmented intelligence for decision-making; Information storage and retrieval systems -- Periodicals; Information science -- Periodicals; Systèmes d'information -- Périodiques; Sciences de l'information -- Périodiques; Information science; Information storage and retrieval systems; Periodicals; 658.4038
• DOI: 10.1016/j.ipm.2022.103242
• Languages: English
• ISSNs: 0306-4573

Abstract

Abstract: Among existing knowledge graph based question answering (KGQA) methods, relation supervision methods require labeled intermediate relations for stepwise reasoning. To avoid this enormous cost of labeling on large-scale knowledge graphs, weak supervision methods, which use only the answer entity to evaluate rewards as supervision, have been introduced. However, lacking intermediate supervision raises the issue of sparse rewards, which may result in two types of incorrect reasoning path: (1) incorrectly reasoned relations, even when the final answer entity may be correct; (2) correctly reasoned relations in a wrong order, which leads to an incorrect answer entity. To address these issues, this paper considers the multi-hop KGQA task as a Markov decision process, and proposes a model based on Reward Integration and Policy Evaluation (RIPE). In this model, an integrated reward function is designed to evaluate the reasoning process by leveraging both terminal and instant rewards. The intermediate supervision for each single reasoning hop is constructed with regard to both the fitness of the taken action and the evaluation of the unreasoned information remained in the updated question embeddings. In addition, to lead the agent to the answer entity along the correct reasoning path, an evaluation network is designed to evaluate the taken action in each hop. Extensive ablation studies and comparative experiments are conducted on four KGQA benchmark datasets. The results demonstrate that the proposed model outperforms the state-of-the-art approaches in terms of answering accuracy. Highlights: We propose a KGQA model based on Reward Integration and Policy Evaluation. We use instant reword to enhance the supervision on intermediate hops of reasoning. We use an attention activation mechanism to capture varied focuses in the question. Experiments on four datasets show the performance of the proposed method..