ARL: An adaptive reinforcement learning framework for complex question answering over knowledge base. Issue 3 (May 2022)

• Record Type: Journal Article
• Authors: Zhang, Qixuan; Weng, Xinyi; Zhou, Guangyou; Zhang, Yi; Huang, Jimmy Xiangji
• Is Part Of: Information processing & management. Volume 59:Issue 3(2022)
• Publication Date: 2022-05
• Subjects: Question answering -- Knowledge base -- Text mining -- Reinforcement learning; Information storage and retrieval systems -- Periodicals; Information science -- Periodicals; Systèmes d'information -- Périodiques; Sciences de l'information -- Périodiques; Information science; Information storage and retrieval systems; Periodicals; 658.4038
• DOI: 10.1016/j.ipm.2022.102933
• Languages: English
• ISSNs: 0306-4573

Abstract

Abstract: Recently, reinforcement learning (RL)-based methods have achieved remarkable progress in both effectiveness and interpretability for complex question answering over knowledge base (KBQA). However, existing RL-based methods share a common limitation: the agent is usually misled by aimless exploration, as well as sparse and delayed rewards, leading to a large number of spurious relation paths. To address this issue, a new adaptive reinforcement learning (ARL) framework is proposed to learn a better and interpretable model for complex KBQA. First, instead of using a random walk agent, an adaptive path generator is developed with three atomic operations to sequentially generate the relation paths until the agent reaches the target entity. Second, a semantic policy network is presented with both character-level and sentence-level information to better guide the agent. Finally, a new reward function is introduced by considering both the relation paths and the target entity to alleviate sparse and delayed rewards. The empirical results on five benchmark datasets show that our model is more effective than state-of-the-art approaches. Compared with the strong baseline model SRN, the proposed model achieves performance improvements of 23.7% on MetaQA-3 using the metric Hits@1. Highlights: We propose a new adaptive reinforcement learning (ARL) framework to adaptively extend the relation paths. We propose a semantic policy network to choose the optimal actions. We introduce a new reward function, with the aim of alleviating the issue of delayed and sparse rewards.