Reinforcement learning with dynamic completion for answering multi-hop questions over incomplete knowledge graph. Issue 3 (May 2023)

• Record Type: Journal Article
• Authors: Cui, Hai; Peng, Tao; Han, Ridong; Zhu, Beibei; Bi, Haijia; Liu, Lu
• Is Part Of: Information processing & management. Volume 60:Issue 3(2023)
• Publication Date: 2023-05
• Subjects: Knowledge graph -- Question answering -- Reinforcement learning -- Path-based reasoning -- Dynamic completion; Information storage and retrieval systems -- Periodicals; Information science -- Periodicals; Systèmes d'information -- Périodiques; Sciences de l'information -- Périodiques; Information science; Information storage and retrieval systems; Periodicals; 658.4038
• DOI: 10.1016/j.ipm.2023.103283
• Languages: English
• ISSNs: 0306-4573

Abstract

Abstract: Text-enhanced and implicit reasoning methods are proposed for answering questions over incomplete knowledge graph (KG), whereas prior studies either rely on external resources or lack necessary interpretability. This article desires to extend the line of reinforcement learning (RL) methods for better interpretability and dynamically augment original KG action space with additional actions. To this end, we propose a RL framework along with a dynamic completion mechanism, namely D ynamic C ompletion R easoning N etwork (DCRN). DCRN consists of an action space completion module and a policy network. The action space completion module exploits three sub-modules (relation selector, relation pruner and tail entity predictor) to enrich options for decision making. The policy network calculates probability distribution over joint action space and selects promising next-step actions. Simultaneously, we employ the beam search-based action selection strategy to alleviate delayed and sparse rewards. Extensive experiments conducted on WebQSP, CWQ and MetaQA demonstrate the effectiveness of DCRN. Specifically, under 50% KG setting, the Hits@1 performance improvements of DCRN on MetaQA-1H and MetaQA-3H are 2.94% and 1.18% respectively. Moreover, under 30% and 10% KG settings, DCRN prevails over all baselines by 0.9% and 1.5% on WebQSP, indicating the robustness to sparse KGs. Highlights: The sparse KG is dynamically augmented with additional unseen actions. This paper extends the line of RL-based methods for better interpretability. The process of action space completion is separated into two sub-steps. The relation pruner filters out noisy and inappropriate candidate relations. The beam search-based action selection strategy maintains multiple partial paths.