Optimal Graph Transformer Viterbi knowledge inference network for more successful visual navigation. (January 2023)

• Record Type: Journal Article
• Authors: Zhou, Kang; Guo, Chi; Zhang, Huyin; Yang, Bohan
• Is Part Of: Advanced engineering informatics. Volume 55(2023)
• Publication Date: 2023-01
• Subjects: Visual navigation -- Knowledge graph -- Reinforcement learning -- Graph transformer network -- Knowledge inference; Computer-aided engineering -- Periodicals; Engineering -- Data processing -- Periodicals; 620.00285
• DOI: 10.1016/j.aei.2023.101889
• Languages: English
• ISSNs: 1474-0346

Abstract

Abstract: Visual navigation recently incorporates priors in reinforcement learning (RL), which endows agent's associative ability in searching object task and has made promising results. However, there is the problem of poor exploration in end-to-end learning to new scenes. Besides, most of the priors only serve as the spatial features of scene layout without fully and directly providing guidance for RL's policy, thus making the navigation inefficient. To address these issues, we design a Graph Transformer Viterbi inference network (GTV), which finds new object relations and explores potential graph-based optimal actions in policy. Our method decomposes the RL's action to learn the adaptive priors in state representations and GTV based actions in policy, which generates a dense end-to-end learning signal and new graph edges in novel scenes. This, in turn, enables object-centric relational RL agents to learn policies faster and improve generalization ability. The results demonstrate our framework outperforms the baseline by 25.85% relatively on SPL (Success weighted by Path Length) and 30.64% on success rate, which also requires only 1 / 5 number of iterations to converge, compared with the baselines. Our code will be made publicly available in the scientific community.