A unified deep sparse graph attention network for scene graph generation. (March 2022)

• Record Type: Journal Article
• Authors: Zhou, Hao; Yang, Yazhou; Luo, Tingjin; Zhang, Jun; Li, Shuohao
• Is Part Of: Pattern recognition. Volume 123(2022)
• Publication Date: 2022-03
• Subjects: Scene graph generation -- Statistical co-occurrence knowledge -- Relationship measurement network -- Graph attention network -- Sparse graph; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2021.108367
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: Propose an efficient framework of feature interaction and knowledge learning for SGG. Construct sparse graph by classifying dense edges into foreground and background. Jointly learn object and relationship features via graphical message passing in GAT. Explore the prior information of statistical probability and build sparse knowledge graph. Our method outperforms several state-of-the-art methods in VG datasets. Abstract: Scene graph generation (SGG) plays an important role in deep understanding of the visual scene. Despite the empirical success of traditional methods in many applications, they still have several challenges in the high computational complexity of dense graph and the inaccurate pruning of sparse graph. To tackle these problems, we propose a novel deep sparse graph attention network to mine the rich contextual clues and simultaneously preserve the statistical co-occurrence knowledge of SGG. Specifically, our Relationship Measurement Network (RelMN) is adapted to first classify all object pairs in dense graph as the foreground and background categories to filter the false relationships and then construct a sparse graph efficiently. Meanwhile, we design a novel feature aggregation and update method via graphical message passing to jointly learn the node and edge features for object recognition and relationship classification in the graph attention network. Extensive experimental results on the large scale VG and VRD datasets demonstrate our proposed method outperforms several state-of-the-art approaches.