Talking-heads attention-based knowledge representation for link prediction. (July 2022)

• Record Type: Journal Article
• Authors: Wang, Shirui; Zhou, Wen'an; Zhou, Qiang
• Is Part Of: Computer speech & language. Volume 74(2022)
• Publication Date: 2022-07
• Subjects: Knowledge representation -- Link prediction -- Talking-heads attention; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2021.101340
• Languages: English
• ISSNs: 0885-2308

Abstract

Abstract: State-of-the-art methods for link prediction, also known as knowledge graph embedding, aim to represent both entities and relations in the given knowledge graphs (KGs) into a continuous low-dimensional vector space and thus could be used to fill the missing facts or identify the spurious facts in KGs, where a fact is represented as a triple in the form of (head entity, relation, tail entity). Most previous attempts solely learn triples independently and thus fail to utilize the rich hidden inference and semantic information in the local neighbourhood existed surrounding each triple in KGs. To this effect, this paper proposes a talking-heads attention-based knowledge representation method, a novel graph attention networks-based method for link prediction which learns the knowledge graph embedding with talking-heads attention guidance from multi-hop neighbourhood triples. We evaluate our model in Freebase, WordNet and Kinship datasets on link prediction, experimental results demonstrate that the injection of talking-heads attention mechanism could better capture the semantic relationship of the neighbourhood surrounding triples and indeed achieve promising performance on link prediction.