Node classification using kernel propagation in graph neural networks. (15th July 2021)

• Record Type: Journal Article
• Authors: Arul Prakash, Sakthi Kumar; Tucker, Conrad S.
• Is Part Of: Expert systems with applications. Volume 174(2021)
• Publication Date: 2021-07-15
• Subjects: Deep learning -- Node classification -- Network embedding -- Graph neural networks -- Attention; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2021.114655
• Languages: English
• ISSNs: 0957-4174

Abstract

Highlights: Spectral Kernel propagation layer for differentiating local/global connectivity. Multiplicative attention mechanism that improves the stability of learning. Node classification without the use of additional node attributes/features. Abstract: In this work, we introduce a kernel propagation method that enables graph neural networks (GNNs) to leverage higher-order network structural information without increasing the complexity of the networks. Recent studies have introduced GNNs that include higher-order neighborhood features containing global network information by propagating node features using a higher-order feature propagation rule. Though these GNNs have shown to improve node classification performance, they fail to include local connectivity information. Alternatively, GNNs also concatenate increasing orders of adjacency matrix in deeper layers in order to include higher-order structural information. In addition to global network information, GNNs also make use of node features which are network and node dependent features that serve to distinguish structurally isomorphic sub-structures within graphs. However, such node features may not always be available or depending on the network, may lead to deteriorating classification performance. Hence, to resolve these limitations, we propose a kernel propagation method that introduces a pre-processing step for GNNs to leverage higher-order structural features. The higher-order structural features are computed using a weighted random walk matrix which is node independent while using the first-order spectral propagation rule which explicitly considers local connectivity. Through our benchmark experiments, we find that the computed higher-order structural features are capable of replacing node dependent features while performing node classification task with performance on par with the state of the art approaches. Further, we also find that including both node features and higher-order structural features increases the performance of GNNs on large scale benchmark networks considered in this study. Our results show that considering local and global structural information as input to GNNs lead to an improvement in node classification performance in the absence/presence of node features without loss of performance.