Every node counts: Self-ensembling graph convolutional networks for semi-supervised learning. (October 2020)

• Record Type: Journal Article
• Authors: Luo, Yawei; Ji, Rongrong; Guan, Tao; Yu, Junqing; Liu, Ping; Yang, Yi
• Is Part Of: Pattern recognition. Volume 106(2020:Oct.)
• Publication Date: 2020-10
• Subjects: Teacher-student models -- Self-ensemble learning -- Graph convolutional networks -- Semi-supervised learning; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2020.107451
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: To the best of our knowledge, our method (SEGCN) is the first to introduce a teacher-student ensemble strategy in the Graph Convolutional Networks (GCNs) design. By proposing to combine the ensemble model with classic GCNs, we em-phasize the importance of exploitation of unlabeled nodes in graph-structured data classification in the context of semi-supervised learning. Analogy to the noise added to student model in regular data, we successfully design new perturbation strategies for the graph-based student model. We make the t-SNE analysis in latent feature space and observe that SEGCN can generate better embedding representations in latent feature space thus leading to better classification accuracy. Our results are on par with the state-of-the-art methods on four node classification benchmarks in terms of accuracy, i.e. Citeseer (69.9% → 73.4%), Core (80.4% → 83.5%), Pubmed (78.6% → 78.9%) and NELL(67.8% → 73.5%). Abstract: Graph convolutional network (GCN) provides a powerful means for graph-based semi-supervised tasks. However, as a localized first-order approximation of spectral graph convolution, the classic GCN can not take full advantage of unlabeled data, especially when the unlabeled node is far from labeled ones. To capitalize on the information from unlabeled nodes to boost the training for GCN, we propose a novel framework named Self-Ensembling GCN (SEGCN), which marries GCN with Mean Teacher – a powerful self-ensemble learning mechanism for semi-supervised task. SEGCN contains a student model and a teacher model. As a student, it not only learns to correctly classify the labeled nodes, but also tries to be consistent with the teacher on unlabeled nodes in more challenging situations, such as a high dropout rate and graph corrosion. As a teacher, it averages the student model weights and generates more accurate predictions to lead the student. In such a mutual-promoting process, both labeled and unlabeled samples can be fully utilized for backpropagating effective gradients to train GCN. In a variety of semi-supervised classification benchmarks, i.e. Citeseer, Cora, Pubmed and NELL, we validate that the proposed method matches the state of the arts in the classification accuracy. The code is publicly available at https://github.com/RoyalVane/SEGCN .