An interpretive constrained linear model for ResNet and MgNet. (May 2023)

• Record Type: Journal Article
• Authors: He, Juncai; Xu, Jinchao; Zhang, Lian; Zhu, Jianqing
• Is Part Of: Neural networks. Volume 162(2023)
• Page Start: 384
• Page End: 392
• Publication Date: 2023-05
• Subjects: Convolutional neural networks -- Data-feature mapping -- Multigrid iterative methods -- ResNet -- MgNet; Neural computers -- Periodicals; Neural networks (Computer science) -- Periodicals; Neural networks (Neurobiology) -- Periodicals; Nervous System -- Periodicals; Ordinateurs neuronaux -- Périodiques; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux neuronaux (Neurobiologie) -- Périodiques; Neural computers; Neural networks (Computer science); Neural networks (Neurobiology); Periodicals; 006.32
• DOI: 10.1016/j.neunet.2023.03.011
• Languages: English
• ISSNs: 0893-6080

Abstract

Abstract: We propose a constrained linear data-feature-mapping model as an interpretable mathematical model for image classification using a convolutional neural network (CNN). From this viewpoint, we establish detailed connections between the traditional iterative schemes for linear systems and the architectures of the basic blocks of ResNet- and MgNet-type models. Using these connections, we present some modified ResNet models that, compared with the original models, have fewer parameters but can produce more accurate results, thereby demonstrating the validity of this constrained learning data-feature-mapping assumption. Based on this assumption, we further propose a general data-feature iterative scheme to demonstrate the rationality of MgNet. We also provide a systematic numerical study on MgNet to show its success and advantages in image classification problems, particularly in comparison with established networks.