IC neuron: An efficient unit to construct neural networks. (January 2022)

• Record Type: Journal Article
• Authors: An, Junyi; Liu, Fengshan; Shen, Furao; Zhao, Jian; Li, Ruotong; Gao, Kepan
• Is Part Of: Neural networks. Volume 145(2022)
• Page Start: 177
• Page End: 188
• Publication Date: 2022-01
• Subjects: Neural network -- Neuron model -- Elastic collision -- Inter-layer collision neuron; Neural computers -- Periodicals; Neural networks (Computer science) -- Periodicals; Neural networks (Neurobiology) -- Periodicals; Nervous System -- Periodicals; Ordinateurs neuronaux -- Périodiques; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux neuronaux (Neurobiologie) -- Périodiques; Neural computers; Neural networks (Computer science); Neural networks (Neurobiology); Periodicals; 006.32
• DOI: 10.1016/j.neunet.2021.10.005
• Languages: English
• ISSNs: 0893-6080

Abstract

Abstract: As a popular machine learning method, neural networks can be used to solve many complex tasks. Their strong generalization ability comes from the representation ability of the basic neuron models. The most popular neuron model is the McCulloch–Pitts (MP) neuron, which uses a simple transformation to process the input signal. A common trend is to use the MP neuron to design various neural networks. However, the optimization of the neuron structure is rarely considered. Inspired by the elastic collision model in physics, we propose a new neuron model that can represent more complex distributions. We term it the Inter-layer Collision (IC) neuron which divides the input space into multiple subspaces to represent different linear transformations. Through this operation, the IC neuron enhances the non-linear representation ability and emphasizes useful input features for a given task. We build the IC networks by integrating the IC neurons into the fully-connected, the convolutional, and the recurrent structures. The IC networks outperform the traditional neural networks in a wide range of tasks. Besides, we combine the IC neuron with deep learning models and show the superiority of the IC neuron. Our research proves that the IC neuron can be an effective basic unit to build network structures and make the network performance better.