Neural network with multiple connection weights. (November 2020)

• Record Type: Journal Article
• Authors: Zhang, Jiangshe; Hu, Junying; Liu, Junmin
• Is Part Of: Pattern recognition. Volume 107(2020:Nov.)
• Publication Date: 2020-11
• Subjects: Neural network -- Neurotransmitter -- Interpretability -- Extending dimension; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2020.107481
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: Biological studies have shown that one neuron simultaneously releases different types of neurotransmitters to another neuron for information transfer, of which different types of neurotransmitters play different roles. Motivated by this biological discovery, a novel neural networks with Multiple Connection Weights (MNN) is proposed. The greedy layer-by-layer pre-training method is proposed to pre-train MNN model. The universal approximation of MNN has been proved. The experimental results on MNIST, NORB and seven UCI datasets have been done and show that the performance of MNN outperformed the traditional neural network. The dimension expansion of weights provide new insights into structure design of the neural network model. Abstract: Biological studies have shown that the interaction between neurons are based on neurotransmitters, which transmit signals between neurons, and that one neuron sends information to another neuron by releasing a number of different neurotransmitters, which play different roles. Motivated by this biological discovery, a novel neural networks model is proposed by extending the dimension of connection weights from one to multiple, i.e. there are multiple not only one connections between each two units. The number of dimensions of connection weight represents the number of categories of neurotransmitters and different components of the weight correspond to different neurotransmitters. In order to make these neurotransmitters collaborate and compete appropriately, the input and output for each unit in our proposed model have been heuristically defined. From the biological perspective, the proposed neural network is much closer to biological neural network. From the viewpoint of new model structure, the characteristic that the activation of each hidden unit is based on several filters, can improve the interpretability of features learned by the proposed neural network. Experimental results on MNIST, NORB and several other data sets have demonstrated that the performances of traditional neural networks can be improved by extending the dimension of connection weight between units, and the idea of multiple connection weights provides a new paradigm for the design of neural networks.