A sparsity-based stochastic pooling mechanism for deep convolutional neural networks. (September 2018)

• Record Type: Journal Article
• Authors: Song, Zhenhua; Liu, Yan; Song, Rong; Chen, Zhenguang; Yang, Jianyong; Zhang, Chao; Jiang, Qing
• Is Part Of: Neural networks. Volume 105(2018)
• Page Start: 340
• Page End: 345
• Publication Date: 2018-09
• Subjects: Deep learning -- Pooling mechanism -- Degree of sparsity -- Representative feature value -- Recognition accuracy; Neural computers -- Periodicals; Neural networks (Computer science) -- Periodicals; Neural networks (Neurobiology) -- Periodicals; Nervous System -- Periodicals; Ordinateurs neuronaux -- Périodiques; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux neuronaux (Neurobiologie) -- Périodiques; Neural computers; Neural networks (Computer science); Neural networks (Neurobiology); Periodicals; 006.32
• DOI: 10.1016/j.neunet.2018.05.015
• Languages: English
• ISSNs: 0893-6080

Abstract

Abstract: A novel sparsity-based stochastic pooling which integrates the advantages of max-pooling, average-pooling and stochastic pooling is introduced. The proposed pooling is designed to balance the advantages and disadvantages of max-pooling and average-pooling by using the degree of sparsity of activations and a control function to obtain an optimized representative feature value ranging from average value to maximum value of a pooling region. The optimized representative feature value is employed for probability weights assignment of activations in normal distribution. The proposed pooling also adopts weighted random sampling with a reservoir for the sampling process to preserve the advantages of stochastic pooling. This proposed pooling is evaluated on several standard datasets in deep learning framework to compare with various classic pooling methods. Experimental results show that it has good performance on improving recognition accuracy. The influence of changes to the feature parameter on recognition accuracy is also investigated.