A noise-based stabilizer for convolutional neural networks. Issue 11 (24th July 2019)

• Record Type: Journal Article
• Authors: Geete, Kanu; Pandey, Manish
• Is Part Of: Journal of statistical computation and simulation. Volume 89:Issue 11(2019)
• Page Start: 2102
• Page End: 2120
• Publication Date: 2019-07-24
• Subjects: Deep neural networks -- convolutional neural networks -- regularization -- stabilizer; 68 Computer Science; Mathematical statistics -- Data processing -- Periodicals; Digital computer simulation -- Periodicals; 519.5028505
• DOI: 10.1080/00949655.2019.1610883
• Languages: English
• ISSNs: 0094-9655

Abstract

ABSTRACT: Overfitting occurs when one tries to train a large model on small amount of data. Regularizing a neural network using prior knowledge remains a topic of research as it is not concluded how much prior information can be given to the neural network. In this paper, a novel algorithm is introduced which uses regularization to train a neural network without increasing the dataset. A trivial prior information of a class label is supplied to the model while training. Laplace noise is introduced to the intermediate layer for more generalization. The results show significant improvement in accuracy on the standard datasets for a simple Convolutional Neural Network (CNN). While the proposed method outperforms previous regularization techniques like dropout and batch normalization, it can also be applied with them for further improvement in the performance. On the variants of MNIST, proposed algorithm achieved an average 48% increment in the test accuracy.