A weight perturbation-based regularisation technique for convolutional neural networks and the application in medical imaging. (1st July 2020)

• Record Type: Journal Article
• Authors: Khatami, Amin; Nazari, Asef; Khosravi, Abbas; Lim, Chee Peng; Nahavandi, Saeid
• Is Part Of: Expert systems with applications. Volume 149(2020)
• Publication Date: 2020-07-01
• Subjects: Convolutional Neural Network -- Regularisation -- Generalisation -- Weight perturbation; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2020.113196
• Languages: English
• ISSNs: 0957-4174

Abstract

Highlights: A regularisation model based on noise perturbation for convolutional neural networks. Accelerated convergence speed, circumvent over-fitting, and improved generalisation. Applying additive noise to earlier convolutional layers achieves better performance. Abstract: A convolutional neural network has the capacity to learn multiple representation levels and abstraction in order to provide a better understanding of image data. In addition, a good multi-level representation of data typically results in a better generalisation capability. This fact emphasises the importance of concentrating on the regularity information of training data in order to improve generalisation. However, the training data contain erroneous information owing to noise and outliers. In this paper, we propose a new regularisation approach for convolutional neural networks with better generalisation properties. Specifically, the weights of the convolution layers are perturbed by additive noise in each learning iteration. The approach provides a better model for prediction, as shown by the experimental results on a number of medical benchmark data sets. Furthermore, the effectiveness and accuracy of the proposed convolutional neural network are demonstrated by comparing with several recent perturbation techniques.