Accelerating convolutional neural network training using ProMoD backpropagation algorithm. Issue 13 (5th October 2020)

• Record Type: Journal Article
• Authors: Gürhanlı, Ahmet
• Is Part Of: IET image processing. Volume 14:Issue 13(2020)
• Page Start: 2957
• Page End: 2964
• Publication Date: 2020-10-05
• Subjects: gradient methods -- backpropagation -- image recognition -- learning (artificial intelligence) -- convolutional neural nets; convolutional neural network training -- ProMoD backpropagation algorithm -- convolutional neural networks -- CNN -- image recognition applications -- fast training -- image recognition systems -- crucial point -- image class -- lengthy calculations -- stable optimisation method -- momentum technique -- backpropagation algorithms -- proportional–integral controller -- automatic control systems -- training targets -- proportional + momentum + derivative method -- training process -- PI–derivative controller -- target accuracy levels -- calculated update matrices -- ProMoD method; Image processing -- Periodicals; 621.36705
• DOI: 10.1049/iet-ipr.2019.0761
• Languages: English
• ISSNs: 1751-9659

Abstract

Abstract : Convolutional neural networks (CNNs) play an important role in image recognition applications. Fast training of image recognition systems is a crucial point, because the system should be trained for each new image class. These networks are trained using lengthy calculations. Focus of engineering is on obtaining a fast, but stable optimisation method. Momentum technique which is used in backpropagation algorithms is like a proportional–integral (PI) controller that is widely employed in automatic control systems. It takes the integral of past errors and helps reaching the training targets. Proportional + momentum + derivative (ProMoD) method adds gradient of update matrices to the training process and builds an optimiser such as the widely used PI–derivative controller. The method accelerates the movement toward the target accuracy levels. This is achieved by doing bigger corrections in the beginning using the differences in the calculated update matrices. In this research, ProMoD method is tested on image recognition applications and CNNs. Modified national institute of standards and technology database (MNIST) and Fashion‐MNIST datasets are used for evaluating the performance. Experimental results showed that ProMoD might perform much faster in training of CNNs and consume proportionally less power with respect to the momentum and stochastic gradient descent (SGD) techniques.