Gain parameter and dropout-based fine tuning of deep networks. (2018)

• Record Type: Journal Article
• Authors: Wani, M. Arif; Afzal, Saduf
• Is Part Of: International journal of intelligent information and database systems. Volume 11:Number 4(2018)
• Page Start: 236
• Page End: 254
• Publication Date: 2018
• Subjects: deep learning -- deep neural networks -- fine tuning -- gain parameter -- dropout technique; Database management -- Computer programs -- Periodicals; Information retrieval -- Computer programs -- Periodicals; Information storage and retrieval systems -- Computer programs -- Periodicals; Artificial intelligence -- Periodicals; Expert systems (Computer science) -- Periodicals; Intelligent agents (Computer software) -- Periodicals; 006.33
• Languages: English
• ISSNs: 1751-5858

Abstract

Training of deep neural networks can involve two phases: unsupervised pre-training and supervised fine tuning. Unsupervised pre-training is used to learn the initial parameter values of deep networks, while as supervised fine tuning improves upon what has been learned in the pre-training stage. Backpropagation algorithm can be used for supervised fine tuning of deep neural networks. In this paper we evaluate the use of backpropagation with gain parameter algorithm for fine tuning of deep networks. We further propose a modification where backpropagation with gain parameter algorithm is integrated with the dropout technique and evaluate its performance in fine tuning of deep networks. The effectiveness of fine tuning done by proposed technique is also compared with other variants of backpropagation algorithm on benchmark datasets. The experimental results show that the fine tuning of deep networks using the proposed technique yields promising results among all the studied methods on the tested datasets.