Pruning Neural Networks Using Multi-Armed Bandits. (26th September 2019)

• Record Type: Journal Article
• Authors: Ameen, Salem; Vadera, Sunil
• Is Part Of: Computer journal. Volume 63:Number 7(2020)
• Page Start: 1099
• Page End: 1108
• Publication Date: 2019-09-26
• Subjects: neural networks -- multi-armed bandits -- pruning weights; Computers -- Periodicals; 005.1
• DOI: 10.1093/comjnl/bxz078
• Languages: English
• ISSNs: 0010-4620

Abstract

Abstract: The successful application of deep learning has led to increasing expectations of their use in embedded systems. This, in turn, has created the need to find ways of reducing the size of neural networks. Decreasing the size of a neural network requires deciding which weights should be removed without compromising accuracy, which is analogous to the kind of problems addressed by multi-armed bandits (MABs). Hence, this paper explores the use of MABs for reducing the number of parameters of a neural network. Different MAB algorithms, namely $\epsilon $ -greedy, win-stay, lose-shift, UCB1, KL-UCB, BayesUCB, UGapEb, successive rejects and Thompson sampling are evaluated and their performance compared to existing approaches. The results show that MAB pruning methods, especially those based on UCB, outperform other pruning methods.