Efficient adaptive inference for deep convolutional neural networks using hierarchical early exits. (September 2020)

• Record Type: Journal Article
• Authors: Passalis, Nikolaos; Raitoharju, Jenni; Tefas, Anastasios; Gabbouj, Moncef
• Is Part Of: Pattern recognition. Volume 105(2020:Sep.)
• Publication Date: 2020-09
• Subjects: Adaptive inference -- Early exits -- Bag-of-Features -- Deep convolutional neural networks -- Hierarchical representations; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2020.107346
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: An efficient hierarchical aggregation scheme for early exits is proposed. A classification layer reuse approach is used to further reduce the complexity. An adaptive classification approach is proposed further improving the performance. The effectiveness of the proposed method is demonstrated using four image datasets. Abstract: Early exits are capable of providing deep learning models with adaptive computational graphs that can readily adapt on-the-fly to the available resources. Despite their advantages, existing early exit methods suffer from many limitations which limit their performance, e.g., they ignore the information extracted from previous exit layers, they are unable to efficiently handle feature maps with large sizes, etc. To overcome these limitations we propose a Bag-of-Features (BoF)-based method that is capable of constructing efficient hierarchical early exit layers with minimal computational overhead, while also providing an adaptive inference method that allows for early stopping the inference process when the network is confident enough for its output, leading to significant performance benefits. To this end, the BoF model is extended and adapted to the needs of early exits by constructing additive shared histogram spaces that gradually refine the information extracted from the various layers of a network, in a hierarchical manner, while also employing a classification layer reuse strategy to further reduce the number of parameters needed per exit layer. Note that the proposed method is generic and can be readily combined with any neural network architecture. The effectiveness of the proposed method is demonstrated using five different image datasets, proving that early exits can be readily transformed into a practical tool, which can be effectively used in various real-world embedded applications.