MicroNets: A multi-phase pruning pipeline to deep ensemble learning in IoT devices. (December 2021)

• Record Type: Journal Article
• Authors: Alhalabi, Besher; Gaber, Mohamed Medhat; Basura, Shadi
• Is Part Of: Computers & electrical engineering. Volume 96:Part B(2021)
• Publication Date: 2021-12
• Subjects: IoT -- Edge devices -- Resource-limited environments -- Deep neural networks -- Edge-ai -- Quantisation -- Weight pruning -- Ensemble pruning -- Ensemble learning; Computer engineering -- Periodicals; Electrical engineering -- Periodicals; Electrical engineering -- Data processing -- Periodicals; Ordinateurs -- Conception et construction -- Périodiques; Électrotechnique -- Périodiques; Électrotechnique -- Informatique -- Périodiques; Computer engineering; Electrical engineering; Electrical engineering -- Data processing; Periodicals; Electronic journals; 621.302854
• DOI: 10.1016/j.compeleceng.2021.107581
• Languages: English
• ISSNs: 0045-7906

Abstract

Abstract: With the proliferation of Internet of Things (IoT) devices and the launch of 5G networks, there are more than 8 billion IoT devices worldwide generating sheer amounts of data causing a throughput burden on the global network. In such an environment, Deep Neural Networks (DNNs) are pushed towards the network's edge to unleash the power of the data on edge devices. However, there are two main challenges: (1) DNNs are computationally expensive to run on resource-constrained IoT devices, and (2) IoT typically generate noisy data due to the surrounding environment causing DNN overfitting as it learns the noise along with the underlying patterns in data. To tackle this, we propose MicroNets, a multi-phase pruning framework to enable deep ensemble learning on edge devices. The experiments on Raspberry PIs show that MicroNets generates lightweight models, and utilise ensemble learning to outperform the predictability levels of a ResNet, CIFAR10CNN baseline models (up to 7%). Graphical abstract: Highlights: A novel framework to enable deep ensemble learning on edge devices; A multi-phase pruning pipeline to generate light-weight deep learning models; A new pruning technique to reduce the number of models in deep learning ensembles.