Random vector functional link neural network based ensemble deep learning. (September 2021)

• Record Type: Journal Article
• Authors: Shi, Qiushi; Katuwal, Rakesh; Suganthan, P.N.; Tanveer, M.
• Is Part Of: Pattern recognition. Volume 117(2021)
• Publication Date: 2021-09
• Subjects: Random Vector Functional Link (RVFL) -- Deep RVFL -- Multi-layer RVFL -- Ensemble deep learning -- Randomized neural network -- Extreme learning machine (ELM); Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2021.107978
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: Inspired by the principles of Random Vector Functional Link (RVFL) network, we propose a deep RVFL network (dRVFL) with rich feature extraction capabilities through several hidden layers. We also propose an ensemble deep network (edRVFL) based on a single dRVFL network. We demonstrate the generic nature of the proposed methods by integrating them with a recent RVFL variant called sparse-pretrained RVFL (SP-RVFL). Experiments on 46 tabular UCI classification datasets demonstrate that the proposed ensemble deep RVFL networks outperform state-of-the-art deep feed-forward neural networks. Experiments on 12 sparse classification datasets demonstrate that the proposed ensemble deep SP-RVFL networks outperform the best. Abstract: In this paper, we propose deep learning frameworks based on the randomized neural network. Inspired by the principles of Random Vector Functional Link (RVFL) network, we present a deep RVFL network (dRVFL) with stacked layers. The parameters of the hidden layers of the dRVFL are randomly generated within a suitable range and kept fixed while the output weights are computed using the closed-form solution as in a standard RVFL network. We also propose an ensemble deep network (edRVFL) that can be regarded as a marriage of ensemble learning with deep learning. Unlike traditional ensembling approaches that require training several models independently from scratch, edRVFL is obtained by training a single dRVFL network once. Both dRVFL and edRVFL frameworks are generic and can be used with any RVFL variant. To illustrate this, we integrate the deep learning RVFL networks with a recently proposed sparse pre-trained RVFL (SP-RVFL). Experiments on 46 tabular UCI classification datasets and 12 sparse datasets demonstrate that the proposed deep RVFL networks outperform state-of-the-art deep feed-forward neural networks (FNNs).