A hybrid regularization approach for random vector functional-link networks. (February 2020)

• Record Type: Journal Article
• Authors: Ye, Hailiang; Cao, Feilong; Wang, Dianhui
• Is Part Of: Expert systems with applications. Volume 140(2020)
• Publication Date: 2020-02
• Subjects: Neural networks -- Random vector functional-link networks (RVFL) -- Regularization -- Sparsity -- Stability; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2019.112912
• Languages: English
• ISSNs: 0957-4174

Abstract

Highlights: Propose a new method for solving the hybrid regularization model. An efficient hybrid regularization algorithm is developed. Algorithms convergence, sparsity, and stability are proved theoretically. Some efficient comparisons experiments are studied. Abstract: Neural networks have been widely applied to expert and intelligent systems in the fields of business, fault diagnosis, and forecasting. Especially, random vector functional-link networks (RVFL), an important structure, have also been considerably studied and used in recent years. This paper addresses in the investigation of regularization model for RVFL, and proposes a hybrid regularization approach for RVFL by adding the ℓ2 and ℓ1 norm penalty terms simultaneously. A novel iterative learning algorithm is developed by using a fixed point contractive map. Further, some theoretical properties, including the convergence, sparsity, and stability of the proposed algorithm, are discussed and analyzed concretely under reasonable assumptions. The proposed method greatly improves the learner's sparsity and stability, guaranteeing the feasibility and effectiveness of network training. Experimental results on some benchmarks as well as face recognition database collected from the expert and intelligent systems, particularly the use of the statistical analysis strategy, verify the effectiveness and superiority of the proposed method, i.e. this new algorithm systematically outperforms the original RVFL and its variants in terms of the accuracy, sparsity, and stability of the solution.