A new learning paradigm for random vector functional-link network: RVFL+. (February 2020)

• Record Type: Journal Article
• Authors: Zhang, Peng-Bo; Yang, Zhi-Xin
• Is Part Of: Neural networks. Volume 122(2020)
• Page Start: 94
• Page End: 105
• Publication Date: 2020-02
• Subjects: RVFL+ -- KRVFL+ -- Learning using privileged information -- The Rademacher complexity -- SVM+ -- Random vector functional link networks; Neural computers -- Periodicals; Neural networks (Computer science) -- Periodicals; Neural networks (Neurobiology) -- Periodicals; Nervous System -- Periodicals; Ordinateurs neuronaux -- Périodiques; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux neuronaux (Neurobiologie) -- Périodiques; Neural computers; Neural networks (Computer science); Neural networks (Neurobiology); Periodicals; 006.32
• DOI: 10.1016/j.neunet.2019.09.039
• Languages: English
• ISSNs: 0893-6080

Abstract

Abstract: In school, a teacher plays an important role in various classroom teaching patterns. Likewise to this human learning activity, the learning using privileged information (LUPI) paradigm provides additional information generated by the teacher to 'teach' learning models during the training stage. Therefore, this novel learning paradigm is a typical Teacher–Student Interaction mechanism. This paper is the first to present a random vector functional link (RVFL) network based on the LUPI paradigm, called RVFL+. The novel RVFL+ incorporates the LUPI paradigm that can leverage additional source of information into the RVFL, which offers an alternative way to train the RVFL. Rather than simply combining two existing approaches, the newly-derived RVFL+ fills the gap between classical randomized neural networks and the newfashioned LUPI paradigm. Moreover, the proposed RVFL+ can perform in conjunction with the kernel trick for highly complicated nonlinear feature learning, termed KRVFL+. Furthermore, the statistical property of the proposed RVFL+ is investigated, and the authors present a sharp and high-quality generalization error bound based on the Rademacher complexity. Competitive experimental results on 14 real-world datasets illustrate the great effectiveness and efficiency of the novel RVFL+ and KRVFL+, which can achieve better generalization performance than state-of-the-art methods. Highlights: This paper presents a new RVFL+, which is an alternative way to train the RVFL. The RVFL+ bridges the gap between randomized neural networks and the LUPI. The KRVFL+ is also proposed in order to handle highly nonlinear relationships. This paper provides a theoretical guarantee using the Rademacher complexity. Performances are evaluated on 14 real datasets using state-of-the-art methods.