On-line sequential extreme learning machine based on recursive partial least squares. (March 2015) - Matias, Tiago; Souza, Francisco; Araújo, Rui; Gonçalves, Nuno; Barreto, João P.
Abstract : Highlights: A new SLFN learning method using an online sequential extreme learning machine algorithm based on recursive partial least-squares is proposed. The proposed method is an improvement of the method proposed in (Huang et al., 2005 [1]). The proposed method was compared with three methods. The performance of the proposed method was better in all data sets. Abstract: This paper proposes the online sequential extreme learning machine algorithm based on the recursive partial least-squares method (OS-ELM-RPLS). It is an improvement to the online sequential extreme learning machine based on recursive least-squares (OS-ELM-RLS) introduced in (Huang et al., 2005 [1]). Like in the batch extreme learning machine (ELM), in OS-ELM-RLS the input weights of a single-hidden layer feedforward neural network (SLFN) are randomly generated, however the output weights are obtained by a recursive least-squares (RLS) solution. However, due to multicollinearities in the columns of the hidden-layer output matrix caused by presence of redundant input variables or by the large number of hidden-layer neurons, the problem of estimation the output weights can become ill-conditioned. In order to circumvent or mitigate such ill-conditioning problem, it is proposed to replace the RLS method by the recursive partial least-squares (RPLS) method. OS-ELM-RPLS was applied and compared with three other methods over three real-world data sets. In all the experiments, the proposed method always exhibits the best prediction performance.