Sparse neural network regression with variable selection. (2nd November 2022)

• Record Type: Journal Article
• Authors: Shin, Jae‐Kyung; Bak, Kwan‐Young; Koo, Ja‐Yong
• Is Part Of: Computational intelligence. Volume 38:Number 6(2022)
• Page Start: 2075
• Page End: 2094
• Publication Date: 2022-11-02
• Subjects: artificial neural network -- initialization -- multivariate regression -- penalization; Artificial intelligence -- Periodicals; Computational linguistics -- Periodicals; 006.3
• DOI: 10.1111/coin.12557
• Languages: English
• ISSNs: 0824-7935

Abstract

Abstract: This paper reports on our study of a sparse neural network regression method based on a single hidden layer architecture and sparsity‐inducing penalties. To determine the size of network in a data‐adaptive way, we adopt the lasso and group lasso penalty functions to simultaneously induce sparsity at node and predictor levels. We also devise several techniques to improve performance of the neural network regression estimator. We adopt a B‐spline activation function with a compact support to identify local trends of data. In addition, we develop an algorithm based on node addition process, in which additional nodes come into the network as a complexity parameter decreases. At each value of the complexity parameter, the algorithm initializes the additional nodes along the direction that best captures the unexplained functional relationship between the response and predictors. The optimization step is conducted based on an efficient coordinate descent algorithm. Numerical studies based on simulated and real datasets illustrate that the combination of the aforementioned devices significantly improves the performance of the neural network estimator.