Approximation in shift-invariant spaces with deep ReLU neural networks. (September 2022)

• Record Type: Journal Article
• Authors: Yang, Yunfei; Li, Zhen; Wang, Yang
• Is Part Of: Neural networks. Volume 153(2022)
• Page Start: 269
• Page End: 281
• Publication Date: 2022-09
• Subjects: Deep neural networks -- Approximation complexity -- Shift-invariant spaces -- Sobolev spaces -- Besov spaces; Neural computers -- Periodicals; Neural networks (Computer science) -- Periodicals; Neural networks (Neurobiology) -- Periodicals; Nervous System -- Periodicals; Ordinateurs neuronaux -- Périodiques; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux neuronaux (Neurobiologie) -- Périodiques; Neural computers; Neural networks (Computer science); Neural networks (Neurobiology); Periodicals; 006.32
• DOI: 10.1016/j.neunet.2022.06.013
• Languages: English
• ISSNs: 0893-6080

Abstract

Abstract: We study the expressive power of deep ReLU neural networks for approximating functions in dilated shift-invariant spaces, which are widely used in signal processing, image processing, communications and so on. Approximation error bounds are estimated with respect to the width and depth of neural networks. The network construction is based on the bit extraction and data-fitting capacity of deep neural networks. As applications of our main results, the approximation rates of classical function spaces such as Sobolev spaces and Besov spaces are obtained. We also give lower bounds of the L p ( 1 ≤ p ≤ ∞ ) approximation error for Sobolev spaces, which show that our construction of neural network is asymptotically optimal up to a logarithmic factor.