A comparative investigation of neural networks in solving differential equations. (March 2021)

• Record Type: Journal Article
• Authors: Shi, Enze; Xu, Chuanju
• Is Part Of: Journal of algorithms & computational technology. Volume 15(2021)
• Publication Date: 2021-03
• Subjects: Neural networks -- differential equations -- loss function -- parameter sensitivity; Computer algorithms -- Periodicals; Numerical calculations -- Periodicals; Computer algorithms; Numerical calculations; Periodicals; 518.1
• DOI: 10.1177/1748302621998605
• Languages: English
• ISSNs: 1748-3018

Abstract

Methods for solving differential equations based on neural networks have been widely proposed in recent years. However, limited open literature to date has reported the choice of loss functions and the hyperparameters of the network and how it influences the quality of numerical solutions. In the present work we intend to address this issue. Precisely we will focus on possible choices of loss functions and compare their efficiency in solving differential equations through a series of numerical experiments. In particular, a comparative investigation is performed between the natural neural networks and Ritz neural networks, with and without penalty for the boundary conditions. The sensitivity on the accuracy of the neural networks with respect to the size of training set, the number of nodes, and the penalty parameter is also studied. In order to better understand the training behavior of the proposed neural networks, we further investigate the approximation properties of the neural networks in function fitting. A particular attention is paid to approximating Müntz polynomials by neural networks.