A Dual-Dimer method for training physics-constrained neural networks with minimax architecture. (April 2021)

• Record Type: Journal Article
• Authors: Liu, Dehao; Wang, Yan
• Is Part Of: Neural networks. Volume 136(2021)
• Page Start: 112
• Page End: 125
• Publication Date: 2021-04
• Subjects: Machine learning -- Physics-constrained neural networks -- Partial differential equation -- Minimax problem -- Saddle point search; Neural computers -- Periodicals; Neural networks (Computer science) -- Periodicals; Neural networks (Neurobiology) -- Periodicals; Nervous System -- Periodicals; Ordinateurs neuronaux -- Périodiques; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux neuronaux (Neurobiologie) -- Périodiques; Neural computers; Neural networks (Computer science); Neural networks (Neurobiology); Periodicals; 006.32
• DOI: 10.1016/j.neunet.2020.12.028
• Languages: English
• ISSNs: 0893-6080

Abstract

Abstract: Data sparsity is a common issue to train machine learning tools such as neural networks for engineering and scientific applications, where experiments and simulations are expensive. Recently physics-constrained neural networks (PCNNs) were developed to reduce the required amount of training data. However, the weights of different losses from data and physical constraints are adjusted empirically in PCNNs. In this paper, a new physics-constrained neural network with the minimax architecture (PCNN-MM) is proposed so that the weights of different losses can be adjusted systematically. The training of the PCNN-MM is searching the high-order saddle points of the objective function. A novel saddle point search algorithm called Dual-Dimer method is developed. It is demonstrated that the Dual-Dimer method is computationally more efficient than the gradient descent ascent method for nonconvex–nonconcave functions and provides additional eigenvalue information to verify search results. A heat transfer example also shows that the convergence of PCNN-MMs is faster than that of traditional PCNNs. Highlights: Physics-constrained neural networks can be trained with reduced amount of data. Training of physics-constrained neural networks is formulated as a minimax problem. A Dual-Dimer algorithm is developed to search for high-order saddle points in training.