A measure theoretical approach to the mean-field maximum principle for training NeurODEs. (February 2023)

• Record Type: Journal Article
• Authors: Bonnet, Benoît; Cipriani, Cristina; Fornasier, Massimo; Huang, Hui
• Is Part Of: Nonlinear analysis. Volume 227(2023)
• Publication Date: 2023-02
• Subjects: NeurODEs -- Mean-field optimal control -- Mean-field maximum principle -- Lagrange Multiplier Theorem; Mathematical analysis -- Periodicals; Functional analysis -- Periodicals; Nonlinear theories -- Periodicals; Analyse mathématique -- Périodiques; Analyse fonctionnelle -- Périodiques; Théories non linéaires -- Périodiques; Functional analysis; Mathematical analysis; Nonlinear theories; Periodicals; Electronic journals; 515.7248
• DOI: 10.1016/j.na.2022.113161
• Languages: English
• ISSNs: 0362-546X

Abstract

Abstract: In this paper we consider a measure-theoretical formulation of the training of NeurODEs in the form of a mean-field optimal control with L 2 -regularization of the control. We derive first order optimality conditions for the NeurODE training problem in the form of a mean-field maximum principle, and show that it admits a unique control solution, which is Lipschitz continuous in time. As a consequence of this uniqueness property, the mean-field maximum principle also provides a strong quantitative generalization error for finite sample approximations, yielding a rigorous justification of a phenomenon that we call coupled descent, indicating the simultaneous decrease of generalization and training errors. We consider two approaches to the derivation of the mean-field maximum principle, including one that is based on a generalized Lagrange multiplier theorem on convex sets of spaces of measures, which is arguably much simpler than those currently available in the literature for mean-field optimal control problems. The latter is also new, and can be considered as a result of independent interest.