A singular Riemannian geometry approach to Deep Neural Networks I. Theoretical foundations. (January 2023)

• Record Type: Journal Article
• Authors: Benfenati, Alessandro; Marta, Alessio
• Is Part Of: Neural networks. Volume 158(2023)
• Page Start: 331
• Page End: 343
• Publication Date: 2023-01
• Subjects: Deep learning -- Riemann geometry -- Classification -- Degenerate metrics -- Neural networks; Neural computers -- Periodicals; Neural networks (Computer science) -- Periodicals; Neural networks (Neurobiology) -- Periodicals; Nervous System -- Periodicals; Ordinateurs neuronaux -- Périodiques; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux neuronaux (Neurobiologie) -- Périodiques; Neural computers; Neural networks (Computer science); Neural networks (Neurobiology); Periodicals; 006.32
• DOI: 10.1016/j.neunet.2022.11.022
• Languages: English
• ISSNs: 0893-6080

Abstract

Abstract: Deep Neural Networks are widely used for solving complex problems in several scientific areas, such as speech recognition, machine translation, image analysis. The strategies employed to investigate their theoretical properties mainly rely on Euclidean geometry, but in the last years new approaches based on Riemannian geometry have been developed. Motivated by some open problems, we study a particular sequence of maps between manifolds, with the last manifold of the sequence equipped with a Riemannian metric. We investigate the structures induced through pullbacks on the other manifolds of the sequence and on some related quotients. In particular, we show that the pullbacks of the final Riemannian metric to any manifolds of the sequence is a degenerate Riemannian metric inducing a structure of pseudometric space. We prove that the Kolmogorov quotient of this pseudometric space yields a smooth manifold, which is the base space of a particular vertical bundle. We investigate the theoretical properties of the maps of such sequence, eventually we focus on the case of maps between manifolds implementing neural networks of practical interest and we present some applications of the geometric framework we introduced in the first part of the paper. Highlights: We consider Neural Networks as sequences of smooth maps between manifolds. We investigate the structures induced through pull-backs on the manifolds. The pull-backs of the final Riemannian metric to any manifolds is a degenerate metric. The theoretical study leads to construct equivalence classes in the input manifold.