When is deep learning better and when is shallow learning better: qualitative analysis. Issue 5 (3rd September 2022)

• Record Type: Journal Article
• Authors: Robles Herrera, Salvador; Ceberio, Martine; Kreinovich, Vladik
• Is Part Of: International journal of parallel, emergent and distributed systems. Volume 37:Issue 5(2022)
• Page Start: 589
• Page End: 595
• Publication Date: 2022-09-03
• Subjects: Neural networks -- deep learning -- shallow learning -- symmetry -- Invariance; Parallel computers -- Periodicals; Electronic data processing -- Distributed processing -- Periodicals; Computer algorithms -- Periodicals; 004.35
• DOI: 10.1080/17445760.2022.2070748
• Languages: English
• ISSNs: 1744-5760

Abstract

Abstract : In many practical situations, deep neural networks work better than the traditional 'shallow' ones; however, in some cases, the shallow neural networks lead to better results. At present, deciding which type of neural networks will work better is mostly done by trial and error. It is therefore desirable to come up with some criterion of when deep learning is better and when shallow is better. In this paper, we argue that this depends on whether the corresponding situation has natural symmetries: if it does, we expect deep learning to work better, otherwise we expect shallow learning to be more effective. Our general qualitative arguments are strengthened by the fact that in the simplest case, the connection between symmetries and effectiveness of deep learning can be theoretically proven.