Next-generation deep learning based on simulators and synthetic data. Issue 2 (February 2022)

• Record Type: Journal Article
• Authors: de Melo, Celso M.; Torralba, Antonio; Guibas, Leonidas; DiCarlo, James; Chellappa, Rama; Hodgins, Jessica
• Is Part Of: Trends in cognitive sciences. Volume 26:Issue 2(2022)
• Page Start: 174
• Page End: 187
• Publication Date: 2022-02
• Subjects: deep neural networks -- synthetic data -- graphics-rendering pipelines -- generative adversarial networks -- domain adaptation -- next-generation learning; Cognitive science -- Periodicals; Cognitive neuroscience -- Periodicals; 153.05
• DOI: 10.1016/j.tics.2021.11.008
• Languages: English
• ISSNs: 1364-6613

Abstract

Abstract : Deep learning (DL) is being successfully applied across multiple domains, yet these models learn in a most artificial way: they require large quantities of labeled data to grasp even simple concepts. Thus, the main bottleneck is often access to supervised data. Here, we highlight a trend in a potential solution to this challenge: synthetic data. Synthetic data are becoming accessible due to progress in rendering pipelines, generative adversarial models, and fusion models. Moreover, advancements in domain adaptation techniques help close the statistical gap between synthetic and real data. Paradoxically, this artificial solution is also likely to enable more natural learning, as seen in biological systems, including continual, multimodal, and embodied learning. Complementary to this, simulators and deep neural networks (DNNs) will also have a critical role in providing insight into the cognitive and neural functioning of biological systems. We also review the strengths of, and opportunities and novel challenges associated with, synthetic data. Highlights: Despite their initial successes, it is becoming apparent that modern deep learning (DL) models are hindered by an important bottleneck: the need for large quantities of annotated data to train the models. Synthetic data provide a solution to this challenge. They are easy to generate, error-free, inexhaustible, pre-annotated, and avoid many ethical and practical concerns. The past decade has experienced unprecedented progress in data synthesis and domain adaptation techniques that close the (statistical) gap between synthetic and real data. Beyond sustaining the DL revolution, synthetic data will enable a next generation of DL models that understand the physical composition of the world and learn continually, multimodally, and interactively. Integrated synthesis and learning pipelines can support life-long structured learning that is more similar to biological learning systems.