A brain-inspired network architecture for cost-efficient object recognition in shallow hierarchical neural networks. (February 2021)

• Record Type: Journal Article
• Authors: Park, Youngjin; Baek, Seungdae; Paik, Se-Bum
• Is Part Of: Neural networks. Volume 134(2021)
• Page Start: 76
• Page End: 85
• Publication Date: 2021-02
• Subjects: Visual cortex -- Long-range horizontal connection -- Object recognition -- Shallow network -- Artificial neural network -- Cost-efficiency; Neural computers -- Periodicals; Neural networks (Computer science) -- Periodicals; Neural networks (Neurobiology) -- Periodicals; Nervous System -- Periodicals; Ordinateurs neuronaux -- Périodiques; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux neuronaux (Neurobiologie) -- Périodiques; Neural computers; Neural networks (Computer science); Neural networks (Neurobiology); Periodicals; 006.32
• DOI: 10.1016/j.neunet.2020.11.013
• Languages: English
• ISSNs: 0893-6080

Abstract

Abstract: The brain successfully performs visual object recognition with a limited number of hierarchical networks that are much shallower than artificial deep neural networks (DNNs) that perform similar tasks. Here, we show that long-range horizontal connections (LRCs), often observed in the visual cortex of mammalian species, enable such a cost-efficient visual object recognition in shallow neural networks. Using simulations of a model hierarchical network with convergent feedforward connections and LRCs, we found that the addition of LRCs to the shallow feedforward network significantly enhances the performance of networks for image classification, to a degree that is comparable to much deeper networks. We found that a combination of sparse LRCs and dense local connections dramatically increases performance per wiring cost. From network pruning with gradient-based optimization, we also confirmed that LRCs could emerge spontaneously by minimizing the total connection length while maintaining performance. Ablation of emerged LRCs led to a significant reduction of classification performance, which implies these LRCs are crucial for performing image classification. Taken together, our findings suggest a brain-inspired strategy for constructing a cost-efficient network architecture to implement parsimonious object recognition under physical constraints such as shallow hierarchical depth. Highlights: Long-range connections (LRCs) enable recognition of various objects under constraints. LRCs added to shallow networks enhance performance comparable to deeper networks. LRCs can emerge spontaneously while balancing performance and wiring cost. LRCs reveal a biological strategy for a cost-efficient neural network architecture.