NeuroLISP: High-level symbolic programming with attractor neural networks. (February 2022)

• Record Type: Journal Article
• Authors: Davis, Gregory P.; Katz, Garrett E.; Gentili, Rodolphe J.; Reggia, James A.
• Is Part Of: Neural networks. Volume 146(2022)
• Page Start: 200
• Page End: 219
• Publication Date: 2022-02
• Subjects: Programmable neural networks -- Working memory -- Symbolic processing -- Cognitive control -- Compositionality -- Associative learning; Neural computers -- Periodicals; Neural networks (Computer science) -- Periodicals; Neural networks (Neurobiology) -- Periodicals; Nervous System -- Periodicals; Ordinateurs neuronaux -- Périodiques; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux neuronaux (Neurobiologie) -- Périodiques; Neural computers; Neural networks (Computer science); Neural networks (Neurobiology); Periodicals; 006.32
• DOI: 10.1016/j.neunet.2021.11.009
• Languages: English
• ISSNs: 0893-6080

Abstract

Abstract: Despite significant improvements in contemporary machine learning, symbolic methods currently outperform artificial neural networks on tasks that involve compositional reasoning, such as goal-directed planning and logical inference. This illustrates a computational explanatory gap between cognitive and neurocomputational algorithms that obscures the neurobiological mechanisms underlying cognition and impedes progress toward human-level artificial intelligence. Because of the strong relationship between cognition and working memory control, we suggest that the cognitive abilities of contemporary neural networks are limited by biologically-implausible working memory systems that rely on persistent activity maintenance and/or temporal nonlocality. Here we present NeuroLISP, an attractor neural network that can represent and execute programs written in the LISP programming language. Unlike previous approaches to high-level programming with neural networks, NeuroLISP features a temporally-local working memory based on itinerant attractor dynamics, top-down gating, and fast associative learning, and implements several high-level programming constructs such as compositional data structures, scoped variable binding, and the ability to manipulate and execute programmatic expressions in working memory (i.e., programs can be treated as data). Our computational experiments demonstrate the correctness of the NeuroLISP interpreter, and show that it can learn non-trivial programs that manipulate complex derived data structures (multiway trees), perform compositional string manipulation operations (PCFG SET task), and implement high-level symbolic AI algorithms (first-order unification). We conclude that NeuroLISP is an effective neurocognitive controller that can replace the symbolic components of hybrid models, and serves as a proof of concept for further development of high-level symbolic programming in neural networks. Highlights: Gated attractor neural networks can learn to represent and interpret programs. NeuroLISP performs symbolic AI algorithms that are readily implemented in LISP. High-level programming constructs enrich cognitive control of neural working memory. Program expressions can be treated as data to model reasoning about behavior.