A differential Hebbian framework for biologically-plausible motor control. (June 2022)

• Record Type: Journal Article
• Authors: Verduzco-Flores, Sergio; Dorrell, William; De Schutter, Erik
• Is Part Of: Neural networks. Volume 150(2022)
• Page Start: 237
• Page End: 258
• Publication Date: 2022-06
• Subjects: Synaptic plasticity -- Motor control -- Reinforcement learning -- Feedback control; Neural computers -- Periodicals; Neural networks (Computer science) -- Periodicals; Neural networks (Neurobiology) -- Periodicals; Nervous System -- Periodicals; Ordinateurs neuronaux -- Périodiques; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux neuronaux (Neurobiologie) -- Périodiques; Neural computers; Neural networks (Computer science); Neural networks (Neurobiology); Periodicals; 006.32
• DOI: 10.1016/j.neunet.2022.03.002
• Languages: English
• ISSNs: 0893-6080

Abstract

Abstract: In this paper we explore a neural control architecture that is both biologically plausible, and capable of fully autonomous learning. It consists of feedback controllers that learn to achieve a desired state by selecting the errors that should drive them. This selection happens through a family of differential Hebbian learning rules that, through interaction with the environment, can learn to control systems where the error responds monotonically to the control signal. We next show that in a more general case, neural reinforcement learning can be coupled with a feedback controller to reduce errors that arise non-monotonically from the control signal. The use of feedback control can reduce the complexity of the reinforcement learning problem, because only a desired value must be learned, with the controller handling the details of how it is reached. This makes the function to be learned simpler, potentially allowing learning of more complex actions. We use simple examples to illustrate our approach, and discuss how it could be extended to hierarchical architectures. Highlights: A biologically-plausible neural network performs simple continuous–time motor tasks. Local synaptic learning is used to configure feedback control. Neural reinforcement learning can be included to control more complex systems. Ideas for handling temporal credit assignment are demonstrated. Hierarchical extensions and biological relevance are discussed.