A Reinforcement Meta-Learning framework of executive function and information demand. (January 2023)

• Record Type: Journal Article
• Authors: Silvetti, Massimo; Lasaponara, Stefano; Daddaoua, Nabil; Horan, Mattias; Gottlieb, Jacqueline
• Is Part Of: Neural networks. Volume 157(2023)
• Page Start: 103
• Page End: 113
• Publication Date: 2023-01
• Subjects: Information seeking -- Effort -- Meta Reinforcement Learning -- Visual attention -- MPFC -- Free energy; Neural computers -- Periodicals; Neural networks (Computer science) -- Periodicals; Neural networks (Neurobiology) -- Periodicals; Nervous System -- Periodicals; Ordinateurs neuronaux -- Périodiques; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux neuronaux (Neurobiologie) -- Périodiques; Neural computers; Neural networks (Computer science); Neural networks (Neurobiology); Periodicals; 006.32
• DOI: 10.1016/j.neunet.2022.10.004
• Languages: English
• ISSNs: 0893-6080

Abstract

Abstract: Gathering information is crucial for maximizing fitness, but requires diverting resources from searching directly for primary rewards to actively exploring the environment. Optimal decision-making thus maximizes information while reducing effort costs, but little is known about the neuro-computational implementation of this tradeoff. We present a Reinforcement Meta-Learning (RML) computational model that solves the trade-off between the value and costs of gathering information. We implement the RML in a biologically plausible architecture that links catecholaminergic neuromodulators, the medial prefrontal cortex and topographically organized visual maps and show that it accounts for neural and behavioral findings on information demand motivated by instrumental incentives and intrinsic utility. Moreover, the utility function used by the RML, encoded by dopamine, is an approximation of variational free energy. Thus, the RML presents a biologically plausible mechanism for coordinating motivational, executive and sensory systems generate visual information gathering policies that minimize free energy. Highlights: Optimal decision-making is needed to maximize information gain while reducing costs. We propose that a cortical-subcortical network solves this tradeoff by meta-RL. The network minimizes free energy instead of maximizing reward. It explains behavioral and neural aspects of visual information gathering.