The algorithmic architecture of exploration in the human brain. (April 2019)

• Record Type: Journal Article
• Authors: Schulz, Eric; Gershman, Samuel J.
• Is Part Of: Current opinion in neurobiology. Volume 55(2019)
• Page Start: 7
• Page End: 14
• Publication Date: 2019-04
• Subjects: Neurobiology -- Periodicals; 573.8
• DOI: 10.1016/j.conb.2018.11.003
• Languages: English
• ISSNs: 0959-4388

Abstract

Highlights: Humans use uncertainty-based algorithms to balance exploration and exploitation. Humans utilize structure in the state space to make exploration more efficient. In some cases, humans explore non-myopically. Abstract : Balancing exploration and exploitation is one of the central problems in reinforcement learning. We review recent studies that have identified multiple algorithmic strategies underlying exploration. In particular, humans use a combination of random and uncertainty-directed exploration strategies, which rely on different brain systems, have different developmental trajectories, and are sensitive to different task manipulations. Humans are also able to exploit sophisticated structural knowledge to aid their exploration, such as information about correlations between options. New computational models, drawing inspiration from machine learning, have begun to formalize these ideas and offer new ways to understand the neural basis of reinforcement learning.