Generalization guides human exploration in vast decision spaces. (December 2018)

• Record Type: Journal Article
• Authors: Wu, Charley; Schulz, Eric; Speekenbrink, Maarten; Nelson, Jonathan; Meder, Björn
• Is Part Of: Nature human behaviour. Volume 2:Number 12(2018)
• Page Start: 915
• Page End: 924
• Publication Date: 2018-12
• Subjects: Human behavior -- Periodicals; Psychology -- Periodicals; Sociology -- Periodicals; 300
• DOI: 10.1038/s41562-018-0467-4
• Languages: English
• ISSNs: 2397-3374

Abstract

Abstract From foraging for food to learning complex games, many aspects of human behaviour can be framed as a search problem with a vast space of possible actions. Under finite search horizons, optimal solutions are generally unobtainable. Yet, how do humans navigate vast problem spaces, which require intelligent exploration of unobserved actions? Using various bandit tasks with up to 121 arms, we study how humans search for rewards under limited search horizons, in which the spatial correlation of rewards (in both generated and natural environments) provides traction for generalization. Across various different probabilistic and heuristic models, we find evidence that Gaussian process function learning—combined with an optimistic upper confidence bound sampling strategy—provides a robust account of how people use generalization to guide search. Our modelling results and parameter estimates are recoverable and can be used to simulate human-like performance, providing insights about human behaviour in complex environments. When searching for rewards in complex, unfamiliar environments, it is often impossible to explore all options. Wu et al. show how a combination of generalization and optimistic sampling guides efficient human exploration in complex environments.