Happiness as an intrinsic motivator in reinforcement learning. (October 2016)

• Record Type: Journal Article
• Authors: Gao, Yue; Edelman, Shimon
• Other Names: Lowe Robert guest-editor.; Barakova Emilia guest-editor.; Billing Erik guest-editor.; Broekens Joost guest-editor.
• Is Part Of: Adaptive behavior. Volume 24:Number 5(2016)
• Page Start: 292
• Page End: 305
• Publication Date: 2016-10
• Subjects: Happiness -- well-being -- hedonic -- eudaimonic -- intrinsic motivation -- reinforcement learning -- reward design; Animal behavior -- Periodicals; Animals -- Adaptation -- Periodicals; Adaptability (Psychology) -- Periodicals; Adaptation, Psychological -- Periodicals; Artificial intelligence -- Periodicals; 591.5
• DOI: 10.1177/1059712316667202
• Languages: English
• ISSNs: 1741-2633

Abstract

Reinforcement learning, a general and universally useful framework for learning from experience, has been broadly recognized as a critically important concept for understanding and shaping adaptive behavior, both in ethology and in artificial intelligence. A key component in reinforcement learning is the reward function, which, according to an emerging consensus, should be intrinsic to the learning agent and a matter of appraisal rather than a simple reflection of external outcomes. We describe an approach to intrinsically motivated reinforcement learning that involves various aspects of happiness, operationalized as dynamic estimates of well-being. In four experiments, in which simulated agents learned to explore and forage in simulated environments, we show that agents whose reward function properly balances momentary (hedonic) and longer-term (eudaimonic) well-being outperform agents equipped with standard fitness-oriented reward functions. Our findings suggest that happiness-based features can be useful in developing robust, general-purpose reward mechanisms for intrinsically motivated autonomous agents.