Abstract inference of unchosen option values. (23rd October 2019)

• Record Type: Journal Article
• Authors: Cox, Karin M.; Fiez, Julie A.
• Is Part Of: European journal of neuroscience. Volume 51:Number 3(2020)
• Page Start: 909
• Page End: 921
• Publication Date: 2019-10-23
• Subjects: decision making -- inference -- learning -- reinforcement -- reward; Nervous system -- Periodicals; 612.8
• DOI: 10.1111/ejn.14577
• Languages: English
• ISSNs: 0953-816X

Abstract

Abstract: Reinforcement learning research has pursued a persistent question: Does reward feedback prompt inferences that transcend simple associations? Reversal learning data suggest an affirmative answer: When the positive stimulus (S+) becomes the negative stimulus (S−), trained humans rapidly switch to choosing the former S−. The operations supporting such inferences remain ambiguous. Do participants identify transitions between stimulus‐specific contexts (i.e., A+B− and A−B+), or deduce values by learning the abstract contingency structure? Across two experiments, we probed humans' use of abstract rules to infer the values of unchosen alternatives. In Experiment 1, 37 participants attempted a task that originally demonstrated monkeys' difficulty with this form of inference. We presented modified discrimination problems in which the initially chosen stimulus (abstract inference group) or unchosen stimulus (control group) was replaced with a novel stimulus of identical status on Trial 2. In the abstract inference condition, accurate performance can be achieved by applying the consistent contingency structure (but not memory of stimulus‐specific reward associations) to infer to the unchosen stimulus' value. The abstract inference group learned to make accurate choices, but only after committing substantially more errors than were observed among control participants—suggesting that unchosen value inferences are infrequently drawn in standard discrimination scenarios. In Experiment 2, 17 participants completed abstract inference problems that had been modified to be suitable for fMRI investigations. Behavioral results both corroborated the Experiment 1 trends and further revealed marked individual differences in explicit awareness of the novel stimulus values. Abstract : We presented discrimination problems for which the initially chosen option was replaced with a novel option of equal value on Trial 2. Optimal choices could be made by inferring the value of the unchosen option, and appropriately approaching or avoiding it on Trial 2. Humans found this behavior difficult to acquire, indicating that unchosen value inferences are unlikely to be made routinely. Yet, their eventual learning contrasts with a past report of monkeys' failure to use such inferences.