Reversal Learning and Dopamine: A Bayesian Perspective

被引:94
|
作者
Costa, Vincent D. [1 ]
Tran, Valery L. [1 ]
Turchi, Janita [1 ]
Averbeck, Bruno B. [1 ]
机构
[1] NIMH, Neuropsychol Lab, NIH, Bethesda, MD 20892 USA
来源
JOURNAL OF NEUROSCIENCE | 2015年 / 35卷 / 06期
关键词
Bayesian; dopamine; haloperidol; L-DOPA; reinforcement learning; reversal learning; PREFRONTAL CORTEX; PARKINSONS-DISEASE; ORBITOFRONTAL CORTEX; NUCLEUS-ACCUMBENS; FRONTAL-CORTEX; IN-VIVO; LESIONS; DISCRIMINATION; REINFORCEMENT; SENSITIVITY;
D O I
10.1523/JNEUROSCI.1989-14.2015
中图分类号
Q189 [神经科学];
学科分类号
071006 ;
摘要
Reversal learning has been studied as the process of learning to inhibit previously rewarded actions. Deficits in reversal learning have been seen after manipulations of dopamine and lesions of the orbitofrontal cortex. However, reversal learning is often studied in animals that have limited experience with reversals. As such, the animals are learning that reversals occur during data collection. We have examined a task regime in which monkeys have extensive experience with reversals and stable behavioral performance on a probabilistic two-arm bandit reversal learning task. We developed a Bayesian analysis approach to examine the effects of manipulations of dopamine on reversal performance in this regime. We find that the analysis can clarify the strategy of the animal. Specifically, at reversal, the monkeys switch quickly from choosing one stimulus to choosing the other, as opposed to gradually transitioning, which might be expected if they were using a naive reinforcement learning (RL) update of value. Furthermore, we found that administration of haloperidol affects the way the animals integrate prior knowledge into their choice behavior. Animals had a stronger prior on where reversals would occur on haloperidol than on levodopa (L-DOPA) or placebo. This strong prior was appropriate, because the animals had extensive experience with reversals occurring in the middle of the block. Overall, we find that Bayesian dissection of the behavior clarifies the strategy of the animals and reveals an effect of haloperidol on integration of prior information with evidence in favor of a choice reversal.
引用
收藏
页码:2407 / 2416
页数:10
相关论文
共 50 条
  • [21] NMDA receptor deletion on dopamine neurons disrupts visual discrimination and reversal learning
    Radke, Anna K.
    Zweifel, Larry S.
    Holmes, Andrew
    NEUROSCIENCE LETTERS, 2019, 699 : 109 - 114
  • [22] Reinforcement learning and Bayesian inference provide complementary models for the of adolescents in stochastic reversal
    Eckstein, Maria K.
    Master, Sarah L.
    Dahl, Ronald E.
    Wilbrecht, Linda
    Collins, Anne G. E.
    DEVELOPMENTAL COGNITIVE NEUROSCIENCE, 2022, 55
  • [23] Learning Dynamical Systems From Quantized Observations: A Bayesian Perspective
    Piga, Dario
    Mejari, Manas
    Forgione, Marco
    IEEE TRANSACTIONS ON AUTOMATIC CONTROL, 2022, 67 (10) : 5471 - 5478
  • [24] Active Learning and Bayesian Optimization: A Unified Perspective to Learn with a Goal
    Di Fiore, Francesco
    Nardelli, Michela
    Mainini, Laura
    ARCHIVES OF COMPUTATIONAL METHODS IN ENGINEERING, 2024, 31 (05) : 2985 - 3013
  • [25] REVERSAL OF RESPIRATORY RESPONSES TO DOPAMINE AFTER DOPAMINE ANTAGONISTS
    ZAPATA, P
    ZUAZO, A
    RESPIRATION PHYSIOLOGY, 1982, 47 (02): : 239 - 255
  • [27] Phasic dopamine release induced by positive feedback predicts individual differences in reversal learning
    Klanker, Marianne
    Sandberg, Tessa
    Joosten, Ruud
    Willuhn, Ingo
    Feenstra, Matthijs
    Denys, Damiaan
    NEUROBIOLOGY OF LEARNING AND MEMORY, 2015, 125 : 135 - 145
  • [28] Establishing the Dopamine Dependency of Human Striatal Signals During Reward and Punishment Reversal Learning
    van der Schaaf, Marieke E.
    van Schouwenburg, Martine R.
    Geurts, Dirk E. M.
    Schellekens, Arnt F. A.
    Buitelaar, Jan K.
    Verkes, Robbert Jan
    Cools, Roshan
    CEREBRAL CORTEX, 2014, 24 (03) : 633 - 642
  • [29] Dissociable Effects of Dopamine and Serotonin on Reversal Learning (vol 80, pg 1090, 2013)
    den Ouden, Hanneke E. M.
    Daw, Nathaniel D.
    Fernandez, Guillen
    Elshout, Joris A.
    Rijpkema, Mark
    Hoogman, Martine
    Franke, Barbara
    Cools, Roshan
    NEURON, 2013, 80 (06) : CP4 - CP4
  • [30] Dopamine DRD2 Polymorphism Alters Reversal Learning and Associated Neural Activity
    Jocham, Gerhard
    Klein, Tilmann A.
    Neumann, Jane
    von Cramon, D. Yves
    Reuter, Martin
    Ullsperger, Markus
    JOURNAL OF NEUROSCIENCE, 2009, 29 (12): : 3695 - 3704