Effects of depression on reinforcement learning

Abstract

Depression is characterized by deficits in the reinforcement learning (RL) process. Although many computational and neural studies have extended our knowledge of the impact of depression on RL, most focus on habitual control (model-free RL), yielding a relatively poor understanding of goal-directed control (model-based RL) and arbitration control to find a balance between the two. We investigate the effects of depression on goal-directed and habitual control in the prefrontal–striatal circuitry. We find that depression is associated with attenuated state and reward prediction error representation in the insula and caudate, a disruption of arbitration control in the predominantly inferior lateral prefrontal cortex and frontopolar cortex, and suboptimal value–action conversion. These findings fully characterize how depression influences different levels of RL, challenging previous conflicting views that depression simply influences either habitual or goal-directed control. Our study creates possibilities for various clinical applications, such as early diagnosis and behavioral therapy design.