RH Logo

5

Reinforcement learning links spontaneous cortical dopamine impulses to reward

Paper Title:

Reinforcement learning links spontaneous cortical dopamine impulses to reward

5

Abstract

Highlights •Extrasynaptic levels of dopamine in mouse cortex exhibit spontaneous impulses •Impulses are broadly distributed in amplitude and time, with a rate of about 0.01/s •Feedback was used to train mice to volitionally control their spontaneous impulses •Mice learned to reliably modulate dopamine impulses in order to receive a reward Summary In their pioneering study on dopamine release, Romo and Schultz speculated “...that the amount of dopamine released by unmodulated spontaneous impulse activity exerts a tonic, permissive influence on neuronal processes more actively engaged in preparation of self-initiated movements....”1 Motivated by the suggestion of “spontaneous impulses,” as well as by the “ramp up” of dopaminergic neuronal activity that occurs when rodents navigate to a reward,2, 3, 4, 5 we asked two questions. First, are there spontaneous impulses of dopamine that are released in cortex? Using cell-based optical sensors of extrasynaptic dopamine, [DA]ex,6 we found that spontaneous dopamine impulses in cortex of naive mice occur at a rate of ∼0.01 per second. Next, can mice be trained to change the amplitude and/or timing of dopamine events triggered by internal brain dynamics, much as they can change the amplitude and timing of dopamine impulses based on an external cue?7, 8, 9 Using a reinforcement learning paradigm based solely on rewards that were gated by feedback from real-time measurements of [DA]ex, we found that mice can volitionally modulate their spontaneous [DA]ex. In particular, by only the second session of daily, hour-long training, mice increased the rate of impulses of [DA]ex, increased the amplitude of the impulses, and increased their tonic level of [DA]ex for a reward. Critically, mice learned to reliably elicit [DA]ex impulses prior to receiving a reward. These effects reversed when the reward was removed. We posit that spontaneous dopamine impulses may serve as a salient cognitive event in behavioral planning.

Comments1

Top

Table of contents

Paper PDF

Empty State
This academic paper hasn't been uploaded yet
View the paper now by clicking the link below or upload the PDF
or