Journal The of Neuroscience, 2001, Vol. 21 RC159 1 of 5
Anticipation of Increasing Monetary Reward Selectively Recruits Nucleus Accumbens
Brian Knutson, Charles M. Adams, Grace W. Fong, and Daniel Hommer National Institute on Alcohol Abuse and Alcoholism, National Institutes of Health, Bethesda, Maryland 20892-1610
Comparative studies have implicated the nucleus accumbens (NAcc) in the anticipation of incentives, but the relative responsiveness of this neural substrate during anticipation of rewards versus punishments remains unclear. Using event-related functional magnetic resonance imaging, we investigated whether the anticipation of increasing monetary rewards and punishments would increase NAcc blood oxygen level-dependent contrast (hereafter, activation) in eight healthy volunteers. Whereas anticipation of increasing rewards elicited both increasing self-reported happiness and NAcc activation, anticiThe ventral striatum has been implicated as a critical neuroanatomical substrate for the anticipation of rewards in mammals (Ikemoto and Panksepp, 1999). For example, electrophysiological studies of monkeys indicate that dopamine projections from the ventral tegmental area of the midbrain to the nucleus accumbens (NAcc) of the ventral striatum re selectively in response to presentation of reward cues (Schultz et al., 1992). However, theorists have questioned the selectivity of NAcc dopamine release for anticipation of rewards versus punishments, because rat studies indicate that stressors can also increase dopamine release in the NAcc and that NAcc lesions can impair active avoidance as well as approach behaviors (Salomone et al., 1997). Comparative research also suggests that dopamine release occurs more robustly in the NAcc during reward anticipation than during reward consumption (Berridge and Robinson, 1998; Ikemoto and Panksepp, 1999). However, no human brain-imaging studies that have examined ventral striatal activity during incentive tasks have explicitly focused on the anticipation of rewards versus punishments (Thut et al., 1997; Koepp et al., 1998; Delgado et al., 2000; Elliott et al., 2000; Knutson et al., 2000; ODoherty et al., 2001). In the present study, we were able to visualize brain activity during anticipatory intervals because of the enhanced temporal resolution afforded by event-related functional magnetic resonance imaging (FMRI) ( 2 sec for multislice volumes) relative to other brain imaging modalities such as positron emission tomography (PET). In addition, we were able to focus on neural responses in small regions of the ventral striatum (e.g., the NAcc) because of the relatively ne spatial resolution of FMRI ( 4 mm).
Received April 12, 2001; revised May 9, 2001; accepted May 17, 2001. This work was supported by the National Institute on Alcohol Abuse and Alcoholism Intramural Research Program. We thank Jerald Varner, Michael Kerich, Thomas Lionetti, and Jonathan Walker for support, as well as Peter Bandettini, Steven Grant, and Wayne Drevets for comments on earlier drafts of this manuscript. Correspondence should be addressed to Dr. Brian Knutson, National Institute on Alcohol Abuse and Alcoholism, National Institutes of Health, Building 10, Room 6S240, MS 1610, Bethesda MD 20892-1610. E-mail: [e-mail]. Copyright 2001 Society for Neuroscience 0270-6474/01/210001-05$15.00/0
pation of increasing punishment elicited neither. However, anticipation of both rewards and punishments activated a different striatal region (the medial caudate). At the highest reward level ($5.00), NAcc activation was correlated with individual differences in self-reported happiness elicited by the reward cues. These ndings suggest that whereas other striatal areas may code for expected incentive magnitude, a region in the NAcc codes for expected positive incentive value. Key words: nucleus accumbens; caudate; reward; anticipation; FMRI; human Based on primate work (Schultz et al., 1997), we have adapted a paradigm for FMRI that elicits anticipation of monetary reward or punishment, called the monetary incentive delay (MID) task (Knutson et al., 2000). During the MID task, participants see cues that indicate that they may win or lose money, then wait for a variable anticipatory delay period, and nally respond to a rapidly presented target with a single button press to try to either win or avoid losing money. In this study, we used a parametric version of the MID task to examine whether the NAcc would respond during anticipation of varying amounts of potential reward versus punishment in a graded manner and whether this activity would be related to cue-elicited emotional responses. If anticipation of increasing reward most potently recruits the NAcc, we hypothesized that (1) regions of the NAcc would show increased activation during anticipation of monetary reward versus anticipation of no monetary consequences, and (2) these same areas should show increased activation during anticipation of larger versus smaller monetary rewards.
MATERIALS AND METHODS
Eight physically and psychiatrically healthy volunteers (four women and four men, right-handed, mean age 31) participated in the study. Before entering the scanner, participants completed a practice version of the task lasting 10 min. This practice task both minimized later learning effects and produced an estimate of each individuals reaction time for standardizing task difculty in the scanner. Participants were also shown
This article is published in The Journal of Neuroscience, Rapid Communications Section, which publishes brief, peerreviewed papers online, not in print. Rapid Communications are posted online approximately one month earlier than they would appear if printed. They are listed in the Table of Contents of the next open issue of JNeurosci. Cite this article as: JNeurosci, 2001, 21:RC159 (15). The publication date is the date of posting online at www.jneurosci.org. http://www.jneurosci.org/cgi/content/full/5472
2 of 5 J. Neurosci., 2001, Vol. 21
Knutson et al. Anticipation of Reward Recruits Nucleus Accumbens
Figure 1. Task design and orthogonal regressors of interest, which contrasted (1) general anticipation versus response and feedback, (2) anticipation of monetary reward versus no monetary outcome, (3) anticipation of monetary punishment versus no monetary outcome, (4) anticipation of a large ( $5.00) versus small ( $0.20) monetary reward, and (5) anticipation of a large ( $5.00) versus small ( $0.20) monetary punishment. the money that they could earn by performing the task successf ully. All participants correctly believed that they would receive money at the end of the experiment. Once in the scanner, anatomical and f unctional scans were collected. Participants engaged in two 10 min sessions of the M I D task during f unctional scan acquisition. After each session, participants retrospectively rated how they felt when they saw each of the seven cues on four-point Likert scales indexing cue-elicited affective valence (i.e., happy and unhappy). All participants gave written informed consent, and the study was approved by the Institutional Review Board of the National Institute on Alcohol Abuse and Alcoholism (N IAAA). MID task . Each of the two M I D task sessions consisted of 72, 6 sec trials, yielding a total of 144 trials. During each trial, participants saw one of seven cue shapes (cue, 250 msec), xated on a crosshair as they waited a variable interval (delay, 2000 2500 msec), and then responded to a white target square that appeared for a variable length of time (target, 160 260 msec) with a button press. Feedback (feedback, 1650 msec), which followed the disappearance of the target, notied participants of whether they had won or lost money during that trial and indicated their cumulative total at that point. On incentive trials, participants could win or avoid losing money by pressing the button during target presentation. Task difculty, based on reaction times collected during the practice session before scanning, was set such that each participant should succeed on 66% of his or her target responses. FMRI volume acquisitions were time-locked to the offset of each cue and thus were acquired during anticipatory delay periods (Fig. 1). Cues signaled potential reward (n 54; denoted by circles), potential punishment (n 54; denoted by squares), or no monetary outcome (n 36; denoted by triangles). Reward cues signaled the possibility of winning $0.20 (n 18; a circle with one horizontal line), $1.00 (n 18; a circle with two horizontal lines), or $5.00 (n 18; a circle with three horizontal lines). Similarly, punishment cues signaled the possibility of losing $0.20 (n 18; a square with one horizontal line), $1.00 (n 18; a square with two horizontal lines), or $5.00 (n 18; a square with three horizontal lines). Trial types were pseudorandomly ordered within each session (Knutson et al., 2000). FMR I acquisition. Imaging was performed using a 1.5 T General Electric MRI scanner (General Electric, Milwaukee, W I) and a standard quadrature head coil. Sixteen 3.8-mm-thick slices (in-plane resolution, 3.75 3.75 mm) centered around the intrahemispheric ssure were sagittally acquired with no interslice gap. This plane of acquisition and voxel size provided adequate resolution of subcortical regions of interest, such as the NAcc and amygdala, as well as of the anterior orbital frontal cortex, although the posterior orbital frontal cortex showed signal dropout because of proximity to tissue boundaries. Functional scans were acquired using a T2*-sensitive gradient echo sequence with the parameters of repetition time (TR) (2000 msec), echo time (TE) (40 msec), ip (90), and number of volumes (432). Structural scans were acquired using a T1-weighted spoiled grass sequence (TR, 100 msec; TE, 7 msec; ip, 90), which facilitated localization and coregistration of f unctional data. FMR I anal ysis. Analyses focused only on changes in blood oxygen level-dependent (BOLD) contrast that occurred during anticipatory delay periods and were conducted using Analysis of Functional Neural Images software (Cox, 1996). For preprocessing, voxel time series were interpolated to correct for nonsimultaneous slice acquisition within each volume (using sinc interpolation and the rightmost slice as a reference), concatenated across both task sessions, and then corrected for threedimensional motion (using the third volume of the rst session as a reference). Visual inspection of motion-correction estimates conrmed that no participants head moved 1.5 mm in any dimension from one volume acquisition to the next. Preprocessed time series data for each individual were analyzed by multiple regression (Neter et al., 1996), which allowed us to statistically covary out nuisance variables related to head motion and scanning session, to optimally localize f unctionally relevant volumes of interest (VOIs). The regression model consisted of a set of ve orthogonal regressors of interest, six regressors describing residual motion, and four regressors modeling baseline differences and linear trends for each of the two experimental sessions. Regressors of interest were convolved with a -variate f unction that modeled a prototypical hemodynamic response before inclusion in the regression model (Cohen, 1997). Maps of t statistics representing each of the regressors of interest were transformed into Z scores, which were spatially normalized by warping to Talairach space, slightly spatially smoothed to approximate the original voxel size (rms, 4 mm), and combined into a group map using a metaanalytic formula [average Z * square root (n)] (Table 1) (Knutson et al., 2000; Donaldson et al., 2001). Separate conjunction maps were calculated for reward and punishment by thresholding (0, no activation; 1, activation) and multiplying orthogonal regressor maps for incentive versus neutral anticipation ( p 0.05) with maps for high versus low incentive anticipation ( p 0.05) (Friston et al., 1999). In addition to yielding a conjoined probability threshold appropriate for the NAcc VOIs ( p 0.0025; n 10 voxels on either side), these conjunction maps allowed us to test for parametric incentive effects in the VOIs without assuming a linear relationship between incentive magnitude and brain activation response. Overlapping thresholded regions that met both f unctional criteria and also fell within the anatomical boundaries of the regions of interest (Breiter et al., 1997) were used to construct right NAcc and right caudate VOIs. The percentages of change in BOLD contrast for the anticipatory periods of each trial type (modeled with a 4 sec lag) were extracted from these VOIs and averaged (n 18 per cue). The mean percentage BOLD contrast change scores were then analyzed with 4 (magnitude, within) 2 (valence, within) repeated-measures ANOVAs. The mean performance and cue-elicited affect for each trial type were analyzed with similarly constructed ANOVAs. Differences between various incentive conditions were tested using T ukeys honestly signicant difference post hoc paired comparisons. For VOI correlational analysis with brain activation, a cue-elicited effect was mean-corrected within each item and within each participant across different incentive conditions.
RESULTS
Hit rate (i.e., proportion of successful button presses during target presentation) (mean, 70%; SD, 7.62%) and reaction times for hits (mean, 200.73 msec; SD, 17.66 msec) did not signicantly differ across incentive conditions. Thus, participants maintained a consistent rate of effort across trials, regardless of incentive condition, as instructed by the experimenter. However, the incentive value of each cue did alter affect ratings. Interactions of cue valence and magnitude indicated that participants ratings of cue-elicited happiness increased as reward cue magnitude increased (F(3,21) 11.84; p 0.001). Specically, paired comparisons indicated that participants reported experiencing more happiness when $1.00 and $5.00 cues appeared, relative to
Knutson et al. Anticipation of Reward Recruits Nucleus Accumbens
J. Neurosci., 2001, Vol. 21 3 of 5
Table 1. Regressor of interest Z scores and Talairach coordinates of peak activation foci right/anterior/superior (RAS) Area (Brodmanns area) Left NAcc Right NAcc Left caudate Right caudate Left putamen Right putamen Anterior thalamus Left Amygdala Right Amygdala Anterior cingulate (24) Mesial prefrontal cortex (32) Supplementary motor area (6) Posterior cingulate (28) Cerebellar vermis
Underlining indicates conjunction. n 8; p 0.05; uncorrected
Anticipation versus response 2.75 ( 5,15,3) 2.33 (3,4,3) 2.96 ( 22,9, 1) 2.13 (20,10, 2) 2.80 (0, 19,14) 2.94 ( 1, 4,55) 2.21 (1, 28,34)
Reward versus neutral anticipation 2.20 (12,19, 1) 2.41 ( 6,6,7) 2.82 (9.2,11) 2.14 (18,8,6) 2.40 (2, 4,11) 1.97 ( 14, 2, 9) 2.09 (2,24,37) 2.14 (3,14,41) 2.13 (2, 2,48) 2.48 ( 1, 33,26)
Large versus small reward anticipation 2.73 (12,17, 2) 3.07 ( 7,0,12) 3.43 (8,3,10) 2.07 ( 17,14, 4) 2.38 (23, 1,6) 3.20 (4, 2,9) 2.31 (1,21,30) 2.24 (1,18,30) 2.34 (6,34,24) 2.53 ( 1, 2,60) 2.54 (1, 25,32) 2.24 (0, 49, 10)
Punishment versus neutral anticipation 2.35 (7,2,9) 2.05 (3, 2.8)
Large versus small punishment anticipation 2.20 ( 6, 1,12) 2.41 (8,4,10) 2.29 (2, 2,9) 2.44 (0,6,36) 2.45 (0, 10,40) 2.54 (0,27,33) 2.04 (5, 62, 29)
$0.00 cues ( p 0.01). In contrast, ratings of cue-elicited unhappiness increased as the magnitude of punishment cues increased (F(3,21) 5.57; p 0.01). Accordingly, paired comparisons indicated that participants reported more unhappiness when presented with $0.20, $1.00, and $5.00 cues, relative to $0.00 cues ( p 0.01). Conjunction of orthogonal regressor maps indicated that brain regions showing overlapping activation for both anticipation of reward versus no outcome as well as anticipation of large versus small rewards included foci in the right nucleus accumbens, bilateral caudate, and However, thalamus. brain regions showing overlapping activation to anticipation of punishment versus no outcome as well as anticipation of large versus small punishments included foci in the right caudate and thalamus but not in the nucleus accumbens (Table 1). The right NAcc [Tailairach coordinates (TC), 12,17, 2; 495 mm 3) and right caudate (TC, 8,3,10; 1525 mm 3] striatal VOIs that met both reward-related functional criteria and fell within the anatomical boundaries of those subcortical regions were selected for additional analysis. A main effect of magnitude (F(3,21) 9.63; p 0.001) indicated that on average, the right caudate VOI showed signicantly increased activation during anticipation of both $5.00 punishments and $5.00 rewards relative to anticipation of no monetary outcome ( p 0.001) (Fig. 2). However, an interaction of valence and magnitude (F(3,21) 6.36; p 0.01) indicated that on average, the right NAcc VOI showed signicantly increased activation only during anticipation of $5.00 rewards, relative to anticipation of no monetary outcome ( p 0.001). To examine whether individual differences in positive affective reaction to reward cues were associated with individual differences in NAcc activity, we correlated $5.00 cue-elicited happiness (mean-corrected) with the right NAcc and right caudate VOI mean percentage of activation change during anticipation of winning a potential $5.00 reward. This correlational analysis revealed a signicant positive relationship between right NAcc
activity and $5.00 cue-elicited happiness (r 0.74; n 8; p 0.05) but not between right caudate activity and $5.00 cue-elicited happiness (r 0.55; NS) (Fig. 3).
DISCUSSION
To our knowledge, this is the rst study to demonstrate proportional activation of the NAcc in humans anticipating increasing rewards but not punishments. The selectivity of the NAcc response for reward anticipation cannot necessarily be predicted on the basis of comparative research, because NAcc dopamine release has been reported in both appetitive and aversive circumstances in other species (Salomone et al., 1997). However, inclusion of human subjects in the present study enabled us to compare anticipation of symbolically equivalent rewards and punishments. Although anticipation of both rewards and punishments increased activation in the medial caudate, only anticipation of reward signicantly increased activation in the ventral striatal NAcc. These results suggest a functional dissociation in which the medial caudate codes for expected incentive magnitude, whereas the NAcc codes for expected positive incentive value. Anticipation of increasing rewards elicited increasing selfreported happiness in our participants. Within the large reward condition and across participants, NAcc activity was also correlated with self-reported happiness. Increased NAcc activation may be associated with dopamine release, because NAcc dopamine release can increase NAcc BOLD contrast in rats (Marota et al., 2000). In addition, PET studies have demonstrated positive correlations between stimulant-induced dopamine release in the ventral striatum and ratings of euphoria in humans (Volkow et al., 1999; Drevets et al., 2001). Thus, the association between NAcc BOLD contrast and increased ratings of happiness observed in this study might be accounted for, in part, by dopamine release in the ventral striatum. Although these results support a positive hedonic interpretation of NAcc function, another FMRI study suggests that NAcc
4 of 5 J. Neurosci., 2001, Vol. 21
Knutson et al. Anticipation of Reward Recruits Nucleus Accumbens
Figure 2. Caudate group regressor maps for anticipation of large versus small reward ( a), reward versus no outcome ( b), large versus small punishment ( c), and punishment versus no outcome ( d); anterior 3. Overlapping areas for a and b were conjoined to construct a right caudate VOI, from which the mean ( SEM) percentage of activation change was extracted and depicted in the graph (n 18 trials per condition per participant). Anticipation of both $5.00 punishment and $5.00 reward led to a signicant percentage of activation change in this VOI, relative to anticipation of no outcome.
activity is modulated by unpredictability of delivery of uid rewards (Berns et al., 2001). Although primate research conrms that delivery of unpredictable rewards enhances activity in the ventral striatum (Schultz et al., 1997), delivery of unpredictable punishments does not necessarily have this effect (Mirenowicz and Schultz, 1996). In the present study, anticipated success at gaining rewards and avoiding punishments was kept constant across different incentive conditions, and participants did not signicantly vary in their performance across incentive conditions. Thus, although the NAcc may be modulated by reward unpredictability, the present ndings suggest that it also plays a selective role in the anticipation of rewards versus punishments. However, reward unpredictability may magnify both immediate positive hedonic reactions as well as anticipation of subsequent rewards. These possibilities would be consistent with the current ndings and pose intriguing possibilities for future research. Our present focus on anticipation necessitated that we compare activations that occurred only 4 sec after anticipatory intervals. This conservatively short lag was selected to minimize potentially confounding activations because of cue perception, which occurred before anticipatory intervals, and also motor response or feedback, which occurred after anticipatory intervals. Comparison of anticipatory activation that occurred before reward feedback versus nonreward feedback revealed no signicant differences, demonstrating that brain activity during subsequent reward feedback did not contaminate the activation observed during the anticipatory intervals. However, hemodynamic lags in
Figure 3. Nucleus accumbens group regressor maps for anticipation of large versus small reward ( a), reward versus no outcome ( b), large versus small punishment ( c), and punishment versus no outcome ( d); anterior 18. Overlapping areas for a and b were conjoined to construct a right NAcc VOI, from which the mean ( SEM) percentage of activation change was extracted and depicted in the graph (n 18 trials per condition per participant). Anticipation of a $5.00 reward only led to a signicant percentage of activation change in this VOI relative to anticipation of no outcome. The scatterplot depicts the correlation of the percentage of activation change during anticipation of a potential $5.00 reward and mean-corrected ratings of $5.00 reward cue-elicited happiness.
peak BOLD response may vary across different regions of the brain as well as across different individuals (Buckner, 1998). In addition, reward feedback may induce more prolonged activation than either punishment or neutral feedback in the caudate (Delgado et al., 2000). Thus, the 4 sec lag may have failed to illuminate later or more sustained activations evoked by anticipation of incentives. Nonetheless, comparisons of activations that occurred at a later lag (6 sec) yielded similar results, which were less robust in the NAcc and more robust in the caudate. Therefore, although the NAcc may respond earlier or more phasically than the caudate during reward anticipation, the observed pattern persists over time. Although we report an apparently lateralized response of the right NAcc, reduction of signicance thresholds for the group maps revealed similar activation patterns in the left NAcc during anticipation of reward ( p 0.10, uncorrected) but not during anticipation of punishment. The apparently unilateral nding reported here may result from asynchronies in the timing of slice
Knutson et al. Anticipation of Reward Recruits Nucleus Accumbens
J. Neurosci., 2001, Vol. 21 5 of 5
acquisition, rather than from true lateralization of function. Choice of a small voxel size ( 4 mm on each side) and smoothing kernel (4 mm rms) may have enabled us to better resolve the NAcc focus of activity and to minimize partial voluming effects. Our group activation focus fell squarely within the anatomical boundaries of the rostral NAcc, in contrast to other ventral striatal foci reported in FMRI studies of monetary reward feedback such as the putamen (Elliott et al., 2000) and sublenticular extended amygdala (Delgado et al., 2000). Although rat studies implicate the shell of the NAcc more prominently than the core of the NAcc in reward anticipation (Ikemoto and Panksepp, 1999), the NAcc shell shows anatomical dispersion across different areas of the ventral striatum in primates (Gerfen et al., 1985). Thus, the current spatial resolution of brain imaging technology cannot resolve activation associated with NAcc subcompartments (Drevets et al., 2001). Notably, all of the regions dened by conjunction maps (i.e., those that responded in a parametric manner) lay below the cortex. This subcortical localization is in contrast to the prominent mesial cortical activations that we observed in a previous study of incentive response (Knutson et al., 2000) and is in contrast to the orbitofrontal cortical activations reported by others in studies of reward feedback (Thut et al., 1997; Elliott et al., 2000; ODoherty et al., 2001). Unlike electrophysiological studies in monkeys, we did not observe parametric activation during reward delays in the ventral orbitofrontal cortex (Hikosaka and Watanabe, 2000; Schultz et al., 2000). However, our scanning protocol was designed to focus on the ventral striatum, and signal dropout in t...