Published online: 04 July 2004; | doi:10.1038/nn1279 【Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops】
Saori C Tanaka1, 2, 3, Kenji Doya1, 2, 3, Go Okada3, 4, Kazutaka Ueda3, 4, Yasumasa Okamoto3, 4 & Shigeto Yamawaki3, 4 1 Department of Bioinformatics and Genomics, Nara Institute of Science and Technology, 8916-5 Takayama, Ikoma, Nara 630-0101, Japan. 2 Department of Computational Neurobiology, ATR Computational Neuroscience Laboratories, 2-2-2 Hikaridai, Keihanna Science City, Kyoto 619-0288, Japan. 3 CREST, Japan Science and Technology Agency, 2-2-2 Hikaridai, Keihanna Science City, Kyoto 619-0288, Japan. 4 Department of Psychiatry and Neurosciences, Hiroshima University, 1-2-3 Kasumi, Minamiku, Hiroshima 734-8551, Japan. Correspondence should be addressed to Kenji Doya [email protected]
Evaluation of both immediate and future outcomes of one's actions is a critical requirement for intelligent behavior. Using functional magnetic resonance imaging (fMRI), we investigated brain mechanisms for reward prediction at different time scales in a Markov decision task. When human subjects learned actions on the basis of immediate rewards, significant activity was seen in the lateral orbitofrontal cortex and the striatum. When subjects learned to act in order to obtain large future rewards while incurring small immediate losses, the dorsolateral prefrontal cortex, inferior parietal cortex, dorsal raphe nucleus and cerebellum were also activated. Computational model-based regression analysis using the predicted future rewards and prediction errors estimated from subjects' performance data revealed graded maps of time scale within the insula and the striatum: ventroanterior regions were involved in predicting immediate rewards and dorsoposterior regions were involved in predicting future rewards.
These results suggest differential involvement of the cortico-basal ganglia loops in reward prediction at different time scales.