- Article
- Published:
Optimal decision making and the anterior cingulate cortex
- Steven W Kennerley1 nAff3,
- Mark E Walton1,
- Timothy E J Behrens1,2,
- Mark J Buckley1 &
- …
- Matthew F S Rushworth1,2
Nature Neurosciencevolume 9, pages940–947 (2006)Cite this article
11kAccesses
38Altmetric
Abstract
Learning the value of options in an uncertain environment is central to optimal decision making. The anterior cingulate cortex (ACC) has been implicated in using reinforcement information to control behavior. Here we demonstrate that the ACC's critical role in reinforcement-guided behavior is neither in detecting nor in correcting errors, but in guiding voluntary choices based on the history of actions and outcomes. ACC lesions did not impair the performance of monkeys (Macaca mulatta) immediately after errors, but made them unable to sustain rewarded responses in a reinforcement-guided choice task and to integrate risk and payoff in a dynamic foraging task. These data suggest that the ACC is essential for learning the value of actions.
This is a preview of subscription content,access via your institution
Access options
Subscription info for Japanese customers
We have a dedicated website for our Japanese customers. Please go tonatureasia.com to subscribe to this journal.
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
Reward-related choices determine information timing and flow across macaque lateral prefrontal cortex
References
McCoy, A.N. & Platt, M.L. Risk-sensitive neurons in macaque posterior cingulate cortex.Nat. Neurosci.8, 1220–1227 (2005).
Platt, M.L. & Glimcher, P.W. Neural correlates of decision variables in parietal cortex.Nature400, 233–238 (1999).
Sugrue, L.P., Corrado, G.S. & Newsome, W.T. Matching behavior and the representation of value in the parietal cortex.Science304, 1782–1787 (2004).
Murray, E.A., Bussey, T.J. & Wise, S.P. Role of prefrontal cortex in a network for arbitrary visuomotor mapping.Exp. Brain Res.133, 114–129 (2000).
Sutton, R.S. & Barto, A.G.Reinforcement Learning (MIT Press, Cambridge, Massachusetts, 1998).
Bayer, H.M. & Glimcher, P.W. Midbrain dopamine neurons encode a quantitative reward prediction error signal.Neuron47, 129–141 (2005).
Schultz, W. Getting formal with dopamine and reward.Neuron36, 241–263 (2002).
Nakahara, H., Itoh, H., Kawagoe, R., Takikawa, Y. & Hikosaka, O. Dopamine neurons can represent context-dependent prediction error.Neuron41, 269–280 (2004).
Satoh, T., Nakai, S., Sato, T. & Kimura, M. Correlated coding of motivation and outcome of decision by dopamine neurons.J. Neurosci.23, 9913–9923 (2003).
Paus, T. Primate anterior cingulate cortex: where motor control, drive and cognition interface.Nat. Rev. Neurosci.2, 417–424 (2001).
Rushworth, M.F., Walton, M.E., Kennerley, S.W. & Bannerman, D.M. Action sets and decisions in the medial frontal cortex.Trends Cogn. Sci.8, 410–417 (2004).
Botvinick, M.M., Braver, T.S., Barch, D.M., Carter, C.S. & Cohen, J.D. Conflict monitoring and cognitive control.Psychol. Rev.108, 624–652 (2001).
Matsumoto, K., Suzuki, W. & Tanaka, K. Neuronal correlates of goal-based motor selection in the prefrontal cortex.Science301, 229–232 (2003).
Walton, M.E., Devlin, J.T. & Rushworth, M.F. Interactions between decision making and performance monitoring within prefrontal cortex.Nat. Neurosci.7, 1259–1265 (2004).
Hadland, K.A., Rushworth, M.F., Gaffan, D. & Passingham, R.E. The anterior cingulate and reward-guided selection of actions.J. Neurophysiol.89, 1161–1164 (2003).
Morecraft, R.J. & Van Hoesen, G.W. Convergence of limbic input to the cingulate motor cortex in the rhesus monkey.Brain Res. Bull.45, 209–232 (1998).
Amiez, C., Joseph, J.P. & Procyk, E. Anterior cingulate error-related activity is modulated by predicted reward.Eur. J. Neurosci.21, 3447–3452 (2005).
Shima, K. & Tanji, J. Role for cingulate motor area cells in voluntary movement selection based on reward.Science282, 1335–1338 (1998).
Ito, S., Stuphorn, V., Brown, J.W. & Schall, J.D. Performance monitoring by the anterior cingulate cortex during saccade countermanding.Science302, 120–122 (2003).
Ullsperger, M. & von Cramon, D.Y. Error monitoring using external feedback: specific roles of the habenular complex, the reward system, and the cingulate motor area revealed by functional magnetic resonance imaging.J. Neurosci.23, 4308–4314 (2003).
Gehring, W.J., Goss, B., Coles, M.G., Meyer, D.E & Donchin, E. A neural system for error detection and compensation.Psychol. Sci.4, 385–390 (1993).
Holroyd, C.B. & Coles, M.G. The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity.Psychol. Rev.109, 679–709 (2002).
Brown, J.W. & Braver, T.S. Learned predictions of error likelihood in the anterior cingulate cortex.Science307, 1118–1121 (2005).
Fellows, L.K. & Farah, M.J. Is anterior cingulate cortex necessary for cognitive control?Brain128, 788–796 (2005).
Swick, D. & Turken, A.U. Dissociation between conflict detection and error monitoring in the human anterior cingulate cortex.Proc. Natl Acad. Sci. USA99, 16354–16359 (2002).
Rushworth, M.F., Hadland, K.A., Gaffan, D. & Passingham, R.E. The effect of cingulate cortex lesions on task switching and working memory.J. Cogn. Neurosci.15, 338–353 (2003).
Amiez, C., Joseph, J.P. & Procyk, E. Reward encoding in the monkey anterior cingulate cortex.Cereb. Cortex, published online 5 October 2005 (doi:10.1093/cercor/bhj046).
Lau, B. & Glimcher, P.W. Dynamic response-by-response models of matching behavior in rhesus monkeys.J. Exp. Anal. Behav.84, 555–579 (2005).
Lee, D., Conroy, M.L., McGreevy, B.P. & Barraclough, D.J. Reinforcement learning and decision making in monkeys during a competitive game.Brain Res. Cogn. Brain Res.22, 45–58 (2004).
Herrnstein, R.J.The Matching Law: Papers in Psychology and Economics (eds. Rachlin, H. & Laibson, D.I.) (Harvard Univ. Press, Cambridge, Massachusetts, 1997).
Bush, G. et al. Dorsal anterior cingulate cortex: a role in reward-based decision making.Proc. Natl Acad. Sci. USA99, 523–528 (2002).
Niki, H. & Watanabe, M. Prefrontal and cingulate unit activity during timing behavior in the monkey.Brain Res.171, 213–224 (1979).
Genovesio, A., Brasted, P.J., Mitz, A.R. & Wise, S.P. Prefrontal cortex activity related to abstract response strategies.Neuron47, 307–320 (2005).
Williams, S.M. & Goldman-Rakic, P.S. Widespread origin of the primate mesofrontal dopamine system.Cereb. Cortex8, 321–345 (1998).
Murray, E.A., Davidson, M., Gaffan, D., Olton, D.S. & Suomi, S. Effects of fornix transection and cingulate cortical ablation on spatial memory in rhesus monkeys.Exp. Brain Res.74, 173–186 (1989).
Morris, R., Pandya, D.N. & Petrides, M. Fiber system linking the mid-dorsolateral frontal cortex with the retrosplenial/presubicular region in the rhesus monkey.J. Comp. Neurol.407, 183–192 (1999).
Aron, A.R., Robbins, T.W. & Poldrack, R.A. Inhibition and the right inferior frontal cortex.Trends Cogn. Sci.8, 170–177 (2004).
Roberts, A.C. & Wallis, J.D. Inhibitory control and affective processing in the prefrontal cortex: neuropsychological studies in the common marmoset.Cereb. Cortex10, 252–262 (2000).
Knutson, B., Taylor, J., Kaufman, M., Peterson, R. & Glover, G. Distributed neural representation of expected value.J. Neurosci.25, 4806–4812 (2005).
Shidara, M. & Richmond, B.J. Anterior cingulate: single neuronal signals related to degree of reward expectancy.Science296, 1709–1711 (2002).
Wallis, J.D. & Miller, E.K. Neuronal activity in primate dorsolateral and orbital prefrontal cortex during performance of a reward preference task.Eur. J. Neurosci.18, 2069–2081 (2003).
Roesch, M.R. & Olson, C.R. Neuronal activity related to reward value and motivation in primate frontal cortex.Science304, 307–310 (2004).
Samejima, K., Ueda, Y., Doya, K. & Kimura, M. Representation of action-specific reward values in the striatum.Science310, 1337–1340 (2005).
Barraclough, D.J., Conroy, M.L. & Lee, D. Prefrontal cortex and decision making in a mixed-strategy game.Nat. Neurosci.7, 404–410 (2004).
Nakamura, K., Roesch, M.R. & Olson, C.R. Neuronal activity in macaque SEF and ACC during performance of tasks involving conflict.J. Neurophysiol.93, 884–908 (2005).
Aston-Jones, G. & Cohen, J.D. An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance.Annu. Rev. Neurosci.28, 403–450 (2005).
Yu, A.J. & Dayan, P. Uncertainty, neuromodulation, and attention.Neuron46, 681–692 (2005).
He, S.Q., Dum, R.P. & Strick, P.L. Topographic organization of corticospinal projections from the frontal lobe: motor areas on the medial surface of the hemisphere.J. Neurosci.15, 3284–3306 (1995).
Bates, J.F. & Goldman-Rakic, P.S. Prefrontal connections of medial motor areas in the rhesus monkey.J. Comp. Neurol.336, 211–228 (1993).
Paxinos, G., Huang, X.F. & Toga, A.W.The Rhesus Monkey Brain in Stereotaxic Coordinates (Academic Press, San Diego, 1999).
Acknowledgements
We are grateful to D. Gaffan for advice and encouragement. Supported by the Medical Research Council. Additional support from the Clarendon Foundation (S.W.K.), the Wellcome Trust (M.E.W.) and the Royal Society (M.J.B and M.F.S.R.).
Author information
Steven W Kennerley
Present address: Helen Wills Neuroscience Institute, University of California Berkeley, 132 Barker Hall, MC #3190, Berkeley, California, 94720-3190, USA
Authors and Affiliations
Department of Experimental Psychology, South Parks Road, Oxford, OX1 3UD, UK
Steven W Kennerley, Mark E Walton, Timothy E J Behrens, Mark J Buckley & Matthew F S Rushworth
Oxford Centre for Functional Magnetic Resonance Imaging of the Brain (FMRIB), John Radcliffe Hospital, Headington, Oxford, OX3 9DU, UK
Timothy E J Behrens & Matthew F S Rushworth
- Steven W Kennerley
You can also search for this author inPubMed Google Scholar
- Mark E Walton
You can also search for this author inPubMed Google Scholar
- Timothy E J Behrens
You can also search for this author inPubMed Google Scholar
- Mark J Buckley
You can also search for this author inPubMed Google Scholar
- Matthew F S Rushworth
You can also search for this author inPubMed Google Scholar
Corresponding author
Correspondence toSteven W Kennerley.
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Supplementary information
Supplementary Fig. 1
Number of trials to update to the other response after a task-imposed response switch in Experiment 1. (PDF 57 kb)
Supplementary Fig. 2
Inter-response time performance in Experiment 1A. (PDF 54 kb)
Supplementary Fig. 3
Performance for sustaining rewarded behavior following errors that occurred at a task-imposed switch trial or all other errors. (PDF 59 kb)
Supplementary Fig. 4
Relative contributions of the high and low reward-probability responses in determining ropt for the 0.4:0.1 action-reward ratio pairing in Experiment 2. (PDF 84 kb)
Supplementary Fig. 5
The average expected reward per response for each action-reward ratio pair in Experiment 2. (PDF 64 kb)
Supplementary Fig. 6
Group response ratio plot for the action-reward ratio 0.4:0.1. (PDF 79 kb)
Rights and permissions
About this article
Cite this article
Kennerley, S., Walton, M., Behrens, T.et al. Optimal decision making and the anterior cingulate cortex.Nat Neurosci9, 940–947 (2006). https://doi.org/10.1038/nn1724
Received:
Accepted:
Published:
Issue Date:
