Scientists Identify Brain Region Responsible for Calculating Risk versus Reward

June 15, 2006

As any gambler knows, the most important decision is where to play. Some flit from table to table, machine to machine and game to game. Others prefer to settle in for the long haul. Now researchers have used those tendencies to probe the function of the human brain as it chooses between the familiar and the unknown. Nathaniel Daw and John O'Doherty of University College London and their colleagues employed slot machines and functional magnetic resonance imaging (fMRI) to investigate how 14 healthy subjects decided between reaping steady profits at a given slot machine or testing the profit potential of a new one. Scientists call the behavior of utilizing a known resource exploitation; the term they give to the behavior of seeking an even better resource is exploration. Although exploitation seems the safe bet, survival can depend on judicious use of exploration.

"The desire to select what seems the richest option is always balanced against the desire to choose a less familiar option that might turn out to serve better," Daw explains. "Most people switch between exploring and exploiting seamlessly and this has always made it hard to distinguish between someone who is doing something they know will offer the highest payout and a person who is testing out new options." To so distinguish, the neuroscientists set up four slot machines to pay off at four different average rates. After each trial, these payoffs changed randomly from machine to machine. In order to discover which slot machine paid the most, a given subject would have to select it at the risk of abandoning a higher paying machine. After the tests were completed, the subjects reported that they had occasionally tried different machines to find the highest reward and sometimes stuck with a slot that they thought offered the most.

The researchers were thus able to categorize whether the subjects were exploring or exploiting in any given trial. They found that human exploration follows the so-called softmax mathematical rule, in which subjects choose whether to explore or not based on the probability of a better payout. In other words, if you determine your reward at a given machine will be small, you are more likely to change. That much the slot machines and interviews revealed. The fMRI showed that the frontopolar cortex and sulcus of a given subject strongly activated when they chose to explore. Other studies have implicated these regions in behavioral control and decision making, according to the paper presenting the finding in today's Nature, but this is the first time neuroscientists have marked these areas of the brain as associated with investigating the unknown. Exploration turns out to be a controlled gamble after all. --David Biello