Scanning for similarities between human decision-making, AI algorithm

Where does the human brain even start when faced with a new task? It immediately checks with a mental library of solutions that worked pretty well in the past.

A recent study, published in the journal PLOS Biology, used neuroimaging to find similarities between human intelligence and a decision-making process developed by AI researchers. The influential algorithm works by learning optimal solutions to a set of tasks that can be generalized, in high-performance ways, to subsequent situations.

“The person who created this model is a bread-and-butter computer scientist,” said lead author Sam Hall-McMaster, a cognitive neuroscientist with expertise in both experimental psychology and neuroimaging. “I don’t think he necessarily expected that his model, developed for teaching machines how to learn, would be picked up and used to understand how people are making decisions.”

Overseeing the research was professor of psychology Samuel J. Gershman, whose Harvard-based Computational Cognitive Neuroscience Lab works at the intersection of human behavior and technology. “Our approach is taking ideas from AI that have been successful from an engineering perspective and designing experiments to see whether the brain is doing something similar,” explained Gershman, who holds joint appointments in the Department of Psychology and Center for Brain Science.

Researchers in the lab have produced a body of work suggesting human decision-making works analogously to some AI algorithms. “We are particularly interested in what makes these algorithms work in contexts where you have to generalize from one task to another,” Gershman noted. “But our evidence was somewhat indirect in the sense that we inferred the use of these different kinds of algorithms based on the choices people made.”

Only recently has the lab deployed neuroimaging to look for evidence of these computational processes via the brain’s internal representations, or its physical encoding of ideas and experiences.

For this study, a total of 38 participants were placed inside functional Magnetic Resonance Imaging (fMRI) machines while playing multiple rounds of a specially designed video game. Data was collected at the Max Planck Institute for Human Development in Berlin before it was analyzed at Gershman’s Cambridge lab.

Specifically tested was an algorithm, introduced by research scientist André Barreto and a team of collaborators at Google DeepMind, called successor feature and generalized policy improvement (SF&GPI). “It allows for representations that are compact and efficient but still allow you to be flexible in different settings,” explained Hall-McMaster, currently a postdoctoral researcher in Gershman’s lab.

Applying the algorithm would mean recycling strategies that worked in prior rounds of the video game. Sure enough, behavioral observations showed study participants emphasizing previous solutions with each new round — even when better options were available.

Researchers had strong ideas about which brain regions would be engaged by an SF&GPI-like process. The dorsolateral prefrontal cortex, associated with complex decision-making, was a good place to look for reactivation of successful solutions used in the past. To illustrate the concept, Hall-McMaster gave the example of a friend swinging into town and asking where to meet for a last-minute lunch date. The brain immediately calls up the café that recently served a satisfying meal in a very different scenario: being extra hungry and desperate for sustenance.

Neural markers were also expected in the medial temporal lobe and orbitofrontal cortex, both associated with gaming out the possible results of reusing any one solution. The medial temporal lobe, in particular, plays an important role in encoding various features associated with different strategies. “That could be something like the atmosphere of the café or the quality of its food,” Hall-McMaster said.

Imaging data produced strong evidence of the human brain leaning heavily on strategies that worked during past rounds of the game. Markings of this process kept repeating in the dorsolateral-prefrontal cortex and — to the researchers’ surprise — in the occipitotemporal cortex (or the OTC), a region at the brain’s back-bumper normally associated with processing complex visual stimuli.

The latter region lit up at the top of each round, as the video game presented a new set of tasks, but before study participants could implement any decisions. It was as if the OTC was queuing up library materials in anticipation.

However, fMRI scans turned up less evidence of neural reactivation of the features associated with past solutions.

“These findings are exciting,” Hall-McMaster summarized, “because they mean we now understand some of what the brain’s doing when people are engaged with complex problem-solving on new tasks. But there are still some mysteries left to solve.”