What’s in a face?

When you meet people for the first time, what’s the first thing you think you notice? Is it their hair color, or eye color? Maybe it’s whether they’re wearing a suit or a T-shirt and jeans, or whether they have a firm handshake.

But your brain, Harvard researchers say, immediately takes note of two key characteristics: their race and gender.

Using real-time scans of the brain, recent Harvard Ph.D. Juan Manuel Contreras, Richard Clarke Cabot Professor of Social Ethics Mahzarin R. Banaji, and Psychology Professor Jason P. Mitchell found a brain region in which patterns of neural activity change when people look at black and white faces, and at male and female faces. The study is described in a paper published last month in the journal PLOS ONE.

“We found that a brain region called the fusiform face area, or the FFA for short, seems to play a key role in differentiating faces along these two dimensions,” said Contreras, the study’s first author, who earned his doctorate in psychology. “When we studied the patterns of activation in this region, we found they were different for black and white faces, and for female and male faces.”

Interestingly, Contreras said, the fact that those patterns appear in the FFA, which earlier studies had shown to be active just 200 milliseconds after seeing a face, suggests that differences in race and sex are parsed early in visual perception. While the brain seems to be collecting information about race and sex, Contreras said, it’s not until later in visual processing that meaning is attached to those differences.

“It’s important to note that previous research suggests the FFA does not endow visual stimuli with meaning, so it probably does not know anything about sex and race. It’s simply a brain region in the visual system that sees faces as belonging to two different sets,” Contreras said. “The information is simply being gathered, and is then handed off to other parts of the brain that begin to process what those differences mean — other regions that have information about what men and women are like or what it means for a face to belong to a black person or a white person”

To understand how the brain gathers that information and begins to process it, Contreras and his colleagues turned to functional magnetic resonance imaging, or fMRI, a technique that allows researchers to monitor changes in blood flow in the brain in real time.

After placing participants in a scanner, the subjects were shown a series of images on a computer screen. For some images, the participants were asked to identify whether the faces were male or female, and for others whether the faces were black or white.

“We take images every few seconds,” Contreras explained. “Using statistical analysis, we can identify patterns of neural activity that correspond to different social categories. We could then look for differences in those patterns between the faces of blacks and whites, and between the faces of men and women.

“We also found evidence that, when we asked participants to pay attention only to the sex of a person, this region was still recognizing race. When we told them to pay attention to race, the FFA was still recognizing sex, so it appears as though this region is constantly categorizing faces by sex and race.”

The question now facing researchers, Contreras said, is why.

One possible reason is that, whether for evolutionary or developmental reasons, it can be important to know the sex and race of other people, especially in contexts in which those differences should change the way in which you interact with them.

“Sex and race can be important things to know about another person, so it would make sense that as soon as you see another person, you need to know figure out the social categories to which they belong,” Contreras said.

“What’s interesting is that the FFA is also believed to be involved in some aspects of processing identity,” he added. “Obviously, characteristics that are inextricably linked to you, like your race and your sex, are part of identity. Other scientists have shown that we perceive identity by perceiving the sex and race of faces, and what we’re showing here is a sort of neural correlate of that. If this region is responsible for identity processing, it might make sense that it’s also responsible for recognizing race and sex differences.”