A new Harvard study of how mice respond to scent cues from potential mates, competitors, and nearby predators has laid a foundation for further investigations that may lead to a greater understanding of social recognition in the animal brain, with implications for a host of human disorders ranging from autism to post-traumatic stress disorder.

While it has long been known that many animals rely on a secondary olfactory organ, known as the vomeronasal organ, to detect certain scents, new research, directed by Higgins Professor of Molecular and Cellular Biology Catherine Dulac, identifies for the first time exactly how such scent cues are detected and interpreted.

In the Sept. 21 online version of the journal Nature, Dulac’s group reports identifying 88 proteins that act as receptor molecules, together with the range of specific signals each protein is responding to. Scents may indicate a potential mate or the presence of a predator. Depending on which receptor is activated, the mouse’s brain receives a signal to approach or to flee and avoid becoming another animal’s dinner.

“This discovery is an enormous achievement. I have been trying to get to this point for 15 years,” said Dulac, who is also chair of the Department of Molecular and Cellular Biology. “Now that I know the identity of these chemical cues, I can design experiments to help us understand how these receptors are activated, what signals are sent to the brain, how the brain makes sense of that signal, and how that leads to specific behaviors. These are absolutely essential questions in neuroscience that we can address because we now have the molecular and genetic tools to do so.”

Making sense of how those social and defensive signals are processed in the brain of mice, Dulac said, could shed light on how similar signals are processed in the brains of humans.

“It doesn’t matter if it’s olfactory input for a mouse or visual input for a human,” Dulac said. “The behavior circuits are built along the same architecture; they are simply fed by different sensory modalities. Our goal now is to understand how the brain makes sense of that input. That research could offer insight into mental disorders related to problems with social recognition, like autism and even schizophrenia.”

The discovery of the scent-specific proteins grew out of Dulac’s identification, more than a decade ago, of the genes that encode such vomeronasal receptor proteins. Though approximately 300 such genes were eventually identified, it wasn’t until her current research that she was able to answer two critical questions — exactly what type of signals the receptors were detecting, and how those signals are interpreted.

While researchers expected the receptors might be used to help mice interpret much of their surroundings, they were surprised to find that the vast majority of the scent signals were not devoted to finding mates or other mice, but to identifying and evading predators.

“These findings suggest that it’s relatively easy for a mouse to identify another mouse, but it’s much more complicated to identify a predator,” Dulac continued. “Predators may be reptiles, birds of prey, or other mammals, so it seems, evolutionarily speaking, that the mouse needed to multiply the number of receptors to predators to be sure it can detect as many as possible.”

The findings also clarify whether such signals are interpreted through a combination of receptors — similar to the way a human eye uses just three receptors to recognize many subtle gradations of color — or if vomeronasal receptors respond only to individual stimuli.

“What we discovered is that the second model — the use of many discrete receptors — is more accurate,” Dulac said. “Most of the receptors seem to be tied to specific predators.”

 

Further studies, Dulac said, will seek to link the activation of specific receptors to the initiation of identifiable behaviors, and try to answer a very old question: What are the “innate” and “learned” components in social and predator recognition?

Funding for the research was provided by the National Institutes of Health and the Howard Hughes Medical Institute.

 

Major study on schizophrenia, bipolar