{"id":318645,"date":"2021-02-25T14:01:45","date_gmt":"2021-02-25T19:01:45","guid":{"rendered":"https:\/\/news.harvard.edu\/gazette\/?p=318645"},"modified":"2023-11-08T20:07:24","modified_gmt":"2023-11-09T01:07:24","slug":"tech-breakthrough-combines-motion-capture-and-neural-learning","status":"publish","type":"post","link":"https:\/\/news.harvard.edu\/gazette\/story\/2021\/02\/tech-breakthrough-combines-motion-capture-and-neural-learning\/","title":{"rendered":"CAPTURE-ing movement in freely behaving animals"},"content":{"rendered":"\n\t
\"An

An artist\u2019s interpretation of CAPTURE.<\/p>

All images courtesy of K.J. Herrera, J.D. Marshall, B.P. Olveczky<\/p><\/figcaption><\/figure>\n\n\t

\n\t\t\t\n\t\t\tScience & Tech\t\t<\/a>\n\t\t\n\t\t

\n\t\tCAPTURE-ing movement in freely behaving animals\t<\/h1>\n\n\t\n\t\t\t<\/div>\n\t\t\n\t
\n\t\t
\n\t\t\t
\n\t\t\t\t\t

\n\t\tJuan Siliezar\t<\/p>\n\t\t\t

\n\t\t\tHarvard Staff Writer\t\t<\/p>\n\t\t\t\t\t<\/address>\n\t\t<\/div>\n\n\t\t

\n\t\t\tNew behavioral-monitoring system combines motion capture, deep learning\t\t<\/h2>\n\t\t\n<\/header>\n\n\n\n
\n\n\n\t\t

Scientists studying the movement of animals have longed for a motion-capture method similar to the one Hollywood animators use to create spectacular big-screen villains (think Thanos in \u201cThe Avengers\u201d).<\/p>\n

Now a team of Harvard-led scientists has made a breakthrough, assembling a new system combining motion capture and deep learning to continuously track the 3D movements of freely behaving animals. The project, which monitors how the brain controls behavior, has the potential to help combat human disease or advance the creation of artificial intelligence.<\/p>\n

The system, called continuous appendicular and postural tracking using retroreflector embedding \u2014 CAPTURE, for short \u2014 delivers what\u2019s believed to be an unprecedented look at how animals move and behave naturally. This can one day lead to new understandings of how the brain functions.<\/p>\n

\u201cIt\u2019s really a technique that will end up informing a lot of neuroscience, psychology, drug discovery, [and disciplines] where questions of behavioral characterization and phenotyping are important,\u201d said Bence \u00d6lveczky<\/a>, a professor in the Harvard Department of Organismic and Evolutionary Biology<\/a>.<\/p>\n

Up to now, no technology has captured intricate details of an animal\u2019s natural behavior for extended periods of time. Recording precise animal movements during simple tasks, like pressing a lever, is possible, but because of the limited range of movements and behaviors scientists have been able to explore, it isn\u2019t clear if the insights gained can lead to a general understanding of brain function.<\/p>\n

CAPTURE starts the push beyond those limitations. It uses a series of custom markers that are attached to an animal, like tiny earrings, to track the position of the animal\u2019s whole body nonstop with a 12-camera array. This lets them digitally reconstruct the animal\u2019s skeletal pose and measure its normal movements for weeks at a time. With that data, the scientists can then develop new algorithms to create a foundational map of an animal\u2019s normal behaviors. These behavioral maps can then be compared with maps when the animal is in an altered state, giving researchers an exact look at even some of the most subtle differences.<\/p>\n

CAPTURE is described in a study recently published<\/a> in Neuron. Harvard postdoctoral fellow Jesse Marshall<\/a> led the project. Working with him and \u00d6lveczky were William Wang \u201920 and Diego E. Aldarondo, a student at the Graduate School of Arts and Sciences<\/a>. Right now, CAPTURE has been designed to work only on rats, but the team plans to expand to other animals in the future.<\/p>\n\r\n

\n
\n\t\t\t\t\n\t\t\t
\n\t\t\t\t\n\n\t
\"Artists<\/figure>\n\t\n\t\r\n\t\t\t<\/div>\n\t\t\t\n\t\t\t\t\t\n\t\t\t
\n\t\t\t\t\n\n\t
\"Photograph<\/figure>\n\t\n\t\r\n\t\t\t<\/div>\n\t\t\t\n\t\t<\/div>\n\n

Researchers attached markers to the mouse and let it actively explore a large arena (top). They then used a 12-camera array to track the position of the markers (lower left) and reconstruct the animal\u2019s pose (right). A photograph of the actual CAPTURE recording arena. <\/p><\/figcaption><\/figure>\r\n\n

\u201cThe large camera array works the way you would track actors for making Gollum [in the \u2018Lord of the Rings\u2019] or other types of animation,\u201d Marshall said. \u201cWe use it to precisely measure the position of their head and limbs and then there\u2019s a lot of analysis on top of that that enables the detection of behaviors and the isolation of other components [like behavioral organization].\u201d<\/p>\n

Marshall spent months researching the constraints and advantages of existing movement technologies before settling on motion capture. He then spent six months figuring out what type of marker would stay attached for long periods of time.<\/p>\n

\u201cTraditional markers from Hollywood are made of foam, and that wasn\u2019t going to fly with rats,\u201d Marshall said.<\/p>\n

Working with local veterinarians, the team designed custom body piercings made of specialized reflective glass and attached them to 20 locations on the rats\u2019 bodies. With the markers in place, they let the rats explore a naturalistic area and tracked their movements using the cameras 24\/7 for weeks.<\/p>\n

The researchers mapped out natural behaviors such as grooming, rearing, and walking and showed how those movements are organized into structured patterns, like the arrangement of words into sentences.<\/p>\n

The team then looked at how those behaviors and patterns changed in response to two stimulants, caffeine and amphetamine. As expected, both drugs caused the rats to move around more, but they did so in different ways. Caffeine amped them up, but they explored their cage normally. Amphetamine, on the other hand, shifted their behavior in novel ways and made them quite disturbed, Marshall said. They ran around in repeated, sequential patterns.<\/p>\n\r\n

\n\t
\n\t\t\t\t\t\t

More like this<\/h2>\n\t\t\t\t