Perfecting digital imaging

Computer graphics and digital video lag behind reality; despite advances, the best software and video cameras cannot seem to get computer-generated images and digital film to look exactly the way our eyes expect them to.

But Hanspeter Pfister and Todd Zickler, computer science faculty at the Harvard School of Engineering and Applied Sciences (SEAS), are working to narrow the gap between “virtual” and “real” by asking the question: How do we see what we see?

Between them, Pfister and Zickler are presenting three papers this week at SIGGRAPH 2013 (the acronym stands for for Special Interest Group on GRAPHics and Interactive Techniques), the 40th International Conference and Exhibition on Computer Graphics and Interactive Techniques.

Realistic soap

One project led by Zickler, the William and Ami Kuan Danoff Professor of Electrical Engineering and Computer Science, tried to find better ways to mimic the appearance of a translucent object, such as a bar of soap. The paper elucidates how humans perceive and recognize real objects and how software can exploit the details of that process to make the most realistic computer-rendered images possible.

“If I put a block of butter and a block of cheese in front of you, and they’re the same color, and you’re looking for something to put on your bread, you know which is which,” says Zickler. “The question is, how do you know that? What in the image is telling you something about the material?”

His hope is to eventually understand these properties well enough to instruct a computer with a camera to identify the material an object is made of and how it should be handled — how much it weighs or how much pressure can be safely applied to it — the way humans do.

Zickler’s co-authors were Ioannis Gkioulekas, a graduate student at SEAS; Bei Xiao and Edward H. Adelson of the Massachusetts Institute of Technology; and Shuang Zhao and Kavita Bala of Cornell University.

For a full description of the three papers, visit the SEAS website. The Association for Computing Machinery SIGGRAPH conference continues through Friday in Anaheim, Calif. The three papers will be published in ACM Transactions on Graphics.