Digital doppelgängers of anyone can be generated from hundreds of images of them collected from the Internet, from celebrities such as Tom Hanks and Arnold Schwarzenegger to family members and historical figures, say researchers at the University of Washington in Seattle.
Their work suggests that creating 3-D digital models of people based on anything from family photo albums and videos to historical collections could one day be possible. Such models could be controlled like puppets and made to do and say anything.
In fact, it could be relatively easy to create such models of “anyone in the future, when there are tons of digital photos,” says study lead author Supasorn Suwajanakorn, a computer scientist at the University of Washington. “Photos on Facebook, for some users, are already enough to reconstruct their controllable models.” Suwajanakorn and his colleagues Steven Seitz and Ira Kemelmacher-Shlizerman will detail their findings on 16 December at the International Conference on Computer Vision in Chile. Their research was funded by Samsung, Google, Intel and the University of Washington.
Benign uses of such technology could include using augmented reality (AR) glasses to see 3-D models of family and friends at home, or donning virtual reality (VR) goggles to interact with superstars and newsmakers in games, the researchers say.
However, such technology could also make it easier to fake photos and videos of people, making it appear as though they did and said things they never did.
Existing technologies for creating detailed 3-D models of people often rely on painstakingly capturing their every angle. The researchers at the University of Washington instead sought to generate animatable models of people based solely on random collections of existing images.
The scientists digitally reconstructed celebrities and notables such as Daniel Craig, Neil Patrick Harris, George W. Bush and Barack Obama using roughly 200 images gleaned of each from the Internet. These pictures were taken over time in various scenarios and poses.
"We're trying to find a way to capture the essence of a person," Suwajanakorn says.
“The time when a rendering is indistinguishable from the real could be quite scary,” Suwajanakorn says. “I think people will start treating videos with caution just like when it became easy enough to fake a photo with Photoshop.”
This strategy first calls for the creation of a 3-D model of a person that averages their images. The team developed ways to modify the pose and expression of that model in tiny ways, such as those that occur when a person smiles or looks puzzled.
The researchers demonstrated they could use a photo or video of one person to control the digital model of another person, as if the model were a puppet.
This method can work on images of faces with any expression, in any pose, and under any lighting conditions—which makes it compatible with just about any snapshot one can expect to find in the average person's photo collection. “Unlike other 3-D reconstruction techniques, it never requires the subject to be in front of a camera, so it has potential to scale to many people,” Suwajanakorn says.
Suwajanakorn notes that he and his colleagues’ doppelgänger technique currently works only on photos of faces. “Right now, we're missing voice, the rest of the head, or specific ways a person does certain things,” he says. “However, these data are more difficult to gather, especially for normal people, and can raise some privacy concerns.”
Charles Q. Choi is a science reporter who contributes regularly to IEEE Spectrum. He has written for Scientific American, The New York Times, Wired, and Science, among others.