The July 2022 issue of IEEE Spectrum is here!

Close bar

AI Creates Fake Obama

Videos of Barack Obama made from existing audio, video of him

3 min read
Two pictures of Barack Obama side-by-side
Photo: University of Washington.

Artificial intelligence software could generate highly realistic fake videos of former president Barack Obama using existing audio and video clips of him, a new study [PDF] finds.

Such work could one day help generate digital models of a person for virtual reality or augmented reality applications, researchers say.

Computer scientists at the University of Washington previously revealed they could generate digital doppelgängers of anyone by analyzing images of them collected from the Internet, from celebrities such as Tom Hanks and Arnold Schwarzenegger to public figures such as George W. Bush and Barack Obama. Such work suggested it could one day be relatively easy to create such models of anybody, when there are untold numbers of digital photos of everyone on the Internet.

The researchers chose Obama for their latest work because there were hours of high-definition video of him available online in the public domain. The research team had a neural net analyze millions of frames of video to determine how elements of Obama's face moved as he talked, such as his lips and teeth and wrinkles around his mouth and chin.

In an artificial neural network, components known as artificial neurons are fed data, and work together to solve a problem such as identifying faces or recognizing speech. The neural net can then alter the pattern of connections among those neurons to change the way they interact, and the network tries solving the problem again. Over time, the neural net learns which patterns are best at computing solutions, an AI strategy that mimics the human brain.

In the new study, the neural net learned what mouth shapes were linked to various sounds. The researchers took audio clips and dubbed them over the original sound files of a video. They next took mouth shapes that matched the new audio clips and grafted and blended them onto the video. Essentially, the researchers synthesized videos where Obama lip-synched words he said up to decades beforehand.

The researchers note that similar previous research involved filming people saying sentences over and over again to map what mouth shapes were linked to various sounds, which is expensive, tedious and time-consuming. In contrast, this new work can learn from millions of hours of video that already exist on the Internet or elsewhere.

One potential application for this new technology is improving videoconferencing, says study co-author Ira Kemelmacher-Shlizerman at the University of Washington. Although teleconferencing video feeds may stutter, freeze or suffer from low-resolution, the audio feeds often work, so in the future, videoconferencing may simply transmit audio from people and use this software to reconstruct what they might have looked like while they talked. This work could also help people talk with digital copies of a person in virtual reality or augmented reality applications, Kemelmacher-Shlizerman says.

The researchers note their videos are currently not always perfect. For example, when Obama tilted his face away from the camera in a target video, imperfect 3-D modeling of his face could cause parts of his mouth to get superimposed outside the face and onto the background.

In addition, the research team notes their work did not model emotions, and so Obama's facial expressions in the output videos could appear too serious for casual speeches or too happy for serious speeches. However, they suggest that it would be interesting to see if their neural network could learn to predict emotional states from audio to produce corresponding visuals.

The researchers were careful to not generate videos where they put words in Obama's mouth that he did not at some other time utter himself. However, such fake videos are “likely possible soon,” says study lead author Supasorn Suwajanakorn, a computer scientist at the University of Washington. 

However, this new research also suggests ways to detect fake videos in the future. For instance, the video manipulation the researchers practiced can blur mouths and teeth. “This may be not noticeable by human eyes, but a program that compares the blurriness of the mouth region to the rest of the video can easily be developed and will work quite reliably,” Suwajanakorn says.

The researchers speculated that the link between mouth shapes and utterances may be to some extent universal for people. This suggests that a neural network trained on Obama and other public figures could be adapted to work for many different people.

The research was funded by Samsung, Google, Facebook Intel and the University of Washington. The scientists will detail their findings [PDF] on Aug. 2 at the SIGGRAPH conference in Los Angeles.

The Conversation (0)

How the U.S. Army Is Turning Robots Into Team Players

Engineers battle the limits of deep learning for battlefield bots

11 min read
Robot with threads near a fallen branch

RoMan, the Army Research Laboratory's robotic manipulator, considers the best way to grasp and move a tree branch at the Adelphi Laboratory Center, in Maryland.

Evan Ackerman
LightGreen

“I should probably not be standing this close," I think to myself, as the robot slowly approaches a large tree branch on the floor in front of me. It's not the size of the branch that makes me nervous—it's that the robot is operating autonomously, and that while I know what it's supposed to do, I'm not entirely sure what it will do. If everything works the way the roboticists at the U.S. Army Research Laboratory (ARL) in Adelphi, Md., expect, the robot will identify the branch, grasp it, and drag it out of the way. These folks know what they're doing, but I've spent enough time around robots that I take a small step backwards anyway.

This article is part of our special report on AI, “The Great AI Reckoning.”

The robot, named RoMan, for Robotic Manipulator, is about the size of a large lawn mower, with a tracked base that helps it handle most kinds of terrain. At the front, it has a squat torso equipped with cameras and depth sensors, as well as a pair of arms that were harvested from a prototype disaster-response robot originally developed at NASA's Jet Propulsion Laboratory for a DARPA robotics competition. RoMan's job today is roadway clearing, a multistep task that ARL wants the robot to complete as autonomously as possible. Instead of instructing the robot to grasp specific objects in specific ways and move them to specific places, the operators tell RoMan to "go clear a path." It's then up to the robot to make all the decisions necessary to achieve that objective.

Keep Reading ↓Show less