Camera-Filled Dome Recreates Full 3-D Motion Scenes

The same technique could theoretically recreate scenes captured by hundreds of spectator cameras

2 min read
Camera-Filled Dome Recreates Full 3-D Motion Scenes
Photo: Carnegie Mellon University

Those thrilling moments when a soccer player kicks home the winning goal in the World Cup final or Beyonce debuts new dance choreography in concert might someday be recreated in full 3-D motion down to the smallest piece of confetti and played back from almost any angle. Such a possibility comes from a new motion-capture technique capable of reconstructing scenes captured by more than 500 video cameras mounted inside a two-story geodesic dome.

The new technique comes from Carnegie Mellon University researchers working in the Panoptic Studio—a video lab with a camera system capable of capturing 100,000 different points in motion at any time. Researchers developed a technique that uses consistent motion patterns as a cue for identifying and tracking certain points on an object captured by cameras. And it all works without the need for physical markers, such as those used by Hollywood motion-capture systems to translate the acting performance of Andy Serkis into the movements of the ape leader Caesar in the newest "Planet of the Apes" films.

"At some point, extra camera views just become 'noise,'" said Hanbyul Joo, a Ph.D. student in the Robotics Institute at Carnegie Mellon University, in a press release. "To fully leverage hundreds of cameras, we need to figure out which cameras can see each target point at any given time."

The camera system uses the motion patterns to figure out which of the Panopticon's 480 video cameras—plus an additional 30 high-definition video cameras—arrayed around the geodesic dome have the best views of a particular moving object. If many cameras are tracking a particular point on a person's body as moving to the right, a camera that detects motion moving in the opposite direction is likely picking up another object or can't see that particular point from its angle. The system can then figure out which video feeds to use or ignore to reconstruct the 3-D motion of that particular point.

Such technology could prove a huge help on feature film sets. Current motion-capture systems used by Hollywood films such as "Dawn of the Planet of the Apes" also use an array of many cameras, but specifically track certain markers placed on the bodies of actors. Carnegie Mellon's system has no need for using such markers, which could enable future Hollywood productions to create both digital actors and scenes.

Other researchers have shown that they can reconstruct 3-D still scenes based on using a large number of cameras, including the cameras of smartphones. But Carnegie Mellon's system has proven capable of reconstructing incredibly detailed, full-motion scenes such as a person tossing confetti in the air that tracks the individual confetti pieces as they get fed into a fan.

The researchers also raised the possibility of a similar system harnessing the power of video cameras in spectators' phones at events such as concerts and sporting events. But that would depend on somehow getting permission to access everyone's personal devices.

The Conversation (0)

Why Functional Programming Should Be the Future of Software Development

It’s hard to learn, but your code will produce fewer nasty surprises

11 min read
A plate of spaghetti made from code
Shira Inbar

You’d expectthe longest and most costly phase in the lifecycle of a software product to be the initial development of the system, when all those great features are first imagined and then created. In fact, the hardest part comes later, during the maintenance phase. That’s when programmers pay the price for the shortcuts they took during development.

So why did they take shortcuts? Maybe they didn’t realize that they were cutting any corners. Only when their code was deployed and exercised by a lot of users did its hidden flaws come to light. And maybe the developers were rushed. Time-to-market pressures would almost guarantee that their software will contain more bugs than it would otherwise.

Keep Reading ↓Show less