By Senior Editor Tekla S. Perry
A new motion-capture technique from the person who helped bring you QuickTime is turning actors into digital characters capable of anything imaginable.
Tekla S. Perry
I've been captured. Motion captured, that is, with one of the new types of motion-capture technology that may soon revolutionize the movie and videogame industry and make living actors indistinguishable from those that live inside computers. You can read all about this coming revolution in "Ready for Their Close-Ups" (by Eric Pavey). And you can see my digital head in action in this month's Back Story column, "Face Time".
To turn myself into a new-generation Max Headroom, I drove from my Palo Alto office to San Francisco's Dogpatch district on a rainy day in February. There, a company called Mova is putting actors in a cage and turning their performances into digital files that can be altered at a director's will, making the actors older, younger, or into space aliens.
The neighborhood, a mix of old factories, auto body shops, construction supply yards, and a power distribution substation a few blocks from the city's waterfront, isn't exactly posh. After a long climb up several flights of concrete stairs (the elevator was broken), I entered Mova's studios, where company founder Steve Perlman and several technicians and production assistants waited.
Steve Perlman? That name ought to ring a bell. Remember WebTV, one of the first companies that tried to bring entertainment content from the Internet to the home television screen. Perlman co-founded it, and Microsoft eventually bought it. How about QuickTime? Perlman led the development of the technologies that became that software within Apple, and then went on to join General Magic before founding three investor-backed companies. Now, he runs Rearden Companies, formerly Rearden Steel, a technology incubator. And Mova's new motion-capture system, called Contour, is one of the technologies Rearden incubated.
PHOTOS: MARK RICHARDS; MOVA
Given that I had to spend about an hour in make-up—Contour records and analyzes the pattern of marks made on an actor's skin when phosphorescent makeup is dabbed on with a rough, exfoliating sponge—it was nice to have Perlman there. (Click on this link to view a slideshow of the Contour process.) Because besides being an incredibly prolific entrepreneur, he is an entertaining storyteller and didn't need much prompting to tell me the story behind Mova.
Perlman always loved movies and animation. He made his first Claymation films back in junior high. He bought a motion-capture system in 1997, the kind that requires sticking little balls all over an actor and shining lights that would reflect back from the balls and be recorded as dots. He played with motion capture as more of a hobbyist than a businessman, getting more serious in 2000 when he started doing motion capture under contract to movie and game companies, eventually spinning that business out of Rearden into Mova. And he began to try to invent the next generation of motion-capture technology, one that wouldn't use markers and would be optimized for capturing facial motion and expression.
"It was an Edisonian journey," Perlman told me, reflecting on all the dead-end roads Edison took in his development of the light bulb. "We tried ultrasonics, seeing if we could measure sound waves. We tried sending light waves and trying to measure the time of flight. We tried optical flow, which is a technology that tracks pores in the face, the problem with that is actresses spend a lot of time making sure you can't see the pores. We tried retroreflective materials, but that involved tiny glass beads, which wouldn't be very safe to use on skin."
At this point, Perlman and his research team were working in an office subleased from Android, a start-up company in downtown Palo Alto. They used the central, windowless space, because, at times, they needed to be able to work in the dark. In August 2005, their officemates moved out, and another start-up, the now famous Facebook, moved in. Mova's development continued uninterrupted surrounded by the chaos of the nascent Facebook.
The group finally settled on phosphorescent Halloween makeup, typically yellow, but which, mixed with theatrical makeup, approximated a flesh tone. To give the best information on facial movement to the computer, the makeup artist had to apply the makeup to the actors face in a noticeable but random pattern. The development team tried an airbrush, which made nice spots in some areas but dense blobs in others. They tried paintbrushes, and a variety of sponges, finally settling on an exfoliating sponge. They unveiled their technology last summer, moved it up to their San Francisco studios, and are now working full out on projects for movie and videogame companies.
Back in Palo Alto, development on the next generation of Contour continues, currently in Perlman's garage. The company's lease on the space ran out at the end of last year, and Facebook booted them out. They're looking for new Palo Alto office space now.
"We like being there for a bunch of reasons," Perlman said. "Palo Alto has the northernmost Fry's. [Fry's is a chain of electronics stores that is a mecca for Silicon Valley engineers.] We rely on Fry's enormously for quick access to parts, processors, boards, and connectors. Palo Alto is also a Caltrain stop, and we have another stop here near our studio."
Perlman could have talked all day, but the makeup on my face was completely dry and I was ready to be captured. Perlman stepped aside, a director slammed a clap-board in front of my face, and it was take one. I said my piece, and, after about five takes, made a series of extreme faces; those will go to production studios as test data. Could be some famous director will decide to turn me into a strange sea creature or alien.
Newsletter Sign Up
Sign up for the Tech Alert newsletter and receive ground-breaking technology and science news from IEEE Spectrum every Thursday.
IBM's Brain-Inspired Chip Tested for Deep Learning
IBM's TrueNorth computer chip shows it can do deep learning despite not having been designed for it
Novel Nanomaterial Could Yield Lossless Charge and Energy Transport
First observation of theorized phenomenon has implications for spintronics and chip thermal management
How to Build a Theremin You Can Play With Your Whole Body
The TeraRanger One range finder lets you make a musical instrument with infrared light
HERE Mapping Service to Automate Finding a Parking Spot
Sensors in Audi, BMW, and Mercedes cars provide data for a service to launch in 2017
“Too Cheap to Meter” Nuclear Power Revisited
After 50 years of operation, nuclear electric generation is still having teething pains
China Unveils World's Largest Single-Dish Radio Telescope
The 500-meter-wide FAST telescope boasts greater sensitivity and a deformable mirror
The 2017 Acura NSX: A Hybrid Supercar
Honda’s new Acura NSX is the first to marry a V-6 engine to three electric motors for high-speed steering
The Megaprocessor Laughs at Your Puny Integrated Circuits
The wondrous insanity of a 42,300-transistor CPU the size of a room
Flexible Nonvolatile Memory Just Got a Lot Closer
A novel molecule changes the game in flexible nonvolatile memory, potentially ushering a new era in wearable electronics
Video Friday: LEGO Drone Kits, Robots in the Desert, and Pepper Learns New Tricks
Your weekly selection of awesome robot videos
The Internet of Fewer Things
Early predictions of 50 billion connected devices by 2020 are being scaled back
Clinton Calls for Tech Firms to Fight Terrorism, but Students Could Be the Key
Stanford’s “Hacking for Defense” class to be offered at 13 universities next year; U.S. Defense Department gets ready to send students classified problems
Bionic Arms Get a Thought-Control Upgrade
Pattern recognition software enables amputees to control prostheses in a natural and intuitive way
LED Streetlights Are Giving Neighborhoods the Blues
Early adopters of LED street lighting are struggling with glare and light pollution
D-Wave Founder's New Startup Combines AI, Robots, and Monkeys in Exo-Suits
Quantum computing pioneers want to patent AI telerobotics controlled by humans—and monkeys
Fusing of Organic Molecules With Graphene Opens Up New Applications
A new manufacturing approach for hybrid materials provides an avenue for new device designs in a range of electronic applications
Tesla's Massive New Autopilot Update Is Released, Promising Safer Driving
Autopilot 8.0 makes more use of cars' existing radar and works harder to keep the driver focused on the road
Cheap Lidar: The Key to Making
Self-Driving Cars Affordable
Startups and established players think they have the key to inexpensive lidar
Electrostatic Glider Could Maneuver Around Asteroids Without Expending Fuel
The charged dust around airless bodies provides a surfable environment for tiny electric spacecraft
BIG-i Social Home Robot Has a Big Eye, Launches on Kickstarter
A unique mobile design and easy voice programming help this social robot stand out