Illustration: Greg Mably
A lot of us have recently become familiar with what what’s being called Zoom fatigue—the drained feeling that follows hours spent performing before coworkers or clients or superiors, all while scanning a sea of faces that for some of us evokes the opening sequence from “The Brady Bunch.” Seeing only disembodied squares of video, you have to work hard to make out the social cues that might reveal anyone else’s feelings.
Some people have retreated to using the telephone, which allows you to relax in a way that videoconferencing doesn’t. Even professionals with significant experience in television broadcasting never truly relax when they’re on camera. If we don’t want to melt down under such stresses, we need tools better suited to meeting one another remotely.
Practice, practice, and more practice remain the only answer to video stage fright. But the solution to mental strain during Zoom meetings could be at hand.
Back in 2016, I got a boxing lesson from a world-famous coach in a converted warehouse in Los Angeles. Only he wasn’t really there. His three-dimensional likeness had been captured elsewhere using a technique called videogrammetry (or confusingly, holography, even though it has nothing at all to do with coherent light or interference).
Encircled by banks of high-resolution cameras, that coach offered an aspiring boxer advice on stance and balance and what makes for a good right hook. Vast cloud-computing resources crunched those high-resolution images, extracting depth measurements from their different perspectives (much as your eyes do).
Peering through VR goggles at my end, I saw a 3D representation of the coach. He was just an animated video sculpture—less resolved than in the flesh—but it still seemed almost as if he were standing next to me.
Because I sensed his physical presence, I subconsciously shifted my pose and balance to mirror his. That’s human behavior at its most empathetic: following cues embedded in another person’s posture or movements—things we don’t typically perceive when videoconferencing.
That demo blew my mind: I knew I’d experienced the future.
And just a few months after my go in the sparring ring, Microsoft revealed its Holoportation demo, a real-time videogrammetry system using its Kinect2 depth cameras and HoloLens AR displays. Data from an array of Kinect2s was piped through a room full of high-powered servers for real-time image processing and compression before being sent to the HoloLens. The technology gave a tantalizing preview of the possibilities, but with a hefty price tag, making it just too difficult to pull off this process well.
Today, we have lidar-equipped iPad Pros and Azure-connected Kinects available by the cartload—an infrastructure for videogrammetry making everything that was once pricey and difficult now cheap and easy. We’re at a moment analogous to 2014, when two Google engineers used a pizza box, a pair of plastic lenses and a bit of software to create Google Cardboard, instantly giving the world a path to a half billion VR systems that were good enough for many applications.
The moment is ripe for another such innovation, one that gives us the sense of human presence that we often can’t safely offer to one another in person at the moment. We need to be there without being there, to touch without touching. Videogrammetry, however imperfect, will bring some much-needed depth—figuratively as well as literally—to the experience of meeting remotely.
This article appears in the August 2020 print issue as “Being There, Virtually.”
