Augmented Reality Slims Down With AI and Holograms

Augmented-reality glasses can upgrade our view of the world with digital images and data. However, even industry titans such as Google and Microsoft have failed to devise augmented-reality headsets that aren’t bulky, cause headaches, or both. Now, with a combination of 3D holograms, artificial intelligence, and the kind of physics that make invisibility cloaks possible, researchers have developed a wearable full-color 3D AR display as comfortable to wear as eyeglasses, a new study finds.

Whereas virtual reality (VR) headsets envelop a person’s field of vision, blocking their view of the real world, augmented reality (AR) headsets superimpose images onto the real world to create a mixed reality. Potential applications may include remote assistance for home repairs and other tasks, or displaying maps, directions, and other data to help users reach their destinations, find places to shop, or learn more about their surroundings.

“We’re taking a big step towards what I think could be the killer app of holography—ultrathin, lifelike, 3D VR/AR displays that will eventually be as thin as conventional eyeglasses.” —Gordon Wetzstein, Stanford University

And while the range of possible applications is broad and getting broader, widespread adoption of AR has faced major hurdles. For example, AR displays are ideally no larger than conventional eyeglasses for comfortable everyday use, but until now, they had to rely on complex, bulky arrays of lenses and other optics.

“Nobody wants to wear a big bulky headset—you get a sore neck, and it’s just not comfortable,” says Gordon Wetzstein, an associate professor of electrical engineering at Stanford University.

In addition, like conventional 3D displays, AR headsets typically create the illusion of depth by showing each eye a different 2D image. However, this strategy can lead to eyestrain.

Instead of using 2D images to simulate a 3D view, scientists have previously explored creating holographic video displays. A hologram is an image that essentially resembles a 2D window looking onto a 3D scene, leading to pictures with depth that people can view without discomfort.

“With our work, we’re taking a big step towards what I think could be the killer app of holography—ultrathin, lifelike, 3D VR/AR displays that will eventually be as thin as conventional eyeglasses,” Wetzstein says. “We’re not quite there yet, but our work takes a big step towards this vision.”

Prior attempts to create holographic AR displays faced challenges producing compact devices with high-quality 3D images. In the new study, researchers at Stanford, the University of Hong Kong, and tech giant Nvidia have overcome these problems with the help of optical metasurfaces, which are components engineered to bend light in unusual ways. Research on metasurfaces and other metamaterials has led to invisibility cloaks that can hide objects from light, sound, heat, and other types of waves, among other discoveries.

Optical metasurfaces contain structures with repeating patterns at scales that are smaller than the wavelengths of light they are designed to influence. The scientists created a metasurface that circumvents the need for complicated, ungainly optics in their AR display.

The new display shines red, green, and blue laser diodes at a spatial light modulator. This component is like a tiny display that controls the phase of the light directed at it to “basically create a small 3D hologram,” Wetzstein says. This hologram is then sent to a metasurface grating, which is made of lead-based glass with grooves 220 nanometers deep etched into it.

The scientists used AI to help design and optimize the structure of the metasurface. This made it very thin and very efficient at scattering light uniformly to where the researchers wanted it—at the eyes of viewers—instead of randomly scattering light in uncontrollable ways, Wetzstein says.

Another AI algorithm was used to compute how to turn 3-D images into high-quality holograms, Wetzstein adds. AI also helps calibrate the entire display, including the optics, electronics, and lasers, he says.

The result is a holographic AR display resembling standard eyeglasses that can display full-color 3-D moving images. “Our AI display is thinner that current AR displays and, importantly, it shows 3-D images to each eye,” Wetzstein says. “This lets you focus your eyes at different distances in the digital scene, which is a capability that is not supported by any existing AR headset.” The lack of focus cues in conventional AR and VR displays is what results in problems such as eyestrain, double vision, and reduced visual clarity, he adds.

In addition to AR displays, these new findings could also help lead to compact VR displays, says Gun-Yeal Lee, a postdoctoral researcher at Stanford. “Applications for VR displays might be simpler than AR displays, because AR requires both virtual image and see-through efficiency, making AR design more challenging.”

Currently the new AR display can overlay images only across a narrow field of view. Whereas each human eye can supply a roughly 130-degree field of vision, and both eyes working together can provide a nearly 180-degree field of vision when looking forward, the new device can display images only over a roughly 12-degree arc in front of the viewer. Although this is comparable to many commercial AR systems, the scientists note future research could improve this field of view—for instance, by using a material that is better at bending light than the glass currently used in the metasurface.

However, don’t expect to buy holographic AR eyeglasses anytime soon. Wetzstein cautions the technology they have developed “is not quite ready yet to go into mass production. It might take years to develop this proof-of-principle technology into a consumer product.”

The scientists detailed their findings online today in the journal Nature.