This morning I found myself looking all over my apartment for an interesting news article I wasn’t able to finish yesterday. The search eventually took me to the living room, which was littered with 3-D models. Thankfully, it took me only a couple of hand gestures to tidy up. Then I found the article I was looking for, floating in midair just behind the spot where a life-size model of an astronaut had been only moments before.
That narrative sounds like something out of a Harry Potter book. But it encapsulates the essence of how Microsoft’s HoloLens augmented-reality glasses provide a new kind of experience.
More polished than a prototype but still far from being a consumer product, Microsoft’s new vision for interacting with computers is now finally finding its way into the hands of people outside the company. Available as the US $3,000 Development Edition, the HoloLens headset, at just 580 grams, is a self-contained wearable Windows 10 computer. It’s able to map the wearer’s environment while displaying virtual objects anchored to that environment. The result is the best mixed-reality experience to date by a stand-alone, untethered device.
Powering each HoloLens is a 32-bit Intel Atom processor and Microsoft’s custom Holographic Processing Unit, or HPU. Using data generated by an inertial measurement unit, four spatial-mapping cameras, and a depth camera, the HPU creates a 3-D model of the surrounding environment. Apps can then use this model to position holograms without wasting the limited processing capacity of the energy-efficient Atom processor on handling sensor data. (As a Windows 10 device, HoloLens can run most 32-bit-processor-compatible Windows Universal Platform apps, which are available at the Windows Store. Nonholographic applications run as floating virtual screens that you place in your environment.)
By opting for such a specialized coprocessor, Microsoft has traded some flexibility for increased power and thermal efficiency. Still, the HoloLens would not be feasible without it. That is because the form factor of a head-mounted computer puts serious constraints on battery size and how much heat can be dissipated without causing discomfort.
Despite these limitations, Microsoft’s efforts amount to the most impressive device I have had a chance to work with in a long time. This is not a statement I make lightly, because my position at Lucidscape Technologies, a research lab specializing in virtual and augmented reality, affords me the opportunity to experience a wide range of related technology on a daily basis.
Once initial setup and calibration are complete, the HoloLens experience starts with a hand gesture that invokes the holographic equivalent of the Windows start menu. The pointer is controlled by your gaze, and clicking is done with a finger gesture. Most items can be dragged through space using a pinching gesture. You enter text by using a gaze-activated keyboard. The gesture recognition is very reliable, although the number of actions appears to be fixed—meaning, at least for now, that developers will not be able to define custom gestures.
Present is also Cortana, Windows’ voice-activated personal assistant. In theory, most hand gestures and keyboard input can be replaced with voice commands. However, I have given up using Cortana due to its unreliability.
While not perfect, the HoloLens’s ability to anchor apps and holograms in the real world is an impressive technological feat. If you place a hologram on top of a desk, it will stay where you put it as you walk around the room. The HoloLens also builds a catalog of the different rooms it maps. This means you can return to a room after several days away and you’ll find your holograms and apps where you left them.
There are currently only a handful of made-for-HoloLens apps of varying utility. Skype for HoloLens allows the person on the other end of the conversation to draw lines that appear as holograms in your field of view. HoloStudio is a fun-to-use, albeit limited 3-D editor with support for 3-D printing. There are three games you can adapt to your real-world environment, which gives you a taste of how future holographic titles might turn your living room into a video-game level.
The bottom line is that despite the high cost and shortcomings of the Development Edition, it is clear that future iterations of the HoloLens could profoundly change how we relate to our computers and even to our environment. Wearing a HoloLens feels like having a glimpse of an unfinished future.
This article appears in the June 2016 print issue as “The Future of Augmented Reality: Hololens.”