Tilt Brush: The Killer App for VR

Tilt Brush could be a game changer for virtual reality

There was a certain amount of media hand-wringing surrounding January’s CES, the big kahuna of consumer electronics trade shows held annually in Las Vegas. Where was the big game-changing product? CES was so over.

It’s true that it’s been a couple of years since, say, smart watches arrived on the scene, and a full five years since the last really big new product category—tablets—was introduced. But new categories tend to leave an innovation hangover: In a rush to grab market share, funding and engineering talent tend to get focused on me-too products. In contrast, this year’s exhibitors were presenting a lot of different ideas pushing in a lot of different directions.

The result can be bemusement—for example, there was a US $300 “game console” for dogs, a machine for writing names in coffee foam, and a wristband that lets you give yourself electric shocks to break bad habits. But there were also a lot of ideas with the potential for a big impact, such as a wearable two-way voice translator that doesn’t rely on an Internet connection, a design for a two-dimensional tactile display for the visually impaired (the makers hope to have a working prototype later this year), and smart-fabric ideas that could help push fashion tech forward.

But the most interesting emerging trend was in virtual reality. Not so much in the hardware, the basic outlines of which have pretty much revolved around smartphone-display-based headsets for several years now. What was interesting was that people are coming up with engaging content and applications for VR.

For example, the Lowe’s chain of home improvement stores was showing off a system it is testing in half a dozen stores. Customers can design a kitchen or bathroom and see how it would look to stand in the room, with an employee making adjustments in real time. When I tested it out, the result was good enough that I kept catching myself trying to lean against a purely virtual countertop. Another company, with the aid of a chair that can tilt a few centimeters forward and back and side to side, provided a simulated carnival ride that was somatically close enough to the real thing to produce literal white knuckles.

But the killer app was something I almost missed, coming across it outside the main halls in the last minutes of the show. This was a demo of HTC’s Vive VR system—which uses two pole-mounted locating beacons and hand controllers in addition to a headset—running Google’s Tilt Brush software.

Those of us over 40 will recall the era of purely character-based interfaces. Remember then the psychological impact the first time you actually got to use MacPaint or Microsoft Paint and you spent an hour just trying out each type of brush?

Using Tilt Brush was like that, only more so. The system creates a 3-D space—the boundaries are demarcated in your view with a light-grid pattern in best holodeck tradition—in which you are free to walk around. One hand controller becomes a universal palette, which you rotate with your wrist to call up things like a color picker, brush selection, and so on. The other controller becomes your brush: Pull its trigger and “paint” begins to appear at your hand’s location.

The tracking is exquisite: As your hand traces a path through space, the brush follows perfectly. Sketch out a shape, and then walk around it to look at it from another side. It’s incredibly intuitive (and a lot easier than when we all tried to write our names with a mouse in MacPaint). Despite my limited artistic abilities, within a few moments I had painted a rough 3-D volcano, complete with glowing lava spilling down the sides. Were it not for the line of folks waiting their turn, I could have spent hours in there.

The system is expected to ship later this year, with pricing to be announced. Apart from its value to artists of all stripes, unless the system is a complete flop, it should only be a matter of time before it interoperates with existing computer-aided design software, bringing 3-D design out of the realm of high-end outfits.

This article appears in the March 2016 print issue as “CES and the Future of VR.”

This article is based in part on material published online as part of IEEE Spectrum’s live CES coverage.