I hate CES. It’s crowded and noisy, with a frantic energy that borders on desperate. That energy is stoked by everything that’s wrong (and right) about Las Vegas. For the press, it’s essentially impossible to find everything worth covering among 3,600 exhibitors announcing 20,000 new products staged across more than 18.5 hectares (two million square feet) of convention space. But we kill ourselves trying anyway.
Why do I keep coming back? Because of the tiny handful of companies that are using technology to make our lives not just better, but magical. I also do it in order to see those companies transform their tech from trade show prototypes to consumer products. This year, Tobii and SteelSeries are demonstrating gaze tracking interfaces that have made just such a leap.
At CES 2011, Tobii demonstrated an arcade cabinet featuring a classic game of Asteroids that you could play with your eyes. Using infrared cameras and illuminators, the game system tracked the player’s pupils to determine exactly where on the screen his or her gaze was focused. Gamers were able to blow up incoming asteroids just by looking at them. It’s an effortless and intuitive way to interact with a computer, and it feels like magic.
A year later at CES, Tobii had more demos, this time allowing gaze control of a desktop environment. Imagine a gallery of thumbnailed pictures: whichever picture you’re looking at expands to fill half the screen, and then shrinks again when you look away. If you could control a computer directly with your brain, it would be exactly like this. Here’s how I tried to describe it at the time:
The kind of experience that you get with Tobii is a little hard to describe in pictures or video, but it's easily the most natural and fluid interface we've ever used. You don't have to think about anything at all, it's simply that wherever you're looking, that's where the cursor is. It's forgiving, too: there's no need to be at an exact distance from the screen or dead center or perfectly still. You just sit there like you normally do and look at stuff. It's incredible.
Basically, a mouse is just a klunky way of telling the computer where your attention is focused, which is what your eyes do naturally. The eye tracker conrols the cursor with your gaze, effectively eliminating the middleman. You still push a button (on the keyboard) to denote an action (a click), but your hands can stay on the keyboard. And since the computer can tell where your attention is, as opposed to just where the mouse cursor happens to be, it can much more easily figure out how to intuitively respond to you, such as by automatically scrolling up when you’ve read to the bottom of your screen.
By 2013, Tobii was selling development kits of its eye tracking hardware for $1,000, and it became immediately obvious that the technology was going to have a huge impact on the gaming industry. At the 2013 Game Developer’s Conference, we saw an absolutely insane demo of gaze control in StarCraft II. It wasn’t a huge surprise when, at CES 2014, Tobii announced a partnership with gaming peripheral manufacturer SteelSeries. And now, a year later, Tobii and SteelSeries have released the Sentry, a consumer eye tracker that you can (finally) buy.
Initially, the Sentry is targeting hardcore gamers and game streamers, and is intended to passively collect statistics and build heat maps of where a gamer’s attention is focused while he or she plays. For example, Sentry can tell you that you never check your radar, while pro gamers check their radar every 10 seconds. Or, it can show you how much time you spend fixated on a single point on the screen and not paying attention to anything else that’s going on.
Of course, what we all really want is gaze control, not just gaze tracking. But there isn’t going to be a big market for games that offer gaze control until lots of gamers have the hardware, and gamers need a reason to buy the hardware before there are games that make good use of it.
Having said that, there’s an enormous amount of potential for gaming. Imagine games where characters react differently to you depending on how much attention you give them. Or puzzle solving games where what you look at changes how the game evolves. Or first person shooters that allow you to move, look, and aim in all different directions, and to shoot at something as fast as you can see it.
If gaming isn’t your thing, the Sentry can sit atop just about any application you want it to, replacing mouse movement with eye movement. Personally, I’m looking forward to not having to continually move my hands from keyboard to mouse or trackpad and then back again. The interface is easy to program even if you have no coding experience, and you can set up custom gaze-directed actions such as zooming or scrolling.
Even though the Sentry doesn’t yet offer control, Tobii and SteelSeries say that it’s totally forward compatible, so that as soon as games (or other software) with eye tracking support come out, this version of the Sentry will take advantage of that ability immediately. The companies are predicting that lots of software will be leveraging the Sentry’s capabilities in 2015, and having used it, I’m sure they’re right. It’s one of those new and amazing technologies that we’re going to look back on in a few years and wonder how we ever got along without it.
The SteelSeries Sentry Gaming Eye Tracker attaches to your monitor with a slick magnetic system that makes it easy to remove, and can be calibrated to work on desktops or laptops. The only requirement is a USB 3.0 port. It’s available now (or within the next few days), and should ship by early February. At $199, it’s not the cheapest peripheral you’ll ever buy. But that’s not too much to ask for a piece of the future.