Sulon Cortex Headset Seeks to Meld Real and Virtual Worlds

This display superimposes a virtual environment on your physical surroundings

4 min read

Evan Ackerman is IEEE Spectrum’s robotics editor.

Sulon Cortex Headset Seeks to Meld Real and Virtual Worlds

Last week at the 2014 Game Developer's Conference in San Francisco, three promising new virtual reality headsets were introduced. One of them, from a Canadian startup called Sulon Technologies, caught our eye because they decided to tackle one of the most challenging issues that VR has to conquer: when you're wandering around a virtual environment, how do you keep from running into real walls?

As much progress as we've seen with virtual reality hardware in the past year or two, there's one major constraint to nearly every visual immersion system: they can't handle movement. You can look around, and perhaps move your head, but moving your virtual body in a VR world generally means that you have to sit still and fiddle with a game controller. Options for full body motion are limited: for $500, you can strap yourself into a Virtuix Omni, using low-friction shoes to walk and run in place. The Omni is an example of a compromise that works (sort of), but it's certainly not the Holodeck that Star Trek promised us. Fundamentally, the problem is that virtual reality is all about putting your mind into a place that doesn't actually exist, while your body stays right where it is.

Sulon Technologies is taking a somewhat different approach to the problem of moving around virtual worlds. Rather than give you complete freedom of movement, Sulon instead is combining what's real with what's not, overlaying a virtual environment on your physical surroundings. The Cortex is an immersive headset with a head position tracker and a game controller tracker, but what's unique about Sulon's system is that it also includes a complex sensor package (pictured below) that actively maps the world around you.

That cylindrical thing on the bottom is a laser scanner, and there are also ultrasonic sensors and a sensitive inertial measurement unit (IMU). All of the sensor fusion processing and map generation takes place on a computer that sits at the back of your head. Once the Cortex figures the size of the room you're in, it pipes that information to the display that you're looking at, which overlays an augmented or completely virtual environment. The result is that you're free to walk around as much as you want in real life, because the walls and obstacles that you see in the virtual world match up exactly with the walls and obstacles that you'll experience in the real world. 

The mapping process itself, which has to happen separately if you want to take advantage of large or complex spaces (like multiple rooms or floors of a house), involves walking around while the system performs SLAM (simultaneous localization and mapping). Once you're finished, you save the map, and from that point forward you can access that environment, and so can any virtual or augmented reality games that you play. 

At the Game Developer's Conference in San Francisco, we got to try this thing on for ourselves. As you can see from the photos, it's very much a prototype, but it's functional in all of the ways that matter. And it works as promised: putting the Cortex on turned the small room that we had our demo in into a room of the exact same size near the top of a castle. We could walk over to a window and look out at the view, and putting a hand out to touch a virtual window meant touching a real wall in the real world. It didn't feel like we were touching glass, of course, but the effect was compelling nonetheless, and having the freedom to walk around was a massive increase in the overall immersive experience.

The Cortex does augmented reality, too: a camera on the front of the headset can stream a first-person view to the display, which can recognize elements like doors and windows and add contextually aware characters or other elements to the scene. You're still restricted by the dimensions of whatever room you're standing in, of course, but Sulon is playing around with ways to maybe get past that, like using visual feedback to trick you into walking in circles when you think you're walking straight.

The highlight of the Cortex is the mapping technology and the full-body movement that it enables. The visual experience (all of the hardware and software that combine to try to fool your eyes into thinking that they're looking at something real) isn't nearly as impressive in its current iteration. This will definitely improve, however: rather than locking themselves into a specific screen, Sulon opted to go with a modular system. The screen that you look at in the Cortex is actually just a smartphone with a six-inch, 2K resolution display, and you can pop it in and out of the headset:

With this modularity, Sulon hopes to both make it easier for people to afford the Cortex (by providing their own screen and brain), and to piggyback on the rapid improvements in consumer smartphone technology. As 4K and 8K smartphones become available (with faster processors and more powerful graphics engines), you'll be able to slide them right into your Cortex, instantly upgrading it.

For now, Sulon isn't looking to sell the Cortex directly to consumers. Instead, developer kits are available to pre-order now, and Sulon is hoping that game developers will start getting creative with the new possibilities that the mapping tech enables. For $500, you'll get something that looks like the picture above, shipping in Q4 of this year.

[ Sulon Technologies ]

The Conversation (0)