I’m generally not the person you want testing your virtual, augmented, or otherwise “enhanced” reality technology. I am horribly susceptible to motion sickness, my presbyopia makes focusing on Google glass–like displays pretty much impossible, and even 3D movies do not make my eyes happy. Using a good virtual reality system, I can go maybe 30 seconds before I have escape to the real world; with a phone-based system, even a couple of seconds is too much.
But last week I spent at least 15 minutes (though it felt like less than five) completely engaged in a sampling of virtual worlds seen through Avegant’s mixed reality viewer. The experience was magical, enthralling, amazing, wonderful—pick your superlative. I didn’t get nauseous, or headachy, or feel any eyestrain at all. Indeed my eyes felt rested (probably because that was 15 minutes not spent in front of a computer or phone screen). Also a wonderful part of the experience: the fact that the company didn’t bother with extreme security measures or nondisclosure agreements (though executives are not talking specific technical details until patent filings are complete.
Avegant is a four-year-old Belmont, Calif., based startup. Its first product, the Glyph head-mounted display typically used for personal entertainment viewing, has been shipping since February of last year. (The name is a mashup of the names of the founders—Edward Tang and Allan Evans.)
The company announced its transparent Light Field Display technology last month. It hasn’t said when this will be ready for manufacture, though Tang points out that the Glyph’s success shows that the company knows how to design products for manufacture and bring them to market.
Avegant cofounder Edward Tang holds a mockup of a future version of Avegant's light field mixed reality wearable displayPhoto: Tekla Perry
Avegant’s prototype mixed reality system uses a headband to position the Avegant display. It is driven by an IBM Windows PC with an Intel i7 processor and an Nvidia graphics card running the Unity game engine.
The images, explained cofounder Tang, now chief technology officer, are projected onto the retina by an array of MEMS micromirrors, each of which controls one pixel.
That, so far, is the same as the company’s Glyph system. But unlike a standard micromirror display, which reflects light straight at the person viewing it, these light field images are projected at different angles, mimicking the way light in the real world reflects off objects to hit a person’s eyes. The difference in these angles is particularly dramatic the closer someone is to the object, creating distinct and separate focal planes; the eye naturally refocuses when it moves from one plane to another.
To avoid having the eyes deal with these multiple focal planes, explained Tang, mixed reality systems like Microsoft’s HoloLens tend to keep viewers a meter or two away from objects. Light field technology, however, can use different focal planes for different objects simultaneously, so the user perceives even very close-up objects to be realistic. (Tang makes the case for light field technology in the video below.)
To date, Tang says, most attempts to bring light field technology into head-mounted displays have involved tricky-to-manufacture technology like deformable mirrors or liquid lenses, or approaches that take huge amounts of computing power to operate, like stacked LCDs.
“We created a new method,” he said, “that has no mechanical parts and uses existing manufacturing capabilities, with a level of computation that isn’t particularly high; it can run on standard PCs with graphics cards or mobile chipsets.”
The effect is designed to be natural—that is, you see virtual objects in the same way you normally see real objects, with no eye strain caused from struggling to focus. And, in the demo I was shown, it absolutely was.
I went through two mixed reality experiences in a slightly dim but not dark room with some basic furniture. The room was rigged with off-the-shelf motion tracking cameras to help map my position; the headset I wore was tethered to a PC. After a short calibration effort that allowed me to adjust the display to match the distance between my pupils, I entered a solar system visualization, walking among planets, peering up close at particular features (Earth seemed to be a little smaller than my head in this demo), and leaning even closer to trigger the playing of related audio.
Clear labels hovered near each planet, which brings up an interesting side note: I wasn’t wearing my reading glasses, but the labels, even close at hand, were quite clear. Tang mentioned that the developers have been discussing whether, for those of us who do need reading glasses, it would be more realistic to make the virtual objects as blurry as the real ones. I vote no, I didn’t find it jarring that my hand as I used it to reach for planets was a little fuzzy, particularly, perhaps, since the virtual objects were appearing brighter than real world ones. And it was quite lovely having so much of what I was seeing be clear.
At one point in the demo, while I was checking out asteroids near Saturn, Tang suggested that I step into the asteroid belt. I was a bit apprehensive; with my VR sickness history, it seemed that watching a flow of asteroids whizzing by me on both sides would be a uniquely bad idea, but it went just fine, and I could observe quite a bit of detail in the asteroids as they flowed past me.
The second demo involved a virtual fish tank. Tang asked me to walk over to a coffee table and look down at the surface; the fish tank then appeared, sitting on top of the table. I squatted next to the tank and put my hand into it. I reached out for a sea turtle; it was just the right size to fit in my palm. I followed it with my cupped hand for a while, and started feeling a whoosh of air across my palm whenever it swept its flippers back. I wondered for a moment if there was some virtual touch gear around, but it turned out to just be my mind filling in a few blanks in the very real scene. Tang then expanded the fish tank to fill the room; now that sea turtle was too big to hold, but I couldn’t resist trying to pet it. Then, he told me, “Check out that chair,” and in a moment, a school of tiny fish swept out from under the chair legs and swooped around the nearby furniture.
After convincing me to leave the fish demo (I was enjoying the experience of snorkeling without getting wet), Tang directed me to walk towards a female avatar. She was a computer-generated human that didn’t quite leave the uncanny valley—just a standard videogame avatar downloaded from a library, Tang said. But he pointed out that I could move up and invade her personal space and watch her expression change. And it certainly did seem that this avatar was in the room with me.
Throughout all the demos, I didn’t encounter any vision issues, focus struggles, or other discomfort as I looked back and forth between near and far and real and virtual objects.
I have not been one of the anointed few who have tested Magic Leap’s much-ballyhooed light-field-based mixed reality technology (and given the company’s extreme nondisclosure agreements, I likely couldn’t say much about it if I had). So, I don’t know how Avegant’s approach compares, though I’d be willing to put Avegant’s turtle up against Magic Leap’s elephant any day.
What I do know is that it absolutely blew me away. I’m eager to see what developers eventually do with it, and I’m thrilled that I no longer have to struggle physically to visit virtual worlds.
