Hey there, human — the robots need you! Vote for IEEE’s Robots Guide in the Webby Awards.

Close bar

Open-Source Eyewear Is AR Without Walls

Brilliant Labs leans into generative AI with tiny, affordable lens

4 min read
A close-up of a thick glass lens with hardware sitting on the front of a pair of glasses.

The Monocle is an open-source augmented-reality device that can be clipped to eyeglasses, as seen here, or held to a user’s eye.

Brilliant Labs

What if you could have the power of ChatGPT packed into an augmented-reality device that fits in the palm of your hand?

That was a trick question: This device already exists, and it’s called Monocle. Built by Brilliant Labs, this US $349 open-source AR device clips onto ordinary glasses, fits in the palm of your hand, and stows in an included recharging case. Bobak Tavangar, the company’s CEO, describes the device as “cute” and believes it will offer a personable, approachable on-ramp for anyone looking to experiment with AR.

“[Monocle] was something that was just kind of unobtrusive; it clipped on. And then when you’re in a social setting, or when you’re finished using it, you clip it off or you take it off and just like [Apple] AirPods, it goes in your pocket,” says Tavangar.

Monocle’s open-source design is a boon for AI experiments

Monocle enjoyed a burst of attention in March 2023 when RizzGPT, a head-up display for ChatGPT, went viral on Twitter. A group of Stanford students put together the project, which is available on Github, to see how ChatGPT might be useful in face-to-face conversations. Its name is a play on “rizzing”—slang for flirting—and the project’s creators describe it as “charisma as a service (CaaS).” The students followed up on RizzGPT with an AI agent that can recognize friends through Monocle’s built-in camera and suggest conversation topics based on recent texts.

This is all a bit tongue-in-cheek, of course, but push past the humor and you’ll find the motivation behind Monocle’s open-source approach. Monocle prioritizes affordability, access, and ease of development over immersive interactions, performance, and graphical fidelity. These traits stand in contrast to those of larger, more complex AR devices like the Microsoft HoloLens, Vuzix Blade, and Nreal Air.

“If multitouch was the enabling technology for the iPhone and that really kind of led to the birth of the modern smartphone, we view generative AI as the multitouch for AR,” says Tavangar.

Monocle offers a small, 640-by-400-pixel micro OLED display and a slim 20-degree field of view, limitations that make the visual experience inferior to alternatives with a wider field of view. This, however, is a less significant barrier when a text interface can be used to tap into AI services. Tavangar mentioned the 2013 film “Her” as an example. In the movie, a powerful AI assistant, Samantha, interacts entirely through speech. RizzGPT isn’t as capable as Samantha, to be sure, but the ideas behind its user interface are similar.

A schematic of the Monocle augmented reality device. It has a single lens which is paired with several components including a camera, MicroOLED display, and camera.Monocle’s compact hardware is simple compared with that of larger, more expensive headsets like Microsoft’s HoloLens and Nreal’s Air. Brilliant Labs

“If multitouch was the enabling technology for the iPhone and that really kind of led to the birth of the modern smartphone, we view generative AI as the multitouch for AR,” says Tavangar.

Brilliant Labs also leans into open-source principles to make AI development more accessible. The company provides a drag-and-drop interface, called StreamLogic, that taps into Monocle’s included field-programmable gate-array processor. The FPGA, in turn, interfaces with the camera, microphone, and display. Tavangar says StreamLogic makes developing AI apps for Monocle “almost like you’re building Legos,” adding that it can easily serve as the platform for apps that scan physical spaces or product barcodes, among other uses.

Schematics and 3D files of Monocle’s physical design are available, too, and should help developers looking to tweak, improve, and customize the hardware. Proof of this in action remains to be seen—Monocle is rather new, having been released in February 2023. But other open-source projects, such as the MNT Reform and Pocket Reform laptops, have shown the benefits of this approach with design improvements that originated in the community.

Open source could benefit from Apple’s entry

Brliliant Labs’ decision to go open source sets it apart from a sea of expensive closed-source headsets that typically provide a mix of AR and VR capabilities.

“The AR space is limited regarding open-source software,” says Anshel Sag, principal analyst at Moor Insights & Strategy. He notes that the Khronos Group offers an open-source VR/AR API called OpenXR but adds that “most of those capabilities have been in VR and few in AR.” Magic Leap has turned toward open source in recent years, but the Magic Leap 2’s intimidating $3,299.99 MSRP is a barrier.

Open-source AR’s meager prospects are further clouded by the highly anticipated entry of a giant famous for walled gardens: Apple. Rumors about Apple’s mixed reality headset gained steam in 2021 and continue to build, with the latest tidbits suggesting a reveal at WWDC 2023.

Whether this proves true remains to be seen, but Apple’s presence in AR is already felt through ARKit, an API that enables AR development for most iPhones and iPads. ARKit is powerful but, true to form, only supports Apple devices, while open-source alternatives like Khronos’s OpenXR don’t support iOS. This fragmentation is a problem for developers looking to build for AR, as choosing to support one API typically means dropping support for another. Apple’s AR headset, if it arrives, will only worsen this dilemma.

A hand holding a clear lens that has electronic components embedded in it. The Monocle is an open-source augmented-reality device that can be clipped to eyeglasses or held to a user’s eye. Brilliant Labs

Despite this, Sag believes that an Apple AR headset would ultimately prove a tailwind for Monocle and others that embrace an open-source approach. “Apple’s launch will likely create more appetite for an open-source approach, especially considering Apple’s stranglehold on its developers regarding development and monetization. There will still be a lot of developers that will utilize Apple’s platform, but I also believe that making the market bigger will increase the demand for an open-source approach.”

Tavangar, who himself spent time with Apple as a program lead, agrees with Sag’s take. He says Apple’s rumored launch can’t come soon enough and sounds confident it will inject innovation and investment into a stagnant AR industry. “We’re excited for Apple to get into this space now,” he says.

The Conversation (0)