AR Glasses Spawn a Whole New Social Dynamic

Remember that viral video-call mishap of a lawyer getting stuck with the cat filter on? Imagine yourself in their shoes. Let’s say you’re talking to someone wearing eyewear akin to “smart” sunglasses that can discreetly overlay a cat filter on your face without your knowledge. How would that make you feel?

Researchers at Cornell University and Brown University teamed up to explore how this scenario would play out in a world with more ubiquitous augmented reality (AR) usage. In their study, presented in July at the 2023 ACM SIGCHI Conference on Designing Interactive Systems, the authors uncovered a social dynamic wherein a power imbalance favors the wearer of “smart” glasses, and offer insights on how the tech industry can design more inclusive AR technologies.

“This AR filter interaction is likely to happen in the future with the commercial emergence of AR glasses,” says Jenny Fu, a doctoral student at Cornell University’s Bowers College of Computing and Information Science and one of the two lead authors of the study. “How will that look like, and what are the social and emotional consequences of interacting and communicating through AR glasses?”

That was the main question Fu and her collaborators—including lead author Ji Won Chung, a doctoral student at Brown University’s Human-Computer Interaction (HCI) Research Lab, and coauthor Malte Jung, an associate professor of information science at Cornell—aimed to answer. They randomly grouped 10 participants into pairs, with one person in each pair wearing Spectacles, AR glasses loaned by Snap Inc., the company behind Snapchat. This prototype version of Spectacles is equipped with a camera and five filters that can transform a nonwearer into a bear, cat, clown, deer, or pig-bunny.

“We often forget these ‘onlookers’ and we’re not designing with them in mind.” —Malte Jung, associate professor of information science, Cornell University

Each pair engaged in a desert-survival exercise, listing items they need to survive in the desert. Wearers reported feeling more relaxed, with the face filters easing their nervousness and reducing their social anxiety when talking to strangers. The experience wasn’t so rosy for nonwearers. They felt uneasy not knowing what was happening on the other side of the AR glasses. And as with sunglasses, the dark shade of the lenses made it difficult to read social cues.

Additionally, nonwearers felt disempowered due to having no agency over their own appearance and a lack of control over how others view them. Some participants even raised concerns about potential nefarious uses of AR glasses, such as filters that could impose nudity onto someone, deepfakes generated with their likeness, or being discretely recorded without their consent.

The pairs also took part in a design session, where they were asked to sketch ideas on how to improve the AR glasses for both wearers and nonwearers. Most of them included features only for the wearer.

“When we think about design in HCI, there is often a tendency to focus on the primary user and design just for them,” Jung says. “Because these technologies are so deeply embedded in social interactions and are used with others and around others, we often forget these ‘onlookers’ and we’re not designing with them in mind.”

Some participants suggested features such as an indicator light to inform nonwearers when the AR glasses are being used to change their appearance, as well as a holographic projection so others can view what the wearer sees. This type of participatory design approach can help “better capture and simulate a positive interaction in the lab when doing usability testing and prototyping,” Fu says.

Moreover, involving nonusers is especially key in developing more equitable tech products and creating more inclusive experiences. “That’s one of the points why previous AR iterations may not have worked—they designed it for the individual and not for the people surrounding them,” says Chung. She adds that a mindset shift is needed to actively make tech that doesn’t exclude people, which could lead to social systems that promote engagement and foster a sense of belonging for everyone.

The authors hope that other researchers can build upon their results by increasing the study size and having groups larger than two, as well as including people whose social relationships are more defined, such as friends and family members. Broadening the age demographic might also help, as participants in the current study are mostly young adults between the ages of 20 and 24.

What Jung ultimately envisions, though, is for the study to raise awareness of the social opportunities and risks that come with designing for a looming mixed-reality future.

“It’s about understanding how these devices shape how we interact with others and how they perceive us, and how these technologies affect our ability to form and maintain social relationships,” he says. “These are what make us human.”

This article appears in the January 2024 print issue as “AR Glasses Upset the Social Dynamic.”