Virtual Reality and Social Networks Will Be a Powerful Combination

Photo: Dan Saelinger

This is part of IEEE Spectrum's special report on the battle for the future of the social Web.

Half a billion people around the world spend more than 20 hours a week "wearing" avatars, that is, using digital representations of themselves.

Did you know that a virtual representation of a politician could be more convincing than the politician himself? That your heart beats just as fast when your girlfriend winks at you from your computer screen as when she walks into the room? That if you make your onscreen surrogate mimic your friend's head movements, he's more likely to do what you say?

Within three years, many online interactions will not simply involve passages of text typed on a keyboard but will instead be rich exchanges involving sophisticated representations of ourselves. Those digital beings are called avatars, and today at least half a billion people around the world are routinely socializing through them over the Internet, for example in online games.

Others spend many hours each week controlling avatars inside their own homes, using game platforms like the Wii and Xbox.

And avatars aren't just for recreation. A vice president at IBM has predicted that by 2015 all IBMers will have avatars to send to work-related meetings and presentations.

---

Half a billion people around the world spend more than 20 hours a week "wearing" avatars, that is, using digital representations of themselves.

---

The use of avatars is booming. But even more people use social media—at least a billion people around the world.

It's only a matter of time before these two trends—realistic avatars and social media—intersect. As more people gain experience with avatars, and as the technology for enriching them improves, we can expect avatars to play far bigger roles in communications. Evidence suggests that the use of avatars could profoundly alter our social behaviors and work performance—for better or worse.

Avatars have special advantages. A videoconference system continuously captures what someone is doing in the real world and sends that image to another locale. But avatar technology works differently—it captures only selected movements, sending enough information about them over a network to allow the computer on the other end to re-create the action. That difference has two helpful consequences. First, the avatar can be shown doing things that a camera doesn't see, or the system can ignore certain motions. Second, the avatar can be controlled without any network delay, because there is no need to send bandwidth-hogging HD pixels.

Remember Second Life? That was the first commercially successful home for avatars that was not a video game. Its cartoonish characters can be controlled (awkwardly) only through the computer mouse and keyboard, and network lags deter users, though developers are trying to improve the experience.

But avatars have evolved over the years and are growing to resemble their human controllers, both in how they look and how they behave. Look at the Kinect, Nintendo's Wii, and Sony's PlayStation Move, which are all now equipped with devices that transform players' physical actions into virtual body movements. Digital avatars move in tandem with their wearers as the latter jump, point guns, and swing rackets.

Soon avatars are going to get even more compelling. Advances in 3-D are changing the entertainment industries; they will change avatars as well. And you can already purchase inexpensive devices from such makers as Novint Technologies and Scent Sciences Corp. that augment video games with tactile feedback or a sequence of scents. So, besides hearing and seeing conversational partners, you can potentially feel and smell them.

While avatar technology has been steadily improving, social networking, with its text, still images, and jerky videos, has been skyrocketing. Adults in developed countries who have access to any kind of digital tools have quickly come to depend on the Internet to shop, do business, and entertain and educate themselves. Online dating, which was somewhat stigmatized just a few years ago, is now the norm.

Photo: Dan Saelinger

Within the next three years we're going to see the advanced avatars and other technology developed for gamers migrate into social networks. Imagine your friends seeing you in 3-D, in the form of a fully embodied avatar projected into the center of a room. You'll need no keyboard for typing or mouse for clicking; rather, unobtrusive devices will measure and capture your movements.

These avatars can simply be realistic representations of who you are and what you are doing, helped along by motion-capture technologies that let them mirror every movement. But they can also be strategically filtered or altered representations that show only what you want them to. Their capabilities can be modified to allow them to do things that no real, physical person could do in a given situation. And behavioral studies have shown that those differences can carry a remarkable social power to influence others.

Should we be at all worried about turning over most—or even some—of our online communications to avatars? The scientific record is mixed.

As researchers and authors of the book Infinite Reality, we know from studies of people's behavior that the brain often fails to differentiate between virtual experiences and real ones, and between computerized characters and human beings. In one of many experiments that support this conclusion, Clifford Nass and Byron Reeves, of the department of communication at Stanford University, conducted a "politeness study" in the mid-1990s. Researchers asked participants who were completing a digital survey to evaluate the performance of the computer they had just used. If they answered those evaluation questions on the same computer—literally, the same physical unit—they rated the computer more positively than if they were moved to a different machine to take the evaluation survey; those who switched machines were more critical of the performance of the first computer. We agree with Reeves and Nass's interpretation that participants (perhaps unconsciously) did not want to offend the machine they had been working on.

More recent work examining people's experience of the virtual versus the real involved functional magnetic resonance imaging (fMRI) data. Researchers at various universities across the country have conducted experiments over the past six years that analyzed and compared the patterns made by neurons that fire when people watch a digital 3-D re-creation of other people with patterns from those that fire when people view photographs or video of real people. Most experiments failed to find differences in those patterns.

It's not too hard to see why some of us prefer virtual life to physical reality. In virtual spaces aging is optional, and weight loss happens via Photoshop. In cyberspace, there is no such thing as a bad hair day. Avatars can even be more socially gifted than the humans who drive them.

For years, for example, researchers have been fascinated by mimicry, regarded by developmental psychologists as essential to the acquisition of communications skills. Psychologists also believe that mimicry has a lot to do with a person's likability. In a classic experiment, Tanya Chartrand, a social psychologist now at Duke University, demonstrated that research participants acting as interviewers significantly favored the "job applicants" who mimicked the interviewers' gestures over ones who did not. Furthermore, the interviewers were not consciously aware of the mimicry. Chartrand labeled this the "chameleon effect."

We wondered if that chameleon effect applied in virtual reality. So in 2005 we carried out a study in which Stanford undergraduates watched and listened to a virtual character in a video-game-like setting while wearing a head-mounted display that allowed them to see in 3-D and to control their point of view by moving their heads. The character delivered a message advocating a campus security policy that would require all students to carry their identification at all times.

We varied the mimicry behavior of that character, using the head-mounted display to track each undergraduate's head movements and programming the virtual character to make those same head movements four seconds later. The undergraduates rated mimicking agents as more persuasive and also more credible, trustworthy, and intelligent than identical-looking virtual characters who delivered the same messages without the mimicked head movements.

In fact, avatars are better at strategically planned mimicry than people are. A human who consciously tries to mimic another human generally doesn't do it well. (Ironically, though, the many people who tend to do this mimicry unconsciously do just fine.) A conscious mimic can be too obvious about it, can miss seeing the other's gestures, and can slip up when tired. Conscious mimicry takes a lot of effort, potentially making it harder to keep track of the conversation and speak persuasively. Avatars face none of these problems.

Could this type of social influence become important? Imagine how a virtual car salesperson on a future social networking site could be programmed to mimic the movements of a potential buyer detected by webcam. Or how a politician who uses these sites to reach constituents might use this technology to augment the effectiveness of his message.

Small changes to avatar appearance could drastically change history. In a study performed by Stanford researchers, 200 respondents from a national random sample received electronic surveys with images of George W. Bush and John Kerry, U.S. presidential candidates in 2004. One third of participants received an image of Bush manipulated to look on a subconscious level like the respondent, a third received a similarly manipulated image of Kerry, and a third saw the original images only. Nobody explicitly detected the changes, but the manipulation caused a 20 percent shift in the votes of those seeing the manipulated samples as compared with the control group. They repeated this experiment with other candidates, covering more than a thousand people in five total studies, and demonstrated conclusively that subconscious similarities could swing an election. People don't consciously notice that the candidates look like them, but they are swayed by the implicit affinity.

This technology may be manipulative, but it can be a force for good. Teachers could use virtual mimicry to ensure that most, if not all, students in a class are more in tune with their lessons. Because the rendering computer in an immersive virtual-reality system sends information to all of the students' systems individually, it can tailor that information to make the teacher look and behave like each student simultaneously.

Similarly, avatars can always maintain eye contact. With the possible exception of physical touch, looking another person in the eye is the most potent nonverbal tool humans possess for commanding attention. When a teacher looks at a student, the student is actually more likely to master the educational material. But making direct eye contact with a hundred students simultaneously is impossible. A teacher can try to spread her gaze around, but even if she spreads it evenly among the students in a large lecture hall, she can only gaze at each student perhaps 1 percent of the time. Yet digital avatars on students' individual displays can engage in a mutual gaze with everyone at the same time—and every student will respond as if she were the sole recipient of that gaze.

We have run hundreds of experimental participants through what we call "non-zero-sum gaze" virtual-classroom studies to test this premise. So far, not one student realized that the gaze of the teacher's avatar was not genuine—they all thought the teacher was looking at them individually while ignoring the other students. Moreover, students' attention and learning improved. As a group, they focused on the teacher's avatar with the simulated direct gaze more than on avatars not set to use the technology. The students also retained more information. So a relatively simple chunk of code (about 20 lines) allowed us to augment the behavior of a virtual teacher in socially influential ways, improving students' behaviors, attitudes, and learning. For example, in one study, students paid attention almost 20 percent longer when receiving an augmented gaze than when receiving a natural gaze.

In our experiments, even small changes in avatar appearance resulted in huge changes in behavior. Taller avatars can make people behave more confidently; attractive ones can make them more social. Avatars that lose weight can make their owners exercise more and eat healthier. Showing young people their future selves can make them save more money for the future, and showing people's avatars living with the consequences of energy use—for example, in a forest with the trees all cut down—can increase conservation.

These examples suggest a synergy between the mind and immersive virtual technology. On the one hand, the brain treats real and virtual experiences as the same. On the other hand, in virtual reality, the rules of grounded reality are suspended. In virtual reality, avatars can age, grow, or become supermodels at the touch of a button. They can use conversational superpowers, including large-scale mimicry and gaze, or wear other people's faces and bodies. Rationally, people bring specific expectations, perhaps guarded ones, when interacting with virtual others. However, the brain cannot consistently keep this guard up for long stretches of social interaction, and these superavatars tend to elicit very human responses.

Current social networking and other online sites are just precursors of what we'll see when social networking encompasses immersive virtual-reality technology. When people interact with others for substantial periods of time, much as they do now on Facebook but with fully tracked and rendered avatars, entirely new forms of social interaction will emerge. Avatars can be more human than humans, and what that means for the brains responding to them will be understood only many years from now.

We will be watching what unfolds as the dawn of the virtual revolution turns to high noon. We expect billions of people to benefit greatly from the resulting expansion of personal freedom and the resulting changes to society's collective institutions; the democratic social movements now unfolding in North Africa and the Middle East are enabled in large part by social networking, and they are just the beginning. But there will be downsides, especially in terms of privacy, personal mental health, and intimate relationships.

Pope Benedict XVI recently said: "New technologies and the progress they bring can make it impossible to distinguish truth from illusion and can lead to confusion between reality and virtual reality" and might result in "indifference towards real life." But Mark Twain cautioned, "Don't part with your illusions. When they are gone you may still exist, but you have ceased to live."

This is part of IEEE Spectrum's special report on the battle for the future of the social Web.

This article originally appeared in print as "This Is Your Mind Online".