Telecommunications

Low Latency 5G Networks Could Be a Game-Changer for AR and VR (But Not Until 2020)

New 5G service could enable multi-player VR games and maybe even eliminate nausea

Visitors to MWC Barcelona test VR glasses with Deutsche Telekom's Hyperglobe.
Photo: Clara Margais/dpa/Getty Images

That’s not me, but these guys seem to be enjoying the Hyperglobe.Photo: Clara Margais/dpa/Getty Images

I’m not going to tell you what I saw in the Hyperglobe.

For those of you who have not experienced the Hyperglobe’s mysteries firsthand, you should know that it’s a virtual reality (VR) demonstration in Deutsche Telekom’s booth at MWC Barcelona (formerly called Mobile World Congress). It’s a large sphere, and—well, that’s about it. It is smooth, featureless, and a dull pink. Its dullness even stands out in the back of Deutsche Telekom’s aggressively bright pink and green aesthetic.

In fact, the Hyperglobe doesn’t do much until you, and up to two other people, don wireless VR headsets. Then, it becomes a large, cooperative game. The edges of the virtual Hyperglobe—which is a transparent bubble in VR—is a surface on which to manipulate objects. Then, by plunging an object into the bubble’s center, a player can build bizarre virtual sculptures. (Okay, here’s what I saw—I took a model skyscraper, flipped it upside down, and painted it pink).

If you haven’t already heard, 5G is no longer on the way—it’s here, with rollouts happening around the world. With that in mind, I’m trying to answer a question at MWC Barcelona this year: What should we expect from this new generation of wireless? If you’re just waiting for better smartphone service, you might not be thinking far enough outside the box. Plenty of companies—Deutsche Telekom, Nokia, and Qualcomm, to name a few—believe augmented reality (AR) and VR will massively benefit from 5G’s high data throughput and low latency.

To return to the Hyperglobe once more, two of its key features are enabled by a wireless 5G connection in the 59- to 61-gigahertz band. The first is that a player can interact, in a virtual space, with a virtual object. While Hyperglobers ram together models of skyscrapers in virtual space, they’re using the sphere as a reference point to navigate that space. For example, a player can hover the right hand over the sphere and rotate the chosen object with swipes of the hand. Similarly, swiping with the left hand changes the color of the selected object.

Doing this successfully requires a low-latency connection so that, as you move around the sphere, your view updates without delay—otherwise, you might blemish the Hyperglobe’s pristine surface with vomit. While you could create a similar setup on a Wi-Fi network with a couple of today’s wireless headsets, like the HTC Vive, you would find it difficult to navigate around and coordinate your efforts because the network would take too long to update you on what the others are doing.

Hyperglobe also lets you experience all of this with friends. Remember when Pokemon Go was cool? While you may have been playing Pokemon Go “with” other people, in the sense that you were searching for each Pikachu or Bulbasaur or Scyther (the best Pokemon) with friends, the actual, in-phone AR experience was single-player. You and your friends couldn’t truly Pokemon Go it together in the augmented world of the game itself.

Deutsche Telekom isn’t the only company demonstrating the possibility of multiperson experiences enabled by 5G. Nokia, for example, has a Ping-Pong table set up with cameras to track the movements of people playing the game together in a virtual space with a virtual ball and table, and determine whether or not they have managed to hit the ball back to their opponent.

Ping-Pong is already great, but apparently it’s even better with 5G.Photo: Michael Koziol

In the demonstration, the players started out on a 4G network with latency of 40 to 80 milliseconds, which, in the VR world, means that, for a player trying to react to a ball coming toward them, the ball is actually anywhere from 120 to 248 centimeters closer than it appears. After a few lag-afflicted points in which players almost always missed the ball, the network switched over to a 5G connection in the 3.4- to 3.8-GHz band, and suddenly the ball was only 9 to 31 centimeters off—plenty close enough to make it possible for the players to return hits.

As with robotics, it’s becoming clear that the key quality of 5G is not the higher data throughput—though that certainly enables these complex, three-dimensional virtual spaces—but the low latency. In fact, VR headsets have been plagued for years by their nasty habit of inducing motion sickness because of a noticeable delay between the time when a players move their head and the virtual view tracks accordingly. It’s worth repeating: Virtual reality is no fun if you puke on the Hyperglobe.

While the Hyperglobe is an interesting piece of collaborative art, it’s not the sort of thing that will drive a new industry of applications for 5G (sadly, neither is VR Ping-Pong). So, what will drive AR and VR in the coming months and years?

Alexander Lautz, Deutsche Telekom’s senior vice president for 5G, thinks the “killer application” will be something like AR-enhanced Google Maps—something ubiquitous and useful and which can also be clearly enhanced by AR elements. Imagine, for example, a real-time view of the street in front of you, courtesy of your phone’s camera, with arrows and directions directly overlaid on that view to tell you where to go.

Other areas where AR and VR—or what some companies call mixed reality (MR) or extended reality (XR)—could break through are gaming and remote training. Both are applications that require the lowest latency possible.

But not this year. Whatever killer apps are or aren’t delivered by AR and VR—or MR, or XR, or WhatevR—don’t look for them until 2020, at the earliest. That’s because the industry is still sorting out the technical details.

“A lot of these use cases, they’re not 2019 use cases,” says Jane Rygaard, head of marketing for mobile networks at Nokia. “For more dedicated use cases, we have to get the specifications right, and the specifications we’re waiting for right now are Release 16.”

The standard that defines 5G is Release 15 of the Mobile Broadband Standard, which is put out by the telecom industry association 3GPP. The latest version of Release 15 was finalized in July 2018, and it focused most heavily on smartphones and traditional cellular networks. “There’s a lot more that’s coming in the evolution of 5G in Release 16 and beyond, and a lot of that is still being discussed,” says Danny Tseng, 5G and LTE technical marketing manager at Qualcomm.

But perhaps the biggest thing that will need to happen to make sure AR and VR do catch on is finding a comfortable, natural way for people to interact with these technologies. “People are not used to doing it, because it’s not very comfortable to hold a phone in this position for very long,” says Deustche Telekom’s Lautz, extending out his arm. “But you want to get used to that as a preface, then you want to have it on the glasses.”

Shadow Creator’s stylish New Air 2 headset on display at MWC Barcelona.Photo: Michael Koziol

So, while your 2019 5G smartphone will certainly handle improved AR and VR applications, the real sea change will come next year. That’s when, with the help of Release 16, we should see the rollout of lighter, more comfortable, wireless headsets and glasses for AR and VR. And that might not be such an impossible task—already on display at Qualcomm’s booth this year was the wireless New Air 2 headset by Shadow Creator, which is the sleekest headset I’ve seen to date.

It’s only a few months away, but if you’re wondering when you can get your hands on a Hyperglobe, I’d say no sooner than 2020.