The October 2022 issue of IEEE Spectrum is here!

Close bar

Low Latency 5G Networks Could Be a Game-Changer for AR and VR (But Not Until 2020)

New 5G service could enable multi-player VR games and maybe even eliminate nausea

5 min read
Visitors to MWC Barcelona test VR glasses with Deutsche Telekom's Hyperglobe.
Photo: Clara Margais/dpa/Getty Images

Visitors to MWC Barcelona test VR glasses with Deutsche Telekom's Hyperglobe.That’s not me, but these guys seem to be enjoying the Hyperglobe.Photo: Clara Margais/dpa/Getty Images

I’m not going to tell you what I saw in the Hyperglobe.

For those of you who have not experienced the Hyperglobe’s mysteries firsthand, you should know that it’s a virtual reality (VR) demonstration in Deutsche Telekom’s booth at MWC Barcelona (formerly called Mobile World Congress). It’s a large sphere, and—well, that’s about it. It is smooth, featureless, and a dull pink. Its dullness even stands out in the back of Deutsche Telekom’s aggressively bright pink and green aesthetic.

In fact, the Hyperglobe doesn’t do much until you, and up to two other people, don wireless VR headsets. Then, it becomes a large, cooperative game. The edges of the virtual Hyperglobe—which is a transparent bubble in VR—is a surface on which to manipulate objects. Then, by plunging an object into the bubble’s center, a player can build bizarre virtual sculptures. (Okay, here’s what I saw—I took a model skyscraper, flipped it upside down, and painted it pink).

If you haven’t already heard, 5G is no longer on the way—it’s here, with rollouts happening around the world. With that in mind, I’m trying to answer a question at MWC Barcelona this year: What should we expect from this new generation of wireless? If you’re just waiting for better smartphone service, you might not be thinking far enough outside the box. Plenty of companies—Deutsche Telekom, Nokia, and Qualcomm, to name a few—believe augmented reality (AR) and VR will massively benefit from 5G’s high data throughput and low latency.

To return to the Hyperglobe once more, two of its key features are enabled by a wireless 5G connection in the 59- to 61-gigahertz band. The first is that a player can interact, in a virtual space, with a virtual object. While Hyperglobers ram together models of skyscrapers in virtual space, they’re using the sphere as a reference point to navigate that space. For example, a player can hover the right hand over the sphere and rotate the chosen object with swipes of the hand. Similarly, swiping with the left hand changes the color of the selected object.

Doing this successfully requires a low-latency connection so that, as you move around the sphere, your view updates without delay—otherwise, you might blemish the Hyperglobe’s pristine surface with vomit. While you could create a similar setup on a Wi-Fi network with a couple of today’s wireless headsets, like the HTC Vive, you would find it difficult to navigate around and coordinate your efforts because the network would take too long to update you on what the others are doing.

Hyperglobe also lets you experience all of this with friends. Remember when Pokemon Go was cool? While you may have been playing Pokemon Go “with” other people, in the sense that you were searching for each Pikachu or Bulbasaur or Scyther (the best Pokemon) with friends, the actual, in-phone AR experience was single-player. You and your friends couldn’t truly Pokemon Go it together in the augmented world of the game itself.

Deutsche Telekom isn’t the only company demonstrating the possibility of multiperson experiences enabled by 5G. Nokia, for example, has a Ping-Pong table set up with cameras to track the movements of people playing the game together in a virtual space with a virtual ball and table, and determine whether or not they have managed to hit the ball back to their opponent.

Nokia's ping pong VR game at MWC Barcelona.Ping-Pong is already great, but apparently it’s even better with 5G.Photo: Michael Koziol

In the demonstration, the players started out on a 4G network with latency of 40 to 80 milliseconds, which, in the VR world, means that, for a player trying to react to a ball coming toward them, the ball is actually anywhere from 120 to 248 centimeters closer than it appears. After a few lag-afflicted points in which players almost always missed the ball, the network switched over to a 5G connection in the 3.4- to 3.8-GHz band, and suddenly the ball was only 9 to 31 centimeters off—plenty close enough to make it possible for the players to return hits.

As with robotics, it’s becoming clear that the key quality of 5G is not the higher data throughput—though that certainly enables these complex, three-dimensional virtual spaces—but the low latency. In fact, VR headsets have been plagued for years by their nasty habit of inducing motion sickness because of a noticeable delay between the time when a players move their head and the virtual view tracks accordingly. It’s worth repeating: Virtual reality is no fun if you puke on the Hyperglobe.

While the Hyperglobe is an interesting piece of collaborative art, it’s not the sort of thing that will drive a new industry of applications for 5G (sadly, neither is VR Ping-Pong). So, what will drive AR and VR in the coming months and years?

Alexander Lautz, Deutsche Telekom’s senior vice president for 5G, thinks the “killer application” will be something like AR-enhanced Google Maps—something ubiquitous and useful and which can also be clearly enhanced by AR elements. Imagine, for example, a real-time view of the street in front of you, courtesy of your phone’s camera, with arrows and directions directly overlaid on that view to tell you where to go.

Other areas where AR and VR—or what some companies call mixed reality (MR) or extended reality (XR)—could break through are gaming and remote training. Both are applications that require the lowest latency possible.

But not this year. Whatever killer apps are or aren’t delivered by AR and VR—or MR, or XR, or WhatevR—don’t look for them until 2020, at the earliest. That’s because the industry is still sorting out the technical details.

“A lot of these use cases, they’re not 2019 use cases,” says Jane Rygaard, head of marketing for mobile networks at Nokia. “For more dedicated use cases, we have to get the specifications right, and the specifications we’re waiting for right now are Release 16.”

The standard that defines 5G is Release 15 of the Mobile Broadband Standard, which is put out by the telecom industry association 3GPP. The latest version of Release 15 was finalized in July 2018, and it focused most heavily on smartphones and traditional cellular networks. “There’s a lot more that’s coming in the evolution of 5G in Release 16 and beyond, and a lot of that is still being discussed,” says Danny Tseng, 5G and LTE technical marketing manager at Qualcomm.

But perhaps the biggest thing that will need to happen to make sure AR and VR do catch on is finding a comfortable, natural way for people to interact with these technologies. “People are not used to doing it, because it’s not very comfortable to hold a phone in this position for very long,” says Deustche Telekom’s Lautz, extending out his arm. “But you want to get used to that as a preface, then you want to have it on the glasses.”

New Air 2 headset by Shadow CreatorShadow Creator’s stylish New Air 2 headset on display at MWC Barcelona.Photo: Michael Koziol

So, while your 2019 5G smartphone will certainly handle improved AR and VR applications, the real sea change will come next year. That’s when, with the help of Release 16, we should see the rollout of lighter, more comfortable, wireless headsets and glasses for AR and VR. And that might not be such an impossible task—already on display at Qualcomm’s booth this year was the wireless New Air 2 headset by Shadow Creator, which is the sleekest headset I’ve seen to date.

It’s only a few months away, but if you’re wondering when you can get your hands on a Hyperglobe, I’d say no sooner than 2020.

The Conversation (0)

Metamaterials Could Solve One of 6G’s Big Problems

There’s plenty of bandwidth available if we use reconfigurable intelligent surfaces

12 min read
An illustration depicting cellphone users at street level in a city, with wireless signals reaching them via reflecting surfaces.

Ground level in a typical urban canyon, shielded by tall buildings, will be inaccessible to some 6G frequencies. Deft placement of reconfigurable intelligent surfaces [yellow] will enable the signals to pervade these areas.

Chris Philpot

For all the tumultuous revolution in wireless technology over the past several decades, there have been a couple of constants. One is the overcrowding of radio bands, and the other is the move to escape that congestion by exploiting higher and higher frequencies. And today, as engineers roll out 5G and plan for 6G wireless, they find themselves at a crossroads: After years of designing superefficient transmitters and receivers, and of compensating for the signal losses at the end points of a radio channel, they’re beginning to realize that they are approaching the practical limits of transmitter and receiver efficiency. From now on, to get high performance as we go to higher frequencies, we will need to engineer the wireless channel itself. But how can we possibly engineer and control a wireless environment, which is determined by a host of factors, many of them random and therefore unpredictable?

Perhaps the most promising solution, right now, is to use reconfigurable intelligent surfaces. These are planar structures typically ranging in size from about 100 square centimeters to about 5 square meters or more, depending on the frequency and other factors. These surfaces use advanced substances called metamaterials to reflect and refract electromagnetic waves. Thin two-dimensional metamaterials, known as metasurfaces, can be designed to sense the local electromagnetic environment and tune the wave’s key properties, such as its amplitude, phase, and polarization, as the wave is reflected or refracted by the surface. So as the waves fall on such a surface, it can alter the incident waves’ direction so as to strengthen the channel. In fact, these metasurfaces can be programmed to make these changes dynamically, reconfiguring the signal in real time in response to changes in the wireless channel. Think of reconfigurable intelligent surfaces as the next evolution of the repeater concept.

Keep Reading ↓Show less
{"imageShortcodeIds":[]}