Augmented Reality Competes With the Real Thing at the 2013 America's Cup

Spectators on the scene in San Francisco get an education in sailboat racing from the AC Liveline system

5 min read
Augmented Reality Competes With the Real Thing at the 2013 America's Cup
Tekla Perry

Spectators sprawled on comfortable mats and beanbag chairs watch Race 4 of the 2013 America's Cup in America's Cup Park in San Francisco.


I went up to the San Francisco waterfront on Sunday to see the America’s Cup action. This series of races, held uniquely close to shore, offers two main official viewing areas (and lots of unofficial ones all along the waterfront). Over at the America’s Cup Village at Marina Green, hardcore race fans pay for grandstand tickets and get a closeup look at the start and the early maneuvering of the race. More casual fans seem to assemble at America’s Cup Park on Piers 27 and 29; that’s where I watched the action.

Much of the crowd, with the exception of a dozen or so New Zealanders wearing their team’s logo on hats, shirts, and flags, seemed new to the sailing scene. But hey, the day was sunny, admission was free, and the spectators filling the park were open to checking out a new sport —in spite of the competition from a Giants baseball game at AT&T Park and a 49ers football game at Candlestick Park. (They were also open to settling into one of the comfy beanbag chairs positioned around the many video screens, or finding a seat in one of the many TV-saturated bars on the pier.)

These are the people—people that generally like sports but don’t quite get sailing—that engineers Stan Honey and Ken Milnes hope to make sailing fans. Honey and Milnes hope these, and other casual spectators who happen to catch a race on TV this week, find the America’s Cup races so exciting that in the future they’ll clamor to watch sailing races on television all the time, not just when there’s a huge international event. Honey and Milnes, hired by the America’s Cup Event Authority, have spent the past three years building a tracking, telemetry, and augmented reality system to make sailing more understandable and exciting for TV viewers. They explain their work in “The Augmented Reality America’s Cup” in this month’s IEEE Spectrum.

Engineers Stan Honey (right), Ken Milnes (second from right) and other members of the AC Liveline team operate the system during Race 3 of the 2013 America's Cup.


A fascinating technical challenge for sure; but could a few graphics that, though complex to create, appear rather simple on screen really make TV coverage of a sailing race into something thrilling?”

My answer? Yes. I knew Honey and Milnes had nailed it when, watching Race 4 on one of the outdoor TV screens at America’s Cup Park, I heard a rising murmur from the crowd—“20,19,18…uh oh…”  The specators were counting meters aloud as a simple white number hovering above the water on the TV image changed. That number indicated the distance in meters separating the boats, and it was dropping fast; Team New Zealand, already three races ahead, was catching up. That’s when I knew that Honey's and Milnes’ effort was worth it—it would have been very hard to tell, at least so quickly, that the distance was closing without that information.

Backup positioning and communications gear waits behind the scenes at the 2013 America's Cup.


I’m sure that it never occurred to most of those spectators that people used to watch sailing races without that information. Or how anyone possibly could.

At another point in the race the crowd gasped when the indicated speed of the Oracle boat—again in a simple white graphic—dropped dramatically. And when the commentator mentioned a layline (the point at which a boat can sail directly to the next buoy that defines the course, without making any turns on the way), folks may not have known the definition, but they easily grasped that it was a good thing for the boat on screen to be traveling along the virtual yellow line, not so good for the boat that had yet to reach it.

Watching the race with augmented reality on the TV screens did create a dilemma for many in the crowd—about ten minutes before the finish folks really should have been getting out of those comfy chairs and strolling over to the edge of the pier to see the boats round the final turn and speed to the finish line, right in front of us. But it would be a lot harder to follow the race action without the TV graphics. Some just stayed put in front of the screens.

America’s Cup does have an app for that. As I walked across the pier I started watching the live feed on my phone (and folks around me, not previously aware there was an app, madly started downloading it). I heard a few comments about how surprised people were that the streaming video feed to the phone worked so well.

The one thing that the TV coverage doesn’t quite capture is just how fast these boats move—they were coming right towards us for what turned out to be a tight finish, Oracle Team USA winning by about eight seconds and then turning sharply to pass alongside the cheering crowd on the pier.

The Oracle Team USA and Emirates Team New Zealand boats sprint to the finish of Race 4 of the 2013 America's cup. The helicopter trailing the boats is capturing video and positioning information that helps the AC Liveline system produce graphics to augment the live television broadcast.


Behind the scenes, Honey and Milnes, while they weren’t actually cheering out loud, were breathing a big sigh of relief. They, along with other members of their team, sat inside one of the many production trailers collected in a Pier 23 warehouse. And they were, not surprisingly, a little tense during the actual races; this, after all, is leading edge (a.k.a. bleeding edge) technology. Would the wireless communications that every piece of their system depended on hold up in what would be a very noisy radio environment? Would the servers running the app handle the demand? Would the equipment on the boats work properly? (Honey admitted to feeling a little nervous about one of the systems that track position, installed on the New Zealand boat; it had earlier responded oddly to a few tests, so he had replaced it that day).

But it all went just fine.

The America’s Cup Series continues Thursday with races 6 and 7, televised around the world starting at 1 pm Pacific Time (over NBC Sports Network in the U.S., TVNZ in New Zealand, and a variety of satellite, cable, air, and internet providers elsewhere). The series will continue on Saturday, Sunday, Tuesday, and Thursday until one team scores 9 wins. An earlier penalty against Oracle Team USA requires that team to win two races before its wins count, so now, after five races, of which Oracle Team USA won one, the score stands 4 to 0 in favor of Emirates Team New Zealand.

For more on the augmented reality technology, see "The Augmented Reality America’s Cup."

To watch replays of the first five races, or to download the app, go to the America’s Cup website.

Photos: Tekla Perry

The Conversation (0)