Attention Rogue Drone Pilots: AI Can See You!

The minute details of rogue drone’s movements in the air may unwittingly reveal the drone pilot’s location—possibly enabling authorities to bring the drone down before, say, it has the opportunity to disrupt air traffic or cause an accident. And it’s possible without requiring expensive arrays of radio triangulation and signal-location antennas.

So says a team of Israeli researchers who have trained an AI drone-tracking algorithm to reveal the drone operator’s whereabouts, with a better than 80 per cent accuracy level. They are now investigating whether the algorithm can also uncover the pilot’s level of expertise and even possibly their identity.

Gera Weiss—professor of computer science at Ben-Gurion University of the Negev in Beersheba, Israel—said the algorithm his team has developed partly relies on the specific terrain around an airport or other high-security location.

After testing neural nets including dense networks and convolutional neural networks, the researchers found that a kind of recurrent neural net called a “gated-recurrent unit” (GRU) network worked best for drone tracking. “Recurrent networks are good at this,” Weiss said. “They consider the sequenced reality of the data—not just in space but in time.”

So, he said, a security professional at an airport, for instance, would hire white-hat malefactors to launch a drone from various locations around the airport. The security team would then record the drone’s exact movements on airport radar systems.

Ultimately, the GRU algorithm would then train on this data—knowing in this case the pilot’s location and the peculiar details of the drone’s flight patterns.

Depending on the specific terrain at any given airport, a pilot operating a drone near a camouflaging patch of forest, for instance, might have an unobstructed view of the runway. But that location might also be a long distance away, possibly making the operator more prone to errors in precise tracking of the drone. Whereas a pilot operating nearer to the runway may not make those same tracking errors but may also have to contend with big blind spots because of their proximity to, say, a parking garage or control tower.

And in every case, he said, simple geometry could begin to reveal important clues about a pilot’s location, too. When a drone is far enough away, motion along a pilot’s line of sight can be harder for the pilot to detect than motion perpendicular to their line of sight. This also could become a significant factor in an AI algorithm working to discover pilot location from a particular drone flight pattern.

The sum total of these various terrain-specific and terrain-agnostic effects, then, could be a giant finger pointing to the operator. This AI application would also be unaffected by any relay towers or other signal spoofing mechanisms the pilot may have put in place.

Weiss said his group tested their drone tracking algorithm using Microsoft Research’s open source drone and autonomous vehicle simulator AirSim. The group presented their work-in-progress at the Fourth International Symposium on Cyber Security, Cryptology and Machine Learning at Ben-Gurion University earlier this month.

Their paper boasts a 73 per cent accuracy rate in discovering drone pilots’ locations. Weiss said that in the few weeks since publishing that result, they’ve now improved the accuracy rate to 83 per cent.

Now that the researchers have proved the algorithm’s concept, Weiss said, they’re hoping next to test it in real-world airport settings. “I’ve already been approached by people who have the flight permissions,” he said. “I am a university professor. I’m not a trained pilot. Now people that do have the facility to fly drones [can] run this physical experiment.”

Weiss said it’s as yet unclear how terrain-agnostic their algorithm is. Could a neural net trained on the terrain surrounding one airport then be effectively deployed at another airport—or another untrained region of the same airport?

Another open question, he said, involves whether the algorithm could also be reversed: Could drone flight patterns around an unmapped terrain then be used to discover features of the terrain?

Weiss said they hope to tackle these questions in future research, alongside possible applications of a series of recent findings that attempt to rate operator skill levels from motion tracking data.

One finding even goes so far as to pinpoint and classify idiosyncrasies in piloting skills—so perhaps repeat offenders might one day be spotted by the thumbprint their copter leaves on the sky.