Is it a bird? A plane? Or is it a remotely operated quadrotor conducting surveillance or preparing to drop a deadly payload? Human observers won’t have to guess—or keep their eyes glued to computer monitors—now that there’s superhuman artificial intelligence capable of distinguishing drones from those other flying objects. Automated watchfulness, thanks to machine learning, has given police and other agencies tasked with maintaining security an important countermeasure to help them keep pace with swarms of new drones taking to the skies.
The security challenge has only grown over the past few years: Millions of people have bought consumer drones and sometimes flown them into offlimits areas where they pose a hazard to crowds on the ground or larger aircraft in the sky. Off-the-shelf drones have also become affordable and dangerous weapons for the Islamic State and other militant groups in war-torn regions such as Iraq and Syria.
The need to track and possibly take down these flying intruders has spawned an antidrone market projected to be worth close to US $2 billion by the mid-2020s. The lion’s share of that haul will likely go to companies that can best leverage the power of machine-learning AI based on neural networks.
But much of the antidrone industry still lags behind the rest of the tech sector in making effective use of machine learning AI, says David Romero, founder and managing partner of Black Sage Technologies, based in Boise, Idaho. “With machine learning, 90 percent of the work is figuring out how to make it so simple so that the customer doesn’t have to know how machine learning works,” says Romero. “Many companies do that well, but not in the defense community.”
He and Ross Lam, his Black Sage cofounder, are poised to take advantage of this opening for the upstarts looking to take on the defense industry’s giants. They initially collaborated on a project that trained machine-learning algorithms to automatically detect deer on highways based on radar and infrared camera data. Eventually, they realized that the same approach could help spot drones and other unidentified flying objects.
Since the self-funded startup’s launch in 2015, it has won multiple contracts from the United States government—including for U.S. military forces deployed in Iraq and Afghanistan—and from U.S. allies.
Romero says it’s fairly straightforward to apply machine learning to the task of automatically detecting and classifying flying objects. But because the stakes are high—mistakenly shooting down a small passenger plane or failing to take out an explosives-laden drone intruder could be equally disastrous—Black Sage puts its system through a rigorous training phase when it’s installed at a new site. The system’s radar and infrared cameras capture information about each unidentified flying object’s velocity, size, altitude, and so forth. Then a human operator helps train the machine-learning algorithms by positively identifying certain classes of drones (rotor or fixed-wing) as well as other objects such as birds or manned aircraft. For proof that it has learned its lessons well, the AI is tested against 20 percent of the positively identified data set—the part reserved specifically for cross validation.
Another company called Dedrone—originally based in Kassel, Germany, but currently headquartered in San Francisco—is taking a similar approach. When a Dedrone system is being installed at a new site, humans label unfamiliar objects as part of the training process, which also updates the company’s proprietary DroneDNA library. Since its launch in 2014, Dedrone’s machine-learning software has helped safeguard events and locations such as a Clinton-Trump presidential debate, the World Economic Forum, and CitiField, home of the New York Mets baseball team.
“Each time we update DroneDNA, we process over 250 million different images of drones, aircraft, birds, and other objects,” says Michael Dyballa, Dedrone’s director of engineering. “In the past eight months, we’ve annotated 3 million drone images.”
Though Black Sage’s and Dedrone’s automated detection systems are said to be capable of running without human assistance after their respective training phases, the companies’ clients may choose to put humans in the loop for engaging active defenses, such as jammers or lasers, to take down flying intruders. Such caution is critical at sites like airports, where drone detection accuracy greater than 90 percent still means the occasional false alarm or case of mistaken identity. Even so, a human’s interpretive ability can only supplement the ceaseless vigilance that AI systems will need to provide as the number of drones continues to rise.
Jeremy Hsu has been working as a science and technology journalist in New York City since 2008. He has written on subjects as diverse as supercomputing and wearable electronics for IEEE Spectrum. When he’s not trying to wrap his head around the latest quantum computing news for Spectrum, he also contributes to a variety of publications such as Scientific American, Discover, Popular Science, and others. He is a graduate of New York University’s Science, Health & Environmental Reporting Program.