This article is part of our exclusive IEEE Journal Watch series in partnership with IEEE Xplore.

Anyone walking in the woods who hears the growl of a bear will want to pinpoint where the sound is coming from as quickly as possible—and head in the opposite direction.

“Directional hearing is key for living beings because we don’t just want to hear sounds around us. Ideally, we want to know where they’re coming from as well,” says Lara Díaz-García, a third-year Ph.D. student in the electronic and electrical engineering department of the University of Strathclyde, in Glasgow. “For example, we want to find the potential partner who’s making a mating call, or we want to track the origin of the sounds of the prey that we are trying to hunt.”

Researchers are keen to understand directional hearing not just from a biology perspective but also to create bioinspired products, such as microphones and hearing aids, which could provide simple yet effective methods for determining the direction of sound.

“It would be great to see the ear of this tiny mothinspire some sort of hearing aid one day in the future.”
—Lara Díaz-García, University of Strathclyde

Díaz-García’s Ph.D. thesis aims to unravel the mysteries of a particularly enigmatic hearing system—that of the lesser wax moth, Achroia grisella.

Notably, large animals have ears far enough apart to let them distinguish the differences in timing when a sound reaches one ear relative to the time it takes to reach the other ear. Their brains are then able to use this information to estimate the direction from which the sound came. However, the ears of small animals, such as insects, are so close together that these creatures must rely on other ways of processing sound directionality.

Díaz-García notes that various theories have been proposed about the wayA. grisella is able to determine sound directionality, despite the two inner ears of this moth species being just 600 micrometers apart.

“There’s one last puzzling element to the situation, which is that females with one ear pierced were found to still be able to track the mating call, which led us to the hypothesis that the morphology of the [ear] itself grants the moth directional hearing,” says Díaz-García.

As part of her Ph.D. thesis, Díaz-García sought to test this theory. She used simulations to re-create the structure of the moth’s ear, starting with an oversimplified model and making it increasing complex.

“Our first model was a simple, circular plate, which is very far from the moth ear and didn’t show any directionality whatsoever. Then we added elements until we reached an elliptical plate, with two sections of different thicknesses and an added mass to account for the neuron connection,” she explains.

Orange toned close up photo of a gel like ring 3D-printed sample of an elliptical plate with two thicknesses. Lara Díaz-García

The research team then created a 3D-printed model of the simulated ear and tested it, describing the results in a study published on 17 August in IEEE Sensors Journal. At the part of the ear where neurons are typically attached, the team found that the sound was picked up more strongly when coming from the front of the ear than when it came from the back of the ear. Indeed, it appears that the structure of the ear can help animals determine sound directionality, opening the door for new bioinspired technologies.

“The main advantage of this research is the fact that directionality is achieved with just one element of specific characteristics, in contrast with most directional acoustic sensors we see nowadays, which require arrays to compare the response of at least two microphones,” Díaz-García explains.

Animated gif of a blue sphere on the left and waving movement on an axis on the right.This COMSOL simulation shows how the resonance mode of the shell resembles the movement of the moth’s tympanum.Lara Díaz-García

She says the team is interested in eventually turning their passive structure into a piezoelectric device, which could be useful for various products.

“Directional microphones are incredibly useful in smartphones, where we want to reject ambient noise to focus solely on the speaker,” Díaz-García notes. “They are also desirable for hearing aids. It would be great to see the ear of this tiny mothinspire some sort of hearing aid one day in the future.”

The Conversation (0)

Europe Expands Virtual Borders To Thwart Migrants

Our investigation reveals that Europe is turning to remote sensing to detect seafaring migrants so African countries can pull them back

14 min read
A photo of a number of people sitting in a inflatable boat on the water with a patrol ship in the background.

Migrants in a dinghy accompanied by a Frontex vessel at the village of Skala Sikaminias, on the Greek island of Lesbos, after crossing the Aegean sea from Turkey, on 28 February 2020.

ASSOCIATED PRESS

It was after midnight in the Maltese search-and-rescue zone of the Mediterranean when a rubber boat originating from Libya carrying dozens of migrants encountered a hulking cargo ship from Madeira and a European military aircraft. The ship’s captain stopped the engines, and the aircraft flashed its lights at the rubber boat. But neither the ship nor the aircraft came to the rescue. Instead, Maltese authorities told the ship’s captain to wait for vessels from Malta to pick up the migrants. By the time those boats arrived, three migrants had drowned trying to swim to the idle ship.

The private, Malta-based vessels picked up the survivors, steamed about 237 kilometers south, and handed over the migrants to authorities in Libya, which was and is in the midst of a civil war, rather than return to Malta, 160 km away. Five more migrants died on the southward journey. By delivering the migrants there, the masters of the Maltese vessels, and perhaps the European rescue authorities involved, may have violated the international law of the sea, which requires ship masters to return people they rescue to a safe port. Instead, migrants returned to Libya over the last decade have reported enslavement, physical abuse, extortion, and murders while they try to cross the Mediterranean.

If it were legal to deliver rescued migrants to Libya, it would be as cheap as sending rescue boats a few extra kilometers south instead of east. But over the last few years, Europe’s maritime military patrols have conducted fewer and fewer sea rescue operations, while adding crewed and uncrewed aerial patrols and investing in remote-sensing technology to create expanded virtual borders to stop migrants before they get near a physical border.

Europe’s 125,000 irregular immigrants in 2019 also number fewer per capita than irregular immigrants to the United States, which has just three quarters the population of the European Union yet reported more than 1 million irregular immigrants at its borders in 2019.

Meanwhile, the European Union spends at least €2 billion (US $2.13 billion) a year internally on managing migration, not counting national-level spending. In 2015, under pressure to address migration from Syria’s civil war, European leaders failed to build a working redistribution of asylum seekers, but they did set in motion a legal framework for a newly empowered European border agency.

At that time Frontex had an annual budget of €142 million (US $156 million) and acted as a kind of liaison network between national border agencies. But the post-2015 rules ballooned its budget. By 2020, when Frontex had gained a more independent legal status as an agency of the European Union, its budget had more than tripled to €450 million and was scheduled to climb another 20.6 percent to €543 million in 2021.

Now, Frontex is refocusing its resources from shipboard patrols to aerial and remote sensing, according to its requests for orientation on the latest technology. The cost of shifting from rescue operations to border enforcement may be harder journeys for migrants and the deaths of some.

One migrant with no money

In early 2017, in the forest highlands of eastern Guinea, a man I’ll call Jacob began a journey that would take him across five Saharan countries and multiple failed sea crossings. He first set out from home after his father died, to look for work in Mali, he says. Mali was a conflict zone, so he moved on to Algeria, but he lacked a work permit, and employers would underpay him or fail to pay him altogether. The police hassled him and other migrant workers.

The workers created informal networks and shared information about where they could get work and how to avoid the police. Following those tips, Jacob worked his way across the desert, sometimes accepting loans from employers or traffickers that turned him into a modern indentured servant.

Meantime, the E.U. was slowly changing its hodgepodge of barriers to keep out Jacob and hundreds of thousands of other migrants. When the 2015 migration surge to Europe began, Spain employed one of the most technologically advanced border-control systems in Europe: the Integrated External Vigilance System, or SIVE (the Spanish acronym). Migrants in those years faced a multisensor gauntlet, involving radar and infrared cameras on towers, aboard ships, and on ground vehicles, that sought to centralize situational awareness by combining as much of that data as possible in a control center in Algeciras, in Spain.

If a boat following a smuggler’s route reflected a radar ping back to one of these sensors, and an officer of one of Spain’s national police forces, the Guardia Civil, happened to be watching the screen, Spain could send a ship to intercept the boat. The Guardia Civil credits SIVE with nudging a larger share of Mediterranean migration to the central and eastern routes. The popularity of other routes, such as to Italy’s Lampedusa Island, 140 km from Tunis, and to Greece, some of whose islands are within sight of Turkish beaches, grew.

Disparities in border technology and the adaptability of migrants and their traffickers are among the reasons Europe decided to convert Frontex into a full agency and triple its budget and staff: “Frontex seeks to create cross-border collaboration in a situation which might otherwise result in a spending arms race on border control between E.U. Member States,” says geographer Dan Fisher of the University of Glasgow, who has published on SIVE.

Following Spain’s SIVE experience, in 2011 Frontex invited industry partners to demonstrate tethered surveillance balloons and now uses them in at least two locations to detect migrants who manage to get past its aerial and space-borne sensors. The present model of balloons can remain several hundred meters up in the air for up to 40 days, providing a persistent visual, infrared, and radar sensing capability across an area of around 11,310 square kilometers.

But that’s a tool of last resort, capable of monitoring people who are already on or near European land. First, migrants must cross the Mediterranean, which is dangerous. Jacob, like many Mediterranean migrants, made multiple attempts from several African countries to reach different European countries. He had limited information about the best way to Europe, but the traffickers who had captured him adapt all the time to changing border security situations. They, in turn, take advantage to coerce people like Jacob into working for them. “When I got to Libya, I didn’t have money,” Jacob says. “The traffickers who got me said, ‘Here you have to pay for your jail, then you pay for your journey.’ I told them, ‘I don’t have money, I have a mother who doesn’t work, I’m just a farmer, not a worker for a ministry or government.’”

EU pushed migrants back until courts intervened

While migrants and traffickers fight over the cost of their crossings, European entities have fought over how to stop migrant crossings. For more than a decade, European courts have documented and declared illegal routine European state border agency actions that include navy vessels pushing migrant boats back into international waters and abandoning them there.

In response to those rulings, European governments shifted tactics but not their goal: They began funding Libya and other North African governments through migration control and security pacts. Then European governments and Frontex began investing in tools to detect migrant boats before they reached European waters and calling the Libyan coast guard to pull back migrant boats.

After the 2015 border crisis and in the wake of court rulings against European countries’ “pushback” policies, Frontex, its sister agency the European Maritime Safety Agency (EMSA), and national border agencies invested in large, long-range drones to monitor slices of the Mediterranean, alongside crewed aircraft.

The E.U., through its main research funding program, also began calling on academia to help it make sense of all the new border data. For example, researchers at the Information Technologies Institute (ITI) in Thessaloniki, Greece, won grants as part of a consortia to fuse video data from uncrewed aerial vehicles, ground robots, and sea drones and use machine-vision software to flag likely items of interest, as seen in this demo video:

The systems send alerts to a dashboard accessible on a computer by border staff. The video system is similar to the security cameras that consumers install on their doorbells or in their homes to detect motion, but with a more sophisticated algorithm designed to detect migration and other illegal activity.

“The previous state of the art was that users had one pilot per drone. Our longer-term goal is to move toward operators managing several UxVs [uncrewed vehicles],” says Athanasios Kapoutsis, an ITI engineering researcher. One of the ways they will do that is by simplifying information from each data feed to prevent overwhelming the user. Rather than display raw footage from a camera or radar, or a dashboard with pages and pages of information, consortia members built an augmented-reality display that might suggest, for example, an 80 percent probability of a particular object being a boat. The group conducted its first real-world tests of the system’s ability to detect ships and humans in 2021.

Border authorities are also experimenting with using migrants’ electromagnetic signatures as tracking tools. It’s feasible because traffickers often hand a satellite phone to migrants and tell them to call for help once they are in international or European waters. Satellite phones emit signals that are detectable from space: In 2019, Frontex issued a contract for satellite-phone-detection services. Hawkeye 360, which also markets its service to the shipping industry and security services in countries such as the United States, was the only bidder and won the contract.

Photo of two people in masks sitting in front of screens on a dashboard aboard a patrol aircraft.Crew members of the Royal Danish Air Force inspecting computer screens on board a Frontex aircraft, during a press day in December 2021. Sipa USA via AP

In 2021, Hawkeye 360’s satellite network offered downloads of their detection data a few times a day, using two clusters of three satellites. Now Hawkeye 360 is on track to have 30 satellites in orbit by early 2023, the company says. Frontex officials must have considered the technology promising, because in 2020 they issued another public tender for a new satellite-radio-detection contract. If the technology matures as fast as Hawkeye 360 predicts, border agencies might soon be capable of detecting satellite-phone-carrying migrant boats on a near-hourly basis, long before the boats leave the search-and-rescue zones of North African countries, without needing to keep aloft a fleet of aircraft or drones or sifting through hours of video feeds.

Social scientist Özgün Topak of York University, in Toronto, Canada, calls the shift to remote detection of migrants and cooperation with origin countries an adaptation strategy by European authorities that helps them comply with court orders and laws but continues to shift responsibility to other countries. In other words, European agencies are treating their direct contact with migrants as the problem, instead of treating the migrants’ precarious situations at sea or in North Africa as the problem. The tech is there to rescue Europe from the migrants, not the migrants from the sea.

What the tech is up against

Jacob says he spent somewhere between three and four years trying to migrate from Guinea to Spain. He eventually found work for someone Jacob describes as “connected to traffickers.”

“There are people who work for the traffickers for a year or so,” he says, to pay their way across. One day, Jacob’s contact told him it was his turn to try crossing. But getting in a boat isn’t the same as getting across.

The sea is hard, Jacob says, the boats insecure. The ones he used were made of a plastic that couldn’t endure many hours of exposure to seawater or the gasoline splashed on it by the rickety motor. The weather would change. “People panic from one minute to the next, the boat can have a problem, and then the Moroccan police can catch you at sea and take you back,” he recalls.

On one of those failed journeys, Moroccan police accused Jacob of being a trafficker himself, perhaps because when they asked for someone to help drive the boat back to shore, he agreed. The judge jailed him and couple of the others, then the police dropped them off in a different inland town.

Caught, cuffed

Two men handcuffed together in a police vehicle.

Two men caught by Moroccan police after a failed crossing to Spain. Migrants say the police drove them far inland to delay subsequent attempts.

Courtesy "Jacob"

Resting on a long journey

A group of people dressed for the cold rest against a border wall.

Migrants rest on the Algerian-Moroccan border.

Courtesy "Jacob"

First by land, then by sea

A group of people ride in an open cargo area of a truck in a sandy desert.

Migrants often travel great distances overland before they board a boat to cross the Mediterranean Sea, exposing themselves to dangerous human traffickers and extreme heat.

Courtesy "Jacob"

If his boat had managed to reach international waters and called for help, passing merchant vessels might have taken its passengers, as they have hundreds of other migrants, and then called authorities for guidance on where to deliver them. That is the goal of most migrant boats: Their owners do not equip the boats for success in a full crossing. Passengers are lucky if they can carry more than their ID and mobile phones. The boats often have just enough fuel to get away from the North African coast but not enough to reach Europe.

Some people—not Jacob—also have reasonable grounds for requesting asylum in a safe country. International law protects those people, but if North African authorities capture them first, those people cannot ask European countries for asylum.

Pushing the border toward your neighbors

That may be why the E.U. funds a bevy of projects that aim to stop people from reaching international or European waters. The most cutting-edge involve looking past Europe’s borders. The European Space Agency has pitched using satellite imagery to look for migrant activity before people cross borders, Frontex uses aircraft overflying international portions of the Mediterranean for “early-warning” of potential crossers into European waters, and a since-canceled Frontex tender mentioned a “Pre-warning Mechanism” and forecasting irregular migration using social media. In other words, the border agency wants to see the future on the other side of its borders.

Just as the United States pressured Mexico to crack down on its border with Central American countries, Europe’s real goal may be to predict, detect, and prevent irregular migration from North Africa long before would-be migrants ever leave the shores of Tripoli, Libya’s capital. For example, one E.U.-funded research project published the results of using Google search data to predict international migration. Another set of E.U.-funded projects seek not only to understand perceptions of the E.U. by potential migrants, but also to reach across the border and change those perceptions.

They might not need to. Migrants, including Jacob, give their compatriots back home a clear-eyed perspective: “I tell people who want to cross that it’s very dangerous…it’s not easy like people imagine it in Morocco,” Jacob says.

Civil-society groups have argued that the European practice of calling North African countries to pick up remotely detected migrant boats sidesteps the law of the sea and the right to request asylum ( here and here). The present E.U. strategy may satisfy the narrow orders resulting from pushback-related court rulings in the early 2010s, but a raft of new cases may reshape Europe’s border policies all over again.

In 2019, human rights lawyers took a case against the E.U. to the International Criminal Court in The Hague, in the Netherlands, which most often addresses war crimes. That case is a long shot, says international law scholar David Fernández Rojo of the University of Deusto, in Bilbao, Spain, but it or similar cases could create a more open record of misbehavior and put moral pressure on the E.U. in response to its policies and actions.

For example, in May 2021 the Office of the United Nations High Commissioner for Human Rights reported multiple instances in which Frontex had shared surveillance information with Libyan authorities so that Libyan coast guards could pull migrant boats back to unsafe ports. While the office of the commissioner has limited power to act on its findings, it did recommend that the E.U. stop disembarking migrants in Libya and noted that Europe has an “obligation…to prevent mistreatment by third parties, including private actors or other States operating within their jurisdiction or effective control.”

Citizen sousveillance

While the main players in Europe’s border surveillance and enforcement are large companies and government agencies, it is easier than ever for armchair detectives to find and report on bad border behavior. One tool, called Alarm Phone, is an alliance between migrants and a civil-society group that invites embarked migrants to call or send messages requesting rescue during their sea crossings. The group then alerts maritime rescue authorities and the public, to hold the authorities responsible.

Civil-society groups are also turning to public data to police Europe’s border police. In late 2020 Bellingcat, an investigative group, used public ship and air tracking data together with videos from migrants to accuse Frontex of participating in illegal pushbacks led by the Greek coastguard. Another group, called Space-Eye, developed tools to detect migrant boats in publicly available satellite imagery. It claims to have corroborated at least one pushback using satellite data.

“If it wasn’t for civil society using technology you wouldn’t be hearing about this,” says David Hammond, founder and trustee of Human Rights at Sea, a civil-society organization in Havant, United Kingdom.

Disparities in border technology and the adaptability of migrants and their traffickers are among the reasons Europe decided to convert Frontex into a full agency and triple its budget and staff.

Academic researchers had been nibbling at satellite-powered migrant boat detection for years. Growing amounts of data and processing power mean that now even dilettante data scientists can write blog posts ( here and here) about coding your own boat-detection software. But ability isn’t the same as responsibility: At least one of those data scientists was competing in a public contest funded by Airbus, a major border-technology provider.

“When you deal with people and satellites, it’s always a problem because it’s a thin line between helping and surveillance,” says remote sensing researcher Urša Kanjir at the Research Center of the Slovenian Academy of Arts and Sciences in Ljubljana, Slovenia, who published an Acta Astronautica paper on the possibility of using Sentinel-2 satellite data for rescuing migrant boats.

It’s too early to tell whether sousveillance, or watching the authorities from below, will work in favor of migrants’ human rights or against them. But civil-society accusations of pushbacks have provoked scrutiny from several European oversight organizations: Europe’s antifraud office in January 2021 began investigating Frontex for its alleged participation in pushbacks, among other things. In June 2021 the European Ombudsman reported that Frontex had not implemented its recommendations on dealing with human rights complaints and recommended the agency provide more transparency in its interactions with civil-society groups. In July 2021 a European Parliament working group concluded that it lacked enough evidence to accuse Frontex of violating human rights, despite acknowledging the Bellingcat and other reports. It did say, however, that Frontex “did not prevent [European member state] violations, nor reduced the risk of future fundamental violations.” The series of investigations may have contributed to the European Parliament’s decision in October 2021 to withhold 12% of Frontex’s 2022 budget.

Tech postpones or relocates border confrontations

The border technology arms race isn’t a race to solve the causes of migration. Like any wall, it can only force potential migrants to think harder about how to get across and raise the stakes of failure. While European authorities have invested tens of millions of euros to make it harder for citizens to see how their border patrols push back against migration, whistle-blowing participants, disgruntled neighbors, errors, or mere contempt for international rule of law will reveal what even the most sophisticated technology might hide for a time.

Even if the destination countries detect them along the way, those migrants have rights enshrined in laws set by the destination countries that entitle them to rescue at sea and to apply for asylum in a safe country.

Outbursts of violence, economic precarity, and perhaps even climate change will motivate migrants despite the technological barriers. “The border policy is just a policy to bother migrants, but it will never stop them,” Jacob says. He says he would have preferred to work in a North African country than migrate all the way to Spain, but those countries’ unwillingness to offer legal residency to West Africans and their police brutality made it too dangerous to stay. One friend died after a fall—or being thrown—from a police station’s upper floor in Algeria, he says.

Now his lack of legal residency in Spain hobbles his earning power and prevents him from visiting home, but it is not as dangerous as it was in North Africa. Still, he is isolated from his family: His mother and brother have poor mobile connectivity, so he only has intermittent contact with them. “I don’t know when I’ll see my mother and brother. That’s one of the hardest things now,” Jacob says.

Having migrated through six countries and survived more attempted sea crossings than he can remember, Jacob says migrating to yet another country probably won’t improve his lot. “I don’t want my whole life to be running. I’ve run what I can run, from my country, crossing the sea. The day I’m tired I’ll return to my country.”

By then, thousands of other young people will have begun the same journey. Even if the destination countries detect them along the way, those migrants have rights enshrined in laws set by the destination countries that entitle them to rescue at sea and to apply for asylum in a safe country. Technology won’t change that, says Hammond, the human rights worker. Instead, he says, finding the right balance between the safety of migrants and the safety of the countries to which they want to travel, “has got to be dealt with at a geopolitical level.”

Editor's note: Article was updated on 14 February 2022 to more precisely describe ITI's research project.

Keep Reading ↓Show less
{"imageShortcodeIds":["29177566"]}