Loser: The Ads Have Eyes

This is part of IEEE Spectrum’s SPECIAL REPORT: WINNERS & LOSERS 2009, The Year’s Best and Worst of Technology.

Every year, the Gallup Organization asks Americans to rate various professions for their honesty and ethics, and every year, people in advertising end up at the bottom of the list, alongside used-car salesmen. Fairly or unfairly, both groups are thought to sell people things they wouldn’t want if only they knew more about the products.

How low, then, do you think most people would place a company that wanted to help advertisers learn more about you than you’d agreed to tell? That’s just what TruMedia Technologies, headquartered in Tampa, Fla., proudly claims to do with a facial recognition–based system it has developed for digital advertising screens in airports and malls. The system counts the looks an ad gets (thus TruMedia’s motto, “Every Face Counts”). It also measures how long each look lasts. Finally, it determines the viewer’s gender and fits him or her into one of three age groups. The point of it all is to display the right ad to any given viewer. To a man, the screen might hawk treatments for erectile dysfunction; to a woman, a purse; to a teenager, a video game.

At first glance this practice may seem hardly different from a search engine that flashes certain ads in response to the keywords used in a search. But the key difference is that here the participation is involuntary. TruMedia takes pride in its stealth technology, one that “does not require audience cooperation for measurement or for gathering demographics, thus providing true and unbiased information.” It sounds more like espionage than market research, and indeed, the company’s research division, based in Yehud, Israel, recruits many of its engineers from the Israeli government’s security agencies.

The TruMedia team has perfected a technology that one can easily imagine being used to thwart terrorist attacks to meet the information-gathering needs of commercial entities: a wide-angle, high-resolution camera installed in a billboard captures a digital image and feeds it to recognition software. The software picks out faces in the image and then extracts key metrics—about the geometry of the bone structure, for example, or the distance between the eyes and the mouth. Then it compares the results to what it has extracted from thousands of other portrait photos that have already been tagged according to age and gender.

Not that age and gender are the only possible categories. At an October WebbyConnect meeting in Laguna Niguel, Calif., TruMedia’s chief measurement officer, Vicki Rabenou, said the technology could classify a viewer by his ethnic group and serve up an ad that targets that group. She added that TruMedia would start doing this in 2009. Much of the audience seemed stunned, and now the chief technology officer, Itzhak Wilf, denies having any such plan—right now, anyway. But he won’t rule the idea out, either. If he gets an offer to incorporate ethnicity classification, he says he’ll “make the decision at that time.”

Does the thought of your face being photographed while you stare at an ad make you queasy? TruMedia says not to worry, because its software sorts images in real time and besides, it’s the company’s policy to store nothing, not even when advertisers have asked that it do so. “If you want to do surveillance, you need to put your own camera,” Wilf says. Besides, he says, the software isn’t sophisticated enough to recognize specific facial features. “We don’t have a database of people, and we don’t attempt to identify people. If you take one of our boxes home from a mall, you will find nothing there.”

You can already find camera-equipped digital signs in malls, supermarkets, and airports. AdSpace Networks, a New York City–based company that operates the largest network of digital video screens in U.S. malls, has embedded TruMedia’s technology in three malls. All Samsung-brand digital signs will carry the technology starting 2009. And it is deployed on an even larger scale in Europe.

TruMedia isn’t the only company producing billboard technology. In May The New York Times reported that Paris-based Quividi had embedded similar technology in small digital signs in Europe and Singapore and was entering the U.S. advertising arena.

Advertisers seem to be happy. TruMedia won a 2008 best product/service award from the Digital Signage Group, a digital sign distributor. Wilf says that the technology is critical for people who buy ad space and time, helping them schedule and plan ad campaigns more effectively. “If you know during lunchtime that viewers are mostly female and view time is 10 to 15 seconds, that’s important information to plan your next campaign,” he says.

Why, you might ask, should anyone care about ads that watch us when we’re already being monitored in public by other means? After all, today’s shops and malls routinely use surveillance cameras to record our every move, which Wilf notes is “much more intrusive” than what TruMedia is doing.

But this trend in the attitude toward privacy in public places is precisely what we should be worried about, says Lee Tien, a senior staff attorney at the Electronic Frontier Foundation, in San Francisco. It’s bad enough, he says, that people have no way to avoid being videotaped for security purposes; it’s worse still to have your face captured for the purpose of selling products. “People not liking it is the entire creepiness of this type of technology,” he says. “The point is, you don’t have any privacy.”

Perhaps Internet services that track what we view and what we click on have inured us to surveillance. Tien insists that there is a difference: “At least you have a known relationship with Google,” he says, adding “that’s not to say that covert online tracking is good. But covert physical real-world tracking is pretty sick.”

Tien is also skeptical of the no-record policy. The software might not be set up to store data right now, but it has the potential to, and there’s no guarantee that the company can’t do it down the road, he says—especially if prodded by law enforcement or the U.S. Department of Homeland Security.

And even if the software cannot identify people right now, it could easily be made to do so with just a few tweaks. If it already has enough information to judge an onlooker’s age and gender, says Anil Jain, a biometrics recognition expert at Michigan State University, then it can also re-create the face image, although probably at a much lower resolution. Again, that’s information that people could use if they wanted to. “Maybe the authorities want to know if a suspect went by that street on a particular day and time,” he suggests.

For now, we have TruMedia’s pledge that nothing about anyone’s face could get into the wrong hands. And, perhaps, we can look forward to the day when digital screens in malls all over the country will show lipstick ads around lunchtime.

For more articles, go to Winners & Losers 2009 Special Report.