The December 2022 issue of IEEE Spectrum is here!

Close bar

Loser: The Ads Have Eyes

TruMedia’s facial-recognition system lets advertisers watch you watching their ads—whether you want them to or not

5 min read
Photo of an electronic billboard.
Photo: TruMedia Technologies

This is part of IEEE Spectrum’s SPECIAL REPORT: WINNERS & LOSERS 2009, The Year’s Best and Worst of Technology.

Every year, the Gallup Organization asks Americans to rate various professions for their honesty and ethics, and every year, people in advertising end up at the bottom of the list, alongside used-car salesmen. Fairly or unfairly, both groups are thought to sell people things they wouldn’t want if only they knew more about the products.

How low, then, do you think most people would place a company that wanted to help advertisers learn more about you than you’d agreed to tell? That’s just what TruMedia Technologies, headquartered in Tampa, Fla., proudly claims to do with a facial recognition–based system it has developed for digital advertising screens in airports and malls. The system counts the looks an ad gets (thus TruMedia’s motto, “Every Face Counts”). It also measures how long each look lasts. Finally, it determines the viewer’s gender and fits him or her into one of three age groups. The point of it all is to display the right ad to any given viewer. To a man, the screen might hawk treatments for erectile dysfunction; to a woman, a purse; to a teenager, a video game.

At first glance this practice may seem hardly different from a search engine that flashes certain ads in response to the keywords used in a search. But the key difference is that here the participation is involuntary. TruMedia takes pride in its stealth technology, one that “does not require audience cooperation for measurement or for gathering demographics, thus providing true and unbiased information.” It sounds more like espionage than market research, and indeed, the company’s research division, based in Yehud, Israel, recruits many of its engineers from the Israeli government’s security agencies.

The TruMedia team has perfected a technology that one can easily imagine being used to thwart terrorist attacks to meet the information-gathering needs of commercial entities: a wide-angle, high-resolution camera installed in a billboard captures a digital image and feeds it to recognition software. The software picks out faces in the image and then extracts key metrics—about the geometry of the bone structure, for example, or the distance between the eyes and the mouth. Then it compares the results to what it has extracted from thousands of other portrait photos that have already been tagged according to age and gender.

Not that age and gender are the only possible categories. At an October WebbyConnect meeting in Laguna Niguel, Calif., TruMedia’s chief measurement officer, Vicki Rabenou, said the technology could classify a viewer by his ethnic group and serve up an ad that targets that group. She added that TruMedia would start doing this in 2009. Much of the audience seemed stunned, and now the chief technology officer, Itzhak Wilf, denies having any such plan—right now, anyway. But he won’t rule the idea out, either. If he gets an offer to incorporate ethnicity classification, he says he’ll “make the decision at that time.”

Does the thought of your face being photographed while you stare at an ad make you queasy? TruMedia says not to worry, because its software sorts images in real time and besides, it’s the company’s policy to store nothing, not even when advertisers have asked that it do so. “If you want to do surveillance, you need to put your own camera,” Wilf says. Besides, he says, the software isn’t sophisticated enough to recognize specific facial features. “We don’t have a database of people, and we don’t attempt to identify people. If you take one of our boxes home from a mall, you will find nothing there.”

You can already find camera-equipped digital signs in malls, supermarkets, and airports. AdSpace Networks, a New York City–based company that operates the largest network of digital video screens in U.S. malls, has embedded TruMedia’s technology in three malls. All Samsung-brand digital signs will carry the technology starting 2009. And it is deployed on an even larger scale in Europe.

TruMedia isn’t the only company producing billboard technology. In May The New York Times reported that Paris-based Quividi had embedded similar technology in small digital signs in Europe and Singapore and was entering the U.S. advertising arena.

Advertisers seem to be happy. TruMedia won a 2008 best product/service award from the Digital Signage Group, a digital sign distributor. Wilf says that the technology is critical for people who buy ad space and time, helping them schedule and plan ad campaigns more effectively. “If you know during lunchtime that viewers are mostly female and view time is 10 to 15 seconds, that’s important information to plan your next campaign,” he says.

Why, you might ask, should anyone care about ads that watch us when we’re already being monitored in public by other means? After all, today’s shops and malls routinely use surveillance cameras to record our every move, which Wilf notes is “much more intrusive” than what TruMedia is doing.

But this trend in the attitude toward privacy in public places is precisely what we should be worried about, says Lee Tien, a senior staff attorney at the Electronic Frontier Foundation, in San Francisco. It’s bad enough, he says, that people have no way to avoid being videotaped for security purposes; it’s worse still to have your face captured for the purpose of selling products. “People not liking it is the entire creepiness of this type of technology,” he says. “The point is, you don’t have any privacy.”

Perhaps Internet services that track what we view and what we click on have inured us to surveillance. Tien insists that there is a difference: “At least you have a known relationship with Google,” he says, adding “that’s not to say that covert online tracking is good. But covert physical real-world tracking is pretty sick.”

Tien is also skeptical of the no-record policy. The software might not be set up to store data right now, but it has the potential to, and there’s no guarantee that the company can’t do it down the road, he says—especially if prodded by law enforcement or the U.S. Department of Homeland Security.

And even if the software cannot identify people right now, it could easily be made to do so with just a few tweaks. If it already has enough information to judge an onlooker’s age and gender, says Anil Jain, a biometrics recognition expert at Michigan State University, then it can also re-create the face image, although probably at a much lower resolution. Again, that’s information that people could use if they wanted to. “Maybe the authorities want to know if a suspect went by that street on a particular day and time,” he suggests.

For now, we have TruMedia’s pledge that nothing about anyone’s face could get into the wrong hands. And, perhaps, we can look forward to the day when digital screens in malls all over the country will show lipstick ads around lunchtime.

For more articles, go to Winners & Losers 2009 Special Report.

About the Author

PRACHI PATEL-PREDD, a regular contributor to IEEE Spectrum, is a freelance writer who covers technology, energy, and the environment. She is also frequently heard on Spectrum Radio.

Keep reading...Show less

This article is for IEEE members only. Join IEEE to access our full archive.

Join the world’s largest professional organization devoted to engineering and applied sciences and get access to all of Spectrum’s articles, podcasts, and special reports. Learn more →

If you're already an IEEE member, please sign in to continue reading.

Membership includes:

  • Get unlimited access to IEEE Spectrum content
  • Follow your favorite topics to create a personalized feed of IEEE Spectrum content
  • Save Spectrum articles to read later
  • Network with other technology professionals
  • Establish a professional profile
  • Create a group to share and collaborate on projects
  • Discover IEEE events and activities
  • Join and participate in discussions

Deep Learning Gets a Boost From New Reconfigurable Processor

The ReAAP processor allows AI to be faster, more efficient

2 min read
different colored beams of light shooting up

This article is part of our exclusive IEEE Journal Watch series in partnership with IEEE Xplore.

Deep learning is a critical computing approach that is pushing the boundaries of technology – crunching immense amounts of data and uncovering subtle patterns that humans could never discern on their own. But for optimal performance, deep learning algorithms need to be supported with the right software compiler and hardware combinations. In particular, reconfigurable processors, which allow for flexible use of hardware resources for computing as needed, are key.

Keep Reading ↓Show less

Rory Cooper’s Wheelchair Tech Makes the World More Accessible

He has introduced customized controls and builds wheelchairs for rough terrain

6 min read
portrait of a man in a navy blue polo with greenery in the background
Abigail Albright

For more than 25 years, Rory Cooper has been developing technology to improve the lives of people with disabilities.

Cooper began his work after a spinal cord injury in 1980 left him paralyzed from the waist down. First he modified the back brace he was required to wear. He then turned to building a better wheelchair and came up with an electric-powered version that helped its user stand up. He eventually discovered biomedical engineering and was inspired to focus his career on developing assistive technology. His inventions have helped countless wheelchair users get around with more ease and comfort.

Keep Reading ↓Show less

Learn How Global Configuration Management and IBM CLM Work Together

In this presentation we will build the case for component-based requirements management

2 min read

This is a sponsored article brought to you by 321 Gang.

To fully support Requirements Management (RM) best practices, a tool needs to support traceability, versioning, reuse, and Product Line Engineering (PLE). This is especially true when designing large complex systems or systems that follow standards and regulations. Most modern requirement tools do a decent job of capturing requirements and related metadata. Some tools also support rudimentary mechanisms for baselining and traceability capabilities (“linking” requirements). The earlier versions of IBM DOORS Next supported a rich configurable traceability and even a rudimentary form of reuse. DOORS Next became a complete solution for managing requirements a few years ago when IBM invented and implemented Global Configuration Management (GCM) as part of its Engineering Lifecycle Management (ELM, formerly known as Collaborative Lifecycle Management or simply CLM) suite of integrated tools. On the surface, it seems that GCM just provides versioning capability, but it is so much more than that. GCM arms product/system development organizations with support for advanced requirement reuse, traceability that supports versioning, release management and variant management. It is also possible to manage collections of related Application Lifecycle Management (ALM) and Systems Engineering artifacts in a single configuration.

Keep Reading ↓Show less