LiShield Can Block Smartphone Cameras for Privacy's Sake

Rules that prohibit photos or videos can prove almost impossible to enforce when nearly everyone carries a smartphone. But a new indoor privacy system has shown how the power of smart LED lighting could prevent people from taking illegal videos of a live Broadway show or surreptitiously snapping photos during a secretive trade show presentation.

The prototype privacy measure uses a smart LED to give off a high-frequency flickering pattern that interferes with the camera sensors on mobile devices such as smartphones. Such flickering creates a vertical striped pattern effect in the photo or video frames taken by digital smartphone cameras without interfering with human eyesight or harming human eyes. It’s a first step toward enabling a cloak of privacy to be thrown over live events or exhibits, even in the presence of dozens of smartphones.

“Previously, when you are in locations like gyms or exam rooms or trade shows, in those locations the cameras are forbidden but there was no way to enforce the compliance,” says Xinyu Zhang, assistant professor in electrical and computer engineering at the University of California San Diego. “Our main achievement with LiShield is automating digital privacy.”

The “LiShield” privacy system specifically targets consumer digital cameras such as smartphone-grade cameras by exploiting their limitations in capturing images. Zhang and his colleagues presented their research on the first day of the ACM Mobicom 2017 conference held from 16-20 October at the Snowbird Ski Resort near Salt Lake City.

Digital cameras use a rolling shutter sampling approach to capture images by exposing each row of pixels in sequence. The on-off flickering patterns of the LiShield system ensure that some rows of pixels are overexposed and appear bright while other rows are underexposed and appear dark. That creates the striped patterning in the captured images or video frames to ruin the overall photo or video.

Such a privacy system is imperceptible to human eyes because it uses high-frequency flickering patterns beyond the limits of human eye sensitivity at around 80 Hertz. And while human eyes have the capacity to perceive very different levels of brightness across a dynamic range within the same scene, cameras tend to suffer from the overexposure or underexposure problems when exposed to lighting with vastly different intensities within the same scene. This is what makes the LiShield’s flickering patterns so effective.

LiShield’s capability to corrupt digital camera images and videos without interfering with human eyesight is an advantage over some past attempts to enable similar privacy protection. “Strong infrared light can saturate the camera and block it from capturing images, but that infrared light would burn human eye retinas,” says Shilin Zhu, a Ph.D. student in computer science at the University of California San Diego and co-primary author on the paper. “We have to use the visible light spectrum for safety reasons.”

LiShield can also allow specific “authorized users” to use their smartphones or digital cameras. For capturing video, authorized camera must use secure side channels to communicate its exposure time setting and synchronize its clock with the smart LED’s clock. That allows the smart LED to send recoverable waveform information about the flickering pattern in use during the authorized camera’s operation. For capturing still images, the smart LED synchronizes with the authorized camera to help reconstruct a still image taken from several complementary frames in a very short video.

The current LiShield prototype relies upon custom-built smart LED hardware and software algorithms used to control both the defensive flickering patterns and synchronization with an authorized camera. Such a system could easily be integrated into existing commercial smart LED lights already on the market, according to the researchers. 

This system still only works within indoor areas that can be illuminated by smart LED lighting, and only under certain situations. For example, LiShield cannot disrupt the photos or videos taken by professional global-shutter cameras that capture the entire scene all at once. Its flickering pattern defense will also fail when the smart LED lighting is overwhelmed by the brightness of stronger ambient light such as sunlight.

Still, LiShield has one more trick to play if strong ambient light threatens to weaken the protection of its flickering pattern defense. Its patterns can still form the equivalent of faint, low-contrast stripes that act as nearly invisible barcodes within smartphone camera images or videos. Such barcodes could then be automatically detected by the online servers of social media networks such as Facebook or Snapchat, alerting the social media servers to block such images or videos taken without permission.

To make LiShield truly practical, the researchers need to go beyond their initial test phase, which involved a single smart LED and a fairly small indoor space. Eventually, the researchers hope to find commercial partners who might be willing to team up for more ambitious real-world testing during live events.

“If you want to protect a large area like a theater, you need to install multiple smart LEDs,” says Chi Zhang, a Ph.D. student in electrical and computer engineering at the University of Wisconsin-Madison and the second co-primary author on the paper. “Synchronization between multiple LEDs in this entire area is a problem to solve in the future.”

The team also wants to figure out ways to enable multiple authorized camera or smartphone users to take photos or videos during LiShield’s operation—a challenge given how the current system can only synchronize between a single digital camera and the smart LED. One possibility is that LiShield could quickly hop between authorized users and enable them to take photos or videos in a round-robin scheduling fashion.

There is also a possibility of taking LiShield’s privacy protection even further sometime in the future. Some potential military customers asked if the researchers could develop a privacy system that fully protects a target area by blacking out the video or images taken at the scene. 

“That’s a kind of demanding application,” Xinyu Zhang says. “We still want the system to be as simple as replacing a light bulb, but we probably need to design smart algorithms like specialized waveforms that can realize that full protection.”