# IEEE Spectrum Online: A Blogging Sampler

For those of you who haven’t yet spent much time on IEEE Spectrum’s Web site (), we thought we’d bring to your attention some of the interesting stuff that’s being discussed in our blog, Tech Talk (/techtalk). We’ll be adding lots of new bells and whistles to the site in the coming months, and blogging (by staff members and industry experts) will be greatly expanded. Here are some excerpts from recent postings. Visit! Comment!

Things You May Not Know About Numbers

23 February

Senior Associate Editor Steven Cherry looked into a Web site that wants to have something nice to say about every integer from 0 to 9999.

Now this is what makes the Web special.

Erich Friedman, an associate professor of mathematics at Stetson University, in DeLand, Fla., has a page called ”What’s Special About This Number?” (http://www.stetson.edu/~efriedma/numbers.html). It’s a list of interesting facts about some of the numbers from 0 to 9999.

Here are a few examples: 5 is the number of Platonic solids; 6 is the smallest perfect number; 7 is the smallest number of faces of a regular polygon that is not constructible by straightedge and compass; 8 is the largest cube in the Fibonacci sequence.

Did you know that 38 is the last Roman numeral when listed alphabetically (”lexicographically” in math lingo)? Or that 40 is the only number whose letters are in alphabetical order? Or that 727 has the property that its square is the concatenation of two consecutive numbers? I sure didn’t.

Friedman doesn’t find an interesting fact for every number between 0 and 9999, but he does for 2849 of them, which is remarkable. Best of all, he is asking people to eâ''mail him ( efriedma@stetson.edu ) if they know of a distinctive fact about a number that hasn’t made his list yet.

We have a suggestion for the number 2600: it’s the number of nuclear power plants needed to equal the energy contained in 1 cubic mile of oil--which happens to be the amount the world currently uses ­annually. And 200 is the number of Three Gorges Dams it would take, and 5200 is the number of coal-fired plants needed. You see, Spectrum ’s editors are almost as obsessed with numbers as Friedman. [ Editor’s note: For proof of that principle, see ” Joules, BTUs, Quads--Let’s Call the Whole Thing Off” in the News section of January’s issue.]

Eyes On The Automotive X Prize

28 February

Automotive Editor John Voelcker got an early view of the competition for the X Foundation’s latest prize.

As everyone in the auto industry will tell you ad nauseam, cars are different. It’s a brutally competitive, capital-intensive, globe-spanning business, and these days, consumer tastes change faster than tooling cycles. So, how do you spur radical cuts in energy use and carbon dioxide emissions? Throw money at it! At least, that’s the theory behind the Automotive X Prize (http://www.xprize.org/xprizes/automotive_x_prize.html). It’s the latest privately funded competition to encourage technology innovation in a variety of disciplines, all courtesy of the X Prize Foundation.

The foundation’s previous competitions each had a fairly clear goal: to create the first privately developed spacecraft that launches successfully, for instance. The AXP is proving--well, different. And it brings with it some challenging questions: How should you weight the goal of reducing carbon emissions against that of lowering overall energy consumption? How do you compare energy usage among vehicles powered by gasoline, regular diesel or biodiesel, ethanol, methanol, natural gas--and electricity? Whose grid do you use to measure the greenhouse gases produced by generating that electricity? Should you take life-cycle energy consumption into account? And so on.

Mark Goodstein, the AXP’s cheerfully energetic executive director, took the classic approach. He invited two dozen world experts into a windowless conference room to air their views for a day, carefully moderating the discussions to prevent bloodshed. The decisions from this Energy Efficiency Working Group, held 9 February in Pasadena, Calif., will feed into the final set of rules for the Automotive X Prize, which are to be released 4 April at the New York Auto Show. I’ve been sworn to secrecy about the proceedings, but we’ll have a full story online the day the rules come out. [Editor’s note: In the meantime, check out Voelcker’s latest story, ”Top Ten Tech Cars,” in this issue.]

The editorial content of IEEE Spectrum magazine does not reflect official positions of the IEEE or its organizational units. Please address comments to Forum at n.hantman@ieee.org .

Join the world’s largest professional organization devoted to engineering and applied sciences and get access to all of Spectrum’s articles, podcasts, and special reports. Learn more →

## Membership includes:

• Follow your favorite topics to create a personalized feed of IEEE Spectrum content
• Save Spectrum articles to read later
• Network with other technology professionals
• Establish a professional profile
• Create a group to share and collaborate on projects
• Discover IEEE events and activities
• Join and participate in discussions

## How Police Exploited the Capitol Riot’s Digital Records

### Forensic technology is powerful, but is it worth the privacy trade-offs?

Vertical
Gabriel Zimmer
Green

The group of well-dressed young men who gathered on the outskirts of Baltimore on the night of 5 January 2021 hardly looked like extremists. But the next day, prosecutors allege, they would all breach the United States Capitol during the deadly insurrection. Several would loot and destroy media equipment, and one would assault a policeman.

No strangers to protest, the men, members of the America First movement, diligently donned masks to obscure their faces. None boasted of their exploits on social media, and none of their friends or family would come forward to denounce them. But on 5 January, they made one piping hot, family-size mistake: They shared a pizza.

According to charging documents, at 10:57 that evening, a PayPal account registered to a Gmail address paid US \$84.72 to Domino’s Pizza in Arbutus, Md. Minutes later, that email account received Venmo payments from users called Thomas Carey, Gabe Chase, and Jon Lizak. A separate Venmo email showed a payment from “Broseph Broseph,” a nickname of another friend, Joseph Brody.

After the horrific events of the next day, the Federal Bureau of Investigation swung into action. It served cell service and tech companies with geofence warrants—search warrants demanding details on every device and app active within a specified geographic area. One of these warrants, served on Google and covering the interior of the Capitol, showed that a device associated with the Gmail account in question entered the Senate Wing door at 2:18 p.m. on 6 January.

Connecting that Gmail account to a phone number and then to its owner, Paul Lovley of Halethorpe, Md., was just a matter of a few keystrokes on law-enforcement databases. All that remained was for an FBI agent on stakeout to observe Lovley taking out the trash one night and match his photo to one of a figure captured by Senate surveillance cameras during the riot. Lovley and his four compatriots were charged with a range of federal crimes in September 2022.

The riot was an unprecedented attack on American democracy, with thousands of citizens, most of them previously unknown to federal investigators, violently storming the seat of government. The resulting investigations were the largest in U.S. history, offering a snapshot of the rapidly evolving nature of law enforcement and how heavily it now relies on data provided, wittingly or not, by suspects themselves.

While it might seem as though the Capitol-riot investigations represent state-of-the-art digital forensics, “those surveillance technologies are being used in even minor low-level criminal cases across the country every single day,” says Jennifer Lynch, surveillance litigation director at the Electronic Frontier Foundation (EFF). “The FBI did not use anything new. They just used it at a much larger scale.”

IEEE Spectrum analyzed hundreds of criminal complaints and other legal filings from the Capitol attacks to understand that reach and scale, and to consider the legal and social consequences of the government’s power to delve into its citizens’ digital lives. That power might seem reassuring when applied to a mob intent on overturning a presidential election, but perhaps less so when brought to bear on people protesting, say, human-rights violations.

## Social media provides clues for digital forensics

Police work has always involved the connecting of dots, whether photos, phone calls, testimony, or physical evidence. The 6 January investigation showed the power of seeking the digital connections between those dots.

Over the past two years, the U.S. Department of Justice and the Program on Extremism at George Washington University have made available thousands of legal documents about those charged in connection with the 6 January riot. Spectrum analyzed all those containing details of how alleged perpetrators were identified and investigated: 884 individuals by mid-December. Many were identified using time-honored techniques: Wanted posters remain a powerful tool, these days reaching a global audience via news organizations, the FBI’s website, and social media. Nearly two-thirds of all those people were first identified via tips from witnesses, friends, family, and other human sources. The FBI ultimately received more than 300,000 such tips.

But the ways in which those sources spotted the alleged perpetrators have changed enormously. Only a tiny fraction of sources were on the ground in Washington, D.C., on 6 January. And although some suspects were recognized in TV reports or news stories, most were spotted on social media.

In almost two-thirds of the cases, evidence was cited from one or more social-media platforms. Facebook appeared in almost half of all cases, cited 388 times, followed by Instagram and Twitter with a combined total of 188 mentions. But almost every major social-media app was mentioned in at least one case: LinkedIn, MeWe, Parler, Signal, Snapchat, Telegram, TikTok, even dating app Bumble and shopping-focused Pinterest.

Investigators immediately exploited the rioters’ use of Facebook. On the day of the attack, the FBI requested that Facebook identify “any users that broadcasted live videos which may have been streamed and/or uploaded to Facebook from physically within the building of the United States Capitol during the time on January 6, 2021, in which the mob had stormed and occupied the Capitol building.” Complying with this request was possible because Facebook records the latitude and longitude of every uploaded photo and video by default.

Facebook responded the very same day, and again over the next few weeks, with an unknown number of user IDs—unique identifiers assigned to accounts on Facebook and Instagram (which Facebook’s parent company, Meta, also owns). The legal documents suggest that about 35 rioters were identified this way, without first being named by witnesses. In many cases, the FBI then requested that Facebook send it the relevant images and videos and other account data.

Investigators gleaned further clues from many hours of professional news footage, as well as 14,000 hours of high-resolution video from dozens of fixed security cameras and 2,000 hours of video from body-worn cameras operated by police responding to the riot. Surveillance cameras were referenced in 63 percent of DOJ cases, open-source videos and social-media images in 41 percent, and body-camera and news footage each in about 20 percent of cases.

Processing these files involved a huge amount of human effort. The body-camera footage alone required a team of 60, who laboriously completed a 752-page spreadsheet detailing relevant clips.

Shortly after the 6 January riot, Spectrum reported on how automated image-recognition systems could be brought to bear on this flood of audiovisual information. The FBI assigned its FACE Services Unit to compare suspects’ faces with images in state and federal face-recognition systems. However, according to the legal documents, only 25 rioters appear to have been first identified through such automated image searches, mostly after comparisons with state driver’s license photos and passport applications.

Hoan Ton-That, CEO of Clearview AI, a face-recognition search engine that indexes 30 billion images from the open Internet, told Spectrum that the court filings do not necessarily reflect how often such technology was used. “Law enforcement don’t always have to disclose that they found a certain person’s information through facial recognition,” he says.

Crowds throng the U.S. Capitol Building, in Washington, D.C., on 6 January 2021.Evelyn Hockstein/Washington Post/Getty Images

Ton-That notes that Clearview’s algorithm is not yet admissible in court, and that any identification it makes from open-source imagery requires further vetting and confirmation. Without providing specifics, he suggested that Clearview’s system was used by the FBI. “As a company, it was gratifying for us to play a small role in helping apprehend people who caused damage and stormed the Capitol,” he told Spectrum. The Capitol riot wouldn’t have been the first time that such technology was applied in this way. Facial recognition was reportedly used to identify protestors at a Black Lives Matter event in New York City in 2020 and at similar protests across the United States.

Computers are generally much better at recognizing letters and numbers than faces; automatic license plate reader (ALPR) technology was cited in 20 of the DOJ cases. There are likely tens of thousands of fixed and mobile ALPR systems in the United States alone, at toll plazas, bridge crossings, and elsewhere, capturing hundreds of millions of car journeys each month.

## How digital data makes it easier to connect the dots

A single stream of data may help a little, but the integration of many such streams can do wonders. Take the case of William Vogel. He was first named by a tipster who sent the FBI a Snapchat video filmed by someone, unpictured, inside the Capitol building. Sure enough, a Facebook account associated with the Snapchat account listed Vogel as its owner and included a cellphone number.

But maybe someone stole Vogel’s cellphone and his Snapchat login to shoot and upload the video. Vogel’s phone number led to an address in Pawling, N.Y., and to a car registered to Vogel. The FBI then logged on to ALPR systems across several states, revealing that Vogel’s vehicle had taken the Henry Hudson Bridge from the Bronx into Manhattan at 6:06 a.m. on 6 January, entered New Jersey at 7:54, and proceeded southbound through Baltimore at 9:15. The car made its return journey late that afternoon, eventually crossing back into New York a minute before midnight.

But, again, perhaps someone had borrowed Vogel’s car? Not according to an ALPR photo snapped in rural Maryland at 8:44 a.m. It shows a distinctive large red “Make America Great Again” hat on the car’s dashboard, just like one that Vogel was wearing when he was filmed on a news broadcast outside the Capitol later that day, and in a Facebook selfie.

“They’re trying to report me to the FBI/DOJ and put me away for 10 years for domestic terrorism, because of my Snapchat story,” Vogel complained later via Facebook Messenger, after admitting to a friend that he had in fact shot the Capitol video, charging documents allege. Vogel’s case goes to trial in February 2023, when he will face charges of violently entering the Capitol and disorderly conduct.

Investigators also homed in on people by looking at data from their cellphones. At least 2,000 digital devices were searched by the FBI for images, data, and messages. The FBI’s Cellular Analysis Survey Team is dedicated to locating cellphones based on which cell towers they access. Although the FBI got rough locations for about one-fifth of the Capitol-riot defendants this way, it’s too imprecise to reliably indicate whether someone actually breached the Capitol itself or remained outside the building.

Far more accurate are the geolocation data gathered by Google Maps and other apps, on both Android and Apple devices. By bolstering cell-tower data with information from nearby Wi-Fi routers and Bluetooth beacons, these apps can locate a target to within about 10 meters (better in urban areas, worse in the countryside). They can even work on phones that have been put in airplane mode.

Until the 6 January attacks, geofence search warrants served on Google—for example, by agents investigating a bank robbery—might produce just a dozen suspect devices. The Capitol breach resulted in 5,723, by far the largest such production. It took until early May 2021 for Google to hand over the data to the FBI; when it did so, the results were comprehensive. That data included the latitude and longitude of each device to seven decimal places, and how long it was inside the Capitol. After narrowing the results to only those most likely to have breached the Capitol, Google eventually delivered the names, phone numbers, and emails associated with the accounts—everything investigators needed to identify and track someone inside the Capitol that day.

And track they did. The legal documents indicate that the Google geofence warrants yielded more initial identifications—50 individuals—than did any other technology, and they were cited in a total of 128 cases. Investigators were able to match interior surveillance footage of one suspect, Raul Jarrin, with a photo he was taking on his Samsung cellphone at the exact same moment. They later acquired the photo from Google under a separate warrant. Jarrin was arrested in March 2022.

On top of the Google data, the FBI served geofence search warrants for anonymized location data from 10 data-aggregation companies. But none of these companies were cited in a criminal complaint, and there are no further details.

The EFF sees the tremendous scope and power of geofence warrants as a bug, not a feature. “We believe that geofence warrants are unconstitutional because they don’t start with a suspect,” says Lynch. “They don’t rely on individualized suspicion, which is what’s required under the Fourth Amendment [to the U.S. Constitution]. In the January 6th context, it’s likely that there were many journalists whose data was provided to the police.”

Lynch points out that geofence warrants were also used to investigate possible arsons that occurred during protests over police brutality in Seattle, in 2020. Even though the fires were set at a known location at a known time, the warrants sought location data for all devices on an entire city block over a 75-minute period, during a Black Lives Matter protest. “I think that we would all agree that [the protest] was constitutionally protected First Amendment activity,” she says. “That information should never be in the hands of law enforcement, because it chills people from feeling comfortable speaking out against the government.”

Google told Spectrum that it examines all geofence warrants closely for legal validity and constitutional concerns. It says it routinely pushes back on overbroad demands, and in some cases refuses to produce any information at all.

## Geofences target places, not people—and that’s a problem

Of course, the idea of staking out a particular area for scrutiny is old hat. “Look at every car parked on Elm Street,” says the detective, in just about any procedural, ever. What’s new is the ability to survey any area immediately, easily, and over a wide range of databases—every phone call placed, car parked, person employed, credit-card transaction made, and pizza sold.

And indeed, the high-tech investigations around the Capitol breach went far beyond suspects’ phones to include Uber rides, users’ search history, Apple iCloud, and Amazon. The FBI noted that one suspect, Hatchet Speed, a U.S. Navy reserve officer assigned to the U.S. National Reconnaissance Office, had purchased a black face mask and black “Samurai Tactical Wakizashi Tactical” backpack on Amazon, both of which he was seen wearing in Capitol CCTV footage on 6 January. Speed was arrested in June 2022.

Gabriel Zimmer

Unsurprisingly, after the deadly riot, some of those present deleted their social-media posts, pictures, and accounts. One suspect threw his phone into the Atlantic Ocean. Annie Howell of Swoyersville, Pa., allegedly posted videos of her clashes inside the Capitol with law enforcement. According to her charging document, on 26 January 2021, Howell conducted a factory reset of her Apple iPhone, without backing up data from her online iCloud account. In a Facebook conversation with her father from her computer, he told her, “Stay off the clouds! They are how they are screwing with us.”

The legal documents allege that around 150 others also attempted to delete data and accounts. For many, it was far too late. “The FBI’s really good at finding information that’s deleted, because, as you might know, if you delete a text or an app on a cellphone, it’s not really deleted,” an FBI agent told a January 6 suspect during an interrogation, as reported in one court filing. Investigators were indeed able to recover chats, social-media posts, call records, photos, videos, and location data from many devices and accounts that suspects thought they had permanently consigned to the digital trash can. The FBI even used such efforts to identify suspects: It asked Google to single out those devices in the geofence warrant whose users had attempted to delete their location history in the days following the siege. That process netted an additional 37 people. In March 2022, Howell was sentenced to 60 days in jail.

## Raising a hue and cry—digitally

Perhaps the biggest innovation in the 6 January investigations was nothing that law enforcement itself did, but rather the general public’s response. Using tools and processes pioneered by open-source investigation organizations like Bellingcat, websites such as Jan6attack.com and Sedition Hunters provided a forum for ordinary people in the United States and around the world to analyze and speculate (sometimes correctly, sometimes wrongly) on the identity of rioters. The FBI cited such efforts in 63 legal documents.

Nonprofit investigative newsroom ProPublica became involved when a source provided 30 terabytes of video—over a million video clips—that had been scraped from the social-media network Parler. “One thing that was really helpful was that Parler wasn’t built very well,” says Al Shaw, deputy editor on ProPublica’s News Application Team. “There was all this metadata still attached to the files when they were leaked. We had geo information, what cellphone they were using, time stamps, and a bunch of other data.”

ProPublica filtered the videos by geolocation and other metadata, but soon realized that not all the data was accurate. So journalists went through videos manually to check that those that appeared to have been shot inside the Capitol actually were. ProPublica ended up with 2,500 videos that it could definitively place in the Senate complex on 6 January.

It quickly published 500 of these videos online. Scrolling through the videos is like fast-forwarding through that chaotic day all over again. “One of the design ideas was, can we build a ‘sad TikTok’?” says Shaw. “It’s got a similar interface to TikTok or Instagram, where you’re seeing what’s going on generally in chronological order.” ProPublica’s videos were cited by the DOJ in at least 24 cases.

The remaining 2,000 Parler videos shot from 6 January are now languishing on ProPublica’s servers and could almost certainly help identify more rioters. And the hundreds of thousands of videos discarded in the filtering process could very well contain evidence of further crimes and misdemeanors, as could the thousands of unsearched smartphones and unscraped social-media accounts of other people who went to Washington that day.

But at some point, says EFF’s Lynch, we should ask what we’re really fighting for. “We could, of course, solve more crime if we let police into everybody’s house,” she says. “But that’s not the way our country is set up, and if we want to maintain a democracy, there have to be limits on surveillance technologies. The technology has advanced faster than the law can keep up.”

In practice, that means that some federal courts have found geofence warrants unconstitutional, while others continue to permit their use. Similarly, some jurisdictions are limiting the retention of ALPR data by law-enforcement agencies and the use of facial-recognition technologies by police. Meanwhile, though, private companies are mining ever more open-source images and location information for profit.

In the eternal struggle between security and privacy, the best that digital-rights activists can hope for is to watch the investigators as closely as they are watching us.

This article appears in the February 2023 print issue as “The Panopticon v. the Capitol Rioters.”