Q&A: In the Quest for Personalized Medicine, Beware the Data Deluge, says Theresa MacPhail

Big data is good for identifying gaps in knowledge but not so good for prediction

2 min read
Illustration of Theresa MacPhail.
Illustration: Jacob Thomas

human os icon

illustration of MacPhailIllustration: Jacob Thomas

One person, properly motivated and equipped, can generate terabytes of personal data. While all that fine-grained information could prove invaluable, it also threatens to overwhelm the health care system. Medical anthropologist Theresa MacPhail of Stevens Institute of Technology spent several months during the H1N1 flu pandemic observing researchers with the Centers for Disease Control and Prevention (CDC) as they struggled to make sense of all that big data.

IEEE Spectrum: What did you see at the CDC on a typical day?

Theresa MacPhail: During the first wave of the pandemic, the analysts were getting 1,000 to 1,200 e-mails a day—spreadsheets, lab reports, surveillance alerts, direct answers to their own questions. They had multiple conference calls and at least two or three in-person meetings every day. It was intense.

As you recount in your bookThe Viral Network, your CDC sources told you “the data were crap.” What did they mean?

That the data was divorced from any context—information needed to interpret the data, like what lab did it come from, how were they running their assays. “Good” information came from sources they knew and trusted. There are some older CDC analysts I think of as “context repositories”—they can instantly provide the context for a new piece of information.

Is that something a machine can do?

The CDC, WHO [World Health Organization], and other institutions are trying to design better surveillance algorithms. When you’re identifying a new outbreak, it’s important to look at tweets, blogs, news reports—all that unstructured data that’s so hard to handle. These massive computer programs like Google Flu Trends are designed to scan for certain keywords and spit out reports. But a human still has to go through it.

I think we’re looking at big data wrong. The real usefulness of big data is to show gaps in our thinking. In the Medicare system, for instance, many people weren’t opting for drug coverage, even though it would save them money. So they redesigned the enrollment campaign to get more people signed up. Big data didn’t say why those people didn’t sign up, and it didn’t solve the problem, but it showed us we had a problem.

This article originally appeared in print as “Will Our Data Drown Us?.”

This article is for IEEE members only. Join IEEE to access our full archive.

Join the world’s largest professional organization devoted to engineering and applied sciences and get access to all of Spectrum’s articles, podcasts, and special reports. Learn more →

If you're already an IEEE member, please sign in to continue reading.

Membership includes:

  • Get unlimited access to IEEE Spectrum content
  • Follow your favorite topics to create a personalized feed of IEEE Spectrum content
  • Save Spectrum articles to read later
  • Network with other technology professionals
  • Establish a professional profile
  • Create a group to share and collaborate on projects
  • Discover IEEE events and activities
  • Join and participate in discussions