Data Divination

Using collected data to predict events shouldn’t blind us to the humans behind it

3 min read
opening illustration for technically speaking
Illustration: Greg Mably
To get better at forecasting big political events, we need both better data and sharper reporting, a clearer read on the numbers, and a more penetrating portrait of on-the-ground realities.

The results of recent votes, particularly the U.S. presidential election and the United Kingdom’s referendum on leaving the European Union (better known as Brexit) left many surprised. In both cases, the postdecision lament went something like this: “In this age of big data, how could the pollsters and pundits have been so wrong in their predictions?”

I’m just a language guy, so I don’t pretend to have an answer, although surely it’s an open question whether polls, with their sample sizes in the few thousands, count as “big” data. (Polling statistics probably fall more under the rubric of medium data.) Perhaps that was the problem: If preelection and prereferendum analyses could have accessed data points in the millions, then might the results have been less surprising? Or perhaps what’s needed isn’t big data on its own but an approach that takes advantage of the many new types of data that are available.

Keep Reading ↓Show less

This article is for IEEE members only. Join IEEE to access our full archive.

Join the world’s largest professional organization devoted to engineering and applied sciences and get access to all of Spectrum’s articles, podcasts, and special reports. Learn more →

If you're already an IEEE member, please sign in to continue reading.

Membership includes:

  • Get unlimited access to IEEE Spectrum content
  • Follow your favorite topics to create a personalized feed of IEEE Spectrum content
  • Save Spectrum articles to read later
  • Network with other technology professionals
  • Establish a professional profile
  • Create a group to share and collaborate on projects
  • Discover IEEE events and activities
  • Join and participate in discussions

Why Functional Programming Should Be the Future of Software Development

It’s hard to learn, but your code will produce fewer nasty surprises

11 min read
A plate of spaghetti made from code
Shira Inbar

You’d expectthe longest and most costly phase in the lifecycle of a software product to be the initial development of the system, when all those great features are first imagined and then created. In fact, the hardest part comes later, during the maintenance phase. That’s when programmers pay the price for the shortcuts they took during development.

So why did they take shortcuts? Maybe they didn’t realize that they were cutting any corners. Only when their code was deployed and exercised by a lot of users did its hidden flaws come to light. And maybe the developers were rushed. Time-to-market pressures would almost guarantee that their software will contain more bugs than it would otherwise.

Keep Reading ↓Show less