IT Hiccups of the Week: Excel Spreadsheet Error Heard Around the World

After the previous week’s quietude, IT-related problems, issues and faults returned to their normal rate of occurrence. We start off with a human-induced spreadsheet error that is reverberating around the economic and government financial policy worlds.

Oops: Excel Error Calls Into Question Widely-Cited Economic Study on the Impact of Government Spending

Economics has long been called the “dismal science.” Last week, an error in an Excel spreadsheet used by two Harvard University professors served to help reinforce that moniker.

Back in 2009, Carmen M. Reinhart and Kenneth S. Rogoff published a book with the provocative title, “This Time It’s Different.” The professors asserted in their book that, among other things, their empirical research demonstrated that when advanced economies’ public liabilities reach or exceed “the important marker of 90 percent of GDP,” long-term economic growth and stability are placed at peril. Upon reaching that 90 percent of GDP point, the two argue, governments need to act swiftly to rein in public spending or increasingly risk stifling future economic recovery and growth.

Reinhart and Rogoff based their conclusions, which many governments (especially in Europe) embraced as a sound rationale for their current policies of cutting public spending, on what the two said was “clear” and “sharp” empirical analysis of “comprehensive” financial data. The information they collected “cover[ed] sixty-six countries across five continents” going back to the early 1800s. The two professors have not been shy about promoting the importance of their research; a quick scan of the book’s website turns up this statement: “An important book that will affect policy discussions for a long time to come, This Time Is Different exposes centuries of financial missteps.”

The book's analysis drew the expected reactions from the two main opposing factions of economists: those who support Reinhart and Rogoff and their call to reduce high public debt levels and governmental spending, and those who argue that in tough economic times, it is imperative that governments increase spending to stimulate their economies. The latter group, while arguing that Reinhart and Rogoff’s conclusions were wrong, have been off-footed by the duo’s seemingly strong, data-driven analysis of 200 years of financial data. That is, until last week.

Last Monday, Thomas Herndon, Michael Ash, and Robert Pollin, three economists at the University of Massachusetts, published a review of the conclusions reached by Reinhart and Rogoff using the original data set upon which "This Time It's Different" is based.  What did they find? The Massachusetts economists say they spotted “coding errors, selective exclusion of available data, and unconventional weighting of summary statistics.” These, they said, “lead to serious errors that inaccurately represent the relationship between public debt and GDP growth among 20 advanced economies in the post-war period.” The takeaway: “when properly calculated, the average real GDP growth rate for countries carrying a public-debt-to-GDP ratio of over 90 percent is actually 2.2 percent, not -0.1 percent, as published in Reinhart and Rogoff...average GDP growth at public debt/GDP ratios over 90 percent is not dramatically different than when debt/GDP ratios are lower.”

Reinhart and Rogoff, as they admitted after the UMass paper was published, accidentally forgot to include the first five rows covering data from five countries (Australia, Austria, Belgium, Canada, and Denmark) from an Excel spreadsheet in their analysis—a “coding error” which they said was “a significant lapse on our part.” Others who were being nice called the oversight a "numbskull error." However, the professors took exception to the charge of deliberately manipulating the data to match their beliefs about the need for government austerity, and insisted that their conclusions were not much affected by the error anyway.

As expected, discovery of the error has caused quite a stir over the veracity of Reinhart and Rogoff's findings and the resulting implications for public economic and financial policies across the world. In other words, does high public debt indeed cause slow economic growth (as Reinhart and Rogoff argue) or does slow economic growth cause high public debt? I'll let Risk Factor readers argue the merits of either case if they are so inclined.

The Excel kerfuffle probably doesn’t matter much in the long run, as economists are fond of saying, given that most governmental economic data (especially old data) is so noisy as to be pretty useless for detailed analysis. Just as important is the fact that no one seems to understand how to spur economic recovery happen anyway, regardless of what side of the economic policy debate they sit. Further, as this commentary at Forbes insightfully notes,  “In reality, the only lesson to be drawn from this episode is that academic economics, like many social sciences, is grounded in hubris and pseudoprecision. And that the modern urge to demand an academic study to ‘prove’ or justify inherently complex and ambiguous decisions is antithetical to clear thinking.”

The other lesson the Reinhart and Rogoff Excel error shows is that “this time it isn’t different,” at least in regard to human-related data error. Two compilations of previous spreadsheet and other data-driven errors aptly demonstrate the point.

H&R Block Sued Over Tax Filling Issues That Delayed Refunds

As we noted a few weeks ago, H&R Block, one of the world's largest tax services providers, had a problem electronically transmitting Form 8863 (American Opportunity and Lifetime Learning Credits) to the IRS. The issue delayed refunds for hundreds of thousands of taxpayers. At the time, H&R Block’s CEO said that he “sincerely” apologized for the filing snafu. However, that apology hasn't counted for much in some quarters, apparently, since according to a story at Consumer Affairs, lawsuits in California, Michigan, and Illinois have been filed against H&R Block. The lawsuits are generally claiming that the company did not live up to its advertised 100 percent accuracy claim, and or failed to provide adequate compensation to those affected by the refund delay.

And speaking of tax filing problems, on the evening before U.S. tax returns were due on 15 April, Turbo Tax Online was “intermittently unavailable” for about an hour, which scared more than a few last minute filers, according to Forbes.  No lawsuits are expected to be filed over the temporary increase in taxpayer heart rates brought on by the issue, but you never know.

In what is hopefully the last of the tax season-related computer problems, a faulty switch and router made it impossible for Utah taxpayers to access the state's tax filing website on 14 April. Utah addressed the issue by giving residents an extra day to file their state tax forms.

Server Malfunction Affects 15 000 Minnesota Students Taking Math Test

Freaking out over a problem with your tax software probably pales in comparison to the emotions experienced by the 15 000 Minnesota students who were trying to take their Minnesota Comprehensive Assessment online math test last Tuesday, but either couldn’t sign in or had their tests ended prematurely.  According to the Star Tribune, American Institutes for Research, the testing vendor, attributes the snafu to a problem on one of its servers. A story at the Pioneer Press quoted Charlene Briner, chief of staff for the Minnesota Department of Education, as saying the problem was “unacceptable.” In response, the vendor stated that while the company’s online testing system isn’t perfect, “it's pretty damn good.” Maybe the vendor should solicit the opinions of the students who had their test ended mid-way through.

Briner said that the affected students will be able to pick up where they left off and review their answers or restart their tests. No doubt the students, who just love taking math tests, were overwhelmed with feelings of joy on hearing that.

New Hampshire’s New Medicaid System Has a Few Problems After All

After telling everyone to expect problems with the roll-out of the state’s new Medicaid Management Information System (MMIS) at the beginning of the month, and then saying that there weren’t any after all, New Hampshire’s Health and Human Services Commissioner Nick Toumpas reversed course again last week. Toumpas admitted to the Union Leader that there were indeed some issues cropping up. He noted that some small Medicaid and Medicare providers have had problems since the system rollout commenced—mainly, the suspension of their submitted invoices for payment. Of the bounced back invoices, he said that, “It was unclear if it was their problem, whether they submitted something incorrectly, or whether it was getting kicked out legitimately.” Interestingly, he did include in his statement the alternative that there may actually be something wrong with the MMIS itself.

To his credit, Toumpas is still allowing payments to be made to the small providers with provisos for getting the money back if it is indeed a problem on the providers’ end, and not with the new MMIS.

American Airlines Still Not Explaining Cause of “Software Issue;” ERCB Outage Continues

American Airlines has still not given a reason for the “software issue” that affected its reservation system and disrupted flights last week, according to the LA Times. It is doubtful that it ever officially will.

And the routine hardware upgrade to the computer system servers at Alberta’s Energy Resources Conservation Board (ERCB), which went bad on 2 April (mentioned here last week), continues. The ERCB reports some progress in fixing the problem, but remains uncertain as to when its servers will be finally fixed.

Also of interest…

Computer Issue Excludes Eligible Jurors in Polk County, Florida

Google Gmail Outage Blamed on Log-in System

Singapore Check Clearing System Hit by Technical “Glitch”

Photo: John Turner/Getty Images

Advertisement

Risk Factor

IEEE Spectrum's risk analysis blog, featuring daily news, updates and analysis on computing and IT projects, software and systems failures, successes and innovations, security threats, and more.

Contributor
Willie D. Jones
 
Advertisement