Five Enduring Government IT Failures

Costly consequences go on and on

An emergency Fire engine responds to an emergency with in London, England.
Photo: John Keeble/Getty Images
Advertisement

Large, costly government IT project failures often create an immediate sensation in the media, but they quickly fall out of public notice as soon another failure grabs attention. However, in many cases, the failure’s after-effects linger for years afterward; but these are little reported. Below are five such IT project failures that have happened in the past decade, and a quick synopsis of how the ramifications of each are still being felt today.

UK FiReControl Project

The FiReControl Project was an attempt at integrating 46 stand-alone fire department control rooms into 9 regional centers. The effort was, in the words of the United Kingdom’s National Audit Office, “one of the worst cases of project failure” ever seen. The planned £100 million project, originally initiated in March 2004 and slated to be completed by November of 2007, was canceled in December 2010 after £469 million had been spent [PDF] on the effort.

That isn’t the final cost, however. The failure forced the government to invest an additional £81 million to improve the operational capability at the local fire control rooms. In addition, since only the London regional center was ever occupied, rent (totaling an additional £181 million over 24 years) would have to be paid on the leases for the other eight built-but-never-used centers. The government would be on the hook for that cost unless tenants could be found for them or they were sold.

Currently, six of the eight regional control centers have either been successfully leased or sold off–the latest being the Taunton center. The transactions have yielded an overall estimated savings of about £100 million expected [PDF] through 2032. But the expected sigh of relief is blunted by the fact that, while the government has touted [PDF] the benefits of its £81 million investment in improving the local fire control IT systems, the Fire Brigades Union has complained that the systems still are “constantly failing” and “putting lives at risk.”

It is highly likely that more IT investments into the local fire control systems will be needed between now and 2033, when the FiReControl system was originally slated to be retired. The final cost of the FiReControl project will not likely be fully understood until then.

California Department of Motor Vehicles Modernization:

The planned $208 million, 6-year California Department of Motor Vehicles IT modernization project to replace its antiquated 40-year old legacy system was canceled in February 2013 after 7 years and $134 million was spent. The failed effort was supposed to erase the painful memories of a previous “hopeless failure” project that was terminated in 1994. That boondoggle also lasted seven years, but cost “only” $44 million. Both DMV IT modernization efforts suffered from project mismanagement, overly-optimistic costs and schedules, and a bevy of project risks that were never effectively managed.

The lack of a modern California Department of Motor Vehicles system has caused numerous problems including long customer waiting lines, incorrect voter registrations, and issues in the implementation of the Real ID law. There were also major IT outages in 2016 and 2018, along with an untold number of minor ones. For example, there were 34 IT outages between January 2017 and mid-August 2018 alone.

The ongoing problems with California’s Department of Motor Vehicles turned into a major campaign issue in last fall’s election for governor. All of the candidates promised to address the lingering problem if elected. Newly elected Governor Gavin Newsom announced soon after taking office that he had created a DMV Reinvention Strike Team to correct what he termed the chronic mismanagement of the department. Newsom also pledged that DMV offices would actually begin to accept credit cards for payment by the end of 2019, something that hasn’t ever been allowed.

Before becoming governor, Newsom stated that any California governor who couldn’t fix the Department of Motor Vehicles “should be recalled.” Long suffering California DMV customers may take his words to heart if the agency isn’t fixed soon.

UK National Program for IT

One of the largest IT project failures of any kind was the UK National Health Service’s attempt to create a national electronic health record system. The project was canceled in September 2011 after blowing through at least £9.8 billion and having realized only two percent of the promised benefits. The National Program for IT project began in 2004 with a projected price tag of £6 billion and was supposed to be completed in 2010.

The reasons for the plan’s failure [PDF] involved all the usual suspects, including: significant project, contracting, and technical complexity; a lack of management realism; and ineptitude.

Since then, the UK government has spent hundreds of millions of pounds trying to keep elements of the NPfIT program operational. Meanwhile, it has  undertaken several new efforts at NHS digitalization (not without notable and costly problems). In February 2016, Health Secretary Jeremy Hunt announced that the government would commit £4.2 billion in new funding in order to create a “paperless” NHS by 2020. However, that amount has now climbed to an estimated £12.9 billion aimed at achieving new Health Secretary Matt Hancock’s grand NHS health IT vision by 2024. Given the government’s track record, the new cost estimate and schedule are both likely too optimistic.

In a major side note, a bitter contract dispute between Fujitsu and the National Health Service that dates back to 2008 may finally be officially resolved this year. The NHS fired Fujitsu, one of the four key NPfIT program suppliers, after contentious contract negotiations fell apart. Fujitsu sued the NHS for £700 million for breach of contract, and after eventually prevailing in a protracted legal battle, Fujitsu may finally receive a settlement totaling about £465 million.

The total cost of the NPfIT effort will likely never be known. Nor will the toll taken on the health of UK residents because they haven’t had a national EHR system in place and because the money wasted on NPfIT wasn’t used to improve the National Health Service instead.

Pennsylvania Unemployment Compensation System

Pennsylvania canceled the modernization of its 40-year-old unemployment compensation system in August 2013 after the IT project had fallen 42 months behind schedule and gone $60 million over its original $106.9 million budget. A report [PDF] describing the failure listed ineffective governance and program management, poor adherence to technical processes such as requirements definition and capture, and poor code quality and testing regime among a host of issues that doomed the project.

The failure has created massive amounts of political turmoil over what to do since the plug was pulled. Some $180 million was spent from 2013 to 2017 with the intent of keeping the unemployment system from falling over dead. But those outlays came without an agreed upon path to an alternative. A state audit report [PDF] describes the situation as going from bad to worse when three of eight unemployment centers were closed and nearly 500 government workers furloughed in 2017 due to a lack of agreement between Governor Tom Wolf and state legislators on how to fix the mess. That impasse forced unemployed Pennsylvanians to endure long waits before applying for and receiving unemployment benefits.

A political agreement was finally reached in late 2017. Another $115 million was earmarked for the development of a new unemployment IT system projected to go live by 2022. The final total cost of the project failure probably won’t be known even then. Why? In March 2017, Gov. Wolf announced that the state was suing IBM, the project’s prime contractor, for $170 million. It will likely be years before that case is settled, and the final bottom line figure is computed.

Secure Border Initiative Network

The Secure Border Initiative Network (SBInet) project, organized with the aim of creating a virtual border fence spanning a 53 mile section of the U.S.-Mexico border, was canceled in 2011. Price tag: $1 billion. SBInet’s failure followed the loss of a combined $500 million by its predecessor, the Integrated Surveillance and Intelligence System, and the American Shield Initiative.

SBInet was originally supposed to cover Arizona’s 600-kilometer border with Mexico using what prime contractor Boeing promised was “proven, low risk, off-the-shelf technology.” But the plan turned out to be anything but. Mix in incompetent government program oversight and management [PDF], and the road to project cancellation was paved with plenty of taxpayer dollars.

After SBInet’s plug was pulled, the government promised to pour hundreds of millions more dollars into virtual fencing along the border. It has kept its word in that regard, initiating the 2011 Arizona Technology [PDF] and 2014 Southwest Border Technology [PDF] plans. Each has had varying degrees of effectiveness, as the Government Accountability Office has thoroughly documented [PDF] in its review of both plans. Their true efficacy is debatable, since, as the GAO pointedly noted in its report, the Custom and Border Patrol “is limited in its ability to determine the mission benefits of its surveillance technologies.”

Despite there being no consensus about what technology works well and what doesn’t, a virtual border wall is being floated once again as a means to secure the border instead of the current administration’s politically contentious physical wall. The technology is ready to be deployed and is much improved over that used before, say its proponents—the same claim that was made when SBInet was being oversold.

Déjà vu all over again is the likely result, which means not only more money spent, but more fuel poured onto raging political and technological quarrels over how to best secure the border.

The Computing Technology Newsletter

Biweekly newsletter about advances in hardware, software and systems.

About the Risk Factor blog

IEEE Spectrum’s risk analysis blog, featuring daily news, updates and analysis on computing and IT projects, software and systems failures, successes and innovations, security threats, and more.