Photo: David Stuart
Photo: David Stuart

img Photo: David Stuart

This issue of IEEE Spectrum contains a special report about custom-made enterprise software and its many spectacular failures—the kind that bankrupt companies and cost governments and whole industries tens of billions of dollars a year. We put a man on the moon. So why can’t we make software that works?

Companies and governments undertake these customized IT ventures to make themselves run more efficiently and more effectively. Some of these projects are huge and extremely complex. It’s now common to see multibillion-dollar efforts that take years or even decades to complete. And when that software is good, it can transform entire organizations, as companies like Wal-Mart and Dell Computer have shown.

But when it’s bad, it’s horrid. Nobody really knows how much is wasted. In the United States, a conservative estimate is US $60 billion to $75 billion dollars every year. Just how many ventures fail outright—meaning that they’re either canceled partway through or abandoned shortly after completion—is controversial. For large projects, it’s probably in the 15 to 20 percent range. And then you have lots of other code that’s delivered late or way over budget.

What’s the problem with custom enterprise software? In “Why Software Fails,” risk-management expert Robert N. Charette, an IEEE member, tackles this question. Basically, far too much of it doesn’t work very well, for reasons that are well understood and preventable: bad or nonexistent process documentation, impossible-to-meet requirements, poor and ever-changing specifications, quality-control issues, and perhaps the biggest problem of all—people. The clients who can’t figure out what they really want or need, the vendors who can’t or won’t rein them in, the managers who see scope creep tearing through a project but look the other way.

One of the most well-publicized software failures is the subject of “Who Killed the Virtual Case File?” Senior Associate Editor Harry Goldstein investigates the trail of missteps that brought down the FBI’s Virtual Case File system. This custom software was supposed to automate the bureau’s paper-based work environment, allowing agents to share investigative information via a computer network. Instead the FBI claims that the Virtual Case File’s contractor, Science Applications International Corp., delivered code so bug-ridden that the bureau scrapped the $170 million project earlier this year. But various government and independent reports show that the FBI shares a good part of the blame for the failure.

As the FBI gears up to spend hundreds of millions more on software during the next several years, questions remain as to how the Virtual Case File project went so wrong and whether an even bigger failure lies ahead. Despite a good deal of attention in the press, the inner workings of the initiative have remained largely invisible—until now. Goldstein’s interviews with many of the people directly involved reveal an effort that succumbed to the most basic mistakes of software development.

Is anything going right? In “The Exterminators," Contributing Editor Philip E. Ross describes how UK-based Praxis High Integrity Systems is applying formal methods to software engineering and emerging with largely bug-free code. These mathematical methods work well for relatively small programs (less than 200 000 lines of code). Because of the added expense, they’re best suited for mission-critical systems, like air-traffic control programs, that must be bulletproof and totally reliable. Issues of scalability remain, however, and that is one of many reasons more people aren’t using such methods: 200 000 lines of code is one thing; the 200 million lines in a supply-chain management system is quite another.

Future software failures are everywhere in the making. The FBI’s replacement for the Virtual Case File system is on deck. The push to automate and digitize medical records looks like another breeding ground for fatal bugs. After you read our report, you will know why so many of these projects end in disaster—and how such costly disasters can often be avoided.

Special Thanks

Many people helped us put this report together, but the IEEE members listed here were particularly generous with their time and their knowledge. Any fault found with these pages rests with the editors. We thank these members for their support and guidance.

Susan K. Land, software engineering section manager, Northrop Grumman IT/TASC, IEEE Senior Member

James C. McGroddy, former director of research, IBM, IEEE Life Fellow

James W. Moore, senior principal engineer, The MITRE Corp., IEEE Senior Member

Peter G. Neumann, principal scientist, SRI International Computer Science Laboratory, IEEE Life Fellow

George Spix, chief architect, Microsoft Corp., IEEE Member

Elaine J. Weyuker, technology leader, AT&T Labs—Research, IEEE Fellow

The editorial content of IEEE Spectrum magazine does not represent official positions of the IEEE or its organizational units.

The Conversation (0)

Q&A With Co-Creator of the 6502 Processor

Bill Mensch on the microprocessor that powered the Atari 2600 and Commodore 64

5 min read
Bill Mensch

Few people have seen their handiwork influence the world more than Bill Mensch. He helped create the legendary 8-bit 6502 microprocessor, launched in 1975, which was the heart of groundbreaking systems including the Atari 2600, Apple II, and Commodore 64. Mensch also created the VIA 65C22 input/output chip—noted for its rich features and which was crucial to the 6502's overall popularity—and the second-generation 65C816, a 16-bit processor that powered machines such as the Apple IIGS, and the Super Nintendo console.

Many of the 65x series of chips are still in production. The processors and their variants are used as microcontrollers in commercial products, and they remain popular among hobbyists who build home-brewed computers. The surge of interest in retrocomputing has led to folks once again swapping tips on how to write polished games using the 6502 assembly code, with new titles being released for the Atari, BBC Micro, and other machines.

Keep Reading ↓ Show less

Spot’s 3.0 Update Adds Increased Autonomy, New Door Tricks

Boston Dynamics' Spot can now handle push-bar doors and dynamically replan in complex environments

5 min read
Boston Dynamics

While Boston Dynamics' Atlas humanoid spends its time learning how to dance and do parkour, the company's Spot quadruped is quietly getting much better at doing useful, valuable tasks in commercial environments. Solving tasks like dynamic path planning and door manipulation in a way that's robust enough that someone can buy your robot and not regret it is, I would argue, just as difficult (if not more difficult) as getting a robot to do a backflip.

With a short blog post today, Boston Dynamics is announcing Spot Release 3.0, representing more than a year of software improvements over Release 2.0 that we covered back in May of 2020. The highlights of Release 3.0 include autonomous dynamic replanning, cloud integration, some clever camera tricks, and a new ability to handle push-bar doors, and earlier today, we spoke with Spot Chief Engineer at Boston Dynamics Zachary Jackowski to learn more about what Spot's been up to.

Keep Reading ↓ Show less

How to Write Exceptionally Clear Requirements: 21 Tips

Avoid bad requirements with these 21 tips

1 min read

Systems Engineers face a major dilemma: More than 50% of project defects are caused by poorly written requirements. It's important to identify problematic language early on, before it develops into late-stage rework, cost-overruns, and recalls. Learn how to identify risks, errors and ambiguities in requirements before they cripple your project.

Trending Stories

The most-read stories on IEEE Spectrum right now