Understanding Failure by Examining Success
In defense acquisitions, success is an aberration. But to understand why these programs routinely fail, it's instructive to study those that have succeeded
This is part of IEEE Spectrum's Special Report on What's Wrong with Weapons Acquisitions?
In Henry Petroski’s highly acclaimed 1992 book To Engineer Is Human: The Role of Failure in Successful Design , he writes, ”No one wants to learn by mistakes, but we cannot learn enough from success to go beyond the state of the art.”
Implicit in Petroski’s observation is that ”success” is the normative state of engineering and that failure is an aberration, albeit a necessary one. When a successful product or system is launched, the design and implementation used to develop the system are highly studied and then widely copied. Over time, the approach is incrementally improved until, as Petroski notes, someone pushes the approach past its limits and a failure occurs. Future engineering success comes from understanding the reasons for the failure, and major failures become case studies for generations of engineers.
Yet in the high-tech engineering world that underlies defense acquisitions, success is an aberration, while failure is the norm. But to understand why defense programs routinely fail, it’s instructive to study those that have succeeded.
One of the most successful programs in defense systems acquisitions was the US $1 billion Peace Shield II, a nationwide ground-air defense, and command, control, and communications system built for the Royal Saudi Air Force in the early 1990s by Hughes Aircraft (now owned by Raytheon). Because the project was considered a ”foreign military sale” between a U.S. defense contractor and another country’s military, the U.S. Air Force acted as an intermediary.
The success of Peace Shield II was striking because it grew out of an extremely unsuccessful program, namely Peace Shield I, which was canceled in 1991 because it was running three years behind schedule and had projected cost overruns of $1.6 billion. Because of the problems encountered on Peace Shield I, some government assessments estimated that Peace Shield II would likely take 116 months to complete, instead of the planned 54. Remarkably, the Peace Shield II system was delivered in 47 months—seven months ahead of schedule and exceeding its functional requirements.
How did that happen? To start with, the contract for Peace Shield II was a firm fixed price with an incentive clause that said Hughes could earn up to $50 million extra for a three-month-early delivery, or be docked up to $50 million for delivering late. While such contract incentive clauses were not new at the time, Hughes management took the unusual step of announcing that it would share with the 900 engineers working on the project 40 percent of the potential bonus. If the program was finished early, individual engineers stood to earn an average $22 000 in bonus pay.
Jeffrey D. Vermeer was a Hughes manager on Peace Shield II, and he gives several reasons for the program’s success. First, Vermeer says, the program manager was very experienced and was open to hearing about the risks and problems confronting the program and willing to do something about them. ”We had a program manager with enough vision to say, ’What things can we do to abate our risks?’ and then immediately got out the checkbook to fund the abatement plans.”
To help identify problems, the Hughes team held a half-hour ”risk” meeting every morning, Vermeer says. ”Everyone was welcome. What was addressed at the meeting was the small number of items that could place the program at risk: either current problems or potentially troublesome problems. The purpose was to talk about what was being done right, then to address these items, or to better understand why an item was of concern.” The team also generated highly detailed scenarios of the various ways the program could get into trouble. The scenarios allowed the engineers to produce histories that showed trends and tendencies that helped identify other troublesome areas.
Vermeer says a spirit of cooperation and problem solving extended to the U.S. government and military personnel assigned to Peace Shield II and to the Saudi customers. ”Everyone worked together towards making the program successful. If you didn’t look at the badge, you could not tell who was who.”
You might think that when the Peace Shield II program ended, the Defense Department would have carefully scrutinized it to understand the program’s extraordinary success. Sadly, though, the approach used in Peace Shield II was never duplicated in any other DOD program. One reason was that some Defense Department officials, while happy with the early program delivery, felt that Hughes had profited too much on the contract. A few seemed to have forgotten the earlier debacle with Peace Shield I and implied that Hughes had overestimated the difficulty of delivering to the 54-month schedule. This highly critical attitude was not unique to Peace Shield II; other successful defense programs since then have also been regarded with suspicion, as if their managers somehow ”cheated” their way to success.
Nor did Hughes Aircraft apply the Peace Shield approach to its other defense acquisition projects. One reason may have been that Hughes management and business direction changed right around the time Peace Shield II ended, in June 1995. Another and probably more significant reason was that Hughes found the bonus earned by engineers on the Peace Shield program caused resentment among engineers working on other projects.
So instead of duplicating this model of success and perhaps improving on it, as is done in ”normal” engineering environments, the defense acquisition community dismissed Peace Shield II’s success as an aberration, with little relevance to solving the DOD’s ongoing acquisition problems.
By now, it is pretty clear that studying defense program failures does not lead to future program successes. Maybe it’s time to go back and study successful efforts like Peace Shield II to understand why they succeeded. I believe this approach would better reveal the true reasons behind program failures and move the DOD closer to a normal engineering paradigm.
To Probe Further
For more articles and special features, go to What's Wrong with Weapons Acquisitions?