The US Department of JusticeInspector General Glenn Fine has just released his seventh in a series of reports (see PDF) concerning the FBI's case management system Sentinel. Sentinel is the replacement for the notoriously failed $170 million Virtual Case File (VCF) system that IEEE Senior Editor Harry Goldstein so exquisitely dissected in detail in the issue of the September 2005 issue of Spectrum.
As you may recall, in September, the FBI announced that it had basically fired prime contractor Lockheed Martin, and was going to take over the management of the project itself.
With that a short intro, let's go to the findings in the latest IG report:
"Our review found that as of August 2010, after spending about $405 million of the $451 million budgeted for the Sentinel project, the FBI has delivered only two of Sentinel’s four phases to its agents and analysts. Moreover, we believe that the most challenging development work for Sentinel still remains."
"In addition, we found that while Sentinel has delivered some improvements to the FBI’s case management system, it has not delivered much of what it originally intended. In July 2010, the FBI deployed Phase 2 of Sentinel, which provides the FBI’s agents and analysts with the beginnings of an electronic case management system. Yet, by July 2010 Sentinel was intended to generate and securely process 18 paperless case-related forms through the review and approval process. Sentinel now only has the capability to generate and process 4 of the 18 forms. Moreover, even these four forms still are not fully automated. Because Sentinel’s four phases have not been completed, FBI agents and analysts can use Sentinel to generate the four forms, but they must still print the forms to obtain approval signatures, and they must maintain hard copy files with the required approval signatures."
"Additionally, because the FBI has not finished the third and fourth Phases of Sentinel, FBI agents and analysts do not have the planned expanded capabilities to search the FBI’s case files. Nor can they use Sentinel to manage evidence, as originally intended. Sentinel also has not replaced ACS [Automated Case Support ], and Sentinel has not yet become the FBI’s official records repository."
"In addition, because of Sentinel’s delays and cost increases, in July 2010 the FBI issued another stop-work order that directed Lockheed Martin to stop all work on the remaining phases of Sentinel - Phases 3 and 4. As of August 1, 2010, the FBI had not decided on an approach for completing Sentinel, and FBI officials did not provide the OIG [Office of the Inspector General] with detailed descriptions of the alternatives under consideration for completing Sentinel. At that time, however, the FBI Chief Technology Officer stated that the alternatives under consideration would allow the FBI to complete Sentinel within its $451 million budget by re-using portions of successful FBI IT projects, including Sentinel, taking advantage of technological advances and industry best practices, and increasing the reliance on FBI personnel to develop Sentinel. Yet, the Chief Technology Officer acknowledged that his estimate did not include the cost of maintaining Sentinel for 2 years after its completion - costs which had been included in all previous Sentinel budgets. In addition, an independent assessment conducted in July 2010 at the FBI’s request by Mitre estimated that completing Sentinel under the FBI’s current development approach would, at a minimum, cost an additional $351 million and take an additional 6 years."
Hmm, Sentinel, shall we say, looks like it is in deep trouble.
Not true, the FBI has said in heated response to the IG's report.
According to Thomas Harrington, FBI associate deputy director, the IG "continues to rely on outdated cost estimates that do not apply to the current FBI plan."
Furthermore, Deputy Director Harrington says that:
"We believe that the interim report does not accurately reflect the FBI's management of the Sentinel project, and fails to credit the FBI with taking corrective action to keep it on budget."
In a parting shot aimed at IG Fine, Deputy Director Harrington says:
"The FBI has and will continue to work with the OIG [Office of the Inspector General] to review our use of agile development to complete the Sentinel project. However, we are concerned that this interim report does not comply with generally accepted government accounting standards, which the report acknowledges. In the future, the FBI requests that the OIG use these auditing standards in its review of the Sentinel project and not rely on 'interim' reports that do not accurately reflect the status of the project."
The FBI apparently believes that it can finish the remaining two phases of Sentinel for about $45 million using agile software development approaches in about a year - or, about 5 to 6 times faster as well as cheaper than what Lockheed Martin had been doing.
Who says silver bullets don't work?
In my mind, all one really needs to know to predict the likely future success of Sentinel is to look at the FBI's past statements on both VCF and Sentinel. During VCF, for instance, FBI's management kept saying everything was okay, even as IG reports and outside expert reviewers were telling management that the program was in deep trouble. If you believed what the FBI was saying, there was nothing wrong with VCF until the day it was cancelled.
Then when Sentinel was initially being contracted for, the FBI said it had learned all the lessons from VCF and this time, it assured everyone that it would be different. Well, it has been different, and a lot less successful and more expensive than promised.
And even though the current FBI way forward on Sentinel is radically different than the original, the FBI would have everyone believe the change was caused by some "minor" technical issues.
In my experience, a telling sign of when an IT project is in trouble whenever management insists that the audit information is out of date. The usually signals management is in deep reality denial or in CYA mode.
The more the FBI says everything is okay on Sentinel, the more you should double your bets against it.
Robert N. Charette is a Contributing Editor to IEEE Spectrum and an acknowledged international authority on information technology and systems risk management. A self-described “risk ecologist,” he is interested in the intersections of business, political, technological, and societal risks. Charette is an award-winning author of multiple books and numerous articles on the subjects of risk management, project and program management, innovation, and entrepreneurship. A Life Senior Member of the IEEE, Charette was a recipient of the IEEE Computer Society’s Golden Core Award in 2008.