The decade long saga of the US Federal Bureau of Investigation's (FBI) effort to deploy a comprehensive information and investigative case management system began with the doomed Virtual Case File system and continues with the already over cost and late Sentinel system development.
Just before Christmas, the US Department of Justice Inspector General's Office released its eighth (PDF) in a series of reviews into Sentinel's development. According to the FBI's original plan, Sentinel was supposed to be fully deployed in December of 2009 at a cost of $425 million. Those numbers slipped and rose, respectively, to June 2010 and $451 million. The deployment slipped again to September 2010 and then again to September 2011 and yet again, with no public fanfare, to late January 2012. We now know from the IG report that in December 2010 the FBI once again slipped the deployment schedule, this time to May 2012.
The good news is that the overall Sentinel project cost has apparently not increased; the bad news is that the IG fully expects the cost of the project to indeed go up—and by how much, no one seems to know. In addition, the IG is skeptical that the current May 2012 deployment date will be met.
Why the skepticism ?
Well, for one thing, the IG points out that in week 44 of Sentinel's (original) 48 week agile development plan, which the FBI put forth with great confidence in October 2010, the project had completed only ".. 50 percent of the functionality originally planned... " This shortfall in delivered functionality necessitated adding 12 weeks to Sentinel's development schedule, thereby moving it from the end of September to near the end of December 2011.
Then, additional problems were encountered during Sentinel system testing which required yet another 8 weeks to be added to the Sentinel development schedule, pushing it out to a total of 68 weeks. The current FBI plan is for Sentinel's development to be completed by the middle of next month, which would allow about three months to fully test and deploy the Sentinel system to its 30 000 users across the FBI beginning in May.
To meet the new February development completion date, however, the IG noted that the agile development team would have to increase its previous average productivity by 150% or more over the final 10 weeks of the project, no small feat.
That assumes, the IG also noted, that nothing unexpected turns up, and that the functionality being promised is actually being delivered. The IG expressed concern in the report that there was no "... consistent application of completion criteria, including verification that [developed] functionality is field ready..." thus creating a situation whereby "... the FBI is unable to accurately assess either the amount of remaining work or the reliability of the functionality that it has designated as complete."
As for the IG's skepticism about the FBI staying within the project budget—which the Bureau's senior management promised Congress—the original contract called for two years of Sentinel operations and maintenance costs to be included in the $451 million project budget, but the IG says that it now looks like at least some of these O & M costs will need to be covered another way.
Finally, the IG is concerned about user acceptance of the Sentinel system. Early systems tests didn't particularly impress rank and file FBI agents, the IG stated.
The IG concluded its report by saying:
"It is too early to judge whether the FBI’s Agile development of Sentinel will meet its newly revised budget and completion goals and the needs of FBI agents and analysts. While the Sentinel Advisory Group responded positively to the version of Sentinel it tested, results from wider testing were not as positive. Also, none of the Agile-developed Sentinel has been deployed to all users to give them the ability to enter actual case data and assist FBI agents and analysts in more efficiently performing their jobs."
The FBI response to the IG report was basically that the concerns raised were merely nits in the wider scheme of things, and that the most recently revised development schedule represented only a "modest extension" of the deployment schedule. And anyway, the extension didn't really have to do with software development related issues, but was instead "... due primarily to the need to implement a standard five-year 'refresh' of computer hardware, so the Sentinel software will provide the required functionality as intended."
If I remember it correctly, that is No. 7 from the universal big book of software development excuses: blame issues with hardware as the reason for the software being late.
Well, we won't have to wait long to see if the latest development schedule is going to be met. On the 10th of February, according to the IG report, Sentinel's functional software is scheduled to be signed off as being officially completed.
Then we'll see whether the wonderful kudos the Sentinel management team gave itself last year are actually deserved.
Robert N. Charette is a Contributing Editor to IEEE Spectrum and an acknowledged international authority on information technology and systems risk management. A self-described “risk ecologist,” he is interested in the intersections of business, political, technological, and societal risks. Charette is an award-winning author of multiple books and numerous articles on the subjects of risk management, project and program management, innovation, and entrepreneurship. A Life Senior Member of the IEEE, Charette was a recipient of the IEEE Computer Society’s Golden Core Award in 2008.