Risk Factor iconRisk Factor

IT Hiccups of the Week: Rough Start for NYC's New 911 System

Yet another quiet week in the land of IT snafus. The most interesting story to crop up involved the problems plaguing the roll out of the New York City Police Department’s new emergency 911 dispatch system.  

New York City’s New Emergency Dispatch System Fails 4 Times in First 48 hours

The Big Apple’s long-troubled effort to modernize its 911 emergency call system ran into additional difficulties when its new, US $88 million NYPD emergency dispatch system suffered four outages within two days of its Wednesday debut.

The dispatch system first went out Wednesday afternoon for 16 minutes beginning at 4:21 pm, the New York Daily News reported. Emergency service operators had to revert to capturing the call information on slips of paper, which were then taken by runners to the separate NYPD and EMS radio rooms where the proper emergency units could be assigned to the call. The scene was described by one experienced operator as sheer “pandemonium.”

A City Hall spokesman down played the incident, saying that the manual back-up system worked and that no calls were missed.

Then, early Thursday morning, the dispatch system suffered a six minute outage, said Police Commissioner Ray Kelly in an interview with CBS News New York. Kelly also felt the need to note that the new system had been “tested for six months.”

I am not sure whether Kelly was trying to vouch for the dispatch system’s reliability, but if he was, his efforts were soon undone. Shortly after Kelley's morning interview, the dispatch system went out again, at 12:09 pm. That time, the outage went on intermittently for about an hour, the Daily News once more reported. Out came the paper, pens, and runners again.

Then at 7:00 Thursday evening, the dispatch system went out one last time—but only for two minutes.

Mayor Michael Bloomberg shrugged off all the outages, saying that, “There are a few bugs in the system. We'll fix them and there'll be more. Every computer system has bugs in it; there's none that does not.”

Those remarks probably didn’t go over well with the emergency system operators who were told that the “new system would never go down.

At least there have been no reported outages the past three days. But if another one happens, at least the emergency dispatch system operators have now had plenty of practice in how to deal with it.

Read More

This Week in Cybercrime: Report Details Stolen U.S. Defense Secrets

What U.S. Defense System Details Have Hackers Accessed?

In the past year, the U.S. government has gone from making thinly-veiled accusations about nation-state sponsored cyberattacks to pointing fingers directly at China as the entity behind a string of hacks in which intellectual property and other sensitive information has been stolen from private firms and government agencies. That was the tone of a report released earlier this month by the U.S. Department of Defense. The 92-page report says the stolen information is helping China build “a picture of U.S. defense networks, logistics and related military capabilities that could be exploited during a crisis.” And this week we learned, courtesy of the Washington Post, some of the elements in that picture. The Post says it obtained a copy of a previously undisclosed section of a report written by the Defense Science Board (DSB), a committee of experts that advises the U.S. Department of Defense on technical and scientific matters.

That report, which was released in January, provided the Pentagon and defense contractors details regarding the data to which cyberthieves gained access. It said that the “DoD and its contractor base have already sustained staggering losses of system design information incorporating decades of combat knowledge and experience that provide adversaries insight to technical designs and system use.” But the public version of the report did not list the weapons whose plans had been stolen. According to an article in the Washington Post, the pilfered information included plans and technical details on several missile defense systems such as the PAC-3 Patriot missile system, the Terminal High Altitude Area Defense (THAAD) system and the U.S. Navy's Aegis ballistic-missile defense system. The cyberthieves—who U.S. government officials say were working at China’s behest—also saw design plans for the F/A-18 fighter jet, the F-35 multirole combat aircraft, the V-22 Osprey aircraft, the Black Hawk helicopter and the Navy's Littoral Combat Ship (LCS) class of vessels.

Read More

BBC Blows £98 Million on Digital Media Initiative

The announcement last week that the BBC pulled the plug on its overly ambitious and admittedly poorly managed Digital Media Initiative (DMI), probably drew a smile from the folks who originally worked for Siemens Information Solutions and Services (SIS) group, now owned by ATOS. The BBC admitted that the project, which was said by BBC Trust's Anthony Fry to have created “little or no assets,” cost license fee holders £98.4 million (about US $150 million at current exchange rates).

To understand the former Siemen’s workers presumptive glee, you need to go back a few years.

In February 2008, the BBC directly awarded a £79 million, fix-priced contract to Siemens under an existing outsourcing contract to implement, roll out, and operate through March 2015 what the BBC called its Digital Media Initiative. According to a 2011 National Audit Office (NAO) review, “DMI is a technology transformation project designed to allow BBC staff to develop, create, share, and manage [all] video and audio content and programming on their desktop, and intended to improve production efficiency across the BBC,” anywhere the BBC operated. In other words, DMI was meant to radically change the way the BBC operated internally.

The BBC's business case put the total DMI project investment, including its management and other administrative costs, at £81.7 million, and claimed that the project would end up generating a total benefit of £99.6 million. The benefit, the NAO indicated, would come from reduced operating costs, the avoidance of some future production costs, and a “creative dividend” savings that would accrue from being able to reuse material instead of having to produce entirely new content.

Read More

IT Hiccups of the Week: RBS Antagonizes Two Million More Customers

Aside from the billion or so 17-year brood cicadas which all seem to be singing directly outside my office window, it was very quiet last week in regard to IT-related snafus, problems and outages. We start off this week with the Royal Bank of Scotland, which can always be counted on to liven up a slow week.

This Time, RBS Mobile Banking App Fails

Two million customers of the UK government nationalized Royal Bank of Scotland and its subsidiaries, NatWest and the Bank of Ulster, were frustrated yet again by an IT problem. According to the BBC, customers were unable to log into their accounts through their mobile phone app for about two and half hours last Friday just before the long bank holiday started. RBS customers reported that the problems began around 7:15 a.m. London time. When they tried to access their bank accounts through the app, they received various error messages telling them that the app couldn't find an Internet connection even though other apps were working fine. The problem was fully cleared up by 1:00 p.m., the BBC stated.

This was the second IT failure for RBS in two months, and follows the disastrous bank IT system meltdown of last year which the banking group is still trying hard to put behind it. Just two weeks ago, RBS announced that it was going to spend an extra £450 million to fix its problem-plagued IT systems. The Financial Times quoted bank chairman Sir Phillip Hampton as saying that, “As the IT incidents over the past year have shown, building and maintaining a top-class infrastructure is fundamental. Our customers deserve banking services that work 100 per cent of the time.”

Friday's failure was embarrassing not only because of Sir Phillip’s comments, but because about a week ago RBS announced that it was eliminating another 1400 jobs in its retail banking sector as part of its move to encourage its 17 million customers to move to online and mobile banking. The latest gaffe may instead encourage RBS customers to decide to move to a rival's online and mobile banking app.

RBS offered up an apology to its customers, who probably aren't listening anymore.

Read More

IT Hiccups of the Week: Lie Detector Lies?

There were a couple of interesting IT-related snafus, errors, and problems last week. We start off this week’s edition of IT Hiccups with a popular polygraph system that may well have incorrectly identified thousands of people as being economical with the truth when they actually weren’t.

Lafayette LX4000 Polygraph System Accused of Minimizing “Technical Glitch” for Years

The McClatchy publishing company ran a series of disturbing stories in its papers over the weekend about a polygraph system called the Lafayette Instrument LX4000, which is widely used by U.S. state, local and federal law enforcement agencies, as well as the military and intelligence agencies. The articles note that the polygraph has had a long-standing “technical glitch” that may have incorrectly shown people as being untruthful when they were not.

Read More

This Week in Cybercrime: Are Strong Passwords Only for Your Important Accounts?

Strong Passwords: Only For Your “Important” Accounts?

How strong are your computer passwords? What influences whether you “secure” an account with a password such as “123456” or never even bother to change it from a default such as “Welcome1” after you’ve registered at a website? A team of researchers from University of California at Berkeley, the University of British Columbia, and Microsoft wanted to know whether the password strength meters more frequently seen on registration pages make a difference in what alphanumeric combinations registrants decide to use. In a paper (pdf) released this week, the researchers report the results of experiments designed to reveal the circumstances under which strong or weak passwords are used. The team wrote that, “meters result in stronger passwords when users are forced to change existing passwords on important accounts and that individual meter design decisions likely have a marginal impact.” But the flip side of that coin, unfortunately, is that when it comes to sites that users view as unimportant (when there is no sensitive information, like their bank balances, to keep hidden), they tend not to make the effort. In those instances, say the researchers, users all too frequently reused passwords from other accounts. What they fail to take heed to, say the researchers, is that regardless of a password’s relative strength, if it is used across several sites, all of a user’s accounts are at risk if a hacker breaks into one site’s poorly guarded password database. The problems with passwords are mostly attributable to “poor policies and…the frequencies we see of databases getting disclosed,” Serge Egelman, a UC Berkeley researcher who was a member of the research team, told Kaspersky Lab’s Threatpost. “If more work was done to secure stored encrypted passwords, less effort would need to be done on the users’ end.”

Read More

IT Hiccups of the Week: Programming Error Rejects Unsuspecting Oregon Trimet Riders' Credit and Debit Cards for 5 Years

This past week saw a hodgepodge of ICT-related issues. We start off with a long-standing software error affecting the credit and debit cards of some unlucky postcode related TriMet transit passengers in Portland, Oregon.

TriMet Ticketing Machine Software Error Flags Credit and Debit Cards as Fraudulent

For years, officials at Portland, Oregon’s, metro TriMet bus, light rail and commuter rail transit system have been trying to deter thieves using stolen credit and debit cards from purchasing TriMet transit tickets as a way to quickly cash in on their theft before a card is reported stolen. According to a 2011 story at the Oregonian, the thieves' modus operandi is using a stolen card to purchase an $88 TriMet pass at a ticket machine, then selling them for huge discounts in a thriving local black market. The fraud costs the transit system tens of thousands of dollars, the article says, because TriMet has made transactions using plastic so easy that “credit processor Visa requires it to cover the cost of every ticket purchased with a stolen credit card.” In 2012, Visa charged back US $95 389 for fraudulent transactions.

Many legitimate purchasers of TriMet tickets have been feeling the effects of the fraudulent activity as well. For the past several years, a large number of TriMet transit riders have been complaining that when they used their credit or debit cards to purchase a ticket, the purchases were not only declined, but their banks put security freezes on their cards out of fear that they had been stolen. Sometimes the banks would even cancel the cards outright, another story in the Oregonian reported last week.

When riders complained to TriMet about the issue, transit officials told the riders that they needed to talk to their banks about it, not them. The Oregonian stated that, “TriMet assumed problems with riders having cards suspended and cancelled were the result of banks using proprietary fraud filters to stop thieves.”

A classic case of what Oscar Wilde said about assumptions: “When you assume, you make an ass out of u and me.”

What was really behind the false positives? A software error in TriMet’s 215 ticketing machines was flagging the credit and debit cards of riders with a certain zip code as being stolen. This was happening 1000 to 2000 times a month over the past five years, Portland television station KATU reported. The error was finally discovered this January. “A data field was passing something other than TriMet's zip code, causing banks to flag the transactions as risky,” the Oregonian reported.

A TriMet official was quoted in the paper as saying, “After addressing [the error], fraud declines for credit cards users at our TVMs decreased significantly from 4 percent to 0.3 percent.”

TriMet issued a roundabout apology for the error, which was buried in a press release detailing the steps the transit agency is taking to reduce another issue angering its ridership, namely the notorious unreliability of its ticketing machines. TriMet suggests in its release that until machine reliability is improved (hopefully this summer), riders should not depend on the machines to purchase a single ticket at the station, but to instead carry a book of pre-bought tickets just in case.

Read More

This Week in Cybercrime: State Court Hack Punishes the Guilty and the Innocent

Up to a Million Washington Residents Affected by Hack of State Court Network

It’s likely that most of the people charged with crimes in Washington State between September 2011 and December 2012 have already been exonerated or have paid their respective debts to society. But for roughly a million of them (at least some of whom were found not guilty at trial, established their innocence before their cases went that far—or were in court simply to fight a traffic ticket) that moment of contact with the state’s court system may lead to another punishment: identity theft. The state government revealed this week that the website for the Washington State Administrative Office of the Courts was hacked and that the attacker may have gotten away with the names and social security numbers of anyone booked into a city or county jail in the state during that time. Officials also couldn’t rule out the possibility that some people charged in the state's superior court criminal system in 2011 or 2012, cited for driving under the influence between 1989 and 2011, or went to court for traffic-related offenses during that period might be at risk. The larger group's names and driver's license information may have been taken.

"The access occurred through a ‘back door' part of a commercial software product [Adobe Systems’ ColdFusion] we were using, and it is patched now," Mike Keeling, information technology operations and maintenance manager for the court system, told reporters on a conference call.

At the same time that state officials were offering up the usual assurances that no financial data such as credit card numbers was accessed as a result of the break-in, they revealed that the breach was discovered in February (and could have been exploited as early as last fall). Since then, the state has attempted to notify only the 94 people (that is not a typo) whose information they could absolutely confirm was taken. Of their delay in reporting the incident, the government employees insisted that they didn’t initially think any confidential personal details had been stolen—despite the fact that a large volume of data had been downloaded through the backdoor. "We regret that this breach has occurred and we have taken immediate action to enhance the security of these sites," Callie T. Dietz, the state’s court administrator, said in a written statement. Dietz also offered this fun fact: The break-in was the first time the court system’s network had been hacked. Hurray! Trophies and orange slices for everyone on the team!

Read More

Another Excuse For Why Tennessee Will Make State IT Workers Reapply for Their Jobs

You may recall that I recently wrote about the apparent success of New Hampshire’s new US $90 million Medicaid Management Information System (MMIS) that went live last month after years of technical difficulties, cost overruns and delays. This was a bit of good news, given that implementations of state Medicaid/Medicare systems have a notoriously bad track record, as the project problems in Maine, Ohio and Idaho have illustrated.

Alas, the difficulty with implementing these systems was highlighted once more when late last month Tennessee announced that it was stopping work on its Vision Integration Platform (VIP) after seven years of development. According to a story in the Tennessean, the state’s Department of Human Services made only a very brief, content-free announcement about the reasons behind its termination decision on a Friday, apparently in the time-honored ploy to reduce the political impact of the news. Tennessee has had a number of high-profile IT state project problems over the past few years affecting the Department of Children’s Services, the Department of Labor and Workforce Development, as well as with the state’s attempt to implement its Project Edison payroll system.

The VIP project was to provide comprehensive automated support for Temporary Assistance for Needy Families, Food Stamps, Medicaid and TennCare, as well as other state supported programs. A February 2005 press release (pdf) from the state’s Department of Human Services said that the $37 million project would take be completed by the summer of 2008.

However, the VIP project has repeatedly missed its deadlines, with the latest being 1 April 2013. A 2012 Tennessee government audit report (pdf) blamed the missed deadlines on “defects in current designs or new functionality requirements,” the Tennessean reported. The state has spent in excess of US $20 million on the VIP project so far, and is now trying to figure out what to do next, such as to start over or to try to use what has been developed so far.

The VIP fiasco is serving to help sell Tennessee’s Governor Bill Haslam’s controversial decision announced in early April to force all of the state’s 1600 information technology workers to reapply for their jobs. Another Tennessean story says that that the purpose of the decision is to weed out “those who can’t master the skills of a rapidly changing field.”  Mark Bengel, the state’s CIO said, apparently with a straight face, “This is really not about getting rid of people. It’s about making sure that we do have the skills and we have the ability to develop and retain staff in the future.”

Read More

IT Hiccups of the Week: Online Testing Problems Spread

Last week saw the ongoing effects of several IT-related problems initially spotted over the past month. We start with problems several U.S. states have been having with the online testing systems upon which they increasingly rely for carrying out all of their standardized testing.

More States Experience Online Testing Issues

In mid-April, I wrote about how 15 000 Minnesota students who were trying to take their Minnesota Comprehensive Assessment online math test either couldn’t sign in or had their tests ended prematurely. The snag was because of a server issue at the American Institutes for Research (AIR), which the state had hired to run the testing. Those kids weren’t alone; more testing problems cropped up about a week later, frustrating the plans of several other Minnesota school districts, Minnesota Public Radio reported. However, a story at the Minneapolis StarTribune says that AIR is denying that the latter testing problems had anything to do with its system. Nevertheless, the ongoing online testing issues forced the state’s education officials to announce last week that they are pushing back the deadlines by which the exams have to be administered. The math and reading tests were to have been completed by May 10 and science tests by May 17. No new completion dates have yet to be set.

The Minnesota students having trouble taking their online tests apparently have lots of company. According to an Associated Press story, students in Oklahoma, Kentucky and Indiana also have had difficulties with their online tests. Other news reports highlighted testing issues in Ohio and Alabama. Again, server-related issues seemed to be a root cause. The problems in these states were similar to those experienced in Minnesota: students couldn’t log on, suffered slow response times, or were kicked off in the middle of their tests. Testing in Oklahoma and Indiana is run by CTB/McGraw-Hill; in Kentucky, ACT oversees the student assessments.

The problems in Indiana not only have students on edge, but teachers too, as their merit pay is tied to student test scores. McGraw-Hill, which has a four-year, $95 million contract to operate the state’s online testing system, could be fined “$50 000 for each day last week the test was down,” WTRV Channel 6 in Indianapolis reported. McGraw-Hill apologized for the problems students had in both Oklahoma and Indiana, saying “We sincerely regret the problems we have caused,” and offering assurances that everything is fine now. The apology rang pretty hollow to many teachers and parents of students affected by the outages, however.

In Kentucky, the problems have caused state officials to suspend online testing for students who haven’t completed their tests and move back to the tried-and-true, paper-and-pencil approach. According to various news reports, state officials acted after ACT gave conflicting reports on the status of its testing system.

ACT was said to have “apologized for any inconvenience" caused.

Read More
Advertisement

Risk Factor

IEEE Spectrum's risk analysis blog, featuring daily news, updates and analysis on computing and IT projects, software and systems failures, successes and innovations, security threats, and more.

Contributor
Willie D. Jones
 
Load More