Risk Factor iconRisk Factor

Toyota Enters into Settlement Talks over Sudden Unintended Acceleration

IT Hiccups of the WeekThis week’s edition of IT hiccups, snarls, and general foul-ups begins with the surprising announcement last Thursday by U.S. District Judge James V. Selna who, according to Bloomberg News, issued an order stopping lawsuits into claims of sudden unintended accelerations in vehicles manufactured by Toyota. The reason: to give time requested by both Toyota and plaintiff lawyers to find a way to settle claims against the car manufacturer.

As long time readers of the Risk Factor may recall, the issue of sudden unintended acceleration (SUA) really came to the fore in 2009 when Toyota issued an initial recall of 3.8 million vehicles over the possibility that floor mats were jamming accelerator pedals, keeping them in the full open position. A fatal crash in California the same year took the life of a veteran California Highway Patrol Officer (along with his wife, teenage daughter and brother-in-law) who could not find a way to stop a runaway 2009 Lexus ES 350. That incident helped highlight claims of additional sources of SUA problems with Toyotas such as software/hardware-related defects inadvertently affecting Toyota’s electronic throttle control system. These claims (along with Congressional pressure) forced the National Highway Traffic Safety Administration to conduct an investigation which reported no such defects could be uncovered. Toyota had long insisted that most cases of SUA were the result of driver error and not electronic-related, and used to the NHTSA investigation to bolster its argument.

Even though the NHTSA couldn't uncover anything wrong with Toyota's electronic throttle system, that finding didn’t stop SUA lawsuits from being filed against the company, which were to date unsuccessful at showing anything other than possible floor mats or driver error being responsible for SUA. In early October, for example, the NBC News reported that Toyota yet again prevailed in an SUA lawsuit against it.

However, later that same month, the LA Times reported that an Oklahoma jury found that electronic defects were indeed responsible for causing SUA in a 2005 Toyota Camry which “caused it to accelerate out of control and crash into a wall, killing a passenger and seriously injuring the driver.” The jury found that Toyota was guilty of “reckless disregard” in the case after defense software forensic experts convinced it that there were indeed, as the EE Times stated, fatal flaws in Toyota’s electronic throttle source code.

Toyota, stunned by the $3 million verdict and its implications, moved quickly to settle the case. Toyota continued to strongly argue—at least publicly—that SUA was not caused by electronic issues; but privately, the company must have worried that the jury verdict was the proverbial straw that broke the camel’s back. As a result, Toyota apparently decided that it had more to lose by going through hundreds of trials than in reaching a broad settlement agreement. Already, Toyota has reached a settlement in another lawsuit in West Virginia. I’ll continue to report on the proposed settlement as it becomes public, and especially whether Toyota now admits that there were software defects in its electronic throttle control software after all.

In other IT snafu news, Yahoo Mail experienced outages for days last week due to a hardware problem in one of Yahoo’s storage systems beginning Monday night. Yahoo, after steadfastly refusing to say how many of its 100 million daily users were affected,  finally conceded at the end of the week that about a million users were affected—although I doubt anyone believes that number is really representative, given breadth and depth of the user complaints voiced.

Last week also saw continued problems with Florida’s new US $63 million unemployment system that was launched in mid-October. While the state government insists that the system is generally working successfully, news stories including one at the Miami Herald continue to report thousands of user complaints that paint a portrait of a dysfunctional system. The technical problems are now morphing into a political headache for Gov. Rick Scott, as politicians of all stripes continue their call for an investigation into what went wrong and why it is taking much longer to fix than the state promised.

Finally, IT issues with the Affordable Care Act website, which was rebooted 16 days ago, continue to be reported. The Washington Post reported over the weekend that thousands of people who thought they had enrolled for insurance actually weren’t because their enrollment records were never transmitted to insurers. The Obama Administration insists that problems with enrollments are being quickly solved, but the New York Times says insurers beg to disagree. In addition, several states continue to report trouble with their health insurance website implementations, with Oregon’s being termed an absolute fiasco. Past history gives me great confidence that additional IT-related problems will surface well into the foreseeable future.

Toyota Decides to Cut Its Losses over Sudden Unintended Acceleration Lawsuits

Toyota Cars, Coding and Carelessness

After Four Years, Toyota Enters Settlement Talks

Toyota Seeks Settlement Over Sudden Acceleration Cases

Toyota Suddenly Flies White Flag in Sudden Acceleration Lawsuits

Toyota SUA Settlement Options Explained

Yahoo Apologizes for Embarrassing Email Outage

Yahoo Outage Hits 70% of Messages

Yahoo Silent over Outage

Yahoo Mail Outage Enters Fifth Day

Marissa Mayer Apologizes for Yahoo Mail Outage

Florida’s New Unemployment System Woes Now Becomes a Political Issue

Florida’s Unemployment System Payments Remain Tied Up

Gov. Scott Brushes Off New Unemployment System Complaints

Unemployment System Woes Becoming a Florida Campaign Issue

Florida Fines Deloitte US$1.5 million over Unemployment System Problems

Of Other Interest …

New Zealand Novopay Snafus Persist a Year On

UK Waitrose Supermarket Suffers Online Delivery Glitch

Data Issue Affects Melbourne Australia Air Traffic Control System

Electronic Benefits Transfer Card Glitch Affects Massachusetts Assistance Recipients

Billing System Problems Hits Johannesburg's Finances

Cable Theft Causes Three-day Broadband Blackout in West London

Image: Mixmike/iStockPhoto

Financial Exchanges Close Ranks to Fight Off Cybercrime

Following a string of confidence-shaking cyberattacks on stock exchanges across the globe that affected their operations, 57 stock, futures, and options exchanges have come together to collaborate on cybersecurity best practices. I guess they've come to the same conclusion expressed in a coinage attributed to Benjamin Franklin: "We must, indeed, all hang together, or assuredly we shall all hang separately."

A hair-raising example of how vulnerable the exchanges are came in August when NASDAQ’s systems were besieged by more than double the amount of data they could process. The data torrent, abetted by a software design flaw, caused a three-hour stoppage in trading for thousands of U.S. stocks. Though the culprit was eventually revealed to be human error instead of a cyberattacker, the event revealed one avenue that a crafty hacker could exploit.

The new group, a committee established under the aegis of the World Federation of Exchanges, will try to figure out how to best share information on attackers, their tools, and attack trends, as well as techniques and technologies for fighting off attacks. It’s easier said than done, explains Mark Graff, NASDAQ's chief information security officer and chairman of the new working group. “When I took the job at NASDAQ, I found it was easy to connect with people within the [U.S.] financial community,” Graff told Computer World. “But I just couldn't see who my opposite numbers were in exchanges overseas,” he said.

G-20 Governments in Hackers’ Crosshairs

Researchers at online security firm FireEye say that In the month leading up to the G-20 Summit in September, hackers they presumed to be Chinese nationals broke into the computer networks of five European foreign affairs ministries.  FireEye was temporarily able to monitor the activity of the attack, which it calls Ke3chang, via one of the command-and-control (CnC) servers the hackers used. The campaign began with a series of spear-phishing e-mails laced with a malicious attachment called US_military_options_in_Syria.zip. The attackers knew that the targets would go for the bait because in the run up to the G-20 meeting, the world’s attention was focused on the Syrian civil war and whether the United States would intervene in response to the use of chemical weapons.

For a few days, FireEye researchers were able to snoop on one of the at least 23 different CnC servers the hackers used. They saw 21 compromised computers connect to that server.

In Other Cybercrime News…

  • A hacker who tried to make money by selling access to several corporate, university, and government computer networks—including two supercomputers at the Lawrence Livermore National Laboratory—fell into a familiar trap. It just so happened that the person on the other end of a US $50 000 transaction that would have given the buyer access to the Lawrence Livermore machines was an undercover FBI agent. This week, 24-year-old Andrew Miller, hacker and police-procedural TV show stereotype, was sentenced to 18 months in prison.
  • The makers of a popular Android flashlight application apparently kept users in the dark about its money-making side business: covertly tracking the locations of  “Brightest Flashlight Free” users and selling that information to advertising firms. The company, Goldenshore Technologies, reached a settlement this week with the U.S. Federal Trade Commission, which threatened to come down hard on the app maker.  
  • AT&T cares about you. So much, in fact, that the company refuses to issue a transparency report providing details regarding what data it has turned over to the U.S. National Security Agency. In a letter to the Securities and Exchange Commission, AT&T says that telling the world about the extent to which it divulged information about its customers would upset its efforts to protect its customers’ privacy. You can’t make this stuff up.
  • Eight of the world’s leading tech companies—Facebook, Apple, and Google among them—have created a new coalition whose aim is to provide pushback on U.S. surveillance practices. The group, Reform Government Surveillance, says that tactics such as National Security Letters, which demand that a company turn over data about customers and keep quiet about it, undermine trust in the companies and in the Internet as a dependable medium for communication and commerce.
  • Kaspersky Lab’s ThreatPost reports that Open WhisperSystems’ TextSecure protocol has been integrated into an app that will bring end-to-end encrypted text messaging to 10 million Android users.

Photo: vladru/iStockPhoto

IBM Sued Over Queensland Health Payroll System Debacle

It hasn’t been a good few weeks for IBM. You may recall, recently Bridgestone Tire filed a US $600 million lawsuit against IBM alleging fraud over an SAP-based invoicing, accounting, and product delivery system went that went live in January 2012 but didn’t operate as Bridgestone expected to say the least. Now news has come out that IBM is being sued by Australia’s Queensland government over its role in the disastrous Queensland Health payroll system implementation. The government wants compensation from IBM, but it did not disclosed the amount it is seeking.

As you may also remember from my years of covering this debacle, IBM was the lead contractor on the effort to replace Queensland Health’s legacy payroll system at an expected cost of A$6.19 million (fixed price) that turned into one that will cost an estimated A$1.2 billion to develop and operate properly when all is said and done. A formal commission of inquiry into the payroll system acquisition and development characterized it in its 264-page report [pdf] that was released in July as being one that must take place in the front rank of failures in public administration in this country. It may be the worst.”

Read More

UK Air Traffic Control Problem Snarls Flights over Weekend

IT Hiccups of the Week Trainspotting is still a popular hobby in UK; spotting computer-related foul-ups may soon become as popular, for last week UK residents (and many visitors) experienced a full train-yard-worth of computer woes.

We start off this week’s review of IT hiccups with the UK National Air Traffic Services (NATS) nighttime to daytime operations switchover that didn’t happen as scheduled at 0600 London time Saturday morning. As a result of the failure, which affected controller communications, hundreds of domestic and international flights into and out of the UK and Ireland were delayed and many cancelled. NATS went to its back-up system, which allowed it to operate at about 80 percent of capacity; full operations were not restored until 1900 Saturday night. The effects of the problems were felt well into Sunday.  

Early last Monday evening, the Royal Bank of Scotland Group's computer systems, which support RBS along with the two other banks (NatWest and Ulster Bank), went down for three hours, halting all three banks' financial transactions. The banks’ 15.7 million customers were not amused, it being Cyber Monday, one of the busiest shopping days of the year.  As you may recall, the RBS Group suffered a massive computer system meltdown in June 2012 that lasted nearly two months before it was fully straightened out. That snafu was preceded by a major outage in November 2011. Bank CEO Ross McEwan apologized for the latest cock-up, blaming it on RBS failing to “invest properly” in its IT systems “for decades.”  I am sure that apology was just the tonic to mollify customer anger. Just to add to the fun, on Wednesday, the three banks’ online systems were unavailable for about an hour because of a denial of service attack.

Also last week, the German-owned gas and electricity supplier Npower sent out letters to its 3.4 million English and Welsh customers apologizing for  “service issues resulting from the installation of a new billing system and a promise that customers will not lose out financially as a direct result of these issues.” It is estimated that over a million Npower customers either owe money or are owed money because of problems with the £200 million billing system that was installed in 2011. At the time, Npower was bragging that because of its deliberate approach, it wasn’t expecting any problems with its roll out.

Lest we forget, the Affordable Care Act website that was rebooted 10 days ago hasn’t fully escaped the IT-related problem orbit. The good news is that people are increasingly able to enroll for health insurance through the federal website, with more enrollments in two days after the reboot than all of October, when it was first launched. The bad news is that, of the 127 000 people who enrolled through the website in October and November, roughly one-fourth of their applications contained errors. The result: enrollees may not have insurance even though they think they do. The reboot has reduced the error rate to “only” 10 percent, the Obama Administration says, but with many more folks being able to sign up, that may not be exactly positive news. In addition, the Administration is now trying to discourage the use of paper ACA applications “because of concerns those applications would not be processed in time.”

State health insurance exchanges in Maryland and Oregon continue to have problems, while in California, the exchange secretly sent the names, addresses, phone numbers and addresses of tens of thousands to insurance agents of anyone who started a health insurance application, even if they didn’t complete it. That news hasn’t gone over well, even though California says that what it did is perfectly legal.

Finally, today is Grace Hopper’s 107th birthday, appropriately marked by a Google Doodle. I was privileged to meet her twice when I worked as an electronic engineer for the Department of the Navy in the 1970s; she was truly a remarkable person.

UK National Air Traffic Services Night to Day Switchover Doesn’t

UK Air Traffic Control Outage Causes Flying Misery

Computer Issue Hits UK National Air Traffic Control

NATS Apologizes for Flight Disruptions

Ryanair Rages at NATS over Outage

NATS Says Outage a “Just a One-off”

Royal Bank of Scotland Irritates Millions of Customers Once More

RBS Suffers Third IT Meltdown in 18 Months

Customers Furious with RBS over Latest Fiasco

Customers Skeptical of RBS Promises of Compensation

IT Cost Cutting Blamed for Problems

RBS CEO Apologizes For Latest IT Failure

Npower Apologies to English and Welsh Customers over Unacceptable Computer Billing Errors

Electricity and Gas Supplier Npower Apologizes to Customers

Npower Says “Sorry” for Those Billing Foul-ups

Npower Customers Angry at Incorrect Bills

Customer Service to be Outsourced to India Npower Announces

Of Other Interest …

Key West Flights Affected by Computer Problems

US Veterans Administration Claims System “Spontaneously” Shuts Down

UAE and Gulf HSBC Bank Customers Angry over Glitch

First Niagara Bank Customers Can’t Access Online Accounts

US Treasury Delays Securities Sale Due to Glitch

Arizona’s Motor Vehicle Department Computers Crash

Florida’s Unemployment Department Sends Tens of Identical Letters to Thousands

Photo: Steve Parsons/AP Photo

Treaty Limiting Weapons Exports Updated to Include Cyberweapons

Diplomats representing several Western governments are huddling in Vienna this week in the hopes of finalizing new, Internet-related additions to the Wassenaar Arrangement. That pact—under which the United States, Russia, Japan, France, Germany and dozens of other signatories agree to strictly limit exports of certain weapons—is being updated in order to control access to complex surveillance and hacking software and cryptography. These countries hope to keep sophisticated cyberweapons out of what they consider to be the wrong hands despite explosive growth (pun intended) in the cybersnooping market.

An example of the technology the signatories hope to keep inside the group’s proverbial fence is “deep package inspection.” According to a Financial Times article, “Western intelligence agencies are particularly concerned [about restricting access to such advances]” because they don’t want their enemies to “foil cyber attacks or gain an intimate understanding of Western screening systems and their fallibilities.” A spokesperson for the UK’s Department for Business, which deals with the Britain's export license regime, told FT that: “The government agrees that further regulation is necessary. These products have legitimate uses in defending networks and tracking and disrupting criminals but we recognize that they may also be used to conduct espionage.”

No Such Thing As a Completely Isolated Computer

Researchers at the Fraunhofer Institute for Communication, Information Processing and Ergonomics in Germany have just published a paper describing how they created a wireless mesh network capable of sending short bits of code to or intercepting data from air-gapped machines.

How does it work? Audio signals in the low ultrasonic frequency range (around 20 kilohertz) were transmitted from one machine to another over a maximum distance of about 20 meters. According to a Computer World article,

The data was transmitted using two different acoustical modem software applications called Minimodem and Adaptive Communication System (ACS) modem, the latter delivering the best results. On the network layer, the researchers used an ad-hoc routing protocol called GUWMANET (Gossiping in Underwater Mobile Ad-hoc Networks) that was developed by FKIE for underwater communication.

The nodes on the network, in this case laptop computers, have to be in direct line of sight, but the researchers note that it’s not unusual to find computers in such an arrangement in labs and open-plan offices.

Though the network—a dream come true for cybercrooks including nation states looking to engage in espionage or sabotage—currently limits data transmission to about 20 bits per second, that’s still enough to snatch login credentials and encryption keys or relay an attacker’s commands.

In Other Cybercrime News…

Image: Getty Images

The U.S. Air Force Explains its $1 Billion ECSS Bonfire

“We learn from failure, not from success!”

Well, if we apply Dracula author Bram Stoker's maxim to the U.S. Air Force, it could make the case that it has learned the most of all the U.S. military services.

A few weeks ago, the Air Force finally released the executive summary [pdf] of its investigation into its Expeditionary Combat Support System (ECSS). The system was a development blunder that the service mercifully terminated last year after spending US $1.03 billion over seven years and producing a system—if you can even call it that—without “any significant military capability.”   The  ECSS project  began in 2004 as an ambitious and risky effort to replace some 240 outdated Air Force computer systems with a single integrated enterprise resource planning  (ERP) system aimed at modernizing the service's global supply chain. It was also meant to help provide the core financial information required to meet a Congressional mandate that demanded an auditable set of books by 2017.

Read More

Los Angeles Department of Water and Power Scrambles to Fix Billing System Mess

IT Hiccups of the WeekAs it has the previous few weeks, news about the reboot of the Affordable Care Act website again overflowed the IT-related problem space last week, for the final time Obama Administration officials hope.

Obamacare website 2.0 was launched over the weekend, with the Administration claiming that the updated site is superbly better than when it was first rolled out on 1 October. For instance, according to a new Center for Medicare and Medicaid Services progress and performance report (pdf), the website's response time is now less than 1 second instead of the previous 8 seconds, the per page system times out are now only 1 percent of the time instead of over 6 percent, and some 50 000 concurrent users can now access the site, instead of the measly 500 or less on 1 October.

However, Health and Human Services Secretary Kathleen Sebelius, even as she was touting the ACA website’s “dramatic improvement,” also urged potential users to visit the ACA website during “off-peak hours when there is less traffic — mornings, evenings, or on weekends” or to “sign up for coverage… by phone, in person, and by mail. In many cases, you can also directly enroll through an insurance company.” That is probably good advice, for news reports from yesterday indicate that instead of the website being able to support 50 000 concurrent users, about 35 000 concurrent users is actually the reality.

Insurers have been less than impressed with the new and improved website, though. According to the New York Times, customers may be able to sign up for insurance, but that doesn’t necessarily mean that they actually been enrolled for insurance because sign-up information isn’t reaching the insurers or the information sent contains corrupted or incomplete data. As a result, the Times reports, insurers are saying “they had received calls from consumers requesting insurance cards because they thought they had enrolled in a health plan through the federal website, but the insurers said they had not been notified.”

Insurers were also unhappy last week when the Administration announced that the back-end system needed to pay insurers was being delayed from being finished in January to a date not yet specified. The insurers have been told they now need estimate what they are owed, and then they and the government can reconcile the differences.  Small businesses also joined the unhappiness queue last week, as the Administration delayed the small business health insurance exchange by a year.  Also in line are Oregonians, who have seen that state’s exchange fall into a technological abyss compounded by admissions of multiple security breaches.

Despite all of this disquieting news, there is hope on the horizon, the Administration says. For according to the CMS progress and performance report, the team that is working on ACA website and back office systems “is operating with private sector velocity and effectiveness, and will continue their work to improve and enhance the website in the weeks and months ahead.” In fact, the team is making such good progress, that former Obama senior adviser David Plouffe was moved to optimistically predict on Sunday that the ACA will “work really well” by 2017. Plouffe didn’t hazard an estimate of how much getting to that state of ACA nirvana will ultimately cost in both financial and personal terms, however.

The other IT-related impediments, deficiencies and malfunctions of the week centered on the teeth-gnashing issues involving the Los Angeles Department of Water and Power (DWP) new $162 million customer billing system. News reports state that over 70 000  faulty bills have been issued by its new customer information and billing system that was rolled out in September (pdf), which has led in some cases to DWP customers having their utilities incorrectly shut off. And in another bit of embarrassment for the DWP, it was scrambling to explain to LA taxpayers last week why it hid the fact that the true cost of the new billing system is nearly three times higher than what it had been previously publicly proclaiming.

Finally, last week’s IT hiccup news included various financially-related IT irritations to consumers during the annual period of U.S. shopping madness disguised as the Thanksgiving holiday, as well as hardware and software problems that accompanied the launches of the new Sony PlayStation 4 and Microsoft Xbox One consoles.

Los Angeles Department of Water and Power Scrambles to Fix Billing System Mess

Over 70 000 Faulty Bills Sent out By LA Department of Water and Power

LA City Council Unanimously Votes To Halt DWP Utility Shutoffs

DWP Agrees To Halt Utility Shutoffs Until End of the Year

LA DWP Admits Major  Billing  Problems Won’t be Fixed Until Spring 2014

Shoppers Experience Holiday Buying Frustrations

WalMart’s Black Friday One-hour Guarantee That Wasn’t

WalMart Suffers Another Online Pricing “Technical Glitch”

SunBank’s Multiple Transaction Error Hits Shoppers across the Country

Academy Bank “Glitch” Multiplies and Declines Customers’ Purchase Transactions

Hiccups Mar New Sony and Microsoft Consoles Launches

Sony to Replace PlayStation Consoles Suffering “Blue Light of Death”

PlayStation Network in Europe Struggling with Launch of PlayStation 4

Some Microsoft’s Xbox One Consoles Have “Disk Drive of Doom”

Of Other Interest …

Florida’s New Unemployment System Continues to Frustrate Unemployed Workers

Hardware Failure Takes Out FirstLight Federal Credit Union Online Banking

Ford Recalling 7 100 2013-2014 Model Year Lincoln MKZ Hybrids to Fix Transmission Software

Software Problem Affects Issuance of Disability Certificates in India

New Speed Camera Issues Ticket to Parked Car in Chicago

Reebok Trainers Are “Free” Thanks to Online Sales Error


Photo: Nick Ut/AP Photo

San Francisco's BART System Went Down Due to Server Upgrade Gone Bad

IT Hiccups of the WeekOnce more with feeling: the mêlée involving the Affordable Care Act website yet again dwarfed last week’s other IT-related impediments, which were relatively few for a change.

During last week’s round-the-clock Obamacare website glitch watch, for instance, we heard a government official admit that somewhere around 30 percent to 40 percent (no one seems to know for certain) of the required ACA back-office computing functionality related to how insurance companies get paid hasn’t been built yet. Documents were revealed showing that senior Obama Administration officials were worried, just before the website’s roll out, that there could be major problems—even though these same officials have claimed they had no inkling that website’s operation would lay down and play dead once going live. It was also revealed that, in a load test conducted just days before the website went live, the system choked when 500 users attempted to access the website simultaneously.  We also heard the Administration redefine operational success: a website that would work smoothly for 80 percent who try to enroll. This was immediately followed by debates about what that 80 percent measure actually means—if anything other than that a lot of people won’t be able to enroll for ACA health insurance via the website despite the promise of an “optimally functioning” website that would “work smoothly” by the end of November. These events had more than a little bit to do with extensions to the ACA 2014 and 2015 enrollment periods in order to help meet both Administration technical and political objectives. HealthCare.gov had company in its misery: There were continued delays to CuidadoDeSalud.gov, the Spanish-language version of the ACA website. Finally, despite everything, the White House released an upbeat report assuring the nation that everything will indeed soon be fine.  

The other IT-related obstacles, impairments and nervous breakdowns of the week included two rail system uffdas—one computer-related, and one apparently mechanical-cum-human error related. The first concerns a service outage on San Francisco’s BART (Bay Area Rapid Transit) system that lasted from late Thursday night into Friday morning. It was apparently caused by a server upgrade Thursday night that didn’t go according to plan. The second rail outage involved a New York City-bound Amtrak train that ended up going to Bala Cynwyd, Pennsylvania (outside Philadelphia) instead.

Finally, Boeing warned the 15 operators of Boeing’s 787 Dreamliner and 747-8 jumbo aircraft equipped with GEnx engines by GE not to fly at high attitude within 50 nautical miles of thunderstorms that may contain ice crystals. Apparently, there’s a risk of engine icing problems. Boeing and GE say that they are looking at a software fix to the engine control system which should be available early next year.

San Francisco’s BART System Goes Down for Several Hours

BART System Restored, But Commuters Left Seething

Software Problems Blamed for BART System Outage Trapping a Thousand Passengers

BART Explains Outage Caused by Bad Upgrade to Network Server

Amtrak Train 664 to New York City Ends Up in Philadelphia Suburb

Amtrak Train Crew Misreads Signal, Gets Lost

Amtrak Gets Turned Around on Way to New York City

Train Mechanical Problem Leads to Human-Error on Lost Amtrak Train

Boeing Tells 787 Dreamliner and 747-8 Jumbo Operators to Avoid Thunderstorms

Six Boeing Aircraft With GEnx Engines Have Had Engine Icing Problems

Boeing Issues Ice Risk Warning for GE-Powered 787 and 747-8 Aircraft

JAL Pulls 787 Off Two Routes

Of Other Interest …

Glitch delays 7500 Hennepin County Minnesota Employee Paychecks

Property Taxes Doubled In Princeton New Jersey Due to Software Glitch

New Election System Fails in Swaziland

Barclays Bank UK Online Systems Goes Out

Technical Glitch Takes Down Mexico Stock Exchange

Technical Glitch Blamed for Trading Halt on Qatar Exchange

Tesco Pricing Glitch Allows £9 Wine to Sell for £2.75

Restaurant Reputations in Northern Colorado Tainted by Health Department Software Error

Emergency Response System at SF Airport Failed Due To Software Problem Soon After July Crash


Photo: Maurits90/Wikipedia

Bridgestone Sues IBM for Fraud in $600 Million Lawsuit over Failed IT Implementation

This is already turning into one nasty, public fight.

On Monday, the newspaper The Tennessean ran an article about Nashville-based Bridgestone Americas, Inc., which is part of the Japanese firm Bridgestone Tire and Auto-service Corporation, bringing a US$600 million lawsuit against IBM. Bridgestone alleged in its complaint (pdf) that when the new US$75 million plus SAP-based invoicing, accounting, and product delivery system went live in January 2012, it found "that there were extremely serious defects in the IBM SAP design solution as implemented which Bridgestone had no reason to expect and for which IBM offered no explanation consistent with the purported concerns IBM had raised.”

As a result, the lawsuit states, “Bridgestone has suffered damages in excess of $200,000,000, and continues to suffer damages from injury to its reputation and customer relations.”

The lawsuit, which was filed 29 October, was sealed until recently. While the legal complaint is heavily redacted, in it Bridgestone alleges that IBM engaged in a “pattern of deception, intentional misrepresentation, and concealment” over its capabilities and the actual status of the project risks and problems. For example, Bridgestone states that IBM “assigned individuals, including the chief technical architect for the project, who did not possess the proper knowledge, skill, education, training, experience, technical expertise, and qualifications to perform the services necessary for the successful design and implementation." The lawsuit also says a lot of the work was outsourced to IBM workers in India and China who possessed less than stellar development skills and practices.

Bridgestone’s lawsuit alleges: (1) Fraud in the inducement and contract performance; (2) misrepresentation in business transactions; (3) constructive fraud; (4) violations of the Tennessee Consumer Protection Act; (5) gross negligence, and (6) breach of contract. The company wants a jury trial.

IBM, which has taken a battering over other failed IT implementations, including the Queensland Health payroll fiasco, the Indiana government outsourcing farce which is still unresolved, the Texas government outsourcing debacle, and the recent botched Pennsylvania government system implementation, has come out swinging. IBM immediately, publicly, and vehemently rejected the claims brought by Bridgestone. IBM gave its side of the story Wednesday to Business Insider, claiming in a statement that:

“Bridgestone filed a lawsuit claiming breach of contract and fraud against IBM regarding a recent SAP implementation. These claims against IBM are exaggerated, factually wrong and without merit. From the outset of this project, Bridgestone failed to meet critical commitments upon which the performance of IBM’s obligations were predicated.

Ultimately, Bridgestone’s repeated failures had a significant impact on the project’s cost and schedule, and its decision to prematurely roll-out the implementation across its entire business negatively impacted its operations."

Among the claims IBM made were that:

  • Bridgestone understood that this would be a challenging project. It had tried several times with other vendors and failed to upgrade its system. IBM was the only vendor to succeed in completing the upgrade to SAP.
  • Notwithstanding the complexity of the project and its negative history, Bridgestone failed to staff the project with people who sufficiently understood its own legacy systems and could assist IBM in designing and converting them into a new SAP system. Throughout, Bridgestone lacked the necessary leadership to effectively manage the project; it replaced its CIO on six occasions in a 2 year period during the project term.
  • Bridgestone failed to supply the necessary software, hardware and network infrastructure for the system to operate properly. In many instances, Bridgestone supplied inferior resources or no resources at all.

There is a lot longer laundry list of complaints which you can read in the Business Insider piece, but you get IBM's gist. Bridgestone, when asked to comment on IBM's statement blaming it for all the system's resulting problems, said its only response is contained in the complaint filed with the lawsuit.

A careful reading of Bridgestone’s complaint includes all of IBM’s points above and says why the tire company thinks those points don’t hold any (legal) water. The redacted proprietary parts of the complaint (which due to someone’s poor understanding of how to use redaction in PDF documents, is easily readable) discusses what appears to be the specific promises by IBM regarding its skills and capabilities, as well as how IBM said it would manage the implementation and any problems that would arise.

Bridgestone in its complaint says that it brought the lawsuit after mediation failed. It also indicated that it was during the mediation effort that it found out “that IBM had been engaged in a course of intentional deception, fraud, and misrepresentation throughout the project.” This seems to indicate that some sort of out of court settlement, like what happened when Avantor brought a lawsuit against IBM a year ago for “reckless indifference"  on another bungled SAP project, is not likely.

How much of Bridgestone’s lawsuit will stand is anyone’s guess. Some of the specific allegations in the complaint, many of which include IBM’s representations in the redacted bits, could, to my distinctly non-lawyerly eye, be thrown out as IBM merely engaging in puffery over its skills and capabilities. That's what happened when Marin County, Calif., sued Deloitte Consulting for fraud over an SAP project in 2010. Other allegations including IBM's agreement to only use personnel possessing the proper expertise and knowledge to carry out the statement of work may be more promising.

I’ll keep you updated on the progress of both the lawsuit and public brawl.

Photo: Tomohiro Ohsumi/Bloomberg/Getty Images

How Much Does Cybercrime Cost? $113 Billion

According to Internet security awareness training firm KnowBe4, the losses attributable to cybercrime total US $113 billion. Take a moment to let that astounding number sink in.

Now here's some more: The fourth annual Cost of Cyber Crime Study conducted by Ponemon Institute and sponsored by HP notes that costs for businesses that are victims of Internet-based attacks has risen 78 percent per year, on average, over the past four years. And from 2010 through this year, the time needed to recover from a breach has increased 130 percent. The losses in terms of personal information, intellectual property, and system damage are staggering enough. But now the average cost of cleaning up after a successful attack has passed the $1-million mark—not counting the cost of customer lawsuits against companies whose systems have been breached.  

Meanwhile, Symantec’s just-released 2013 Norton Report notes that although the overall number of victims of online attacks has actually decreased, the average cost per victim has risen by 50 percent. "Today's cybercriminals are using more sophisticated attacks, such as ransomware and spear-phishing, which yield them more money per attack than ever before," said Stephen Trilling, Symantec’s CTO in a press release.

In Other Cybercrime News…

Image: iStockphoto


Risk Factor

IEEE Spectrum's risk analysis blog, featuring daily news, updates and analysis on computing and IT projects, software and systems failures, successes and innovations, security threats, and more.

Willie D. Jones
Load More