There was a plethora of IT-related faults, errors and defects reported last week. We start off this week’s edition of IT Hiccups with what arguably is not a classic IT problem but a more general electronically-related one that affects over 1.6 million GM vehicles.
A few weeks ago, a New York Times article reported a world-wide GM recall (pdf) of 778 000 or so 2007-model-year Pontiac G5 and the 2005-7 Chevrolet Cobalt (619 000 in the U.S.) because a “jarring event” such as a crash, bumping the ignition, or a heavy key chain could inadvertently cause the cars’ ignition switches to move from the run position to the accessory position. Switching into that mode would disable the cars’ engines and prevent their air bags from deploying.
At the time, the Times reported that GM “knew of six deaths in five crashes in which the front air bags did not deploy” as well as 17 additional crashes “involving some type of frontal impact and nonfatal injuries where the air bags did not deploy.”
A GM spokesperson, having insisted that, “Safety of our consumers is paramount to G.M,” also tried to minimize the recall by saying that, “All of these crashes occurred off-road and at high speeds, where the probability of serious or fatal injuries was high regardless of air bag deployment. In addition, failure to wear seat belts and alcohol use were factors in some of these cases.”
GM recommended using only the key issued with the affected vehicles until the ignition switch is repaired.
In addition, GM disclosed the news that it had issued a service bulletin in 2005 about the issue to its dealers, although the car manufacturer was not exactly forthcoming in saying whether dealers felt obligated to inform vehicle owners of the potential problem before (or after) purchasing their GM vehicles. GM did say that “the ignition switch torque performance may not meet General Motors’ specification.” In other words, dealers were going to tighten the switch on the recalled vehicles.
A Detroit News story quoted several auto analysts who said “because the company took quick action” the reputation risk impact of the recall to GM was minimal, rating it as only a “6” on a scale of 10.
However, last week, GM expanded the recall by another 748 000 cars in the United States as it disclosed that four more of its vehicles—the 2003-07 Saturn Ion and the 2006-07 Chevrolet HHR, Pontiac Solstice, and Saturn Sky—also used the same ignition switch. The additions brought the total to nearly 1.37 million vehicles in the U.S., and 1.6 million vehicles worldwide. What’s more, GM revealed that the number of related deaths has reached thirteen and the number of reported crashes due to the defect rose from 22 to 31.
GM declined to explain to various news media inquiries why it did not include those vehicles in its original recall, why the additional crashes and deaths were not reported or linked to the others, or why it had taken so long for the company to issue a recall since it admitted that it knew of and had been studying the problem since 2004.
In fact, the Detroit News reported, GM “spent nearly a decade studying the issue and repeatedly opted not to recall the vehicles or pay for potentially expensive fixes.” GM, the Detroit News stated, “downplayed the ignition switch issue in prior years, including canceling in 2005 an approved redesign of the ignition key head. By the end of 2007, GM said it knew of 10 frontal crashes in which air bags didn’t deploy—linked to the ignition problem—but the automaker opted not to recall the cars.”
GM North America President Alan Batey, sensing that the recall issue had moved considerably higher on GM’s reputation risk meter, issued a “rare apology.” He said that GM was “deeply sorry” for the problem, and that the chronology of its actions, reported to the National Highway Traffic Safety Administration (NHTSA), showed that “the process employed to examine this phenomenon was not as robust as it should have been.”
NHTSA shortly thereafter announced it was launching an investigation into GM’s recall delay. GM faces up to a $35 million fine for not reporting problem in a timely manner to NHTSA. However, NHTSA was itself under heavy criticism from at least one U.S. senator for not acting earlier when it became aware of the problem with the ignition switch back in 2007. GM, in light of the NHTSA’s announcement, issued an unheard of second apology saying that, “We deeply regret the events that led to the recall and this investigation. We intend to fully cooperate with NHTSA and we welcome the opportunity to help the agency have a full understanding of the facts. Today’s GM is committed to learning from the past while embracing the highest standards now and in the future.”
A story in yesterday’s New York Times provides a bit more information about the chronology of the ignition switch issue. Among the damning details is the fact that back in 2004, GM engineers were able to replicate the problem and suggested a fix, but GM executives decided against it after “consideration of the lead time required, cost and effectiveness.” Another example of company executives playing “pay me now or pay me later” roulette.
[Update 06 March 2014: GM's new CEO Mary Barra announced in an email to GM employees this week that she had put into place a "working group of senior executives, which I lead, to direct our response, monitor our progress and make adjustments as necessary." In addition, Barra stated there is now "an internal review to give us an unvarnished report on what happened. We will hold ourselves accountable and improve our processes so our customers do not experience this again." She added that, "We sincerely apologized to our customers and others who have a stake in GM's success."
To "help" GM with its inquiry, NHSTA has now sent a 27-page list of 107-questions seeking all GM information about the recall and why it wasn't initiated earlier.
GM says that replacement parts for the defective ignition switches will begin to become available early next month.]
GM Chevy Silverado and GMC Sierra Truck Steering Control Malfunction
GM was also the source of an IT-related hiccup last week. According to Edmunds.com, GM is trying to convince NHTSA that a “glitch” affecting its steering wheel controls is “inconsequential to motor vehicle safety” and doesn’t warrant the recall of more than 200 000 GMC Sierra and Chevy Silverado trucks manufactured between 29 January and 28 October 2013. Another recall would be quite embarrassing considering that the Silverado was recently named 2014 North American Truck of the Year. You may remember that just a few weeks ago, GM recalled 370 000 of those same two vehicles for a software update in order to reduce the likelihood that their exhaust systems would overheat and catch fire.
In this latest problem, GM says that, “under certain circumstances when an owner uses the steering wheel controls to browse and select songs to play from an external device (i.e., MP3 player) that is plugged into one of the vehicle's USB ports, the instrument cluster may reset. When the instrument cluster resets the analog gauges and identifications, the PRNDM [shift position] indicator, and the cruise control telltale will briefly turn off. In addition, some of the instrument cluster telltales may also illuminate briefly without the condition the telltale is designed to indicate being present.”
It doesn’t sound like a major safety issue, but it is one that could annoy or even distract a driver. However, given aforementioned GM ignition switch recall debacle, the previous software recall on the same vehicles, and the interesting interaction of what would seem to be at least on the surface disparate vehicle systems, NHTSA may want a bit more information from GM before granting it a waiver.
New Zealand Hospital EHR Outage Sparks Political Row
While not nearly on the same technical level as the Affordable Care Act health insurance exchange problems in the United States, a relatively minor electronic health record system outage in New Zealand has created much the same political hue and cry, apparently.
Last week, Dunedin Hospital, which serves the Otago catchment of New Zealand, suffered an electronic health record outage that lasted a little more than a day due to an apparent hardware problem. Staff reportedly resorted to paper records during the outage and the hospital said patients were not put at any risk by the outage. Coincidentally, New Zealand Prime Minister John Key, who leads a National-led government, was visiting the hospital the day after the EHR system crash. Key, who was on hand to open the hospital’s new neonatal intensive care unit, reiterated to the press that the incident was not a big deal, saying that anyone working with computers “will know that at some point they break down.”
Others, however, such as out of power Labour Associate Health spokesman David Clark, saw something sinister in the outage. Clark vigorously proclaimed that, “Patient care has been compromised, there's no doubt; radiation treatment didn't happen yesterday; there have been other monitoring mechanisms that are in place that just simply weren't working.” Clark said the EHR outage was obviously the result of the ruling government’s “cost cutting pressures.”
While Clark was clearly trying to score political points, the hospital apparently has no robust back-up systems in place to handle equipment or software outages. Given that the hospital recently admitted that a different IT failure that resulted in the loss of 4000 mammogram images taken in 2012, it is probably fair that some IT professionals are calling for a review of the hospital’s IT systems.
DMV Headaches Abound
We close this week’s edition of IT Hiccups with a couple of stories of motorists having trouble with their local department of motor vehicles, something all of us can readily relate with. The first is about a software error in the computer systems of the Washington, D.C., DMV that has existed for at least 5 years (and possibly as long as 15). The bug had made it extremely difficult if not impossible for motorists to get refunds for incorrectly issued traffic citations. At least 450 paid traffic tickets that were later voided were not refunded because of the error—a problem the DMV says is now fixed, but was addressed only after television stations started reporting on motorists fighting for years to get their refunds.
Next, news reports emanating from North Carolina indicate that problems with that state’s new DMV computer system have resulted in its offices “losing or delaying up to 35,000 vehicle inspections per month.” The computer system, the DMV says, “has had logic and code issues” that frequently keep the record of a successful vehicle inspection from being successfully communicated to the DMV from the state’s 7500 certified inspection stations.
Typically, the DMV sends out a bill to the motorist once it is notified of a passed car inspection. As a result of the computer problems, a motorist can’t pay for their car inspection nor can they pay their annual vehicle fee when it is due since the state doesn’t know that the vehicle has been successfully inspected as required by state law. According to media reports, there is no timetable for when the problem will be resolved.
Finally, news reports from across New York State indicate that local DMV offices have been experiencing repeated IT outages that are driving both DMV employees and motorists crazy. As in North Carolina, no one in charge seems to have a date in mind as to when the outage problem will be fixed.
Alas, would you expect anything different from the DMV?
GM Recalls 1.6 Million Cars for Ignition Switch Fix
GM Recalls 778 000 Small Cars for Ignition Switch Problem
GM Ignition Switch Recall Not a Big Deal Auto Analysts Claim
GM Adds 750 000 Cars to Recall Bringing Total to 1.6 Million
GM Delayed Ignition Switch Fix for Years
GM Issues Rare Public Apology Over Recall
NHTSA Launches Probe into GM Cobalt Recall
NHTSA Aware of Ignition Switch Issue in 2007
Massachusetts Senator Markey Asks Where Was NHTSA?
Trail of Inaction in Ignition Switch Recall Debacle
Truck Glitch is Inconsequential GM Tells NHTSA
New Zealand EHR Meltdown Sparks Political Row
New Zealand Hospital Suffered EHR Outage
Labour Claims Patients Placed at Risk During EHR Outage
Review of Hospital IT Systems Called For
Motorists Suffer Because of State DMV Computer Woes
DC DMV Finally Fixes Software Error That Stopped Ticket Refunds
NC DMV Computer Error Affects Vehicle Inspections
NY DMV Offices Suffer Plague of Computer Problems
In Other News…
California Healthcare Insurance Website Software Problem Forces 14 500 Applicants to Start Over
Maryland Healthcare Insurance Website Flaws Cause $30 Million in Unnecessary Medicaid Payments
Washington State Healthcare Insurance Exchange Keeps 15 000 Applications in Limbo
Software Issue Delays Payment for Indian Eye Surgeries
Social Security Numbers Printed on Banner Health Address Labels
Milwaukee Fire Department’s New CAD System Has Flaws
New Digital Water Meters in LaVergne, Tennesse, Cause Billing Problems
Louisiana Disqualifies Food Stamp Recipients for Exploiting Benefits Glitch
Florida Says New Unemployment System Fixed, But Many Still Express Doubts
NPower Utility Billing Errors Power On
Swedish Company Accidentally Invites 61 000 to Job Interview
Pizzeria Unwanted Recipient of American Idol Votes
DeGeneres' Selfie Causes Oscar Twitter Crash
ABC’s Live Oscar Internet Stream Goes Out
Photo: GM/AP Photo
Robert N. Charette is a Contributing Editor to IEEE Spectrum and an acknowledged international authority on information technology and systems risk management. A self-described “risk ecologist,” he is interested in the intersections of business, political, technological, and societal risks. Charette is an award-winning author of multiple books and numerous articles on the subjects of risk management, project and program management, innovation, and entrepreneurship. A Life Senior Member of the IEEE, Charette was a recipient of the IEEE Computer Society’s Golden Core Award in 2008.