Risk Factor iconRisk Factor

San Francisco's BART System Went Down Due to Server Upgrade Gone Bad

IT Hiccups of the WeekOnce more with feeling: the mêlée involving the Affordable Care Act website yet again dwarfed last week’s other IT-related impediments, which were relatively few for a change.

During last week’s round-the-clock Obamacare website glitch watch, for instance, we heard a government official admit that somewhere around 30 percent to 40 percent (no one seems to know for certain) of the required ACA back-office computing functionality related to how insurance companies get paid hasn’t been built yet. Documents were revealed showing that senior Obama Administration officials were worried, just before the website’s roll out, that there could be major problems—even though these same officials have claimed they had no inkling that website’s operation would lay down and play dead once going live. It was also revealed that, in a load test conducted just days before the website went live, the system choked when 500 users attempted to access the website simultaneously.  We also heard the Administration redefine operational success: a website that would work smoothly for 80 percent who try to enroll. This was immediately followed by debates about what that 80 percent measure actually means—if anything other than that a lot of people won’t be able to enroll for ACA health insurance via the website despite the promise of an “optimally functioning” website that would “work smoothly” by the end of November. These events had more than a little bit to do with extensions to the ACA 2014 and 2015 enrollment periods in order to help meet both Administration technical and political objectives. HealthCare.gov had company in its misery: There were continued delays to CuidadoDeSalud.gov, the Spanish-language version of the ACA website. Finally, despite everything, the White House released an upbeat report assuring the nation that everything will indeed soon be fine.  

The other IT-related obstacles, impairments and nervous breakdowns of the week included two rail system uffdas—one computer-related, and one apparently mechanical-cum-human error related. The first concerns a service outage on San Francisco’s BART (Bay Area Rapid Transit) system that lasted from late Thursday night into Friday morning. It was apparently caused by a server upgrade Thursday night that didn’t go according to plan. The second rail outage involved a New York City-bound Amtrak train that ended up going to Bala Cynwyd, Pennsylvania (outside Philadelphia) instead.

Finally, Boeing warned the 15 operators of Boeing’s 787 Dreamliner and 747-8 jumbo aircraft equipped with GEnx engines by GE not to fly at high attitude within 50 nautical miles of thunderstorms that may contain ice crystals. Apparently, there’s a risk of engine icing problems. Boeing and GE say that they are looking at a software fix to the engine control system which should be available early next year.

San Francisco’s BART System Goes Down for Several Hours

BART System Restored, But Commuters Left Seething

Software Problems Blamed for BART System Outage Trapping a Thousand Passengers

BART Explains Outage Caused by Bad Upgrade to Network Server

Amtrak Train 664 to New York City Ends Up in Philadelphia Suburb

Amtrak Train Crew Misreads Signal, Gets Lost

Amtrak Gets Turned Around on Way to New York City

Train Mechanical Problem Leads to Human-Error on Lost Amtrak Train

Boeing Tells 787 Dreamliner and 747-8 Jumbo Operators to Avoid Thunderstorms

Six Boeing Aircraft With GEnx Engines Have Had Engine Icing Problems

Boeing Issues Ice Risk Warning for GE-Powered 787 and 747-8 Aircraft

JAL Pulls 787 Off Two Routes

Of Other Interest …

Glitch delays 7500 Hennepin County Minnesota Employee Paychecks

Property Taxes Doubled In Princeton New Jersey Due to Software Glitch

New Election System Fails in Swaziland

Barclays Bank UK Online Systems Goes Out

Technical Glitch Takes Down Mexico Stock Exchange

Technical Glitch Blamed for Trading Halt on Qatar Exchange

Tesco Pricing Glitch Allows £9 Wine to Sell for £2.75

Restaurant Reputations in Northern Colorado Tainted by Health Department Software Error

Emergency Response System at SF Airport Failed Due To Software Problem Soon After July Crash


Photo: Maurits90/Wikipedia

Bridgestone Sues IBM for Fraud in $600 Million Lawsuit over Failed IT Implementation

This is already turning into one nasty, public fight.

On Monday, the newspaper The Tennessean ran an article about Nashville-based Bridgestone Americas, Inc., which is part of the Japanese firm Bridgestone Tire and Auto-service Corporation, bringing a US$600 million lawsuit against IBM. Bridgestone alleged in its complaint (pdf) that when the new US$75 million plus SAP-based invoicing, accounting, and product delivery system went live in January 2012, it found "that there were extremely serious defects in the IBM SAP design solution as implemented which Bridgestone had no reason to expect and for which IBM offered no explanation consistent with the purported concerns IBM had raised.”

As a result, the lawsuit states, “Bridgestone has suffered damages in excess of $200,000,000, and continues to suffer damages from injury to its reputation and customer relations.”

The lawsuit, which was filed 29 October, was sealed until recently. While the legal complaint is heavily redacted, in it Bridgestone alleges that IBM engaged in a “pattern of deception, intentional misrepresentation, and concealment” over its capabilities and the actual status of the project risks and problems. For example, Bridgestone states that IBM “assigned individuals, including the chief technical architect for the project, who did not possess the proper knowledge, skill, education, training, experience, technical expertise, and qualifications to perform the services necessary for the successful design and implementation." The lawsuit also says a lot of the work was outsourced to IBM workers in India and China who possessed less than stellar development skills and practices.

Bridgestone’s lawsuit alleges: (1) Fraud in the inducement and contract performance; (2) misrepresentation in business transactions; (3) constructive fraud; (4) violations of the Tennessee Consumer Protection Act; (5) gross negligence, and (6) breach of contract. The company wants a jury trial.

IBM, which has taken a battering over other failed IT implementations, including the Queensland Health payroll fiasco, the Indiana government outsourcing farce which is still unresolved, the Texas government outsourcing debacle, and the recent botched Pennsylvania government system implementation, has come out swinging. IBM immediately, publicly, and vehemently rejected the claims brought by Bridgestone. IBM gave its side of the story Wednesday to Business Insider, claiming in a statement that:

“Bridgestone filed a lawsuit claiming breach of contract and fraud against IBM regarding a recent SAP implementation. These claims against IBM are exaggerated, factually wrong and without merit. From the outset of this project, Bridgestone failed to meet critical commitments upon which the performance of IBM’s obligations were predicated.

Ultimately, Bridgestone’s repeated failures had a significant impact on the project’s cost and schedule, and its decision to prematurely roll-out the implementation across its entire business negatively impacted its operations."

Among the claims IBM made were that:

  • Bridgestone understood that this would be a challenging project. It had tried several times with other vendors and failed to upgrade its system. IBM was the only vendor to succeed in completing the upgrade to SAP.
  • Notwithstanding the complexity of the project and its negative history, Bridgestone failed to staff the project with people who sufficiently understood its own legacy systems and could assist IBM in designing and converting them into a new SAP system. Throughout, Bridgestone lacked the necessary leadership to effectively manage the project; it replaced its CIO on six occasions in a 2 year period during the project term.
  • Bridgestone failed to supply the necessary software, hardware and network infrastructure for the system to operate properly. In many instances, Bridgestone supplied inferior resources or no resources at all.

There is a lot longer laundry list of complaints which you can read in the Business Insider piece, but you get IBM's gist. Bridgestone, when asked to comment on IBM's statement blaming it for all the system's resulting problems, said its only response is contained in the complaint filed with the lawsuit.

A careful reading of Bridgestone’s complaint includes all of IBM’s points above and says why the tire company thinks those points don’t hold any (legal) water. The redacted proprietary parts of the complaint (which due to someone’s poor understanding of how to use redaction in PDF documents, is easily readable) discusses what appears to be the specific promises by IBM regarding its skills and capabilities, as well as how IBM said it would manage the implementation and any problems that would arise.

Bridgestone in its complaint says that it brought the lawsuit after mediation failed. It also indicated that it was during the mediation effort that it found out “that IBM had been engaged in a course of intentional deception, fraud, and misrepresentation throughout the project.” This seems to indicate that some sort of out of court settlement, like what happened when Avantor brought a lawsuit against IBM a year ago for “reckless indifference"  on another bungled SAP project, is not likely.

How much of Bridgestone’s lawsuit will stand is anyone’s guess. Some of the specific allegations in the complaint, many of which include IBM’s representations in the redacted bits, could, to my distinctly non-lawyerly eye, be thrown out as IBM merely engaging in puffery over its skills and capabilities. That's what happened when Marin County, Calif., sued Deloitte Consulting for fraud over an SAP project in 2010. Other allegations including IBM's agreement to only use personnel possessing the proper expertise and knowledge to carry out the statement of work may be more promising.

I’ll keep you updated on the progress of both the lawsuit and public brawl.

Photo: Tomohiro Ohsumi/Bloomberg/Getty Images

How Much Does Cybercrime Cost? $113 Billion

According to Internet security awareness training firm KnowBe4, the losses attributable to cybercrime total US $113 billion. Take a moment to let that astounding number sink in.

Now here's some more: The fourth annual Cost of Cyber Crime Study conducted by Ponemon Institute and sponsored by HP notes that costs for businesses that are victims of Internet-based attacks has risen 78 percent per year, on average, over the past four years. And from 2010 through this year, the time needed to recover from a breach has increased 130 percent. The losses in terms of personal information, intellectual property, and system damage are staggering enough. But now the average cost of cleaning up after a successful attack has passed the $1-million mark—not counting the cost of customer lawsuits against companies whose systems have been breached.  

Meanwhile, Symantec’s just-released 2013 Norton Report notes that although the overall number of victims of online attacks has actually decreased, the average cost per victim has risen by 50 percent. "Today's cybercriminals are using more sophisticated attacks, such as ransomware and spear-phishing, which yield them more money per attack than ever before," said Stephen Trilling, Symantec’s CTO in a press release.

In Other Cybercrime News…

Image: iStockphoto

Is It Fair to Steer Students into STEM Disciplines Facing a Glut of Workers?

The argument over whether or not there is a shortage of qualified STEM workers was replayed once more in a story this past week in a Chronicle of Higher Education article titled, “The STEM Crisis: Reality or Myth.” Unfortunately, you need to be a subscriber to gain full access to the article, but I thought a few quotes from the usual suspects claiming there is a STEM crisis in the United States would be enlightening.

For example, there's Robert D. Atkinson, president of the Information Technology & Innovation Foundation (ITIF), which receives a lot of its funding from high-tech companies. ITIF vehemently insists that the STEM crisis is real and that anyone who says differently is hopelessly misguided and uninformed. Atkinson argued that, among other things, college students need to be channeled towards “more useful” majors.

“We should be making some value judgments on what kind of people we'll need for the nation to move forward...The distribution of degrees right now is entirely up to students. Shouldn't we be steering them into degree types that are of more value to society, such as computer science or engineering? The American tradition is one of hard-core pragmatism. We're at risk of losing that, and we're in trouble now in regards to competitiveness.”

Atkinson goes on to imply that IT workers in the U.S. will just have to get accustomed to lower wages given that, “Companies can go overseas for workers.” Of course, the ITIF is a strong supporter of expanding the H-1B visa program for its high-tech paymasters, which has helped erode STEM wages, especially for engineers. Additionally, Atkinson maintains that, “there will be work in IT for people with the right set of skills…[and] that lower wages probably won't keep them from accepting jobs.

I would bet, however, it might discourage many potential engineering and computer students from pursuing those careers, as it has in the past.

The Chronicle article goes on to quote Anthony Carnevale, a research professor and director of Georgetown University's Center on Education and the Workforce, who also insists that there is a STEM student/worker shortfall (but who also once in a moment of candor admitted that any college student with math talent would be “crazy to go into STEM”). However, in the Chronicle article, Carnevale reasons that even if there is a glut of STEM graduates moving into the workforce, that’s okay because STEM grads “do better than other types of majors and tend to move into management pretty quickly.”

There's nothing like hedging your bets.

In fact, Carnevale continues:

“Having experience in technical matters helps them [STEM students] land good non-STEM jobs. They might work in places like marketing or medical-device sales, where their technical backgrounds helped them get in.” 

Yep, get an EE or CS degree, and you too can strive to get a job shilling medical devices.  Sounds to me like a winning slogan for convincing high-school students to pursue engineering or similar STEM majors. Maybe Carnevale can make up posters and send them to all the high schools to put up in their science and math classrooms.

On another related note, last week I had the opportunity to attend a Congressional briefing hosted by IEEE-USA and the AFL-CIO (a federation of trade unions in the United States) on the impact of the H-1B visa program on the economy, innovation, and the workforce. The panel was moderated by Ron Hira, associate professor of public policy at the Rochester Institute of Technology, and included Neeraj Gupta, CEO of Systems in MotionKaren Panetta, professor of electrical and computer engineering at Tuffs University and editor and chief of IEEE Women in Engineering magazine; and Hal Salzman, professor of public Policy at Rutgers University. The briefing drew a standing-room only crowd of House of Representatives staffers.

Hira provided a quick overview of the current H-1B visa program, and highlighted the fact that no one knows (or tracks) exactly how many H-1B visa holders there are in the U.S. He estimated that the total is around 650 000, with most working in the high tech arena. Hira also reported that the program does not require U.S. companies to actively recruit U.S. workers before seeking out H-1B visa workers, and that company compliance with the H-1B visa requirements is only maintained through whistle blowers such as Jay Palmer, who exposed Indian outsourcing company Infosys’s rampant abuse of the program. Palmer was supposed to attend the briefing to describe his Infosys experience, but unfortunately, his flight was canceled.

Gupta, who came to the United States as a student, was hired under an H-1B, and later became a U.S. citizen, talked (ironically) of the difficulty he faces as the CEO of a growing IT services company competing against H-1B outsourcers. He emphasized that H-1B workers are hired by U.S. companies as well as Indian and other foreign outsourcing companies primarily to lower their labor costs using mostly high-tech workers with average skills. Gupta argued that the H-1B program needs to return to its original purpose, which was to bring the truly best and brightest from across the world, not just primarily India, to work in the United States. This is not likely to happen, since the world's truly “best and brightest” are not likely to sign up to be treated as high-tech “indentured servants” as many H-1B visa holders do.

Salzman spoke of the latest data on STEM graduates and jobs, reiterating that STEM programs turn out at least 50 percent more IT graduates every year than there are U.S. job openings. He also said that if the H-1B program is ramped up to the numbers that are being advocated (up from 85 000 to 185 000), that worker oversupply could possibly increase to the 90 percent mark or more. Salzman called attention to Georgetown University’s report earlier this year that showed recent information system majors had a 14.7 percent unemployment rate, the highest of the majors it tracks. Even contemporary computer science graduates were experiencing an 8.7 percent unemployment rate.

Well, there are always those jobs selling medical devices.

Panetta noted that expansion of the H-1B visa program has had the effect of keeping down the already small numbers of women and minorities getting computer science and computer engineering degrees, since the more visa holders there are, the fewer job opportunities are available for U.S. workers. She also noted that only a small proportion of H-1B visas is given to female STEM graduates, even though 40 percent of the STEM graduates in India are women (this is more than double the U.S. percentage, she said). Panetta also noted how U.S. STEM students are facing school loan debts which are discouraging many to pursue graduate studies, a problem many foreign STEM students don’t have.

You can read more about the H-1B briefing in a ComputerWorld story as well.

Coincidentally, a few hours after the panel briefing, House Speaker John A. Boehner announced that full immigration reform would not happen this year. Boehner wouldn’t indicate whether it might be looked into again in 2014. While it may look like the H-1B visa cap will remain at 85 000 for the foreseeable future, President Obama signaled yesterday that he is open to “piecemeal” immigration legislation. This means that the H-1B cap may in fact be raised sooner than later, which would make Facebook and other tech companies very happy.

However, with CIO’s indicating that a slowdown in IT hiring may be in store for 2014, there seems little need for raising the H-1B cap anytime soon if ever.

Photo: Getty Images

Verizon Agrees to $60 Million Settlement over NY City Emergency 911 System Delays

IT Hiccups of the WeekDare I say it? The brouhaha with the Affordable Care Act website once more overshadowed other IT-related obstructions, complications and malfunctions reported in last week’s news.  

During last week’s news cycle, we learned a great many disturbing things. Among them: only a total of 106 185 people were able to sign up for ACA health insurance through October (27 794 of this total via the federal website); IT success is now defined as 4 out of 5 (in the best case) people being able to sign up for health insurance through the federal ACA website; confidence that the federal ACA website will be working by 30 November as promised is dwindling; the main Federal ACA contractor, CGI, has a less than exemplary record on government IT projects; and the unsurprising revelation that everyone involved with the federal ACA website development knew it was in deep trouble long ago, but no one had the guts to come out and forthrightly say so. And, of course, there was President Obama calling a press conference to apologize for the federal ACA website problems, and offer a temporary reprieve for those who saw their health insurance cancelled because it didn’t meet minimum ACA standards. Obama, who insisted that if only someone had told him that the website was so terrible, he wouldn’t have gone forward with its rollout, may be ginning up IT headaches for health insurers with the administrative changes related to the Affordable Care Act that he announced. We also learned that several states, including Oregon and New York, are reporting problems with their health insurance exchanges. Finally, over $4 billion is estimated to have been spent so far implementing the state health insurance exchanges, while the Federal effort accounts for at least $350 million at last count. Multiply this total amount by three or more to get the estimated IT maintenance cost over the next 15 or so years.

Yet, while the ACA ruckus was going on, several other IT-related inconveniences were reported. For example, Pennsylvania home care workers saw their paychecks delayed or lost for months because Pennsylvania's Department of Public Welfare mismanaged an IT program consolidation. There was news that Verizon has agreed to pay $60 million for botching New York City’s Emergency 911 system implementation, and disclosure of plans by major stock exchanges to try to reduce the IT outages that have been striking with increasing regularity over the past few years.

Verizon Agrees to $60 Million Settlement over Emergency 911 System Problems

New York City Mayor and Comptroller Argue Over How Much Verizon Owes in 911 Screw-ups

Verizon Settles for $60 million to Resolve Delayed Emergency 911 System Development

Mayor Bloomberg, Comptroller Liu Announce $60 Million Settlement Agreement with Verizon

Verizon Agrees to Pay New York City for Cost Overruns

Stock Exchanges Outline Plans to Stem Future Problems

Stock Exchanges Offer Plans to Stop Future Glitches

NYSE and Nasdaq Offer to Support Each Other in Event of Data Stream Issues

OTC Markets to Make Changes after Outage

Asia Markets and Regulators Work to Grapple With High Frequency Trading

Pennsylvania State Auditor Slams Department of Public Welfare for Payroll Mismanagement

Taxpayers Lose US $7 Million over Payroll System Mismanagement

Thousands of Home Care Workers Go For Months without Pay

Performance Audit Department of Public Welfare’s Oversight of Financial Services Providers (PDF)

Of Other Interest …

Living Social Suffers Multiple Day Outage

Facebook Messages Goes Down

Minnesota's "Give to the Max Day" Charity Drive Hit by Website Crash

NASA Curiosity Okay after Software Upgrade Problems Fixed

Montgomery County, Tennessee, Tax Bills Delayed Due to Software Issues

California Sends Incorrect Information to 246 000 New Medicaid Enrollees

Computer Glitch Does in Parking Pay Stations in La Crosse, Wisconsin

Woes with Florida’s New Unemployment System Could Last through Holidays

New Problems Emerge with Massachusetts New Unemployment System

Computer Problem Jams Main Causeway between Saudi Arabia and Bahrain Again

Bad Computer Data Sends Firefighters to Wrong Address In Spokane, Washington

GM Recalls 44 000 Chevy Malibu Midsized Cars for Computer Fix

Google, HP Halt Sales of Chromebook 11 over Faulty Chargers


Photo: Getty Images

Canada’s Missing STEM Skills Shortage

As in the United States and some other countries, there has been much hand-wringing in Canada about the lack of STEM and other skilled workers. For instance, a study late  last year by IBM proclaimed that Canada would be short 100 000 workers by 2016. In addition, in October, a report underwritten by pharmaceutical company Amgen Canada argued that Canada wasn’t producing a sufficient supply of STEM students while the Canadian Manufacturers and Exporters Association argued a few days ago that a skills shortage is significantly hurting Canadian companies’ competitiveness. Canadian government ministers have also been loudly asserting a “skills crisis.” Prime Minister Stephen Harper has gone so far as stating that the lack of skilled workers, scientists, and engineers was “the biggest challenge our country faces.”

Some Canadian economists and others have questioned the validity of these claims (including using the government’s own data to contradict Canada's official position). But the counter-claims didn't carry much weight until a few weeks ago. That’s when senior economists at TD Bank, the second-largest bank and financial services company in Canada, published an in-depth analysis (pdf) of the alleged wide-spread skills shortage in Canada and found the claims “exaggerated.”

TD's deputy chief economist Derek Burleton was quoted by CBC News as saying, “Evidence of economy-wide shortages is hard to find. Yes, across regions and occupations, skills mismatches (exist) because you are never going to get a perfect match. So it's not a complete myth, but it's not as extreme as people believe.”

Even in country’s Western provinces, which report the greatest skills shortages, wages have not risen measurably—something that happens when there is a shortage. The TD report says, “The story on the wage data remains curious, as wage gains out West have not increased to the extent that one might have thought given the signs of tightness.”

Of course, soon after the report was published, those with a vested interest in promoting the claim of a skills shortage took umbrage to it. The Information Technology Association of Canada (ITAC) and Information and Communications Technology Council (ICTC), for example, immediately issued a press release saying the bank’s analysis “does not hold up to scrutiny,” that the skills shortage was  real indeed, and that “addressing it is critical to Canada’s economy.”

This to and fro should all sound familiar to readers of my IEEE Spectrum article "The STEM Crisis is a Myth".

Photo: iStockphoto

U.S. Spy Agencies Losing Carte Blanche for Digital Data Gathering?

This Week in Cybercrime Until the revelations based on documents leaked by Edward Snowden came to light, the world had to take U.S. intelligence agencies’ word that they were adhering to legal limits on domestic and foreign data gathering. Now that we know better, all of the assurances they’ve made about the nature of their surveillance programs are under scrutiny. One such conceit—that the collection of metadata shouldn’t be viewed as surveillance—is being put to the test by researchers at the Stanford Security Lab at Stanford University. A new project, called Metaphone, will use metadata collected from the cellphones of volunteers to see how much additional information can be discovered when starting with logs of phone calls and text messages.

Meanwhile, the U.S. Senate began debate this week over the Surveillance Transparency Act introduced by Sen. Al Franken (D-Minn.). The bill would require that the U.S. National Security Agency (NSA) make revelations of its own. Among them: how broad a net it is casting in its data collection programs; what proportion of the people having their data collected are U.S. citizens or permanent residents; and whose information was actually reviewed by a government agent. The legislation would also eliminate the gag orders that prevent phone and Internet companies from divulging the number of orders they receive demanding customer data and the number of requests with which they comply.

More On the U.S. Government and Digital Surveillance

Data Insecurity Heightened by Government-Installed Backdoors In Hardware, Software, and Networks, says New Report

Google Fielded More Than 10 000 User Data Requests from the U.S. Government in the First Half of 2013—More Than Twice the Number of Requests Received in 2010

State Obamacare Exchanges Not Secure

Obamacare Update: Security Expert says State Healthcare Insurance Exchanges “Built In Such a Way as to Almost Attract Attackers"

In Other Cybercrime News…

New Microsoft Cybercrime Center Puts Security Engineers, Digital Forensics Experts, and Lawyers Trained in Fighting Cybercrime All Under One Roof

Hackers Steal $1.2 Million from Australian Bitcoin Wallet

Facebook Posts Alert Telling Potential Adobe Hack Victims to Reset Their Passwords

Internet Explorer 11 and Google Chrome Hacked at Mobile Pwn2Own

Security Researchers Say Svpeng, an Android banking Trojan Created by Russian Hackers, Can Phish for Bank Card Access Credentials and Issue Commands to Empty Victims’ Accounts

Microsoft Provides Patch for Windows Vulnerability Discovered in the Wake of a Watering Hole Attack Targeting Visitors of an Unnamed U.S.-based Domestic and Foreign Security Policy Website


Wal-Mart Apologizes for Unrealistically Low Prices

IT Hiccups of the WeekThe complications with the roll out of the Affordable Care Act (ACA) website continued its unbroken streak of dominating the news cycle related to IT-related snarls, snags and inconveniences. Among the list of IT health issues bedeviling the ACA’s website that cropped up last week are: the admission of a dramatic rise in the number of items on the “punch list,” from dozens to hundreds of problems that require fixing; the lack of interest from those to whom the ACA’s website was specifically targeted; a host of folks, including well-known IT luminaries such as economist Larry Summers, offering what I'm sure is welcomed at the White House, unsolicited advice on how to successfully manage the ACA website development effort; and counsel that if a user finds him or herself on an Obamacare website that seems to actually work, it is more than likely being operated by a scam artist.

But the ACA website was not the only IT system that was reportedly in poor health last week.  There was, for example, a Wal-Mart website pricing error that gave new meaning to the company’s slogan, “Always low prices.” A Fox News website called for World Zombie Day, and problems continued for Florida’s new unemployment insurance system website and back office systems.

WalMart Website Glitch Excites and then Disappoints Many with Low, Low Prices

WalMart Advertises $9 Computer Monitors

WalMart Glitch Sparks Buying Frenzy

WalMart Blames “Technical Issue” for Low Pricing Errors

WalMart Cancels Orders, Offers $10 Gift Certificate Consolation

Fox News Website Calls on All Zombies to Honor World Zombie Day

Fox News Website Claims “World Zombie Day”

Fox News Website Says “Weeeeeeee”

Fox Insists It Wasn’t Hacked, Just Had Production Problems

Florida’s New Unemployment Insurance System Website Still Causing Frustration

Florida Still Trying to Fix Deloitte-Built Unemployment System Website

Florida Officials Claim Only “Minor Glitches” with System, Others Disagree

More Trouble With Florida’s Unemployment System Upgrade

Of Other Interest …

Stanbic IBTC Bank in Nigeria Has ATM Problems

San Juan County, Washington, Declares Phone Outage Emergency after Fiber Optic Cable Cut

Cable Cut Takes out Bermuda’s Transport Control Department’s Website

Network Failure Halts OTC Market Trading

Instagram Goes Down Due to “System Issues”

Over 91 000 Model Year 2014 Jeep Grand Cherokee Recalled for Electrical & Software Problems

Canadian Broadcasting Corporation Labels Online Stories From 2003 as Being From 2013

NYC MetroCard Glitch Leaves Riders Unable to Pay by Credit or Debit Cards

Kentucky’s Online School Testing System Still Unreliable

Consumer Reports Says the Monster 7 Tablet, Sold Exclusively at Wal-Mart, Is Glitch-Prone

Computer “Glitch” Affects Missouri Courts

Researchers Claim IT Glitch Could Threaten 2015 UK Elections


Photo: Carlos Barria/Reuters

Have Heads Begun to Roll Over Obamacare Imbroglio?

Wednesday morning, as U.S. Health and Human Services (HHS) Secretary Kathleen Sebelius testified about the ongoing problems with healthcare.gov at a Senate Finance Committee hearing, the first head rolled as a result of the Obamacare website debacle. It was announced that Tony Trenkle, the CIO of the Centers for Medicare and Medicaid Services, the HHS entity responsible for creating the healthcare insurance portal, would be leaving his post as of 15 November. According to an e-mail sent to agency employees on Wednesday, Trenkle is “leaving for the private sector”; the agency’s chief operating officer, Michelle Snyder, announced a reshuffling meant to temporarily fill Trenkle’s role.

As for Sebelius, she told senators that, “We’re not where we need to be” with regard to meeting a 30 November deadline set by the administration for making the HealthCare.gov site functional. The HHS secretary revealed that “a couple of hundred functional fixes” would still need be made so the website would not get hung up, display error messages, or make other errors such as displaying blank drop-down boxes. (In this week’s IEEE Spectrum Q&A with risk management expert Robert Charette, he imagined that tackling a punch list containing only 30 items strained credulity.)

Read More

The Obamacare Rollout: What Really Happened?

By now, everyone with access to a newspaper or an Internet connection knows just how badly the first few weeks have gone for the U.S. healthcare insurance portal HealthCare.gov and some of the independent state-run insurance marketplaces. To say we are experiencing technical difficulties is an understatement. Despite testimony last week from the main contractors for the federal website and from Health and Human Services Secretary Kathleen Sebelius, we still have few details about what precipitated such a colossal failure and why no alarms went off before the site's ill-fated 1 October debut. IEEE Spectrum Assistant Editor Willie D. Jones talks with risk management expert Robert Charette about what likely happened and what we can expect going forward.

Jones: You wrote an article for IEEE Spectrum in 2005 that could have been used as the playbook for the HealthCare.gov debacle. I guess you’re somewhat of a prophet.

Charette: You can go back to the first book I wrote on software systems risk management back in 1989. It’s easy to make any project fail. You just don’t give it enough time, enough money, or requirements that are understandable.

Jones: Last week, Health and Human Services Secretary Kathleen Sebelius assured her inquisitors at a congressional hearing that her department has brought in experts that have a handle on the problems the site is facing. How confident should we be in Sebelius’ assurances?

Charette: Not very. They’re talking about dozens and dozens of items on their punch list—both in terms of functionality and performance issues. They’ve got just over 30 days to get through the list. Let’s just say that there are 30 items on it. What do you think is the actual probability of getting through testing them, making sure that the system works end to end and that there are no security holes all in a single month? How do you expect to get that done, knowing that every time you make a fix, there’s a high probability that you’re going to introduce an error somewhere else?

Jones: Let’s spin this forward a bit. How do you think this next month will actually go?

Charette: They said that they needed five weeks at the minimum to test it, and they’re still making all these changes. Where will that five-week window fit? If they had stopped right then and tested it for five weeks, they wouldn’t have been able to finish on time. And five weeks was probably the absolute minimum they needed, assuming everything worked. They’re patching the system as they go along and as Sebelius admitted, they’re doing very local unit tests (which, by the way, is what got them into this mess in the first place, with each contractor saying, "Well, my stuff works"). If they discover something major, they may have to run the whole system test again.

Read More

Risk Factor

IEEE Spectrum's risk analysis blog, featuring daily news, updates and analysis on computing and IT projects, software and systems failures, successes and innovations, security threats, and more.

Robert Charette
Spotsylvania, Va.
Willie D. Jones
New York City
Load More