Risk Factor iconRisk Factor

IT Hiccups of the Week: Sabre Outage Hit Flights Worldwide

It’s been another eventful week in the land of IT snags, snarls and complications. We start with another major airline reservation system hiccup, but this time one with world-wide effects.

Sabre Reservation System Outage Affects 400 Airlines

The Sabre airline reservation system experienced a still unexplained two-hour-plus “system issue” beginning at 8:40 p.m. PDT on Monday evening. The outage affected airlines around the world. (Sabre’s website states that about 380 airlines use its reservation system.) As a result, ticket agents needed to hand write boarding passes, and the affected airlines’ websites couldn’t book or change reservations.  Where in the world you happened to be at the time impacted how severely you felt the system outage.

In the U.S., a small number of late-night West Coast domestic and international flights experienced delays. In Europe and the Middle East, the effects were felt a bit more, as it was early to mid morning. The greatest problems were felt in Australia, where it was early afternoon Sidney time. Virgin Australia got the worst of it.

Virgin Australia reportedly had to cancel 35 domestic and international flights and delay many others.  For whatever reason, Virgin seems snake-bit when it comes to reservation system IT issues. This complication came on the heels of a router outage that occurred just a few weeks ago. You may remember its nearly two week reservation meltdown back in 2010, as well. Virgin moved from Navitaire’s New Skies platform, which was at the heart of the 2010 meltdown, to the Sabre system earlier this year. The costs incurred in the switch helped send the company into the red.

Although the exact number of passengers affected is unknown, it was undoubtedly thousands worldwide given that Sabre says some 300 million passenger reservations are processed by its system every year. On Tuesday, in the wake of the outage, Sabre sent out the standard, “We apologize and regret the inconvenience caused.”

Denver International Airport officials were also apologizing on Tuesday to passengers as all 1200 airport flight boards were out for most of the day.Maintenance had been performed on the system that runs the flight boards on Monday night. The boards were not restored to operation until late Tuesday evening.

False Emergency Warnings Sent in Japan, Virginia, California

The Japan Meteorological Agency sent an emergency message last Thursday warning most of Japan to expect “violent shaking” after detecting a magnitude-2.3 earthquake in Wakayama prefecture in western Japan at 4:56 pm local time, Bloomberg News reported. According to the story, the Wakayama earthquake prompted JMA’s warning system to predict that a magnitude 7.8 earthquake was possible.

As a result of the warning, Central Japan Railway suspended some bullet train operations, and a number of mobile phone networks became jammed as a multitude of people called friends and family.

However, it soon became clear that the prediction was in error. The JMA blamed the false warning on “electrical noise” on the ocean floor, and offered a televised apology. The JMA admitted Thursday's incorrect warning to be the “biggest misreading” since the early warning system was begun in 2007.

On Wednesday morning, human error was blamed for a tornado alert mistakenly being sent to 500 people in the Charlottesville-Albemarle County area of Virginia, the AP reported. Apparently the notification was sent during a training session on how to use the local emergency alert system.

Also on Wednesday, a real emergency alert of a reported gas leak was sent to more people than intended. The automated wireless emergency alert message, which urged residents and businesses to evacuate immediately and only take essential belongings with them, was sent out across all of Contra Costa County, California, instead of the homes and businesses within a 1000 foot radius of a damaged gas pipe. County officials said that they would be working with the vendor the county uses to send alerts to ensure that the messages are better targeted, the San Jose Mercury News reported.

Xerox to Patch Scanner Feature

I doubt many people would think to look at a document they scanned to check whether in fact what was scanned actually match that as on the original. It might be a good idea to do so in the future, however.

Last week, a story Tuesday at BBC News reported that German computer scientist David Kessel  “discovered” that the compression software used by several Xerox scanner models had the nasty habit of changing the characters in the scanned document from those on the original document. The Daily Mail published an article showing some of the changes that could result.  The legal implications, a London lawyer told the BBC, were “Interesting.”

Xerox played down the error, however, saying that the vast majority of scanner users would never experience the problem since it only happened when the scanner's default resolution setting was reset to low resolution in order to save smaller-sized computer files.  The character changing/substitution issue was long known to be a possibility, and a warning about it was in all its user manuals, Xerox said. However, in light of the uproar the BBC News story generated—which was also fueled by Xerox’s nonchalant response to the issue—Xerox said it would be sending out a patch in the next few weeks to disable the highest compression mode which it claimed would eliminate the problem.

Even so, it might be a good idea to routinely check over your scanned documents just in case, and also maybe read  your scanner’s user manual. There may actually be something useful in it.

Also of Interest…

Navy Explains Why the USS Guardian Got Stuck on “Misplaced” Philippine Reef

BATS Exchange Experiences Another Outage

New Zealand Vodafone's Data Networks Problems Affect Mobile EFTPOS

X-Ray Computer System Problems Continue Unabated in Kent, England

Wisconsin State Government Recovering From Computer Crash

Los Angeles Fire Department 911 System Repeatedly Breaks Down

Japanese Luxury Toilet Has Computer Hardware Flaw

Photo: iStockphoto

This Week in Cybercrime: Is there Anonymity on Anonymity Networks?

Security researchers studying malware that exploits a hole in the Firefox browser’s security to unmask users of the privacy-protecting Tor anonymity network suspect that the author of the malicious code is…wait for it…the U.S. government. Journalists and human rights activists depend on Tor and services like it to evade surveillance or protect users’ privacy. But the hidden services have found themselves in U.S. law enforcement’s crosshairs because they cloak the activities of criminals such as Eric Eoin Marques, who was recently described by an FBI special agent as “the largest facilitator of child porn on the planet.”

A Sunday attack on several websites hosted by Freedom Hosting originated at “some IP in Reston, Virginia,” security engineer Vlad Tsyrklevich told Wired. “It’s pretty clear that it’s FBI or it’s some other law enforcement agency that’s U.S.-based.” So much for China being the nexus of cyber espionage.

Tsrklevich and other researchers think the malicious code is an example of the FBI’s decade-old “computer and internet protocol address verifier,” or CIPAV, the tool it has used to track down hackers, sexual predators, and other cybercriminals who use proxy servers or anonymity services like Tor to hide their identities. Wired reported on the spyware way back in 2007.

“Court documents and FBI files released under the FOIA have described the CIPAV as software the FBI can deliver through a browser exploit to gather information from the target’s machine and send it to an FBI server,” says a Wired article. Where is the FBI server in question? In Virginia.

The first clue that law enforcement is behind the hack is that the malware doesn’t steal anything nor does it lay any groundwork for future access to the systems. All it does is “look up the victim’s MAC address—a unique hardware identifier for the computer’s network or wireless card—and the victim’s Windows hostname. Then it sends it to the Virginia server, outside of Tor, to expose the user’s real IP address…”

DIY Femtocell Hack Sniffs Out Malware on Mobile Phones

In last week’s edition, we highlighted a presentation at Black Hat Las Vegas by researchers who figured out how to hack a femtocell portable cellular base station in order to intercept all data transmitted by nearby mobile handsets. They informed device makers such as Verizon about the exploit so it could be remedied. This week, Wired reported that the good guys have devised a method for using a femtocell to detect malware on mobile phones. In a presentation at the Def Con hacker conference in Las Vegas, researchers from LMG Security demonstrated a system they built for less than $300 that can view data transmitted from smartphones, through a femtocell, to a cellular carrier’s network. This allows a phone’s user to monitor his or her own data traffic for malicious activity.

“If your phone is infected … it can send audio recordings, copies of your text messages, and even intercept copies of your text messages so you never receive them,” LMG’s Sherri Davidoff told Wired. “Our goal is to give people the ability to see the network traffic” to determine if this is occurring.” The LMG jury rig not only allows traffic monitoring, says Wired, it also gives the user the ability “to stop the data from being passed to attackers from infected phones, alter it to feed the attackers false data, or pass commands back to the smart phone to remotely disable the malware.”

The researchers went a step further, releasing a paper describing their method that includes information so consumers can build the system as a DIY project.  for others to use to develop their own system.

Cybersecurity Expert Advocates Fighting Hackers With Hackers

“Large organizations are shooting themselves in the foot if they're not willing to hire a reformed computer hacker to aid with cyber security.” That’s the bottom line, at least according to Robert Hansen, director of product management for security firm WhiteHat Security. In an interview with Computing Magazine, Hansen goes on to say not only is shunning so-called black hat hackers a bad idea, but that many large businesses unknowingly employ them anyway.

"One guy I know who does training for military contractors, he lives in a state where they're not allowed to do background checks on people for whatever reason. But he's been to jail before, for hacking," Hansen told Computing.

"He's gone to jail for something and now he's teaching the best of the best how to defend against hackers and they're not allowed to ask the question if he's gone to jail or not. " Hansen, who regularly talks with black hats, reasons that, if a company is going to have people on the payroll who at one time or another went to prison for hacking or committed cybercrimes but weren’t caught, it’s better to do so know knowingly. "If you intentionally do it then at least it's on the table and they can do the things they need to do to help you [avoid becoming the victim of cybercrime]," he said.

In Other Cybercrime News…

A Chinese hacker gang infiltrated more than 100 companies, sat in on private teleconferences

Two providers of secure e-mail shut down rather than comply with secret U.S. government court orders for access to their customers' data

The Cybercrime of Things: Adding Internet connectivity to everything in your home means convenience. It also means greater vulnerability.

Photo: Getty Images

Queensland Government Bans IBM from IT Contracts

That didn’t take long.

A day after the Queensland Health Payroll System Commission of Inquiry delivered its 264-page blistering report (pdf) on the Queensland Health payroll system project cock-up, which I have been following for the past few years, Australia’s Queensland Premier Campbell Newman released a statement today in which he banned IBM from entering into “any new contracts with the State Government until it improves its governance and contracting practices.” This comes right on the heels of last week's news that Pennsylvania had opted out of renewing its contract with IBM to modernize the state's unemployment compensation computer system, and yesterday's news that a Credit Suisse analyst cut IBM's stock rating, saying that, “Organically, we believe IBM is effectively in decline.” It's starting to seem like a very bad month for IBM indeed.

Newman cited as a reason for the ban the commission report’s finding that the fiasco—which saw an effort to replace Queensland Health’s legacy payroll system at an expected cost of A$6.19 million (fixed price) explode into one that will cost around A$1.2 billion to develop and operate properly when all is said and done—“must take place in the front rank of failures in public administration in this country. It may be the worst.”

Maybe the scariest thing for Queensland taxpayers reading the report’s conclusion was that the commission wasn’t sure that this farce was the worst in Australian government’s history. Maybe it was unsure because the commission has so many worthy candidates to choose from.

Newman went on to say, “It appears that IBM took the state of Queensland for a ride.”

Newman’s statement warned that before IBM will be allowed to bid again it “must prove it has dealt with past misconduct and will prevent future misconduct.”

Naturally, IBM took exception to the report’s conclusion, saying it was at best only minimally responsible for the project blunder—a major admission for the company which has long insisted it “successfully delivered” what it was contracted for. An IBM spokesperson is quoted in an article today at the Delimiter as saying, “IBM cooperated fully with the Commission of Inquiry into Queensland Health Payroll, and while we will not discuss specifics of the report we do not accept many of these findings as they are contrary to the weight of evidence presented.”

Additionally, the IBM spokesperson stated, “As the prime contractor on a complex project IBM must accept some responsibility for the issues experienced when the system went live in 2010. However, as acknowledged by the Commission’s report, the successful delivery of the project was rendered near impossible by the State failing to properly articulate its requirements or commit to a fixed scope. IBM operated in a complex governance structure to deliver a technically sound system. When the system went live it was hindered primarily through business process and data migration issues outside of IBM’s contractual, and practical, control.”

“Reports that suggest that IBM is accountable for the $1.2 billion costs to remedy the Queensland Health payroll system are completely incorrect. IBM’s fees of $25.7 million accounted for less than 2 percent of the total amount. The balance of costs is made up of work streams which were never part of IBM’s scope.”

In other words, all that money that Queensland has had to spend on those "work streams" to get the payroll system to work correctly since we were kicked out ain’t on us.

IBM’s statement, however, studiously avoided tackling head-on a commission conclusion in the report that: “The only finding possible is that IBM should not have been appointed” to the contract in the first place in part because of “ethical transgressions” on the part of some of its employees, including “the obligation not to use the State’s confidential [bid and proposal] information” that it had somehow couldn't explain came into its possession from a restricted government database along with the apparent privileged insider information from a government consultant to the project who happened to be a former-IBM employee.

The report damningly states that IBM’s “conduct shows such disregard for the responsibilities of a tenderer, and a readiness to take advantage of the State’s lapse in security, as to make it untrustworthy.”

This is not to say that the government was blameless in the shamble, either. The commission report states that:

“a. the scoping of the system (ie its definition) was seriously deficient and remained highly unstable for the duration of the Project. That being so, and although the problem was firmly known to each party, no effective measures were taken to rectify the problem or to reset the Project;

b. the State, who would ultimately bear the risk of a dysfunctional payroll system, gave up several important opportunities to restore the Project to a stable footing and to ensure that the system of which it would ultimately take delivery was functional. [An expert] characterised the approach of both parties as being ‘Plan A or die’;

c. the decision to Go Live miscarried, both because it ought to have been obvious to those with responsibility for making that decision that the system would not be functional and because the decision to Go Live involved no proper and measured assessment of the true risks involved in doing so;

d. the system, when it went live, failed to function in a way in which any payroll system, even one which was interim and to have minimal functionality only, ought to have done.”

The report did “clear” former Premier Anna Bligh, former Health Minister Paul Lucas and former Public Works Minister Robert Schwarten of any wrong doing in the decisions they made during and after the screw-up because the three politicos were merely following the advice given to them by their senior public servants. However, the report did not endorse their decisions as necessarily being correct: it notes that, “Those who read the Report, and have an interest in good government, can judge for themselves” whether the decisions made by the former senior Queensland politicians were “improvident” or not.

The commission report makes for a jolly good read for those students of major government IT project catastrophes. I found nothing in it particularly surprising or what I haven't been writing about for years here at the Risk Factor, and the numerous recommendations for the government for avoiding misadventures in the future can be summed up as, “Don’t take on IT projects that you don’t have the professional competence or capability to ensure you aren’t being taken for a ride.”

Good advice that undoubtedly everyone will agree with but will also be studiously ignored in practice.

Photo: IBM logo/Wikimedia Commons, Denied stamp/iStockphoto

IT Hiccups of the Week: Port of New York and New Jersey’s Computer Problems are Virginia’s Gain

There was a clustering of IT-related problems, outages, and apologies last week in the government and banking spheres. However, we'll start off this week’s IT Hiccups review with a story of how a problem with a new terminal operating system at the Port of New York and New Jersey that has become a boon to the port's competitors at the Port of Virginia and elsewhere.

Port Authority of New York and New Jersey Tells Shippers to Come Back

It is hard to overestimate the importance (pdf) of an effective, fully-functioning terminal operating system (TOS) to a port’s attractiveness to shippers, and therefore, to the port’s profitability. And when a TOS doesn’t work well, it doesn’t take long for shippers to decide to go somewhere else, as the Port Authority of New York and New Jersey—the busiest on the East Coast despite still being in recovery from the ravages of Hurricane Sandy—is finding out to its dismay. The Port of Virginia, among others, is delighting in the shift.

As part of a US $3 billion investment by the Port Authority and New Jersey, Maher Terminals, one of the largest multi user container terminal operators in the world, began on 20 April to upgrade its Navis TOS over several phases at Maher’s Elizabeth, N.J., facility. The project's planners originally expected that the upgrade would be completed by 8 June. The early phases of the upgrade seemed to go well, but the final phase has proved troublesome.

According to a joint Maher-Navis statement issued on 20 June, the system’s “operations has encountered some unexpected issues” which “have led to delays.” The companies stated that they were committing “all available resources to identify and resolve” the technical issues involved, and expected the issues to be resolved shortly.

The statement also said, “With the implementation of new systems, there is always a risk of initial declines in productivity as new operating procedures and processes are streamlined into the operation.” This was a hint that the companies saw that some of the operational problems were caused by the shippers themselves, and to ensure the hint wasn’t missed, the statement added, “Noticeable improvements are already being realized as users adjust to new systems and processes.”

Apparently the “blame the customer” excuse for the port's problems didn’t go over very well, especially as the TOS-related issues continued for not days, but for several weeks. As a result, the Port of Virginia as well as other East Coast ports have seen an increase in their container traffic, a story in the Hampton Roads, Virginia, paper Daily Press reported last week. The paper said that Hapag-Lloyd, a German shipping line that operates150 ships, even went so far as to urge “its rail and local cargo customers on July 26 to seek out alternative ports” and also told its customers that the issues in NY/NJ were being made worse by a labor and trucking shortage.  

The Port of Virginia successfully migrated to the Navis SPARCS N4, the same TOS that the Port Authority of New York and New Jersey is having trouble with, last year.

A story in today’s Wall Street Journal adds insights into the problems being encountered at the Port of New York and New Jersey.  It says that “some trucks endured waits of an extra four to five hours for routine jobs that require one hour on a good day” and that “ships have been diverted to nearby terminals in New Jersey and Staten Island, causing more delays.” Thousands of containers worth millions of dollars are still stuck at the port, many of which contain holiday goods that are stores are anxious to receive.

One trucking company executive was quoted in the Journal as saying, “There's no word for it other than ‘hell’… I've been in business thirtysome-odd years, and this is the most stressful time I've had.”

A joint Maher-Navis statement last week said the companies had “determined that the real-time interactions between the various [TOS] system components deployed in the container yard were not operating as designed” and that they created a temporary fix: certain automated features were turned off and would be phased back in “on a controlled basis.” When they would be phased in, the companies didn't say. The companies also claimed that service had returned to “acceptable levels during the past several weeks, albeit at reduced volume.”

In the June joint-statement, the companies stated that they had expected the Port of New York and New Jersey to be returning to its previous “exceptional performance” soon. Somehow, I don’t think shippers will settle for the currently touted “acceptable” service; other Port Authorities such as those in Virginia, Baltimore, and Boston will likely keep reminding the shippers that their ports aren’t having any problems.

There is a neat real-time animation of a TOS in action at the Bremerhaven port for those interested.

Banking IT Systems Have a Bad Week

Gross understatement alert: Banking customers took it on the chin last week.

We start off with reports today that Westpac Bank customers in Australia and New Zealand are having trouble with their online and mobile banking services for a second straight week. Last Thursday and Friday, the bank apologized for intermittent connectivity problems, while today it apologized for intermittent slowdowns. The bank said last week that it didn’t know what the cause of the problems was: it hadn’t performed any updates and it wasn’t experiencing a cyber-attack, two of the usual culprits. Westpac said it would ensure that no one was out any bank fees because of the problems.

Also in Australia, the National Australia Bank apologized today to customers who were experiencing connectivity issues with its online and mobile banking services. It too denied experiencing a cyberattack, but didn’t give any further explanation for its problems. NAB, which is in the midst of a 10-year IT system modernization effort, suffered another major system issue a few weeks ago.

NAB suffered a separate black eye, but in the U.K. A week ago, customers of Clydesdale and Yorkshire banks, which are owned by NAB, discovered that they were unable to access the banks’ online services. The banks originally said it had nothing to do with their IT systems, and urged customers to check with their local ISPs. However, it soon turned out that NAB had failed to renew its domain names on time, a story at Computer World UK reported.  Interestingly, the banks haven’t admitted that this oversight was the cause of their customers’ access problems, instead reiterating the previous excuse, Computer World stated.

Also in the UK, the Telegraph reported that payment processor Streamline, which the Telegraph stated “handles half of all face-to-face card transactions in Britain,” had trouble processing payments made between 26 and 28 July. As a result, an unknown number of customers may “have not received settlement for transactions processed” with their banks over those days.  “We apologise for any inconvenience caused,” the processor said, although it didn’t promise to ensure any bank fees caused by the settlement issues would be covered.

Finally, in Vermont, the State Treasurer’s Office said that a processing problem at TD Bank, which is the state’s bank of record, would be delaying by a day electronic payment of retirement checks to 7059 retirees of the Vermont State Teachers’ Retirement System. TD Bank said that it would cover any late charges, overdrafts or any financial penalties assessed caused by the delay. TD Bank said it “apologizes for the inconvenience.”

US Government IT Systems Have Another Bad Week

A few weeks ago, I wrote about a number of U.S. state governments that suffered serious IT meltdowns. Last week, there was more of the same.

We start off with a mainframe system upgrade a week ago Sunday at New York State’s Department of Motor Vehicles that didn’t go as planned. A hiccup ended up taking the department’s computer system offline for several hours on Monday morning. A story at the Times Union quoted a DMV official as saying that, “Although the system was fully tested after the upgrade was completed, an issue presented itself under this morning's high transaction volumes that was not detected during testing.

Even after the DMV’s system came back up, it was said to be operating very slowly. It wasn't until Tuesday that the system resumed normal operation.

Also on Monday, a network connectivity problem prevented the Texas Department of Motor Vehicles from registering or titling vehicles, a Star-Telegram story reported. That problem was also fixed on Tuesday.

On Tuesday, a problem with a mainframe computer in Georgia “caused the computer systems at [the Department of Driver Services], the Department of Revenue, the Department of Human Services and the Office of the Secretary of State to crash,” WGCL, CBS television's Atlanta affiliate, reported. The effects from the crash were still being felt on Wednesday. The cause of the problem was not disclosed.

Then on Thursday, the city of Alamogordo, New Mexico, said that all of the city’s e-mail and scheduled appointments between 19 and 31 July were apparently lost “while technicians were transitioning from one email system to another,” the Alamogordo Daily News reported. In addition, city employees can’t currently access their e-mail or their online calendars because of the “glitch.” The city said it has a backup system, but it also failed. There is no date being given when the situation will be rectified.

Also on Thursday, the computer system used by Nevada's Welfare System was reportedly out for the entire day. No reason was provided for the outage, other than that it was an “internal” issue, whatever that means.

Finally, on Friday, the computer system supporting all Oklahoma state agencies, including telephone services, went down because of a power failure. The system was successfully rebooted Friday night, but child support payments made by the Oklahoma Department of Human Services are going to be delayed by about a week because of the outage.

And the reason for the power outage? Apparently, “someone inadvertently hit the emergency shut off system.” Who that “someone” is has not yet been revealed.

Also of Interest…

Victoria Australia 000 Emergency System Goes Down for the Fifth Time in Six Weeks

Fat Finger Error Adds US$18 Billion to Kemper Corporation Capitalization

Glitch at M&G Accounting, UK's Biggest Bond Fund Manager, Hits 32 000 Investors

Indiana ISTEP Testing Glitches Didn’t Hurt Much (We Think)

Photo: Captain Albert E. Theberge/NOAA

This Week in Cybercrime: Black Hat USA 2013 Uncovers a Bevy of Exploits

Spy Chief Addresses Hacker Nation

The highlight of this week in cybercrime was the Black Hat USA 2013 conference that took place in Las Vegas. Though dozens of cybersecurity researchers showed up to alert the world to the wide-ranging vulnerabilities that could be exploited by cybercriminals, the top story was the appearance of Gen. Keith Alexander, director of the National Security Agency and chief of U.S. Cyber Command. Alexander was booked to deliver the gathering’s opening keynote address well before Edward Snowden’s revelation’s about the NSA’s PRISM program for collecting phone call metadata. So there was much speculation about whether Alexander would show up, whether he should, and what type of reception he would receive. In video of the talk, recorded by Kaspersky Lab’s Threatpost, the audience,

“was initially cordial and attentive, but soon turned somewhat restive and hostile. While Alexander defended the NSA’s intelligence-gathering efforts and provided examples of how they had led to the disruption of terror attacks in recent years, some people in the audience were uninterested and shouted criticisms and accusations at him.”

What a nice way to get the party started.

Read More

Pennsylvania Won’t Renew IBM's Contract for Botched Project

Pennsylvania’s Labor and Industry Secretary Julia Hearthway announced Wednesday that the state has decided not to renew its contract with IBM to modernize the state’s 40-year-old unemployment compensation computer system. According to an AP report, the contract, which was awarded in 2006 and is set to expire in 2013, is currently 42 months late and over $60 million above its original contract amount of $106.9 million.

IBM's Fred Brooks once famously wrote in his 1975 groundbreaking IT project management book The Mythical Man-Month that IT projects become a year late one day at a time. Of course, Brooks meant it as a warning, not as a goal, something that Brooks may want to explain to his old company.

The state decided to end the contract after a US$800 000 assessment of the effort by the Software Engineering Institute (SEI) indicated that the critical objective of the modernization effort, namely a demonstrated “capability to reliably, consistently, and accurately process unemployment claims, calculate payments, and enable payment to eligible citizens who are out of work” doesn’t yet exist, and apparently, there is no agreement when, if ever, such a capability will ever exist.

Read More

Researchers Hack Into Car Immobilizers, But Can’t Say How They Did It

Where do you draw the line between deciding what people need to know and what should be kept out of the wrong hands? It’s never been easy. The Guardian broke a story about three computer scientists who tried to publish a paper analyzing a faulty algorithm that could let criminals steal cars—that is, before the English High Court of Justice stepped in and issued a provisional ban.

Flavio Garcia, a University of Birmingham computer science lecturer, decoded the algorithm that allows the engine immobilizer to verify the authenticity of a car key. He had hoped to publish his findings in a paper called “Decoding Megamos Crypto: Wirelessly Lock-picking a Vehicle Immobilizer,” at a Usenix Security Symposium next month in Washington, D.C. But Volkswagen and the creators of the algorithm, the French security company Thales, were none too happy about this.  

Megamos Crypto is a system that uses RFID (radio-frequency identification) to disable a feature that prevents the engine from starting. The crypto algorithm sends a signal from the car key to the engine immobilizer.

Read More

IT Hiccups of the Week: Ohio Bank Blames GPS for Wrong House Repossession

It has been another busy week in the land of IT-related snarls, malfunctions and snafus. We start off this week’s edition with a bank which blamed a house repossession error on bad GPS coordinates.

Bank Repo Crew Repossesses Wrong House

Katie Barnett of McArthur, Ohio, returned to her house after a two week vacation to find that the locks on her doors had been changed. Fearing that a squatter had tried to take over her home while she was away, Barnett broke in only to find that all her possessions were gone, says a story in the Daily Mail.

What Barnett soon found out was that a repo crew sent by First National Bank of Wellston, Ohio, had mistaken her house for the bank-owned home across the street, even though her house number is 514 and the one across the street is numbered 509.

According to an ABC News story, the bank’s president and CEO Anthony Thorne put out a statement (pdf) on the bank's website that blamed the problem on a GPS error. Thorne’s statement says that the GPS locator led the bank's “representatives” to the “wrong home, which was located on the same street as the target property (we have since retraced their route using the same GPS, and it again took us to the same wrong location).”

The representatives, Thorne claims, “noted that the grass was overgrown, the door was unlocked, and the utilities had been turned off. The home was also nearly empty, with two dressers being the only furniture inside the premises, and a neighbor indicated that the home had been vacant for some time.” So they apparently assumed that even though the house number was wrong, they were at the right place.

Thorne stated, “This situation was a mistake on the part of our bank and—as we have done previously—we sincerely apologize to the homeowner for the inconvenience and concern it may have caused. In addition, we communicated to the homeowner our desire to compensate her fairly and equitably for her inconvenience and loss.”

There seems to be a disagreement, however, over the amount of compensation owed. Barnett provided the bank with an itemized list of what she claimed was lost, which she said amounted to US $18 000 of personal property. However, the bank disagrees that the two dressers and other personal property it says its representatives discovered in the house were worth that much. The bank wants Barnett to come up with receipts for what she claims the representatives removed, and then the bank will decide how much to pay based on a fair market valuation. Exactly how Barnett is supposed to come up receipts that would be likely be in the “trash” that the bank’s representatives admittedly threw out wasn’t explained by the bank in it's statement.

Barnett is now planning a lawsuit against the bank.

Technology, Equipment Problems Blamed for 1 in 4 Surgical Errors

The prospect of undergoing a surgical procedure is always likely to cause anxiety. A research study published last week in the UK journal BMJ Quality & Safety will probably only make going under the knife even more anxiety inducing. According to the study of surgical technology and operating-room safety failures, nearly one-quarter (23.5 percent) of all surgical suite errors can be attributed to “failures of equipment/technology.”

The research study, which examined published studies on surgical safety failures, discovered that “an average of 2.4 errors was recorded for each procedure, although this figure rose to 15.5 when an independent observer recorded the errors,” a story at Science Daily noted.

Of the failures categorized as being related to equipment issues, 37 percent were attributed to equipment availability, 44 percent to problems with equipment configurations and setting, and 33 percent to device malfunctions. It is unclear how many device malfunctions are the direct result of software problems, or if equipment issues also involve operator error—which has been a concern recently with robotic surgery.

A story at LiveScience indicated that medical errors affect around 15 percent of patients, with half of these resulting in adverse events. Not surprisingly, LiveScience noted that the BMJ Quality & Safety study found “operations that rely heavily on technology, such as heart surgeries, had higher rates of equipment problems than did general surgeries.”

Using pre-operative checklists, standardizing the use of briefing tools, and improving staff training reduced error rates by half, the study found.

In addition, a blog post in the New York Times last week reported on another interesting research study (pdf) to be published in next month’s issue of the Mayo Clinic Proceedings; the paper found that “more than 40 percent of established practices studied were found to be ineffective or harmful, 38 percent beneficial, and the remaining 22 percent unknown.”

I can hardly wait until the BMJ Quality & Safety and Mayo Clinic Proceedings studies are compared to see how many equipment/technology surgical safety errors happen in conjunction with ineffective or harmful medical procedures.

Gold’s Gym Customers Hit By Billing Error

While not as traumatic as having your house repossessed, some customers of a Gold’s Gym in Palm Springs, Calif., were less than amused when they found out that a “computer glitch” in the electronic billing system the gym uses charged “thousands of dollars” to their credit card accounts early last week, a story at the Desert Sun reported. According to the Sun’s story, numerous members found they were charged anywhere from US $1500 to $7000 for their monthly membership fees. As a result, many of those gym members found their credit cards and for some, their bank accounts, reportedly overdrawn.

The software error, which originated at third-party Ohio-based credit card billing company Vanitiv, was rectified within a day. Vantiv said no money actually was charged; only a “preauthorization” charge (a hold) was put on the credit cards. Of course, the effect for the cardholder when trying to access his or her money is basically the same. Gold’s Gym says it will refund any bank fees incurred by customers due to the error.

Gold’s Gym wasn’t the only gym hit by Vantiv’s error. According to a story at television station WRAL in Raleigh, N.C., some members of Fitness Connections in that city discovered that credit and debit cards with which they paid their monthly fees had been hit with “preauthorization” charges in the thousands of dollars. The WRAL story says the problem was caused by Vantiv’s billing system dropping a decimal point which turned a US $29.90 monthly membership charge into a $2990 one instead.

Fitness Connection said it would also refund any customer bank fees incurred as a result.

Also of Interest…

New York City’s Emergency 911 System Again Crashes Multiple Times

Las Vegas' McCarran International Airport Network Crashes, Severely Delaying Passenger Check-in

Arizona Motor Vehicle Department Sees Uptick in Computer Problems

New Jersey Motor Vehicle Commission Suffers Computer Problems for Second Time in a Week

"X Factor New Zealand" Fans Outraged at Voting Problems

Lincoln City, Oregon Sends Out Incorrect Water Bills

Louisiana Department of Revenue Sends Out Erroneous Tax Notices

Image: iStockphoto

This Week in Cybercrime: Online Bank Heists Just the Latest in a Long String

Late last month, I began an edition of This Week in Cybercrime by noting that, “The idea that cybercrimes are the work of miscreants or gangs of hackers picking targets at random is outmoded. Analysts now see a mature industry with an underground economy based on the development and distribution of ever more sophisticated tools for theft or wreaking havoc.” That updated thinking was backed up by a report released a few days earlier by researchers at 41st Parameter, a fraud detection and prevention firm.

Further reinforcement came this week when U.S. federal prosecutors filed charges against five people for orchestrating what is said to be the largest hacking/data breach/bank robbery case ever reported. “The defendants and their co-conspirators penetrated the secure computer networks of several of the largest payment processing companies, retailers and financial institutions in the world, and stole the personal identifying information of others, such as user names and passwords,” prosecutors said. The crew of cybercrooks netted at least 160 million credit and debit card numbers. Let that number sink in for a second.

The estimated financial losses stemming from the thefts reach into the hundreds of millions of dollars.

Of the five defendants—Vladimir Drinkman, Aleksandr Kalinin, Roman Kotov, and Dmitriy Smilianets of Russia, and Mikhail Rytikov of Ukraine—only two, Drinkman and Smilianets, are in custody. The other three are on the run, which is unfortunate, because Kalinin has been prolific in his efforts to pick financial institutions clean. According to the prosecutors, from December 2005 through November 2008, Kalinin and a separate co-conspirator hacked into Citibank and PNC Bank computer systems and stole account information they subsequently used to withdraw millions of dollars from victims’ bank accounts.

For example, in January 2006, they launched a cyberattack on PNC Bank’s online banking site and walked away with hundreds of the personal identification numbers associated customers’ ATM cards. Kalinin’s partner in crime turned the data over to associates who used it to encode blank ATM cards and withdraw $1.3 million from victims’ accounts. In 2007, Kalinin repeated the feat, that time going after the network of a firm that processed ATM transactions for Citibank and other banks. He stole PINs for half a million bank accounts, including those of 100 000 Citibank customers. He and his co-conspirators used the spoofed ATM cards they created to rob Citibank—in broad daylight, without guns or masks—to the tune of $2.9 million. A year later, Citbank was again in the online bank robbers’ crosshairs. Amid a cyberattack against the bank’s website, they stole 300 000 account holders’ information. This time the haul from faked ATM cards was $3.6 million.

In a separate case, for which Kalinin was also charged on Thursday, prosecutors allege that between 2007 and the fall of 2010, malware he placed on servers used by the Nasdaq financial exchange allowed him to incrementally elevate his level of administrative access until the point, in January 2008, that he had unfettered control. He marked the occasion with a jubilant instant message: "NASDAQ is owned." The government alleges that the 26-year-old native of St. Petersburg, Russia, got his foot into the door of Nasdaq's systems when he noticed a small vulnerability on its website.

Prosecutors named 16 separate corporate victims of the Russian and Ukrainian cyberthieves’ reign of terror. Most of the damage was done in breaches of systems at: Heartland Payment Systems Inc., a credit and debit card payment processor that had 130 million card numbers stolen from its databases; Commidea Ltd., a European electronic payment processor for retailers, that had 30 million card numbers stolen; and Euronet, a Leawood, Kansas–based electronic payment processor that lost roughly 2 million card numbers to malware. Other notable names that were targeted include Visa, Discover, Dow Jones, and J.C. Penney.

The alarm with which the gang of five’s activities was reported is itself startling. It’s not as if a certain blog doesn’t update readers on what happens each week in the world of cybercrime. And it’s not as if, in 2013 alone, This Week in Cybercrime hasn’t highlighted several other techno bank robberies, in the process making it clear that financial institutions are increasingly vulnerable.

In May, for example, we reported that seven people were arrested for coordinating 40 000 fraudulent cash machine withdrawals in 27 countries (with a total haul of $45 million) within hours after they hacked into the servers of credit card processors in the United Arab Emirates and Oman. And in June, we noted that U.S. prosecutors charged a gang of Ukranian cyberthieves with stealing account information from 15 different payment processors, banks, and online brokers and using it to transfer funds to prepaid debit cards. After the transfers, they would subsequently have roving teams of “cashers” hit ATMs to withdraw cash or make purchases with the ill-gotten loot.

The banks should do a better job at securing their networks, you say? That may not be possible, says a report from the FireEye Malware Intelligence Lab, that we discussed back in April. The report, on advanced persistent threats [pdf], found that some companies (banks fit the description) have to fend off attacks as often as once every three minutes. It’s not realistic to expect them to stand up to the growing volume and increasing sophistication of digital assaults. "This nearly continuous rate of attacks and activities is indicative of a fundamental reality: these attacks are working, yielding dividends," said the report. An Ars Technica article describing the Nasdaq hack notes that,

"The indictments give a birds' eye view of the patience and meticulousness hackers employ when penetrating some of the world's most well-fortified networks."

Photo: Tim Robberts/Getty Images

IT Hiccups of the Week: U.S. State Government IT System Meltdowns Galore

After a couple of quiet weeks, IT related snafus, snarls and ooftas reappeared with a vengeance last week. We start off with several U.S. state governments’ IT systems that have had better weeks.

Oregon, New Mexico, Kansas, North Carolina, New Jersey and Iowa Experience IT Problems

Last Monday evening at 7 p.m. Pacific time, government contractor Hitachi initiated a planned hardware upgrade to add storage capacity to Oregon’s State Data Center (SDC) computer systems located in Salem, Ore. The work, according to Matt Shelby, spokesman for the Department of Administrative Services, “was not supposed to cause any disruptions,” a story at the Oregonian reported. However, Shelby told the paper, “During the course of that work, a catastrophic failure and major connectivity issues arose.”

As a result, 90 state agencies were unable to connect with the data center and thus to each other. The outage knocked out a range of services including Oregon’s Department of Transportation TripCheck road cameras, government e-mail and websites, and the processing of some 70 000 state unemployment checks. The hardware problem was fixed Tuesday morning, and state services were restored by midday Tuesday. 

Next, New Mexico continues to have trouble with its new unemployment computer system as well, an AP story from last week reports. An audit (pdf) of the $48 million system—which was originally supposed to cost half that—was released last week by the state's Legislative Finance Committee. The report stated that the new system, which went into operation in January, “is complex and can be difficult for users to navigate.” According to the AP story, businesses and individuals who've used the system say that the phrase “can be difficult” should be changed to “is exceedingly difficult”—so much so that a New Mexico state senator said that the state should consider going “straight back to paper because the system doesn’t work.”

The head of the state agency responsible for the system, Workforce Solutions Secretary Celina Bussey, told the AP that she considers the unemployment system implementation a success, but concedes that maybe the system interface could be more user-friendly. That may be a bit of an understatement: for example, it previously took unemployed workers 15-minutes to make a claim; now, with the new system, it takes up to an hour, the audit reported. Unemployed workers calling the state's help lines with their applications face long telephone wait times as well, the audit states.

In addition, the audit reports that the unemployment system suffers from “data conversion defects, limited application testing and the lack of a contingency and disaster recovery plan,” the latter of which it says creates “conditions of risk.” In other words, don't be surprised if the new unemployment system suddenly keels over.

Moving on to Wichita, Kansas, where television station KWCH reported last week that the unemployment system in Kansas has been having problems of its own. According to the story, the Kansas Department of Labor “was upgrading its servers when it discovered a software problem” that caused a delay in the issuing of an unknown number of unemployment checks for over a week.  On top of not being able to tell how many Kansans were affected by the software problem, the Department of Labor didn't offer an explanation on why it kept the information about the problem quiet until the television station made inquiries about it.

North Carolina got a double dose of government IT system problems last week. First, the state’s new family assistance system NC FAST (North Carolina Families Accessing Services through Technology) reportedly experienced difficulties, according to a story Fox News Channel WGHP in Highpoint, North Carolina. The story reported that, “several Department of Social Services [offices] across the state [were] reporting glitches” that were keeping families from receiving food stamps. The state said it was trying to determine what was causing the problems and fix them.

North Carolina also continued to have problems with its new, expensive and controversial computerized Medicaid billing system NCTracks, which went live on 1 July. Some businesses have been claiming the new system has been a nightmare to deal with. I'll be writing more about the issues with NCTracks later this week.

Then on Friday, the New Jersey Motor Vehicle Commission’s computer system was offline for the entire day. According to a story at the New Jersey Journal, a “fire alarm was activated at one of the state’s data centers, causing the state’s website, including the MVC division, to shut down automatically” at about 2 a.m. Friday morning. While data center technicians were able to get most other state agencies back online by mid-morning Friday, they weren't successful in getting the MVC and “one section of the Department of Labor and Workforce Development” up and running, an article at the Record reported. MVC offices stayed open an extra hour on Saturday to compensate for Friday's outage.

Finally, last Friday night, a “faulty piece of equipment on the Iowa Communications Network, the state-owned fiber-optic system,” caused emergency 911 calls made from cell phones across Iowa to be “routed to out-of-town” call centers, a story at the Des Moines Register reported. An AP story stated that “a vendor tried to install new software to fix the problem, but that made it worse.” In addition, the “backup system also failed to activate,” the AP stated.  The routing problem was finally fixed late Saturday morning.

PayPal: What's $92 Quadrillion Between Friends?

Back in 2007, Joe Martins closed a bank account at Wachovia Bank and got a letter inquiring about when he was going to pay off his US $211,010,028,257,303.00 outstanding balance.  Well, if that happened to Chris Reynolds, it would not be a problem.

According to a story last week at the Philadelphia Inquirer, thanks to the generosity of PayPal, Reynolds was worth $92,233,720,368,547,800.00. Heck, even after paying off Martins’ $211 trillion debt, Reynolds would still have a nice $92 quadrillion and change left over to maybe buy himself a nice country or three.

Alas, it was all a mistake. PayPal told Reynolds that his real account balance was a measly $0.00, but it would “donate an unspecified amount of money to a cause of Reynolds' choice,” as a way to make up for the error, CNN reported.

Texas Lubbock Power & Light Apologizes for Billing Error

While Reynolds and the folks at PayPal were having a good laugh at the quadrillion dollar accounting mistake, customers of Lubbock Power & Light were less amused at the billing error they were told about last week.

According to the Lubbock Avalanche-Journal, Lubbock Power & Light officials held a news conference last Friday where they confirmed that there was a software “billing glitch” with its new customer billing system. The error meant some 44 000 customers were undercharged for their June electric bills, and the difference was applied to their July bills. The simultaneous addition of a 9.7 percent rate hike that went into effect on 1 June, as well as an increase in LP&P’s standard service charge, caused the July bills to, in the words of the utility's spokespersons, “appear disproportionately high.”

LP&P apologized for the “unfortunate and regrettable” billing error and its “communication errors” in not informing customers about the problem before the higher bills started to arrive in customer mailboxes.

Also of Interest…

18 700 University of Virginia Students’ Social Security Numbers Printed on Brochure Address Labels

Computer Issues Delay X-ray and Scan Results for Weeks at Hospitals Across Kent, England

30 000 Ireland Welfare Payments Delayed by Computer Problems

Google Accidentally Makes Scotland's Jura Island Invisible

Another National Australia Bank Computer Problem Riles Customers

Taiwan Stock Exchange Experiences Update Problems

Virgin Australia Ticketing System Suffers Outage

Illustration: iStockphoto


Risk Factor

IEEE Spectrum's risk analysis blog, featuring daily news, updates and analysis on computing and IT projects, software and systems failures, successes and innovations, security threats, and more.

Robert Charette
Spotsylvania, Va.
Willie D. Jones
New York City
Load More