Risk Factor iconRisk Factor

Waiting For REAL ID? Take a Seat, It'll Be a While

There's an interesting story in next month’s National Defense magazine on the long gestation of the REAL ID Act.

As you may remember, eight years ago the U.S. Congress passed the REAL ID Act of 2005. It would have forced states to start issuing tamper-proof driver licenses and identify cards by 11 May 2008. The reason for the act, a brainchild of Congressman Jim Sensenbrenner of Wisconsin, was to make it harder for terrorists and other criminals to be able to pass off fake IDs in the commission of their crimes. And a REAL ID card would be required to enter a federal building or board a commercial airline flight.

After an outcry from state governors over the projected cost—upwards to US $12 billion they claimed—and from privacy advocates over this creation of a de facto national identity card, the Department of Homeland Security (DHS) decided in March 2007 to move the act's compliance date to December 2009. Then, in January 2008, DHS decided again to postpone the deadline for states to the 11 May 2011 and also changed some of the documentation requirements needed to get a REAL ID in hopes of quieting the critics. DHS estimated then that the states’ implementation costs would not be any greater than $3.9 billion, which DHS would help cover with $280 million in state grants.

After continued grumbling by the states about the cost, and some two dozen state legislatures passing laws or resolutions refusing to comply with the REAL ID requirements, in March 2011, DHS postponed the compliance deadline yet again, this time to 15 January 2013. And then, as this deadline approached and with most states still in non-compliance, late last month DHS for the fourth time delayed the compliance deadline. It will apparently be to sometime in 2015; the department won't announce the exact date until later this year.

A DHS press release announcing this latest delay praised the 13 states that it says have met REAL ID standards: Colorado, Connecticut, Delaware, Georgia, Iowa, Indiana, Maryland, Ohio, South Dakota, Tennessee, West Virginia, Wisconsin, and Wyoming. However, as the National Defense magazine article points out, the Real ID act requires that there exist “five different national databases for states to tap into to verify identities [but those] are not up and running.”

Hmm, I guess meeting the REAL ID standard all depends what you mean by “standard.” Or maybe the word "is."

In addition, as noted in an acerbic post at the CATO Institute, there are pretty good odds that the states who haven’t complied with the REAL ID act will probably never have to, making suckers out of those that did. As CATO points out, it is highly unlikely that the federal government is going to tell the citizens of 37 states they can’t fly in planes or enter federal buildings. How will federal judges feel about all those empty jury boxes? As it happens, I've been called to federal court jury duty next week, in a state that doesn’t meet the REAL ID act.

The pleasure of watching the endless tug of war over federal (unfunded) mandates versus states’ rights exposed by the REAL ID act is compounded by the risible and ever-changing cost estimates to the states of implementing it. Sensenbrenner originally estimated (i.e., pulled out of the air if not another place) that the cost to change state department of motor vehicle computer systems would be about $2 million per state over 5 years, or $100 million overall. The Congressional Budget Office, sharing the same fantasy, generally concurred, estimating that it would be closer to $120 million over the 5 years total.

However, a  2006 study by the  National Conference of State Legislatures (NCSL), the National Governors Association (NGA), and the American Association of Motor Vehicle Administrators (AAMVA) said that Sensenbrenner and the CBO were way off, and did not account for the vast majority of costs that would be incurred. This group estimated that the REAL ID act could cost states more than $11 billion over five years (pdf).

That number was thought to be way off until the DHS admitted in March 2007 that its own estimates of the REAL ID act implementation costs would be from $10.7 billion to $14.6 billion—with another $7.8 billion or so in costs borne by individuals in fees—over ten years.

After its January 2008 changes to the REAL ID requirements, DHS revised its own estimate and claimed that compliance would now cost the states a mere $3.9 billion or so over ten years.  In 2011, the Center for Immigration Studies, an advocate for REAL ID, estimated the cost  to the states would be even less: somewhere  between $350 million and $750 million. That seems remarkably low, given that DHS has said that it has already awarded $263 million in grants from FY 2008 to FY 2011 to states to help them meet the REAL ID requirements—and three-fourths of the states aren't done yet.

Exactly how much money the states have spent so far on top of this DHS grant amount is unknown, but even the DHS knows that meeting the Real ID “standards” aren't cheap. One reason for the latest delay, DHS says, is that “in a period of declining state revenues,” the states are having a hard time finding the money to implement the act's requirements.

My guess is that the DHS will continue to set Real ID compliance deadlines only to postpone them at the last moment, and hope that over time, the vast majority of states will ultimately albeit grudgingly implement REAL ID as they eventually replace their DMV legacy systems. Me, I'm rooting for some genuine enforcement of compliance by 2015. I probably wouldn't ever have to report for federal jury duty again.

IT Hiccups of the Week: I Don’t Have Your Cellphone, Honest

We start off this week’s potpourri of IT–related snafus and snarls with an unusual one from North Las Vegas.

The Case of the Missing Sprint Cellphones

According to a story in the Las Vegas Review-Journal, since 2011, people keep showing up at Wayne Dobson’s house demanding that he return their lost or stolen Sprint cellphones. Police also have shown up demanding entrance after being sent to his house on suspicion of domestic violence because of calls 911 operators received from Sprint cell phones. The only trouble is that Hobson, who lives alone, doesn’t have any of the phones.

The Review-Journal cited telecom experts who speculated the problem might be an intermittent error in a Sprint’s local switchboard software that is used to determine the GPS coordinates of its cell phones. As a resul, they say, some owners of lost or stolen Sprint cell phones, as well as the police, are being directed to Dobson’s house by mistake.

Dobson, who has been awakened at all hours of the night by both the police and irate cellphone owners demanding he return their cellphones, is not amused. He has posted a sign on his house saying that he doesn’t have any lost or stolen cellphones, but that isn’t likely to deter someone who thinks their phone is at his house. It definitely is not going to deter the police, who although aware of the glitch, say that if they get a 911 domestic violence call, “they will still send officers to the scene unless they can confirm that there isn’t actually a problem there.”

Sprint told the Review-Journal last week that it “will research the issue thoroughly and try to get to the bottom of what is going on and if it has anything to do with our company.”

And according to a story today at the Review-Journal, Sprint says it has indeed gotten to bottom of the problem. There isn’t any error on our part, Sprint told the paper; the issue is a result of people who don’t understand “the inaccuracy of cellphone location software.”

Sprint told the paper that, “Location search results … are intended to be interpreted as anywhere within a several-hundred-meter-wide circular area - not the center point of the circle itself.”

I think that's news to most people.

Sprint went on to tell the Review-Journal that it can help the police understand when there is inaccurate location information coming from their cellphones, but “as for private citizens who use the technology to track their lost or stolen cellphones, there's nothing the company can do beyond educating them,” Sprint said. In other words, Dobson may still receive knocks on his door at all times of night.

Sprint's statement somewhat begs the question of what "inaccurate location information" means - being anywhere within a several-hundred-meter-wide circular area seems pretty inaccurate to me to begin with. Does Sprint mean that if the circular area displayed to the police is a several-thousand-meter-wide circular area it will help reduce it to a several-hundred-meter-wide circular area?

Sprint also told the paper, “We sincerely regret the inconvenience experienced by Mr. Dobson."

The Review-Journal found that Dobson’s experience is not unique. According to the paper, the same "knock at the door" has happened to folks living in New Orleans, Louisiana, Decatur, Georgia and San Antonio, Texas, all involving Sprint phones.

I wonder if Sprint sincerely regrets the inconvenience experienced by them, too.

Navy Minesweeper Runs Aground: Digital Map Error May Be Involved

The Defense News reported over the weekend that the minesweeper USS Guardian which ran hard aground on 17 January and remains stuck on a reef within the protected Tubbataha Reefs Natural Park in Philippine waters may have been following a digital navigation map that “misplaced the location of a reef by about eight nautical miles.”  As a result of the grounding, U.S. Navy ships have been ordered “to operate with caution when using [National Geospatial-Intelligence Agency]-supplied Coastal Digital Nautical Charts due to an identified error in the accuracy of charting in the Sulu Sea.”

The U.S. Navy is currently trying to minimize the damage to the reef, which is in a Unesco World Heritage restricted zone.  So far there have been no reports of fuel or oil leaks from the ship, although the ship is reportedly taking on water.  However, the U.S. Navy can expect to pay heavy fines any damage caused to the reef.

A few years ago, the British Maritime Accident Investigation Branch (MAIB) issued a warning to commercial ship operators about the dangers of relying too much on electronic navigation charts.

New Computer System Confuses Paternity

A story at The Tribune-Democrat last week reported that “the Division of Vital Records at the [Pennsylvania] Department of Health, which was switching to a new computer system” had sent out official birth certificates to 500 families that incorrectly listed the name of the father.

According to the story, the names are correct on the state’s main computer system, but when the Division of Vital Records “went to print out the new birth certificates, data for the father's first and last names were pulled from the wrong fields, which caused the documents to be filled out incorrectly.”

The state is telling those families that received the incorrectly printed birth certificates to send them back and they will get new ones.

And as far as I can tell from looking through the various news reports, the Division of Vital Records spokesperson didn’t bother with expressing “it regrets the inconvenience” tagline. How refreshingly honest.

This Week in Cybercrime: Hackers Build Better Mousetraps

U.S. Military Wants Ability to Jump Air Gaps, Attack Isolated Systems

According to a 15 January report by Defense News, the U.S. Army is looking to create sophisticated new techniques in cyberwarfare that solve a problem created by a well-known moment of success. It is looking for a way to remotely penetrate the defenses of industrial control systems—even if they are supposedly isolated from the Internet by so-called air gaps. Stuxnet, a cyberwarfare tool unleashed by the United States and Israel, used multiple zero-day exploits to inject malicious code that caused centrifuges at Iran’s Natanz nuclear enrichment facility to spin out of control. But it wouldn’t have gotten in the door if someone hadn’t carried it in on a USB flash drive. In the wake of revelations about the cyberattack, operators of secure systems such as Natanz have stiffened their security. Among the new protocols are bans on connecting thumb drives and other external storage devices to ostensibly secure systems. So now the Pentagon is interested in new ways to infiltrate isolated computer systems without gaining physical access. Defense News cites sources familiar with the program who say that the Army’s Intelligence and Information Warfare Directorate (I2WD) met with representatives from about 60 organizations to start figuring out how to, for example, send malicious code through the air into an enemy facility from a van parked outside or a drone hovering far above. 

Pay Attention, Class

Speaking of security updates, administrators at an unnamed U.S.-based power plant clearly didn’t get the memo. The U.S. Computer Emergency Readiness Team (CERT) reported in a just-released quarterly report that the power generating facility was shut down after malware infiltrated its turbine control systems and engineering workstations. The agency, which is part of the U.S. Department of Homeland Security, wouldn’t reveal the name, location, or type of plant, but said that the malicious code was introduced by a contract employee using a USB drive to perform software updates. And get this: None of the computers were equipped with antivirus software. Why, you ask? The reasoning, at least until recently, was that because industrial control systems in such facilities aren’t connected to other networks, malware couldn’t get in.

The problem wasn’t discovered until the contractor noticed glitches in the operation of the USB drive. A cursory check by the IT staff at the power plant revealed that it was infected with a two different types of malware. CERT says it removed the malicious code from the control systems and workstations and offered some recommendations for tightening security there. I imagine the first recommendation was: Get a clue.

Is Your Identity Worth Stealing?

According to an old saying, beggars can’t be choosers. But it seems that thieves have no such governing principles. A Security Week article reports the discovery of a new phishing technique that courts a preselected group of victims and doesn’t bother infecting the machines of people who are not on the so-called “bouncer list.” According to researchers at EMC’s RSA Security division, attackers begin with a list of email addresses and assign each person on the list a unique user ID. When someone stumbles upon the Web page hosting the malware, the site first checks to see if the person has been assigned an ID number. If so, the browser is directed to the phishing page; if not, the user is shown a “404 page not found” message. Being selective, say security experts, allows the perpetrators of such schemes to attack many “quality” victims without setting off the alarms that would be triggered by casting a wide net. The RSA researchers say each of these schemes typically targeted 3000 people. “Obviously quality data fetches a higher price in the underground,” Daniel Cohen, RSA’s head of business for online threats, told Security Week. He added that these attacks are most likely the work of someone looking to sell the information for profit rather than an illicit end user.

Malware Comes Calling Via Skype

As if phishing schemes and other come-ons weren’t leading to enough online havoc, CSIS Security Group, a Denmark-based IT security firm, has reported in a blog post that Shylock, a malware program designed to steal credentials for online banking accounts, has been armed with a new propagation method. A new plug-in added to the program this week allows it to send messages and files through Skype, then cover its tracks by deleting them from Skype’s history folder. Addding to the plug-in’s stealth is its ability get in and out without triggering the warning and confirmation request that a user normally sees when a third-party program tries to connect to Skype. Researchers already knew that Shylock could copy itself to removable drives and local network shares

Observers suspect that the move to use Skype as a transmission mechanism is related to Microsoft’s announcement that it plans to scrap its MSN Messenger service on 15 March. Microsoft advised users to switch to Skype. Also important, from the cybercrook’s perspective, is the ability to use Skype to reach any point on the globe instead of being mostly limited to small regions because users of infected machines tended to connect with a limited circle of friends.

Hacker Prosecutors Face Scrutiny

On 11 January, Internet pioneer and activist Aaron Swartz committed suicide at age 26. He was facing the prospect of a 35-year prison sentence if convicted of violating the United States’ federal Computer Fraud and Abuse Act (CFAA).  In the wake of Swartz’s death, the prosecutors in the case—and MIT, whose systems Swartz used to pull off the misappropriation of thousands of subscription-based scholarly papers—have been tried in the court of public opinion. Swartz supporters and other observers say the potential punishment did not fit the crime.

In a petition on the White House's website started on 14 January, some legal experts indicated their desire to see the government initiate a review of the CFAA that would result in a more nuanced application of the 1986 law. The statute “makes it illegal to knowingly access a computer without authorization, to exceed authorized use of a system, or to access information valued at more than $5,000.” But the petitioners note that the law was originally intended to bring the hammer down on hackers aiming to steal for personal gain or to sabotage systems. Neither of those motives was behind Swartz’s caper, they point out. "The government should never have thrown the book at Aaron for accessing MIT's network and downloading scholarly research," the Electronic Frontier Foundation (EFF) said in a 14 January blog post. Hanni Fakhoury, staff attorney at EFF, told Computerworld that “Over the years, creative prosecutors have taken advantage of the law and applied it to situations that it was never meant to tackle.” 

F-35 Software: DoD's Chief Tester Not Impressed

Last September, U.S. Air Force Maj. Gen. Christopher Bogdan, the then incoming director of the troubled  F-35 program, said that he was not optimistic that all the program's current problems—especially those related to software, which has long been a sore point (pdf)—would be fixed in time to meet the services’ planned initial operational capabilities, beginning with the Marine Cops in about 2 years. The 2012 Annual Report (pdf) on major defense acquisitions, by the Department of Defense's Director of Operational Test and Evaluation, J. Michael Gilmore, isn’t likely to increase Bogdan’s optimism any.

In his report, Gilmore states that in regards to operational suitability, the F-35 currently “demonstrates [a] lack of maturity… as a system in developmental testing and as a fielded system at the training center.” While Gilmore’s report details a host of other engineering-related issues as well, software remains a major area of concern.

For instance, the report states that, “Software delivery to flight test was behind schedule or not complete when delivered” and that, “Block 1 software has not been completed; approximately 20 percent of the planned capability has yet to be integrated and delivered to flight test.” Block 1 software, which provides initial training capability, was first flown in November 2010.

Read More

Red October Detected but Still Dangerous

Like the fictional nuclear submarine with the same name, the Rocra or Red October computer espionage campaign was designed to escape notice. It operated undetected by most antivirus products until unnamed researchers discovered it five years after it began stealing data on workstations, mobile devices and networking gear. Kaspersky Lab said it was alerted of the Rocra attacks by a partner in October; that’s when it began tracking the campaign’s myriad tentacles, which extended mainly to Eastern Europe, former Soviet nations, and Central Asian countries. In a report released today, Kaspersky described the cybercrime operation as, “still active with data being sent to multiple command-and-control servers through an infrastructure which rivals the complexity of the Flame malware.”

Kaspersky researchers say they haven’t found any connections between Rocra and Flame, but like Flame, the new campaign comprises more than a thousand unique malware files that carry out tasks such as reconnaissance, scanning for new machines to infect, recording keystrokes and screenshots, and capturing data in e-mail and USB drives. According to Kaspersky’s Threatpost blog:

“The command and control infrastructure behind this campaign is made up of 60 domains and a number of server host locations in Russia and Germany, most of which act as proxies in order to hide the true C&C server. Kaspersky said it was able to sinkhole six of the domains and watch them over since Nov. 2. More than 55,000 connections were made to the sinkhole from close to 250 IP addresses. Most of those IP addresses were in Switzerland, Kazakstan, Greece and Belarus; there are victims in 39 countries.”

This level of sophistication, say the researchers, requires resources that bespeak the participation—or at least the purse strings—of a nation-state. Still, Kaspersky wouldn’t go so far as to make that claim—even though the targets of the attacks, which include oil and gas companies, aerospace and nuclear research firms, and trade and commerce organizations, suggest a country looking to improve its fortunes or gain strategic advantage by getting its hands on proprietary information without paying for it.

Read More

IT Hiccups of the Week: BATS Global Long-hidden Programming Errors

It’s been another relatively quiet week on the IT glitch front. We start off, again, with news of errors involving a stock exchange—this time ones that have avoided detection for four years before being discovered.

BATS' Undiscovered Errors

According to Bloomberg News, BATS Global Markets, the third-largest U. S. stock exchange, announced on the 9th of January that it had discovered during internal system audits two situations where “its computers allowed trades that violated [U.S.] rules intended to ensure all investors get the best prices for equities.” In one case, a programming error in BATS’ Matching Engine that matches “orders for two Bats equity exchanges and an options venue allowed some trades to occur at prices inferior to the best available bid or offer.” In the second case, another programming error allowed short sales that violated trading rules. You can read the details in the BATS press release.

A story at the Financial Times said that the errors first occurred in October 2008 and weren’t discovered until about a week ago. Some 436,528 trades were affected, and “customers lost more than $420,000 from inferior prices they received,” the FT reported. BATS CEO Joe Ratterman defended how BATS tests its systems, telling the FT that the affected trades were “anomalies,” and implying that, given the complexity of the exchange operating environment, no one should be surprised by these issues. At least he didn’t call his programmers “knuckleheads.”

Ratterman did try to deflect responsibility, placing the blame on government regulators for creating the complexity. “The regulatory environment is getting layered with additional guidelines and requirements and what you end up with is a fair amount of rules . . . as you layer on over time, it gets more complex,” Ratterman explained. Ratterman conveniently neglected to mention that if there weren’t so many technical issues cropping up (as well as unbridled growth in complex and risky financial engineering instruments few understand), maybe there wouldn’t be so many regulations being piled on the exchanges.

While in artful dodger mode, Ratterman added that customers hadn't complained about being disadvantaged, as if that is an excuse. That said, at least Ratterman told the FT that BATS wants to compensate customers affected by the programming errors.

BATS is understandably defensive about this latest issue—even though it was small from a financial standpoint—given the major hit the exchange took to its reputation last March when it botched its own IPO. That screw-up helped Standards & Poor's decide to assign BATS a BB- corporate credit rating late last year; the S&P in its announcement said it believed “that BATS is highly vulnerable to operational risk.”

Forewarned is forearmed.

RIM Takes a Hit

Speaking of reputation hits, the last thing that RIM wanted last week was being associated with yet another service outage, even if it was relatively minor. According to a Wall Street Journal report, an apparent Vodafone U.K. router error caused e-mail and instant-messaging services throughout Europe, the Middle East and Africa to be affected for about five hours on Friday. Vodafone U.K. BlackBerry customers were the worst hit. Voice and text messages were not affected, however.  Vodafone said in an emailed statement that, “We apologize to customers for any inconvenience caused.

RIM is hoping that its new BlackBerry 10, scheduled for launch by the end of the month, will turn around its fortunes. With that in mind, it certainly doesn’t need any visits from ghosts of problems past.

NAV Canada Regrets the Inconvenience

Finally, last Thursday evening, flights into and out of Toronto’s Pearson International Airport were delayed and some were canceled when a flight planning system computer belonging to NAV Canada, the country's civil air navigation services provider, crashed.

Everything was back to normal by 0300 local time Friday morning, a Canadian Press story reported. NAV Canada said that it “regretted the inconvenience,” and although it wasn’t sure what caused the problem, it was “looking into the matter to make sure it never happens again.”

That's most reassuring, given that they don't know what caused the crash yet.

Android in My Rice Cooker: Gateway to Future Cyber Home Invasion?

Over the past few days, there were several interesting stories in the news that caught my eye. The first was at Bloomberg News on 8 January; it reported how Google’s Android operating system software is increasingly being embedded into everything from refrigerators to rice-cookers. According to the story, Android creates a nice, symbiotic relationship between Google and product manufacturers. The manufacturers get a free and easy-to-use OS that allows them to create Internet-connected products, while Google is in a position to “collect more data to build its lucrative search business and one-up software rivals Microsoft Corp. (MSFT) and Apple Inc.”

Another goal of the manufacturers is to also create products that can “exchange information with less human intervention.” The Bloomberg story goes on to state that, “A television, for example, might show a pop-up message from a clothes dryer in the basement, indicating that the homeowner’s jeans are not yet dry. The user could press a button on the TV remote to automatically add 15 minutes to the dryer cycle. A connected rice cooker could determine what type of rice is being used and set cooking instructions accordingly.”

Given that your rice cooker might also inform others what type of rice you’re cooking, like a competitor to the brand you're using who might then try to convince you otherwise, consumers might view their relationships with their Android-enabled devices as parasitic in nature, but that is a discussion for another time.

For the past 20 years or so, observers have predicted that digital convergence—what used to be called “smart” or “intelligent” devices and has now has morphed into “The Internet of Things"—is just over the horizon. Finally, it's actually is looking like it is going to be an everyday reality sooner than later. A story in 9 January’s Wall Street Journal reports that 10 consumer-oriented product companies have launched the Internet of Things (IoT) Consortium whose aim is, “cooperation between hardware, software, and service providers.”

The IoT website states further that, “The IoT Consortium is primarily focused on those Internet enabled devices and related software services that directly touch consumers in the form of home automation, entertainment, and productivity. One of the goals of the consortium is to see billions of connected devices that benefit from communication with other devices and services.”

The IoT charter members include Active Mind Technologies, BASIS Science, Coin, Kease, Logitech, MOVL (KontrolTV), Ouya, Poly-Control, SmartThings, and Ube.  The members of IoT essentially break along two lines: new companies that are basically start-ups, like Active Mind, Kease, Ouya, Smart Things, and Ube, who are developing products that include video game consoles (Ouya), smart electrical outlets (Ube) and home automation controllers (Smart Things), and more established product companies like Logitech (peripherals), Basis (personal health trackers), and Poly-Control (access security products). Active Minds has a nice description of what all these companies are ultimately aiming to do in this age of the Internet of Things, namely, provide tools and technology to support the “emerging world of the quantified self.”

The third story, from the 9 January London Telegraph, concerns a report from the UK Commons Select Committee on Defense which finds that, “Evidence received by the Committee suggested that in the event of a sustained cyber attack the ability of the Armed Forces to operate effectively could be fatally compromised due to their dependence on information and communication technology.”

The report points out that because of the increasing extent of digital technology in defense systems that communicate with one another, there are “many more points of vulnerability.” Therefore, says the Defense Committee, “cyber threats can evolve with almost unimaginable speed and serious consequences for the nation’s security." The Government, it added, "must be more vigorous in its approach to cyber security.”

The committee wants the UK Ministry of Defense to make a heavier investment in “mechanisms, people, education, skills, thinking and policies that take into account both the opportunities and the vulnerabilities which cyberspace presents.” The Parliament members also supported the view (pdf) expressed last summer by the Cabinet Office Intelligence and Security Committee that the UK should not just try to keep cyber criminals out, but engage in “active defense”, i.e., "actively interfere with the systems of those trying to hack into UK networks.”

All of which leads us back to our Android-enabled rice cookers.  As more everyday appliances become nodes in home networks that connect into national networks, and become increasing interdependent (and vulnerable) in the manner that military systems now are, how will the inevitable cyber attacks against them be viewed? For instance, consider a coordinated cyber attack against home automation networks using some insecure Internet-enabled appliance connected to the network as a gateway which successfully shuts down tens of thousands of home heating systems during a major blizzard like that of 1978 that hit the Northeast United States. Will such a widespread attack against non-vital government or infrastructure IT systems be seen as mere nuisances or as something more threatening?

It may sound far-fetched, but once everything communicates with everything else, I don't put it past malicious programmers to figure out clever ways to exert control over devices remotely in ways no one planned against.

It is interesting that the IoT Consortium doesn’t mention improving the security of Internet-enabled devices and related software services in its mission statement. Maybe it should think about doing so in order to ensure that the above question never has to be answered. Until then, I think I'll hang on to my dumb rice cooker.

IT Hiccups of the Week: Amazon’s Christmas Eve Incident Hits Netflix

Luckily, there have been rather few IT uff das over the holiday season. That said, there were still a few worth noting.

The world’s stock exchanges continued their streak of glitches over the last year with one at the New York Stock Exchange ending 2012 and two new ones to start 2013, one at the Nasdaq and the other at the London Stock Exchange, which served to undercut the exchanges’ promises to reduce operational snafus.

Netflix experienced two glitches, one at the hands of Amazon on Christmas Eve and one of their own making. At about 12:30 PST on Christmas Eve, Netflix’s streaming video went out  for customers across Canada, Latin America, and the United States  and wasn’t fully restored until late Christmas morning. Netflix placed the blame on a problem with Amazon’s Web Services (AWS) cloud computing center in Northern Virginia, which Amazon admitted experienced an “event” on 24 December. Amazon, in a long and detailed explanation, said that the root cause of the error was a programmer doing system maintenance who accidentally logically deleted “a portion of the ELB [Elastic Load Balancing] state data.” It took a couple of hours before the accident showed up in a way that it could be diagnosed correctly and then many more before an effective fix took hold. Other Amazon AWS users were also affected by the incident, but given that it was Christmas Eve, there weren’t wide-spread complaints reported.

Amazon apologized for the incident, and claimed that it would “use it to drive further improvement” in its services. Netflix also apologized for the outage, explaining how it had thought it had built in enough redundancy to handle such an incident. In a backhand swipe at Amazon’s vaunted claims for it cloud’s service reliability, Netflix said, “It is still early days for cloud innovation and there is certainly more to do in terms of building resiliency in the cloud,” and would be investigating further approaches on how to improve its reliability.

The other Netflix problem occurred on New Year’s Eve. In this case, the “technical difficulties” didn’t affect streaming video but the capability of some Netflix customers to add discs to their mailing queue. Apparently the minor issue was fixed by New Year’s Day.

A more impactful technical difficulty was felt in Michigan when the state’s Department of Technology, Management and Budget (DTMB) to load food assistance benefits on state-issued debit cards at the beginning of the year, the AP reported. It happened because of a “human error” in which a the department “failed to give a vendor a computer file required”. Some 85 000 food assistance recipients out of 1.8 million were affected; those affected had identification numbers ending in “0,” “1,” and “2,” television station CBS Detroit reported. The situation was fixed by noon on Saturday, and the DTMB promised to “find out why exactly this happened” and “figure out how to make sure this doesn’t happen again.”

A similar “forgot to load the data” excuse was pointed to as the reason why some Augusta, Georgia city employees discovered that their Blue Cross Blue Shield of Georgia health insurance cards were showing up as being invalid on New Year’s Day. According to a story at the Augusta Chronicle, “Mike Blanchard, the deputy information technology director … said [in an email to city employees] the problem was caused when … [an] employee benefits file failed to load into the Blue Cross system.” The insurance policy was still valid, but until new cards were mailed out sometime this week, “employees should present a copy of an attached letter at the doctor’s office and pharmacy and have the office call Blue Cross or pharmacy services for confirmation.”

The Chronicle reported that city employees weren't happy, especially given that, “City officials offered similar explanations when hundreds of Augusta employees’ and retirees’ insurance previously turned up canceled,” last year.

Finally, there was an AP report that those dependent on unemployment benefits in Arizona wouldn’t have to wait for their checks after all. The Arizona Department of Economic Security had warned benefit recipients that the late decision by the U.S. government to extend unemployment benefits as part of the “fiscal cliff” agreement might delay unemployment checks for up to a week because of the computer programming changes required.  However, the AP reported, the programming changes were completed ahead of schedule and no delay would result.

A programming change done ahead of schedule? Almost sounds like as A Christmas Carol ending.

This Week in Cybercrime: Danger May Not Come from What You Do As Much as Where You Go

Mathematicians have their beloved Erdös Numbers (coauthors of the prolific Paul Erdös are a 1, their coauthors have an Erdös Number 2, and so on), and movie lovers have the game “Six Degrees of Kevin Bacon,” which posits that because Bacon has appeared in so many films, it is possible to link him to any other actor within six steps of relatedness.

In the past week, the world of computer security experienced something analogous, with the top stories demonstrating the interrelatedness of things on the Internet. In this case, a mass transit operator in Turkey can be linked in a few steps to an energy manufacturer in the United States.

On 3 January, the Kaspersky Lab Threatpost reported that Capstone Turbine Corp., a company specializing in power generation equipment for utilities, had become the most recently discovered victim of malware exploiting a vulnerability in Microsoft’s Internet Explorer 6, 7, and 8 browsers. The cybercriminals carried out a so-called watering hole attack. Instead of attacking the desired victims directly, the hacker profiles the individuals or companies, finding out what websites they frequent. The attacker scans those sites for vulnerabilities. Having found one or more whose defenses can be penetrated, the attacker injects code at those sites that causes the victim’s computer to automatically redirect to a separate site. The site to which the victim is diverted hosts a zero-day exploit that is lying in wait—like a lion at a watering hole—to give the attacker access to the victim’s computer so he or she can install more malware, steal data, or monitor the victim’s activities. According to Kaspersky Labs, “Attackers are generally state-sponsored and hope to spy on their victims’ activities and siphon off business or military intelligence.”

The revelation that Capstone had been attacked at the watering hole comes just days after it was reported that the website of the U.S. Council on Foreign Relations, a high-level think tank, has been compromised by the same code since early December, and security firm FireEye confirmed that the site was still hosting malware as of 26 December. “We can confirm that the malicious content hosted on the [CFR] website does appear to use Adobe Flash to generate a heap spray attack against Internet Explorer version 8.0 (fully patched), which was the source of the zero-day vulnerability,” wrote FireEye’s Darien Kindlund on the company’s blog. “We have chosen not to release the technical details of this exploit, as Microsoft is still investigating the vulnerability at this time.”

Microsoft released a temporary patch on 31 December and noted that it is still working on a security update for the browser vulnerability, says Kaspersky’s Threatpost. Computerworld reports that in addition to the announcement about the fix—which offered no timetable about when an update that would eliminate the zero-day exploit would be ready—Microsoft announced on 3 January that it will release seven security updates next week “including one rated critical for Windows 8 and Windows RT -- to patch 12 vulnerabilities in Windows, Office, SharePoint Server and the company's website design software.”

Microsoft also took that opportunity to warn computer users that hackers were using digital certificates wrongfully obtained from a Turkish certificate authority and urge them to make sure that they have installed a Windows update that handles the decertification process whenever warranted.

On the same day, Google noted in a corporate blog post that someone attempted to impersonate Google.com on 24 December with the aim of carrying out a man-in-the-middle attack. What did the hacker use as a disguise? A fraudulent certificate generated after the Turkish trusted root certificate authority Turktrust mistakenly gave the power to issue certificates to two companies that were not supposed to have it. One of the two firms, which maintains a site at ego.gov.tr, is a Turkish transit authority.

Google Chrome detected and blocked the errantly issued certificate, says Wired. If the man-in-the-middle attack using the certificate had been successful, the hacker would have been able to “intercept and read any communication that passed from users on the ego.gov.tr network to any google.com domain, including encrypted Gmail traffic,” says Wired.

Image: Marcellus Lindsay

Another Year, Another ERP Lawsuit: Accenture/Microsoft Joint-Venture Sued for Bait-and-Switch Tactics

ScanSource, a South Carolina-headquartered wholesale distributor of specialty technology, announced this week that it had filed a lawsuit against Avanade, a joint venture formed in 2000 by Accenture and Microsoft. The suit alleges that Avanade purposely underestimated the cost of the enterprise resource planning (ERP) software system Microsoft Dynamics AX that ScanSource had contracted with them to install. According to ScanSource’s press release, Avanade promised to install the ERP system—claimed by ScanSource “to be one of the largest global enterprise level Microsoft Dynamics AX implementations to date”—for $17 million in 11 months; however, “within a few months” the cost escalated to an estimated $66 million and more than three years to complete.

In its lawsuit, filed in U.S. District Court in Atlanta, ScanSource alleges “fraud, tortious misrepresentation and breach of contract," and seeks "tens of millions of dollars in damages that ScanSource has incurred and will continue to incur, as a result of Avanade’s misconduct.” The company claims that Avanade misrepresented its “skills and abilities” in order to land the contract, using “bait-and-switch” tactics that included making ScanSource believe that Avanade would use “highly-skilled consultants to perform the implementation” but instead used “a revolving door of consultants who knew little or nothing about implementing Microsoft Dynamics AX software or managing large-scale global ERP projects.”

ScanSource may have a hard time making the bait and switch charge stick, since courts often view "vague, exaggerated, generalized or highly subjective statements" related to representations of skills and abilities as mere "puffery."

ScanSource may have better luck with its claims that the consultants created “an incomplete and defectively designed system that suffered from numerous flaws and required extensive remedial work.  Avanade’s incompetence was reflected in, among other things, the staggering 500 000 lines of software code it wrote in an attempt to have the Microsoft Dynamics AX platform meet ScanSource’s business requirements.”

In addition, the code developed apparently was riddled with “hundreds” of Priority 1 fixes that a Microsoft quality assurance assessment uncovered. ScanSource claims that the poorly designed and developed software would likely have increased the cost of future upgrades and maintainability of the ERP system.

The press release says that after ScanSource received the Microsoft QA assessment, it delivered a Notice of Default to Avanade in May of 2012 demanding that it fix the identified problems. The press release goes on to say that, “When it became evident to ScanSource that Avanade had no intention of curing its defaults, on 28 September 2012, ScanSource provided Avanade with a Notice of Termination removing Avanade from the Project.”

ScanSource has hired a new implementation partner and plans to complete the ERP project, still using Microsoft Dynamics AX. The project's cost to complete is now estimated at between $58 and $72 million, and it may not be completed until late this year or early next. The company says it has spent some $37 million already, and “seeks tens of millions of dollars in damages that ScanSource has incurred and will continue to incur, as a result of Avanade’s misconduct.”

A blog post over at the Wall Street Journal says that neither Avanade nor Microsoft will comment on the lawsuit. Doubtless, one or both will soon file a counter suit saying all the resulting problems were the result of ScanSource's unreasonable demands and expectations, and that they were entirely blameless for the fiasco.

There may be yet another ERP-related lawsuit in the offing, as well. An AP story from last month reported that California’s State Controller’s Office wrote a letter to SAP claiming that the California government payroll system modernization effort called the 21st Century Project is foundering and is in danger of collapsing.”  The state controller’s office demanded that SAP fix the state-identified problems, the AP said, “including replacing inexperienced project managers and staff.”

Hmm, that sounds familiar.

An SAP spokesperson told the AP in response to the letter, which still hasn't been made public, that there were “some challenges” on the project, but not to worry; “SAP remains committed to the overall success of the project.”

How long California remains committed to SAP remains to be seen. California fired BearingPoint in January 2009 for “non-performance” on the project, which is now over 5 years late and triple its original cost of $131 million.

In a less contentious bit of IT project failure news, also last month, Vermont announced that the state and HP had agreed to a “no-fault” divorce over a botched six-year, $18.5 million Department of Motor Vehicles project computer system modernization effort. According to a press release from Governor Peter Shumlin, HP will refund Vermont the $8.37 million it was paid, and Vermont will return to HP “physical and virtual rights to all software and documents created by HP.” Vermont will keep some of the components developed by HP that are now in use at a cost of about $262 000.

The two parties agreed to terminate the contract for “mutual convenience.” The governor’s press release said that, “The state and HP entered into the settlement agreement because they recognize that the DMV contract was flawed and that a mutual termination is the best path to resolve outstanding issues.”

The AP was onto this story as well, noting that the state is out “nearly $5 million in state staff time DMV employees have put into trying to implement system changes that now are deemed to have failed. Other unreimbursed state expenses include money Vermont spent for ‘change management consulting’ and other services.” Those consulting costs look to be in the several million dollar range.

For now, Vermont doesn’t plan another attempt at modernizing the now 36-year old DMV computer system.  DMV Commissioner Robert Ide says that the focus will be on keeping the current system working, adding that, “My people need a break. It's been incredibly discouraging for our people who were working on this project.”

At least Vermont has one of the most progressive health care systems in the United States. I'm not sure if it covers emotional therapy or "stress relaxation" to help DMV workers deal with their discouragement. Maybe Vermont can provide it by using some of the money HP gave back.

Most Commented Posts

Risk Factor

IEEE Spectrum's risk analysis blog, featuring daily news, updates and analysis on computing and IT projects, software and systems failures, successes and innovations, security threats, and more.

Contributor
Willie D. Jones
 

Newsletter Sign Up

Sign up for the ComputerWise newsletter and get biweekly news and analysis on software, systems, and IT delivered directly to your inbox.

Advertisement
Advertisement
Load More