In December, when Bernard Madoff’s hedge fund turned out to be a gigantic Ponzi scheme that was US $50 billion in the red, everybody wondered how this had gone undetected for more than a decade. Amazingly, in separate audits, government auditors saw no evidence of wrongdoing.
Embarrassed into action, the U.S. Securities and Exchange Commission (SEC) has announced plans to implement an off-the-shelf electronic monitoring system that tracks transactions conducted by 8000 hedge funds. The system will allow SEC accountants to examine hedge funds using scenario analysis and dozens of different graphical representations of the data it collects. But the SEC is intended to be the backstop in the financial accounting system, with most crooked firms getting caught by internal or outside auditors. So technological sleuthing capability belongs in the average auditor’s toolbox and not just the government’s.
Firms such as PricewaterhouseCoopers, a Big Four auditor, have been working on data-mining systems—software that looks for fraud by finding hidden patterns in financial data. PwC’s automated Ledger Analyzer (formerly known as Sherlock) detects anomalies such as a higher-than-normal number of credits to expense accounts or late-posted transactions. These are among the more than 100 indicators of a significant risk of error or fraud taken into account by the system, says David M. Steier, director of research at the company’s Center for Advanced Research in San Jose, Calif. Researchers at the auditing firm are refining the software so that as more data points are added, it becomes better at spotting these anomalies, while reducing the number of false positives.
But there are big limits to data mining’s fraud-finding potential, says Chao-Hsien Chu, a professor of information sciences and technology at Pennsylvania State University’s Smeal College of Business, in University Park. He’s conducted a review of the literature on the subject and found that although data mining alone has been used to detect illicit activity such as credit card fraud, insurance fraud, and unauthorized computer system access, it is not a reliable method for checking all financial statements.
One problem, says Chu, is that companies are less likely than ever to share the amount of general ledger data required to make data mining’s prediction, classification, and clustering functions effective, citing privacy as a concern.
Mark J. Nigrini, a professor of accounting and information systems at the College of New Jersey’s School of Business, in Ewing, is working on the problem using a method that might require less invasive access to data. Nigrini determines whether a given data set has been ginned up artificially by using Benford’s Law. This statistical phenomenon describes the relative frequency with which the digits 1 through 9 should appear in a particular decimal place. For example, the digit 1 should appear roughly 30 percent of the time in the decimal place farthest to the left. Lists of numbers manufactured by people, regardless of the purpose, tend to violate this statistical principle, giving an auditor a clue that something is amiss.
Nigrini uses off-the-shelf software to create statistical plots showing where and how widely a company’s numbers deviate. Asked if Benford analysis could be used to reverse engineer a passable list of fraudulent numbers, Nigrini says, ”Most fraudsters aren’t that sophisticated.” However, Nigrini acknowledges that his method has limitations, making it clear that a suite of tools is necessary.
Though data mining and pattern recognition make it possible to search through a general ledger with millions of transactions, some of which have 50 000 lines of detail, ”technology alone can’t solve the fraud detection problem,” says Steier. ”No matter how good the system is, it’s no substitute for an auditor’s professional judgment.”
Asked whether Benford analysis would have foiled the schemes of Enron and WorldCom executives, Nigrini says that the Enron gang would have been caught. But WorldCom’s plot, which used real-world numbers that had been intentionally mislabeled, would have passed the Benford sniff test. As this issue went to press, Nigrini was trying it out on Madoff.