How to Fight Crime in Real Time

New York City has a population of 8.2 million, and the municipal police force of its five boroughs is the world’s largest. The police have to worry about what is going on in the entire metropolitan area, which has a population of close to 20 million, as well as all the cargo and passengers entering and leaving the city through its port. Taking into account, too, the city’s immense ethnic and linguistic complexity and the social problems of its worst-off neighborhoods, law enforcement is a megaproblem by any standards. For better and worse, New York has always been filled and refilled with people who have the talent and ambition that make the United States what it is, but who also sometimes bring with them bad habits, discredited beliefs, and unfortunate associations from their homelands.

Take, to pick some notable examples, the Russian mobs that began to infiltrate Brooklyn’s Brighton Beach neighborhood in the early 1990s following the collapse of the Soviet Union, and the Chinese gangs discovered to be importing slave prostitutes into lower Manhattan, and the former Port-au-Prince torture specialists who could be lying low in the city’s burgeoning Haitian communities.

Considering the scope of the problems facing New York City law enforcement and the sheer size of the police force created to deal with it—some 38 000 officers, bigger than the FBI—you might expect to find an organization rife with bureaucratic inefficiency and corruption, unable to cope. What’s remarkable, then, is the degree to which the city has risen to the occasion, even as control of the city’s government has bounced back and forth between the political parties during the past few decades.

Most recently, the city created a central computerized control room, the Real Time Crime Center (RTCC). There members of the New York Police Department (NYPD) can monitor crimes as they are reported, immediately summon up all relevant information from centrally organized databases, and get that information to investigators in the field [see photos, "RTCC"]. The same system can also be used to counter any immediate terrorist threat, though the city’s counterterrorism operation emphasizes intelligent deployment of personnel and resources, not technology as such.

New York’s computerized crime center is but the latest in a series of police innovations that have brought the city’s violent-crime rates down in the last 15 years. Reforms aimed at strengthening old-fashioned gumshoe police work began in the early 1990s during the administration of David Dinkins, the city’s first black mayor, who brought in the dynamic and highly regarded Raymond Kelly as police commissioner to get officers back on the beat and more visible on every city block. Further big changes came with the next mayor, Rudolph Giuliani, and his first police commissioner, William Bratton. They inaugurated a policy of energetically going after seemingly minor “quality of life” crimes—teenagers jumping subway turnstiles, graffiti artists defacing buildings and trains—on the theory that often the people committing such misdeeds will be connected with much more serious offenses as well.

Although that aggressive approach to crime fighting gave rise to civil-liberties concerns—some well founded—it also turned out to be effective. As people apprehended for minor infractions were taken off the streets, the city began to experience dramatic decreases in violent crime, far exceeding improvements seen in most other U.S. cities. Sure enough, the person with a ­squeegee demanding money in return for a windshield ­cleaning—a ­practice generally experienced by New Yorkers as a form of petty ­extortion—often turned out to be wanted for more serious crimes or needed as a witness in an ongoing investigation. The police just had to check the suspect’s name or fingerprints against those in police files.

It was a natural second step to streamline the records themselves. In 1994 Giuliani and Bratton inaugurated a program that came to be known as CompStat. The city centralized its data on major crimes in a database, and detectives began to meet daily to analyze emerging crime patterns, redeploy resources to address threats, and take stock of successes and failures.

In the 1970s and well into the 1980s, New York had been known as a place where corporations didn’t want to locate and tourists didn’t want to visit, because of the threat of violent crime. That situation changed in the 1990s, as the number of murders dropped from nearly 2000 in 1993 to 629 in 1998 and 584 in 2002. CompStat got a lot of the credit, though other factors obviously played an important role, too—violent crime rates also declined in several other cities around the country. By the end of the decade, CompStat was being copied by at least half the large cities in the United States. Police chiefs in other parts of the world were sitting up and taking notice as well. The Manhattan Institute, a New York City think tank, could argue without much fear of contradiction that CompStat was “perhaps the single most important organizational/administrative innovation in policing during the latter half of the 20th century.”

Then came 9/11. Soon after the 2001 attacks on the World Trade Center, a newly elected mayor, Michael Bloomberg, took office. A tech-savvy and progressive-minded ­billionaire—trained, by the way, as an electrical engineer—he had made his fortune by tailoring a computer system to serve the needs of investors. Recognizing that the leadership of the NYPD was in disarray after a falling-out between Giuliani and Bratton, Bloomberg promptly brought back Dinkins’s man, Kelly, to serve again as chief. Bloomberg was impressed not only by Kelly’s earlier record in New York but also by the experience he had gained afterward working with the federal government on projects of national and international import.

To beef up the city’s electronic crime-fighting and counterterrorism capabilities, Kelly reached far outside the usual channels, hiring as the NYPD’s first chief information officer V. James Onalfo, who had been CIO for the international operations of Kraft Foods. Former IBM Chairman Louis V. Gerstner Jr. had recommended Onalfo to Kelly, and during his job interview with Bloomberg, Onalfo said he was interested only if the city was really serious and would provide the support needed to get the job done. The city was, and it did.

Kelly’s showcase project was the creation of the RTCC. He conceived it personally and, in October 2003, ordered Onalfo to have it set up by the following July. The idea was to focus investigative resources on crimes as they are committed, not merely after the fact. What NYPD wanted to do, explains Deputy Chief Joseph D’Amico, who runs the RTCC, “was to give a high-octane shot to CompStat.” Previously, investigators in the field had “nothing, no technology,” and so it was decided that the reporting and initial investigation of crimes was a “good place to focus technology.” The ultimate objective, Onalfo says, was “simply to speed up police reactions in emergencies, where seconds can be a matter of life and death.”

At the time Onalfo arrived on the scene, he says, there was no general e-mail capability across the whole system, no provision for emergency data recovery, and no laptops in police cars. A lot of potentially valuable information was “stovepiped” in incompatible systems—databases that had been jerry-built over the years to different specifications, without ever getting integrated.

Working mainly with IBM and with the Hauppauge, N.Y., office of Dimension Data, a South African software-­applications company, Onalfo has built a greatly enhanced computer and communications system for the NYPD, with a big emphasis on redundancy and data security. Using practices proven in industry, Onalfo devised a phased, iterative approach, so that gains could be registered periodically, whether or not support for the next phases proved forthcoming.

That was crucial, observes David Petri of Dimension Data, who was program manager in the center’s first year. The kinds of things Onalfo was doing had been standard in the private sector for many years, Petri says, but in the public sector, you can’t count on “ever-flowing money.”

Crucial to the success of the system’s implementation, Onalfo says, was the streamlined command structure and the absence of red tape. Eyeing the elaborate review committees and interagency politics that FBI Director Robert S. Mueller III has contended with in attempting to computerize that agency’s operations, Onalfo wonders “how he gets up in the morning.”

The NYPD’s communications are now handled by a fiber-optic ring, with many officers able to access information wirelessly from laptops in patrol cars. A dozen large vans equipped with wireless laptops and high-speed printers, along with audio, video, and fax capabilities, can be deployed to major crime or terror scenes to serve as mobile operations centers.

Information that previously was dispersed in many different repositories is now centralized in one data warehouse that can be accessed from the RTCC or from the streets. The creators of the centralized system took great care not to disturb legacy systems, notably the city’s 911 emergency telephone program, which handles more than 12 million calls each year and has to be able to deal with nearly 200 languages. The city’s police officers speak about 150 languages, says Onalfo, but all data are entered into the computerized system in English.

Access to the data warehouse is granted only with passwords backed by the biometric ID cards that all NYPD employees carry, and every query is logged so that any suspicious entry into the system can be investigated. Special forensic tools, developed mainly by Dimension Data, enable detectives to perform sophisticated searches and to call up all manner of links, allowing them to determine ­rapidly whether suspects really are who they say they are, what other incidents they have been involved in, and so on. RTCC staff members can, for example, tell investigators in the field whether the witnesses they are interviewing are wanted in connection with other crimes, enabling officers to apply a little extra pressure—the kind of scene regularly portrayed in episodes of television’s “Law & Order.”

Standing in the RTCC with Onalfo and D’Amico on a quiet spring afternoon, facing a theater-size display divided into six or seven changing rectangles, the lead author of this article gets a briefing on how the system works. To the lower left, there’s what’s often called a ticker (though it does not really resemble a list of stock quotations), showing a list of crimes reported in emergency calls to 911 or by officers to dispatchers. A red dot alongside an entry indicates the crime is in progress and deserves priority attention; yellow dots mark resolved situations. Another system of red and yellow dots shows whether police cars are immediately available for action.

The RTCC shares a wall with the NYPD’s emergency operations center, facilitating immediate exchange of information and ideas in a crisis. The counterterrorism command centers, located in undisclosed locations in the outer boroughs, have full access to the RTCC’s database.

D’Amico called up another display that showed a suspect in the middle, with concentric rings of associated names. Such networks can be extended indefinitely, he says, to indicate immediate family, business associates, partners in crime, and so on. There are about 25 display options, and the database can be searched using what D’Amico calls a “superfinder” tool, which can work on all sorts of structured and unstructured information, such as a phrase a robber might habitually use during holdups.

“Before,” D’Amico says, “when you called up a crime report, you could only query perhaps a half-dozen fields. Now we’re able to pull out every last word. We can develop partial information; we can do searches on the data such as the suspect’s modus operandi, the type of crime, his weapon, and the location.”

On this particular day, not much is going on, and what seems to be attracting the most attention is a CNN news report that two-thirds of Americans consider the Iraq war to be as big a mess as America’s former involvement in Vietnam. Onalfo recalls his surprise when he first joined the NYPD at how news-oriented the department is. Just about every office, he says, has a television running, and everybody’s watching for reports of al-Qaeda threats, terror incidents in other parts of the world, or other events that could inflame passions in the city’s streets.

Those occupying the RTCC are nervously awaiting a decision from a grand jury in Queens, one of the city’s five boroughs. The grand jury is deciding whether to indict police officers who, late one night in November, shot at a group of unarmed black men, killing one and seriously wounding two others, outside a nightclub.

The Queens shootings were but the latest in a string of wrongful-death complaints against the NYPD. There’s no obvious connection between the application of CompStat methods and those incidents, but concerns linger about the Big Brother ways in which electronically archived information can be used.

Onalfo and D’Amico insist that there are sharp boundaries between what can be stored and accessed and what cannot. They are not allowed to archive even publicly available records, they say, unless there is good reason to link such information to a particular criminal investigation. But, in fact, the boundaries are controversial and still being defined.

A particular point of contention has been the city’s attitude toward public demonstrations and information gathered by the police at such events. During the Republican National Convention, in August 2004, New York City police arrested 1806 people and routinely fingerprinted people who were charged with such minor offenses as parking violations. Also, the NYPD conducted extensive intelligence operations before the convention, spying occasionally outside the city and on some groups with no history of violence.

Just a few weeks before IEEE Spectrum’s visit to the RTCC, a U.S. District Court judge dealt the NYPD a setback, ruling that it could not routinely make video recordings of public gatherings, because of the misuses to which the images could be put. The city and the NYPD had argued that such videos could be useful in training and in protecting the department against allegations of abuse.

A week prior to Spectrum’s visit, the Reverend Al Sharpton, a civil rights activist, said he would file a class-action suit against the NYPD on behalf of those who see a racist pattern in the department’s stop-and-frisk policy. Since 2002, the number of individuals searched by city police has increased fivefold, according to figures released in February by the NYPD. More than 55 percent of those targeted were black, though blacks represent less than a third of the city’s population.

Whichever way such public-interest and private-rights issues are ultimately resolved, at least this much can be said: the same electronic tools that enable aggressive police action also permit the public to exercise a heightened level of scrutiny, as long as transparency is maintained.

Meanwhile, the main facts would seem to speak for themselves. At a cost of US $11 million, the RTCC has been created, allowing almost instant access around the clock to 120 million New York City criminal and arrest records and 911 logs, going back to 1995, as well as 5 million New York state crime records, 31 million national crime records, and 25 million other public records.

As that operation has gained a footing, the amount of time needed to complete the initial investigation of a crime has shrunk from an average of 9 hours to 2, says D’Amico, who has just left New York City to join the state attorney general’s office as chief investigator. About 80 ­percent of the city’s homicides were solved in 2006, and 74 percent the year before, statistics that would be the envy of most big cities, not just in the United States, but worldwide.

To be sure, even now more than 10 people are killed each week in the city, and in 2006 New York’s murder rate rose for the first time in many years, by about 10 percent, according to the Police Executive Research Forum, in Washington, D.C. However, in many other large U.S. cities, murder rates went up by as much as 20 percent, part of a recent national tendency toward rising violent-crime rates. New York City has largely bucked the trend.

And so, in light of New York’s impressive achievements, troops of foreign police officials continue to visit the RTCC from Austria, Brazil, Israel, the Netherlands, Norway, Sweden, Turkey, and other countries. They learn about the city’s crime-fighting system straight from Onalfo, who in 2006 was named one of Computerworld magazine’s CIOs of the year.

New York has made its computerized crime-­fighting methods a model for police departments the world over—but it’s a model that should be copied with care. In every system of law worthy of the name, individuals are innocent until proven guilty, and all police procedure must be grounded in that principle.

To see all of Spectrum’s special report on The Megacity, including online extras and audio and video exclusives, go to /moremegacity.