Internet security professionals are, by occupational temperament, a pretty nervous bunch. But lately they've had more reason than ever to be jumpy. Early this year, a new kind of worm, known as Storm, began to sweep through the Internet. It hasn't received much attention in the mainstream press, but it has given security professionals more than a few sleepless nights. Storm is far more sophisticated than previous worms, because it uses peer-to-peer technologies and other novel techniques to evade detection and to spread. The popular press hasn't paid much attention to Storm, because it has yet to wreak devastating havoc on businesses, as some previous worms have. But we shouldn't be fooled by that relative quiet: Storm's designers appear to be biding their time, building an attack network far more disruptive than any before seen.

Storm methodically infiltrates computers with dormant code that could be used to take down the entire network of a corporation, creating opportunities for blackmail or for profiting by selling the company's stock short. And Storm's creators, whoever they are, continue to modify and refine their malevolent progeny even as it already stands as a dark cloud poised over the Internet.

Network security software products on the market today offer only limited defense. They use firewalls, which simply block access to unauthorized users, and software patches, which can be created only after a worm or virus's unique bit pattern is discerned. By the time this laborious process of hand coding is complete, the infestation has had hours and hours to spread, mutate, or be modified by its creators.

A new kind of answer is needed. Network security researchers--including ones at our company, Narus, in Mountain View, Calif.--are developing software that can rapidly detect a wide variety of intrusions from worms, viruses, and other attacks without the high rate of false alarms that plagues many conventional Internet security products. These new programs can detect anomalous network behavior in seconds, as opposed to hours or days--even on so-called backbone networks running at 10 billion bits per second. That means the software is fast enough to block threats that can span the globe in minutes, a rate that far outpaces what a firewall can monitor.

This new generation of algorithms is based on concepts related to the thermodynamic concept of entropy. Often defined briefly as a measure of the disorder of a system, entropy as a cornerstone of thermodynamic theory goes back more than a century and a half. But as a construct of information theory it is only 60 years old, and its application to data communications began only in the last decade or so.

In essence, an entropy-based defense works because a worm's malicious activity changes, in subtle but unavoidable ways, the character of the flow of data on a network. Those data flow changes alter, in clearly measurable ways, the entropy of the network--a measure of the endlessly shifting ebb and flow between the predictability and randomness of the movement of data on the network.

Researchers at Intel, Microsoft, Boston University, and the University of Massachusetts are among those plumbing the mysteries of randomness and order in data flows to get a leg up on network attackers. Although ours is the only company we know of whose commercial products apply entropy to network security, we are confident that the approach will find much wider favor in the next few years.

We'll have lots more to say about entropy and how algorithms that measure changes to the order and disorder of a network can detect a worm outbreak long before traditional methods can. But to get a grip on those algorithms, first consider how viruses and worms attack.

Virus or worm? Security experts distinguish between them, but their differences are less important than their similarities. Either can render computers on a network unstable, and in many cases unusable. A virus is a program that can copy itself and infect a computer without the knowledge of the user. It can, and often does, damage a computer's files or the hardware itself. A worm is, similarly, a self-replicating computer program that uses a network to send copies of itself from one computer, which we will call a ”host” of the infection, to other computers on the network. Worms usually harm the network, if only by consuming bandwidth.