When Robots Turn Deadly...

...and the venture capitalists funding them are not supposed to know

3 min read

Prey: Michael Crichton, US $27 HarperCollins , ISBN 0-0662-1412-2

"Things never turn out the way you think they will." So begins Michael Crichton's latest novel, Prey, an account of seven nerve-wracking days in which programmer Jack Forman deals with mischievous children, his possibly adulterous wife Julia, and malfunctioning robots.

What makes the tale fascinating is that these robots are the minute products of nanotechnology, biotechnology, and artificial intelligence(AI) all combined. What makes it chilling is that the robots start operating in ways their designers never anticipated. What makes it thrilling is that as Jack struggles to debug the miscreants, you can never be sure how things will turn out.

This recipe would make Prey an entertaining read for almost anyone. But the author's skillful extrapolation of current technology and his examination of its unintended consequences makes this book especially thought-provoking for engineers and computer scientists.

Crichton has a knack for identifying emerging technologies before they make the headlines. His 1969 novel, The Andromeda Strain, about a biological warfare development program run amok, had such a ring of authenticity that when scientists reached the scene of the first recorded Ebola outbreak in 1976, one of them later recalled thinking that "Andromeda had landed." Jurassic Park, published in 1990, featured dinosaurs created through cloning—seven years before scientists announced the cloning of Dolly the sheep.

Once again, Crichton appears to have spotted a hot emerging technology—in fact, three of them. First, nanotechnology: already laboratories are manufacturing microscopic devices molecule by molecule using traditional chemistry. Now nanotechnology is merging with biotechnology, the second hot field, in experiments using bacteria and viruses to facilitate the assembly process. Commercial use is expected within the decade.

Finally, there is evolutionary computing. Unlike AI programs based on complex rules written by humans, a program based on evolutionary computing strives to teach itself to do things by following only a few basic principles. [See "What is evolutionary computation?," IEEE Spectrum, February 2000, pp. 26-32.]

Such programs already exist. One is Blondie24, a program that learned to play checkers so well it beat more than 99.5 percent of players who challenged it online. Indeed, an interesting companion to Prey is the nonfiction Blondie24: Playing at the edge of AI, by David Fogel (Morgan Kaufmann Publishers, 2001). Fogel, one of Blondie's creators, makes understandable the concepts and current status of evolutionary computing.

Of course, Prey wouldn't be a Crichton novel without the specter of unintended consequences. Instead of biochemists and doctors, this time the novelist challenges engineers and computer scientists to deal with things that don't turn out the way you think they will.

A central question raised by Crichton is how do we prevent technical processes from running away? In the days of steam, the invention of the governor prevented run-away engines. But how do you build a governor in self-replicating robots that learn independently from their environment?

Crichton poses another timely question: has technical judgment become hopelessly subservient to business requirements? Should these robots be released into the environment before it is fully understood how well the controls are functioning?

Most of the engineers and programmers in Crichton's story fail to deal successfully with either of these issues. To maximize the efficiency of operation, they remove any self-limiting components from their designs and rush to test the robots in the desert. Even after problems arise, they sheepishly continue to build more robots.

Prey is not the first novel to deal with the possibly disastrous results of mixing nanotechnology and artificial evolution. Will McCarthy's 1999 Bloom, for example, is a fascinating look at a future solar system nearly overrun by all-devouring nanotechnology. But, like all Crichton's books, Prey is lent weight by being set firmly in our contemporary world. Recently laid off, Jack is a stay-at-home dad and wife Julia, a fast-track executive at a Silicon Valley start-up that makes the problematic robots.

And why are the engineers so reckless ? Because Julia and the other executives tell them to be so. The execs are pursuing venture capital financing and want to show rapid progress. Only Jack has the insight to comprehend the danger and the tenacity to oppose the execs—wife included—and to try to take corrective action.

Perhaps the lesson in all this is that engineers and computer scientists should read more science fiction. If Prey's characters had read Isaac Asimov, they would have known his first law of robotics: "A robot may not injure a human being, or, through inaction, allow a human being to come to harm."

This article is for IEEE members only. Join IEEE to access our full archive.

Join the world’s largest professional organization devoted to engineering and applied sciences and get access to all of Spectrum’s articles, podcasts, and special reports. Learn more →

If you're already an IEEE member, please sign in to continue reading.

Membership includes:

  • Get unlimited access to IEEE Spectrum content
  • Follow your favorite topics to create a personalized feed of IEEE Spectrum content
  • Save Spectrum articles to read later
  • Network with other technology professionals
  • Establish a professional profile
  • Create a group to share and collaborate on projects
  • Discover IEEE events and activities
  • Join and participate in discussions