Backyard Star Wars

1. A long time ago, in a suburb far, far away...

”So, how would you kill mosquitoes with a laser?”

Nathan Myhrvold asked us. Lowell Wood, Rod Hyde, and I smiled. The three of us were meeting with Myhrvold in the fall of 2006, in an office at Intellectual Ventures Management, a company in Bellevue, Wash., that he founded in 2000 to create and invest in inventions. We smiled because we had just spent the afternoon arguing over that very question, scribbling ideas and calculations on a whiteboard, and had come up with what we thought was a pretty good answer: a ”photonic fence” in the form of a row of vertical posts that would use optical sensors and lasers to spot, identify, and zap bad bugs on the wing.

The idea of building a high-tech defense against disease-carrying pests had come up in discussions that Myhrvold and Wood had been having with Bill Gates, who was Myhrvold’s boss when he was chief technology officer at Microsoft in the 1990s. Through the Bill and Melinda Gates Foundation, Gates has been trying to improve living conditions in some of the world’s poorest countries and in particular to come up with ways to eradicate malaria, a mosquito-borne disease that sickens about a quarter billion people a year and kills nearly a million annually, including roughly 2000 children a day (see Web-only sidebar, ”New Techniques Against a Tenacious Disease”).

Illustration: Jude Buffum

Click on the image for the full illustration view.

Wood, a veteran of advanced weapons development at Lawrence Livermore National Laboratory, in California, and one of the scientists behind the Strategic Defense Initiative (otherwise known as ”Star Wars”), had suggested trying a similarly high-tech approach against malarial mosquitoes—to take advantage of inexpensive, low-power sensors and computers to somehow track individual mosquitoes and shoot them out of the air. If it could be done cheaply enough, this might offer the first really new way in many years to combat malaria, as well as other diseases transmitted by flying insects, such as West Nile virus and dengue fever.

Hyde and I had worked with Wood at Livermore. Hyde now manages Intellectual Ventures’ stable of staff and consulting inventors, and he assigned me the challenge of making the idea work—or showing why it couldn’t.

Three years later, my colleagues and I at Intellectual Ventures have now worked out many of the trickiest aspects of the photonic fence and have constructed prototypes that can indeed identify mosquitoes from many meters away, track the bugs in flight, and hit them with debilitating blasts of laser fire. And we did it without a multimillion-dollar grant from some national Department of Entomological Defense. Nearly everything we used can be purchased from standard electronics retailers or online auction sites.

In fact, for a few thousand dollars, a reasonably skilled engineer (such as a typical IEEE Spectrum reader) could probably assemble a version of our fence to shield backyard barbecue parties from voracious mosquitoes. We therefore present the following how-to guide to building a photonic bug killer, in five parts: selecting an appropriate weapon, spotting the bugs, distinguishing friends from foes, getting a pest in your sights, and finally shooting to kill.

Choose Your Weapon

Why build a fence? You could try to lure bugs to one spot and kill them, but more will just keep arriving. For all practical intents, the supply of mosquitoes is infinite.

You could build a system that scans your entire yard for bugs. But as any infantryman knows, the first step in defense is to establish a perimeter; it’s much more effective (and safer) to concentrate your firepower in a narrow zone. Mosquitoes generally don’t spend their whole lives in your backyard, unless you live in a swamp. They fly back and forth from their breeding grounds, so to get to you they have to cross your laser-guarded perimeter. A few may fly over the fence, but not many. The average flying altitude varies among mosquito species but is usually only about 2 meters. They will fly over obstacles when necessary—even into an upper-story window—but if your virtual fence is 3 to 5 meters high, it can catch almost all mosquitoes that fly by.

Of course, there are any number of ways to build a fencelike barrier. You could detect mosquitoes acoustically, with radar, or with ultrasonic sonar. You could shoot them down with tiny bullets, break them apart with sonic pulses, or cook them with microwaves. We considered these and many other possibilities, but it’s hard to beat the range, precision, and literally lightning-fast response of optics. Good digital cameras and powerful diode lasers are relatively affordable and easy to find. So for us, photonic technology is the way to go.

Getting a bug in the crosshairs

To detect mosquitoes, you’ll need several video cameras—four per fence post for a full-coverage fence (two at the top and two at the bottom facing the two adjacent fence posts—see ”Spot the Skeeter,” left). The cameras don’t have to be sharp enough to take a good photo of a mosquito many meters away, but they should have a resolution high enough so that a distant insect will at least fill up most of one pixel. If the bug occupies several pixels in the frame, that’s even better, as this will allow you to find its center. A 1.3-megapixel (1280- by 1024-pixel) camera turned sideways can cover a 4-meter-high fence post with pixels 3 millimeters on a side. If you’re economizing, you could even try VGA-resolution (640- by 480-pixel) cameras.

The speed of your cameras matters as much as the resolution. It’s no good if the bug is already through the fence by the time your system registers its presence. Mosquitoes fly up to a meter per second, so if the active zone is 10 centimeters wide, the frame rate must be more than 10 frames per second. Standard video cameras supply 30 frames per second, but the frames are interlaced—odd and even rows of pixels are delivered separately, 1/60 second apart—which will make it hard to follow a small moving object. So you’ll want to use noninterlaced cameras, or at least units on which you can disable the interlacing.

You’re interested only in a narrow vertical strip of image, less than 100 pixels wide, so you can drastically cut down the amount of data your cameras produce if you capture only that strip. Many digital video cameras allow you to do this by selecting a rectangular ”region of interest.” Using less than the full frame may also let you increase the frame rate.

Next you need to get all those pixels into a computer. Not only do you need a high data rate but also low latency—as little delay as possible between the moment the pixels are captured and when they’re available for processing. USB connections are probably too slow; we’ve found that IEEE 1394 (FireWire) or gigabit Ethernet interfaces work fine. (Best of all would be to create a custom image-processing chip and integrate it directly into the camera. But that’s a different project.)

Mosquitoes are dark and typically fly at dusk, so they’re hard to see by reflected light. It’s much easier to pick out their silhouettes against a bright background. Fortunately, in a fence, we can use one post as the background, as seen from the next post in line.

Put a light source next to each camera and aim both the light and the camera at an adjacent fence post. Cover the target post with retroreflective tape, which will reflect the incoming light directly back toward the camera, much as a highway sign does. We often use 3M’s Diamond Scotchlite material, which reflects 3000 times as much light back toward the source as a matte white surface does. Other ”safety marker” retroreflective materials should work too.

Single-color LEDs or diode lasers are good choices for the light source, because you can put a filter on your camera to block out stray visible light from the sun or nearby lamps. Infrared LEDs work better if you’d rather not attract insects, pets, or curious children to your fence; if you prefer a high-tech look, you can use red LEDs. You can add lenses or reflectors to focus the light at the target post, so that as little is wasted as possible.

Now that you have streaming video of bug shadows, the next step is to track the bogies. In simplest form, this just means generating a list of darker-than-usual pixels and grouping sets of adjacent dark pixels into objects. More sophisticated software might calculate the geometric center, or centroid, for each object, to achieve much better than single-pixel accuracy, or it could match up objects from one frame to the next, thus generating flight paths and measuring velocity. Extrapolate a track into the future and you can even predict where an insect will be—at least until its next zig or zag.

If you have more than one camera looking at the same target area, you can experiment with stereo imaging to estimate the range of each insect. The OpenCV software library (http://opencv.willowgarage.com) has many prebuilt functions for tracking and ranging.

Who flies there—friend or foe?

Let’s assume you don’t want to shoot down bumblebees, scare off hummingbirds, or freak out the neighbor’s cat while annihilating mosquitoes. How do you pick out the pests? The first test is size: Anything bigger than, say, 2 centimeters is not a mosquito (except perhaps in Minnesota, where the mosquito is reputed to be the state bird).

Another useful check is whether the target is isolated, with clear pixels on all sides. This test will rule out fingers, tree branches, or other objects that are thin but long. It should also prevent your fence from shooting at anything that’s too close to a large object, like your neighbor.

You may want to tune your system to reject things flying too fast (any more than 1 meter per second, unless there’s a breeze) or too slow. And anything moving vertically downward is more likely to be a raindrop or a falling leaf than a mosquito.

Simple filters such as these may be sufficient for backyard use. Although you don’t want to wipe out beneficial insects wholesale, it is probably okay to shoot gnats, flies, and small moths that attempt to crash your barbecue. Certainly the fence will inflict no more damage on the local ecology than an insecticide spray or an electric bug zapper. For large-scale malaria control, though, we need to be more selective (and more energy efficient), and so we have one more filter: We check how fast an insect beats its wings (see Web-only sidebar, ”Hold Your Fire Until You Hear the Beat of Their Wings”). This allows us not only to accurately identify the mosquitoes but also to tell in a split second whether they’re male or female. That way we can conserve power by sparing the lives of the males, which do not suck blood.

Ready, aim…

An insect has now wandered, unsuspecting, into the forbidden zone, and your program has decided it’s a bad bug. It’s time to point your death ray at it.

How fast and accurate your aiming system needs to be, and how large your killing pulse is, will depend on the geometry of your fence. As an example, if you want to be able to place a lethal pulse of energy anywhere within a target zone that’s 4 meters high and 10 centimeters wide (at a distance of 10 meters), then your beam must be able to swing 22 degrees up and down and about half a degree from left to right.

The most common way to steer a laser beam is to use galvanometers. A galvo is essentially a simple motor with a mirror mounted on the shaft. Drive a current through the galvo and the mirror will rotate by an amount proportional to the current. A built-in encoder feeds back the mirror position for closed-loop control. Individual galvos are often mounted in pairs to provide x-y motion.

High-quality galvos are carefully matched to their drive circuits and tuned by the manufacturer to provide maximum speed without overshoot or oscillation. They can be expensive: The Scanlab units we use—designed for laser marking and micromachining systems—cost upward of US $10 000 new. We managed to find some at an online industrial auction for about $500 each (although we then had to buy a controller card for close to $2000). But lower-cost galvos, designed for laser light shows, are widely available.

You can also experiment with other beam-deflecting techniques. Acousto-optic modulators deflect narrow laser beams over small angles (up to a few degrees) at high speed. Yet another approach is to rotate a pair of prisms relative to one another.

Whatever strategy you choose, keep in mind that the laws of diffraction will limit how well you can focus your killing laser. So the smaller the spot you want to produce, the wider the beam must be before it enters the focusing optics.

If, for example, your kill laser is an 808-nanometer infrared laser diode (with a high-quality single-mode beam), then a 4-mm-wide beam will remain nearly constant in diameter over 10 meters and will illuminate a 4-mm spot on the far fence post. But if you want to focus that beam on a 1-mm spot at 10 meters or on 4 mm at 40 meters, you’ll need to start with a beam that’s 16 mm in diameter.

Photo: Ryan Smith/Intellectual Ventures

LIGHTS, CAMERA... Here’s some of the equipment needed to image a tiny bloodsucker flying at up to 1 meter per second.  Click on the image for the full illustration view.

In any case, you’ll need a beam expander to enlarge the narrow beam produced by your laser, as well as galvo mirrors or other aiming optics big enough to accommodate the beam. Or you can deflect the beam first, then magnify it, which means you’d need to trade smaller galvo mirrors for bigger (but stationary) lenses. In our test system, the large output lens is actually a low-power telescope, which doubles the beam diameter (see photo, ”Lights, Camera...”).

For the simplest aiming system, use a dichroic (wavelength-selective) beam splitter and arrange the camera and galvo so they share a common optical center. Then you don’t need to know the distance to your target; just as if you were sighting along a rifle barrel, what you see is what you hit. You’ll still need to calibrate the system to match up the camera and beam-steering coordinates, though. If everything is linear, that’s easy; you just need to steer the laser to three different spots and find the corresponding pixels on the camera.

If your camera can’t see your laser wavelength, you’ll need a way to find where the laser is pointed, such as an infrared-sensitive card, and mark the spot in a way the camera can see. A little algebra will give you the offsets and scale factors you need to point your laser accurately.

Usually, though, life won’t be so simple. Either your camera or your pointing system may be nonlinear; for instance, many camera lenses have ”pincushion” or ”barrel” distortion, which is really a change in magnification with the angle. You can measure the distortion by pointing your camera at a calibration grid and noting the nonlinearities. Or you can just move the beam a few more points, make a table of galvo input versus pixel position, and let your software interpolate between table entries. It’s worth the time to set up an efficient alignment process, though, because you’ll have to calibrate each camera, and you’ll need to recalibrate each time you change your optics or move a laser and camera relative to one another.

If the camera and beam deflector don’t share the same pivot point (as in our test system, where the camera lens and beam-output lens are a few inches apart), you’ll end up with a parallax error whenever you rotate the line of sight: A bug that shows up in a particular camera pixel will need slightly different laser-beam pointing, depending on how far away it is. I mentioned stereo imagery above as one way to find the range to bug; astute readers will no doubt think of other approaches.

Image: Ryan Smith/Intellectual Ventures

A PEST’S LAST MOMENTS: The laser pulse [purple dot] induces an instant case of “heatstroke” that causes the critter to burn and crash.

Blast ’em!

Now comes the fun part: zapping the nasty little things. How much of a wallop should your laser pack? We’ve yet to find an engineering handbook—or even a paper in the entomological literature—that specifies the energy that’s lethal to mosquitoes. Certainly it must be less than about 2 joules, because that’s enough to boil the water inside a 1-milligram bug.

So we at the Intellectual Ventures laboratory have undertaken the world’s largest effort to discover what it takes to kill a mosquito using various wavelengths, pulse lengths, and fluxes of light. Our conclusion: For Anopheles stephensi (not your typical backyard pest but a close relative of the malaria-carrying strains), a few tens of millijoules, delivered within a few milliseconds, will cause most mosquitoes to expire within 24 hours. The blast gives them a sudden high fever, but not an obvious injury. Even under a microscope it isn’t clear what they actually die of. Let’s call it heatstroke.

We’ve shot high-speed video of mosquitoes being zapped with 50 to 100 mJ of light and found that although they keep flapping their wings, they lose attitude control and fall out of the sky. Boost the energy further and wings burn through or fall off, bodies emit puffs of steam, legs and antennae char, and so forth. Yes, it is a waste of laser energy, but revenge is sweet.

For a backyard system, the safest route would be to insist on ”eye-safe” lasers, which emit wavelengths that the eye will not focus onto the retina. Alas, there are few inexpensive high-power lasers at eye-safe wavelengths, other than carbon dioxide lasers, whose wavelengths are so long (10.6 micrometers) that they require very large optics to focus over any distance. We have used an eye-safe near-infrared fiber laser operating at 1570 nm to kill mosquitoes, but even with diligent bargain hunting at surplus stores, you’re unlikely to find a comparable laser for less than several thousand dollars.

Ultraviolet lasers (shorter than 400 nm) are safe for the eyes, and pulsed UV light seems to be quite good for killing bugs, but they are also expensive. In addition, shorter UV wavelengths, particularly 266 nm (sometimes used for laser machining), can cause severe photochemical damage, including cataracts, even though they aren’t focused by the eye.

Therefore, if you can observe safety precautions—particularly wearing goggles at all times if there’s a chance of a stray beam—the best option may be a visible or near-infrared diode laser, or perhaps an older flashlamp-pumped laser. (SSY-1 flashlamp-pumped neodymium-doped yttrium aluminum garnet lasers, which can put out about one mosquito-lethal pulse a second, are often available on eBay for around $100.) Of course,making your guests wear FDA-approved goggles may put a crimp in your barbecue, but so would a cloud of thirsty mosquitoes.

We’ve found it helpful, as we assemble and align all the components, to combine a visible guide beam with the main laser beam, a color-selective beam splitter, or a filter. This makes it much easier to see where the laser will go. If your system has lenses downstream of the galvo, though, you will have a more complicated job aligning everything, because beams of different wavelengths will be steered and focused differently.

Before pressing the big red button, make sure your safety systems are working. Double-check that the kill laser won’t fire if there’s anything bigger than a mosquito in the way. Cover the optical path with an interlock to keep unauthorized folks away from beam paths (and high voltage, if any) inside.

All ready? Then turn everything on and wait for your first invader to breach the photonic plane. Track! ID! Aim! Fire!

Intellectual Ventures developed the photonic fence to help in an epic struggle against malaria and other vicious insect-borne diseases. But this technology could prove valuable in other roles as well, such as protecting crops from airborne pests or simply tallying insect populations rather than reducing them.

We don’t really expect many Spectrum readers will build a photonic fence for their backyards—although given the number of people whose first reaction to the concept is ”How soon can I get one?” we wouldn’t be shocked to hear that some of you do. Our compliments: You will be helping to make the world a better place. Or at least a less itchy one.