Games Soldiers Play

Weeks before U.S. pilots took to the skies above Afghanistan last October, they had a pretty good idea what they would see there. Already they had logged many hours doing virtual fly-throughs over the rugged mountain terrain, using a mission rehearsal system known as Topscene (for tactical operational scene). Built for the U.S. Department of Defense by Anteon Corp., Fairfax, Va., Topscene combines aerial photos, satellite images, and intelligence data to create high-resolution three-dimensional databases of a region.

Seated at computer consoles running on Silicon Graphics Onyx processors, pilots could visualize flying from ground level up to 12 000 meters, at speeds up to 2250 km/h. The detailed renderings, showing roads, buildings, and even vehicles, helped them plot the best approach, scout for landmarks, and identify designated targets.

Topscene is just one of many powerful new simulation tools that the U.S. military is using to prepare soldiers and their leaders for battle. Over the last three decades, sophisticated computer modeling and graphics, faster processor speeds, and advances in artificial intelligence have gone into building simulation technology that can create a reality that stops just short of war.

In turn, the use of simulators has helped bring about a sea change in military training. Troops today practice exhaustively, taught by simulators not only how to use their ever more complex equipment, but also how to work in teams, move efficiently through a battlespace, and negotiate a wide range of conflicts, which may or may not involve military force.

Simulation also gives military and political leaders insight into potential conflicts. Commanders can now recreate on computer the complex choreography of thousands of soldiers, weapons, vehicles, and aircraft moving across a battlefield that extends over thousands of square kilometers. In this way, military decision-makers can test strategic options before launching a campaign in earnest. They can also assess the performance of new weapons systems under consideration.

The result has been nothing less than remarkable. Low U.S. casualties in Desert Storm, the Balkans, and now Afghanistan stem in large part from the growing use of training simulators, according to a task force of the U.S. Defense Science Board, whose 35 civilian members advise the Secretary of Defense on matters of military R&D. In its 2000 report, "Training Superiority and Training Surprise," the task force concluded that "the new combat training approach invented 30 years ago develops, without bloodshed, individuals and units into aces."

The push toward training simulation has also spawned a huge industry. According to the trade publication Military Training and Simulation News, the U.S. Department of Defense spends about US $4 billion each year on simulation and training equipment. No other country has invested nearly as much.

And that is just the beginning. Today, the Microsoft Xbox and Sony Playstation 2 game consoles are being adapted for distributed and networked military gaming. Meanwhile, an Army-sponsored group of artists, Hollywood special-effects experts, and researchers at the University of Southern California are working on the next generation of military trainers: immersive virtual-reality environments akin to the "Star Trek" holodeck, in which real soldiers interact with synthetic yet life-like actors.

"The shift from live range training to computer-based training is fundamentally changing the way we prepare our soldiers for the future," noted W. H. ("Dell") Lunceford Jr., director of the Army Model and Simulation Office in Arlington, Va. "Every soldier today needs to understand the value and the pitfalls of simulation, just as he or she must understand military science."

The limits of live training

Live training is still the most common way of readying troops. The U.S. Navy pioneered this approach after combat data revealed that a pilot's first few encounters with the enemy tended to be the deadliest; those who survived that early phase were more apt to survive in general. Thus was born the Navy's Top Gun school in 1969. Located originally at Miramar Naval Air Station in San Diego, Calif., and now part of the Naval Strike Warfare Center at Fallon Naval Air Station in Nevada, it trained pilots in conditions as close to real combat as possible.

All the U.S. armed forces now have similar live-training sites. In recent years their effectiveness has been boosted by advanced electronics and instrumentation. At the urban warfare training village at Fort Polk, La., for instance, video cameras record the action as Army Rangers and Marines fight building to building and room to room. A form of laser tag called Miles--for Multiple Integrated Laser Engagement System--identifies hits and misses.

At the Army's National Training Center in Fort Irwin, Calif., mock armor battles involving thousands of troops rage across the Mojave Desert, and every vehicle is tracked by the global positioning system (GPS).

But maintaining such a vast training infrastructure is costly. Just transporting a unit to a remote location for training can run several million dollars. Typically, units spend just three weeks at the site, returning every 18 months or so. The equipment is also pricey. A single round for the infantry-carried Javelin antitank weapon costs over $30 000. A Javelin simulator, in contrast, costs about the same but can be fired over and over.

Then, too, the land available for live training is limited. While some air-launched missiles can travel 40 km, few training areas are that big. Nor do local residents much appreciate live ammunition used so close to home, as was made clear in Vieques, Puerto Rico.

Another issue is the sophistication of new weapons systems. A pilot operating a Predator unmanned aerial vehicle (UAV), for example, sits hundreds or thousands of kilometers from his vehicle. He must learn to navigate through a camera lens and compensate for the delay between his command and the aircraft's response. Not surprisingly, pilots new to UAV flight crash their vehicles a lot. Getting a feel for the nuances of telerobotic flight is better done with computer simulation, rather than live.

Given live training's limitations, the military is relying more and more on virtual simulations. Such systems have proven effective for enhancing motor control, as in driving a tank or firing a rifle; decision-making, as in calculating the resources needed for combat; and leadership, as in responding to an ambush.

Virtual war

Even as computer simulations achieve greater realism, military operations themselves have become more computer-driven, and more synthetic. From the viewpoint of a Predator pilot, a real combat mission feels much like a simulated one.

The same holds true even at the highest levels of command. In his memoirs, It Doesn't Take a Hero (Bantam, 1992), General H. Norman Schwarzkopf discussed the training exercises conducted just before the Gulf War. Recalling the uncanny similarities between the simulated war game, dubbed Internal Look, and the real thing, Schwarzkopf wrote:

"We played Internal Look in late July 1990, setting up a mock headquarters complete with computers and communication gear at Eglin Air Force Base in the Florida panhandle. As the exercise got under way, the movements of Iraq's real-world ground and air forces eerily paralleled the imaginary scenario of the game....As the war game began, the message center also passed along routine intelligence bulletins about the real Middle East. Those concerning Iraq were so similar to the game dispatches that the message center ended up having to stamp the fictional reports with a prominent disclaimer: 'Exercise Only.' "

Over the two-week course of the exercise, U.S. Central Command staff, based at MacDill Air Force Base in Tampa, Fla., endured all the emotional highs and lows of battle--what virtual- reality researchers call "presence." Lessons learned from Internal Look shaped the defensive plan for Desert Shield, and drove home the power of computer simulation in preparing for war.

Such constructive simulations, more commonly known as war games, have become a common tool for military decision-makers who are exploring strategic options. These simulations often involve hundreds of players and are based on complex computer-driven models of the battlefield. Real people supply the inputs and the key, high-level decisions; the synthetic forces they command execute the outcomes. Though constructive simulations are the heirs of manual board games [see "From Kriegspiel to Simnet,"], the sheer scope of operations today requires computers even for the lesser tasks of tracking troop movements, logistics, and intelligence flow.

The next generation of constructive simulations are being developed under the auspices of the Joint Simulation System (JSIMS) program. JSIMS is an ambitious effort to knit together disparate war-gaming systems now used by the U.S. Army, Navy, and Air Force, as well as the intelligence community.

In JSIMS, models representing the land, maritime, air/space, and intelligence agency domains interact in a Joint Synthetic Battlespace (JSB), which encompasses the three levels of war--tactical, operational, and strategic. The first major test of the program will come this summer, as part of an exercise called Millennium Challenge. The event will bring together about 15 000 personnel from all the armed forces for combined computer and live operations in the air, on the sea, and on land, extending across southern California and Nevada.

With such a large-scale model, many details get lost, however. JSIMS may represent an entire tank company as a single entity, and its terrain databases have a resolution of only 100 meters.

A finer-grained view is offered by the OneSAF program, being developed for the Army by an Orlando, Fla.-based division of Science Applications International Corp. SAF is short for Semi-Automated Forces, and the program simulates the behavior of hundreds of tanks and foot soldiers, much like the AI characters a player encounters in video games. In a OneSAF simulation, each soldier or tank, representing either friendly or enemy forces, maneuvers autonomously over 1-meter-resolution terrain. Human players thus get a more realistic picture of battlefield conditions.

Training Generation Xbox

Computerized military simulators have also taken hold further down the chain of command. Not too long ago, cadets at the U.S. Military Academy at West Point, N.Y., only read about military strategy; war games in the field would begin after graduation. Today's academy cadets do battle in virtual M1 tanks and infantry using a commercial game called Steel Beasts. The game lets them practice individually or in Internet-linked groups; they can face down a computer-simulated enemy, or another squad of cadets.

Employing these computer simulations has proved to be a smooth transition for younger generations of soldiers, who, after all, were spoon fed on Nintendo and computer games. And they expect more: current PCs can do as much as the supercomputers of a decade ago, while the computational muscle of game consoles now far surpasses that of a DEC VAX.

Only a huge investment could have endowed the Xbox with its 80 gigaflops of computing power. Microsoft Corp., Redmond, Wash., reportedly developed it at a cost of over $2 billion. Rather than reinvent the wheel, the Pentagon decided to customize it and other consoles and PC-based simulations for its own purposes. The idea is not entirely new. Back in the early 1980s, the Army briefly experimented with Atari Inc.'s tank game Battlezone in the hopes of enhancing a gunner's eye-hand coordination. But such efforts never made it to the mainstream.

Perhaps the first commercial 3-D game to be used for group military training was the fantasy game Doom. To create its own version, the U.S. Marine Corps Modeling and Simulation Management Office edited the commercial game into an urban combat scenario. Nonhuman characters--the game's AI bad guys--were converted from monsters into opposition forces. Marine Doom, as it came to be known, taught concepts like properly sequencing an attack, protecting the rifleman, conserving ammunition, and observing the chain of command.

Other games have since been similarly altered. The Army, for instance, commissioned NovaLogic Inc., Calabasas, Calif., to include in its popular Delta Force 2 game features found in the Army's Land Warrior. Worn by infantry soldiers, Land Warrior is a complex, integrated system that includes a computer and radio, a GPS receiver, a helmet-mounted LCD display, and a modular weapons system that adds thermal and video sights and laser ranging to the standard M4 carbine or M16A2 rifle. The Army is now evaluating the game's effectiveness compared to other training methods.

More recently, the Marine Corps has developed training games that run on networked laptops. That feature allows Marines to practice a variety of combat roles (pilots, riflemen, tank drivers) and maintain their skills even while on ship.

In all such virtual trainers, the main advantage over live training is time. A commander can restart a training scenario at any point, and the training can be repeated until the soldier or unit performs the operation correctly.

If the thought of soldiers honing their skills on PlayStations still seems peculiar, consider this. In an extensive study of the effectiveness of Microsoft's Flight Simulator, the U.S. Navy found that students who used such microsimulation products during early flight training were 54 percent more likely to receive above-average scores in real flight tests than their peers who never used the software. What's more, most Navy flight-training students now practice on Flight Simulator at home.

Hollywood meets the military

Some of the most innovative war-gaming technology is being developed by a small research institute at the University of Southern California. Founded by the Army in 1999, the Institute for Creative Technologies (ICT), in Marina del Rey, brings together university researchers with entertainment industry veterans and artists in the search for new kinds of virtual experiences. The goal is to create immersive learning environments so compelling that participants will react as if they were real.

In the ICT's Mission Rehearsal Exercise System, for example, a lieutenant takes command of a platoon in a Bosnian village. As the program gets under way, a Humvee injures a young boy, and tensions start to mount as angry townspeople threaten to riot. The lieutenant must soothe the grieving mother and also take advice from his trusted sergeant.

Only the lieutenant is human; the villagers are synthespians (synthetic actors), and the sergeant is a virtual intelligent tutor. Such thinking, seeing, talking simulations are being developed to represent smart opponents, allies, and even robots. The goal is to immerse trainees physically, intellectually, and emotionally.

Developing emotional intelligence is not usually considered the stuff of boot camp. Yet, increasingly, it is part of a good soldier's armament. During the Cold War, the United States really had only one conflict to train for: a total war with the Soviet Union, to be fought on the plains of Central Europe. Today's soldier, in contrast, must be ready not only for a major firefight but also for a wide variety of other crises--peacekeeping, hostage rescue, urban warfare, and counter-terrorism.

To make the ICT's synthespians exude human behavior, researchers are focusing on software agents that model human performance, explained Jonathan Gratch, an artificial-intelligence expert at ICT. "A wealth of empirical research has revealed a complex interplay between emotions, cognition, and behavior," he added. "One's emotional state may impact decision-making, actions, memory, attention, voluntary muscles, and so on, which in turn can influence the emotional state."

Behind the scenes

Creating a virtual-reality (VR) environment that convinces the eyes and ears is another matter. ICT's computer graphics rely on a technique pioneered by Paul Debevec when he was at the University of California at Berkeley in the mid-1990s. Known as image-based rendering (IBR), it generates images directly from photos and other images, rather than building them from geometric primitives such as polygons. For military simulations, IBR could prove useful for quickly rendering high-resolution terrain and creating realistic urban environments.

One of the trickiest problems in creating VR simulations is blending human images into digital stage sets. The custom, as Hollywood special-effects buffs know, is to film human actors against a green screen, and then compositing the images onto computer-generated scenes, perhaps with animated characters. But it's nearly impossible to match the background scene's lighting with that used to film the green screen shots.

ICT's lightstage gets around this problem. It can recreate the lighting condition of any real or synthetic setting, allowing the human actor to perform in the virtual light as he is being filmed. The resulting human character can then be seamlessly incorporated into the VR setting.

ICT's sound group, meanwhile, is aiming to create a 3-D aural environment that remains consistent even as the listener moves around in the simulation. "In recent years, the audio industry has focused on increasing the sampling rate and word length for digital audio," said Chris Kyriakakis, a researcher at USC's Integrated Media Systems Center. "While the new standards--96 kHz and 24 bits--are technically impressive, the benefits [do little to] enhance the experience for the listener."

Kyriakakis' team has instead boosted the number of channels. With USC professor Tomlinson Holman (inventor of the THX system for film sound), they built the world's first 10.2-channel immersive audio-rendering system. Placement of the system's speakers is paramount. Two speakers in front expand the soundfield vertically. The traditional left, center, and right speakers are augmented by two wide speakers at 60 degrees on each side. Two surround loudspeakers are placed at 110 degrees each to the left and right, and another at 180 degrees behind the listener. And directly to each side are two low-frequency subwoofers.

"The result is an immersive experience that places the participant in the middle of a town square, inside a Humvee, or under a low-flying helicopter with a sense of realism that is closely coupled to the wide field-of-view picture on the screen," Kyriakakis said.

To the trenches

Clearly, the ultimate test of such research will be not just how sophisticated the technology is, but how well it prepares our military for an age of uncertainty and new dangers. In real-world conflicts, can our soldiers innovate, understand the consequences of their actions, win wars, and keep peace? Does the simulation accurately reflect 21st century conditions ranging from Bosnia to Afghanistan, and thereby help commanders plan for the next challenge? If the answer to these questions is yes, then the technology will have done its job.

Jean Kumagai, Editor