Building Safer Cars

This is part of IEEE Spectrum's special report: Critical Challenges 2002: Technology Takes On

In the future, cars will help make the world's roads nearly accident free. Humans are fallible: we get sleepy while driving at night, do dumb things like put on makeup or shave while creeping along in bumper-to-bumper traffic, or look away from the road to adjust our car radios. But cars will soon make road travel safer by looking over drivers' shoulders, so to speak, keeping their attention from being pulled away from the road, and finding ways to reduce the hazard should a driver's focus stray.

To this end, researchers envision smart systems that give the driver "the right information, in the right way, at the right time," to quote Mike Gardner, director of Motorola Laboratories' Human Interface Lab, Phoenix, Ariz. At Motorola, Delphi Safety & Interior Systems, Troy, Mich., and DaimlerChrysler's Ulm Research Center in southern Germany, R&D programs are advancing toward a smart car capable of reducing the number of stimuli, some of them simultaneous, to which a driver must react, or taking over elements of the driving task such as braking or steering. These and other companies are developing adaptive cruise control (ACC) systems, which combine radar- or laser-based sensors that scan the road ahead with throttle and brake actuators, to maintain a safe, preset minimum distance between cars in the same lane [see "Keeping Cars from Crashing," IEEE Spectrum, September 2001, pp. 40-45].

ILLUSTRATION: STEVE STANKIEWICZ

Sensors That Can Make Cars Safer: Cars will have to manage input data from myriad sensors, and make split-second decisions that might involve taking control from the driver. Here, when forward collision warning senses that a crash is imminent, data from body mass and position sensors in the cabin [inset, bottom] instantly adjust the amount of force with which air bags are deployed and seat belts are tightened. Click to enlarge.

Now in its early stages, the work will take shape over time, as innovations from around the world are combined into ever more refined subsystems with sensing and control capabilities. By the end of the decade, these subsystems will allow cars not only to periodically take over tasks from the driver but also to ensure that the driver is not overwhelmed by a deluge of information from the road and from electronic devices in the cabin [see illustration], let alone by sleep.

Motorola's, Delphi's, and DaimlerChrysler's wish lists for these smart cars are very similar in what they will sense and tell the driver. In order to perform tasks such as steering, maintaining following distance, or alerting the driver who is not paying close enough attention to the road, cars will need to know, physiologically speaking, what the driver is up to. Sensors in the passenger cabin will gather this data, monitoring, for example, his or her movements, eye-blink pattern, and respiration. This data will be combined with information from sensors measuring such behavior as the number and the severity of the steering corrections the driver uses to keep the car in its lane.

All this data will be screened for signs of a distracted or disabled driver by processors programmed with neural network algorithms. These will assess whether the driver is about to doze off or is preoccupied with tasks other than driving. Engineers even imagine cars some day attempting to rouse drowsy drivers by causing their seats to vibrate, lowering the windows to provide a gust of cool air, or simply telling them to pull off the road.

Systems designed to monitor many outside conditions are also being developed. ACC and forward-, rear-, and side-impact warning systems will work with other vehicle-based sensor systems that warn of wet, frozen, or snowy roads or improper tire pressure. Communications systems will monitor local weather broadcasts and will also alert the driver to upcoming hazards (say, an oil spill or a major collision) relayed from the roadside by digital short-range communications.

Data of this type will be fed into so-called workload management systems--control centers of systems designed to reduce driver distraction. Suppose data from a system tracking the driver's eye movements indicates that she is looking not at the road but at the center console, while data from the GPS mapping system reveals that she is attempting to chart a new route. At that point the workload management system might take over, telling the driver something like, "For safety reasons, navigation information will be presented as audible prompts," and shutting off the navigation system's display. Or someday a driver may have a mobile phone call abruptly disconnected should the car sense impending danger.

Two design philosophies

An important distinction between Motorola's system and those proposed by DaimlerChrysler and Delphi will be in their allocation of driving tasks. The latter two will decide which driving tasks should be performed by the driver and which should be delegated to the car, whereas Motorola engineers, fearing that the transfer of control between driver and vehicle may itself breed accidents, will limit its system to controlling the information the driver receives [see "Cars Won't Drive Themselves"].

Motorola's Driver Advocate System, said Gardner, ranks each type of information on the basis of its relative importance. Naturally, information critical to the driver's safety, such as a warning of an impending collision, is given highest priority. Lower-priority information, such as a stock quote made available through the car's Internet connection, is held in a queue if the car senses the driver is preoccupied. Theoretically, if the car senses it is rapidly accelerating, as when entering an expressway, Motorola's driver advocate may transfer an incoming cell phone call directly to voice mail, while flashing the position and relative speeds of nearby vehicles on, say, a head-up display. When the system senses an easing of the driver's information load, it could present the items in the queue, choosing the best distraction-free way to deliver them.

The Delphi System, like Motorola's, will also limit the number of stimuli to which the driver must respond, but will go further and, if need be, take over the driving task. If it senses that the driver is not alert, it will spring into action, first increasing the distance between cars maintained by the ACC. According to a 1992 study by Daimler-Benz, this alone could prevent nearly 90 percent of rear-end collisions.

The Delphi system will also, in a limited set of circumstances, override the driver's actions to correct for common errors like over- or understeering on turns or driving too fast when the road is wet. If the driver continues to depress the accelerator after repeated visual or audible warnings that, say, a vehicle is zooming in from a cross street, the safety system will take over, slowing the car down.

An automated decision could even steer a car out of trouble. Steer-by-wire is a system that has appeared in the concept vehicles produced for exhibit at auto shows by carmakers such as DaimlerChrysler and BMW. The experimental technology [see "By-Wire Cars Turn the Corner," IEEE Spectrum, April 2001, pp. 68-73] eliminates the mechanical connection between the steering wheel and the road wheels and replaces it with electronic links and actuators that respond to driver input. Should the car's antilock braking system be unable to restore traction on a wet road, the vehicle will be able to override a driver's panicked wrenching of the steering wheel by turning the wheels in whatever direction its traction recovery system, collision avoidance sensors, and GPS mapping system suggest will allow the car to regain footing or get to the side of the road without causing an accident.

DaimlerChrysler's plans are even more ambitious. The carmaker says it is a few years away from perfecting a system that is the closest thing yet to automated driving. Engineers at the Ulm facility have equipped an E-Class Mercedes-Benz sedan with stereo video cameras and an image-processing system that together can generate a 3-D image of everything in the car's field of view--including pedestrians--in 100 ms. The imaging system's software architecture is arranged as a series of modules that focus on separate elements in a car's environment. Vetting images this way will allow cars to maintain a safe distance from the curb or road edge, recognize lane markings as well as moving and stationary vehicles and pedestrians, and respond to road signs, traffic lights, crosswalks, and directions painted on the pavement.

Watching out for speeders

Europeans are jumping ahead with field trials of a technology that overrides the driver's actions. A case in point is the Intelligent Speed Adaptation (ISA) system sponsored in the UK by Leeds University and the Motor Industry Research Association (now MIRA Ltd.), Nuneaton, Warwickshire. An in-car GPS system programmed with detailed road maps produced for the project tracks the vehicle, telling the ISA if the car enters, say, a 50-km/h zone. If the vehicle doesn't slow to the posted speed, the ISA system can give a warning or, when linked to the vehicle's fuel supply, ignition, and braking, can slow the car automatically.

Also under way is the development of systems that are more advanced than current ACC but less complex than Delphi's and Motorola's. A good illustration is the Automotive Collision Avoidance Systems (ACAS) program, a study conducted by Delphi-Delco Electronic Systems, of Kokomo, Ind., and General Motors with funding from the U.S. National Highway Transportation Safety Administration (NHTSA). ACAS has successfully demonstrated a combination of ACC and forward collision warning.

Points to Ponder

LIKE CLOCKWORK Every minute, on average, at least one person somewhere in the world dies in a car crash.

AT A LOSS Annual costs associated with crashes (like hospital bills and damaged property) total nearly 3 percent of the world's gross domestic product (GDP). In 2000, the GDP exceeded US $31.3 trillion.

Today's ACC systems rely on yaw-rate and vehicle-speed sensors to focus the car's forward-facing radar or laser devices. This arrangement keeps good track of the car ahead of the host car but is less effective if the host vehicle is on a straight section of road while the vehicle ahead has entered a curve or vice versa. So for the past two years, prototype vehicles that go a step further have been tested on simulators and test tracks by Delphi and GM, plus their partners, Hughes Research Laboratories, the Environmental Research Institute of Michigan, the University of California at Davis, and Systems Technology. To improve the focus of the forward collision sensors, the team has added cameras that track a vehicle's position in the lane (and may later be used for warnings about lane-keeping and road departure) plus a GPS data map that continuously updates the car's location and reports the shape of the road ahead. This enhancement can also be used to adjust the aim of a car's headlights.

Making the decisions

Sorting out this data (plus inputs from the ACC system) and deciding how the car should respond is the job of what Ron Colgin, a GM engineer who is director of the ACAS project, calls a data fusion package. It is essentially a computer programmed with algorithms that govern whether the car will issue a warning in a given situation. This package is a likely progenitor of future workload management systems that will coordinate streams of data coming from dozens of sensors.

According to Colgin, by the end of the year the ACAS program plans to have built 13 consumer-ready vehicles equipped with ACC and forward collision warning. Each will present four collision-warning levels to the driver on a head-up display projected on the windshield. The final-stage alert (when the interval between the host car and the one ahead is 1.6 seconds or less) will also be audible.

The Australian National University in Canberra, with assistance from AB Volvo, Göteborg, Sweden, also has under way a system for sensing where a driver is looking. FaceLAB is a face- and gaze-tracking system that captures and processes facial images using a monochrome stereo camera hardwired to a paperback-book-sized Pentium III 1-GHz workstation running Windows 2000. Proprietary algorithms use the image sequences to home in on facial landmarks such as the lips, nose, and eyes. This filtering, said the system's developers, yields head position and orientation measurements accurate to within 1 mm and 2 degrees.

Once the system has acquired enough data to determine the head position, it focuses on the eyes. Each of the two image-capturing devices generates a data stream for each eye, yielding four sources that the processor can use to estimate where the driver is looking. Should the driver wink or rub one eye, or if bright light saturates the video image on one side of the face, enough data is still transmitted to make a good gaze-direction estimate.

To keep drivers looking up, the Eye Cue Head-Up Display produced by Delphi has already been integrated into a few of General Motors' high-end vehicles. Head-up displays introduced last autumn by Siemens VDO Automotive AG, Frankfurt, Germany, and Microvision Inc., Bothell, Wash., are being reviewed by several auto manufacturers.

Challenges ahead

With all this work, researchers are only just beginning to plumb the depths of driver distraction. The O'Jays, a soul music recording group that had its heyday in the 1970s, used to sing, "Your body's here with me, but your mind is on the other side of town." Likewise, a driver whose eyes face forward and whose hands rest properly on the steering wheel--someone who is, by biosensor standards, awake and alert--can be seething about a work project, thinking about errands, or organizing tomorrow's to-do list.

Penetrating these and similar states of distraction is by far the most difficult challenge for researchers. Advances in medical electronics have recently begun to give engineers dreaming of accident-free cars a clue as to what goes on in the human brain during driving. But these advances and other strides made by experts in human factors and cognitive studies notwithstanding, we are still a long way from a system that can read a driver's mind.