Pothole Detection for the Visually Impaired

An early prototype applies machine learning algorithms to identify uneven surfaces

3 min read
Pothole Detection for the Visually Impaired
Photo: Jeff Sherer/Getty Images

Over the years, researchers and companies have invented plenty of devices to help people with visual impairments avoid objects such as a desk or chair. Many of these gadgets used ultrasonic sensors to detect such hazards. Just to name a few, there was the discontinued Pathsounder (which hung around a person’s neck), the cumbersome NavBelt (worn around the waist), and the wheeled GuideCane.

However, there’s another type of obstacle that lurks underfoot—slight depressions in the ground such as steps, curbs, or divots that can cause a person to stumble or a wheelchair to suddenly turn awry. For these subtle features, most high-tech detection systems don’t work very well.

“There’s nothing in existence that we know of that detects non-protruding hazards,” says Elaine Wong, an electrical engineer at Australia’s University of Melbourne.

With the help of two nonprofit partners and funding from the Ian Potter Foundation, Wong has crafted a prototype for a detection system that uses a camera paired with lasers to spot potholes and other potential surface hazards. It combines image processing with machine learning to analyze a user’s surroundings and provide auditory cues as he or she approaches an uneven surface. Wong presented her work last week at the IEEE International Conference on Communications in Kuala Lumpur, Malaysia.

The project is still very much in development. So far, her prototype correctly identified potholes at least 90 percent of the time in a very small batch of three tests that processed a total of 15 potholes.

In order for the system to truly be useful, she will need to validate these results in many more tests and further improve its accuracy, which she hopes to do by tweaking the parameters of the machine learning algorithms or adjusting the resolution of the lasers. She must also create a real time version since the image processing for the pilot was done offline. Lastly, the lasers she used only work in the dark so she hopes to switch to other lasers in subsequent versions.

With her prototype, Wong separately tested two laser patterns as a detection method: one that projects a grid and another in the shape of a crosshair. Her system records the strength of these lasers as they scan the path in front of a user and bounce back to the device. If the path is clear, the laser should return at full strength—assuming a flat surface such as a sidewalk. But if the laser must travel further in certain spots due to a sudden dip, that section of the pattern appears more faint.

A camera (currently a GoPro in HD mode) records the laser patterns at 15 frames per second. To interpret those images, Wong has paired machine learning with image processing. She developed algorithms that can make note of a peculiar deviation in the laser’s pattern and note that blip as a pothole. Her system chimes out “Pothole detected” whenever it spots one. (She considered using vibrational cues to alert users but figured it was easier for the system to just say it out loud.) Analyzing the crosshair pattern led to more accurate results than the grid in her initial tests.

Wong’s goal is to eventually develop an affordable, easy-to-use detection system for surface hazards. “You want it to be fast, you want it to be small, you want it to be low complexity,” she says.

Her first version was designed to be strapped to a walker or a wheelchair because many visually impaired people rely on these aides and it can be hard for them to also handle a guide dog or swing a cane to detect obstacles.

In her mind, the potential market for such a device is huge—the World Health Organization says about 285 million people worldwide have visual impairments. Still, Wong keeps one particular potential user in mind—her son, who was born blind. “Hopefully he’ll be proud of me when this gets off the ground,” she says.

The Conversation (0)