Israeli Startup’s Vision Device Can Help the Nearly-Blind Read and Recognize Faces

Independent study by ophthalmologists shows that the clip-on device dramatically improves ability to carry out daily life tasks

Photo: OrCam
Advertisement

Over 21 million adults in the United States suffer from impaired vision. Several companies are have been racing to develop wearables and implants that can improve or restore eyesight in these patients.

One such wearable vision device, made by Israeli startup OrCam, just got an enthusiastic thumbs up from eye doctors who tested the gadget on 12 legally blind patients. OrCam’s little camera device, which clips on to eyeglasses, reads aloud words on labels and signs, and names objects and faces for the wearer. Opthalmologists at the University of California Davis Eye Center published results of their study in the journal JAMA Opthalmology. They found that the device dramatically improved the reading ability of the patients, many of them elderly.

“We haven’t seen anything else comparable to OrCam’s product,” says Mark J. Mannis, director of the UC Davis Eye Center. “While the technology is sophisticated, it is easy to operate even for elderly patients for whom technology is daunting,” says Mannis, who specializes in corneal transplants. “And secondly, it’s very portable, not obtrusive, and it works very efficiently.”

That second part is a direct comparison to the existing crop of visual aids like magnifiers and smartphone apps for reading help. “Magnifiers don’t work for patients with macular degeneration where there’s essentially a big blotch on the center of vision so if they look at a face, it’s blacked out,” Mannis says. “They need to read or recognize faces and objects without needing a working retina.” This is where smartphone reading apps come in, but they have limited capabilities and aren’t easy to use.

OrCam’s camera device clips on magnetically to a user’s eyeglasses. It is connected via a thin wire to a small computer that fits in a pocket. The interface is minimal: The wearer can tap the device or simply point to an object or text that the device recognizes. Like the Google Glass, the audio goes through a bone conduction speaker so only the wearer can hear the machine’s interpretation. The device reads text and recognizes faces and objects such as buses, street lights and signs, and paper money. It comes with a library of objects that the user can add to by waving the object in front of the camera so it can see it from all angles.

This requires sophisticated artificial vision and image recognition wizardry. But unlike the tremendous computing power needed by deep learning techniques used for vision and recognition, the OrCam device uses a computer vision algorithm called ShareBoost. The program was developed by the company’s founder, Amnon Shashua, a professor of computer science at the Hebrew University in Jerusalem. The algorithm bases its decision on the most informative parts of an image, with a reasonable tradeoff between recognition accuracy and speed.

OrCam is reportedly selling the device in the U.S., UK, Israel, and Canada for $2,500, the price of a good hearing aid.

For the UC Davis study, Mannis and a colleague trained the patients to use the device for 10 tasks such as reading from a computer screen, a newspaper and a book, which they couldn’t do on their own. After using the device at home for a week, all of them could perform the tasks. A subset of 7 patients who had used visual aids such as magnifiers and smartphones could do the tasks better with the OrCam device.

One caveat is that the device does not work well in low light conditions. And contrary to the company’s claims, the device might not help the blind or those with profound vision loss. According to Mannis, it’s, “mostly because you do have to direct it to what you’re looking at.”

Retinal implants might be more relevant for the blind. The first such bionic eye, made by Second Sight, is already on the market, and a few others are on the way or being developed for human trials.

But for those suffering from low-vision due to diseases like macular degeneration and glaucoma, wearables like OrCam’s offer a bright glimmer of hope. “External wearable devices are much further along in development and testing,” Mannis says. “They’re much easier than performing implant surgery and are probably more useful for a broader swath of people.”

About the Human OS blog

IEEE Spectrum’s biomedical engineering blog, featuring the wearable sensors, big data analytics, and implanted devices that enable new ventures in personalized medicine.