Eye Scans to Detect Cancer and Alzheimer’s Disease

Computer vision and machine-learning algorithms detect biomarkers in the white of the eye

4 min read
photo: Person holds up a box with smartphone camera facing his face to take a selfie of his eyes to detect early signs of pancreatic cancer via a new app
Photo: Dennis Wise/University of Washington

It is said that eyes are the window to the soul. Well, what about the window to our health?

We recently reported on two AI systems trained to detect eye diseases, specifically diabetic retinopathy and congenital cataracts. Now, we’ve found groups extending that concept to illnesses beyond the eye. Two new projects pair imaging systems with advanced software in an effort to catch early symptoms of pancreatic cancer and Alzheimer’s disease.

At the University of Washington, a team led by computer scientist Shwetak Patel created a smartphone app to screen for pancreatic cancer with a quick selfie. Developed over the last year and a half, the team recently tested their system in a clinical study of 70 people. They were able to identify cases of concern with 89.7 percent sensitivity and 96.8 percent accuracy.

While the app is not yet ready to be used as a diagnostic tool based only a single small trial, it might soon provide a tool for doctors monitor disease progression in patients undergoing treatment with a simple photograph, rather than a blood test, says first author Alex Mariakakis, a UW graduate student.

The app, called BiliScreen, monitors levels of bilirubin, a metabolic byproduct that builds up in cases of jaundice, causing a yellowing of the skin and whites of the eyes. Increased bilirubin in the blood is one of the earliest symptoms of pancreatic cancer, but it is currently detected by a blood test in a doctor’s office and done only for patients at high-risk or with other symptoms.

The BiliScreen app shows bilirubin levels based on eye color measurements The BiliScreen app detects levels of bilirubin in the whites of the eyes. Photo: Dennis Wise/University of Washington

To create an easy-to-use, non-invasive early screen for pancreatic cancer—which claimed the life of study co-author James Taylor’s father—the researchers designed a three-step system. First, users take a selfie with their smartphone using one of two accessories to control for environmental conditions: either a cardboard box to block out the light, or colored glasses to give a color reference. In the study, the team found that the box was slightly better at controlling light conditions than the glasses.

Once an image is captured, the software relies on computer vision algorithms to isolate the sclera, or white of the eye, from the skin and pupil. The clinical trial version of the software was based on an algorithm from Microsoft Research called GrabCut, says Mariakakis. A newer iteration, which works even better to isolate the sclera, uses a fully convolutional neural network to identify the most important local features in each image.

“You give the algorithm enough cases of labeled images…with people of different skin tones, eye colors, and orientation, and it can figure out where the sclera is,” he says.

Next, that image information is matched to levels of bilirubin taken from a blood draw and fed into a machine-learning algorithm to train it to detect images of concern. In the initial 70-person study, roughly half the participants had elevated levels of bilirubin and half did not. BiliScreen was able to detect those individuals with high levels with good accuracy and sensitivity, but both measures could be improved with more data, adds Mariakakis. The team is now seeking funding for additional clinical trials.

At Cedars-Sinai and NeuroVision Imaging LLC in California, researchers have developed a sophisticated camera and retinal imaging appraoch to detect early signs of Alzheimer’s disease (AD). This system, recently detailed in a proof-of-concept trial published in the journal JCI Insight, relies on a specialized ophthalmic camera that is not yet available on a smartphone.

In June 2010, the Cedars-Sinai team, led by Maya Koronyo-Hamaoui and Yosef Koronyo, made a novel discovery: Beta-amyloid protein deposits, a neurotoxic protein that builds up in the brains of Alzheimer’s patients, are also present in the postmortem retina of such patients and even in early-stage cases. Further investigation has shown that beta-amyloid deposits in the eye appear in clusters and seem to be more abundant in certain regions then others in the human retina, says Koronyo-Hamaoui.

She and collaborators at Cedars-Sinai founded a company, NeuroVision, to develop a way to detect and quantify those plaques in the eye, in the hopes of being able to see early signs of the disease before cognitive decline becomes obvious. Today, their system consists of a modified scanning laser ophthalmoscope with a special filter to capture a fluorescence signal emitted by the plaques when tagged with a marker, and advanced software to process the images.

In the JCI Insight study, the team used their imaging system on 16 live patients and the donated eyes and brains of 37 deceased patients, 23 of whom were confirmed with Alzheimer’s disease. Overall, they found a 4.7-fold increase in the post-mortem retinas of patients with AD compared to controls.

The next step is larger scale studies to see if the method is practical for diagnosing AD.  The group is conducting additional clinical trials in the U.S., Europe, and Australia, Koronyo-Hamaoui told IEEE Spectrum in an email. “We are testing larger cohorts of living patients for the possible relationship between retinal amyloid index and the gold standard amyloid-PET brain imaging and other AD biomarkers.”

She believes that the imaging could someday be adapted to less expensive cameras, but currently a doctor’s office would require a high-definition camera and image processing and quantification tools in order to use the system.

The Conversation (0)

This CAD Program Can Design New Organisms

Genetic engineers have a powerful new tool to write and edit DNA code

11 min read
A photo showing machinery in a lab

Foundries such as the Edinburgh Genome Foundry assemble fragments of synthetic DNA and send them to labs for testing in cells.

Edinburgh Genome Foundry, University of Edinburgh

In the next decade, medical science may finally advance cures for some of the most complex diseases that plague humanity. Many diseases are caused by mutations in the human genome, which can either be inherited from our parents (such as in cystic fibrosis), or acquired during life, such as most types of cancer. For some of these conditions, medical researchers have identified the exact mutations that lead to disease; but in many more, they're still seeking answers. And without understanding the cause of a problem, it's pretty tough to find a cure.

We believe that a key enabling technology in this quest is a computer-aided design (CAD) program for genome editing, which our organization is launching this week at the Genome Project-write (GP-write) conference.

With this CAD program, medical researchers will be able to quickly design hundreds of different genomes with any combination of mutations and send the genetic code to a company that manufactures strings of DNA. Those fragments of synthesized DNA can then be sent to a foundry for assembly, and finally to a lab where the designed genomes can be tested in cells. Based on how the cells grow, researchers can use the CAD program to iterate with a new batch of redesigned genomes, sharing data for collaborative efforts. Enabling fast redesign of thousands of variants can only be achieved through automation; at that scale, researchers just might identify the combinations of mutations that are causing genetic diseases. This is the first critical R&D step toward finding cures.

Keep Reading ↓ Show less