The photograph is cropped close on the face of four-year-old Yael, who is smiling and looking as healthy as can be. But a computer analysis of her features says something’s not right. She has MR XL Bain Type, the computer predicts—a very rare syndrome that causes a wide range of health problems.
It turned out that the computer was right.
Yael is one of thousands of children who have contributed to the development of an artificial intelligence system called DeepGestalt that can identify rare genetic disorders based on facial features alone. The system, built by Boston-based FDNA, analyzes photographs of faces using computer vision and deep learning algorithms.
In an article published this week in the journal Nature Medicine, FDNA announced the results of a study of DeepGestalt involving 17,000 children, with more than 200 different syndromes among them. The system outperformed clinicians’ ability to identify disease in three separate experiments.
In its best performance, the AI system correctly distinguished between different subtypes of the genetic disorder Noonan syndrome in 64 percent of the cases. Clinicians looking at images of people with Noonan syndrome in previous studies identified the disease correctly in only 20 percent of the cases.
Combined with DNA sequencing, DeepGestalt could prove useful in helping to identify disease, says Yaron Gurovich, chief technology officer at FDNA. “Some people call it deep phenotyping,” he says. “It’s the ability to get accurate and deep insights on a person and link them correctly to genes that were found as problematic in a [DNA] sequencing process.”
The tool could also help standardize the methods doctors use when looking for visual signs of disease, the study authors say. Trying to describe why a person’s facial features are phenotypic expressions of a disease can be challenging. “It’s like when you look at a child and you look at the mother and you know they’re related, but you’re not able to say why,” says Gurovich. “That’s the difference between [a doctor] looking at the facial features and our Gestalt algorithm. It finds a link that we can’t really describe.”
For example, people with Cornelia de Lange syndrome tend to have a small nose, arched eyebrows, and an atypical mouth. But other syndromes, such as the one Yael has, manifest in different ways or aren’t so readily apparent. (In psychology, Gestalt theory "emphasizes that the whole of anything is greater than its parts," according to Britannica.)
This heat map illustrates facial features that influence the algorithms' predictions.Image: FDNA/Nature Medicine
How the algorithms accomplish the task is a black box—a frustrating problem in many AI systems. To get a peek into the algorithms’ methods, the researchers created a color-coded map of the “hot” areas of the face—those that influence the computer’s predictions. It provides a “visualization for our users to try to look inside the black box and understand what the algorithm thinks and how it chose its results,” says Gurovich.
FDNA has analyzed more than 150,000 cases to date. The company amassed its database by building a community platform called Face2Gene that clinical geneticists can use for free. The doctors upload images into the system (with consent from the patient) and in return get to use the platform to help them narrow down the disease possibilities of their patients.
The system provides the doctors with a short list of about 10 possible syndromes the patient might have—not so much a diagnosis, but an aid to help the doctors narrow down the possibilities. Gurovich says 70 percent of clinical geneticists worldwide are using the Face2Gene system.
Those clinicians are getting reliable results, according to the new study. In a fourth experiment, DeepGestalt analyzed 502 images and generated a suggested list of ten potential syndromes. The list included the patient’s actual disease 91 percent of the time, according to the researchers.
In the paper, Gurovich and his colleagues warn of potential for misuse of the images. “Unlike genomic data, facial images are easily accessible. Payers or employers could potentially analyze facial images and discriminate based on the probability of individuals having pre-existing conditions or developing medical complications,” the authors wrote. They suggest implementing monitoring strategies, such as recording digital footprints on a blockchain, to prevent abuse.
FDNA’s system did not require regulatory approval from the U.S. Food and Drug Administration because it’s considered a reference tool, according to the company.
Yael, who lives in Israel, has become the face of FDNA, with her picture on the home page of the company’s website. An FDNA spokesperson says Yael’s parents are actively searching for more people like Yael who have MR XL Bain Type. There is no treatment yet available for the disease.
Emily Waltz is a contributing editor at Spectrum covering the intersection of technology and the human body. Her favorite topics include electrical stimulation of the nervous system, wearable sensors, and tiny medical robots that dive deep into the human body. She has been writing for Spectrum since 2012, and for the Nature journals since 2005. Emily has a master's degree from Columbia University Graduate School of Journalism and an undergraduate degree from Vanderbilt University. She aims to say something true and useful in every story she writes. Contact her via @EmWaltz on Twitter or through her website.