Ophthalmologists vs. AI: It's a Tie

Artificial intelligence performs just as well as eye docs in diagnosing congenital cataracts

3 min read
A child with cataracts
Photo: Heydt/VWPics/AP Photo

Last week, we reported on an algorithm that recognizes skin cancer as well as the world’s best dermatologists. That computer program was trained using 130,000 images from more than 2,000 diseases. It, like most artificial intelligence (AI) breakthroughs, relied on big data.

Now, a team in China has demonstrated that AI also has the potential to aid medical diagnoses in situations where there is limited high-quality data available. An AI program trained with just 410 images of congenital cataracts (a rare disease that causes irreversible blindness), plus 476 images of disease-free eyes performed as well as doctors in diagnosing the condition, recognizing its severity, and offering a treatment suggestion.

Inspired by a 2015 research paper from Google’s DeepMind artificial-intelligence company, which described how a machine-learning algorithm beat professional human players at a series of arcade games based on minimal starting information, Haotian Lin, an ophthalmologist at Sun Yat-sen University in China, and colleagues had the idea of creating an AI agent to mine their clinical database on childhood cataracts.

Working with Xiyang Liu’s team at Xidian University, they created CC-Cruiser, an AI program able to diagnose congenital cataracts, predict the severity of the disease, and suggest treatment decisions. The program was created using deep-learning algorithms trained with the aforementioned images of affected and control eyes from children.

Then, the researchers put CC-Cruiser to five tests. First, in a computer simulation, the AI program was able to distinguish patients and healthy individuals with 98.87 percent accuracy. It also achieved above 93 percent accuracy in estimating each of three indicators of the disease’s severity: lens opacity areas, density, and location. The program also provided treatment suggestions with 97.56 percent accuracy.

Next, the team conducted a clinical trial using 57 images of children’s eyes from three collaborating hospitals in China. None of the chosen hospitals specialize in diagnosing or treating this condition, because the team hopes the platform will eventually be most useful in helping hospitals like these, which lack on-site specialists. Again, CC-Cruiser performed well: 98.25 percent identification accuracy; over 92 percent on all three severity factors; and 92.86 percent accuracy in treatment suggestions.

In yet another test of the AI’s capabilities, the program was presented with 53 low-quality cases mined from the Internet. It handled them with a high level of accuracy. But the researchers still weren’t done. After that, the program successfully identified three needle-in-a-haystack cases, correctly pointing out three cataract cases in a data set with 300 normal cases.

Finally, in an effort to simulate real-world use, they pitted the program directly against individual ophthalmologists. CC-Cruiser and three ophthalmologists—an expert, competent, and a novice—went head-to-head diagnosing 50 clinical cases. The computer and doctors performed comparably.

The program did incorrectly flag a few cases in the hospital trial, and Lin hopes that a larger dataset could improve its performance. The team plans to build a collaborative cloud platform to do so, but Lin emphasize that the technology is “insufficient” to determine the best course of treatment with 100 percent accuracy. “Doctors should therefore make good use of the machine’s suggestion to identify and prevent the potential misclassification and complement their own judgment,” Lin told IEEE Spectrum in an email.

So, it’s unlikely that CC-Cruiser will make ophthalmologists obsolete anytime soon. Especially because there is one key skill that it cannot do as well as doctors: “The human ability to communicate and interact affectively is indispensable for medical treatment,” said Lin. “The simulation of human emotion is very challenging for machine[s]. The face-to-face interaction between doctors and patients will be one of the last bastions of human intelligence.”

The team hopes that, with further clinical trials, doctors in non-specialized hospitals could use the program to identify the condition and send patients to specialized centers. Patients may eventually use it themselves and see out a specialist if there were a concern, added Lin. “The ultimate goal of artificial intelligence is to leverage it, in combination with human abilities, to make the world a better place.”

The Conversation (0)

This CAD Program Can Design New Organisms

Genetic engineers have a powerful new tool to write and edit DNA code

11 min read
A photo showing machinery in a lab

Foundries such as the Edinburgh Genome Foundry assemble fragments of synthetic DNA and send them to labs for testing in cells.

Edinburgh Genome Foundry, University of Edinburgh

In the next decade, medical science may finally advance cures for some of the most complex diseases that plague humanity. Many diseases are caused by mutations in the human genome, which can either be inherited from our parents (such as in cystic fibrosis), or acquired during life, such as most types of cancer. For some of these conditions, medical researchers have identified the exact mutations that lead to disease; but in many more, they're still seeking answers. And without understanding the cause of a problem, it's pretty tough to find a cure.

We believe that a key enabling technology in this quest is a computer-aided design (CAD) program for genome editing, which our organization is launching this week at the Genome Project-write (GP-write) conference.

With this CAD program, medical researchers will be able to quickly design hundreds of different genomes with any combination of mutations and send the genetic code to a company that manufactures strings of DNA. Those fragments of synthesized DNA can then be sent to a foundry for assembly, and finally to a lab where the designed genomes can be tested in cells. Based on how the cells grow, researchers can use the CAD program to iterate with a new batch of redesigned genomes, sharing data for collaborative efforts. Enabling fast redesign of thousands of variants can only be achieved through automation; at that scale, researchers just might identify the combinations of mutations that are causing genetic diseases. This is the first critical R&D step toward finding cures.

Keep Reading ↓ Show less