The February 2023 issue of IEEE Spectrum is here!

Close bar

Study: Medical Image AIs Need a Good “Hallucination Map”

Astute new algorithms identify false structures in brain scans

2 min read
A series of brain scans

From left to right, an example of a brain image scan with a false structure ("hallucination"), along with error maps and null space hallucination maps.

University of Illinois at Urbana–Champaign and Washington University in St. Louis/IEEE

This article is part of our exclusive IEEE Journal Watch series in partnership with IEEE Xplore.

Medical imaging techniques such as MRIs and CT scans have revolutionized the medical field, helping to advance novel therapies and provide hard data and new perspectives on the health of patients. Such noninvasive approaches allow doctors to peer into the brains and bodies of patients, detecting anything from fractured bones to brain tumors.

Yet while these imaging techniques are critical for modern-day health care, they can sometimes be deceptive—suggesting that an unusual phenomenon is present in the brain or body when in fact it is not.

This is because medical imaging devices do not record images directly. Instead, the raw data collected by the devices is analyzed by a computer, and machine-learning algorithms are used to reconstruct the images that doctors and radiologists use for diagnosing a health complication. Image reconstruction is done based on the known physics of the imaging device, along with a set of assumptions about how the final image should appear.

“However, if certain assumptions are wrong during image reconstruction, false structures may be introduced into the final image,” explains Mark Anastasio, a professor of bioengineering at the University of Illinois at Urbana–Champaign.

As one could imagine, the presence of false structures (also called “hallucinations”) in medical images could have serious consequences in diagnosing patients. “For example, a hallucination that resembles a tumorlike structure may influence a radiologist to conclude that a lesion is present in the tissue or organ being imaged, when it is actually not,” explains Anastasio.

To address this issue, Anastasio and his colleagues have been working on a new technique that can identify when algorithms for image reconstruction are likely creating false structures. It works by mapping out errors in the reconstructed image that are not attributable to the raw image data.

The research team tested their approach against three different image-reconstruction models through numerous simulations. The results, which show that the new mapping technique is effective at detecting hallucinations, are described in a study published in November in IEEE Transactions on Medical Imaging.

As future data-driven image-reconstruction algorithms are designed, the hallucination map can be used to help find and analyze errors introduced by them, ultimately helping to identify and build more accurate methods for reconstructing medical images.

Anastasio notes that, although the hallucination map helps identify the existence of false structures in medical images, it does not tell us how harmful—or even useful—these structures are with respect to a given diagnosis. “Extending the proposed hallucination maps to incorporate such information is an exciting topic for future study,” he says.

The Conversation (1)
Anjan Saha03 Feb, 2022

Brain Scanning & Functionality tests can be done by EEG,MRI,CT

Scan and PET scan as on

date. EEG tests the neurochemicals & Current flow in brain Cells. For PETs Scanning , patients has to take very mild radio active chemical injection. MRI is the safest to Scan Full body

and brain without any side effect. In CT scan high energy electron particles are used and not so safe as MRI in health aspect .In X Ray also high energy EM

waves is used.

In all these Scanning for

Brain testing and examination the normal healthy Brain and Diseased Brains Images are compared and analysed by Medical Practitioners and Doctors.

Financially the cost of Scanning increases in

the order as given below:


On health grounds X RAY & PET SCAN for brain may not be conducted.

Illustration showing an astronaut performing mechanical repairs to a satellite uses two extra mechanical arms that project from a backpack.

Extra limbs, controlled by wearable electrode patches that read and interpret neural signals from the user, could have innumerable uses, such as assisting on spacewalk missions to repair satellites.

Chris Philpot

What could you do with an extra limb? Consider a surgeon performing a delicate operation, one that needs her expertise and steady hands—all three of them. As her two biological hands manipulate surgical instruments, a third robotic limb that’s attached to her torso plays a supporting role. Or picture a construction worker who is thankful for his extra robotic hand as it braces the heavy beam he’s fastening into place with his other two hands. Imagine wearing an exoskeleton that would let you handle multiple objects simultaneously, like Spiderman’s Dr. Octopus. Or contemplate the out-there music a composer could write for a pianist who has 12 fingers to spread across the keyboard.

Such scenarios may seem like science fiction, but recent progress in robotics and neuroscience makes extra robotic limbs conceivable with today’s technology. Our research groups at Imperial College London and the University of Freiburg, in Germany, together with partners in the European project NIMA, are now working to figure out whether such augmentation can be realized in practice to extend human abilities. The main questions we’re tackling involve both neuroscience and neurotechnology: Is the human brain capable of controlling additional body parts as effectively as it controls biological parts? And if so, what neural signals can be used for this control?

Keep Reading ↓Show less