Image: IBM

“As you can see,” the doctor was telling me, “the X-rays don’t show any break.” I nodded in agreement, although I couldn’t tell much of anything from the black-and-white image of my badly swollen and severely hyperextended right wrist, which I’d injured during a softball game. The doctor helpfully explained, as he was taping on a splint, that I obviously had injured a couple of somethings in my wrist with Latin-sounding names, but not to worry, it would just take a few weeks to heal up.

While it may not have changed my diagnosis, at that moment I would have appreciated a nice diagram of the bones, tendons, muscles, and blood vessels of my wrist showing me exactly what I had injured, along with an explanation of all the medical jargon my doctor used. I would have especially liked having a way of understanding what parts of my body were going to be affected before the several surgeries I have experienced.

Well, my wish might be coming true.

Andre Elisseeff leads a research team at IBM’s Zurich Research Lab that in September demonstrated a prototype system that will allow doctors to view their patients’ electronic health record (eHR) using three-dimensional images of the human body. Called the Anatomic and Symbolic Mapper Engine, the system maps the information in a patient’s eHR to a 3-D image of the human body. A doctor first clicks the computer mouse on a particular part of the image, which triggers a search of the patient’s eHR to retrieve the relevant information. The patient’s information corresponding to that part of the image is then displayed, including text entries, lab results, and medical images, such as magnetic resource imaging. The doctor can zoom in on the image to retrieve selective information or narrow the search parameters by time or other factors.

“The 3-D coordinates in the model are mapped to anatomical concepts, which serve as an index onto the electronic health record. This means that you can retrieve the information by just clicking on the relevant anatomical part. It’s both 3-D navigation and a 3-D indexed map,” explains Elisseeff.

Elisseeff makes clear that the mapper engine is not just a 3-D imaging system. In addition to connecting to a patient’s eHR, the images displayed are linked to the 300 000 medical terms defined by the SNOMED (Systematized Nomenclature of Medicine) international standard, a copy of which the mapper engine accesses from a local database. “It is impossible for doctors to remember all these terms, and they will need some assistance in the near future. Medical standards are at least as complicated to doctors as normal medical terms are to patients,” Elisseeff notes.

Furthermore, Elisseeff says, “The SNOMED terminology is also a knowledge repository. It is how we include the medical knowledge into the mapper engine, how we tell the computer that a finger is a part of the hand or that flu and fever can be related. The glue between graphical objects and the electronic health record is fundamentally based on this computerized medical knowledge. This is the core of our work. The visible part of the application is the 3-D model. But the most challenging part is building the links such that they are clinically relevant.”

“You can think of it as being like Google Earth for the body,” is how Elisseeff frames the mapper engine. “We see this as a way to manage the increasing complexity that will come in using computers in medicine.”

Electronic Health Records

A major driver of that complexity is the push by governments worldwide to computerize paper-based medical records.

“By computerizing health records, we can avoid dangerous medical mistakes, reduce costs, and improve care,” said President George W. Bush in his 2004 State of the Union address. In that speech he called for the computerization of the nation’s medical health records. In April 2004 Bush issued an executive order to accomplish this by 2014. Although our ability to meet the 2014 date is highly doubtful, progress is being made toward defining the underlying standards necessary for creating a national, interoperable automated health-record system.

The United States is following the lead of other countries, such as the United Kingdom, Australia, Canada, Finland, Germany, and Denmark, each of which has introduced national programs to eliminate paper-based medical records and replace them with some form of eHR. The UK’s computerization effort, called the National Programme for IT (NPfIT), is considered by many to be the largest nonmilitary IT program ever undertaken.

However, not all is well on the eHR front, not only because of the technological difficulties involved (for instance, the NPfIT implementation, like most national eHR efforts, has experienced both schedule slips and rising costs) but because of the resistance of both physicians and patients to the presence of computers in the exam room. Many doctors complain that eHRs have turned them into clerks, while patients say that doctors using these automated systems seem more interested in typing on their computer keyboard than in listening to their health problems.

Instead of capturing unstructured data from a conversation between doctor and patient, “most of the electronic health record systems have been built as if physicians and clinicians were office workers entering in structured administrative data,” says Elisseeff. This clerical approach to these system designs implicitly excludes the patient as an active participant and makes the computer an intrusive third party to what are often difficult personal discussions.

Elisseeff hopes that by “opening the computer screen to the patient, better communication between doctor and patient can occur.” He also believes that by changing the computer’s role from a physical barrier to a conversation starter that the acceptance of eHRs will increase. He also thinks the mapper engine’s graphical approach will prove helpful for situations where people may have difficulty in conversing with a doctor—children, for instance, or someone who speaks a different language.

How the Project Came About

The ideas that sparked the mapper engine project came when the IBM research team began talking with doctors working at IBM about the problems with using eHR systems. One doctor said that when exposed to the graphical representation of medical information, many doctors seem to be faster at recognizing what’s going on with a patient and at finding the evidence for it. “Since time is often mentioned as being a key point for the acceptance of IT [in medical settings],” Elisseeff says, “we started to think about it.”

Elisseeff’s team then visited a gynecologist in a European hospital who showed them how he used an electronic form to enter patient data. Yet as the doctor prepared for his next patient, he kept looking at both her paper medical records along with a sketch of the human anatomy where he had written some notes, basically ignoring the on-screen information. Elisseeff said this encounter “helped contribute to our belief that we needed to replicate somehow the paper-based ‘spirit’ of the machine, that is, an unstructured, flexible representation of human anatomy, browsing-style navigation with shortcuts, bookmarks, et cetera.”

Elisseeff and his team then created an initial mapper-engine prototype and showed it around to clinicians who use eHR systems to see if they thought it would increase the usefulness and acceptance of these systems. The positive feedback and suggestions for improvements gave the impetus to turning the prototype into a full-fledged project, which now consists of a team of more than a dozen researchers and software engineers.

Elisseeff states that their prime project objective is to move eHR systems away from an administrative work mode toward a clinician’s natural style of working: “We try to adapt our approach to the workflow of the doctor,” while allowing both the doctor and patient to interact as easily as possible with the eHR system.

“We would not be surprised if it helps more the patients actually than doctors,” Elisseeff adds.

Elisseeff and his team are collaborating closely with doctors in two Danish hospitals both in the design and use of mapper engines in a clinical setting. The IBM researchers see a day in the near future when a patient will be able to access their medical information prior to seeing a doctor and annotate the image with their symptoms. They believe this would further increase the value of the conversation between doctor and patient and improve the resulting quality of care.

Are 3-D Images Really That Useful?

One thing that Elisseeff made clear was that the mapper engine was not designed as a diagnostic support tool: “We are focusing on providing the information, not the decision.”

While this may seem like an omission, it is a pragmatic design decision. Studies have shown that doctors are slower to accept IT than other professionals, even when the benefits have been demonstrated. This is especially true of anything that smacks of being a decision support system, which for many doctors is perceived to undermine or call into question their professional judgment.

While the IBM team believes that the mapper engine will improve the doctor-patient conversation, others are somewhat skeptical.

“I see that someone who is concerned with the topology or geography of the body, like a surgeon or a radiologist, may find value,” says Richard Baron, a primary care doctor in Philadelphia and a long-time user of eHRs. “But as an internist, I am way more interested in structured data elements and mixing and matching and aggregating than I am in visual displays of the human body.”

Elisseeff freely admits that not every doctor they have shown the mapper engine to sees it as useful, either. He says that the mapper engine can provide two-dimensional information as well but thinks that most patients, once they see 3-D imagery, will want it.

Another issue is whether having such a high-tech display of the human body linked to a vast information store might create an unintended consequence, namely increasing even more in patients the belief that medicine is an exact science. Kathryn Montgomery makes the point in her book, How Doctors Think: Clinical Judgment and the Practice of Medicine, that while medicine is unarguably scientific, “medicine is not itself a science” but a practice—“the care of sick people and the prevention of disease.”

Montgomery argues, however, that the increased use of scientific knowledge and technology in medicine has made it appear to patients that it is a science. As far back as the 1930s, this view of medicine as a science was reinforced in the public’s mind by popular “doctor movies” like those in the Dr. Kildare series, in which doctors voiced confidently, “It’s the 20th century: we ought to be able to cure anything.”

The public at large today believes that medicine is a cause-to-effect science and therefore expects doctors to be able to diagnose medical problems with certainty and provide treatments that are cures. “While individual physicians may never actually declare that medicine is a science, few explain that it is not,” Montgomery writes.

Elisseeff doesn’t think that having high-definition, 3-D human-body imagery will reinforce the feeling that medicine is a science with deterministic outcomes, thereby raising false expectations on the part of patients. Instead, he says, it may provide the opposite effect. Presenting complex, uncertain medical information in this way, Elisseeff says, “might be a way to make patients aware that sometimes medical decisions, diagnoses, or treatments are really complicated.”