Doc at a Distance
Robot surgeons promise to save lives in remote communities, war zones, and disaster-stricken areas
On a hot morning this past June, our research group at the University of Washington, in Seattle, crammed into two cargo vans and drove 2000 kilometers south to the rangeland north of Simi Valley, in southwestern California. In the back of one of the vans was our latest creation: a prototype surgical robot we’d been developing for the past four years. Our mission was to field-test the robot—by operating not on a person but rather on latex objects mimicking human organs, with a surgeon commanding the robot from a control console 100 meters away.
At the test site, we met the rest of our team—surgeons, aerospace engineers, networking experts—and set up a base camp on a flat expanse circled by undulating hills. Under a blistering sun, some of us assembled the robot, a portable surgery table, and a video camera under a canopy tent, while another group installed the surgical control console and a video monitor in a second tent. With all systems ready, we waited for the communications channel to be set up to link the two locations.
Remote surgery technologies like the ones we were testing have long interested military groups all over the world. On the battlefield, medical response time often determines who lives and who dies. A recent study of combat casualties found that in nearly two-thirds of fatal battlefield injuries, death comes within 30 minutes. There’s precious little time to perform even simple life-saving procedures, such as controlling bleeding.
Surgical robots offer a tantalizing possibility [see “Submerging Surgery”]. They would allow military doctors, stationed safely distant from the front line, to perform operations without once putting their hands on patients. Medical vehicles equipped with such remote-controlled robots could get surgical care to soldiers in a lot less time than it would take to evacuate them to the nearest base or hospital.
For that vision to become reality, however, surgical robots need plenty of improvement. One challenge is designing systems that can work under conditions very different from those of pristine operating rooms. Indeed, what was new about our trial in California was the fact that the whole setup was compact, rugged, and easy to transport. Local conditions for the experiment, organized by Gerald Moses and Timothy Broderick of the U.S. Army’s Telemedicine and Advanced Technology Research Center, in Frederick, Md., included sandy gusts of wind, high humidity, and an ambient temperature crawling toward 40 °C. Gasoline-fueled generators were all we had to power our electronics.
Our system is one of a new generation of surgical robots that may one day bring advanced medical care not only to soldiers but also to people in remote locations lacking specialized physicians. Disaster relief, too, could be revolutionized. Last year’s Hurricane Katrina in the United States, the October 2005 earthquake in northern Pakistan, and the December 2004 Indian Ocean tsunami overwhelmed the medical resources available in those places. The hope is that in the not-so-distant future, fleets of ambulances or helicopters equipped with surgical robots would rush to such areas, allowing faraway doctors to save lives and limbs.
Indeed, some of the military projects seem to be quite easily convertible to civilian use. Our group is participating in another U.S. military–funded initiative, the US $12 million Trauma Pod program, launched last year by the Defense Advanced Research Projects Agency (DARPA). Managed by SRI International, in Menlo Park, Calif., the program aims to develop an unmanned, mobile operating room that is equipped with a host of automated surgical systems and could be quickly dispatched anywhere in a war zone.
But giving surgical robots the necessary mobility requires meeting another challenge: the reliable transmission of the surgeon’s commands to a system often roaming in far-flung places. Surgeons have remotely commanded surgical robots before, even with real patients under the robots’ scalpels. But those setups took place in well-equipped hospitals and relied on dedicated, wired communications channels. How, then, to break free from wires?
Geosynchronous satellites have good data bandwidth, but they don’t cover all regions of the world and their delayed transmissions make surgery difficult. With our experiment in California [see illustration, "Far-Flung Fingers"], we wanted to test an alternative that could be heaved into the air—literally—in a matter of seconds: a wireless link enabled by an unmanned aircraft. Launched from the ground, a small drone was sent to fly in lazy circles above us. Video from the camera near the robot was compressed by special hardware into MPEG format and beamed to the plane, which relayed the feed to the monitor below. At the same time, motion commands from the surgeon’s console were bounced through the plane to the robot, which responded only a fraction of a second later, performing such tasks as tying suture knots.
Sewing IT up: The University of Washington’s surgical robot ties suture knots, one of the most complex tasks in minimally invasive surgery.David Clugston
Nevertheless, surgery today remains fundamentally the same as it was centuries ago in its basic aspects: it relies heavily on the experience of the surgeons and the dexterity of their hands.
The idea of robot surgeons may conjure visions of C-3PO–like androids clad in scrubs excising appendixes, but existing systems and those being developed by our group and others are actually more like robotized laparoscopic instruments. They function as a surgeon’s miniaturized and electromechanically enhanced hands, maneuvering tools with greater dexterity, accuracy, and stability than humans could ever achieve. The tools, positioned by high-precision motors, can reach spaces, such as those around the prostate gland or in the tiny bodies of infants, that surgeons often can’t reach using their hands.
Indeed, robotically assisted surgery is already a bona fide, if small, category of robotics. In its “World Robotics 2005” report, the International Federation of Robotics estimated the number of robots used in surgery and therapy—including electromechanical arms to position instruments, robotic bone drills, and computer-controlled radiotherapy equipment—at 2800 units worldwide. These units include the da Vinci surgical system, by Intuitive Surgical, of Sunnyvale, Calif., currently the only commercially available surgical robot approved by the U.S. Food and Drug Administration. It has two or three arms equipped with surgical tools and an extra arm with a stereoscopic video camera probe. A surgeon controls the machine from a console located in the same room as the patient. Currently, some 400 da Vinci units are installed worldwide, a new unit costing about $1.5 million.
For at least one procedure, the da Vinci system is beginning to show advantages over conventional surgical techniques: the laparoscopic radical prostatectomy, in which the prostate gland is removed through tiny incisions in the lower abdomen. Surgeons have reported that patients undergoing the robotic procedure had less blood loss, fewer complications, and shorter hospital stays. Some experts say that cardiac and gynecological procedures could be next, but further studies have yet to show the benefits of robotic surgery—considering its higher costs—over conventional techniques.
And despite its achievements, the da Vinci system wasn’t engineered to stand up to the extremes of work in the field. It weighs nearly half a ton, and controlling it remotely requires substantial electronic modifications. Moreover, the system is designed specifically for minimally invasive surgery, precluding significant use for trauma procedures.
To develop a surgical robot that can operate in harsh environments, our group has focused on miniaturization and mobility. This design work draws on research we did several years ago. In that research, we collaborated with Mika Sinanan, a professor of surgery at the University of Washington, to attach position and force sensors to minimally invasive tools. We gathered sensor data as Sinanan and other surgeons operated on animals by cutting into the body, severing and suturing arteries, lifting out organs, and sewing up incisions. The measurements revealed the actual workspace the tools require and the magnitudes of force and torque for performing a variety of procedures.
With this information we were able to optimize the robot’s dimensions and motion, minimizing the space it occupies without compromising its manipulation capabilities. To do that, we wrote software that generated 5476 different designs by varying the size and shape of the segments and joints that make up the robot’s arms. Using the program, we selected the smallest design that meets all the requirements of minimally invasive surgery and also of soft-tissue procedures typically performed as part of general, vascular, cardiac, and urological surgery.
Why is size so important? Yulun Wang, one of the pioneers of surgical robotics, once said that the most precious space in the whole world is the operating room space directly above a patient. No matter how big the operating room is, the space above the patient is always roughly the same: about 1 by 0.5 by 0.25 meters for an adult patient. In a robotic surgery setup, that space needs to accommodate at least two and ideally four robotic arms, as well as one or more video cameras for the surgeon to see the results of each robotic maneuver.
Our current prototype has two articulated arms. Each arm consists of four aluminum segments connected by rotary joints. It holds a stainless steel shaft 30 centimeters long and 5 millimeters in diameter to which different surgical tools—scalpels, graspers, clip appliers, scissors—can be attached. An articulation at the extremity of the shaft works as a wrist, giving additional mobility to the tools.
We built the arms with steel cables running through them and converging toward a fixed base, where six brushless dc motors used to pull them are housed. With this configuration, you don’t have to embed the motors into the arms themselves, enabling them to be smaller and lighter. The arms can each move in six distinct ways (six degrees of freedom), and they can be attached to a surgical table or to the walls or ceiling of the operating room.
The robot’s control system uses a standard PC running a real-time operating system based on Linux. One thousand times every second, the control system receives a variety of input signals: readings from the robot’s position sensors, motion commands from the surgeon’s console, and status and safety signals from circuits inside the robot. After processing the data, the control system generates electrical currents to guide the motors.
As for the surgeon’s console, we’ve been using off-the-shelf devices until we have the resources to design a more robust system. The console includes two Phantom Omni haptic devices, which are motion-tracking joysticklike controls, from SensAble Technologies, that connect to a PC. The PC, in turn, communicates with the robot through a standard Ethernet link.
That means the robot and the surgeon can be in the same room, connected by an Ethernet cable, or on different continents, connected through the Internet. Indeed, in a recent transatlantic experiment, surgeons at Imperial College in London successfully controlled our robot in Seattle using a regular Internet connection. They used a PC and haptic devices available there, and the software for the video links—iChat and Skype—was not only off-the-shelf, it was free. Although the quality was not the same as what we get with our MPEG hardware, we were impressed with the detail and smoothness of motion.
Our PC-based control console also includes features important to robotic surgery. For instance, like earlier systems, it can scale down motion. That means that the surgeon’s hands can move relatively large distances while the robot’s arms move shorter distances. This is particularly useful when the surgeon is operating on small body structures while looking at a magnified video image. We hope that with our simple control interface, more surgeons will be able to learn and experiment with our robot.
The distinguishing aspect of our work, as mentioned earlier, was to make surgical robots smaller and easier to deploy in places other than conventional operating rooms—the reason, of course, that we wound up in California this past summer. That experiment was part of a U.S. Army project called High Altitude Platforms Mobile Robotic Telesurgery. Its overall goal was to demonstrate the concept of remote robotic surgery in the field, using a rugged surgical robot and an airborne communications link.
The communications platform was provided by AeroVironment, best known for its unmanned aerial vehicles (UAVs). The Army decided to go with a UAV because earlier remote surgery experiments had turned up problems with satellites, which aren’t always available and whose communication delays—often a second or more—make surgery hard. UAVs are an attractive alternative: they can be compact and low cost, and they can fly as low as 100 meters or as high as 30 000 meters from the ground. AeroVironment alone has about 4000 small UAVs flying in Iraq and Afghanistan.
Our experiment took place at an isolated site north of AeroVironment’s headquarters in Simi Valley. With the robot and the surgeon’s console set up in different tents 100 meters apart, AeroVironment engineers sent one of their UAVs, called the Puma, to fly autonomously in circles about 100 to 200 meters above us. Internet-style data packets streamed through the UAV and between the two tents. To transmit video using the 2 megabit-per-second channel available, a high-performance MPEG encoder-decoder device by HiaVision Systems compressed the video signal.
Taking turns at the surgeon’s console were Timothy Broderick, who is a professor of surgery and biomedical engineering at the University of Cincinnati, and Lynn Huffman, also from the department of surgery at the University of Cincinnati. Their eyes on a video monitor and their hands at the haptic controls, the surgeons moved the tools along specific paths in space and positioned their tips at predetermined spots on the latex objects. They were able to simulate various maneuvers that surgeons normally perform, including suturing, which is one of the most complex surgical tasks in minimally invasive procedures, when surgeons must sew up tissue and vessels well under the patient’s skin. During the experiment, we measured manipulation delays of 20 millisconds and video delays of 200 ms, both noticeable by the surgeons but not enough to interfere with their control of the robot.
In the next phase of the project, expected to take place sometime next year, the UAV will fly much higher and the surgeon’s console will be located not 100 meters from the robot but 2000 km away, at our Seattle lab. The experiments will help us better understand how remote surgery works when relying on UAV-enabled links, Internet connections, and combinations of them. In fact, some people wonder about the vulnerabilities of airborne communications. What would happen if the UAV is shot down, interrupting the surgeon-robot connection? To avoid such problems, the best solution seems to be redundancy. It’s likely that when surgical robots are deployed, they will use multiple communications links, enabled by a fleet of UAVs, which could fly at stratospheric altitudes, making it difficult for rockets to reach them.
In any case, surgical robots need to have a host of monitoring circuits to guarantee that they are fail-safe. One of the hardware safety systems we developed controls the transitions among four states of our robot: E-Stop, in which power is removed from the motors and brakes are applied; Initialize, in which the robot checks its operation and range of motion prior to surgery; Pedalâ''Up, in which the surgeon uses a foot pedal to determine that the robot is ready to respond to commands but still has the brakes applied; and Pedal-Down, in which actual surgery is performed.
The robot’s control system guarantees that the robot is always in one of these four states. In addition, if the control system detects any type of software or hardware failure, it puts the robot in the E-Stop state within 20 ms.
DARPA is promoting its vision of the operating room of the future primarily through its Trauma Pod program. It’s an ambitious initiative managed by Richard M. Satava, a professor of surgery at the University of Washington. Satava, a hospital commander in the first Gulf War, was prompted by his experiences there to think about how technology could improve battlefield medical care.
Satava’s main objective with Trauma Pod is to use robotics to project the skills of surgeons to precisely where they’re needed on the battlefield. How to do that? Using an unmanned, mobile operating room that expert surgeons can control at a distance. The concept is in line with the current trend of reducing personnel and logistics on the battlefield through the use of autonomous and teleoperated systems. The U.S. Department of Defense expects to reduce deployed personnel by up to 30 percent by 2025.
Behind this vision is a multiphased program led by SRI that includes contributions from the University of Washington, the University of Texas at Austin, the University of Maryland, and Oak Ridge National Laboratory, as well as from companies like General Dynamics, Intuitive Surgical, General Electric, Robotic Surgical Tech, and Integrated Medical Systems.
In the first phase of the program, to be completed next spring, the goal is to demonstrate a prototype of a trauma pod. The prototype will be built with commercially available technologies wherever possible. Intuitive Surgical’s da Vinci robot will be the main surgical robot, and Integrated Medical Systems’s Life Support for Trauma and Transport (LSTAT) stretcher system will work as a high-tech surgical bed. LSTAT, now used on helicopters and ships as well as by MASH units in the field, carries a defibrillator, ventilator, oxygen supply, and other equipment.
Other systems, however, will have to be custom made. That includes machines to perform the functions of operating room nurses. Our primary role in the Trauma Pod project is in developing the tool changer—an automated machine that performs some of the functions of the nurse who hands surgical tools to the surgeon. Our current prototype consists of a rotating device that holds up to 15 tools. It can retrieve a surgical tool and present a new one in about 0.7 second.
DARPA is planning a series of proof-of-concept demonstrations. If the tests are successful, in the second phase the agency will fund research aimed at miniaturizing and integrating all the systems, so that they form a portable operating-room-on-a-stretcher platform that could eventually be carried by Humvees, helicopters, or other vehicles.
Here’s DARPA’s vision of how it would work: say an explosion sends shrapnel into the leg of a soldier in an urban war zone circa 2025. The soldier is put into a trauma pod that is accompanying the squadron. The trauma pod scans the soldier’s body with a CT system and detects the leg injury. It then administers antibiotics and anesthesia to the wound. Next a surgical robot, remotely controlled by a doctor, removes the metal fragment, stabilizes the bleeding, and closes the wound. The soldier is then evacuated by aircraft to a base nearby for further treatment.
The concept of surgical robots has gone from crude prototypes to FDA-approved commercial technology in just the past 15 years. The surgical robots of the future promise even more spectacular advances. They will use imaging technologies such as ultrasound, MRI, and CT scans as their “eyes,” and they will break free from centuries of surgical convention, entering the body through existing openings and moving inside the patient as they make their way to the surgery area. Your descendants might even swallow one of these some day.
As the technology matures, surgical robots promise to improve a wide range of procedures in terms of patient recovery time, cost, and safety. Medicine, however, is, as it should be, a conservative field, following Hippocrates’ mantra: “I will keep [patients] from harm and injustice.” In the next several decades, surgical robots, like many technologies introduced in medicine, will prove their value and become mainstream tools—tools always guided by a physician’s judgment and dedication to the delivery of the best health care.
The authors wish to thank graduate students Mitch Lum, Denny Trimble, and Dianna Warden; our colleagues at the University of Washington Jesse Dosher, Mika Sinanan, and Rick Satava; Timothy Broderick and Lynn Huffman of the University of Cincinnati; and many others for their contributions to our surgical robotics work and helpful comments on this article.
About the Authors
JACOB ROSEN is a research associate professor of electrical engineering at the University of Washington, in Seattle. BLAKE HANNAFORD, an IEEE Fellow, is a professor of electrical engineering at the University of Washington. They are codirectors of the university’s BioRobotics Laboratory.
To Probe Further
An extensive backgrounder on the University of Washington’s BioRobotics Laboratory and its surgical robotics projects is available at https://brl.ee.washington.edu.
For more information on the U.S. Army’s Telemedicine and Advanced Technology Research Center, see https://www.tatrc.org.
For technical papers, videos, and other material on DARPA’s Trauma Pod program, see https://depts.washington.edu/biointeland https://www.darpa.mil/DSO/thrust/biosci/traumpod.htm.