Automaton iconAutomaton

NATO's First Combat Casualty in Libya Is a Robot

On Tuesday morning, a Northrop Grumman MQ-8 Fire Scout was shot down by Moammar Gadhafi's forces in Libya, becoming NATO's first combat casualty in the conflict. The U.S. Navy has been testing Fire Scouts for five years or so, and the robots have progressed from shipboard autonomous landings all the way to accidental drug busts, but this is the first we've heard of them actually involved in a major military operation.

Presumably, the Fire Scouts are being used solely as surveillance platforms, although they've also been successfully tested as weapons platforms, as you can see in the second half of this vid:

So far, the Navy hasn't said much about what exactly the Fire Scout was doing when it was shot down beyond the obligatorily vague "performing intelligence, surveillance and reconnaissance over Libya to monitor pro-Gadhafi forces threatening the civilian population." It's remarkable how not big of a deal this incident is relative to what the response would have been had (say) a manned Apache gunship been shot down instead, as Libyan state TV originally claimed.

Photo: Northrop Grumman

[ Fire Scout ] via [ BBC ] and [ Danger Room ]

'Bots High' Documentary Takes You Inside the Fierce and Hormonal World of High School Robotics

It's a blast to go to a combat robotics event and watch two metal behemoths beat the battery packs out of each other in violent deathmatches. Most spectators, though, really don't have a clue about just how much time, effort, skill, creativity, cooperation, and (quite literally) blood, sweat, and tears go into the creation of one of these machines. Also, most spectators likely assume that there's no way they would ever be able to get involved in what looks to be such a technical pastime, but even robotics veterans had to get their start somewhere, and increasingly, that somewhere is high school. 

bots high film movie documentaryBots High, a feature documentary produced and directed by Joey Daoud, takes us along as three teams of high school students spend four months building 15 pound and 120 pound combat robots for a national competition in Florida. With help from faculty, parents, and volunteers, the teams (two of which are made up entirely of girls) have to design and then fabricate robots that have a fair chance of getting totally destroyed over the course of a single match.

Combat robotics is unlike any other robot competition (or most competitive sports, period) in that you run the risk of having months, or even years, of work completely obliterated in the space of three minutes or less. And not just "oh we have to to fix it now" obliterated, but more like "go get a broom to sweep up the sad little pile of metal splinters that used to be your prized combat robot" obliterated.

For the audience, watching this happen is a brief and highly entertaining tragedy. For the teams who build the robots, it's obviously a much bigger deal, and something that's probably impossible to understand without experiencing it yourself. By taking us through much of the process (the human process as well as the mechanical one), Bots High helps to provide some background and context to the sport as a whole, as you watch just how much effort it takes to get a robot from a design concept to something that's able to hold its own in the combat arena.

bots high film movie documentary

As the title implies, Bots High (which is 83 minutes long) spends time not just on the robots, but also on the environment in which they were created: high school. I found myself to be very impressed (perhaps even inspired) by the level of support that the teams received from faculty, parents, and volunteers. I'd like to think that this is because combat robotics (and robotics in general) is recognized as an incredible educational tool, because it absolutely is. You can disguise it in remote-controlled violence all you want, but the bottom line is that these kids are learning to design, to program, and to build with their hands. They're learning to work in teams, to troubleshoot, and to think on the fly. These are things that aren't especially easy to teach in the traditional sense, but they just happen in the context of combat robotics. It's not just about the fun, it's also about the future, and anybody who knows a kid who's interested in building stuff should absolutely watch this movie and then figure out a way to get themselves involved.

The stars of the film, for better or worse, are definitely the high school students on the individual teams. Consequently, there's, like, a lot of, like, you know, high school-ness that you'll have to sit through. To some extent, that's part of the charm: these kids aren't all super-nerds. They have lives, they have other priorities, they have hormones (ugh), and frequently they don't have much of a work-ethic. But they're also extraordinarily passionate about their robots, with a focus and level of commitment that I can't recall ever having at that age.

bots high film movie documentary

There might be a little too much time, though, spent on the obligatory high school boy/girl drama. It doesn't help that Will and Elizabeth (members of opposing teams) are flirting with each other while giggily trying not to be obvious about it, a phase that (even at my relatively advanced age) I definitely remember suffering through, and we're sort of left wondering whether they'll get to first base by the end of the film. Spoiler alert: maybe, but I definitely started to think to myself, "Okay, lock those two in a closet already and let's get back to the robots."

Fortunately, the film does get back to the robots, and in a big way. I certainly won't ruin the ending, but there's laughter and tears and violence (the good kind of violence) and lots and lots of screaming (the good kind of screaming), and by the wonderful end of it all, I was ready and willing to go back to high school (!) and have this whole amazing robotics experience that I so obviously missed out on.

The bottom line: If you can stand to be reminded of high school, this is a very well executed and altogether enjoyable documentary that might just inspire you to get involved in combat robotics yourself. And you absolutely should.

Bots High is currently being shown at film festivals. It will be available on DVD and online on Netflix and iTunes later this year. There will be a special free screening day on October 6th at multiple locations; to find a screening near you (or host one yourself) check out the website below.

[ Bots High ]

Watch This Little Termite Robot Build Itself Some Stairs

Swarms of robots might not always seem like the friendliest things to have in your life, but next time you need a tiny stairway, this hard-working little robot named Kali and a bunch of its friends are here to serve.

Kali is part of Harvard's Termes project, which is developing a swarm construction system where lots of little robots team up to build structures that would be impossible for any one single little robot to put together. It's called "Termes" after our noble and endlessly destructive pals the termites, who use teamwork to fabricate mounds of earth up to 30 feet high. Like termites, Termes robots are simple and autonomous, and are able to cooperatively move heaps of standard building blocks (specially designed to allow the robots to both lift and crawl around on them) to create just about anything, as long as you give them enough blocks and enough time.

For example, the demo below shows Kali using just a few simple sensors to autonomously construct a staircase to scale a wall of an unknown height, based on previous experience with such situations:

Nicely done. While this is just one robot, you can easily extrapolate to what might be possible with swarms of robots, and it's not just bigger staircases. Get enough of these little guys together and they'll build you your very own fort, as the simulation at the end of this next video shows:

The Termes Project comes from Harvard's Self-organizing Systems Research Group, the same dudes responsible for the kill-o-bots kilobots we met last week. They'll be presenting this work at the 2011 Robotics: Science and Systems conference next week in LA, and you can read the full paper at their website.

[ Termes Project ]

READ ALSO:

Kilobots Are Cheap Enough to Swarm in the Thousands
Thu, June 16, 2011

Blog Post: What can you do with a $14 robot? Not much. What can you do with a thousand $14 robots? World domination

Robot Uses Supersonic Jets of Air to Stick to Almost Anything
Tue, May 24, 2011

Blog Post: By blowing air across a surface really, really fast, this robot can suck onto everything from metal to bricks to cloth

Little Amphibious Tumbling Robot Tackles Tough Terrain
Wed, May 25, 2011

Blog Post: The secret to making a small robot that can sink, swim, and slog through sand? Easy: just teach it to do somersaults

Gecko-Inspired Window Washing Robot is Powered Entirely by Water
Fri, May 27, 2011

Blog Post: This climbing robot washes windows with water, sticks with water, and even moves with water power

Autonomous Boats Take On Obstacle Courses, Fake Fires in RoboBoat Competition

Somehow, it's been an entire year since the 2010 RoboBoat Competition. Rather than letting all of those industrious teams improve their robots to be better able to complete the existing course, the organizers added a whole bunch of practically impossible new challenges. Practically impossible, sure, but also pretty sweet, since they involve using deployable rovers to retrieve objects and autonomous water cannons to put out (fake) fires.

You may be wondering why such seemingly trivial tasks like navigating between different colored buoys is so tricky, but remember that this is all taking place on water, which is covered in nasty things like reflections and waves and hostile swans. So whenever the sun angle changes (an event that tends to happen quite often throughout the day), everything looks slightly different for the boats' cameras, sensors, and vision algorithms.

Anyway, luckily for you there's some excellent video recap of all three days of the event, so you can ignore my blathering and just watch things unfold for yourself. Swans beware!

[ RoboBoat 2011 ]

Microsoft Releases Kinect SDK, Roboticists Cackle With Glee

One of the cheapest and most effective pieces of 3D mapping and gesture sensing hardware you could possibly hope for has just gotten an official SDK (software development kit) release. We're talking about Kinect, of course, and Microsoft has benevolently decreed that you no longer have to hack the sensor to get some non-gaming use out of it. Here's a few things you have to look forward to:

  • Raw sensor streams: Access to raw data streams from the depth sensor, color camera sensor, and four-element microphone array enables developers to build upon the low-level streams that are generated by the Kinect sensor.

  • Skeletal tracking: The capability to track the skeleton image of one or two people moving within the Kinect field of view make it easy to create gesture-driven applications.

  • Advanced audio capabilities: Audio processing capabilities include sophisticated acoustic noise suppression and echo cancellation, beam formation to identify the current sound source, and integration with the Windows speech recognition API.

Kinect is just one example of how robotics has been successfully piggybacking on other tech to get access to sensors and other hardware that's super effective and super cheap at the same time. Microsoft isn't making Kinect for robotics, but we don't care, we're perfectly happy to steal it and put it to better use than they ever could. I mean, come on, games? Psh! Try this stuff on for size.

The other advantage of having cheap and effective hardware with an SDK is that it helps the robotics community share ideas. It's the same basic philosophy as the PR2 (and ROS): if everyone's developing for the same platform, you can save yourself tons time and money by sharing code. So from a hobby robotics standpoint, you don't have to know a lot about Kinect to take advantage of it, since you can just adapt the clever things that other people have developed for the platform to your particular project.

You can download the Kinect SDK beta right now; it's free, but Windows 7 only and for use with Visual Studio in C++, C#, or VB. If you still need the hardware, Kinect sensors are a mere $150 at your friendly local gaming emporium.

Oh and by the way, we should also mention that the original Kinect hardware developer, PrimeSense, has partnered with Asus to develop a PC version of Kinect that they're calling "WAVI Xtion." No, I don't know how it's pronounced, but I do know that you can expect it in the second quarter of 2011, i.e. pretty much now.

[ Kinect for Windows Beta SDK ]

READ ALSO:

Top 10 Robotic Kinect Hacks
Mon, March 07, 2011

Blog Post: Microsoft's Kinect 3D motion detector has been hacked into lots of awesome robots, and here are our 10 favorites

TurtleBot: Mobile 3D Mapping and ROS
Tue, April 05, 2011

Blog Post: Willow Garage has created an affordable, but still capable, ROS development platform

Hands-Free Roomba With Kinect
Mon, February 14, 2011

Blog Post: Now you can enjoy autonomous robotic vacuuming with direct gesture control using a hacked Kinect

Kinect Teleoperated Robot Does Pushups
Mon, January 03, 2011

Blog Post: Robots are ready and willing to help take over all of our hard work, and that includes exercise

Kilobots Are Cheap Enough to Swarm in the Thousands

These are Kilobots. They're fairly simple little robots about the size of a quarter that can move around on vibrating legs, blink their lights, and communicate with each other. On an individual basis, this isn't particularly impressive, but Kilobots aren't designed to be used on an individual basis. Costing a mere $14 each and buildable in about five minutes, you don't just get yourself one single Kilobot. Or ten. Or a hundred. They're designed to swarm in the thousands, although the Harvard group that's working on them is starting out with a modest 25:

We've seen lots of examples of swarm robotics, but what we decide to call a "swarm" often isn't, really. There is (or should be, at any rate) a distinction between a group of robots cooperating on a task and a true swarm of robots, and for the purposes of this article, I'm going to arbitrarily assert that a group of robots turns into a swarm of robots when you can't easily count how many individual robots there are. So like, these swarming MAVs? Not really a swarm. Swarmanoid? Not a swarm yet. Swarm bots are getting closer. What definitely makes the cut are projects like RoboSwarm and FlyFire, which use anywhere from hundreds to thousands of small robots all at once.

There's a lot you can do with gigantic swarms of robots, but there are two big obstacles to deploying them: programming, and charging. If you can't figure out a way to do these things efficiently (i.e. not on an individual basis for each robot), it negates a big part of the swarm appeal. In the case of the Kilobots, they can all be programmed at once with an infrared controller, and to charge them, the bots can simply be sandwiched between two conductive surfaces. The fundamental idea here is that any interaction with a robot swarm has to be scalable, such that an increase in the number of robots in the swarm doesn't result in an increase in the amount of time it takes to interact with the swarm.

I should point out that the other big obstacle to robot swarm deployment is price, which is why kilobots are deliberately so cheap: at $14 each, a thousand robots is actually an achievable number with a modest grant, which is something that probably has not been possible before. Generally people who want to experiment with large swarms have had to be content with computer simulations, which is fine, but at some point you have to try things out in the real world (or as close as you can get in a lab), and Kilobots can make that happen.

The Self Organizing Systems Research Group at Harvard is planning to expand their Kilobot collective to 1024 robots, and then they'll teach the swarm to demonstrate behaviors like self-healing and collective transport. Better hide your kids. Also, for the record, I'm pretty sure it's "Kilobots" and not "kill-o-bots." But who really knows until it's too late, right?

[ Kilobots ] via [ Hack A Day ]

READ ALSO:

German Robot Plays Pool, Throws Down Robot Pool Gauntlet
Fri, June 03, 2011

Blog Post: With several different human-sized robots now capable of playing pool on real tables, a confrontation seems inevitable

Cutest Quadcopter Ever Sounds Like a Swarm of Angry Bees
Mon, May 02, 2011

Blog Post: This quadcopter measures only about 10 centimeters on a side, but it's just as agile as its big brothers

Awesomely Bad Ideas: Teaching a Robot to Sword Fight
Fri, May 13, 2011

Blog Post: Georgia Tech has given a robot a sword and told it that humans are out to get it, all in the name of safety

Japanese Ministry of Self-Defense Builds Flying Robot Soccer Ball
Thu, June 09, 2011

Blog Post: Why does the Japanese Ministry of Defense need a flying robot soccer ball? We can only hope it's for flying robot soccer

Libyan Rebels Making Armed Robots From Power Wheels Toys

Libya rebels remote controlled machine gun vehicle

Armed robots have been making their way from science fiction to mainstream combat at an aggressive pace. The U.S. military is trying to be cautious about the whole thing (too cautious for some and not cautious enough for others), but most people would probably acknowledge that increased reliance on unmanned systems is, for better or worse, an inevitability. This is because robots offer many advantages in conflict zones, the first and foremost being that sending a robot into a dangerous situation often means that a human doesn't have to go into that same situation.

These advantages aren't realized solely by the U.S. military. They're not realized solely by governments in general, either. Robots have been getting cheaper and more accessible, and people with an interest in robotics have for years been able build their own systems to take over work that's dull, dirty, or dangerous. It should be no surprise, then, that rebels in Libya have started cobbling together their own armed robots out of Power Wheels toys, video cameras, radios, and machine guns:

So what does this mean for the present and future of military robotics? First, it's a vivid illustration of the potential implications of a rapidly descending barrier to entry for this kind of technology. Anyone can (on principle, at least) build a robot, and given the need or the motivation, anyone can put a gun on one, too. Second, the fact that anyone can build something like this is an equally vivid illustration that despite whatever qualms we may have about military robotics, it's not only going to happen, it's happening already. Whatever the ethical implications may be, this is becoming the new reality faster than we might like, and it's something that we're going to have to prepare for.

[ Al Jazeera ] via [ Jalopnik ]

U.S. Senator Calls Robot Projects Wasteful. Robots Call Senator Wasteful


Tom Coburn, a senator from Oklahoma, and PR2, a robot from California.

A U.S. senator has cited three robotics projects as examples of "wasteful" research that lack useful applications and shouldn't have received government funding.

In a recent report, Senator Tom Coburn of Oklahoma takes aim at the National Science Foundation, the premier source of funding for science and engineering in the United States, raising questions about the agency's management and priorities. In one section of the report, Coburn criticizes the NSF for squandering "millions of dollars on wasteful projects," including three that involve robots.

"A dollar lost to mismanagement, fraud, inefficiency, or a dumb project is a dollar that could have advanced scientific discovery," the report says.

Coburn didn't give the roboticists a chance to respond, so I reached out to the three groups—from the University of California, Berkeley; University of California, Davis; and Rowan University, in Glassboro, N.J.—to hear their side.

Of course, they aren't exactly thrilled to see their work "featured" in the report. One scientist quipped that Coburn has just sparked a robot uprising. Picture hordes of bots descending on Washington, D.C. to show the senator who's wasteful by using him as cookie dough.

The researchers say they welcome scrutiny and agree that there are many improvements the NSF could make. But they argue that the Coburn report evaluated their projects superficially and out of context.

One of the projects deemed questionable involves a PR2 robot, made by Silicon Valley firm Willow Garage and by far one of the most capable robotic systems in existence. Berkeley researchers taught the PR2 how to fold towels, a demonstration that captured people's imagination.

But apparently Coburn wasn't impressed. His report notes that the robot cost $1.5 million and complains that it "took nearly 25 minutes to fold each towel." [UPDATE: The report references the wrong NSF grant; this is the correct one, a $1.2 million award. And the Berkeley researchers got the robot for free.]

Here's the "exclusive" unveiling of the report on ABC's "Good Morning America."

Berkeley computer science professor Pieter Abbeel, one of the researchers behind the project, told me that the towel folding experiment was just a small part of a much broader effort aimed at creating robots that can handle the complexities of real environments. Here's what he wrote in a rebuttal:

"[I]n order to expand the use of robots beyond manufacturing the machines must be far more sophisticated in terms of their ability to deal with complexity. That's what our work is all about. Towel folding is just a first, small step towards a new generation of robotic devices that could, for example, significantly increase the independence of elderly and sick people, protect our soldiers during combat, and a host of other applications that would revolutionize our day-to-day lives."

Coburn also discussed the report with Neil Cavuto from Fox News. After seeing footage of the PR2 folding towels, Cavuto says: "I guess many folks would like that. But how's the robot doing? Did it indeed fold clothes?" The senator admits he doesn't know details about the project. "It just caught my eye," he says.

I asked Coburn's office for more information on how they selected the projects they thought shouldn't have been funded. Did researchers or policy experts with relevant scientific backgrounds help Coburn prepare the report? Who are his co-authors?

Coburn "is the author of the report," John Hart, the senator's spokesman, told me in an e-mail. He added that the senator, who is a physician, "does have a scientific background," in addition to a business, accounting, and public policy background. "This is a multi-dimensional discussion."

I also asked whether Coburn and his staff contacted the researchers prior to the publication of the report to ask for more information or offer them a chance to address the criticism.

"Yes," Hart said. "Scientists and researchers who are privileged to receive federal funds should welcome and expect questions about their work." He added: "There are no sacred cows that should avoid examination and, if necessary, dissection."

But all the researchers I contacted told me they never heard from Coburn's staff. They said they were puzzled that the report relies so much on press reports rather than material with more scientific content—an approach they found a bit, well, unscientific. One researcher asked if Coburn would judge whether a patient is sick just by looking at the person's face.

In another project criticized in the report, a UC Davis group is studying how people interact with and control their bicycles. The researchers also want to build a robotic bike.

Mont Hubbard, a professor of mechanical engineering, is working with other faculty and students to develop dynamic models that can accurately describe how people ride bikes. The goal of the project, which received a $300,000 NSF award, is to understand the design parameters that could lead to bikes that are safer and easier to control by different groups of people and for different tasks.

mont hubbard uc davis instrumented bicycle

The researchers are using a bike equipped with sensors [photo above] and also building a robotic bicycle to identify the parameters that their models need to take into account. As it turns out, Hubbard says, we know very little about how a bike's design affects safety, performance, and our ability to control it. In particular, we need to learn more about how the dynamics of the bike and rider affect each other.

"There's plenty to be discovered," Hubbard says. "Just because Senator Coburn knows how to ride a bicycle, it doesn't mean that's the end of it."

He adds that increasing bicycle usage would have "health benefits, transportation benefits, environmental benefits." Surveys show that although Americans don't bike much, many more would if they felt bikes were safer, he said.

The third project criticized in the report was a "robot rodeo," a three-day event that took place at a conference for computer science educators in Dallas, Texas, last year. The organizers, Jennifer Kay, a computer scientist at Rowan University, and Tom Lauwers, a robotics entrepreneur, say the goal of the event was to "introduce robot programming to the nearly 1200 educators attending the conference, and to raise awareness amongst participants of how robots could be used in their classrooms."

They say that despite evidence that robots can be used as educational tools to excite and motivate students, only a tiny fraction of educators have ever programmed a robot or tried them in their classrooms. They told me that the event—which involved months of planning and dozens of volunteers—received only $6,283 from the NSF, a number that the Coburn report doesn't mention. (Just for reference, that's one-fifth of what the Senate Hair Care Revolving Fund spent last year.)

And yes, Kay and Lauwers say, the event was designed to be fun:

"Perhaps the Robot Hoedown and Rodeo was singled out because it has an intentionally eye-catching name, and because on the surface it appears 'fun.' Indeed in his report Senator Coburn states, 'Videos of the event posted to YouTube suggest the effort was a source of enjoyment for observers.' It is precisely this 'fun' which our program aims to associate with Computer Science education, so that our current students will choose to become the future researchers that make the kinds of transformative discoveries that improve our society and our economy."

Coburn acknowledges that NSF grants have supported many scientific breakthroughs, but he insists that the agency could save between $1 billion and $3 billion by eliminating inefficiencies and duplication.

Among other things, he calls for the NSF to defund its social and behavioral sciences division and sharpen its focus on "truly transformative sciences with practical uses outside of academic circles and clear benefits to mankind and the world." (Full disclosure: IEEE Spectrum has collaborated with the Directorate for Engineering of the National Science Foundation to coproduce "Robots for Real," an award-winning special report with clear benefits to mankind and the world.)

But picking "winners" is a challenge even for experienced NSF program managers and the scientists who help the agency review its grant applications.

"In many cases, it can be difficult to identify, in advance, what kinds of research proposals might lead to transformative results," says Dana Topousis, an NSF spokesperson. "For instance, when NSF funded a graduate research fellow in the early 1990s to study digital libraries, we couldn't predict that that graduate student would co-found Google."

So who knows? The next Google may very well be a robotics company founded by a pair of NSF-funded researchers. Then again, there's only one way to find out.

READ ALSO:

Robots Are the Next Revolution
Mon, March 28, 2011

Blog Post: So why isn't anyone acting like it?

PR2 Does The Impossible, Folds Towels
Wed, March 31, 2010

Blog Post: The robot may not be the fastest at folding towels, but the fact that it can do it entirely autonomously is nuts

Top 20 Robot Videos of 2010
Tue, January 11, 2011

Blog Post: Quadrotors performing acrobatics, ultrarealistic humanoids dancing, dexterous robots folding towels, and more

Who Wants a Free Robot?
Tue, May 04, 2010

Blog Post: Willow Garage is giving away PR2 robots to 11 institutions in the U.S., Europe, and Japan

Batcopter Does High-Tech Robotic Bat Harassment for Science

When bats leave their caves at night to go eat bugs, they can swarm in the millions while somehow managing to not crash into each other, which is a pretty clever trick. Kenn Sebesta, a researcher at Boston University, is wondering just how exactly they pull this off, and there's nothing better than good old fashioned experimentin' with robots to see how the bats do what they do.

This is Batcopter 2.0 (aka "Quady"), a home-built quadrotor made from carbon-fiber arrow shafts, twine, glue, zip ties, bamboo, foam, and netting to make sure that any bats not doing their jobs wouldn't get decapitated by a stray prop. A GoPro camera was stuck on the front and the whole thing was piloted from the ground with an array of three high-speed infrared cameras watching the glowing hot robot-on-bat nighttime aerial action:

To control the Batcopter, Sebesta says he and his colleagues used OpenPilot, an open source autopilot platform for small UAVs, which "allowed us to get so far so fast and was the real hero."

The UAV did end up having an unfortunate accident shortly thereafter, but not before collecting terabytes of high quality video of the bats interacting with movements of the UAV. The Batcopter team is planning to analyze this footage to try and see if there are any fundamental laws of flying that the bats follow to keep from colliding with other bats and wayward robots. If there are, it could lead to better autonomous flight controllers for UAVs, as well as ultrasonic squeaks of relief from bats everywhere as scientists find something else to do with their time.

UPDATE: No animals were harmed in the making of this robot! Professor John Baillieul, who directs Boston University's Laboratory for Intelligent Mechatronic Systems, writes us to say the researchers involved in the project, which includes several biologists, are very careful to design and use technology that is animal-friendly and meets all of the acceptable standards of animal care and use in the laboratory and field. "We do hope to use robotic air vehicles to observe bats and other flying animals in ways that have not been done up to now," Baillieul says, "but I can't emphasize too strongly that we have not harmed and are not seeking to harm or harass animals in any way, including making them fearful."

[ Boston University ] and [ OpenPilot.org ] via [ Slashdot ]

Thanks, Kenn!

Using Robots to Train the Surgeons of Tomorrow

This article is the first of a series that will explore recent advances in surgical and medical robotics and their potential impact on society. More articles, videos, and slideshows will appear throughout the year.

da vinci surgical system robot
Da Vinci surgical system. Photo: Kelleher Guerin

How can the skill of a surgeon be measured? A patient's body has no buzzer that alerts the surgeon when mistakes occur during an operation. There is no Yelp-like website that ranks a surgeon based on user reviews. It is surprising that people can spend less time selecting a surgeon for an operation than they might selecting a restaurant for dinner or a mechanic to fix their car.

According to a study from the U.S. Agency for Healthcare Research and Quality,  surgical complications, including postoperative infections, foreign objects left in wounds, surgical wounds reopening, and post-operative bleeding, resulted in a total of 2.4 million extra days of hospitalization, $9.3 billion excessive charges, and 32,000 mostly surgery-related deaths in the United States in 2000.

To what extent training is responsible for those errors is unknown. Some argue that most surgeons never achieve true expertise. One thing is certain, though: Residents need better, more effective training. It isn’t sufficient to have residents merely go through the motions; they must be able to practice deliberately. The problem is that residents already work inhumanely long hours (recent regulations limit their training to 80-hour work weeks, but they typically work more than that) and they must learn a growing number of surgical techniques and technologies, which means new generations of surgeons are having less and less time for hands-on practice.

In the past few years, several research groups, including our team at Johns Hopkins University, have been working to analyze and automate the training process using modern robotic surgical tools. Our goals are to create an objective, standardized method of surgical training as well as to reduce the time and cost of having an experienced surgeon in the training loop.

Surgical skill can be broken down into theoretical skill (consisting of factual and decision-making knowledge) and practical skill (the ability to carry out manual tasks such as dissection and suturing). Theoretical skill is often taught in a classroom and is thought to be accurately tested with written examinations like the Medical College Admission Test (MCAT) and the United States Medical Licensing Examination (USMLE). Practical skill, on the other hand, is much more difficult to judge.

Practical skills, such as driving a car, swinging a golf club, or throwing a football, are most effectively taught "in the field" through demonstration. In 1889, Sir William Halsted at Johns Hopkins University revolutionized surgical training by developing an apprentice-style technique still being used in most modern training programs of surgical residents today. According to this method, a resident would “see one, do one, teach one,” implying that after minimum exposure and the completion of a procedure once, a resident will have mastered the skill and will be capable of teaching the next novice. (Residents practice certain procedures more than once, but the principle is still that one time is really all the exposure they'd need before going out in the field and performing on their own.) Although many talented surgeons are trained this way, the method is time consuming, and evaluating a student's performance is a subjective task that varies depending on the student/teacher pair. The method also involves a lot of yelling.

With the advent of technologies such as robotic surgical systems and medical  simulators, researchers now have the tools to analyze surgical motion and evaluate surgical skill. Our group is studying human-machine interaction for surgical training and assistance in multiple contexts with increasing levels of complexity. The first level involves a system that understands what the human and environment are doing. The next level of interaction is for machines to provide assistance to a human operator through augmentation. The last level is to have a robot perform a task autonomously. We'll describe the state of research in each of these areas.

Understanding the surgical environment

language of surgery
Language of surgery. Photo: Carol Reiley

There is an active effort to develop new approaches to surgical training and evaluation. Using techniques from speech recognition, our group is developing mathematical models for motion recognition and skill assessment. These models may be the key to standardizing surgical training by decomposing complex surgical tasks like suturing, blunt dissection, and cutting into elementary “chunks” of motion -- and thus decode the "language of surgery."

These motions can be compared to phonemes, the elementary units of speech. Sequences of subtasks can be constructed like words to form sentences (analogous to various surgical tasks), which can then be used to form paragraphs (analogous to surgical operations). And, just as in speech, a recognition program might call attention to poor "pronunciation" or improper "syntax" in surgical execution, and can try to understand the intent of the surgeon from recorded motion and video data. (This research typically focuses on telepresence surgery as performed using the da Vinci system from Intuitive Surgical.) Using our skill evaluation system, trainees can have their trials evaluated offline or see their trial synchronized with a prerecorded expert trial to shorten the learning curve.

Augmenting the surgical environment

kidney stone image overlay
Kidney stone image overlay. Photo: Balazs Vagvolgyi

Super-surgeon performance can be achieved if human intelligence can be combined with robot accuracy and precision. Computer-integrated surgery, using equipment such as a robotic system with a video display, can enhance human senses by providing additional information. For example, the visualization can overlay a reconstructed CT scan of a tumor on the operating site, or the robot can use force feedback to prevent a surgeon’s hand from puncturing a beating heart.

Studies have shown that superimposing graphics, sounds, and forces over the real-world environment in real-time can assist with training.

Robots with intelligent sensors can address humans’ physiological limitations such as poor vision or hand tremor. Even the best surgeons can use intelligent assistance to improve performance. Force sensing “smart” surgical instruments will allow for safer and more effective surgeries. For example, they can be used to measure the local tissue oxygen saturation on the working surfaces of surgical retractors and graspers so that tissue doesn’t become permanently damaged.

steady hand eye robot
JHU Steady Hand-Eye Robot. Photo: Marcin Balicki

The JHU Steady-Hand Eye Robot is a robot used for retinal microsurgery where the surgeon and the robotic manipulator share the control of the instrument. This reduces hand tremors and allows for precise and steady motion. Shaky-handed surgeons, there’s hope for you yet!

The robot surgeons of the future

Researchers are now moving towards understanding how humans and machines can work together as a team to collaboratively finish a surgical task. Training models can be used to automate portions of a tedious task or to predict surgeons’ intent to automate an instrument change. Automation might also allow a surgeon to utilize more than two arms of the system at the same time: although the da Vinci surgical system has four arms (three to hold tools and one for the camera), the third arm generally sits idle, since humans can only control two arms at any given moment.

university of washington raven surgical robot biorobotics laboratory
University of Washington Raven. Photo: BioRobotics Lab

The University of Washington’s Raven System is an impressive mobile surgical robot used for telesurgery. In the next few months, seven schools are receiving this system as a part of a multi-institutional grant: Johns Hopkins University, UC Santa Cruz, University of Washington, UC Berkeley, Harvard, University of Nebraska, and UCLA. A few orders are already in for the next iteration that include schools in Florida, Toronto, and Minnesota. This standardized research surgical platform will lead to exciting new research in telesurgery and surgical training these next few years.

Raven is a mobile laparoscopic surgical system. Because Raven is modular, it is more portable than massive surgical robots used in hospitals and is able to be reassembled by a team of people. And while most commercial surgical robots weigh nearly half a ton, Raven is only 23 kilograms (about 50 lbs). This makes it ideally situated for hazardous environments.

Telesurgery experiments with the Raven generally involve a surgeon at a safe location operating a robot in the field; for example, underwater in a submarine pod or in the desert under scorching temperatures and gusting winds. Control commands and sensor feedback are transferred over a wireless connection.  Research questions include how time delays affect performance, how multiple surgeons can operate robots together to complete a surgery, and how surgeons can train on the platform most effectively.

The surgical environment in the operating room is unlike any other because of the constantly moving objects, because no two procedures are identical, and because of the sterilization/FDA approval issues. The state of surgical robotics is still a long way from one-button autonomous surgery, but the future of surgical training might be undergoing a major “facelift.”

About the authors:

Carol E. Reiley is currently finishing her doctoral research in surgical robotics at Johns Hopkins University and running TinkerBelle Labs, focused on creating low-cost, do-it-yourself projects. Reiley, who was the student chair on the IEEE Robotics and Automation Society for 2008-2010, earned her bachelors at Santa Clara University in computer engineering and her masters in computer science at Johns Hopkins.

Gregory D. Hager, an IEEE Fellow, is a professor in the computer science department at Johns Hopkins University, where his research interests include computer vision, robotics, medical devices, and human-machine systems. He directs the Computational Interaction and Robotics Lab and is the deputy directory of the NSF Engineering Research Center for Computer-Integrated Surgical Systems and Technology (CISST).

Advertisement

Automaton

IEEE Spectrum's award-winning robotics blog, featuring news, articles, and videos on robots, humanoids, automation, artificial intelligence, and more.
Contact us:  e.guizzo@ieee.org

Editor
Erico Guizzo
New York, N.Y.
Senior Writer
Evan Ackerman
Berkeley, Calif.
 
Contributor
Jason Falconer
Canada
Contributor
Angelica Lim
Tokyo, Japan
 

Newsletter Sign Up

Sign up for the Automaton newsletter and get biweekly updates about robotics, automation, and AI, all delivered directly to your inbox.

Advertisement
Load More