Swarms of robots might not always seem like the friendliest things to have in your life, but next time you need a tiny stairway, this hard-working little robot named Kali and a bunch of its friends are here to serve.
Kali is part of Harvard's Termes project, which is developing a swarm construction system where lots of little robots team up to build structures that would be impossible for any one single little robot to put together. It's called "Termes" after our noble and endlessly destructive pals the termites, who use teamwork to fabricate mounds of earth up to 30 feet high. Like termites, Termes robots are simple and autonomous, and are able to cooperatively move heaps of standard building blocks (specially designed to allow the robots to both lift and crawl around on them) to create just about anything, as long as you give them enough blocks and enough time.
For example, the demo below shows Kali using just a few simple sensors to autonomously construct a staircase to scale a wall of an unknown height, based on previous experience with such situations:
Nicely done. While this is just one robot, you can easily extrapolate to what might be possible with swarms of robots, and it's not just bigger staircases. Get enough of these little guys together and they'll build you your very own fort, as the simulation at the end of this next video shows:
The Termes Project comes from Harvard's Self-organizing Systems Research Group, the same dudes responsible for the kill-o-botskilobots we met last week. They'll be presenting this work at the 2011 Robotics: Science and Systems conference next week in LA, and you can read the full paper at their website.
Somehow, it's been an entire year since the 2010 RoboBoat Competition. Rather than letting all of those industrious teams improve their robots to be better able to complete the existing course, the organizers added a whole bunch of practically impossible new challenges. Practically impossible, sure, but also pretty sweet, since they involve using deployable rovers to retrieve objects and autonomous water cannons to put out (fake) fires.
You may be wondering why such seemingly trivial tasks like navigating between different colored buoys is so tricky, but remember that this is all taking place on water, which is covered in nasty things like reflections and waves and hostile swans. So whenever the sun angle changes (an event that tends to happen quite often throughout the day), everything looks slightly different for the boats' cameras, sensors, and vision algorithms.
Anyway, luckily for you there's some excellent video recap of all three days of the event, so you can ignore my blathering and just watch things unfold for yourself. Swans beware!
One of the cheapest and most effective pieces of 3D mapping and gesture sensing hardware you could possibly hope for has just gotten an official SDK (software development kit) release. We're talking about Kinect, of course, and Microsoft has benevolently decreed that you no longer have to hack the sensor to get some non-gaming use out of it. Here's a few things you have to look forward to:
Raw sensor streams: Access to raw data streams from the depth sensor, color camera sensor, and four-element microphone array enables developers to build upon the low-level streams that are generated by the Kinect sensor.
Skeletal tracking: The capability to track the skeleton image of one or two people moving within the Kinect field of view make it easy to create gesture-driven applications.
Advanced audio capabilities: Audio processing capabilities include sophisticated acoustic noise suppression and echo cancellation, beam formation to identify the current sound source, and integration with the Windows speech recognition API.
Kinect is just one example of how robotics has been successfully piggybacking on other tech to get access to sensors and other hardware that's super effective and super cheap at the same time. Microsoft isn't making Kinect for robotics, but we don't care, we're perfectly happy to steal it and put it to better use than they ever could. I mean, come on, games? Psh! Try this stuff on for size.
The other advantage of having cheap and effective hardware with an SDK is that it helps the robotics community share ideas. It's the same basic philosophy as the PR2 (and ROS): if everyone's developing for the same platform, you can save yourself tons time and money by sharing code. So from a hobby robotics standpoint, you don't have to know a lot about Kinect to take advantage of it, since you can just adapt the clever things that other people have developed for the platform to your particular project.
You can download the Kinect SDK beta right now; it's free, but Windows 7 only and for use with Visual Studio in C++, C#, or VB. If you still need the hardware, Kinect sensors are a mere $150 at your friendly local gaming emporium.
Oh and by the way, we should also mention that the original Kinect hardware developer, PrimeSense, has partnered with Asus to develop a PC version of Kinect that they're calling "WAVI Xtion." No, I don't know how it's pronounced, but I do know that you can expect it in the second quarter of 2011, i.e. pretty much now.
These are Kilobots. They're fairly simple little robots about the size of a quarter that can move around on vibrating legs, blink their lights, and communicate with each other. On an individual basis, this isn't particularly impressive, but Kilobots aren't designed to be used on an individual basis. Costing a mere $14 each and buildable in about five minutes, you don't just get yourself one single Kilobot. Or ten. Or a hundred. They're designed to swarm in the thousands, although the Harvard group that's working on them is starting out with a modest 25:
We've seen lots of examples of swarm robotics, but what we decide to call a "swarm" often isn't, really. There is (or should be, at any rate) a distinction between a group of robots cooperating on a task and a true swarm of robots, and for the purposes of this article, I'm going to arbitrarily assert that a group of robots turns into a swarm of robots when you can't easily count how many individual robots there are. So like, these swarming MAVs? Not really a swarm. Swarmanoid? Not a swarm yet. Swarm bots are getting closer. What definitely makes the cut are projects like RoboSwarm and FlyFire, which use anywhere from hundreds to thousands of small robots all at once.
There's a lot you can do with gigantic swarms of robots, but there are two big obstacles to deploying them: programming, and charging. If you can't figure out a way to do these things efficiently (i.e. not on an individual basis for each robot), it negates a big part of the swarm appeal. In the case of the Kilobots, they can all be programmed at once with an infrared controller, and to charge them, the bots can simply be sandwiched between two conductive surfaces. The fundamental idea here is that any interaction with a robot swarm has to be scalable, such that an increase in the number of robots in the swarm doesn't result in an increase in the amount of time it takes to interact with the swarm.
I should point out that the other big obstacle to robot swarm deployment is price, which is why kilobots are deliberately so cheap: at $14 each, a thousand robots is actually an achievable number with a modest grant, which is something that probably has not been possible before. Generally people who want to experiment with large swarms have had to be content with computer simulations, which is fine, but at some point you have to try things out in the real world (or as close as you can get in a lab), and Kilobots can make that happen.
The Self Organizing Systems Research Group at Harvard is planning to expand their Kilobot collective to 1024 robots, and then they'll teach the swarm to demonstrate behaviors like self-healing and collective transport. Better hide your kids. Also, for the record, I'm pretty sure it's "Kilobots" and not "kill-o-bots." But who really knows until it's too late, right?
Armed robots have been making their way from science fiction to mainstream combat at an aggressive pace. The U.S. military is trying tobe cautious about the whole thing (too cautious for some and not cautious enoughfor others), but most people would probably acknowledge that increased reliance on unmanned systems is, for better or worse, an inevitability. This is because robots offer many advantages in conflict zones, the first and foremost being that sending a robot into a dangerous situation often means that a human doesn't have to go into that same situation.
These advantages aren't realized solely by the U.S. military. They're not realized solely by governments in general, either. Robots have been getting cheaper and more accessible, and people with an interest in robotics have for years been able build their own systems to take over work that's dull, dirty, or dangerous. It should be no surprise, then, that rebels in Libya have started cobbling together their own armed robots out of Power Wheels toys, video cameras, radios, and machine guns:
So what does this mean for the present and future of military robotics? First, it's a vivid illustration of the potential implications of a rapidly descending barrier to entry for this kind of technology. Anyone can (on principle, at least) build a robot, and given the need or the motivation, anyone can put a gun on one, too. Second, the fact that anyone can build something like this is an equally vivid illustration that despite whatever qualms we may have about military robotics, it's not only going to happen, it's happening already. Whatever the ethical implications may be, this is becoming the new reality faster than we might like, and it's something that we're going to have to prepare for.
Tom Coburn, a senator from Oklahoma, and PR2, a robot from California.
A U.S. senator has cited three robotics projects as examples of "wasteful" research that lack useful applications and shouldn't have received government funding.
In a recent report, Senator Tom Coburn of Oklahoma takes aim at the National Science Foundation, the premier source of funding for science and engineering in the United States, raising questions about the agency's management and priorities. In one section of the report, Coburn criticizes the NSF for squandering "millions of dollars on wasteful projects," including three that involve robots.
"A dollar lost to mismanagement, fraud, inefficiency, or a dumb project is a dollar that could have advanced scientific discovery," the report says.
Coburn didn't give the roboticists a chance to respond, so I reached out to the three groups—from the University of California, Berkeley; University of California, Davis; and Rowan University, in Glassboro, N.J.—to hear their side.
Of course, they aren't exactly thrilled to see their work "featured" in the report. One scientist quipped that Coburn has just sparked a robot uprising. Picture hordes of bots descending on Washington, D.C. to show the senator who's wasteful by using him as cookie dough.
The researchers say they welcome scrutiny and agree that there are manyimprovements the NSF could make. But they argue that the Coburn report evaluated their projects superficially and out of context.
But apparently Coburn wasn't impressed. His report notes that the robot cost $1.5 million and complains that it "took nearly 25 minutes to fold each towel." [UPDATE: The report references the wrong NSF grant; this is the correct one, a $1.2 million award. And the Berkeley researchers got the robot for free.]
Here's the "exclusive" unveiling of the report on ABC's "Good Morning America."
Berkeley computer science professor Pieter Abbeel, one of the researchers behind the project, told me that the towel folding experiment was just a small part of a much broader effort aimed at creating robots that can handle the complexities of real environments. Here's what he wrote in a rebuttal:
"[I]n order to expand the use of robots beyond manufacturing the machines must be far more sophisticated in terms of their ability to deal with complexity. That's what our work is all about. Towel folding is just a first, small step towards a new generation of robotic devices that could, for example, significantly increase the independence of elderly and sick people, protect our soldiers during combat, and a host of other applications that would revolutionize our day-to-day lives."
Coburn also discussed the report with Neil Cavuto from Fox News. After seeing footage of the PR2 folding towels, Cavuto says: "I guess many folks would like that. But how's the robot doing? Did it indeed fold clothes?" The senator admits he doesn't know details about the project. "It just caught my eye," he says.
I asked Coburn's office for more information on how they selected the projects they thought shouldn't have been funded. Did researchers or policy experts with relevant scientific backgrounds help Coburn prepare the report? Who are his co-authors?
Coburn "is the author of the report," John Hart, the senator's spokesman, told me in an e-mail. He added that the senator, who is a physician, "does have a scientific background," in addition to a business, accounting, and public policy background. "This is a multi-dimensional discussion."
I also asked whether Coburn and his staff contacted the researchers prior to the publication of the report to ask for more information or offer them a chance to address the criticism.
"Yes," Hart said. "Scientists and researchers who are privileged to receive federal funds should welcome and expect questions about their work." He added: "There are no sacred cows that should avoid examination and, if necessary, dissection."
But all the researchers I contacted told me they never heard from Coburn's staff. They said they were puzzled that the report relies so much on press reports rather than material with more scientific content—an approach they found a bit, well, unscientific. One researcher asked if Coburn would judge whether a patient is sick just by looking at the person's face.
In another project criticized in the report, a UC Davis group is studying how people interact with and control their bicycles. The researchers also want to build a robotic bike.
The researchers are using a bike equipped with sensors [photo above] and also building a robotic bicycle to identify the parameters that their models need to take into account. As it turns out, Hubbard says, we know very little about how a bike's design affects safety, performance, and our ability to control it. In particular, we need to learn more about how the dynamics of the bike and rider affect each other.
"There's plenty to be discovered," Hubbard says. "Just because Senator Coburn knows how to ride a bicycle, it doesn't mean that's the end of it."
He adds that increasing bicycle usage would have "health benefits, transportation benefits, environmental benefits." Surveys show that although Americans don't bike much, many more would if they felt bikes were safer, he said.
The third project criticized in the report was a "robot rodeo," a three-day event that took place at a conference for computer science educators in Dallas, Texas, last year. The organizers, Jennifer Kay, a computer scientist at Rowan University, and Tom Lauwers, a robotics entrepreneur, say the goal of the event was to "introduce robot programming to the nearly 1200 educators attending the conference, and to raise awareness amongst participants of how robots could be used in their classrooms."
They say that despite evidence that robots can be used as educational tools to excite and motivate students, only a tiny fraction of educators have ever programmed a robot or tried them in their classrooms. They told me that the event—which involved months of planning and dozens of volunteers—received only $6,283 from the NSF, a number that the Coburn report doesn't mention. (Just for reference, that's one-fifth of what the Senate Hair Care Revolving Fund spent last year.)
And yes, Kay and Lauwers say, the event was designed to be fun:
"Perhaps the Robot Hoedown and Rodeo was singled out because it has an intentionally eye-catching name, and because on the surface it appears 'fun.' Indeed in his report Senator Coburn states, 'Videos of the event posted to YouTube suggest the effort was a source of enjoyment for observers.' It is precisely this 'fun' which our program aims to associate with Computer Science education, so that our current students will choose to become the future researchers that make the kinds of transformative discoveries that improve our society and our economy."
Coburn acknowledges that NSF grants have supported many scientific breakthroughs, but he insists that the agency could save between $1 billion and $3 billion by eliminating inefficiencies and duplication.
Among other things, he calls for the NSF to defund its social and behavioral sciences division and sharpen its focus on "truly transformative sciences with practical uses outside of academic circles and clear benefits to mankind and the world." (Full disclosure: IEEE Spectrum has collaborated with the Directorate for Engineering of the National Science Foundation to coproduce "Robots for Real," an award-winning special report with clear benefits to mankind and the world.)
But picking "winners" is a challenge even for experienced NSF program managers and the scientists who help the agency review its grant applications.
"In many cases, it can be difficult to identify, in advance, what kinds of research proposals might lead to transformative results," says Dana Topousis, an NSF spokesperson. "For instance, when NSF funded a graduate research fellow in the early 1990s to study digital libraries, we couldn't predict that that graduate student would co-found Google."
So who knows? The next Google may very well be a robotics company founded by a pair of NSF-funded researchers. Then again, there's only one way to find out.
When bats leave their caves at night to go eat bugs, they can swarm in the millions while somehow managing to not crash into each other, which is a pretty clever trick. Kenn Sebesta, a researcher at Boston University, is wondering just how exactly they pull this off, and there's nothing better than good old fashioned experimentin' with robots to see how the bats do what they do.
This is Batcopter 2.0 (aka "Quady"), a home-built quadrotor made from carbon-fiber arrow shafts, twine, glue, zip ties, bamboo, foam, and netting to make sure that any bats not doing their jobs wouldn't get decapitated by a stray prop. A GoPro camera was stuck on the front and the whole thing was piloted from the ground with an array of three high-speed infrared cameras watching the glowing hot robot-on-bat nighttime aerial action:
To control the Batcopter, Sebesta says he and his colleagues used OpenPilot, an open source autopilot platform for small UAVs, which "allowed us to get so far so fast and was the real hero."
The UAV did end up having an unfortunate accident shortly thereafter, but not before collecting terabytes of high quality video of the bats interacting with movements of the UAV. The Batcopter team is planning to analyze this footage to try and see if there are any fundamental laws of flying that the bats follow to keep from colliding with other bats and wayward robots. If there are, it could lead to better autonomous flight controllers for UAVs, as well as ultrasonic squeaks of relief from bats everywhere as scientists find something else to do with their time.
UPDATE: No animals were harmed in the making of this robot! Professor John Baillieul, who directs Boston University's Laboratory for Intelligent Mechatronic Systems, writes us to say the researchers involved in the project, which includes several biologists, are very careful to design and use technology that is animal-friendly and meets all of the acceptable standards of animal care and use in the laboratory and field. "We do hope to use robotic air vehicles to observe bats and other flying animals in ways that have not been done up to now," Baillieul says, "but I can't emphasize too strongly that we have not harmed and are not seeking to harm or harass animals in any way, including making them fearful."
This article is the first of a series that will explore recent advances in surgical and medical robotics and their potential impact on society. More articles, videos, and slideshows will appear throughout the year.
Da Vinci surgical system. Photo: Kelleher Guerin
How can the skill of a surgeon be measured? A patient's body has no buzzer that alerts the surgeon when mistakes occur during an operation. There is no Yelp-like website that ranks a surgeon based on user reviews. It is surprising that people can spend less time selecting a surgeon for an operation than they might selecting a restaurant for dinner or a mechanic to fix their car.
According to a study from the U.S. Agency for Healthcare Research and Quality, surgical complications, including postoperative infections, foreign objects left in wounds, surgical wounds reopening, and post-operative bleeding, resulted in a total of 2.4 million extra days of hospitalization, $9.3 billion excessive charges, and 32,000 mostly surgery-related deaths in the United States in 2000.
To what extent training is responsible for those errors is unknown. Some argue that most surgeons never achieve true expertise. One thing is certain, though: Residents need better, more effective training. It isn’t sufficient to have residents merely go through the motions; they must be able to practice deliberately. The problem is that residents already work inhumanely long hours (recent regulations limit their training to 80-hour work weeks, but they typically work more than that) and they must learn a growing number of surgical techniques and technologies, which means new generations of surgeons are having less and less time for hands-on practice.
In the past few years, several research groups, including our team at Johns Hopkins University, have been working to analyze and automate the training process using modern robotic surgical tools. Our goals are to create an objective, standardized method of surgical training as well as to reduce the time and cost of having an experienced surgeon in the training loop.
Surgical skill can be broken down into theoretical skill (consisting of factual and decision-making knowledge) and practical skill (the ability to carry out manual tasks such as dissection and suturing). Theoretical skill is often taught in a classroom and is thought to be accurately tested with written examinations like the Medical College Admission Test (MCAT) and the United States Medical Licensing Examination (USMLE). Practical skill, on the other hand, is much more difficult to judge.
Practical skills, such as driving a car, swinging a golf club, or throwing a football, are most effectively taught "in the field" through demonstration. In 1889, Sir William Halsted at Johns Hopkins University revolutionized surgical training by developing an apprentice-style technique still being used in most modern training programs of surgical residents today. According to this method, a resident would “see one, do one, teach one,” implying that after minimum exposure and the completion of a procedure once, a resident will have mastered the skill and will be capable of teaching the next novice. (Residents practice certain procedures more than once, but the principle is still that one time is really all the exposure they'd need before going out in the field and performing on their own.) Although many talented surgeons are trained this way, the method is time consuming, and evaluating a student's performance is a subjective task that varies depending on the student/teacher pair. The method also involves a lot of yelling.
With the advent of technologies such as robotic surgical systems and medical simulators, researchers now have the tools to analyze surgical motion and evaluate surgical skill. Our group is studying human-machine interaction for surgical training and assistance in multiple contexts with increasing levels of complexity. The first level involves a system that understands what the human and environment are doing. The next level of interaction is for machines to provide assistance to a human operator through augmentation. The last level is to have a robot perform a task autonomously. We'll describe the state of research in each of these areas.
Understanding the surgical environment
Language of surgery. Photo: Carol Reiley
There is an active effort to develop new approaches to surgical training and evaluation. Using techniques from speech recognition, our group is developing mathematical models for motion recognition and skill assessment. These models may be the key to standardizing surgical training by decomposing complex surgical tasks like suturing, blunt dissection, and cutting into elementary “chunks” of motion -- and thus decode the "language of surgery."
These motions can be compared to phonemes, the elementary units of speech. Sequences of subtasks can be constructed like words to form sentences (analogous to various surgical tasks), which can then be used to form paragraphs (analogous to surgical operations). And, just as in speech, a recognition program might call attention to poor "pronunciation" or improper "syntax" in surgical execution, and can try to understand the intent of the surgeon from recorded motion and video data. (This research typically focuses on telepresence surgery as performed using the da Vinci system from Intuitive Surgical.) Using our skill evaluation system, trainees can have their trials evaluated offline or see their trial synchronized with a prerecorded expert trial to shorten the learning curve.
Augmenting the surgical environment
Kidney stone image overlay. Photo: Balazs Vagvolgyi
Super-surgeon performance can be achieved if human intelligence can be combined with robot accuracy and precision. Computer-integrated surgery, using equipment such as a robotic system with a video display, can enhance human senses by providing additional information. For example, the visualization can overlay a reconstructed CT scan of a tumor on the operating site, or the robot can use force feedback to prevent a surgeon’s hand from puncturing a beating heart.
Studies have shown that superimposing graphics, sounds, and forces over the real-world environment in real-time can assist with training.
Robots with intelligent sensors can address humans’ physiological limitations such as poor vision or hand tremor. Even the best surgeons can use intelligent assistance to improve performance. Force sensing “smart” surgical instruments will allow for safer and more effective surgeries. For example, they can be used to measure the local tissue oxygen saturation on the working surfaces of surgical retractors and graspers so that tissue doesn’t become permanently damaged.
JHU Steady Hand-Eye Robot. Photo: Marcin Balicki
The JHU Steady-Hand Eye Robot is a robot used for retinal microsurgery where the surgeon and the robotic manipulator share the control of the instrument. This reduces hand tremors and allows for precise and steady motion. Shaky-handed surgeons, there’s hope for you yet!
The robot surgeons of the future
Researchers are now moving towards understanding how humans and machines can work together as a team to collaboratively finish a surgical task. Training models can be used to automate portions of a tedious task or to predict surgeons’ intent to automate an instrument change. Automation might also allow a surgeon to utilize more than two arms of the system at the same time: although the da Vinci surgical system has four arms (three to hold tools and one for the camera), the third arm generally sits idle, since humans can only control two arms at any given moment.
University of Washington Raven. Photo: BioRobotics Lab
The University of Washington’s Raven System is an impressive mobile surgical robot used for telesurgery. In the next few months, seven schools are receiving this system as a part of a multi-institutional grant: Johns Hopkins University, UC Santa Cruz, University of Washington, UC Berkeley, Harvard, University of Nebraska, and UCLA. A few orders are already in for the next iteration that include schools in Florida, Toronto, and Minnesota. This standardized research surgical platform will lead to exciting new research in telesurgery and surgical training these next few years.
Raven is a mobile laparoscopic surgical system. Because Raven is modular, it is more portable than massive surgical robots used in hospitals and is able to be reassembled by a team of people. And while most commercial surgical robots weigh nearly half a ton, Raven is only 23 kilograms (about 50 lbs). This makes it ideally situated for hazardous environments.
Telesurgery experiments with the Raven generally involve a surgeon at a safe location operating a robot in the field; for example, underwater in a submarine pod or in the desert under scorching temperatures and gusting winds. Control commands and sensor feedback are transferred over a wireless connection. Research questions include how time delays affect performance, how multiple surgeons can operate robots together to complete a surgery, and how surgeons can train on the platform most effectively.
The surgical environment in the operating room is unlike any other because of the constantly moving objects, because no two procedures are identical, and because of the sterilization/FDA approval issues. The state of surgical robotics is still a long way from one-button autonomous surgery, but the future of surgical training might be undergoing a major “facelift.”
About the authors:
Carol E. Reiley is currently finishing her doctoral research in surgical robotics at Johns Hopkins University and running TinkerBelle Labs, focused on creating low-cost, do-it-yourself projects. Reiley, who was the student chair on the IEEE Robotics and Automation Society for 2008-2010, earned her bachelors at Santa Clara University in computer engineering and her masters in computer science at Johns Hopkins.
Gregory D. Hager, an IEEE Fellow, is a professor in the computer science department at Johns Hopkins University, where his research interests include computer vision, robotics, medical devices, and human-machine systems. He directs the Computational Interaction and Robotics Lab and is the deputy directory of the NSF Engineering Research Center for Computer-Integrated Surgical Systems and Technology (CISST).
Once upon a time, a charming American robot called James met a striking German bot by the name of Rosie. They liked each other, so they moved in together. Now they spend their days taking long walks in the lab and doing other things that robots do.
James is a PR2 robot, built by U.S. robotics firm Willow Garage, and it traveled to Germany as part of the PR2 Beta Program, an effort to popularize personal robots. At the Technical University Munich (TUM), James was introduced to Rosie, a dual-arm robot with a curvy figure and four eyes [photo above].
Their courtship was at first a bit mechanical, but they soon found many things in common: Both run ROS (Robot Operating System), use Hokuyo laser scanners and Kinect 3D sensors, and have omnidirectional mobile bases.
On a recent spring morning, James and Rosie were seen together cooking the traditional Weisswurst Frühstück, a Bavarian sausage breakfast.
It was a demonstration prepared by researchers at CoTeSys (Cognition for Technical Systems), a Munich-based high-tech cluster. This is how the researchers summarize the experiment:
TUM-Rosie is collecting the sausages, putting them into the pot with boiling water, waiting for them to be cooked and, finally, finding and getting them out of the pot into the serving bowl. [The PR2 robot] TUM-James is meanwhile slicing the french baguette using a regular electric bread slicer and in the end serving the sausages and the bread to the class of highly regarded roboticists. [...]
TUM-James makes use of recent advances in the field of real-time RGB-D sensing using a Kinect sensor for the detection of the bread slicer and the baguette. In the serving task it uses PR2's haptic capabilities in order to grasp and manipulate the plate.
TUM-Rosie is also using Kinect and perception algorithms from COP [cognitive perception] module in order to calibrate the skimmer and use it as a new tool center point of the arm. Furthermore it learns the 3D models for the pot and the bowls in order to be able to localize them at any arbitrary pose on the table. Lastly, it uses the torque sensors to resolve depth measurement inaccuracies through contact detection with the objects and blob segmentation in order to localize sausages inside the pot.
The couple has a promising life ahead of them, and we look forward to hearing about their future adventures and, hopefully, seeing some baby robots too.
PS: This is not the first romantic meal the robots have together. Last year, the pair prepared a somewhat more mainstream breakfast: pancakes. Guten Appetit!
This is PR2. PR2 plays pool. PR2 brings you beer. And now, or very soon anyway, PR2 will bake you cookies. Warm, gooey, chocolate chip cookies. Seriously, is this not the greatest robot in the world or what?
This video comes from graduate student Mario Bollini, who's a member of Daniela Rus' Distributed Robotics Lab at MIT CSAIL. It's not in the video, but as you can see from the picture, PR2 (or "bakebot" for the purposes of this demo) is also able to cream butter and sugar, and we already know that it can break (or not break) eggs. It does make a bit of a mess, which is the reason for the surgical smock, but a separate group is programming the robot to wipe down the table afterwards. Incidentally, I love how when PR2 finishes adding an ingredient to its mixing bowl, it just drops the container on the floor. Now that's my kind of clean-up.
Bollini hopes to have PR2 making cookies from start to finish within the shockingly short time of a month. Or actually, it'll be just making one single giant cookie at a time, but you know what, I'm totally okay with that.