Yesterday we posted a video showing how robots can use machine learning to adapt to objects and situations that they've never seen before. The same research group at Cornell has also been working on clever ways to allow robots to efficiently interpret scenes and identify objects, which is one of those things that robots are going to have to be good at before they can really become helpful in our homes.
Humans have the ability to look at a scene and immediately pick out important elements while ignoring everything else, because we have brains that are awesome like that. Robots, in general, don't really work that way. They have to examine each and every pixel of what they're looking at and decide whether or not it's something they're interested in. So if you ask a robot go to find you (say) a computer keyboard, it's got to enter a room and methodically search every pixel-equivalent area until it finds what it's looking for. While the robot is likely to find its target in this manner, it'll take roughly forever to do so.
What the Cornell research group has been teaching robots to do is to be able to rapidly break down a scene into general categories, and then recognize how some categories are related to others. In a general sense, this is the same thing that we humans do. For example, if you enter a room and want to locate a computer keyboard, you first (subconsciously) identify the places that you don't need to pay attention to, like the walls and ceiling and probably the floor. Next, if you don't immediately see the keyboard, you might take a closer look at the places where keyboards like to hang out, namely in close proximity to a computer. You see a computer, you get closer, and hey, there's the keyboard. Nicely done.
Now, watch a robot do the exact same thing:Â
The key to getting this to work goes back to the machine learning and adaptability system that Cornell has been working on. First, they showed a robot a series of about 50 different scenes, with most objects in the scene labeled. The robot read the labels and then remembered characteristics of the associated objects (like color, texture, and what other objects are nearby), such that it was then able to use those characteristics categorize entirely new objects in environments it's never experienced.
Armed with this knowledge and way of thinking (if you want to call it that), the robot was then able to enter a room it had never been in before in search of a keyboard, and "know" that since it spotted a monitor, and keyboards are often found near monitors, it should go check out the monitor. And of course, the keyboard is right there.
This ability to follow chains of reasoning to identify, categorize, and locate objects is obviously still being developed, but again, giving robots the ability to understand context and use it to adapt to new things brings them that much closer to being the machines that take over all of our household chores and leave us with nothing better to do than lie around and play video games all day. Because that's what we all secretly want, right? ...Right?
[ Cornell ]
