Robots are getting much, much better at sensing and manipulation. It may not always feel like they’re making progress, but they are. Still, asking them to do anything autonomously, especially in unfamiliar or unstructured environments where things need to happen in real time, is usually only going to lead to disappointment, frustration, tears, and sometimes screaming.
For now, the reliable way of getting a robot to do what you want (when it involves perception and fine motor skills) is teleoperation. RE2 Robotics builds Highly Dexterous Manipulation System (HDMS) that are half the weight of a humanoid torso with twice the strength of a human, intuitively controlled by a sort of Waldo system. It’s primarily designed for military operations, but RE2 has a far more critical task: making balloon animals. See how they do it, plus lots more videos. It’s Friday!