Now that robots aren’t allowed to play with beer anymore, we need a new type of demo that’s guaranteed to be crowd-pleasing. At FZI in Germany (where, for the record, they can still use beer because some things just make more sense over there), they’ve put together a very tasty cookie decorating demo with their humanoid robot HoLLiE. The demo features sprinkles, marshmallows, and a “frosting gun”—a thing I didn’t know existed but now desperately want.
Although the hat and the sign might throw you off, HoLLiE is not, in fact, a professional baking robot. Developed at FZI’s House of Living Labs, HoLLiE is designed to “manage complex tasks easily and support people in everyday situations.” The robot is human-sized and mobile, with a similar workspace to a human, featuring 6-DoF arms and 9-DoF Schunk hands along with a bendable waist that allows it to reach all the way down to the floor.
Obviously, this particular cookie decorating application isn’t a game changer for HoLLiE (or robots in general), but what’s interesting here is how the system is designed to make it effortless for a completely inexperienced human to direct the robot to decorate a customized cookie interactively and in real time. A more traditional solution would be to have people design the cookie digitally first, use some software to determine how to draw the design, and then send a complete set of instructions to the robot. While this works fine, it doesn’t allow the user to change their mind—or more importantly, to modify their design on the fly in response to what the robot is actually doing. This has the potential to be useful for all kinds of collaborative robotics tasks, where humans and robots have to dynamically cooperate to achieve a goal.
For more tasty details about CookieBot, we spoke with Arne Roennau at FZI via email.
IEEE Spectrum: Where did you get the idea for this demonstration?
Arne Roennau: In general, we believe that service robots are already capable of doing cool tasks, and not just in industrial environments. We have this Stallwächter event where we show what can already be done with robots, like the BratWurst Bot from 2016.
There were many crazy ideas. We wanted to do something that includes or integrates the user into the demo, but it should also be interesting and fun to watch somebody else interact with the robot. And at the end the user should be able to gain some reward— the CookieBot fits well to all these points. Besides, HoLLiE needed new arms and this was the perfect event to give it a hardware upgrade.
What was the hardest problem that you had to solve?
Physics! We underestimated the difficulty of adjusting the consistency of icing to be able to continuously write for hours. If the icing is too watery, it will make very blurry lines on the cookie. If it is not watery enough, it will not drip and not flow. Fluid dynamics is tough! But also particle physics is more complex than you realize in everyday life. Even sprinkles and marshmallows don’t always behave in the way you’d expect. Nevertheless, by varying many control and mechanical (hole diameter) parameters, we achieved good results.
The demo integrated a web-based interactive UI with a high precision motion tracking system and the ROS-based FZI motion pipeline.Photo: FZI
Why is this demonstration important for robotics research?
Field applications don’t always create new concepts or algorithms, but they can help to evolve the applied approaches. In the case of HoLLiE’s Patisserie, we were relying on our ROS-based FZI motion pipeline, complex state machines in FlexBE, and intuitive interaction concepts.
We combined a web-based interactive UI with a high precision motion tracking system to create a very clear way of interacting with a highly complex robot. The direct motion control of the robot via motion tracking is not new, but it was extended with semantic interpretations and safety restrictions. Basically, with no training, anybody was able to decorate their cookie individually.
The combination of all these technologies and concepts makes this a complex, but really compelling robotics demo. And we like bringing service robots to the public—this opens new fields and new challenges.
Are there new things that you learned from how people interacted with the robot?
Shared control concepts work really well and people love them. We developed a very advanced UI with fluid and particle dynamics to give the user a realistic impression of their cookie, which looked great and people were going crazy with their designs, but the real cookies did not turn out exactly like the virtual cookies. At least for us, it wasn’t possible to have the simulation in UI closer to the real cookie, but maybe it would be better to use more abstract visualisations or include a video stream of the real cookie next time.
What improvements would you like to make?
Switching the baking tools could be a bit faster—we programmed the motions to be slow to keep the demo safe, but after watching all night it seemed a bit too slow. It would also be great if we could understand (or influence) the consistency of the icing better in the future. This could improve the writing quality, which was already really good from a robotics control perspective. And maybe we could also bake the cookies with a robot next time :)
We think that’s an excellent idea!
[ FZI CookieBot ]
