Sony Partners With CMU to Develop Food Prep and Delivery Robots

A research agreement on robotics and AI will result in robot waiters and chefs

5 min read

Sony Partners With CMU to Develop Food Prep and Delivery Robots
Photo: iStockphoto

Last week, Sony and Carnegie Mellon University announced a collaboration “on artificial intelligence (AI) and robotics research.” Usually, these announcements pretty much just end there, with the implication being that giant corporation X will support academic research institution Y by funding ongoing research or a string of new initiatives. This Sony/CMU announcement is a bit more exciting because of how specific it is: The project will be about food.

Researchers will focus on defining the domain of food ordering, preparation, and delivery. Initially, they will build upon existing manipulation robots and mobile robots, and will plan on developing new domain-specific robots for predefined food preparation items and for mobility in a limited confined space. 


More from the press release:

Initial research and development efforts will focus on optimizing food preparation, cooking and delivery. Depending on the needs of the consumer, food offerings and preparation methods could be adjusted based on personal dietary restrictions and the availability of certain ingredients. Food could be delivered to the home or office, and dining tables could be set elegantly prior to food being served. 

This area of research and development was selected because the technology necessary for a robot to handle the complex and varied task of food preparation and delivery could be applied to a broader set of skills and industries. Applications could include those where machines must handle fragile and irregularly shaped materials and carry out complex household and small business tasks. Additionally, robots that are developed for food preparation and delivery would have to be able to operate in small areas, an ability which could be valuable for many other industries.

That’s about the extent of the info that’s been made public at this point, so to get a few more details, we spoke with Toshimoto Mitomo, a senior investment executive at Sony Innovation Fund, and Professor Manuela Veloso, who directs the Machine Learning Department at CMU’s School of Computer Science.

IEEE Spectrum: Why focus on food preparation and delivery with this project?

Toshimoto Mitomo: We think food preparation and delivery tasks encompass a comprehensive array of the many underlying technologies that will be necessary to advance robotics going forward, starting with human interaction. For example, food preparation and delivery require technologies that allow for readjustment of menus to accommodate for dietary restrictions, recipe planning, handling of ingredients with different sizes and shapes, setting up tables in an elegant way, and delivering to offices or right to the table.

We believe this covers a wide array of technologies relating to handling of different types and sizes of objects, small-scale, diverse production of ingredients for dietary restrictions, and performing services while interacting directly with humans. We consider these to be important functional elements when introducing AI and robotics to places where they can interact more closely with people. By going ahead with this project, we hope to help greatly expand the possibilities of the robotics business in the future. 

Can you describe what successful completion of the project might look like?

Manuela Veloso: There will be a robot that has some object manipulation capabilities and will be connected to a food request system, and then there will be another robot that will be able to deliver food. Within this initially time-limited project, we clearly expect a strong collaboration between humans and these two types of robots. We plan on initially using our existing hardware, such as robot arms and our mobile CoBot robots for navigation and food delivery. We think that the robots will be specialists to this task, but the variety of food for preparation and the diverse type of layouts for food delivery will set the need to have a general flexible approach. We also envision in particular the robots learning from demonstration, instruction, and correction from humans.

“There will be a robot that has some object manipulation capabilities and will be connected to a food request system, and there will be another robot that will be able to deliver food. We think that the robots will be specialists, but the variety of food and layouts will set the need to have a flexible approach. We envision the robots learning from demonstration, instruction, and correction from humans”

End-to-end tasks like these are challenging for robots right now. What specific problems will you be focusing on?

Manuela Veloso: The goal is to research a combined task of food preparation and food delivery to a specific space, similar to a restaurant. Focusing on such a “complete” task for autonomous robots sets up the research on AI, autonomous robots, and human-robot interaction. 

There are challenges in the multiple aspects of robotics and AI, namely perceptual, planning, learning, actuation, and human-interaction aspects. We plan to start with the domain definition and representation, the learning and human-robot interaction opportunities, and the relationship with the specific hardware. 

Is there potential for some of the technology developed in this partnership to become a consumer or commercial product?

Toshimoto Mitomo: With this project, we are tackling the tasks of food preparation and delivery and aiming to develop fundamental technologies that will help expand the robotics business in the years to come. We hope that the results of the project will be useful in a range of areas that go beyond simply preparing and delivering food. This is strictly a research project centered arounds the theme of food preparation and delivery, and is not being carried out with the goal of directly commercializing a cooking or delivery robot in mind.

However, we do believe that it will address some of the potential needs of both households and small-scale enterprises when looking to automate food preparation and delivery and make those processes more efficient.

What area of robotics is most exciting to you personally right now?

Toshimoto Mitomo: I have spent the past few years working on the aibo project. As you are probably aware, Sony recently introduced a new aibo in Japanese market and re-entered the robotics business. Personally, I feel that there are so many areas where we can make people’s lives better by introducing robotics technologies. Sony is looking to expand the possibilities of the robotics business in the future, and that is why we are so excited to be conducting this research with CMU.

With the notable exception of the new Aibo, Sony itself hasn’t been directly involved in robotics (that we know of) since it stopped making the old Aibo back in 2005. And it still isn’t really, although the Sony Innovation Fund has been making some significant robotics investments lately, in companies like Acutronic (developing H-ROS), Embodied (social robot intelligence), and of course Agility Robotics. We’ll try not to draw any premature conclusions from the fact that Agility Robotics’ Cassie is intended for home delivery, but we wouldn’t be surprised if a future version of Cassie (one with arms, maybe) ended up as part of this project.

In any case, we really like what Sony and CMU are doing here. They have a specific goal, tackling specific challenges that are at the forefront of robotics research right now, and we’re expecting to see some tangible results that could be real-world useful. What, if anything, Sony itself does with that remains to be seen, but it’s a good approach, and we hope that it pays off so that we see more collaborations like this.

[ Sony/CMU Press Release ]

The Conversation (0)