Earlier this year, Diligent Robotics introduced a mobile manipulator called Poli, designed to take over non-care related, boring logistical tasks from overworked healthcare professionals who really should be doing better things with their time. Specifically, Diligent wants to automate things like bringing supplies from a central storage area to patient rooms, which sounds like it should be easy, but is actually very difficult. Autonomous mobile manipulation in semi-structured environments is hard at the best of times, and things get even harder in places like hospitals that are full of busy humans rushing around trying to save the lives of other humans.
Over the past few months, Diligent has been busy iterating on the design of their robot, and they’ve made enough changes that it’s no longer called Poli. It’s a completely new robot, called Moxi.
As a friendly, sensitive, and intuitive robot, Moxi not only alleviates clinical staff of routine tasks but does so in a non-threatening and supportive way that encourages positive relationships between humans and robots, further enhancing clinical staff’s ability to and interest in leveraging AI in the healthcare industry. Created with a face to visually communicate social cues and able to show its intention before moving to the next task, Moxi is built to foster trust between patients and staff alike, setting the stage for future innovation and partnerships with developing technology. Moxi’s specific tasks and responsibilities at each hospital will be tailored to fit each hospital’s needs.
While Diligent’s general concept for a mobile manipulator for hospitals is the same as it’s always been, Moxi is much, much different than its predecessor, Poli, that we wrote about in January. Moxi uses a Freight mobile base from Fetch Robotics, which seems like a reasonable thing to do if your company is about manipulation and human-robot interaction (HRI) and you just want the navigation and obstacle avoidance to work without you having to stress about it. Moxi is significantly more human-like than earlier designs (with a pronounced head and torso), which presumably makes HRI more straightforward, although there’s that Velodyne Puck that almost looks like it was added as an afterthought. For manipulation, the robot relies on a Kinova Jaco arm and Adaptive Gripper from Robotiq.
The video shows some fairly standard mobile manipulator capabilities—navigation and obstacle avoidance, plus the ability to pick items out of bins and drop them into what looks like a little partition on the base. We’re told that this is fully autonomous, though I’m not totally clear on what happens at the other end of this process, when presumably the robot needs to pick things out of its storage area (can it see down there?) and place them on shelves or in bins or something. And not to be needlessly suspicious, but there’s only so much we can infer from videos like this, because they invariably show the best case scenarios of how robots operate. Having said that, it’s also important to keep things in context: We’re seeing a prototype during a pilot, after all. If it worked perfectly, it would be a commercial product already, right?
The executive team (from left): Andrea Thomaz (CEO and co-founder), Vivian Chu (CTO and co-founder), and Agata Rozga (head of product). Photo: Diligent Robotics
For more details on how Diligent is working on getting Moxi towards that idea of a perfect commercial product, we spoke with CEO and co-founder Andrea Thomaz, who is also a professor of electrical and computer engineering at the University of Texas at Austin, where she leads the Socially Intelligent Machines Lab.
IEEE Spectrum: We spoke with you about Diligent in January; what have you learned from your pilot customers since then about deploying mobile manipulators in hospitals?
Andrea Thomaz: Our priority from the beginning has been to make healthcare professionals enjoy and feel supported in their work again, so we were most nervous that idea wouldn’t be heard given that many people see AI as a professional threat rather than a tool. But the more research we’ve done and the more enthusiastic healthcare staff we’ve met, we feel even more confident in our vision for Moxi (and many other AI services) to improve the roles of clinical staff by taking away their many repetitive, logistical responsibilities so they can spend more time with patients.
Our early customer engagements have validated the need for a robot to be adaptable to each location, both in terms of the physical environment for navigation and manipulation and in terms of what specific support tasks will be most valuable for different hospital departments depending on the patient population, unit workflows, etc. One expectation we have confirmed is that that millimeter accuracy navigation isn’t solved. As a result, we are coming up with solutions that use vision to get the robot to repeatable millimeter manipulation accuracy.
Another thing we’ve learned with our pilots is that each unit and healthcare environment is different and as a result, a wide variety of sensing is required. Specifically, we’re experimenting with sensors that have different scale, resolution, and distances. We are making use of sensor fusion and multimodal perception in order to allow Moxi better perceive each semi-structured environment.
What iterations did you go through to get from Poli to Moxi? What changes did you make to the hardware and why were they necessary?
We’ve made quite a few changes from Poli to Moxi, all focused on our idea of making a robot that feels more natural and accepted by people experiencing it in their everyday environment. Our very user-centric design approach helped us understand the limitations of Poli so we could improve Moxi. For example, we liked the safety, form-factor, and range of manipulation of the Kinova Jaco2, however, the original orientation limited the workspace of the arm. This change also improved the aesthetics of Moxi.
Another change was designing it so Moxi’s entire torso rests on a telescoping pillar instead of only the arm mounted on a linear actuator with Poli. Both designs achieve the manipulation goal of being able to have that degree of freedom to expand the workspace, reaching high shelves and down to the floor, but having a telescoping pillar solution accomplishes aesthetic and social goals by allowing the robot have a smaller footprint as it is working near people. Also having the arm and head move in tandem is important for it to feel socially appropriate.
For our pilot testing phase of the product, we added a Velodyne Puck (VLP-16), which allows us to perform perception tasks that cannot be done with just an RGB-D sensor in the head. As we mentioned earlier, this is one of the sensors we are fusing to allow Moxi to perceive its environment. For closer range depth perception, we selected the Intel RealSense as a replacement to the Kinect sensor used on Poli. We did a series of benchmarking with a variety of RGB-D cameras and found this was the best in market for us right now.
We are using the Fetch F100 as our mobile base with Moxi. Our biggest constraint in a hospital environment is the footprint of the base, and after piloting the F100 we’ve found that it’s able to navigate all the spaces that we need.
The visual design for Moxi is much different than the design for Poli. Can you talk about some specific HRI-related design decisions that you made, and why you made them?
One of our biggest design goals with our vision for making Moxi more natural in human environments was to downsize Moxi from Poli’s size, so Moxi doesn’t feel obstructive in busy healthcare environments. Our second design goal was to make Moxi feel like it’s friendly, and an important part of the team. Through the brilliant work of our industrial designer, Carla Diana, we determined Moxi needed to have soft, unified, and rounded design features.
From an HRI perspective, we are always trying to find intuitive ways that the robot’s behavior can be transparent for the people around it. We decided for Moxi to demonstrate social intelligence and awareness in human healthcare environments, we needed to design various ambient communication tools it could use to proactively communicate its intentions to people, such as its moving LED arrays and head movements. Poli was a more non-anthropomorphic social robot and we decided to move specifically toward a more explicitly social face with Moxi to show the kind of socially communicative and positive contributor Moxi is on a team. There has already been an immediately positive and warm reaction people are having when meeting Moxi, which we’re thrilled to see as that type of immediate connection will be important for every clinical staff team to have when welcoming Moxi on board.
Photo: Diligent Robotics
Can you describe what we’re seeing in the video?
We created this video to introduce Moxi and show its design, personality, and basic functionalities. We wanted to use it to demonstrate how its many pieces and qualities work together to bring to life not just a single product, but an idea: An idea that robots don’t have to be scary or isolated, but they can and should be friendly and functional members of a team.
Moxi is fully autonomous, so it completes end-to-end tasks—it doesn’t need assistance at either end of its action, its arm allows it to pick something up and drop it off without help. Regarding specific movements such as picking something up from a bin, Moxi uses a variety of sensors to orient itself and execute reliable grasps of objects, regardless of their arrangement. Additionally, Moxi has the ability to verify a failed grasp and retry if needed. All of this has culminated a system that is already quite successful at manipulating a wide range of tasks, especially those healthcare professionals are responsible for.
The robot can navigate around humans that are moving. We take advantage of the navigation stack developed by Fetch, that does a great job with obstacle avoidance and path planning. On top of this we are developing the social intelligence to avoid humans differently than objects, as well as ensuring paths the robot takes are transparent and legible to the people in the environment.
What kinds of non-patient facing, logistical tasks do you think Moxi will be most successful at in the near term? What are the trickiest problems that you’re still working on solving?
With our pilot customers, we are exploring a variety of support tasks related to making sure certain supplies are in particular locations at a given time. In the near term, the robot is going to be most successful at achieving the supply setup for care tasks that might be known several hours in advance, like that an admission is happening because someone is being discharged from surgery. This is as opposed to working in an emergency department where clinical care is happening on a much more unpredictable time scale, and supplies are needed with more immediacy.
Patient care workflows are very dynamic and are constantly changing throughout the day, so one of the challenges we are still working on solving is understanding how to best insert Moxi into these workflows.
Our current platform is explicitly developed with design iteration in mind. We have a lightweight arm and gripper, with a small form factor to be able to operate in the kinds of tight spaces we will be in. In this phase of the product we are testing the set of capabilities this current prototype can perform, as well as learning about any desired capabilities that are outside of the specs for this particular arm and gripper solution. All of these learnings with customers will inform our final product offering, and next revision of the product.
Thomaz tells us that Diligent is just starting Moxi’s pilot program, with the goal of figuring out “how Moxi can best support and work with each clinical team.” It sounds like Moxi is just another step in this process, and more iterations in software, hardware, and overall design will take place before Diligent finalizes their commercial robot. And it may take a little while—the only thing harder than mobile manipulation in semi-structured environments just might be autonomous interaction with humans, and Diligent will need to conquer both of those challenges if they want to end up with a useful robot that people like to work with.