Social robots have had it tough recently. There are lots of reasons for this, but a big part of it is that it’s a challenge to develop a social robot that’s able to spark long-term user interest without driving initial expectations impractically high. This isn’t just the case for commercial robots—social robots designed for long-term user interaction studies have the same sorts of issues. The Honda Research Institute is well aware of how tricky this is, and researchers there have been working on the design of a prototype social robot that achieves a “balance between human expectation, surface appearance, physical affordance, and robot functionality.” It’s called Haru, and Honda Research has provided a fascinating and detailed look into how they came up with its design.
A paper on Haru was presented at the ACM/IEEE International Conference on Human Robot Interaction (HRI) earlier this year, and the introduction does a lovely job of presenting why it’s so important to carefully consider the physical design of social robots intended to interact with humans:
Various studies confirm that human expectations are shaped by the physical attributes of a robot. As a consequence, human expectations can set the bar high depending on the promise it holds as a function of its physical appearance and how this measures up with the robot’s actual affordances. For example, a six-foot-tall humanoid robot with a futuristic look would turn out be a disappointment if it only performed Q&A tasks and nothing more. This indifference does not impact on the smaller and basic-shaped smart devices, as the simple Q&A task completion of current smart devices is proportional to the simple image they project. The physical and aesthetic elements of a robot require considered design as they affect its prospect of acceptance and long-term adoption. It is essential to foresee in advance the implicit illusionary functionality brought upon by the design of the robot’s physical affordance, and to strike a balance between this and human expectation. Keeping human expectation low while stoking interest at the same time may prove to be a good strategy.
It’s possible that this is not just a good strategy, but the best strategy (or in fact the only strategy). We’re tempted to ascribe all kinds of things to robots that look even vaguely human, and that’s been one of the issues that social robots have had in the past—enough human-ness that users think they’re more competent than they are. Commercial social robots are very much aware of this tendency, which is why they often go for a minimalist approach, the thinking being that if it looks less human, the expectations will be less as well. As the same time, though, the goal is to maximize emotional engagement to encourage users to become invested in interacting with the robot long-term.
The approach that Honda Research took with Haru was to tackle the robot design holistically, by “assembling interdisciplinary teams that include professionals from outside the field of robotics from the early stages of the design process.” Their “Design Thinking” methodology emphasized continual iteration, beginning with concepts that weren’t necessarily grounded in engineering reality. Instead, they started with inspiration from animated characters:
Top row: Artists’ sketches of animation characters from Pixar and Disney. Middle row: Artists’ representative sketches of the different tabletop robot candidates during brainstorming. Bottom row: Artists’ sketches of the actual tabletop robot (Haru) after the deliberation process.Image: Honda Research Institute
To further refine Haru's empathetic behavior (empathy failure has, the paper says, been identified as the “main downfall when it comes to maintaining the illusion of life in robotic behavior”), the researchers had 50 real live humans come in and act out different emotional states through both facial expressions and body language. Professional artists sketched the volunteers as they posed, and then adapted those poses in conceptual sketches of Haru:
Top: Acted-out emotive actions from performers. Bottom: Artists’ sketches of robot emotion synthesized from performers’ acted-out emotive actions.Image: Honda Research Institute
Haru’s physical design, meanwhile, needed a compromise between having the ability to display human emotions and staying away from recognizable forms in order to keep expectations reasonable, the researchers explain:
In designing Haru’s physical appearance we aimed to step away from a literal humanoid or animal form, with its accompanying expectations, by experimenting with various form factors and shapes. Haru is an experimental tabletop robot designed as a research platform to test the limitations and potentials of a range of communication modalities within a framework of intentionally designed physical constraints. These constraints aim to strike a balance between keeping human expectations grounded while stoking human interest and supporting long-term interaction between personal robots and their users.
The current Haru prototype has five degrees of freedom: eye tilt, eye roll, eye “pop” (the eyes move in and out in their frames by 15 millimeters), body lean forward and backward, and base rotation. These all combine with animated eyes and an LED array in the base to give Haru an impressive amount of emotional expression from an only vaguely humanoid form:
Haru’s actual hardware “emotive affordance.”Image: Honda Research Institute
Apparently, 20 percent of the entire design time was devoted to figuring out exactly how big to make Haru—a series of prototypes were constructed, then narrowed down to five discrete sizes, ranging from 40 percent to 160 percent of the final size. Team members then voted on which one they liked the most, and the current size (22-centimeter-diameter base and about 15 cm tall) got nearly half the votes.
In initial testing, people had no trouble identifying Haru’s simple emotional displays, like happy, sad, and angry. Haru can do more complex emotions, but the researchers haven’t yet done a study to see how effective it is. It sounds like there were a bunch of other design considerations that went into Haru as well; this particular paper only focuses on the visual, physical elements.
Haru side and front views.Image: Honda Research Institute
Hopefully, we’ll be hearing more about this robot, but in the meantime, first author Randy Gomez from Honda Research Institute in Japan was kind enough to answer some of our questions via email.
IEEE Spectrum: How was the process of designing the Haru prototype different from how most robots are designed, and what human-robot interaction challenges were you trying to address with this approach?
Randy Gomez: Our goals were to design a personal robot that people would really love. Something that people really felt a connection with. This meant that in taking on the challenge of designing Haru aesthetics and communication were very important in the design process. The Haru prototype is different in a sense that “design thinking” approach was the centerpiece of designing Haru. Designers, animators, sketch artists, engineers, and ordinary people were involved right from the very beginning of the design process, including a battery of tests until the first prototype came into fruition.
What do you think is the best way for robot designers to balance interactive potential with user expectations?[shortcode ieee-pullquote quote=""The design has a lot to do with balancing user expectations. Whilst a robot's physical attributes give it an advantage over virtual assistants, you have to be mindful, as too many bells and whistles or too humanoid a form can set the bar for human expectations too high"" float="right" expand=1]
The design has a lot to do with balancing user expectations. Whilst a robot’s physical attributes give it an advantage over virtual assistants, you have to be mindful as too many bells and whistles, or too humanoid a form, can set the bar for human expectations too high. It is important to foresee in advance the implicit illusionary functionality brought upon by the design of the robot’s physical affordance, and to strike a balance between human expectation as a function of physical affordance and the robot’s functionality.We have been very careful in the consideration of the balance between Haru’s physical design and Haru’s abilities. We wanted to keep the design simple but appealing, which is actually very hard. So we conducted a number of tests to see which parts of the body were used for communication and this influenced our end design.
To what extent were you able to effectively translate emotive actions from humans and animated characters to emotive actions on Haru?
We are still in the prototyping stage and it has been fantastic working with animators from such an early stage. They have been using all their skills and knowledge to do with design, emotion and performance, to design eyes, nonverbal acoustics and gestural emotions. In terms of gestures and movement, the animators had a lot of fun, working on a scale of three, seven and ten out of ten on an emotional scale. They have been using all their observation of human behaviour and their skills as animators, however they are evolving this proves to be relevant for robotics. So there is a lot of intersect and overlap but it isn’t as straightforward as a direct translation. Some of the emotive actions they designed and some of their requests test the limits of our engineering, however that is a good thing as it challenges us to find solutions. The relationship is two way as our requests also challenge the design and animation team to find solutions. What we have seen so far is looking good. This is the exciting area of research where we are collaborating in new territory.
Can you elaborate on how you designed Haru’s eyes to serve a similar communicative purpose to arms and hands?
As I mentioned, we conducted a number of tests to understand the distribution of communicative expressiveness that we as humans attribute to areas of the body. We categorized movements and expressiveness into a distribution of activity percentages in three major parts of the body—upper, middle, and lower. Then we looked at the distribution of movement prominence in terms of body movements and emotive affordances. What we found was that it was the upper and middle body that had more impact in these areas, particularly the head, neck, face and eyes, although there was a very significant emotional communication related to hands. However, if we had designed Haru with hands, then people would have expected a robot that had the dexterity that hands and arms provide. As we were looking at the emotional rather than physical affordances, we resolved the issue by eschewing the popular unibody design of many contemporary robots, and separated the eyes. This immediately allowed us to work with a backwards and forward tilt and pivot capacity, which meant that the eyes had similar emotional affordances to hands and arms. They could be used to indicate direction and emphasize particular emotional gestures.
It sounds like some of the design concepts for Haru were impractical to engineer. Without engineering constraints, how would Haru be different? What specific compromises did you have to make?
If there were no engineering restraints we could have lifelike movement and movement similar to what we see in the very best animated movies. However, it has been a very good process working within Haru’s limitations. An interesting outcome from this research has been the back and forth and the challenges that we have all set ourselves. For instance the animators could model and rig a digital prototype of Haru that was governed by the same DoFs [degrees of freedom]. This meant we could test the potential of the engineered DoFs for emotional affordance through a digital avatar. It also meant that the animators could challenge us, which they did. They came back with a wish list of DoFs that asked for 10 degrees here, 10 degrees there and various other movements that required reengineering. However when they asked for these they would physically model and demonstrate what they were asking for, which made communication much easier.
We in turn would ask things of the animators that challenged them. So the whole process has been, and continues to be one of setting a high bar and trying to meet it. Even if we can’t meet each other’s requests one hundred percent, we are all aiming higher and achieving higher. You have to work outside what is comfortable or known to design for the future. In general, a compromise on DoF, speed, and smoothness in exchange for aesthetics have to be made.
What are you working on next?
Haru is human-centered in both design and application. In the short term, we will be working with researchers (partners) to fine tune Haru’s design and to find useful applications through HRI [human-robot interaction] experiments. The results will be used to update Haru’s hardware vis-à-vis Haru’s skills set. In the midterm, we will be building together with our partners the tools that empower Haru, making use of the advancement of AI. We are looking at the prospect of the adoption of Haru as a platform in social robot experiments and applications. In the long term, we wish to see Haru or Haru’s successors inspired by this project as assistive tools that improve the quality of life for various sectors of the community.