How Aldebaran Robotics Built Its Friendly Humanoid Robot, Pepper

The French company worked in secret for two years to create Pepper. Now Japanese telecom giant SoftBank is ready to sell it to consumers

8 min read
Designed to be a companion in the home, Pepper is the world’s first full-scale humanoid to be offered to consumers.
Photo: Antoine Doyen

The robot seems determined to put a bigger smile on the man’s face. “Are you smiling from the bottom of your heart?” it asks. The man chuckles. “That’s what I’m talking about,” the robot quips in a high-pitched voice. Then, just for good measure, it bows its plastic head and apologizes for being “too bossy to our CEO.”

The CEO is Masayoshi Son, founder and chairman of telecom giant SoftBank and Japan’s richest person. As such, he has overseen the development of hundreds of new products as part of a vast conglomerate of mobile-phone carriers, Internet ventures, and media companies. But last June, at a press conference outside Tokyo, Son climbed onstage to unveil a pet project: a humanoid robot named Pepper. Designed to be a companion in the home, it is the world’s first full-scale humanoid to be offered to consumers. In February, SoftBank plans to start selling it in Japan for 198,000 yen (less than US $2,000), plus a monthly subscription fee. Taiwanese electronics manufacturer Foxconn, known for building iPhones and iPads for Apple, will produce the robots.

Video: Erico Guizzo

For that kind of money, don’t expect anything like Rosie, the robot maid from “The Jetsons.” What you’ll get is a two-armed, 1.2-meter-tall robot that rolls around on a wheeled base. It can dance and gesture with some grace, but its manipulation skills are limited, and it’s unclear how much autonomy the robot has; at the launch event, most of its actions were clearly preprogrammed. At home, Pepper won’t be able to fold your laundry or clear the dinner table.

And that’s fine with SoftBank, which says Pepper is not a utilitarian automaton. It is designed to provide advice and company: It’ll tell you jokes, play games with you, teach you a new subject, and help you communicate with family and friends. Pepper will read a recipe aloud while you do the cooking. Onstage with Son, it spoke Japanese, but it is also fluent in English, French, and Spanish.

3 robots cost Data sources: iRobot; SoftBank; Willow Garage

To do all that, the robot is equipped with an “emotion engine”—software that attempts to infer how a user is feeling based on facial expressions, tone of voice, and speech, allowing the robot to respond accordingly. If you arrive home and look a bit down, Pepper will play your favorite song, for instance. “We want to have a robot that maximizes joy and minimizes sadness,” Son said.

SoftBank is betting that people are ready for that kind of experience. Humanoids have long captured our imaginations, but until now they’ve been notably absent from our homes, where the only robotic inhabitants you’ll find today are small mechatronic toys and Roomba vacuum cleaners. It’s still too early to tell whether Pepper will be a hit, but its arrival may be a sign that robotics technology is beginning to catch up with science fiction.

It seems natural that Japan, a nation known for its love of everything robotic, would be the first market for a home humanoid. But it may come as a surprise that SoftBank, as it sought partners to develop Pepper, didn’t join forces with Honda, Sony, Toyota, or any other big Japanese company with robotics expertise. Instead, it turned to a much smaller and lesser-known French robotics firm called Aldebaran.

Tucked in a narrow tree-lined street on the southwest edge of Paris, Aldebaran’s headquarters occupies several floors of a modern office building. Robots and humans mingle casually as if in a scene from Star Wars. A Pepper robot greets employees near the coffee machine. Another—wearing a blond wig that someone thought would make for a stylish upgrade—watches people going by in a hallway. In a glass-walled room, a dozen robots speak and roam aimlessly, testing their own endurance.

I approach a Pepper and strike up a conversation, but the robot doesn’t seem to get what I’m saying. An employee comes over and tells the robot to speak English. Voilà! Now Pepper and I can understand each other—sort of. When I ask the robot what it is capable of doing, it responds by describing a game it wants to play. The employee shows me how to improve my human-robot interaction skills: I have to look at Pepper’s face and speak more clearly. I ask again what it can do. “I can do lots of things,” this Pepper tells me, “because the engineers who programmed me are very smart.”

AntoineDoyen Bruno Maisonnier, founder and CEO of Aldebaran Robotics, wants to bring friendly companion robots into people’s homes. Photo: Antoine Doyen

SoftBank seems to agree. Four years ago, Masayoshi Son decided he wanted robotics to be part of his vast business empire, and he sent emissaries to evaluate the world’s top robotics companies. Aldebaran, despite being relatively small, stood out in its ability to design robots that offer a highly interactive experience. Aldebaran’s flagship robot is a knee-high humanoid called Nao. More than 6,000 Naos are now used in research labs, schools, and hospitals in 70 countries. Longer term, the company is also developing a 1.4-meter-tall legged humanoid called Romeo.

“The most important role of robots will be as kind and emotional companions to enhance our daily lives, to bring happiness, to surprise us, to help people grow,” says Aldebaran founder and CEO Bruno Maisonnier, an executive who quit a career in finance to pursue his dream of creating robots for everyone.

Son and Maisonnier met and realized they shared the vision of taking robots out of the lab and into everyday life. Together, they could do something for robotics “that could be really world changing,” Maisonnier recalls thinking. In early 2012, SoftBank acquired a majority stake in Aldebaran and agreed to fund the company’s growth. For Aldebaran, the acquisition meant that nearly overnight, the company had to focus almost exclusively on the Pepper project.

graphic link to Fast Forward sidebar

SoftBank gave Aldebaran just three months to build the first prototype, and after that it demanded to see a demonstration every two or three months. Aldebaran’s strategy was to build on its experience with Nao and develop a “stretched up” version of the smaller humanoid. As an example, Rodolphe Gelin, Aldebaran’s research director, says the engineers adapted the joint mechanisms in Nao’s arms for use in Pepper. “Our robots share many things in common, and that allowed us to do things in a time frame that otherwise would be impossible,” he says.

EricoGuizzoPeppers In this room at Aldebaran’s Paris headquarters, Pepper heads, torsos, and arms endure a battery of mechanical and thermal tests. Photo: Erico Guizzo

A team of designers and artists created a sleek, friendly looking shell for the robot. And the engineering team stuffed its body with 20 electric motors, an Intel Atom–based computer, two cameras, a 3-D sensor, four microphones, and a lithium-ion battery that lets Pepper run for 12 hours. A tablet on the robot’s chest displays information and provides another way to interact with the robot.

Gwennael Gate, one of the software directors, says that a big challenge was dealing with the robot’s huge computing needs while “making sure that the CPU is not exploding.” Each function is controlled by one of about 20 software engines. If you’re standing far from the robot, for instance, its awareness engine makes the robot move its head and emit sounds to try to get your attention. If you come closer, a dialogue engine kicks in, so you and Pepper can have a conversation. If you ask the robot to dance, a motion engine takes over.

In Aldebaran’s first trip to SoftBank’s Tokyo headquarters, in April 2012, the engineers gathered in a room filled with Japanese executives. Suddenly, the doors opened and everyone went silent. Masayoshi Son entered, took a seat, and stared at the Pepper prototype in front of him. The first demo was simple enough: The engineers turned the robot on, and it did a little dance. Almost immediately, Son was “like a kid,” beaming at the robot, an Aldebaran engineer told me.

Video: Erico Guizzo

What followed was an intense two years for the French company. Engineers worked day and night, with no breaks for weekends or holidays. “Aldebaran never slept during this period,” Gate said. The head count ballooned to 500 people at one point, with offices expanding not only at the Paris headquarters but also in Tokyo, Boston, and Shanghai. The work culminated in the demo of all demos: the highly produced, Apple-esque launch, where Son would introduce Pepper to the world. Aldebaran and SoftBank rehearsed the event several times a day for an entire month, using stand-ins for Son and Maisonnier.

The emotion engine, which Son highlighted in the event, uses the robot’s vision system to detect smiles, frowns, and surprise, and it uses speech recognition to sense the tone of voice and to detect certain words indicative of strong feelings, like “love” and “hate.” The engine then computes a numeric score that quantifies the person’s overall emotion as positive or negative.

Aldebaran admits the system is not very sophisticated, but the company promises that it will improve. In the future, the system could also incorporate ethics, empathy, and other qualities and behaviors that the company believes robots need in order to be part of people’s lives. The emotion engine is something Maisonnier wants “embedded at the core of our humanoid operating system, because it defines who our robots are and how they behave.”

Not everyone is convinced that people will want a Pepper at home—at least not until the robot can do some actual chores. The robot will have “a very difficult time getting off the ground as a viable consumer product,” a robotics observer told PCWorld. Others have accused SoftBank of hyping Pepper’s capabilities. The technology website The Verge found the robot’s emotion-recognition skills disappointing, saying that “Pepper has a heart of COBOL.”

SoftBank counters that it is pricing the robot very aggressively, which should help drive demand. Indeed, Son says he’s willing to lose money selling the robots until the company can ramp up volume and reduce costs—a strategy he’s used successfully in the mobile industry. Still, it’s unclear whether consumers will be convinced of Pepper’s usefulness, especially outside Japan. SoftBank has yet to articulate its plans for international sales.

Maisonnier says Pepper will become more capable over time, as developers create new applications for it. Users will then be able to download and install these new functionalities, just as they add new apps to their smartphones. “The most important thing is to have a huge community of people trying, experimenting,” he says. “This community will create the applications that will make the next wave of people want to have the robot.”

Last September, SoftBank and Aldebaran held a developers conference in Tokyo, where they revealed details about Pepper’s technology and a set of software-development tools. A thousand attendees showed up, 600 of whom preordered a robot.

And while Aldebaran will continue working to make its existing robots smarter and more capable, it doesn’t plan to stop there. With the experience it gained developing Pepper, the company wants to build new robots for other customers—a strategy that SoftBank supports. Aldebaran doesn’t yet know what these Pepper cousins will look like, but it’s exploring ideas with banks, insurance companies, and retail stores.

Maisonnier believes Pepper’s debut is the beginning of a revolution whose effects will eventually be of the same magnitude as those of the PC, the Internet, and mobile phones. “People want robots, and they’re frustrated because there are no robots,” he says. “We’re going to give people the robots they’ve been waiting for.”

Fast Forward

When Will We Have Robot Servants?

Not anytime soon, but personal robots will invade our homes

robot Servents Illustration: MCKIBILLO

By 2020: Pepper appears to be the first of a new breed of personal robots designed to be helpers and companions in the home. In the next five years, several others will become available. One of them, set to ship later this year, is Jibo, a chatty, coffee-machine-size robot created by Cynthia Breazeal, a pioneer of social robotics at MIT. Another robot that will likely hit the market before 2020 is a larger, mobile machine developed by Hoaloha Robotics, a Seattle start-up that targets the health-care market. What we believe Pepper, Jibo, and the Hoaloha robot will have in common is that they’ll be interactive, expressive robots that communicate with users through voice and touch screens; in addition to their built-in functionalities, they’ll also run third-party applications. So with this first generation of home robots, we should see the emergence of a community of developers focused on robot applications. These robot apps will revolve around entertainment, communication, and education, so don’t expect the robots to take over your domestic chores just yet.

By 2025: Within the next 10 years, more personal robots—with different sizes, capabilities, and prices—will arrive in our homes, offered by established robot makers, big electronics companies, and new robotics start-ups. More robo-companions will mean greater demand for robot applications, and a growing population of developers will strive to build the most compelling—and profitable—apps. But the biggest shift will be in the robots’ vision and manipulation skills, which will be vastly improved and allow our robotic cohabitants to begin to perform some useful tasks for us. They still won’t be able to cook or clean as well as a human, but they might help you chop some veggies or pick up toys from the floor. This generation of robot helpers will still be slow and imperfect, but they’ll be smarter than their predecessors. At least when they break, they’ll be able to call customer service themselves. —E.G.

This article originally appeared in print as “A Robot in the Family.”

The Conversation (0)

IoT-ize Your Old Gadgets With a Mechanical Hijacking Device

This tiny connected manipulator can, its creators say, make any device smart

3 min read

An IoTIZER installed on dimmer switch

KAIST/Korea Polytechnic University
IoT

Just about every device you own is probably available with Internet connectivity. Whether or not every device you own actually needs Internet connectivity is up for debate. But if you want it, it's there—as long as you're able to afford it, of course. Connectivity usually comes at a premium, and it also usually involves buying a brand new whatever it is, because new hardware and software and services are required.

If connectivity is important to you, there aren't many options for older devices. It's not hard to turn them on and off with a connected socket adapter of some sort, but as you start to go back more than a few years, things become increasingly designed for direct human interaction rather than for the Internet, with buttons and switches and dials and whatnot.

IoTIZER is a prototype of a mechanical hijacking device (MHD), designed to replace human manipulation of existing products. As the name suggests, it can IoT-ize just about anything designed to be operated by a human, potentially giving a new connected life to your stuff.

Keep Reading ↓ Show less

Video Friday: Android Printing

Your weekly selection of awesome robot videos

4 min read

Your weekly selection of awesome robot videos

Video Friday is your weekly selection of awesome robotics videos, collected by your Automaton bloggers. We'll also be posting a weekly calendar of upcoming robotics events for the next few months; here's what we have so far ( send us your events!):

RO-MAN 2021 – August 8-12, 2021 – [Online Event]

DARPA SubT Finals – September 21-23, 2021 – Louisville, KY, USA

WeRobot 2021 – September 23-25, 2021 – Coral Gables, FL, USA

IROS 2021 – September 27-1, 2021 – [Online Event]

ROSCon 2021 – October 21-23, 2021 – New Orleans, LA, USA

Let us know if you have suggestions for next week, and enjoy today's videos.

Keep Reading ↓ Show less

New AI-Based Augmented Innovation Tool Promises to Transform Engineer Problem Solving

AI-enabled software suite helps automate innovation process – from ideation to disclosure submission.

5 min read

You're an engineer trying to work out a solution to a complicated problem. You have been at this problem for the last three days. You've been leveraging your expertise in innovative methods and other disciplined processes, but you still haven't gotten to where you need to be.

Imagine if you could forego the last thirty hours of work, and instead you could have reached a novel solution in just 30 minutes. In addition to having saved yourself nearly a week of time, you would have not only arrived at a solution to your vexing engineering issue, but you also would have prepared all the necessary documentation to apply for intellectual property (IP) protection for it.

Keep Reading ↓ Show less

Trending Stories

The most-read stories on IEEE Spectrum right now