As if quantum computing wasn’t mind-bending enough, one of D-Wave Systems’ founders is now pursuing another futuristic idea: using artificial intelligence and high-tech exoskeleton suits to allow humans—and, at least according to one description of the technology, monkeys—to control and train an army of intelligent robots.
Geordie Rose is a cofounder and chief technology officer of D-Wave, the Canadian company selling machines that it claims exploit quantum mechanical effects to solve certain problems hundreds of millions times faster than traditional computers.
Now an IEEE Spectrum investigation has discovered that Rose is also CEO of Kindred Systems (aka Kindred AI), a stealthy startup he founded with others in 2014 dedicated to delivering advanced teleoperated and autonomous robots. The goal is making programming robots faster and less costly–and possibly revolutionize the world of work.
Kindred has so far received well over US $10 million in funding, according to Data Collective, the venture capital firm that led one of the rounds. Another Silicon Valley VC firm, Eleven Two Capital, also has a stake in the company. In a blog post, Data Collective described Kindred as using “AI-driven robotics so that one human worker can do the work of four.”
Kindred has kept a very low profile, issuing no press releases and maintaining only a very basic website. However, last November, Suzanne Gildert, a former D-Wave researcher who is a cofounder and CTO of Kindred, told an audience of technologists that the company was building personal robots that use machine learning to recognize patterns and make decisions. “Quantum mechanics is cool, but humanlike intelligence in robots is cooler,” she is reported as saying.
Kindred, based in Vancouver, B.C., Canada, recently filed a U.S. patent application revealing the extent of its ambitions. The document describes a system where an operator wearing a head-mounted display and an exo-suit of sensors and actuators carries out everyday tasks. Data from the suit and from other external sensors is then analyzed by computers in the cloud and used to control distant robots. The data could also be used to train machine learning algorithms that would allow the robots to imitate the operator’s actions autonomously.
“An operator may include a non-human animal such as a monkey,” says the patent, “and the operator interface may be…re-sized to account for the differences between a human operator and a monkey operator.” (This isn’t the first device to enable monkeys to direct robots, but previous research has focused on brain-machine interfaces, not robot control and autonomy.)
A rendering shows the exo-suit of sensors and actuators that an operator would wear to control a robot at a distance. Image: U.S. Patent Application US20160243701A1 via Google Patents
The application goes into some detail about the operator interface, a wearable robotic suit [image, right] that includes head and neck motion sensors, devices to capture arm movements, and haptic gloves. The operator can use foot pedals to control the robot’s movement and a virtual reality headset like the Oculus Rift to experience what the robot is seeing. The suit could even feature chemical and biometric sensors, as well as EEGs and MRI devices to capture brain waves.
The robot is envisioned as a 1.2-meter-tall humanoid, possibly covered with synthetic skin, with two (or more) arms ending in hands or grippers, and wheeled treads for locomotion. Cameras on its head would stream high-definition video to its simian operator, while other sensors might include infrared and ultraviolet imaging, GPS, touch, proximity, and strain sensors, and even a radiation detector.
This system could be used for straightforward teleoperation, where an operator would control a distant robot to complete industrial or domestic tasks. Legendary robotics lab Willow Garage trialed just such a system, called Heaphy, with some success back in 2011.
Now Kindred wants to take telerobotics to the next level. “Although a wealth of information included in human brains for performing various human executable tasks is available, robotic-related devices used to execute these tasks have historically not utilized this information or not made good use of it,” says Kindred’s patent document.
More significant, the company wants its system to be able to learn from its operators, and ultimately carry out tasks without a human—or monkey—in the loop. “Device control instructions and environment sensor information generated over…multiple runs may…be used to derive autonomous control information which may be used to facilitate autonomous behavior in an autonomous device,” says the patent application.
The document suggests that Kindred will manage this using “deep hierarchical learning algorithms” such as a conditional deep belief network (CDBN) or a conditional restricted Boltzmann machine (CRBM), a type of powerful recurrent neural network.
A diagram illustrates how multiple operators and robots could communicate through Kindred’s AI system in the cloud. Image: U.S. Patent Application US20160243701A1 via Google Patents
In fact, there are references to research on CDBN and CRBM done by Graham Taylor, who is one of the inventors named in the patent application. Taylor leads the Machine Learning Research Group at the University of Guelph, in Ontario. He studied at the University of Toronto under deep-learning pioneer Geoff Hinton, who now works part-time for Google and coinvented Boltzmann machines in 1985.
Quantum computing firm D-Wave says that the operation of its system is “analogous to a…restricted Boltzmann machine,” and that its research team is “working to exploit the parallels between these architectures to substantially accelerate learning in deep, hierarchical neural networks.” In 2010, Geordie Rose coauthored a paper that claimed a quantum computer could perform some types of machine learning applications more efficiently than software on a traditional computer. Could this be the beginning of a new field: quantum robotics?
Neither Kindred nor D-Wave replied to Spectrum’s request for comment, but according to LinkedIn and Canadian government records, Kindred has about 25 employees in Vancouver, including several who previously worked at D-Wave. It also appears to have employees in the Bay Area, including a mechatronics engineer focusing on mechanical design and electromechanical integration.
As for practical applications, the patent document mentions industrial manufacturing, domestic chores, and even entertainment. “The task may for example be making a cup of coffee, or performing a choreographed dance,” it says. “An operator…may be an actor providing a recordable set of behaviors (e.g. a set of verbal communications…to be played through a speaker on the [robot]).”
While it is unclear how far along Kindred is in actually building the telerobotics system described in its patent application, the document does include renderings of 3D models of the exo-suit, details of some components, and photos of glove assemblies and robotic tracks.
Robotics experts contacted by Spectrum said that making programming robots easier, as Kindred hopes to do, would be a huge advance for the field. But some were skeptical of the company’s ability to deliver the system as described. “The section on applying machine learning is far ahead of the state of the art,” says Tim Field, who developed the Heaphy telerobotics system at Willow Garage. “The amount of data that’s required is astronomical.” He mentions a Google Research system that required 800,000 attempts for a robot arm to learn to pick up objects from a basket. “Think about the time it would take to perform 800,000 trials using a human operator,” he explains. “It’s just not possible right now.”
“It’s not a bad idea in principle, but the trick is going to be to capture the context of the task,” says Bill Smart, a roboticist at Oregon State University. “Plus, I’m willing to bet that the human isn’t going to move the robot optimally, since it’s going to have a different set of dynamics.”
And what about the idea of using primates to teach robots how to sing and dance? “If you had an infinite number of monkeys, maybe you’d get an optimal controller?” jokes Smart. “But keeping them on task would be a nightmare.”
If you think the whole idea is bananas, you might not have to wait long to learn more. Rose and Gildert are due to speak at a Machine Learning conference in Toronto next month.
Last updated 22 September 2016 6:25 pm E.T.
Mark Harris is an investigative science and technology reporter based in Seattle, with a particular interest in robotics, transportation, green technologies, and medical devices. He’s on Twitter at @meharris and email at mark(at)meharris(dot)com. Email or DM for Signal number for sensitive/encrypted messaging.