It takes a lot of practice to fly a drone with confidence. Whether it’s a multirotor or a fixed-wing drone, there are a lot of complicated things going on all at once, and most of the control systems are not even a little bit intuitive. The first-person viewpoint afforded by drone-mounted cameras and VR headsets helps, but you’re still stuck with trying to use a couple of movable sticks to manage a flying robot, which takes both experience and concentration.
EPFL has developed a much better system for drone control, taking away the sticks and replacing them with intuitive and comfortable movements of your entire body. It’s an upper-body soft exoskeleton called FlyJacket, and with it on, you can pilot a fixed-wing drone by embodying the drone—put your arms out like wings, and pitching or rolling your body will cause the drone to pitch or roll, all while you experience it directly in immersive virtual reality.
We’ve seen a few projects that explore the same basic idea as FlyJacket; the most notable is probably Birdly, which is a sort of platform that you lie down on while wearing a VR headset, flapping your arms like a bird as a fan blows air in your face. It’ll make more sense if you watch the video, and people seem to like it, but it’s not exactly portable (or cheap) and the amount of control that you get is fine for VR but not ideal for flying a drone in real life.
FlyJacket, on the other hand, was designed to be portable and affordable. Developed at EPFL’s Laboratory of Intelligent Systems, led by Professor Dario Floreano, the exosuit doesn’t require much in the way of training since it’s inherently so intuitive. It may look a little bit awkward, but those arm supports (which are removable) make it comfortable for long-term use by offloading the weight of your outstretched arms down to your hips. Sensors in the suit detect body motion, and translate torso pitch (bending forwards and backwards) into drone pitch, and torso roll (bending sideways) into drone roll. It’s worth noting that neither of these motions require your arms to be outstretched, or really for your arms to move at all, but it’s instinctive for people to hold their arms out when they’re pretending to fly, and they’re more comfortable with their level of control when they do so.[shortcode ieee-pullquote quote=""The experience starts when you put the FlyJacket on as it gives the feeling to wear a superhero suit. As the drone control is very intuitive, user is directly immersed into the flight and can directly start to explore the environment. You almost instantly become embodied to the drone"" float="right" expand=1]
Besides being an easy and intuitive method of control, the FlyJacket offers a few other benefits as well. The researchers found that people can fly more consistently with the exosuit than they can with a traditional controller, perhaps because you end up feeling the motion of the drone with your body, providing feedback that you don’t get with a controller. People also reported feeling less VR-induced motion sickness, which may be due to less of a disconnect between the motion that your eyes see and the motion that the rest of your body (and especially your inner ear) feels.
To get more detail on FlyJacket, including what it’s like to use and whether there might be a commercial product in the works, we spoke via email with Carine Rognon, one of the members of EPFL team, which just published a paper on the project in IEEE Robotics and Automation Letters.
IEEE Spectrum: What made you decide to develop the FlyJacket?
Carine Rognon: Drones are becoming more and more pervasive. They can extend human perception to 3D space, and allow us to increase our range of action in an unprecedented way. However, drones need training and dexterity to be well controlled, which limits their use to highly trained users. Therefore, we wanted to make the interaction between drone and human more natural and intuitive in order to create a symbiosis between them. The idea was to go further than just improving the interaction by achieving an embodiment of the human in the drone. To do so, we wanted a device that is directly mounted on the skin and close to an object of the everyday life.
Can you describe what it feels like to fly a drone through the FlyJacket?
The experience starts when you put the FlyJacket on as it gives the feeling to wear a superhero suit. As the drone control is very intuitive, user is directly immersed into the flight and can directly start to explore the environment. You almost instantly become embodied to the drone.
What is it about a “flying” position that makes it easier or more intuitive to pilot a drone?
The movement to control the drone have been derived from a previous study done by our research group, which analyzed which body movement people would intuitively do if they were a drone. Therefore, the movements to control the drone using the FlyJacket are intuitive. While using a joystick needs training, the FlyJacket allows most of the people to control the drone within few tens of seconds. Indeed, the performance variance across participants was significantly lower when using the FlyJacket than when piloting the drone with a joystick. In addition, as people use their full upper body and not only their fingertips, they have kinesthetic and vestibular feedback due to their change of upper body position. Also, the immersion is deepened by the fact that the interaction with the drone is direct as it doesn’t pass through a physical tool such as when using a joystick.
Is there any commercial potential for FlyJacket?
Yes, there is definitely good commercial potential. The design of the jacket was focused on keeping the material and technologies at low price to have an affordable product. In addition, it is small enough to fit into a backpack in order to be taken in the field and adaptable to many morphologies so many body types can use the same jacket. We already have replicated multiple versions of the jacket to be used by other research projects.
What are you working on next?
We are working on using more body degrees of freedom to add more commands to the drone; among other things to control the speed of the drone. Currently, only the torso is used to pilot a fixed wing drone. Therefore, the degrees of freedom of the arms are available to implement new commands. We are currently investigating an intuitive way to control the throttle for a fixed-wing drone, which would allow control of the flight speed. Controlling yaw should also be studied for controlling a quadcopter. Yet, the aim is to keep the control of the drone as natural as possible and therefore should remain simple so the user can focus on other tasks such as inspection or giving vocal commands to other team member.
We are also working on adding haptic feedback in order to have a bidirectional interaction between the drone and the human. In our last study, we have added haptic guidance by the use of cables in order to improve the flight performance. We are currently also investigating if this haptic guidance accelerates learning. And we are working on rendering the sensation of flying through tactile feedback on the torso with a collaboration with the Collaborative Haptics and Robotics in Medicine (CHARM) Lab at Stanford University.
Professor Floreano tells us that FlyJacket is one of the first tangible results from a much larger interdisciplinary project called The Symbiotic Drone, which is “aimed at developing technologies that could create a symbiosis between a human and a non-anthropomorphic robot,” he says. “I conceived this project during a sabbatical leave a few years ago while reading the literature on dreaming of flight, and I became interested into the capability of humans to morph into a different organism at a different scale in a different space and with different sensory-motor capabilities.”
This is an extraordinarily compelling idea that goes well beyond flight. You can imagine, for example, how an exosuit that allows the user to run and jump could be applied to control a quadrupedal robot, with more speed and strength than any human could experience on their own. Or, maybe you can’t imagine it—I’m not entirely sure I can fundamentally understand what that would be like without actually trying it. We could one day find ourselves with the ability to inhabit robots of a multitude of different sizes and shapes, but with intuitive embodied control that allows us to feel like we are them, rather than just controlling them. It’s exciting to think about, and while we’re not there yet, FlyJacket has definitely gotten the concept off the ground.