3D Haptic Display Melds Touch With Tech

What started out as trying to create soft, simulated organs for medical devices and surgical robots, has given us, instead, a touch-sensitive, shape-morphing 3D display. This multifunctional device, developed by researchers at the University of Colorado Boulder and Max Planck Institute for Intelligent Systems, is about as big as a board game, and can create pop-up patterns, manipulate objects across its surface, and shake a beaker of liquid.

“The whole concept of creating the 3D display was…to try to replicate the human body, not biologically, but from a sense and response standpoint,” says Mark Rentschler, a roboticist at CU Boulder. Which meant designing a system with soft actuators and sensors to replicate muscles and nerves within the body, and a support structure to represent the skeleton.

“This type of sensing could create some very interesting surgical simulations for either training medical students or developing medical devices in robotics.”
—Mark Rentschler, University of Colorado Boulder

The result is a display surface comprising a 10-by-10 grid of individual cellular units, with high-speed actuation, sensing, and control. Each single cell is about 6 centimeters by 6 centimeters, and 1.4 cm high, packed with soft actuators and sensors, and supporting electronics. The system is connected to a small PC for computation. A paper about this work was published in Nature Communications in July.

The soft “muscles” of the device come from the earlier work of graduate student Ellen Rumley, who designed the Hydraulically Amplified Self-healing Electrostatic (HASEL) actuators. “These actuators use simple polymer sheets to hold oil inside of them,” Rentschler says. “By passing a current through the components, you can get the actuators to zip close.”

Each cell of the touch-sensitive surface contains a stack of HASEL actuators. Rentschler’s team used a modified folded design of the actuator. The compression of the individual oil chambers causes the entire stack to increase or decrease in size. Apart from being soft, the HASELs have a fast-response rate (of 50 hertz) and morph well enough to move solids and liquids across the entire display surface. It is sensitive to about 5 grams of mass and to deformations as small as 0.1 millimeter. In other words, a very small amount of force on the surface can be detected, says Rentschler.

To create sensing and response, both to external stimuli, and to provide closed-loop actuator control, the researchers used soft magnetic sensors made of silicone. Rentschler says the group placed the sensors directly on the surface layer, giving the system the ability to detect both surface deformation and external stimuli. This also gave the display a small footprint, and allowed it to perform various sequences of actuations, both with a user, and with other objects.

Shape-shifting display for 3D designsyoutu.be

While shape-morphing displays aren’t exactly new, this system is unique in being smaller, faster, quieter, and softer. Its computational and power requirements are low. Plus, it is a continuous surface, not discrete points, Rentschler says, “And that allows us to do a couple of unique things with it.” He is optimistic in making the system even more compact in the future. “It’s really just reducing the actuator size, and as the electronics continue to evolve, having those shrink down as well.”

The device’s versatility also opens up application possibilities—from a consumer electronics interface, to various manufacturing or commercial applications. For example, processes that involve, say, handling toxic or delicate materials. He also sees possible applications in the gaming industry, providing tactile feedback in AR/VR environments.

Then, of course, there are medical applications. “This type of sensing could create some very interesting surgical simulations for either training medical students or developing medical devices in robotics,” Rentschler says. It could, for instance, simulate using such shape-morphing devices in a body that’s not human or an animal model, as a precursor to getting clearance for use on humans.

As for creating simulated organs, where the project began, “we’ve got a few different ideas for that,” he says. He says his lab is now looking at creating a simulated portion of the gastrointestinal tract, maybe the colon, to test surgical robotics.