Scientists have devised a virtual reality platform for lab animals. Let that sink in. Zebrafish have swum with the aliens from the video game Space Invaders, mice were afraid of virtual heights, and fruit flies circled illusory obstacles.
This new holodeck for animals can help researchers see how freely-moving animals respond to a variety of illusions—work that could help scientists better understand human genes and brain circuitry, researchers say. The researchers, from the Vienna Biocenter in Austria, detailed their findings in today’s edition of the journal Nature Methods.
FreemoVR, immersed animals in arenas where the walls or floors were computer displays. Each screen depicted photorealistic images that accounted for each animal’s perspective as it walked, flew, or swam.
Up to 10 high-speed cameras monitored the precise 3D position of each animal. FreemoVR then updated its video imagery within milliseconds of each animal's movements to create the 3D illusion that they were moving in environments that changed in response to their actions.
The researchers compared FreemoVR to the holodeck, “a fictional environment in [the TV show] Star Trek in which humans enter a computer-controlled virtual world,” says Andrew Straw, a neurobiologist at the Vienna Biocenter who was co-senior author of a paper detailing the study. “They can freely move, have no need to wear special clothing or headgear, and are immersed in a computer-controlled environment, which can be made completely realistic or arbitrarily unrealistic.”
The researchers tested FreemoVR on mice, fruit flies, and zebrafish, three species commonly used in lab research. The virtual landscape with which these animals interacted included vertical pillars, floating rings, checkerboard floors, virtual plants, and a swarm of digital aliens from Space Invaders. They even had distinctive “portals” that could instantly alter the virtual environments to make it seem as if zebrafish swimming into them had “teleported” elsewhere.
The animals apparently found the illusions realistic. For instance, fruit flies circled virtual pillars just as they did real ones placed in the platforms. Moreover, mice generally avoided tracks that looked as if they were suspended at great heights, just as they would in real life.
The animals also changed their behavior in response to illusory animals. For example, zebrafish normally circled the periphery of their fishbowl near the screens, but when teleported into settings with swarms of Space Invaders, the zebrafish tended to move toward the middle of the fishbowl.
“We wanted to study collective behavior because that is something incredibly difficult to do with real animals or with robots,” Straw says. "We wanted to show how real fish respond to the motion of a swarm of simulated agents and to show that we could create a hybrid biological-computational swarm.”
In addition, the researchers developed a photorealistic model of a swimming fish, and showed that real zebrafish most reliably followed the digital fish when the virtual animal matched its swim direction to the real fish. The fact that researchers can vary the appearance of virtual animals from cartoonish to realistic “will allow experiments to test how important the exact visual appearance of other animals is as opposed to, say, the pattern of motion,” Straw says.
This new platform will let scientists investigate animals as they behave relatively naturally and unrestrained by conventional VR gear in realistic virtual environments they can manipulate extensively. By tinkering with animal DNA or brains in such experiments, the researchers can learn what role certain genes or brain circuits play in these animals, and potentially in humans as well. “Brains evolved in the real world, and to understand how and why neural circuits process information in the way they do, we need to understand them in this context,” Straw says.
Straw notes that humans would notice several imperfections with FreemoVR. “Primary amongst those is that our system does not create two distinct views for the two eyes, and thus the stereo cues important for depth perception would be gone,” he says. However, Straw notes this is not a major concern with the animals they are experimenting with; the eyes of these animals are so close together that the differences between the view from each eye are limited.
Straw’s lab is now conducting experiments where they can silence the activity of single brain cells in fruit flies and examining the roles these cells play in the insects’ behavior in virtual erality.
Charles Q. Choi is a science reporter who contributes regularly to IEEE Spectrum. He has written for Scientific American, The New York Times, Wired, and Science, among others.