Tech Meets Art: Capturing Life with the Eyes of a Machine

The two women make their way painstakingly around the cramped apartment, dancing in sync with each other in a sort of techno-tango. Ziv Schneider steadily passes a discontinued PrimeSense Carmine 3-D scanner over rows of picture frames, pots and pans. Its infrared sensors take in every curve and divot of the tiny studio apartment. Caitlin Robinson tails her, shotgun microphone in hand, carrying the computer that’s tethered to her partner by a USB cable. Their subject, Telma, sits frozen on her living room sofa bed, caught up in an episode of Downton Abbey.

When Schneider and Robinson finish, they’ll have a miniature, egg-like, 3-D printed replica of Telma inside her home—a single moment in the life of an East Village resident that can be held in the palm of your hand.

“When you don’t have five different rooms, then the one room you have—everything in that space—becomes critical to who you are as a person,” says Robinson. “So we wanted to capture that: people’s personalities as they are reflected on the walls.”

Schneider and Robinson set out hoping to document a time in New York City’s history in which more people are choosing to live alone. Capsule Portraits began as a documentary project at the Interactive Telecommunications Program at NYU’s Tisch School of the Arts. But the project quickly evolved when the technology itself became an integral part of the creation process.

What began as a very human-centered project spurred a fascinating dialogue between the two artists and their frequently-glitchy gear and software—a dialogue that’s influenced their art in surprising and beautiful ways.

Robinson says she’s come to think of the technology as a third collaborator on the team. “It’s kind of interesting,” she says, “to see the world not through my own eyes, but through the eyes of a machine.”

Schneider and Robinson had originally imagined that the scans would be a virtual environment, static worlds experienced with Google Cardboard or a similar VR platform. However, they quickly realized that the detailed 3-D meshes created by the scans were too data-heavy to be processed by most current mobile devices. So as an experiment, they gave 3-D printing a try, incorporating the “watertight” function of their Skanect 3-D software.

Watertight employs an algorithm that uses the internal structure of the apartment to guess at the external shape so the scan can be closed up into an object and printed. The results were odd: curvy, bumpy, organic-looking shell enclosures with strange, unintended color artifacts. Schneider admits that she didn’t like them at first, but quickly fell in love when she realized how the prints resembled a living creature—a physical representation of the symbiosis between an apartment and its occupant.

Other happy accidents continue to inspire possible future directions for their work. Schneider says the Kinect 1 scanner she uses has trouble with reflective surfaces, for instance, causing the software to make strange guesses about mirrors and what exists on the other side. The result is an Alice in Wonderland-like “world beyond the mirror.”

“It’s fascinating to see the sensor’s interpretation of what might be another reality—or another layer of reality,” says Schneider.

Rather than being frustrated by the now-discontinued hardware and problematic software, Schneider says her biggest worry is actually that its creators will make updates, “fixing” the glitches that she’s come to love so much.