Piz Daint Supercomputer Shows the Way Ahead on Efficiency

Scan the latest list of the 10 most powerful supercomputers and the list of the top 10 most energy efficient and you’ll see that only one name appears on both. The Piz Daint supercomputer is both the sixth most powerful machine in the world and the fourth most energy efficient. It’s also one of only two supercomputers in the top 10 of the efficiency ranking (the Green500) that’s capable of maintaining petaflops performance—a million billion floating-point operations per second. How it managed to make its mark in both categories could point the way to both more powerful and more efficient future machines.

“What I’m seeing is that energy efficiency is increasingly becoming a first-order design constraint that is on par with performance,” says Wu Feng, one of the Green500’s leaders and a professor of computer engineering at Virginia Tech. “It’s not going to be any good to be high performance but so power hungry that you don’t even have the budget to power or cool the system.”

Located at the Swiss National Supercomputing Centre (CSCS), in Lugano, Piz Daint is being used to parse huge data sets and simulate processes for projects in geophysics, materials science, chemistry, and other areas, but especially in climate modeling. To power that research, the computer uses a hybrid system that combines the advanced network architecture of the Cray XC30 with two cutting-edge processors, the Intel Xeon E5 CPU and the Nvidia Tesla K20X GPU. The computer’s 5272 compute nodes are bound together in a special low-latency network to form an unusually compact machine. Thomas Schulthess, the director of CSCS, says that Piz Daint uses 28 cabinets to achieve what would take 50 cabinets of the second most powerful computer in the world, Titan, which Schulthess worked on at Oak Ridge National Laboratory, in Tennessee.

But the supercomputer’s real secret is that whenever possible its software keeps data from having to travel between processors. “What we really wanted on Piz Daint was to motivate users to consider how they organize their data in applications,” says Schulthess. He thinks that optimizing where data resides relative to where it’s computed and improving how multiple simultaneous computations can be performed on the same set of data “are the two ingredients to our future algorithms and application design.”

Water Versus Watts: The Swiss supercomputer is cooled by water from nearby Lake Lugano.Photo: Marco Carocari

Applications that run on Piz Daint, which is named for a mountain in the Swiss Alps, use custom algorithms that intentionally position large data sets for processing on either the GPU or the CPU so the data won’t have to be moved much or reconfigured later on. These efficiencies speed computing, because Piz Daint isn’t taking time to locate and cull data. They also decrease energy consumption, by reducing data movement between processors and even within one processor in the case of the GPUs.

The machine needed to be built in two steps because the team had to wait for supercomputer manufacturer Cray to release the hybrid version of the XC30 architecture. But Schulthess says this was almost preferable because it gave the CSCS group time to do extensive testing on the smaller network and iron out any kinks before adding the XC30 to create the hybrid system in October.

It was worth the effort. “It’s bad to say as an engineer when you’re surprised, but Piz Daint in the end was so energy efficient, it was actually better than we expected,” Schulthess says.

Feng hopes that Piz Daint’s success will encourage other research groups to be more aware of their supercomputers’ carbon footprints. This would be a step toward “simulating the climate, not generating it,” he says.

This article originally appeared in print as “Small Swiss Supercomputer Shows the Way Ahead on Efficiency.”