The speed of high-performance computing has soared from around 100 gigaflops in 1993 to over 50 petaflops today and is on course to hit the long-sought exascale (1018 floating-point operations per second) mark in the 2020s. Yet this remarkable supercomputing progress can be something of a super nightmare for the institutes and government agencies asked to invest the hundreds of millions, even billions of dollars that leading systems can cost.
“We are achieving a 1,000[-fold] improvement over 10 years, so after just 5 years a conventional supercomputer is no longer able to perform (at the necessary standard) and has to be trashed,” says Michihiro Koibuchi, a systems architect at Japan’s National Institute of Informatics, in Tokyo. Koibuchi and his colleagues think they have a solution that will let users get more out of older machines: free-space optics, lasers that link supercomputer nodes through the air.
Typically, the thousands of processing nodes that make up a high-end supercomputer are clustered into several popular network topologies depending on the computer’s primary use. The topologies are implemented as cable-connected switches that link a select group of server-stuffed cabinets. With multiple users running different jobs at the same time, mapping a program’s communications needs to the most suitable topology for a particular job is essential for efficient processing. But such mapping becomes problematic when the same system must serve very different types of applications at the same time.
“Day by day, fragmentation occurs as tasks end at different times and others are added, causing mapping to break down,” says Koibuchi. Consequently, some applications end up performing inefficiently—for instance, when their data is forced to hop between a number of switches, or when an ordered parallel application is forced to adapt to a random topology used for irregular applications.
Koibuchi and his colleagues from several universities believe that free-space optics (FSO) can mitigate this breakdown and improve performance and resource use. FSO uses laser light to transmit data through the air from one terminal to another in its line of sight. With its wide, gigabit-scale bandwidth, it is used outdoors in interbuilding links and in aerospace and satellite communications. So handling the 40-gigabit-per-second transfer rates of high-performance computers indoors shouldn’t be a problem.
“Rather than relying on a few fixed topologies, we propose using FSO terminals mounted on the cabinets to provide line-of-sight communications between almost any two cabinets,” says Koibuchi. “Network topologies can then be reconfigured dynamically. This will help applications better maintain their topologies, reduce fragmentation and latency, and also cut the amount of cable used.”
The idea gets support from Erik D’Hollander, a parallel computing expert at Ghent University, in Belgium. “Because FSO terminals can be readily adjusted for any type of topology, they have a huge potential to replace the data-communication fixed backbone between... supercomputer cabinets,” he says.
FSO terminals suitable for supercomputer use are not yet available, however, so Koibuchi and his colleagues simulated several terminal layouts in software. What they found was a reduction in latency of up to 9 percent and a 36 percent reduction in fiber-optic cable length. That’s a significant savings, considering that a system might have thousands of kilometers of cables and they are discarded when machines become obsolete.
To confirm these results physically, the researchers constructed four small prototype bidirectional FSO terminals from off-the-shelf components. These included optical transceivers rated at both 10 and 40 Gb/s, commodity infrared lasers, and collimator lenses used to direct the beam, as well as motors and gears to aim the terminals in any direction with a margin of error of just 0.003 degrees at 40 meters. At this distance, the tests confirmed the simulation results and achieved a data rate of 38 Gb/s.
Despite the tests’ success, Koibuchi admits that commercializing the technology is not likely to happen anytime soon. “There is only one real hurdle left to overcome, but it’s severe. At present there are no mass-produced high-bandwidth FSO terminals.”
D’Hollander, too, sees the economics of FSO as a problem. “I’m less optimistic about the cost effectiveness of FSO terminals employing accurate steering and tracking equipment,” he says. “Perhaps fixed FSO terminals without a steering mechanism would be a viable alternative.”
Koibuchi says his team is studying the high-bandwidth FSO problem with NEC Green Platforms Research Laboratories, a unit of the Japanese maker of electronics and supercomputers.
This article originally appeared in print as “Giving Supercomputers a Second Wind.”