For Better Computing, Liberate CPUs From Garbage Collection

An accelerator unit improves both the performance and efficiency of a system by taking over one simple task

Illustration of a CPU chained to a garbage can, and the chain is broken.
Illustration: IEEE Spectrum; icons: iStockphoto
Advertisement
Journal Watch report logo, link to report landing page

Without you noticing (much), your computer is working hard in the background to organize its memory system. On top of its many tasks, a CPU must do something called “garbage collection,” whereby it identifies and deletes redundant or irrelevant data from applications to free up additional memory space.

Garbage collection is meant to spare programmers from having to manually address this unnecessary data, but the automated process that CPUs are tasked with consumes a lot of computational power—up to 10 percent or more of the total time a CPU spends on an application.

While completing his PhD at the University of California, Berkeley, Martin Maas, who now works at Google, designed a new type of device that relieves the CPU from its garbage collection duties. The design is described in a paper published 23 April in IEEE Micro.

Maas notes that CPUs, which have traditionally been assigned garbage collection, were never specifically designed for the task. “CPUs are built to be flexible and run a wide range of applications. As a result, they are relatively large and can take up a significant amount of power,” he explains.

Instead, Maas and his colleagues created a compact accelerator unit that requires a small amount of chip area and power. It can be added to the CPU, similar to how many modern processor chips are integrated into graphics processing units.

“While the software application is running on the CPU, this unit sits on the side and performs garbage collection for the application,” says Maas. “In principle, this means that you could build a system where the software does not have to worry about garbage collection at all and just keeps using the available memory.”

Notably, the accelerator unit does its garbage collection in a more efficient manner than how CPUs have traditionally done it. For instance, CPUs have to complete some garbage collection actions in a certain order, whereas the new accelerator unit exploits the fact that these actions do not need to be done sequentially. By doing a larger number of memory operations simultaneously, it is more efficient at garbage collection than a CPU.

When Maas’s team compared the accelerator to a small CPU core, they found that their accelerator had 18 times better performance given each device’s respective sizes. In terms of power consumption, they estimate that the accelerator reduces the energy required for garbage collection by 15 percent.

Delegating the task of garbage collection to an accelerator unit could enhance the productivity of computers. Maas points out that although the 10 percent of computation power that CPUs dedicate to garbage collection may not seem like a lot, globally this represents a large amount of computing resources.

He also points out that many garbage collection mechanisms can result in unpredictable pauses, where the computer system stops for a brief moment to clean up its memory. “While you typically don’t notice it, this can be a problem in scenarios where you care about your system being very responsive, and programmers go out of their way to avoid these pauses, often at the cost of spending even more cycles on garbage collection,” he says. The new accelerator unit addresses both of these issues, paving the way for more efficient computing.

In their paper, the researchers suggest that this new device could become a common component of future systems, bypassing the need for CPUs to be burdened by garbage collection.

The Tech Alert Newsletter

Receive latest technology science and technology news & analysis from IEEE Spectrum every Thursday.

About the Tech Talk blog

IEEE Spectrum’s general technology blog, featuring news, analysis, and opinions about engineering, consumer electronics, and technology and society, from the editorial staff and freelance contributors.