GPUs Can Now Analyze a Billion Complex Vectors in Record Time

The advancement boosts the speed of GPU image analysis eight-fold

2 min read

many photos in a grid

The complexity of a digital photo cannot be understated.

Each pixel comprises many data points, and there can be millions of pixels in just a single photo. These many data points in relation to each other are referred to as “high-dimensional” data and can require immense computing power to analyze, say if you were searching for similar photos in a database. Computer programmers and AI experts refer to this as “the curse of high curse of high dimensionality.” 

In a study published July 1 in IEEE Transactions on Big Data, researchers at Facebook AI Research propose a novel solution that aims to ease the burden of this curse. But rather than the traditional means of a computer’s central processing units (CPUs) to analyze high-dimensional media, they’ve harnessed Graphical Processing Units (GPUs). The advancement allows 4 GPUs to analyze more than 95 million high-dimensional images in just 35 minutes. This speed is 8.5 times faster than previous techniques that used GPUs to analyze high-dimensional data.

“The most straightforward technique for searching and indexing [high-dimensional data] is by brute-force comparison, whereby you need to check [each image] against every other image in the database,” explains Jeff Johnson, a research engineer at Facebook AI Research who co-developed the new approach using GPUs. “This is impractical for collections containing billions of vectors.”

CPUs, which have high memory storage and thus can handle large volumes of data, are capable of such a task. However, it takes a substantial amount of time for CPUs to transfer data among the various other supercomputer components, which causes an overall lag in computing time.

In contrast, GPUs offer more raw processing power. Therefore, Johnson and his team developed an algorithm that allows GPUs to both host and analyze a library of vectors. In this way, the data is managed by a small handful of GPUs that do all the work. Notably, GPUs typically have less overall memory storage than CPUs, but Johnson and his colleagues were able to overcome this pitfall using a technique that compresses vector databases and makes them more manageable for the GPUs to analyze.

“By keeping computations purely on a GPU, we can take advantage of the much faster memory available on the accelerator, instead of dealing with the slower memories of CPU servers and even slower machine-to-machine network interconnects within a traditional supercomputer cluster,” explains Johnson.

The researchers tested their approach against a database with one billion vectors, comprising 384 gigabytes of raw data. Their approach reduced the number of vector combinations that need to be analyzed, which would normally be a quintillion (1018), by at least 4 orders of magnitude.

“Both the improvement in speed and the decrease in database size allow for solving problems that would otherwise take hundreds of CPU machines, in effect democratizing large-scale indexing and search techniques using a much smaller amount of hardware,” he says.

Their approach has been made freely available through the Facebook AI Similarity Search (Faiss) open source library. Johnson notes that the computing tech giant Nvidia has already begun building extensions using this approach, which were unveiled at the company’s 2021 GPU Technology Conference.

The Conversation (0)