The December 2022 issue of IEEE Spectrum is here!

Close bar

The complexity of a digital photo cannot be understated.

Each pixel comprises many data points, and there can be millions of pixels in just a single photo. These many data points in relation to each other are referred to as “high-dimensional” data and can require immense computing power to analyze, say if you were searching for similar photos in a database. Computer programmers and AI experts refer to this as “the curse of high curse of high dimensionality.” 

In a study published July 1 in IEEE Transactions on Big Data, researchers at Facebook AI Research propose a novel solution that aims to ease the burden of this curse. But rather than the traditional means of a computer’s central processing units (CPUs) to analyze high-dimensional media, they’ve harnessed Graphical Processing Units (GPUs). The advancement allows 4 GPUs to analyze more than 95 million high-dimensional images in just 35 minutes. This speed is 8.5 times faster than previous techniques that used GPUs to analyze high-dimensional data.

“The most straightforward technique for searching and indexing [high-dimensional data] is by brute-force comparison, whereby you need to check [each image] against every other image in the database,” explains Jeff Johnson, a research engineer at Facebook AI Research who co-developed the new approach using GPUs. “This is impractical for collections containing billions of vectors.”

CPUs, which have high memory storage and thus can handle large volumes of data, are capable of such a task. However, it takes a substantial amount of time for CPUs to transfer data among the various other supercomputer components, which causes an overall lag in computing time.

In contrast, GPUs offer more raw processing power. Therefore, Johnson and his team developed an algorithm that allows GPUs to both host and analyze a library of vectors. In this way, the data is managed by a small handful of GPUs that do all the work. Notably, GPUs typically have less overall memory storage than CPUs, but Johnson and his colleagues were able to overcome this pitfall using a technique that compresses vector databases and makes them more manageable for the GPUs to analyze.

“By keeping computations purely on a GPU, we can take advantage of the much faster memory available on the accelerator, instead of dealing with the slower memories of CPU servers and even slower machine-to-machine network interconnects within a traditional supercomputer cluster,” explains Johnson.

The researchers tested their approach against a database with one billion vectors, comprising 384 gigabytes of raw data. Their approach reduced the number of vector combinations that need to be analyzed, which would normally be a quintillion (1018), by at least 4 orders of magnitude.

“Both the improvement in speed and the decrease in database size allow for solving problems that would otherwise take hundreds of CPU machines, in effect democratizing large-scale indexing and search techniques using a much smaller amount of hardware,” he says.

Their approach has been made freely available through the Facebook AI Similarity Search (Faiss) open source library. Johnson notes that the computing tech giant Nvidia has already begun building extensions using this approach, which were unveiled at the company’s 2021 GPU Technology Conference.

The Conversation (0)

Why Functional Programming Should Be the Future of Software Development

It’s hard to learn, but your code will produce fewer nasty surprises

11 min read
A plate of spaghetti made from code
Shira Inbar

You’d expectthe longest and most costly phase in the lifecycle of a software product to be the initial development of the system, when all those great features are first imagined and then created. In fact, the hardest part comes later, during the maintenance phase. That’s when programmers pay the price for the shortcuts they took during development.

So why did they take shortcuts? Maybe they didn’t realize that they were cutting any corners. Only when their code was deployed and exercised by a lot of users did its hidden flaws come to light. And maybe the developers were rushed. Time-to-market pressures would almost guarantee that their software will contain more bugs than it would otherwise.

Keep Reading ↓Show less