Global regulators have largely stepped out of diagnostics manufacturers’ way to enable them to quickly bring COVID-19 tests to the public. That has led to a deluge of testing options on the market, and in many cases, the reliability and accuracy of these tests is unclear.
That led us to wonder: Is anyone testing the tests?
We found one organization that’s on it. The Foundation for Innovative New Diagnostics, or FIND, headquartered in Geneva, Switzerland, is evaluating its way through a list of over 300 COVID-19 tests manufactured globally, and today published its first results.
FIND’s effort, which it is undertaking in collaboration with the World Health Organization (WHO), involves running thousands of coronavirus samples through the tests and comparing their performance against a gold standard. The organization will rank the tests based on sensitivity.
“There’s a rapid influx of diagnostics without clear knowledge of whether or not they work,” says Cassandra Kelly-Cirino, director of emerging threats at FIND and lead investigator in the program. “The purpose [of our project] is to inform global decision making about which tests to use.”
The effort may turn out to be vital in a season when regulatory agencies are, in the interest of speed, fast-tracking authorizations of tests. “The regulatory agencies do not require extensive data,” from manufacturers during emergencies such as the current pandemic, says Kelly-Cirino.
The U.S. Food and Drug Administration, for example, has authorized through its Emergency Use Authorization pathway more than 30 COVID-19 tests in the past four weeks. That’s lightning speed compared with the normal course of business.
But in the interest of speed, subpar COVID-19 tests are reaching patients, sometimes in large numbers. In two painful examples, authorities in Spain and Britain ordered millions of COVID-19 test kits, largely from manufacturers in China, only to discover that they were unreliable for most patients.
“It’s that kind of thing that’s making everybody quite nervous,” says Kelly-Cirino. “It’s hard to go slow when things are so critical, but if we use tests that don’t perform properly, it can do more harm than good.”
FIND in February announced a call to manufacturers to submit their tests for evaluation. In exchange, the manufacturers get a global stamp of independent evaluation, and a listing on FIND’s pipeline. Manufacturers have submitted more than 300 tests, and FIND today posted the results of the first five it has evaluated.
All five achieved 100 percent clinical sensitivity on positive samples, and at least 96 percent specificity on negative samples. The tests were submitted by KH Medical, SD Biosensor, BGI Health, DAAN Gene, and Tib Molbiol. Each company submitted a molecular test that detects the presence of viral genetic material.
FIND evaluates tests that meet their criteria on a first-come, first-served basis. The five tests described today were the first to arrive, Kelly-Cirino says. She aims to post performance data for two or three more tests each week for the foreseeable future.
How to test a test
The FIND Pandemic Preparedness team regularly evaluates diagnostics for outbreak diseases including Ebola, Lassa fever (pictured, in Nigeria), and now COVID-19.Photo: Devy Emperador/FIND
FIND’s ambitious endeavor will take some time, and depends on several variables. They must operate through a network of laboratory partners, secure the right samples, and depend on companies to send them the actual tests.
Most tests for the novel coronavirus, also knows as SARS-CoV-2, fall into two broad categories: molecular tests and immunoassay. FIND and its partners are evaluating the performance of tests in both categories.
Molecular tests, also known as nucleic acid tests, detect the virus’s genetic code using a decades old technology called reverse transcription polymerase chain reaction, or RT-PCR. These tests can yield accurate results in the earliest days of a COVID-19 infection.
To evaluate performance, FIND researchers first verify the manufacturers’ claims on sensitivity. Then 50 positive and 100 negative samples are run through each molecular test. Then they compare the results to that of a reference test, or gold standard, which is another PCR-based test.
“We’re already finding that several of the new commercially available tests are likely more sensitive than the reference test,” Kelly-Cirino says.
FIND conducts evaluations of the molecular tests at the Hôpitaux Universitaires de Genève, which has a collection of archived patient specimens. FIND plans to run the same 150 specimens through each molecular test to provide a true side-by-side comparison. When the 150 specimens are exhausted, they’ll move on to another set of 150 specimens.
Immunoassays vs PCR
Molecular tests have been widely used during the COVID-19 pandemic. But these typically require special instruments that get plugged into a wall, a biosafety cabinet, and trained personnel. They can be expensive and are often only available in centralized locations. Results usually take at least 24 hours to come back.
Rapid molecular tests, which can provide results in minutes, are available. But they are relatively new and expensive, and it’s a select few, globally, that will have access to such technology.
Immunoassays offer an alternative to molecular tests. One type of immunoassay, an antigen-based test, detects the presence of protein structures, called spikes, that are unique to the virus. Antigen-based tests require no specialized personnel or large instruments, are relatively inexpensive, and can deliver results in minutes.
These attributes make antigen-based tests ideal in resource-poor regions and decentralized health care facilities. “You take the swab, dip it into a buffer, put it on the pregnancy-style test and get the results,” says Kelly-Cirino. “You don’t need them to be run in a hospital or laboratory.”
But antigen-based tests are not as sensitive as molecular tests, especially during the early stages of infection. They are also taking longer to develop. “We know there’s a lot in development, but they’re not going to be ready for market for at least three months, and maybe more like six to ten months,” says Kelly-Cirino.
FIND and its partners plan to evaluate antigen-based tests by enrolling thousands of people with suspected cases of COVID-19. The volunteers will agree to be swabbed twice. One swabbed sample goes through an antigen test and, for comparison, the other swabbed sample goes through PCR. FIND has prioritized five antigen tests to evaluate, says Kelly-Cirino.
The other major category of immunoassay detects the antibodies our immune systems produce in response to the virus. These tests typically can’t determine whether a person currently has the virus. But they can prove that someone was previously infected with it.
FIND’s evaluation of antibody-based tests will require a series of blood samples to be drawn from patients. That’s tricky during a pandemic, because people generally don’t want to come back to a testing site or hospital when they don’t need to. FIND has not yet begun evaluation of antibody-based tests.
In an effort to generate an even larger resource, FIND put out an invitation [PDF] on 27 March to any other groups that are collecting performance data on SARS-CoV-2 tests. FIND is aggregating the data into one freely available online resource.
