‘Test, re-test, re-test’: using inaccurate tests to greatly increase the accuracy of COVID-19 testing

To the Editor — Commenting recently on rapid point-of-care tests, US COVID-19 coordinator Deborah Birx said, “We are very quality-oriented. We don’t want false positives.”¹

“If they are incredibly accurate, we will work out the quickest way to release them. If they are not accurate, we will not release any of them.” echoed UK Chief Medical Officer Chris Whitty².

Given the need for testing³, the end goal is a quick, accurate and cheap test. With scientific innovation, we will, in time, attain this goal. But the best is becoming the enemy of the good. Meanwhile, avoidable infections are growing.

The ‘gold standard’ RT-PCR test for COVID-19 is highly accurate and reproducible, but is costly (US$125 per test kit, and over $15,000 to set up a processing lab) and slow (4–6 hours of processing time, and a turnaround of 2–4 days, including shipping)⁴.

At the other extreme, a Bangladeshi lab has reportedly developed a $3 rapid test kit that gives a result in under 15 minutes (ref. ⁴). But the accuracy of such point-of-care tests is questionable.

Smart tactics can help break this tradeoff between cost and quality.

First, consider two quick, cheap and inaccurate tests, each developed by a different lab, and based on detection of a different antibody — or of the same antibody, but via a different method. Suppose each test has a false-negative rate of 30%, and, for simplicity, zero false-positive results. What if both tests were administered to the same person? If the results of the two tests are independent, the chances of obtaining two false-negative results drops to 9% (and to less than 3% if a third independent test with similar characteristics is administered). Figure 1 illustrates this logic, which also applies to false-positive results, for a test with a 50% false-negative rate. (Reports suggest that the tests being considered for large-scale procurement in the UK are in this range^4,5). As a comparison, since 2017, rapid influenza diagnostic tests cleared by the US Food and Drug Administration have been required to achieve false-negative rates and false-positive rates of below 20% and 5%, respectively, compared with RT-PCR⁶.

Fig. 1.

Why re-testing increases testing accuracy.

Second, this recommendation to test and re-test can apply elsewhere too. Consider a test that displays the same false-negative and false-positive rates as the tests above — and is also unreproducible. If a patient is tested twice in succession with this test, the results could vary. Counterintuitively, this lack of reproducibility may be advantageous. Again, if the results of the two tests are independent, the likelihood of two false-negative results drops to 9%.

The implication is clear: even an inaccurate test tells us something. Or, to misquote the World Health Organization: ‘test, re-test, re-test’.

Use of this strategy would be made easier if there were a database — updated in real time — of point-of-care tests being generated by labs around the world. This database, which could be assembled by an international organization such as the World Health Organization, would list the lab and test name, the antibody that the test detects (e.g., IgG, IgN or both⁷), the detection method (e.g., lateral-flow immunoassay) and its accuracy and reproducibility, the turnaround time, the testing-kit cost and the sample-processing cost. With this information in hand, governments and international organizations could advise scientists on what combination of cheap tests would be optimal for specific nations.

Third, consider a quick and cheap test with a 30% false-positive rate, and for simplicity, zero false-negative results. First, one could test many people with this test, and then test the subset who test positive with a highly accurate test. This economizes on the use of scarce but accurate test kits while allowing much wider testing than would have been possible with the few accurate test kits available. In short: ‘test, triage, re-test’.

Finally, smart tactics can enable cheaper testing with the expensive RT-PCR tests, if a sample obtained can ‘fuel’ multiple tests. Some German hospitals are doing ‘block tests’ using a sample pooled from ten employees, and then are testing individually only if there is a positive result⁸.

One can take this idea further, by applying principles from discrete optimization. If the test is positive, then one would test two blocks of five samples each, and then further test the arm that tests positive. This mimics ‘branch and bound’ algorithms for solving discrete optimization problems such as the famous ‘traveling salesperson’ problem⁹, which requires finding the cheapest route for delivering supplies to a fixed number of stores.

These simple examples are illustrative. Naturally, several factors would come into play in their implementation. For example, block testing would increase time to diagnosis and may be more useful for asymptomatic low-risk cases.

Finally, all inaccuracies are not equal. Right now, tests with a high false-positive rate are less problematic — since people are being advised to stay home anyway — than those with high false-negative rates. Furthermore, a false-positive result for SARS-CoV-2 is unlikely to initiate treatment with negative side effects, as would chemotherapy for misdiagnosed cancer.

The key point here is that creative use of currently available cheap and quick tests — even if they are inaccurate and unreproducible — can go a long way to reaching adequate levels of accuracy and precision, at least until the gold-standard tests can be developed.

Competing interests

A.D. has received a donation of one Axceed 260 machine from Bioscience Diagnostic Medical Technology in China, and is the academic sponsor on a study focusing on PCR and antibody testing for COVID-19 on behalf of the Department of Health.