Scaling up COVID-19 rapid antigen tests: promises and challenges

Abstract

WHO recommends a minimum of 80% sensitivity and 97% specificity for antigen-detection rapid diagnostic tests (Ag-RDTs), which can be used for patients with symptoms consistent with COVID-19. However, after the acute phase when viral load decreases, use of Ag-RDTs might lead to high rates of false negatives, suggesting that the tests should be replaced by a combination of molecular and serological tests. When the likelihood of having COVID-19 is low, such as for asymptomatic individuals in low prevalence settings, for travel, return to schools, workplaces, and mass gatherings, Ag-RDTs with high negative predictive values can be used with confidence to rule out infection. For those who test positive in low prevalence settings, the high false positive rate means that mitigation strategies, such as molecular testing to confirm positive results, are needed. Ag-RDTs, when used appropriately, are promising tools for scaling up testing and ensuring that patient management and public health measures can be implemented without delay.

Introduction

It has been nearly 1 year since the COVID-19 pandemic started and most countries still face major challenges in scaling up testing capacity, coupled with a lack of understanding of the different types of tests and how they can be used. To combat the COVID-19 pandemic, Tedros Adhanom Ghebreyesusm, the director-general of WHO, urged countries to “test, test, test.”.¹ Yet, few countries have managed to scale up testing capacity to gather sufficient population-level data to inform public health decisions on reopening of schools, return to work, mass gatherings, and travel, to allow easing of restrictions. The consequences for individuals, public health, and the economy are substantial.

Molecular assays to diagnose COVID-19 were quickly developed once the genetic sequence of SAR-CoV-2 was published in January, 2020.² These assays typically use PCR to amplify and detect viral RNA, and are highly sensitive and specific. Most assays require laboratory facilities with robust infrastructure and highly trained staff. Normally, results are available in less than 2 h, but many countries are seeing delays of up to 7 days.³ Delays in obtaining molecular testing results can increase the risk of virus transmission. The longer patients wait, the more likely they will not self-isolate at the time that they are most infectious and will resume daily activities before receiving test results. Although rapid technological advances in automated portable sample-to-answer molecular testing platforms have allowed testing to be deployed outside laboratory settings, and provide results in less than 1 h, these technologies are still equipment-dependent and the manufacture and scale up takes time. Hence the supply of devices and cartridges is short of global demand, especially for countries with weak or scarce laboratory infrastructure. Another limitation of molecular testing is the global competition for reagents and supplies, which has severely slowed down testing capacity, particularly in resource-constrained settings. The consequences of these limitations include continued restrictions and delays in confirming or excluding SARS-CoV-2 infection at the individual-level for case management or isolation, and at population-level for surveillance and response purposes.

Molecular testing is inherently difficult to scale up. Pooling samples for molecular testing has been suggested as a cost-effective way of scaling up the number of samples tested per day. Although pooling might be useful for large scale surveys, for individual detection there is a risk of false negative results due to sample dilution in the pooling process. Furthermore, batching a large number of samples followed by testing individual samples of positive pools would result in delays in getting the test results sent back. These limitations show the need for well thought out options in addition to molecular assays.

An alternative assay that can alleviate some of the bottlenecks created by molecular testing would be antigen testing. COVID-19 antigen tests diagnose active infection by detecting SARS-CoV-2 viral proteins in various specimen types. They are available as a single use, lateral flow, antigen-detection rapid diagnostic tests (Ag-RDTs) that can be visually read or processed and read using a small portable device (table 1 ). Both can be done outside the laboratory and provide a result within 15–20 min. These rapid tests can be produced much faster and cheaper in larger quantities for large scale deployment. Although these tests can be highly specific, they are generally not as sensitive as molecular tests. As of Nov 27, 2020, six Ag-RDTs have received US Food and Drug Administration (FDA) Emergency Use Authorization and two have received WHO Emergency Use Listing and are undergoing independent evaluation.2, 4, 5

Table 1.

Examples of COVID-19 antigen-detection rapid diagnostic tests

	Sample type	Time of sample collection*	Result reading	Sensitivity, specificity†	Comments
Abbott BinaxNOW, USA	Nasal swab	0–7 days	Visual, 15 min	97%, 99%	WHO Emergency Use Listing; US FDA Emergency Use Authorization; app for results; influenza A and B tests available
Abbott Panbio, USA	Nasal swab, nasopharyngeal swab	0–7 days	Visual, 15–20min	93%, 99%	WHO Emergency Use Listing; US FDA Emergency Use Authorization pending
Access Bio CareStart, USA	Nasal swab, nasopharyngeal swab	0–5 days	Visual, 15–20min	88%, 100%	US FDA Emergency Use Authorization
BD Veritor, USA	Nasal swab	0–5 days	Instrument, 30 min	84%, 100%	US FDA Emergency Use Authorization
LumiraDx, UK	Nasal swab	0–12 days	Instrument, 12 min	98%, 97%	US FDA Emergency Use Authorization
Quidel Sofia SARS Antigen Fluorescent Immunoassay, USA	Nasal swab, nasopharyngeal swab	0–5 days	Instrument, 20 min	97%, 100%	US FDA Emergency Use Authorization; does not differentiate between SARS-CoV and SARS-CoV-2
Quidel Sofia Flu and SARS Antigen Fluorescent Immunoassay, USA	Nasal swab, nasopharyngeal swab	0–5 days	Instrument, 20 min	95%, 100%	US FDA Emergency Use Authorization
SD Biosensor, South Korea	Nasal swab, nasopharyngeal swab	Not stated	Visual, 15–30min	97%, 100%	WHO Emergency Use Listing

Data from the Foundation for Innovative New Diagnostics.² SARS-CoV=severe acute respiratory syndrome coronavirus. FDA=Food and Drug Administration.

^*Days after symptom onset.

^†Data from manufacturers.

Understanding the limitations of these Ag-RDTs is important when trying to scale up testing to detect cases of COVID-19 and provide data to inform public health measures, and ease restrictions. Here we explore the use of Ag-RDTs across different populations with various risks of acquiring and transmitting COVID-19 for a more informed public health approach.

Test accuracy and prevalence of infection

When considering which test to use and how to interpret results, two components should be considered: the test sensitivity and specificity, which provide information on its performance to accurately measure the presence or absence of the disease. Test sensitivity and specificity are inherent characteristics of a test and do not change when used in different populations. However, the usefulness of a test for a particular population is defined by the positive predictive value (PPV) and negative predictive value (NPV) of the test result for that particular population. The PPV is defined as the probability that a person with a positive test result truly has the disease. The NPV is defined as the probability that a person with a negative test result truly does not have the disease. Both measures vary depending on the true prevalence of the disease or condition in a population, which is a particularly important consideration in the COVID-19 pandemic because disease prevalence varies across different populations and over time.

We compared the relationship between test performance, predictive values, and disease prevalence (table 2 –4). Test specificity is an issue at lower prevalence of infection; a lower prevalence means lower PPV and a higher number of false positive results. As prevalence of infection in the community increases, the PPV of a test also increases, and the number of false positive results decreases. Conversely, sensitivity is a concern at higher prevalence; a higher prevalence means lower NPV and a higher number of false negatives. Thus, a test should be specific enough to minimise the proportion of cases erroneously diagnosed as positive in low prevalence settings, and sensitive enough to avoid missing a diagnosis as COVID-19 prevalence increases.

Table 2.

Relationship between likelihood of testing positive, test performance, PPV, and NPV for symptomatic people

	Predictive values		Distribution of test outcomes among 10 000 people tested
	PPV	NPV	True positive	False positive	True negative	False negative
25% likelihood of testing positive
80% sensitivity, 97% specificity	90%	94%	2000	225	7275	500
80% sensitivity, 98% specificity	93%	94%	2000	150	7350	500
80% sensitivity, 99% specificity	96%	94%	2000	75	7425	500
90% sensitivity, 99% specificity	97%	97%	2250	75	7425	250
50% likelihood of testing positive
80% sensitivity, 97% specificity	96%	83%	4000	150	4850	1000*
80% sensitivity, 98% specificity	98%	83%	4000	100	4900	1000*
80% sensitivity, 99% specificity	99%	83%	4000	50	4950	1000*
90% sensitivity, 99% specificity	99%	91%	4500	50	4950	500

People presenting for care or at testing centres. NPV=negative predictive value. PPV=positive predictive value.

^*Mitigation strategy is to confirm negatives.

WHO has published advice on the use of Ag-RDTs during the COVID-19 pandemic, and reversed previous advice that urged countries not to use immunodiagnostic tests, which included antigen and antibody tests.6, 7 In September, 2020, WHO recommended minimum sensitivity as 80% and specificity as 97% for antigen tests, compared with a molecular test. In its newest interim guidance,⁷ WHO recognises that despite lower sensitivity than molecular tests, antigen tests offer the possibility of rapid, inexpensive detection of SARS-CoV-2 in individuals who have high viral loads and hence are at high risk of transmitting the infection to others.

Test accuracy, accessibility, and time to result

Testing is a crucial tool in the pandemic response to identify and confirm COVID-19 in those who are symptomatic so they can receive appropriate care and follow public health measures such as self-isolation, and contact tracing can be implemented without delay. At the population-level, the goal of testing is to identify those who are infectious and at risk of transmission. Studies8, 9, 10 examining the infectious period of patients with COVID-19 showed that the virus can be cultured from patients within the first 8 days after symptom onset. This finding corresponds to the manufacturers' recommendations that Ag-RDTs can be used within the first 5–7 days after symptom onset (table 1). Ag-RDTs might not be as accurate as molecular tests, but they are more accessible in terms of availability and ease of use, and can be used to scale up testing outside of laboratory settings, including frequent repeat testing if necessary. The availability of results in 15–20 min and frequency of testing has been shown by modelling studies to be more important than sensitivity.¹¹

Although evaluation of Ag-RDTs approved for emergency use by WHO or the US FDA show that their performance conforms with WHO recommendations, independent evaluation of Ag-RDTs done by FIND² showed that their performance varies in different countries depending on the composition of the evaluation panel and the viral load in the specimens used for evaluation (appendix p 1).² Patients with COVID-19 can remain RNA positive for 2–3 weeks after the onset of symptoms but, are undetectable for antigen at 7–8 days after symptom onset, which coincides with the infectious period of a patient with COVID-19.¹² The duration of a positive molecular test beyond the infectious period might affect essential workers returning to work, especially in countries struggling with shortages of health-care workers.

Another important role for testing in the pandemic response is the identification of those who do not have COVID-19 so that they can travel, return to school, work, and attend mass gatherings. Wide availability of Ag-RDTs and the rapid result time offer the promise of efficiently testing a large number of people in community settings to ensure safe environments for resumption of activities, which are important for social, educational, and economic reasons. Careful consideration of PPV and NPV calculated from test performance and estimated prevalence of infection are crucial to weigh the risks and consequences of false positive and false negative results, and could help to guide decision making on further mitigation strategies.

Use of Ag-RDTs and mitigation strategies

Symptomatic patients

The population group that would have the highest likelihood of testing positive would be patients with symptoms consistent with COVID-19 presenting for care at a hospital or testing centre. The purpose of Ag-RDTs in this group is to diagnose patients suspected with COVID-19 in cases where molecular testing is not available or delays in molecular testing results are hampering appropriate patient management and disease control efforts.

If we estimate that the likelihood of this group testing positive (prevalence) is 25–50%, then using an Ag-RDTs with minimum performance characteristics of 80% sensitivity and 97% specificity (per WHO interim guidance) would have acceptable PPVs of 90–96% provided that the test is used within 7 days after symptom onset (table 2).¹³ However, as the prevalence increases above 36%, the NPV decreases to less than 90%, generating unacceptable numbers of false negative results. For example, at a true prevalence of 50%, using a test of 80% sensitivity and 97% specificity in 10 000 symptomatic patients presenting for care, might result in as many as 1000 false negative results. In such scenario, Ag-RDTs can provide a quick result to confirm a clinical suspected case within 1 week after symptom onset. The mitigation strategy for these symptomatic patients who have a negative Ag-RDT result would be to collect another swab for confirmatory molecular testing.

However, for symptomatic patients presenting to care more than 7 days after symptom onset, when viral load has decreased to concentrations which are unlikely to be detected by Ag-RDTs, the best option would be to use a combination of molecular and antibody tests to confirm COVID-19. Studies14, 15, 16 have shown that at 2 weeks after symptom onset, most patients already have detectable concentrations of SARS-CoV-2 antibodies in serum and a combination of molecular and antibody tests can increase the sensitivity of detecting COVID-19.

Combining molecular or antigenic tests with antibody tests to maximise detection of active infection is an approach that has been used for other viral infections such as dengue.17, 18

Health-care workers and contacts of confirmed cases

The next group who has a higher likelihood of testing positive compared with the general population are health-care workers and those working in care homes with older patients, first responders, and contacts of confirmed cases, especially family members. This group is at risk of acquiring COVID-19 and transmitting disease to those in care. The purpose of Ag-RDTs in this group is to screen for asymptomatic SARS-CoV-2 infection so that those with a positive result can be isolated to stop disease transmission and given the appropriate care. Studies19, 20 have shown that symptomatic and asymptomatic individuals have viral loads within the detection limit of Ag-RDTs.

If we estimate that the likelihood of testing positive in this group is 5–10%, then false negative results would not be a concern. The WHO recommended minimum test specificity of 97% shows a PPV of 63–83%, which will lead to an unacceptable number of false positive results (table 3 ). For this population, test specificities of more than 98% would be needed to attain an acceptable PPV of more than 90%. The mitigation strategy in this case would be to consider the use of a two-step testing algorithm, in which a test of high sensitivity is used for initial screening and then Ag-RDT positive results are confirmed with molecular testing or another Ag-RDT with higher specificity. In Cameroon, a strategy that uses two-test diagnostic algorithms and incorporates RDTs has been implemented in testing asymptomatic and symptomatic patients to reduce delays. For the antigen-based diagnostic testing algorithm, symptomatic individuals were first screened for the SARS-CoV-2 antigen using an Ag-RDT. Molecular testing is used for all negative Ag-RDT samples. A positive diagnosis of SARS-CoV-2 infection was made if the Ag-RDT or molecular test results were positive (Yap Boum II, Epicentre, personal communication).

Table 3.

Relationship between test performance, PPV, and NPV for asymptomatic people

	Predictive values		Distribution of test outcomes among 10 000 people tested
	PPV	NPV	True positive	False positive	True negative	False negative
5% likelihood of testing positive
80% sensitivity, 97% specificity	58%	99%	400	285*	9215	100
80% sensitivity, 98% specificity	68%	99%	400	190*	9310	100
80% sensitivity, 99% specificity	81%	99%	400	95*	9405	100
90% sensitivity, 99% specificity	83%	99%	450	95*	9405	50
10% likelihood of testing positive
80% sensitivity, 97% specificity	75%	98%	800	270*	8730	200
80% sensitivity, 98% specificity	82%	98%	800	180*	8820	200
80% sensitivity, 99% specificity	90%	98%	800	90	8910	200
90% sensitivity, 99% specificity	91%	99%	900	90	8910	100

People at higher risk of acquiring or transmitting COVID-19 than the general population such as health-care workers, care home workers, and first responders. NPV=negative predictive value. PPV=positive predictive value.

^*Mitigation strategy is to confirm positives.

Asymptomatic individuals

Finally, we consider the use of Ag-RDTs in asymptomatic individuals in schools, workplaces, mass gatherings, and travellers. The purpose of Ag-RDTs is to ensure a safe environment free from COVID-19 by allowing those who test negative to resume their normal activities.

We estimate that the likelihood of testing positive in these groups would be low (possibly 1–2·5%), unless they are located in a COVID-19 outbreak area.21, 22, 23 An Ag-RDT with 80% sensitivity and 97% will result in NPVs of 99–100% which means that most people testing negative are likely to be true negatives (table 4 ).

Table 4.

Relationship between test performance, PPV, and NPV in the general asymptomatic population

	Predictive values		Distribution of test outcomes among 10 000 people tested
	PPV	NPV	True positive	False positive	True negative	False negative
1% likelihood of testing positive
80% sensitivity, 97% specificity	21%	100%	80	297*	9603	20
80% sensitivity, 98% specificity	29%	100%	80	198*	9702	20
80% sensitivity, 99% specificity	45%	100%	80	99*	9801	20
90% sensitivity, 99% specificity	48%	100%	90	99*	9801	10
2·5% likelihood of testing positive
80% sensitivity, 97% specificity	41%	99%	200	293*	9458	50
80% sensitivity, 98% specificity	51%	99%	200	195*	9555	50
80% sensitivity, 99% specificity	67%	99%	200	98	9653	50
90% sensitivity, 99% specificity	70%	100%	225	98	9653	25

NPV=negative predictive value. PPV=positive predictive value.

^*Mitigation strategy is to confirm all positives.

However, for those in this group who test positive, poor PPVs of 21–41% means that in screening 10 000 people, more people will have false positive results (n=293–97) than true positive results (n=80–200). Hence, Ag-RDT positive results in this group need to be confirmed with a more specific Ag-RDT or a molecular assay. Test performance reduces even further in-between epidemic waves and in nationwide surveys (around 1% point prevalence), when PPV drops to 21%. Increasing test specificity will improve the PPV and reduce the number of false positive results, but even at 90% sensitivity and 99% specificity, using these tests in a low prevalence population will always result in unacceptable PPVs and confirmatory testing is essential (table 4).

If molecular testing is available, why use Ag-RDTs?

A testing strategy for each country should consider scenarios where Ag-RDTs can be used to augment molecular testing. Since most Ag-RDTs have detection limits of 10⁴–10⁵ genome copies or cycle threshold values of 25–30, they can be used to rapidly triage or identify patients with high viral loads, easing the excessive demand or sole dependence on molecular testing.20, 24, 25, 26, 27 Ag-RDTs can be used in settings where molecular testing is available (panel ), but a rapid test offers a distinct advantage of a more effective response.

Panel. Examples of antigen-detection rapid diagnostic tests (Ag-RDTs) use.

• •In cases where the demand for confirming COVID-19 in symptomatic patients exceeds molecular testing capacity, the use of Ag-RDTs can rapidly reduce the number of molecular tests required and hence waiting time for results and costs
• •In specific settings such as busy hospitals or clinics where patients who present with symptoms consistent with COVID-19 are waiting more than 20 h for molecular test results because the laboratory is overwhelmed or there is no laboratory on site and specimens need to be sent to a central testing location to be processed, then the use of Ag-RDTs allows most patients to be triaged within 15 min and only a small proportion of patients need to wait for molecular test results before being admitted to the appropriate ward or sent home for self-isolation
• •The use of Ag-RDTs for any patient in a care home who develops symptoms consistent with COVID-19 would allow improved patient management and immediate implementation of appropriate measures to prevent the spread of COVID-19 to vulnerable patients within the care home

Moreover, the question should not be posed as either or, but rather as both and, according to country capacity and cost-effectiveness. Adopting the proposed two-step approach would make huge savings in the number of molecular tests done. Confirming test negatives at 50% prevalence would reduce the number of molecular tests by 90%, and confirming positive tests at 1–2·5% prevalence would mean around a 95% saving in molecular tests.

Conclusion

Given the current availability of different types of COVID-19 tests, most countries are still struggling to meet crucial testing demands for patient management and surveillance. Molecular tests and Ag-RDTs have different but complimentary roles in the pandemic response and case management. Understanding advantages and limitations of using Ag-RDTs in different populations across a prevalence range will allow the tests to be deployed concomitantly with others to improve the COVID-19 response.

Countries need to balance benefits of having rapid Ag-RDTs results for immediate and appropriate patient management and public health action against the harm of false positive and false negative results. Each country might have different risk–benefit profiles based on epidemiological data and other factors such as age demographics and the ability to control population movement across borders. Understanding limitations regarding the prevalence of infection in each population is important and risk mitigation strategies such as a two-test, two-step algorithm need to be considered, reserving molecular testing for confirmation of Ag-RDT results.

No test is perfect when it comes to the attributes of accuracy, accessibility, affordability, and timeliness of results. The choice for testing should be guided by these attributes and individual risks of acquiring and transmitting infection should be balanced against prevalence of COVID-19 in the population being tested. More work is still needed to better understand how often to use different tests and how to interpret and act on results. These fundamental considerations will inform testing strategies and policies and support effect modelling.

Testing strategies will have to be optimised as the COVID-19 pandemic evolves. Risk mitigation strategies will help to ensure that testing results are sufficiently accurate to provide appropriate care at the patient level, and sound evidence to inform public health action for interrupting disease transmission.