Real-world clinical performance of commercial SARS-CoV-2 rapid antigen tests in suspected COVID-19: A systematic meta-analysis of available data as of November 20, 2020

Graphical abstract

Abstract

Objectives

Rapid antigen tests, or RATs, are a type of lateral flow chromatographic immunoassay utilized to aid the diagnosis of SARS-CoV-2 infection. We performed a systematic meta-analysis to compare the real-world performance of commercially available RATs.

Methods

We searched several databases and websites for manufacturer-independent prospective clinical performance studies comparing SARS-CoV-2 RATs and RT-PCR. Only studies on RATs that did not need a separate reader for result retrieval and that reported data on viral load, patients’ symptom status, sample type, and PCR assay used were included.

Results

19 studies utilizing 11,109 samples with 2,509 RT-PCR-positives were included. RAT sensitivity varied between 28.9% (95% CI 16.4–44.3) and 98.3% (95% CI 91.1–99.7), likely dependent upon population characteristics, viral load, and symptom status. RAT specificity varied between 92.4% (95% CI 87.4–95.9) and 100% (95% CI 99.7–100) with one outlier. The RATs by Roche Diagnostics/SD Biosensor and Abbott had the highest pooled sensitivity (82.4% [95% CI 74.2–88.4] and 76.9% [95% CI 72.1–81.2], respectively). Sensitivity in high-viral-load samples (cycle threshold ≤25) showed heterogeneity among the different RATs.

Conclusion

The RATs offered by Roche Diagnostics/SD Biosensor and Abbott provide sufficient manufacturer-independent, real-world performance data to support their use to detect current SARS-CoV-2 infection, particularly in high-viral-load populations.

Background

Nucleic acid amplification tests, such as real-time reverse transcriptase PCR (RT-PCR), performed on upper respiratory tract samples, are considered the gold standard for clinical diagnostic detection of current SARS-CoV-2 infection (Centers for Disease Control and Prevention, 2020, European Centre for Disease Prevention and Control, 2020). RT-PCR requires a professionally run laboratory with molecular-biological competence and transport infrastructure between the place of sample collection and the laboratory. Rapid antigen tests, or RATs, are a type of lateral flow chromatographic immunoassay used to support the rapid diagnosis of individuals suspected of SARS-CoV-2 infection, either in those presenting symptoms or those who have had contact with positive cases. These point-of-care tests are less clinically sensitive than RT-PCR assays but offer a comparable specificity (Centers for Disease Control and Prevention, 2020). Several RATs have been authorized for use under EUA and/or the CE mark (US Food and Drug Administration, 2020, World Health Organization, 2020b), presenting manufacturer-generated clinical performance data across heterogeneous patient populations.

Numerous variables contribute to the sensitivity and specificity of RATs and, therefore, their applicability in different testing scenarios. Notably, there are significant differences in sensitivity according to viral load (Dinnes et al., 2020). As such, most RATs are intended for use in patients up to 5–7 days after symptom onset to increase the probability of having a sufficiently high viral load for detection. If RATs are used to assess asymptomatic contacts of index cases, time from symptom onset is not available, and the date of infection can only be assumed as the date of contact. The use of a RAT for screening within a low-prevalence population may not be appropriate, as fewer cases with a detectable high viral load are expected within this group, decreasing the test's positive predictive value accordingly (Centers for Disease Control and Prevention, 2020).

While studies that conduct direct head-to-head comparisons benefit from reduced experimental heterogeneity (e.g., PCR assay performance differences), repeat sample extraction is invasive. The ability to conduct these studies is potentially hampered by the high number of screened persons required to detect a sufficient number of positive cases.

To provide clarity on the real-world clinical performance of RATs, we compiled all available manufacturer-independent, prospectively collected clinical data using RATs. Data were from RATs commercially available as of November 20, 2020, and intended for the qualitative detection of SARS-CoV-2 present in the human nasopharynx in individuals suspected of SARS-CoV-2 infection. We aimed to harmonize the data regarding the aforementioned performance-impacting factors, using mathematical methods to ensure that the data are comparable despite varying presentation methods in the publications considered for this analysis.

Materials and methods

Search strategy

We searched MEDLINE®, EMBASE®, BIOSIS™ (ProQuest®), and Derwent Drug File (ProQuest®) for any clinical performance studies using a commercial SARS-CoV-2 RAT for the following search terms: “COVID-19” OR “SARS-CoV-2” OR “2019-nCoV” OR “coronavirus disease 2019” OR “novel coronavirus” OR MESH Entries for Coronaviridae (incl. narrow terms) OR EMTREE Entries for Coronaviridae (incl. narrow terms) OR MESH/EMTREE Entries for “severe acute respiratory syndrome” (incl. narrow terms) AND “rapid antigen test*” OR “rapid antigen assay*” OR “standard Q covid-19 ag” AND “sensitivity” OR “specificity” OR “clinical performance” OR “positive agreement” OR “negative agreement” OR “concordance” OR “validation” OR “evaluation” OR “accuracy.”

Secondly, we searched for studies listed on relevant diagnostic databases and/or websites, including the FIND website, which collates new SARS-CoV-2 test developments and manufacturer-independent validation studies (The Foundation for Innovative New Diagnostics, 2020) (only final reports considered), the COVID-19 Diagnostic Devices and Test Method Database (European Commission, 2020), and the Diagnostics Global Health website (Diagnostics Global Health, 2020).

Selection criteria

Eligible studies were those: i) reporting clinical performance data of standalone RATs (i.e., tests that did not require a separate reading device); ii) that measured the performance of RATs against any RT-PCR assay (reference standard; commercial RT-PCR assay or in-house); iii) performed independent of funding by the manufacturer or distributor; iv) that utilized only nasopharyngeal or combined oro-/nasopharyngeal sample types; v) where tests were performed at the point-of-care or at a laboratory after sample transport in viral transport media; vi) that provided information on cycle threshold (Ct) values or symptom status. We excluded retrospective laboratory studies.

Data analysis

The data were extracted to an electronic database and stratified according to RAT. Information on the test utilized, number of participants, percentage of symptomatic patients, specimens (number of PCR positives), and clinical performance (stratified by Ct if available) were recorded for each study.

Performance results of the RATs were reported as sensitivity and specificity measured against the reference standard of RT-PCR and summarized in tables. As confidence intervals reported in the publications used different methods, all confidence intervals were recalculated using the exact Clopper–Pearson method for better comparability. Due to the heterogeneity in sub-groups and the small number of studies for some RATs, we report the differences between tests descriptively rather than statistically.

The meta-analysis of the performance results of the RATs against the RT-PCR methods was performed using the statistical software R (R Foundation for Statistical Computing, 2020). The metaprop function from the “meta” package (Schwarzer, 2020) was used to calculate the effect size for each individual test and pooled overall. The results of the AAZ-LMB and RapiGEN RATs were included, despite only one study being available for each of the tests. The results are shown as a forest plot summarizing the sensitivities found in the different studies.

The bivariate model (Reitsma et al., 2005) was fitted as a linear mixed model, and variance components were estimated by restricted maximum likelihood, using the reitsma function from the “mada” package (Doebler, 2017) for each system investigated in more than one study. The results are presented as a summary receiver operating characteristic (SROC) curve plot (Rutter and Gatsonis, 2001) showing the results of all systems (including those investigated in only one study). The single studies, summary estimates, SROC curves, and confidence regions are depicted.

The relationship between sensitivity and viral load represented by the Ct value is visualized in a scatterplot. The single study results for sensitivity below a certain Ct threshold are plotted against these Ct values and categorized by the different RATs. If, in a single study, sensitivity estimates for more than one Ct value were available, those results are connected by a line.

Results

According to our search criteria, a total of 59 publications were initially selected, of which 19 studies (ten peer-reviewed and nine pre-prints) were included (Figure 1 ).

Figure 1.

Flowchart of search results.^a

^aNo publications were identified from the COVID-19 Diagnostic Devices and Test Method Database (European Commission, 2020).

Ct, cycle threshold.

Included studies were found to investigate five different RATs: i) the SARS-CoV-2 Rapid Antigen Test from Roche Diagnostics, equivalent to the STANDARD Q COVID-19 Ag Test by SD Biosensor (henceforth called “Roche/SDB”); ii) the Panbio™ COVID-19 Ag Test by Abbott (henceforth called “Abbott”); iii) the COVID-19 Ag Respi-Strip® by Coris BioConcept (henceforth called “Coris”); iv) the COVID-Viro® by AAZ-LMB (henceforth called “AAZ-LMB”); v) the BIOCREDIT COVID-19 Ag by RapiGEN (henceforth called “RapiGEN”).

Clinical performance of the RATs

The 19 clinical studies provided data on 11,109 samples, including 2,509 samples with confirmed SARS-CoV-2 by RT-PCR; see Table 1, Table 2, Table 3, Table 4 .

Table 1.

Study population characteristics and performance: SARS-CoV-2 Rapid Antigen Test/STANDARD Q COVID-19 Ag Test (Roche Diagnostics/SD Biosensor) only.

Study	Description					Specimen Ct values		Antigen test performance
	Country	Participants (N)	PCR+ (n)	Prevalence (%)	Symptomatic (%)	Min-max	Mean/median	Specificity (95% CI)a	Sensitivity (95% CI)a	Sensitivity by Ct (95% CI)a
Cerutti et al. (2020)	Italy	330	109	33	56.1 (overall); 95.4 (PCR+)	12.3–38.1		100 (98.3–100)	70.6 ( 61.2–79.0)	Ct <28: 100; Ct 28–30: 38.5; Ct 30–35: 26.7; Ct >35: 9.1
						Dual-target and multiplex assays – target gene not reported
Chaimayo et al. (2020)	Thailand	454	60	13.2	95.0 (PCR+)	10.5–39.0	E-gene: Mean 22.8 ± 6.7/Median 23.4 RdRP-gene: Mean 24.7 ± 6.6/ Median 24.75 N-gene: Mean 26.1 ± 6.5/ Median 26.3	98.7 (97.1–99.5)	98.3 ( 91.1–99.7)	Ct ≤31: 100
						Multiplex assay – all genes reported
Iglὁi et al. (2020)	The Netherlands	970	186	19.2	91.3 (overall)	15.6–37.4	Median 23.6	99.5 (98.7–99.9)	84.9 ( 79.0–89.8)	Ct ≤25: 99.1 (95.2–100); Ct ≤30: 94.3 (89.6–97.0)
						Dual-target – E-gene reported
Krüttgen et al. (2020)	Germany	150	75	50	Not reported	<20–≥35		96.0 (88.8–99.2)	70.7 ( 59.0–80.6)	Ct <25: 100; Ct 25–<30: 95.0; Ct 30–<35: 44.8; Ct ≥35: 22.2
						Dual-target – target gene not reported
Lindner et al. (2020)	Germany	289	39	13.5	97.6 (overall)	17.3–35.5	Mean 23.7 ± 5.5/ Median 21.9	99.6 (97.8–100)	79.5 ( 63.5–90.7)	Ct ≤24: 100 (85.2–100); Ct ≤25.3: 96.2 (80.4–99.9); Ct ≤29.6: 90.3 (74.2–98.0); Ct ≤32: 88.6 (73.3–96.8)
						Single- and dual-target assays – Ct values stated here are for E-gene
Nalumansi et al. (2020)	Uganda	262	90	34.4	14 (PCR+)	≤29–39		92.4 (87.4–95.9)	70 (59.4–79.2)	Ct ≤29: 91.9% (78.1–98.3); Ct 30–37: 54.5%; Ct 38–39: 55.6%
						Custom oligos – target gene not reported

CI, confidence interval; Ct, cycle threshold; oligos, oligonucleotide; ±, standard deviation.

^aValues were recalculated using the original data, and confidence intervals were calculated using the exact Clopper–Pearson method.

Table 2.

Study population characteristics and performance – Panbio™ COVID-19 Ag Test (Abbott) only.

Study	Description					Specimen Ct values		Antigen test performance
	Country	Participants (N)	PCR+ (n)	Prevalence (%)	Symptomatic (%)	Min–max	Mean/median	Specificity (95% CI)a	Sensitivity ( 95% CI)a	Sensitivity by Ct (95% CI)a
Albert et al. (2020)	Spain	412	54	13.1	100 (overall)	<10–>30		100 (99.0–100)	79.6 (66.5–89.4)	Ct <25: 100
						Multiplex assay – target gene not reported
Bulilete et al. (2020)	Spain	1369	140	10.2	49.7 (overall); 62.1% (PCR+)	<25–>30	Mean 20.3 ± 6.5 (N-gene); Mean 21.9 ± 6.5 (S-gene); Mean 21.0 ± 6.7 (ORF-gene)	99.8 (99.4–100)	71.4 (63.2–78.7)
						Multiplex assay – all genes reported
Drevinek et al. (2020)	Czech Republic	591	223	37.7	49.1 (overall); 75.3 (PCR+)	<10–≥35		100 (99.0–100)	66.4 (59.8–72.5)	Ct <20: 92.2 (81.1–97.8); Ct <25: 92.6 (86.3–96.5); Ct <30: 87.0 (80.8–91.7); Ct ≤35: 77.9 (71.3–83.6)
						Multiplex assay – lowest Ct value reported
Fenollar et al. (2020)	France	341	204	59.8	53.4 (overall)	<10–34		94.9 (89.8–97.9)	75.5 (69.0–81.2)	Ct <10: 100; Ct <15: 95.2; Ct <20: 98.3; Ct <25: 96.4; Ct <30: 89.0
						Dual-target – N-gene
Gremmels et al. (2020)	The Netherlands	1367	139	10.2	97.3 (overall)		Mean 27.5 ± 6.0 (N-gene); Mean 24.7 ± 5.7 (E-gene); Mean 26.4 ± 5.6 (RdRp-gene)	100 (99.7–100)	72.7 (64.5–79.9)	Ct <32: 95.3 (89.3–98.5)
	Aruba	208	63	30.3			Mean 25.69 ± 5.96 ( E-gene); Mean 26.56 ± 6.41 (N-gene); Mean 26.26 ± 6.36 (RdRP-gene)	100 (97.5–100)	81.0 (69.1–89.8)	Ct <32: 98.0 (89.1–99.9)
						Multiplex assay – all targets reported
Linares et al. (2020)	Spain	255	60	23.5	72.2 (overall)	(0–37.8)	Median 23.3	100 (98.1–100)	73.3 (60.3–83.9)	Ct <25: 97.1 (84.7–99.9); Ct 25–30: 77.8; Ct 30–35: 30.0; Ct 35–40: 14.0
						Multiplex assay – Ct values stated here are for N gene (other targets unavailable)

CI, confidence interval; Ct, cycle threshold; ±, standard deviation.

^aValues were recalculated using the original data, and confidence intervals were calculated using the exact Clopper–Pearson method.

Table 3.

Study population characteristics and performance – COVID-19 Ag Respi-Strip (Coris BioConcept) only.

Study	Description					Specimen Ct values		Antigen test performance
	Country	Participants (N)	PCR+ (n)	Prevalence (%)	Symptomatic (%)	Min-max	Mean/median	Specificity (95% CI)a	Sensitivity (95% CI)a	Sensitivity by Ct (95% CI)a
Lambert-Niclot et al. (2020)	France	138	94	68.1	Not reported	≤10–>40		100 (92.0–100)	50.0 (39.5–60.5)	Ct <25: 82.2
						Multiplex and 3x dual-target assays – E gene reported
Scohy et al. (2020)	Belgium	148	106	71.6	88.5 (overall)	16–38	Mean 31.4; Median 33	100 (91.6–100)	30.2 (21.7–39.9)	Ct <25: 100; Ct <30: 70.6; Ct <35: 46.9
						Single target – RdRPb
Veyrenche et al. (2020)	France	45	45	NAb	100	<15–>40		NAc	28.9 (16.4–44.3)	Ct ≤25: 86.7 (59.5–98.3); Ct >25: 0.0 ( 0.0–11.6)
						Multiplex assay – average value of all genes

CI, confidence interval; Ct, cycle threshold; NA, not applicable; ±, standard deviation.

^aValues were recalculated using the original data, and confidence intervals were calculated using the exact Clopper–Pearson method.

^bTarget gene described in the manuscript (RdRP) discordant with manufacturer-described target genes (ORF1a/b).

^cOnly positive clinical samples.

Table 4.

Study population characteristics and performance – studies assessing multiple antigen tests.

Study	Description							Specimen Ct values				Antigen test performance
	Country	Antigen test	Participants (N)	Participants per test (n)	PCR+ (n)	Prevalence (%)	Symptomatic (%)	Min–max		Mean/median		Specificity (95% CI)b	Sensitivity (95% CI)b	Sensitivity by Ct (95% CI)b
Berger et al. (2020)	Switzerland	Abbott	1064	535	124	23.2	97.8 (PCR+)	14.2–39.7	14.2–39.7	Mean 22.4 ± 5.4;	Mean 22.5 ± 5.1;	100 (99.1–100)	85.5 (78.0–91.2)
										Median 21.8	Median 21.5
		Roche/SDB		529	191	36.1		14.4–37.4		Mean 22.6 ± 4.9;		99.7 (98.3–99.9)	89.0 (83.7–93.1)
										Median 21.0
								Dual target – E gene reported
Khairat et al. (2020)	Egypt	RapiGEN	100	100	80	75				Median 18.57		45 (23.1–68.5)	52.5 (41.0–63.8)	Ct < 18.57: 60.0 (43.3–75.1);
														Ct > 18.57: 45.0 (29.3–61.5)
		Roche/SDB										95 (75.1–99.9)	68.8 (57.4–78.7)	Ct < 18.57: 77.5 (61.5–89.2);
														Ct > 18.57: 60.0 (43.3–75.1)
								Assay unreported
Krüger et al. (2020)	Germany + UK	Coris	1688a	1263	8	1.9	68.9 (overall)					95.8 (93.4–97.6)	50.0 (15.7–84.3)	Ct < 25:66.7 (9.4–99.2);
														Ct ≥ 25: 40.0 (5.3–85.3)
		Roche/SDB		425	47	3.7	84.4 (overall)					99.3 (98.6–99.7)	76.6 (62.0–87.7)	Ct < 25: 100 (81.5–100);
														Ct ≥ 25: 62.1 (42.3–79.3)
								2x single, 2x dual, and 1x multiplex assays – E gene reported for German subset
Schwob et al. (2020)	Switzerland	Abbott	928	271	122	45.0	96 (overall)	(Results per viral load available)				100 (97.6–100)	86.1 (78.6–91.7)
		AAZ-LMB		324	138	42.6						100 (98.0–100)	84.1 (76.9–89.7)
		Roche/SDB		333	112	33.6						100 (98.3–100)	92.9 (86.4–96.9)
								Single- and dual-target assay – target gene not reported

CI, confidence interval; Ct, cycle threshold; NA, not applicable; ±, standard deviation.

^aTotal sample number of the study was 2,417, but one RAT (BIOEASY™ 2019-Novel Coronavirus [2019-nCoV] Fluorescence Ag Rapid Test) was excluded from our analysis for not fulfilling inclusion criteria (this test needs a device for readout).

^bValues were recalculated using the original data, and confidence intervals were calculated using the exact Clopper–Pearson method.

The sensitivity of the investigated RATs ranged between 28.9% (95% CI 16.4–44.3) and 98.3% (95% CI 91.1–99.7). Specificity ranged between 92.4% and 100%, with one outlier (45%) (Khairat et al., 2020). The two RATs with the most comprehensive available database of more than eight studies, the Roche/SDB and Abbott, reported a specificity of ≥97% in the majority of the trials. The Coris RAT ranged between 95.8% and 100% for specificity, but this was combined with unacceptably low sensitivity. The AAZ-LMB RAT showed very good results, with a specificity of 100% and sensitivity of 84.1%, but was only evaluated in a single study (Schwob et al., 2020). RapiGEN showed an unacceptably low specificity of 45% combined with low sensitivity in the only published study (Khairat et al., 2020).

Meta-analysis

We undertook a statistical pooling of estimates across the 19 studies. Pooled sensitivities for each test with more than one study ranged between 37.7% (95% CI 27.8–48.7) and 82.4% (95% CI 74.2–88.4) (Figure 2 ). There was substantial heterogeneity across estimates of all the RATs, with I ² values ranging from 59% to 88%.

Figure 2.

Forest plot of studies evaluating rapid antigen test sensitivity, grouped by test manufacturer/distributor.

CI, confidence interval.

SROC analyses for all RATs

SROC curves, summary estimates, and confidence intervals were generated for RATs with multiple data points. Of these, the RATs by Abbott and Roche/SDB had overlapping confidence intervals, showing comparable performance; summary estimates were in the region of 80% (Figure 3 ).

Figure 3.

SROC plots for all rapid antigen tests.^a

^aStudy result for RapiGEN lies outside the plotting region (x = 0.55, y = 0.525) and is therefore not shown.

CI, confidence interval; Coris, Coris BioConcept; SDB, SD biosensor; SROC, summary receiver operating characteristic; Roche, Roche Diagnostics.

RAT sensitivity stratified by Ct value

As expected, all RATs performed better in samples with high viral loads, but sensitivity dropped more rapidly at Ct >20 for the Coris test and less rapidly for the Roche/SDB and Abbott tests (Figure 4 ). All tests showed a lower sensitivity at Ct >30–32 and variable accuracy at Ct 25–30.

Figure 4.

Rapid antigen test sensitivity according to viral load (Ct value).^a

^aWhere available, sensitivity estimates derived from a single study are connected by a line.

Coris, Coris BioConcept; Ct, cycle threshold; SDB, SD biosensor; Roche, Roche Diagnostics.

Characteristics of the included studies

Summaries of the studies are provided in Table 1, Table 2, Table 3, Table 4. All studies provided descriptions of the study populations regarding mean age and gender distribution (data not shown) and symptoms, prevalence rates, and Ct of the RT-PCR-positive samples.

Symptomatic or asymptomatic patients

Local definitions of “patients suspected of SARS-CoV-2 infection” either included i) only patients presenting clinical symptoms or ii) asymptomatic persons with recent direct contact with suspected or confirmed cases. Some asymptomatic contact case groups were limited to persons with high-risk contact with the severely ill (Krüger et al., 2020), contact with a case, or high-risk exposure in a cluster (Berger et al., 2020), but others included travelers returning from high-risk areas (Cerutti et al., 2020, Krüger et al., 2020), which meant that this population varied in terms of pre-test probability. The percentage of symptomatic samples varied widely, ranging between 14.0% (Nalumansi et al., 2020) and 97.8% (Berger et al., 2020). Not every report clearly stated the ratio between symptomatic RT-PCR-positive and symptomatic RT-PCR-negative samples. This ratio seemed to differ considerably; some studies reported >90% symptomatic persons gaining approximately 15% RT-PCR-positive samples, and the study which investigated only 14% symptomatic persons tested 34.4% RT-PCR-positive samples (Nalumansi et al., 2020). No study provided a direct comparison of RAT results between asymptomatic and symptomatic patients.

Prevalence

Prevalence rate – here, meaning the number of RT-PCR-positive samples within the study population – varied between 1.9–100%. The prevalence of SARS-CoV-2 in some of these studies did not reflect the prevalence in the local populations, as additional pre-specified testing criteria qualified patients for study entry, thus creating a preselection bias.

Characterization of viral RNA

The mean Ct value was reported in six studies ranging between 20.3 (Bulilete et al., 2020) and 31.4 (Scohy et al., 2020). The median Ct value was reported by seven studies and ranged between 18.57 (Khairat et al., 2020) and 33 (Scohy et al., 2020). The definition of a high viral load varied considerably within the trials, from Ct <18.57 in an Egyptian study (Khairat et al., 2020) to ≤37 in a Ugandan trial (Nalumansi et al., 2020). One study did not report Ct values but reported RNA copies/mL (Schwob et al., 2020). Notably, the threshold for negativity (Ct >38 or >40) varied between the studies.

The Ct values, which were summarized and analyzed by group, also differed considerably. A majority, but not all, reported data at Ct thresholds close to Ct ≤20, ≤25, ≤30, and/or ≤35.

Analytical parameters

The cobas® SARS-CoV-2 Assay by Roche and the Allplex™ 2019-nCoV Assay by Seegene were the most frequently used RT-PCR assays (Supplemental Table 1), but even within a single study, up to five different assays were used (Krüger et al., 2020). The most commonly reported target was the envelope gene (pan-Sarbecovirus- and SARS-CoV-2-specific); other targets included the nucleocapsid, RNA-dependent RNA polymerase, spike, and ORF1a/b genes. Where dual-target or multiplex assays were used, the target gene used to report the Ct value was frequently not stated (Supplemental Table 1; Table 1, Table 2, Table 3, Table 4).

Discussion

This review presents an overview of manufacturer-independent commercial SARS-CoV-2 RATs not requiring a reading instrument. Altogether, 19 studies investigating five different RATs presented detailed population characteristics and Ct values. Only three commercial SARS-CoV-2 RATs have been assessed in multiple independent real-world studies, and of these, only the Roche/SDB and Abbott tests had adequate levels of performance; their summary estimates were in the region of 80%, exceeding or approaching health authorities’ requirements for sensitivity ≥80% (European Centre for Disease Prevention and Control, 2020, World Health Organization, 2020a). The two RATs with the most comprehensive available database of more than eight studies, Roche/SDB and Abbott, reported a specificity of ≥97% in the majority of the trials, meeting specificity requirements (European Centre for Disease Prevention and Control, 2020, World Health Organization, 2020a).

Critical appraisal of factors influencing the performance of RATs

One major concern to be highlighted when comparing performance data of RATs originating from different performance studies is that data presented in different trials must not directly be compared with each other as numerous variables impact the resulting performance values. A direct comparison can only be made within one trial, when performing a head-to-head comparison of different tests within an identical setting, or when using several studies such that effects average out. Several variables have considerable impact on RAT clinical performance and prohibit any direct comparison:

• 1
  Preanalytical influencers

  • a
    Sample type and way of sampling (same or opposite nostril, combined oro-/nasopharyngeal vs. nasopharyngeal sample only, order of sampling)
  • b
    Collection device, transport media, and volume versus direct testing without dilution by transport media
  • c
    Time to test and storage/transport conditions, the time delay before processing
• 2
  Analytical influencers

  • a
    Viral load of the sample and viral load distribution in the respective cohort, represented by Ct or RNA copies/mL
  • b
    Analytical sensitivity and specificity of the PCR reference standard
  • c
    PCR assay specifics, such as different target genes (E-/RdRp-/N-gene, etc.)
  • d
    Across-laboratory differences (e.g., the definition of a positive sample starting at Ct <38 or <40)
• 3
  Clinical parameters of the tested subject

  • a
    Days post symptom onset of sampling
  • b
    Asymptomatic vs. symptomatic status, the definition of symptoms “suspective of SARS-Cov-2 infection”
  • c
    Severity of symptom presentation

It must be noted that symptom classifications considerably differed between the studies. One study even investigated different populations for self-reported versus physician-defined symptoms (Lindner et al., 2020). A uniform definition of “clinical symptoms suggestive for SARS-CoV-2 infection” would be desirable but is currently unavailable.

Aside from viral load, none of the 19 included studies reported sufficient detail to allow high confidence formal analysis of the effect of these variables on the performance of the RATs evaluated. Additionally, the lack of standardization (e.g., variable cut-offs and study designs), and the low number of positive samples in the individual studies, precluded such analysis.

Moreover, the included studies exhibited high heterogeneity. This heterogeneity was likely attributable to differences in the patient population between studies and other influencing factors, as mentioned above.

Due to methodological reasons, the detection limit for SARS-CoV-2 RNA from clinical samples tested by RT-PCR is always lower than the detection limit for SARS-CoV-2 antigen. The detectability of even the best performing RAT deteriorates with decreasing viral load; however, RATs still have utility in this context. Cell culture studies have shown that the probability for positive viral cell culture (a surrogate of viral transmission and infectivity) is lower at low viral load/high Ct (Bullard et al., 2020, van Beek et al., 2020). This translates into very limited to no infectiousness of the infected patients, even if RT-PCR may still show positive signals for up to three more weeks after peak Ct value (Magleby et al., 2020, Wölfel et al., 2020).

Additionally, a high positive predictive value requires testing at a high pre-test probability setting; a high negative predictive value in a low pre-test probability setting can help to safely rule out infectious or high-viral-load individuals (Peeling et al., 2021). The use of RATs requires careful preselection and confirmation of recent contact to confirmed cases and/or knowledge about the underlying local population prevalence. If RATs are used to screen asymptomatic cases in low-prevalence scenarios, a lower positive predictive value of the according result has to be considered.

Conclusion

Based on a systematic review and meta-analysis of RAT data published until November 2020, only three RATs had been assessed in multiple real-world manufacturer-independent studies. Of these, the Roche/SDB and Abbott RATs had adequate performance levels and provide the strongest evidence base to recommend their use for the detection of current SARS-CoV-2 infection, particularly in high-viral-load patient populations.

Ethical approval

This study did not require ethical approval because the meta-analysis was based on published research.

Competing interests

Johannes Hayer and Dusanka Kasapic are employees of Roche Diagnostics. Claudia Zemmrich works as a freelance contractor for Roche Diagnostics.

Data availability

The data supporting this meta-analysis are from previously reported studies and datasets that have been cited. The processed data are available from the corresponding author upon request.

Funding

This work was supported by Roche Diagnostics.

Appendix A. Supplementary data

The following is Supplementary data to this article: