Performance of point-of care molecular and antigen-based tests for SARS-CoV-2: a living systematic review and meta-analysis

Abstract

Background

Molecular and antigen point-of-care tests (POCTs) have augmented our ability to rapidly identify and manage SARS-CoV-2 infection. However, their clinical performance varies among individual studies.

Objectives

The evaluation of the performance of molecular and antigen-based POCTs in confirmed, suspected, or probable COVID-19 cases compared with that of laboratory-based RT-PCR in real-life settings.

Data sources

MEDLINE/PubMed, Scopus, Embase, Web of Science, Cochrane Library, Cochrane COVID-19 study register, and COVID-19 Living Evidence Database from the University of Bern.

Study eligibility criteria

Peer-reviewed or preprint observational studies or randomized controlled trials that evaluated any type of commercially available antigen and/or molecular POCTs for SARS-CoV-2, including multiplex PCR panels, approved by the United States Food and Drug Administration, with Emergency Use Authorization, and/or marked with Conformitè Europëenne from European Commission/European Union.

Participants

Close contacts and/or patients with symptomatic and/or asymptomatic confirmed, suspected, or probable COVID-19 infection of any age.

Test/s

Molecular and/or antigen-based SARS-CoV-2 POCTs.

Reference standard

Laboratory-based SARS-CoV-2 RT-PCR.

Assessment of risk of bias

Eligible studies were subjected to quality-control and risk-of-bias assessment using the Quality Assessment of Diagnostic Accuracy Studies 2 tool.

Methods of data synthesis

Summary sensitivities and specificities with their 95% CIs were estimated using a bivariate model. Subgroup analysis was performed when at least three studies informed the outcome.

Results

A total of 123 eligible publications (97 and 26 studies assessing antigen-based and molecular POCTs, respectively) were retrieved from 4674 initial records. The pooled sensitivity and specificity for 13 molecular-based POCTs were 92.8% (95% CI, 88.9–95.4%) and 97.6% (95% CI, 96.6–98.3%), respectively. The sensitivity of antigen-based POCTs pooled from 138 individual evaluations was considerably lower than that of molecular POCTs; the pooled sensitivity and specificity rates were 70.6% (95% CI, 67.2–73.8%) and 98.9% (95% CI, 98.5–99.2%), respectively.

Discussion

Further studies are needed to evaluate the performance of molecular and antigen-based POCTs in underrepresented patient subgroups and different respiratory samples.

Introduction

The efforts to contain the COVID-19 pandemic have been facilitated by the expeditious development of antigen and molecular rapid diagnostic tests (RDTs) that complement—or in some instances substitute—the laboratory-based nucleic acid amplification test (NAAT), which is currently the reference standard diagnostic tool. Point-of-care tests (POCTs) have augmented the ability of healthcare systems to rapidly identify and manage SARS-CoV-2 cases by applying large-scale testing and reaching remote areas where laboratory-based NAATs would not be feasible [1]. Alongside their undeniable advantages, including ease of use, low cost, portability, rapid results, and self-testing, concerns still exist regarding their diagnostic accuracy in terms of sensitivity and specificity.

WHO endorses the use of antigen RDTs in symptomatic cases that meet the definition of suspected COVID-19, in asymptomatic individuals who have been exposed to SARS-CoV-2, for the detection of suspected outbreaks of COVID-19, and for screening purposes when a prevalence of >5% is suspected [2]. More recently, WHO also recommended self-testing with RDTs as a reliable and feasible alternative to professionally administered testing [3]. For RDTs, WHO has set the minimum performance requirements of ≥80% sensitivity and ≥97% specificity regarding symptomatic individuals.

This systematic review and meta-analysis contributes to the efforts made by other scientific groups [4,5] to synthesize available data and provide an updated, comprehensive, and detailed account of the overall performance of RDTs. Additionally, the inclusion of molecular POCTs in addition to antigen-based POCTs, the application of stringent criteria for included studies focusing on high-quality real-world data, and the thorough analysis of potential contributing factors that may influence the accuracy of POCTs render this study a valued addition to the available literature and a groundwork for future elaborations based on emerging evidence. Therefore, the scope of this systematic review and meta-analysis is to add an important piece of information to the current literature regarding the performance of commercially available point-of-care antigen and molecular tests that will be updated with new evidence every 18–24 months.

Methods

Search strategy and data sources

This meta-analysis was conducted in accordance with the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) reporting guidelines 2020 [6] and the PRISMA diagnostic test accuracy guideline, which is a stand-alone extension of the PRISMA statement [7].

A systematic literature search was conducted between 1 November 2019 and 14 June 2021 in MEDLINE/PubMed, Scopus, Embase, Web of Science, Cochrane Library, Cochrane COVID-19 study register, and COVID-19 Living Evidence Database from the University of Bern. The main search terms applied to the query of all databases included: ‘point-of-care test’, ‘rapid test’, ‘molecular’, ‘PCR’, ‘antigen-based’, ‘fluorescent immunoassay’, ‘lateral flow’, ‘NAAT’, ‘LAMP’, ‘COVID-19’, and ‘SARS-CoV-2’ or ‘coronavirus’. The respective algorithms for MEDLINE/PubMed, Scopus, and Web of Science searches are available in Appendix 1. Bibliographic database interrogation was performed by two independent teams consisting of two researchers each (PPN and HJ; CDM and DD). All observational studies (prospective or retrospective) that provided data on the performance of molecular and/or antigen-based POCTs compared with that of laboratory-based RT-PCR for the detection of SARS-CoV-2 were identified.

Definitions used in this meta-analysis

The definitions used in this systematic review are provided in Table S1. Case definitions for COVID-19 in particular were selected on the basis of the current definitions used by WHO for surveillance purposes [8,9]. We decided to use the same definitions used by international societies and other stakeholders because this would harmonize our results with future decision-making processes.

Study eligibility criteria

Study eligibility and exclusion criteria are described in Table 1 .

Table 1.

Study eligibility and exclusion criteria as well as the main and secondary outcomes of this systematic review

Eligibility criteria	Exclusion criteria	Outcomes
(1) Studies reported data on the outcomes of interest (2) Studies included close contacts and/or patients with symptomatic and/or asymptomatic confirmed, suspected or probable COVID-19 infection of any age (3) Studies that evaluated any type of FDA approved and/or with EUA and/or CE-marked from EC/EU commercially available antigen and/or molecular POCTs for SARS-CoV-2, including multiplex PCR panels (4) The index and reference standard tests were performed in any type of respiratory samples (5) Both peer-reviewed studies and preprints were included in the analysis	(1) Studies utilizing only serology tests or non-respiratory samples (2) Studies reporting interventions not aligned with our pre-defined inclusion criteria, including tests not fulfilling the definition of POCT, or using in-house or non-commercial assays or commercial non-FDA approved or with non-EUA or CE-marked index test (3) Studies using as a reference standard a non-laboratory-based RT-PCR (4) Case reports, case series, book chapters, conference abstracts (5) Studies with non-available full text (6) Analytical accuracy studies (7) Studies with non-extractable data for individual index tests (8) Studies with only sensitivity data	(1) Evaluation of the performance of molecular-based and antigen-based POCTs in confirmed or suspected or probable COVID-19 patients compared to laboratory-based RT-PCR (2) Evaluation of the performance of molecular-based and antigen-based POCTs in asymptomatic COVID-19 patients or close contacts compared to laboratory-based RT-PCR (3) Evaluation of the performance of molecular-based POCTs in COVID-19 patients compared to antigen-based POCTs (4) The measures of effect were sensitivity and specificity, positive percent agreement and negative percent agreement (5) Subgroup analyses by stratifying our data according to: a. the age of participants (paediatric, adult and mixed population) b. the Ct value of the reference test (Ct<35, Ct<30, Ct<25 and Ct<20) c. the days since the onset of symptoms (≤7 and >7 days of symptoms) d. the way of sample acquisition (self-sampling or self-testing versus testing or sampling by a health professional / trained personnel) e. the different types of biological samples (nasal, nasopharyngeal, oropharyngeal, saliva and combination of them)

CE, Conformitè Europëenne; Ct, cycle threshold; EC, European Commission; EU, European Union; EUA, Emergency Use Authorization; FDA, United States Food and Drug Administration; POCT, point-of-care test.

Participants

Adults and children of any age with symptomatic or asymptomatic, confirmed or probable, or suspected COVID-19 and healthy subjects and/or close contacts of individuals with COVID-19 were included.

Index tests

We included studies evaluating molecular and/or antigen-based POCTs for the diagnosis of SARS-CoV-2 infection. The threshold for test positivity was inherent to the device, and it was defined as per the manufacturer.

Reference standard

We considered laboratory-based RT-PCR as an acceptable reference standard for the purpose of this systematic review [10].

Outcomes

The main and secondary outcomes of this study are shown in Table 1.

Data extraction

Two independent teams, each one consisting of two reviewers (DD and VAS; PPN and HJ), searched the databases. The two teams screened the retrieved articles for eligibility according to the pre-defined criteria. The studies that were selected for inclusion by the two reviewer teams were compared to ensure consistency. Any disagreements were resolved by a senior reviewer (PCF and CDM). Data extraction was performed by two independent teams comprising three reviewers each (CDM, DD, and KD; KAP, MK, and DSYO) after ensuring the coherence of the process among the two teams by simultaneous data extraction training.

Extracted data included the following: study information (author's last name, publication date, sample size, number of patients, and country), baseline characteristics of the study cohort (age, ethnicity/race, sex, and COVID-19 status [case, close contact, or healthy subject]), and data regarding the index test, such as method/assay/manufacturer, time from sample acquisition to result, hands-on time, turn-around time, approval status (United States Food and Drug Administration [FDA] Emergency Use Authorization/ Clinical Laboratory Improvement Amendments (CLIA) waived/Conformitè Europëenne (CE)–marked), type of biological samples, type of setting where the test(s) was performed (community testing sites, primary care/outpatient setting, emergency room, hospital [not otherwise specified], other or multiple sites, or non-specified setting) and all data related to the outcomes of interest. Because the prevalence of COVID-19 in each area during the time of sample acquisition was impossible to be retrieved, we extracted and analysed data on the intra-study prevalence. Potential disagreements in data extraction were resolved by a senior author (CS).

Study quality control and risk-of-bias assessment

Eligible studies were subjected to quality-control and risk-of-bias assessment using the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool within Review Manager, version 5.4 [11,12]. Regarding applicability, any test that was truly a POCT and was performed on human respiratory samples was considered applicable. Therefore, our applicable patient population has been widely defined, including every individual tested with both a POCT and NAAT, including adults and children with symptomatic or asymptomatic, confirmed or probable, or suspected COVID-19 and healthy subjects and/or close contacts of individuals with COVID-19. Regarding the domain of timing, the ideal interval between index and reference test was pre-defined as <48 hours. Quality-control and risk-of-bias assessment was conducted independently by two reviewers (CDM and DD), and disagreements were resolved through discussion with a senior author (PCF and CS).

Linear regression of log ORs on the inverse root of effective sample size test was used to assess the presence of publication bias [13,14]. Publication bias was strongly suspected if the Deek's funnel plot asymmetry test resulted in p value of <0.10.

The certainty of the evidence at the synthesis level was assessed using the Grading of Recommendations Assessment, Development and Evaluation framework (Appendix 1) [15,16].

Statistical analysis and data synthesis

Summary sensitivities and specificities with their 95% CIs were estimated using a bivariate model. Subgroup analysis was performed when at least two studies informed the outcome.

All analyses were conducted in R v4.2.1 [17]. The summary sensitivities and specificities with their 95% CIs were estimated using the bivariate model [18] as implemented in the mada v5.11 package [19] in the function reitsma. Sensitivity-only analyses for the different cycle thresholds (Ct) were conducted with the v5.5-0 package [20] using the metaprop function. The 95% CIs for sensitivity and specificity for the individual studies were recalculated using the Clopper-Pearson procedure [21]. Finally, Deek's funnel plot asymmetry test was performed using the midas package in Stata v.14 [22].

Subgroup analysis was performed when at least three studies informed the outcome, except for the subgroup analysis of individual commercial tests, in which performance was analysed for all of them irrespective of the number of studies assessing their performance. Differences between two subgroups were tested using a t test for logit-transformed group estimates and variances reported by the individual regression results of the reitsma package. Multiple testing was adjusted for using the Benjamini-Hochberg procedure [23].

Plan for establishing living evidence

We plan to update our results with emerging evidence arising from new studies by following the same search method as described in our protocol every 18–24 months. The reviewers that conducted the initial search will evaluate the new evidence according to the pre-defined inclusion and exclusion criteria and will meta-analyse the new data according to our pre-defined methods.

Results

Literature search and included studies

In total, 109 eligible studies were retrieved via database search (Fig. 1 ). Through manual search of reference lists of previously published systematic reviews and meta-analyses as well as free internet searches, we retrieved another 14 papers, leading to a total of 123 eligible publications: 97 and 26 studies assessing antigen and molecular POCTs, respectively (Tables S2 and S3).

Fig. 1.

Systematic review flow chart. COAP, COVID-19 Open Access project; EU, European Union; US FDA, United States Food and Drug Administration; POCT, point-of-care test.

Study characteristics and demographics

A summary of the characteristics of the eligible studies assessing molecular and antigen-based POCTs is presented in Table 2 . The total included population comprised 112 304 subjects, 27 332 males and 30 095 females among the studies that reported participants' sex. Most eligible studies included adult participants (n = 122); five studies reported the performance of antigen POCTs in a paediatric population (aged <18 years) separately. Studies were conducted in diverse settings: community testing sites, 14.8%; primary care, 12.6%; emergency room, 9.6%; hospital (not specified) 37%; and other or multiple sites, 14.8%, whereas 11.1% of studies did not report relevant data.

Table 2.

Summary of characteristics of the included studies

	Molecular POCTs	Antigen-based POCTs
Number of studies	26	97
Number of retrospective studies	9	21
Number of prospective studies	17	76
Number of individual evaluations	44	312
Number of individual commercial tests	16	36
Total population	11 675	82 093
Males, % (n/N)	51.2% (387/756a)	47.4% (25 686/54 224a)
Adults, % (n/N)	99.9% (11 673/11 675)	93.1% (76 412/82 093)
Children, % (n/N)	0.01% (2/11 675)	6.9% (5681/82 093)
Number of IFU conforming evaluations	18	58
Number of self-sampling evaluations	1	9
Number of self-testing evaluations	0	2
Number of non–peer-reviewed studies	2	23

IFU, instructions for use; POCT, point-of-care test.

^aThese numbers correspond to the population that had available data.

Molecular POCTs

Among the 123 included studies, 26 (12 from the Americas, 11 from Europe, and three from Asia) reported the performance of 13 different molecular point-of-care index tests; these studies included a total of 8768 subjects (Table S3 and Appendix 1). Some publications reported the assessment of more than one molecular POCT, leading to a total of 33 individual evaluations across all eligible studies. In terms of study design, the majority (n = 20) were cross-sectional, whereas the remaining (n = 6) were case-control studies. Sampling was conducted according to the instructions for use (IFU) from the manufacturer in 18 studies. In the remaining ones, archived samples were used to evaluate the performance of the index tests. None of the articles reported a self-testing process, whereas sample acquisition by self-sampling was reported only in one study. The turn-around time of molecular POCTs ranged from <13 minutes to 60 minutes.

Overall performance

The overall pooled sensitivity and specificity for the 13 molecular POCTs were 92.8% (95% CI, 88.9–95.4%) and 97.6% (95% CI, 96.6–98.3%), respectively (Fig. 2, Fig. 3 ).

Fig. 2.

Forest plot of the overall performance of molecular point-of-care tests for SARS-CoV-2. TP, True positive; FP, False positive; FN, False negative; TN, True negative.

Fig. 3.

Summary receiver operating characteristic (SROC) curves showing the pooled sensitivity and false positive rate by regression over all studies. The 95% CI is indicated by an ellipse. (A) SROC for molecular point-of-care tests. (B) SROC for antigen-based point-of care tests. AUC, Area under the curve; POCT, Point-of-care tests.

Performance according to clinical, epidemiological, and demographic characteristics

The sensitivity and specificity among nine evaluations of molecular POCTs in patients with signs and symptoms compatible with COVID-19 were 93% (95% CI, 85.9–96.7%) and 98.4% (95% CI, 96.3–99.3%), respectively (Fig. 4 ). The performance of point-of-care NAATs varied according to the different epidemiological profiles of the subjects (confirmed cases, suspected cases, and close contacts), as shown in Table 3 .

Fig. 4.

Forest plot of the performance of molecular-based point-of-care tests for SARS-CoV-2 in symptomatic participants. TP, True positive; FP, False positive; FN, False negative; TN, True negative.

Table 3.

Pooled sensitivity and specificity rates of subgroups for either molecular, antigen-based, and both types of point-of-care tests

Subgroup	Number of evaluations	Sensitivity (95% CI)	Specificity (95% CI)
Molecular point-of-care tests
Close contacts	n/a	n/a	n/a
Suspected cases	17	90.9% (85.5–94.4%)	98.8% (97.9–99.3%)
Confirmed cases	7	91.8% (75.7–97.6%)	96.1% (92.9–97.9%).
Mixed close contacts and suspected cases	3	98.1% (91.9–99.6%)	95.5% (90.2–98%)
Test performed ≤7 d from symptom onset	n/a	n/a	n/a
Test performed >7 d from symptom onset	n/a	n/a	n/a
Cross-sectional studies	23	92.9% (88.7–95.6%)	98.2% (97.0–98.9%)
Case-control studies	10	91.8% (81.2–96.7%)	95% (89.7–97.7%)
IFU	23	94.3% (90.4–96.7%)	97.6% (96.3–98.4%)
Non-IFU	9	87.8% (75.0–94.5%)	97.5% (95.7–98.6%)
Sample size ≤100	7	91.0% (76.5–96.9%)	97.4% (93.2–99%)
Sample size >100	26	93.2% (89.1–95.9%)	97.6% (96.4–98.4%)
Self-sampling	n/a	n/a	n/a
NP sample	21	91.6% (85.8–95.2%)	97.7% (96.3–98.6%)
NS sample	3	83.4% (68.8–91.9%)	99.4% (95.6–99.9%)
Antigen-based point-of-care tests
Close contacts	7	65.3% (45.8–80.7%)	99.6% (99.2–99.7%)
Suspected cases	45	74.8% (68.7–80.1%)	98.6% (97.6–99.2%)
Confirmed cases	3	64.2% (51.3–75.4%)	87.1% (42.2–98.4%)
Mixed confirmed and suspected cases	25	76.2% (70.3–81.3%)	99.2% (98.6–99.5%)
Screening of general populationa	14	49.3% (39.7–59.1%)	98.7% (97–99.5%)
Test performed ≤7 d from symptom onset	9	83.1% (77.3–88.7%)	98.2% (90.1–99.7%)
Test performed >7 d from symptom onset	11	46.2% (36–56.6%)	97.0% (93.4–98.6%)
Cross-sectional studies	116	72.0% (68.5–75.3%)	99.2% (98.9–99.3%)
Case-control studies	22	62.9% (52.3–72.4%)	95.0% (89.7–97.7%)
IFU	72	73.1% (68.7–77.1%)	99.2% (98.8–99.4%)
Non-IFU	59	67.7% (61.9–73%)	98.4% (97.5–99%)
Sample size ≤100	14	64.9% (48.6–78.4%)	89.2% (76.4–95.5%)
Sample size >100	124	71.2% (67.8–74.4%)	99.1% (98.8–99.3%)
Self-sampling	9	71.2% (57.9–81.6%)	98.9% (98.5–99.2%)
Non–self-sampling	129	71.2% (57.9–81.6%)	99.3% (98.3–99.7%)
NP samples	84	71.5% (66.9–75.8%)	98.9% (98.4–99.2%)
NP and OP samples	24	71.5% (64.1–77.9%)	99.2% (98.5–99.5%)
NS samples	39	74.5% (68.9–79.3%)	98.4% (97.1–99.1%)
NS and throat samples	6	76.8% (67.4–84.2%)	98.5% (96.9–99.3%)
Ct < 35	16	82% (73–89%)	n/a
Ct < 30	47	88% (86–90%)	n/a
Ct < 25	45	95% (93–96%)	n/a
Ct < 20	18	98% (95–99%)	n/a
Both antigen and molecular point-of-care tests
Community testing sites	26	71% (63–78%)	99% (98–99%)
Primary care/outpatient setting	20	77% (65–86%)	98% (97–99%)
Emergency room	15	72% (63–80%)	99% (98–100%)
Hospital, not otherwise specified	62	78% (72–83%)	99% (98–99%)
Other or multiple sites	29	75% (66–83%)	98% (97–99%)
Non-specified sites	20	77% (70–83%)	99% (98–99%)

Ct, cycle threshold; IFU, instructions for use; n/a, data not available for meta-analysis; NP, nasopharyngeal; NS, nasal; OP, oropharyngeal.

^aIncludes screening of asymptomatic individuals both in the community and in healthcare facilities (e.g. emergency departments and delivery rooms).

Among evaluations assessing only adult participants, the sensitivity and specificity rates of molecular POCTs were 92.2% (95% CI, 88.3–94.8%) and 97.7% (95% CI, 96.8–98.4%), respectively (Fig. S1). No data were available for the paediatric population; additionally, no data were retrieved for the performance of molecular tests according to the timing of testing since the onset of symptoms.

Antigen POCTs

Ninety-seven articles (68 from Europe, 18 from the Americas, seven from Asia, and four from Africa) reported the performance of 40 different antigen-based POCTs in 85 462 individuals (Table S2). Most studies were prospective in nature (n = 76) and of a cross-sectional design (n = 88). As for molecular POCT studies, some articles reported the performance of more than one assays, leading to a total of 138 individual evaluations of antigen-based POCTs across the included studies. Furthermore, nearly 50% of the studies (n = 60) reported the collection and processing of the samples according to the manufacturer's IFU. Finally, self-sampling was reported in nine studies, and only two studies evaluated antigen tests using a self-testing method.

Overall performance

The sensitivity of all antigen-based POCTs pooled from 138 individual evaluations was considerably lower than that for molecular POCTs (Fig. 3, Fig. 5 B); the pooled sensitivity and specificity rates were 70.6% (95% CI, 67.2–73.8%) and 98.9% (95% CI, 98.5–99.2%), respectively.

Fig. 5.

Forest plot of the overall performance of antigen-based point-of-care tests for SARS-CoV-2. TP, True positive; FP, False positive; FN, False negative; TN, True negative.

Performance according to clinical, epidemiological, and demographic characteristics

Regarding antigen-based POCTs, the subgroup analysis according to the clinical status of the participants showed higher sensitivity in symptomatic patients (Fig. 6 A) than in asymptomatic patients (Fig. 6B) and their close contacts (Table 3). The pooled sensitivity and specificity rates of general population screening (testing of asymptomatic individuals both in the community and in healthcare facilities) for suspected and mixed suspected/confirmed COVID-19 cases are shown in Table 3.

Fig. 6.

(A) Forest plot of the overall performance of antigen-based point-of-care tests for SARS-CoV-2 in symptomatic participants. (B) Forest plot of the overall performance of antigen-based point-of-care tests for SARS-CoV-2 in asymptomatic participants. TP, True positive; FP, False positive; FN, False negative; TN, True negative.

In antigen-based POCTs, 154 and seven evaluations assessed their performance in adults and the paediatric population, respectively (Figs S2–S4); the sensitivities and specificities were 72.7% (95% CI, 69.5–75.6%) and 98.7% (95% CI, 98.3–99.0%) for adults and 65.3% (95% CI, 59.9–70.3%) and 98.3% (95% CI, 95.3–99.4%) for children, respectively. The sensitivity and specificity rates of antigen-based POCTs in samples obtained ≤7 and >7 days since symptom onset are shown in Table 3 and Figs S5 and S6.

Performance according to sample size and study design

Subgroup analysis by study design revealed higher sensitivity and specificity for both molecular and antigen-based POCTs in cross-sectional studies than in case-control studies (Table 3 and Figs S7–S10).

Studies including >100 samples demonstrated only marginally higher pooled sensitivity and comparable specificity compared with those with ≤100 samples for molecular and antigen-based POCTs (Table 3 and Figs S11–S14).

Other subgroup analyses

Sample acquisition and processing according to the manufacturers' IFU, self-acquired and non–self-acquired samples, different types of respiratory biomaterials for both molecular and antigen assays, and the Ct values of the reference test (Ct < 35,Ct < 30, Ct < 25 and Ct < 20) are summarized in Table 3 and Figs S15–S30. We also analysed the performance of the POCTs in different settings, as shown in Table 3 and Figs S31–S36. We finally performed a subgroup analysis according to the intra-study prevalence of COVID-19, which did not explain the observed heterogeneity (Figs S37 and S38).

Performance of individual commercial tests

The performance of individual commercial molecular and antigen-based POCTs are shown in Figs S39 and S40.

The best performing molecular POCTs were as follows: Simplexa® COVID-19 Direct kit, Cepheid Xpert® Xpress SARS-CoV-2, cobas ® SARS-CoV-2, AQ-TOP^TM, BioFire® Respiratory Panel 2.1, and SAMBA II Coronavirus SARS-CoV-2 Test system.

From antigen-based POCTs, the best performing assays (with two evaluations each) were the COVID-VIRO ALL IN® and GenBody COVID-19 Ag test.

Quality control of included studies and risk-of-bias assessment

Graphical summary of the risk of bias and the applicability concerns using the QUADAS-2 tool is shown in Fig. S41, and the assessment for each individual study is presented in Fig. S42. Most eligible studies had a low overall risk of bias and applicability concerns.

Publication bias was assessed separately for molecular and antigen-based POCT studies (Figs S43 and S44). A low risk of publication bias was detected for molecular studies. However, antigen-based studies demonstrated a high risk of publication bias, as shown in Fig. S44 (Deek's Funnel plot asymmetry test, p 0.01).

Discussion

Our meta-analysis on the use of a RDT as a POCT included 123 eligible studies, of which 26 and 97 studies assessed molecular and antigen-based POCTs, respectively. The observed differences in sensitivity were largely attributable to the type of RDT, conformity to IFU, and target population for use. Indeed, as expected, molecular RDTs yielded significantly higher sensitivity rates than those yielded by antigen-based RDTs. Antigen-based RDTs that were performed following IFU had a higher but not statistically significant sensitivity than that yielded with non–IFU conforming testing: 73.1% (95% CI, 68.7–77.1%) versus 67.7% (95% CI, 61.9–73.0%). When antigen-based RDTs were used as screening tools in the general population, sensitivity decreased to 49.3% (95% CI, 39.7–59.1%). Similarly, sensitivity decreased to 46.2% (95% CI, 36–56.6%) when the test was performed >7 days since symptom onset. In contrast, specificity rates were high across all subgroup analyses.

Interestingly, we detected a high risk of publication bias for studies assessing antigen-based POCTs. This may be linked to the fact that antigen-based RDTs yield low sensitivity rates in clinical and community practice, especially among individuals with a low pre-test probability, as shown in our meta-analysis; the low performance in turn may negatively affect the intention of researchers to publish their results.

In comparison to the meta-analysis by Dinnes et al. [4], our analysis included a longer time period during the ongoing COVID-19 pandemic and provides an overall update regarding the diagnostic performance of both molecular and antigen-based POCTs. The recently updated meta-analysis by Brümmer et al. [24] included publications on antigen-based POCTs until August 2021, which did not include molecular tests. In sight of the challenge of considerable heterogeneity of studies included in meta-analyses and possible improvement of RDTs in time, we used stricter selection criteria by including only data from US FDA-approved or Emergency Use Authorization–marked and Conformitè Europëenne–marked RDTs in our analysis. This approach has the advantage that most results of first-generation antigen tests were not included, the results of which might have improved after further optimization or retracted for use in clinical practice. When focusing on the subgroup of patients with symptom duration of <7 days, our approach could explain why there is a higher sensitivity of 83% in this study compared with 78% in the meta-analysis by Dinnes et al. [4] that included search results up to 30 September 2020. However, when comparing our results to those of Brümmer et al. [5], which included search results up to 30 April 2021, sensitivity estimations were similar: 83% versus 83% for testing in the first week and overall pooled sensitivity of 71% versus 71%, respectively. Finally, a recent meta-analysis of the performance of antigen RDTs in paediatric population demonstrated comparably low sensitivity rates, similar with our results: 64.2% (95% CI, 57.4–70.5%) versus 65.3% (95% CI, 59.9–70.3%) [25]. These results should be taken into account for future infection control policies in the paediatric population.

The interpretation of our results (including where can these tests be used and what will be the consequences of testing) should be performed in light of several factors, such as the prevalence of the disease in different areas, the policies each country follows, and of course the available resources in each setting/area. Although valid for symptomatic patients with either suspected or probable COVID-19, no data were available to draw conclusions regarding asymptomatic patients with or without a history of exposure risk or close contact. According to our results, in non-laboratory settings, such as COVID-19 areas of emergency room, the use of molecular POCTs might be considered to accurately and rapidly diagnose SARS-CoV-2 in symptomatic patients. In this context, the significant reduction of time-to-result achieved by molecular POCTs when compared with that achieved by laboratory-based tests may significantly speed up the proper hospital area allocation of the patient (i.e. COVID-19 or non–COVID-19–dedicated areas); bed moves before proper allocation [26,27]; and, indirectly, reduce the risk of nosocomial COVID-19 acquisition [27]. Particularly interesting for infection control purposes are the multi-target molecular POCTs that simultaneously carry out a differential diagnosis of SARS-CoV-2 and other respiratory viruses, including influenza virus [28].

A few months after the onset of the COVID-19 pandemic, faster and cheaper antigen-based POCTs have been added to the diagnostic arsenal of SARS-CoV-2. As early as September 2020, WHO recognized their significant role both in patient management and surveillance of COVID-19, especially in the community [29]. A year later, WHO confirmed its position, albeit acknowledging that the evidence still revealed that many of these tests evaluated in different populations did not meet the minimum performance requirements of ≥80% sensitivity and ≥97% specificity [30]. By meta-analysing the results of a large number of evaluations (up to 160), the results of our study show that although the specificity threshold is often exceeded, the sensitivity threshold is not. This draws attention on the importance of the setting and the population. Expectably, the highest sensitivity values, close to the 80% threshold recommended by WHO, were observed in cohorts of symptomatic patients with a recent history of the disease.

A strength of our study is the systematic, comprehensive, and detailed approach to estimate the diagnostic performance of molecular and antigen-based RDTs in accordance to PRISMA reporting guidelines and to a pre-defined registered protocol. Moreover, most of the eligible studies had a low overall risk of bias and applicability concerns. Approximately 16% of studies had a high risk of bias for the patient selection and flow as well as timing of the test according to the QUADAS-2 assessment, which seemed to be much lower in comparison with 20% to 30% for patient selection in two previous meta-analyses [4,5]. This can be attributed to the fact that we established very strict and specific inclusion and exclusion criteria for the included tests to ensure a high-quality meta-analysis.

There are also study limitations to consider. First, the quality of meta-analyses will always be determined by the quality of the individual studies included in the analysis. The inclusion of only FDA- or European Commission/European Union-approved tests in our analysis may improve the quality and reduce heterogeneity. Second, the performance of an RDT can also be influenced by circulating SARS-CoV-2 variants. The included original studies in this meta-analysis were performed during the pre-Omicron variant era and most of the pre-Delta variant era. Recent data suggest that the Omicron variant impairs the diagnostic performance of antigen RDTs [31]; hence, it remains to be seen whether the overall performance of antigen-based tests will be different for the current Omicron and future variants.

As the pandemic progresses and the waves follow one another, the use of POCTs for SARS-CoV-2 detection will continue to be an important reference in both healthcare settings and in the community. In the future, it will be necessary to monitor the quality of POCT results, both molecular and antigenic, particularly in view of the emergence of new SARS-CoV-2 variants.

Author contributions

PCF and CS conceptualized and designed the study. PCF, CDM, DD, DSYO, KD, PPN, VAS, HJ, EK, and KAP acquired and analysed the data. HRC and CT performed statistical analysis of the data. ST, GDA, and MS performed data interpretation. PCF, CDM, DD, DSYO, and CS interpreted the results. PPN drafted the figures. HRC, GDA, and CT prepared the figures. PCF, CDM, DD, DSYO, GDA, and CS drafted the manuscript. PCF, CDM, DD, ST, GDA, MS, and CS critically appraised the manuscript. PCF, CDM and DD contributed equally to this study. All authors had access to all data in the study and have read and approved the published version of the manuscript.

Transparency declaration

CS is supported by the Universities Giessen and Marburg Lung Center, the German Center for Lung Research (DZL), University Hospital Giessen and Marburg research funding according to article 2, section 3 cooperation agreement, and the Deutsche Forschungsgemeinschaft (German Reuter Foundation) Project-ID 197785619–SFB 1021, KFO 309 (P10), and SK 317/1-1 (Project number 428518790) as well as by the Foundation for Pathobiochemistry and Molecular Diagnostics. In addition, consultancy and research funding was acquired from Hycor Biomedical, Bencard Allergie, and Thermo Fisher Scientific and research funding was acquired from Mead Johnson Nutrition. These funds were received outside the submitted work. The protocol of this systematic review and meta-analysis has been registered to the International Prospective Register of Ongoing Systematic Reviews PROSPERO (Registration number: CRD42020227233). No external funding was received for this study.

Data availability statement

The data underlying this article will be shared upon reasonable request to the corresponding author.

Editor: M. Leeflang

Appendix A. Supplementary data

The following is the Supplementary data to this article: