Diagnostic Accuracy of SARS-CoV-2 Rapid Antigen Detection Testing in Symptomatic and Asymptomatic Children in the Clinical Setting

ABSTRACT

Antigen-based rapid diagnostic tests (RDTs) are used in children despite the lack of data. We evaluated the diagnostic performance of the Panbio-COVID-19 Ag Rapid Test Device (P-RDT) in children. Symptomatic and asymptomatic participants 0 to 16 years old had two nasopharyngeal swabs (NPS) for both reverse transcription-PCR (RT-PCR) and P-RDT. A total of 822 participants completed the study, of which 533 (64.9%) were symptomatic. Among the 119 (14.5%) RT-PCR-positive patients, the P-RDT sensitivity was 0.66 (95% confidence interval [CI] 0.57 to 0.74). Mean viral load (VL) was higher among P-RDT-positive patients than negative ones (P < 0.001). Sensitivity was 0.91 in specimens with VL of >1.0E6 IU/ml (95% CI 0.83 to 0.99) and decreased to 0.75 (95% CI 0.66 to 0.83) for specimens >1.0E3 IU/ml. Among symptomatic participants, the P-RDT displayed a sensitivity of 0.73 (95% CI 0.64 to 0.82), which peaked at 1.00 at 2 days post-onset of symptoms (DPOS) (95% CI 1.00 to 1.00), then decreased to 0.56 (95% CI 0.23 to 0.88) at 5 DPOS. There was a trend toward lower P-RDT sensitivity in symptomatic children <12 years (0.62 [95% CI 0.45 to 0.78]) versus ≥12 years (0.80 [95% CI 0.69 to 0.91]; P = 0.09). In asymptomatic participants, the P-RDT displayed a sensitivity of 0.43 (95% CI 0.26 to 0.61). Specificity was 1.00 in symptomatic and asymptomatic children (95% CI 0.99 to 1.00). The overall 73% and 43% sensitivities of P-RDT in symptomatic and asymptomatic children, respectively, was below the 80% cutoff recommended by the World Health Organization. We observed a correlation between VL and P-RDT sensitivity, as well as variation of sensitivity according to DPOS, a major determinant of VL. These data highlight the limitations of RDTs in children, with the potential exception in early symptomatic children ≥12yrs.

INTRODUCTION

The current coronavirus disease 19 (COVID-19) pandemic induces the need for widespread SARS-CoV-2 testing to control virus circulation. The rapid identification of SARS-CoV-2-infected individuals is important whether persons are symptomatic or not, as the role of asymptomatic persons in SARS-CoV-2 transmission is still unclear. Therefore, easy to use, affordable, and rapid diagnostic methods are required in addition to the gold standard of reverse transcription-PCR (RT-PCR) (1). These devices are increasingly helpful in settings where results are immediately needed, access to a testing facility is limited, or in case of shortage of RT-PCR reagents. Several antigen-based rapid diagnostic tests (RDTs) have been marketed to fill this gap. Among them, the Panbio-COVID-19 Ag Rapid Test Device (referred to here as P-RDT) has displayed an overall sensitivity ranging between 61% and 92% (2–7) in adults compared to nasopharyngeal RT-PCR. Sensitivity was improved for higher viral loads (VLs), shorter duration of symptoms and/or in symptomatic patients (2–6). In symptomatic children, the overall sensitivity of the P-RDT was 45 to 78% (5, 8, 9), but published data do not take into account the effect of VL nor the duration of symptoms. Moreover, to our knowledge, no study has evaluated this assay in asymptomatic children at the time of sampling. The aim of the present study was to provide an independent evaluation of the diagnostic performance of the P-RDT in a large cohort of symptomatic and asymptomatic children. We also aimed to identify situations with optimal P-RDT sensitivity by accounting for VL, day post onset of symptoms (DPOS), type and number of symptoms.

MATERIALS AND METHODS

Setting.

This single-center prospective diagnostic study was performed in Geneva University Hospitals’ (HUG) pediatric testing center, from 10 November 2020 to 26 March 2021, with a maximum 14-day incidence of 1,199/100,000 at the time of study onset (10). Participants 0 to 16 years old who presented with the need for SARS-CoV-2 RT-PCR testing were approached. The indication for RT-PCR testing in symptomatic participants was symptoms suggestive of SARS-CoV-2 infection according to local governmental testing criteria. The indications for RT-PCR testing in asymptomatic participants were notification by local health authorities after contact with a laboratory confirmed a SARS-CoV-2-infected person and pretravel testing. The presence or absence of exposure to a SARS-CoV-2 infected person was not documented.

Study procedures.

For each enrolled participant, two nasopharyngeal swabs (NPS) were collected. Nurses were trained to perform NPS testing through a standardized video-documented procedure. First, a standard flocked swab placed in viral transport media (VTM) was used for viral genome detection by RT-PCR. The second swab, provided in the P-RDT kit, was obtained from the contralateral or ipsilateral nostril and the antigen test was performed immediately at the testing center as per the manufacturer’s instructions. All study participants and/or caregivers provided written informed consent prior to specimen collection. The study was approved by the local research ethics board (Commission cantonale d'éthique de la recherche number 2020-02323).

P-RDT testing.

The Panbio-COVID-19 Ag Rapid Test Device for nasopharyngeal use (Abbott Rapid Diagnostics, USA; ref 41FK10) was chosen for the current study based on adult data from our institution showing optimal sensitivity and ease of use. The P-RDT was used as recommended by the manufacturers, using materials provided in the kit only. P-RDT results were read independently by two members of the study team, both being blinded to the result assigned by their pair as well as to the clinical presentation of the participant. Any discrepant result was considered positive when any of the above-mentioned readers set a positive diagnosis.

RT-PCR testing.

RT-PCR testing was performed either on the Cobas SARS-CoV-2 assay (cobas SARS-CoV-2 Test, Cobas 6800, Roche, Switzerland) or on the TaqPath (Applied Biosystems, Thermo Fisher Scientific, Waltham, USA) RT-PCR assay, using NPS in 3 ml VTM. In order to transform cycle threshold (C_T) values into IU/ml, serial dilutions of the first World Health Organization (WHO) international standard for SARS-CoV-2 RNA (11) (National Institute for Biological Standards and Control [NIBSC], Potters Bar, United Kingdom; product code: 20/146) were performed to calibrate both RT-PCR assays, as per the manufacturer’s instructions. All VLs were calculated for originals specimens in log IU/ml of VTM from the C_T values using the following formulas: for Cobas (E gene target), VL (log IU/ml) = (C_T value − 42.59)/−3.096 and for TaqPath (N gene target), VL (log IU/ml) = (C_T value − 44.333)/−3.08.

Data collection.

The following data collected at enrollment were managed using RedCap electronic data capture tools hosted at HUG: date of enrollment, number of days post onset of symptoms (DPOS), gender, age, type of symptoms (nasal discharge, cough, dyspnea, dysphagia, dysgueusia, anosmia, vomiting, diarrhea, fever, chills, decreased intake, headache, myalgia, fatigue and irritability, abdominal pain, and/or nausea), and comorbidities if present (chronic respiratory disease, cardiopathy, immunosuppression, cancer, diabetes, obesity, hypertension, and organ failure). P-RDT results, RT-PCR and VL/C_T values were included subsequently.

Statistics.

Before study onset, a sample size was calculated to have sufficient power to generate a 95% confidence interval (CI) with a lower bound above the WHO target of 80% if the prevalence was 25% (corresponding to the pediatric positivity rate at study onset) and the measured sensitivity of 85% (corresponding to the sensitivity reported in adults in our institution) (6). The sample size was therefore estimated at 654 participants. Continuous variables were expressed by their mean ± standard deviation (SD) and median (interquartile range [IQR]) upon variable distribution. Categorical variables were presented by their frequencies and relative proportions. For comparisons of continuous variables, parametric Student t tests and nonparametric Mann-Whitney tests upon variable distribution were used. For categorical variables, either Chi-square or Fischer’s exact tests were performed, depending on applicability. Statistical analyses were processed using SPSS software v23.0 (IBM Corp., Armonk, NY). Statistical significance was defined as P < 0.05 (two-sided).

Data availability.

Study protocols, statistical analysis plan, and individual participant data that underlie the results reported in this article can be made available (after deidentification) to researchers who make a methodologically sound proposal. Proposals should be directed to arnaud.lhuillier@hcuge.ch. To gain access, data requestors will need to sign a data access agreement.

RESULTS

A total of 885 pediatric participants were enrolled. Among them, 63 were subsequently excluded (Fig. S1 in the supplemental material). A total of 822 participants completed the study and had both RT-PCR and P-RDT performed. The demographics of symptomatic and asymptomatic study participants are detailed in Table 1.

TABLE 1.

Study participants’ demographics

Characteristicsa	Symptomatic (n = 533)	Asymptomatic (n = 289)	Combined (n = 822)	P valueb
Median age (±IQR)	12.1 (9.4–14.5)	10.9 (8.5–13.7)	11.8 (9.0–14.3)	0.002
Female sex, n (%)	266 (49.9)	138 (47.8)	404 (49.1)	0.555
Comorbidities, n (%)
Chronic respiratory disease	33 (6.2)	13 (4.5)	46 (5.6)
Obesity	5 (2.8)	10 (3.5)	25 (3.0)
Diabetes	1 (0.2)	3 (1.0)	4 (0.5)
Hypertension	2 (0.4)	0	2 (0.2)
Cancer	0	2 (0.7)	2 (0.2)
Cardiopathy	1 (0.2)	0	1 (0.1)
Other immunosuppression	1 (0.2)	0	1 (0.1)
Chronic liver failure	1 (0.2)	0	1 (0.1)
Result of RT-PCR, n (%)				0.014
Negative	444 (83.3)	259 (89.6)	703 (85.5)
Positive	89 (16.7)	30 (10.4)	119 (14.5)
Mean log RNA IU/ml (± SD)	5.9 (±1.8)	4.1 (±1.9)	5.5 (±2.0)	<0.001

^aRT-PCR, reverse transcription-PCR; IQR, interquartile range; SD, standard deviation.

^bSymptomatic versus asymptomatic participants.

Overall, 14.5% (119/822) were positive by RT-PCR with a mean RNA VL of 5.5 log IU/ml (SD 2.0) (Table 1). Among the 822 P-RDT performed, only one P-RDT result displayed a discrepant interpretation between the two observers (κ = 0.999). The corresponding patient was subsequently considered positive for the purpose of the analysis, leaving an overall positivity rate of 9.6% (79/822). The P-RDT’s sensitivity and specificity when challenged against RT-PCR were 0.66 (95% CI 0.57 to 0.74) and 1.00 (95% CI 1.00 to 1.00), respectively (Table 2, Table S1). Mean VL was higher among positive P-RDT specimens than negative ones (6.4 log IU/ml [SD 1.4] versus 3.7 [SD 1.6]; P < 0.001) (Fig. 1).

TABLE 2.

Diagnostic accuracy of the Panbio RDT

Characteristicsa	Symptomatic	Asymptomatic	Combined
Sensitivity (95% CI)	0.73 (0.64–0.82)	0.43 (0.26–0.61)	0.66 (0.57–0.74)
Specificity (95% CI)	1.00 (0.99–1.00)	1.00 (1.00–1.00)	1.00 (1.00–1.00)
Positive predictive value (95% CI)	0.98 (0.92–1.00)	1.00 (1.00–1.00)	0.99 (0.96–1.01)
Negative predictive value (95% CI)	0.95 (0.92–0.97)	0.94 (0.91–0.97)	0.95 (0.93–0.96)

^aCI, confidence interval.

FIG 1.

Mean (standard deviation) SARS-CoV-2 viral load expressed in log IU/ml among RT-PCR-positive individuals according to Panbio RDT results. RDT, antigen-based rapid diagnostic test; RT-PCR, reverse transcription-PCR; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

Sensitivity varied according to RT-PCR VL, even though false-negative results occurred throughout all VL values. Sensitivity was highest at 0.97 in specimens with VL of >1.0E7 IU/ml (95% CI 0.91 to 1.00), decreased slightly to 0.91 (95% CI 0.83 to 0.99) for specimens of >1.0E6 IU/ml, and to 0.87 (95% CI 0.79 to 0.94) and 0.86 (95% CI 0.79 to 0.94) for specimens of >1.0E5 IU/ml and >1.0E4 IU/ml, respectively. Sensitivity then dropped to 0.75 (95% CI 0.66 to 0.83) and 0.69 (95% CI 0.61 to 0.78) for specimens of >1.0E3 IU/ml and >1.0E2 IU/ml, respectively (Fig. 2). Of note, nine P-RDT-negative specimens that showed a high viral load by RT-PCR were retested using another RT-PCR assay which confirmed high viral load in every specimen.

FIG 2.

Sensitivity (95% confidence interval) of Panbio RDT according to SARS-CoV-2 viral load expressed in log IU/ml. RDT, antigen-based rapid diagnostic test; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

Mean viral load was lower in children <12 years old than in older children (5.0 log IU/ml [SD 2.0] versus 5.9 [SD 1.9]; P = 0.011) and there was a trend toward lower P-RDT sensitivity in children <12 years old (0.57 [95% CI 0.44 to 0.70]) than in older children (0.74 [95% CI 0.63 to 0.85]; P = 0.057). Additional demographics stratified by age group are detailed in Table S2.

Symptomatic participants.

Among the 533 (64.9%) symptomatic participants, median duration of symptoms at time of testing was 2 days (IQR 1 to 3) (Table S3). The most frequently reported symptoms were headache (56%), nasal discharge (56%), cough (45%), and fatigue (44%) (Table S3). Eighty-nine symptomatic patients (16.7%) were positive by RT-PCR with a mean RNA VL of 5.9 log IU/ml (SD 1.8) (Table 1). The P-RDT displayed an overall sensitivity and specificity of 0.73 (95% CI 0.64 to 0.82) and 1.00 (95% CI 0.99 to 1.00), respectively (Table 2, Table S1). For specimens with VL of >1.0E6 IU/ml, sensitivity was 0.92 (95% CI 0.84 to 1.00). Mean VL was higher among positive P-RDT specimens than negative ones (6.6 log IU/ml [SD 1.3] versus 4.1 [SD 1.7]; P < 0.001) (Fig. 1).

Sensitivity was 0.68 at 0 to 1 DPOS (95% CI 0.53 to 0.83), peaked at 1.00 at 2 DPOS (95% CI 1.00 to 1.00), then gradually decreased to 0.73 (95% CI 0.46 to 0.99), 0.63 (95% CI 0.29 to 0.96), and 0.56 (95% CI 0.23 to 0.88) at 3, 4, and 5 DPOS, respectively (Fig. 3A). False-negative results occurred across all DPOS except at 2 DPOS.

FIG 3.

Sensitivity (95% confidence interval) of Panbio RDT according to days post onset of symptoms (A) and clinical symptoms (B). RDT, antigen-based rapid diagnostic test.

Additionally, we analyzed sensitivity according to typical acute COVID-19 symptoms. For this analysis, only objective symptoms were reported because of the low ability of children to report more subjective symptoms, such as anosmia, even though very suggestive of COVID-19. Sensitivity was highest in the presence of chills (0.88 [95% CI 0.65 to 1.00]) and cough (0.86 [95% CI 0.75 to 0.96]), followed by fever (0.74 [95% CI 0.56 to 0.92]) and then nonspecific symptoms (0.65 [95% CI 0.50 to 0.80]) (Fig. 3B). Interestingly, sensitivity was significantly better in participants reporting >2 symptoms (0.78 [95% CI 0.68 to 0.87]) than in those reporting only 1 to 2 symptoms (0.53 [95% CI 0.29 to 0.77]; P = 0.038).

Among symptomatic participants, mean viral load did not significantly differ between children <12 years old and older children (5.5 log IU/ml [SD 1.8] versus 6.2 [SD 1.8]; P = 0.085). Moreover, median duration of symptoms did not differ between children <12 years old and older children (2 days [IQR 1 to 4] for both groups; P = 0.790). There was a trend toward lower P-RDT sensitivity in children <12 years old (0.62 [95% CI 0.45 to 0.78]) than in older ones (0.80 [95% CI 0.69 to 0.91]; P = 0.09).

Asymptomatic participants.

Among the 289 (35.1%) asymptomatic participants at the time of sampling, 10.4% (30/289) were positive by RT-PCR with a mean RNA VL of 4.1 log IU/ml (SD 1.9), which was significantly lower than found in symptomatic participants (P < 0.001) (Table 1). The P-RDT displayed an overall sensitivity and specificity of 0.43 (95% CI 0.26 to 0.61) and 1.00 (95% CI 1.00 to 1.00), respectively (Table 2, Table S1). For specimens with VL >1.0E6 IU/ml, sensitivity was 0.86 (95% CI 0.60 to 1.00). Mean VL was higher among positive P-RDT specimens than in negative ones (5.5 log IU/ml [SD 1.6] versus 3.0 [SD 1.2]; P < 0.001) (Fig. 1). Mean viral load did not significantly differ between children <12 years old and older children (4.3 [SD 2.0] versus 3.5 [SD 0.8]; P = 0.158).

DISCUSSION

This study prospectively evaluated the diagnostic accuracy of the Panbio-COVID-19 RDT in the clinical setting in more than 800 symptomatic and asymptomatic children, taking into account VL, DPOS, and clinical parameters such as number and type of symptoms. The study was performed during a period of sustained virus circulation, with an overall positivity RT-PCR rate of 17% among symptomatic participants. The major finding of our work is an overall suboptimal 66% sensitivity of the assay, ranging between 43% and 73% in asymptomatic and symptomatic children, respectively. On the other hand, specificity was 100% regardless of the presence or absence of symptoms. It would therefore seem very unlikely that children are unnecessarily sent into quarantine, which is important from a public health perspective. The WHO RDT target product profile cutoffs of ≥80% for sensitivity and ≥97% for specificity were achieved for specificity but not for sensitivity (12). The relatively low sensitivity of the P-RDT is in line with previous data showing an assay sensitivity of 45 to 78% among symptomatic children (5, 8, 9), and confirms that the assay sensitivity is lower than that in symptomatic adults in whom the largest studies report sensitivity between 67 and 92% (2, 3, 5–7). However, among symptomatic children with high VL, the assay’s sensitivity seemed only marginally lower than symptomatic adults with high VL (6). The suboptimal sensitivity of the assay in children is most likely explained by the increasingly recognized evidence that children have lower SARS-CoV-2 VLs than adults. Indeed, although initial studies suggested similar VLs in adults and children, they were limited in sample size and did not take into account DPOS (13–15), which is a major determinant of VL (16). Recently, studies on larger data sets and/or taking into account DPOS have shown that SARS-CoV-2-infected children have significantly lower VLs than adults (17–19). Another possible explanation for the lower sensitivity could be sampling bias related to the technical challenge of the NPS procedure in children, given that the swab for the P-RDT testing was the second one to be performed. False-negative RDT results have also been observed in adult RDT studies, even though less frequently (6), and are unlikely to be caused by SARS-CoV-2 variants. Indeed, no mutation in the N gene possibly causing false-negative RDTs in circulating SARS-CoV-2 variants have been identified and, so far, all variants are detected with RDTs with comparable sensitivity to earlier circulating variants (20).

Novel findings in our study relate to the evaluation of P-RDT sensitivity in the light of several factors such as VL, DPOS, type and number of symptoms, which has not been reported so far in the pediatric population. As expected, VL was higher among those with positive P-RDT (true positives) than among those with negative P-RDT (false negatives), as already shown in the adult setting (6). Similarly, and as previously shown in adults (2, 3, 6), sensitivity was correlated with VL, peaking at 97% in specimens with >1.0E7 IU/ml and dropping to 75% in specimens with >1.0E3 IU/ml. Sensitivity remained at >80% in specimens with >1.0E4 IU/ml. However, the presence of false-negative P-RDT results in participants with VLs compatible with shedding of infectious virus is important from a public health perspective, as these would not be identified by P-RDT despite being likely contagious. Another interesting finding was the impact of DPOS on the assay sensitivity. Sensitivity was optimal at 2 DPOS, when VL is expected to peak (21, 22). These findings somehow differ from adult data, where sensitivity remained high throughout the first five DPOS (despite being lower at 0 DPOS) (6), likely here again reflecting the impact of higher VLs in adults on P-RDT sensitivity. Similarly, the trend toward lower sensitivity in children <12 years old is likely explained by lower VLs in younger children, as seen in this study and other publications (17–19). Among symptomatic participants, sensitivity was better in those with COVID-19-typical symptoms, as previously shown in adults (6), but also in those with >2 symptoms. Interestingly, the sensitivity of P-RDT in children asymptomatic at time of sampling was low at 43%. This is probably related to the fact that asymptomatic children in our data set had significantly lower VL than symptomatic children.

The strength of our study is related to the large size of a purely pediatric data set. The large subset of asymptomatic children at the time of sampling, and the analysis of diagnostic accuracy based on VL, DPOS, number and specific symptoms, represent additional strengths and novelties. With a majority of mild clinical presentation and 25% being asymptomatic among RT-PCR-positive cases, our data set is representative of the majority of SARS-CoV-2-infected children, even though the extent of the pediatric contribution to community transmission is still debated.

Our study has several limitations. First, the evaluation was based on one RDT only. Comparative studies have shown similar or reduced performance of other RDTs compared to the P-RDT (6, 23). It is therefore highly unlikely that another RDT would perform significantly better in children than the P-RDT. Second, the study was performed using two different validated RT-PCR assays as gold standards, although 90% of the specimens were tested on the Cobas assay, and VLs were reported in IU/ml, allowing comparison between assays. Third, the study was conducted in a high-prevalence setting. Extrapolating the findings to low-prevalence settings must be done with caution. Then, providing that the NPS for P-RDT was always performed after the NPS for RT-PCR, one cannot exclude that the second procedure was more challenging to perform. Finally, we did not evaluate the performance of P-RDT on oropharyngeal, nasal, or saliva specimens. However, given the fact that VL is lower in these anatomical compartments compared to NPS (24–27), one can expect even lower sensitivity of P-RDT if used on oropharyngeal, nasal, or saliva specimens.

In conclusion, this independent study confirms the respective suboptimal sensitivity of P-RDT in symptomatic children and its poor sensitivity in children asymptomatic at the time of sampling, providing additional evidence for cautious routine use of these tests for the detection of SARS-CoV-2, both in symptomatic and asymptomatic children. This study also highlights the impact of VL, DPOS, and clinical presentation on the assay’s sensitivity and shows that the sensitivity was ≥80% in participants with medium and high VLs, suggesting reliable identification of contagious individuals (16, 28). However, it should be discussed whether missing individuals with lower VLs is acceptable, since they might subsequently have an increase in their VL and become contagious. Therefore, public health benefits of rapidly identifying infected children should be balanced with the disadvantages of missed diagnoses (29). For individual diagnosis, P-RDT seems a decent alternative to RT-PCR in symptomatic children ≥12 years, especially if tested at <5 DPOS, based on our findings. For mass pediatric screening, however, such as in school settings or institutions, providing there is no vulnerable contact person, the suboptimal sensitivity of P-RDT is likely outweighed by the advantages of P-RDT, allowing rapid identification of most infected individuals without the need of a laboratory facility.