Diagnostic Accuracy of the Panbio Severe Acute Respiratory Syndrome Coronavirus 2 Antigen Rapid Test Compared with Reverse-Transcriptase Polymerase Chain Reaction Testing of Nasopharyngeal Samples in the Pediatric Population

Abstract

We conducted a multicenter clinical validity study of the Panbio coronavirus disease 2019 Antigen Rapid Test of nasopharyngeal samples in pediatric patients with coronavirus disease 2019–compatible symptoms of ≤5 days of evolution. Our study showed limited accuracy in nasopharyngeal antigen testing: overall sensitivity was 45.4%, and 99.8% of specificity, positive-predictive value was 92.5%.

Since the World Health Organization declared the emergence of the coronavirus disease 2019 (COVID-19) pandemic in March 2020, the development of a rapid and accurate test has been a priority to rapidly identify acute cases. Accurate diagnosis allows case isolation and contact tracing to avoid the spread of the infection. Reverse-transcriptase polymerase chain reaction (RT-PCR) testing performed on a nasopharyngeal sample (NPS) is the most-used test globally.¹ However, it requires skilled staff and special laboratory equipment, is expensive, and during the pandemic, the turnaround time for results may be slow due to the high demand. A gap exists between the large number of patients and the laboratory capacities to perform RT-PCR. Rapid antigen (Ag) tests have appeared recently as point-of-care testing techniques.² The Panbio COVID-19 Ag Rapid Test by Abbott Rapid Diagnostic, based on lateral immunochromatography, is a simple and rapid test that can detect the virus in NPS samples. According to the manufacturer's information, the sensitivity during the first week of symptoms is 93.9% (95% CI 86.5%-97.4%). Studies carried out in adult population showed sensitivity close to 85% in patients with <5 days of symptoms.^{3 , 4} Nevertheless, there is a lack of evidence about the accuracy of rapid Ag tests in the pediatric population.

Our objective was to determine the sensitivity, specificity, positive- and negative-predictive values, and concordance of the Panbio COVID-19 Ag Rapid Test by Abbott Rapid Diagnostic in NPS samples in symptomatic pediatric population compared with RT-PCR testing.

Methods

This was a descriptive, retrospective, multicenter clinical validity study nested in a prospective, observational, multicenter cohort study (Epidemiological Study of COVID-19 in Children of the Spanish Society of Pediatric; EPICO-AEP). We included 1620 pediatric patients aged 0-16 years with symptoms compatible with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection of ≤5 days of evolution attended in the emergency departments of the 7 centers involved. Asymptomatic patients with close contact with patients positive for COVID-19 were excluded. The study was conducted in September and October 2020. Verbal consent for paired testing was obtained. The study was approved by the Fundación de Investigacion Biomédica del Hospital 12 de Octubre Ethics Committee (code 20/101).

SARS-CoV-2 Testing

Two NPS were obtained concurrently from each patient by trained nurses. One was employed to perform the rapid antigenic test and the other to carry out the RT-PCR. We applied on-site the Panbio COVID-19 Ag Rapid Test Device (Abbott Rapid Diagnostic) following the manufacturer's instructions. Results were interpreted by attending pediatricians and nurse staff. The Panbio test is a qualitative, membrane-based immunoassay (immunochromatography) for the detection of nucleocapsid protein of SARS-CoV-2. RT-PCR testing was performed within 24 hours of specimen collection targeting SARS-CoV-2 E and RdRp genes. Results of RT-PCR tests were analyzed by technicians and practitioners specialized in microbiology in the involved centers' laboratories according to the protocol of each center.

Statistical Analyses

Sample size for a noninferiority study was calculated considering 80% power, for a 5% prevalence and a 90% sensitivity, using RT-PCR as the gold standard reference. A minimum sample of 1200 participants was calculated. A confusion matrix was displayed, and accuracy, sensitivity, specificity, and predictive positive and negative values were calculated. Associated 95% CI was estimated using Clopper–Pearson CIs. Noninferiority of sensitivity and specificity between diagnostic tests was assessed using the McNemar test. The agreement between the 2 methods was calculated using Cohen kappa index. Statistical analyses were performed using R Software.⁵

Results

Among 1620 patients tested in 7 hospitals, 77 tested positive by RT-PCR (4.8%), 38 tested positive by Panbio COVID-19 Ag Rapid Test (2.3%), and 35 patients tested positive by both tests (2.1%). Discordant results occurred in 45 Ag test results compared with RT-PCR (2.7%): 3 of 1543 (0.2%) false-positive Ag tests and 42 of 77 (54.5%) false-negative Ag test results were found (Figure 1; available at www.jpeds.com).

Figure 1.

Overall sensitivity, specificity, and predictive values for rapid antigenic test Panbio using RT-PCR as the gold-standard reference.

There is evidence of a systematic difference between results from the 2 diagnostic tests (P = 1.47·10⁻⁰⁸). A moderate agreement (k = 0.6) was found between the 2 methods.

The overall sensitivity for rapid antigenic test Panbio was 45.4% (95% CI 34.1-57.2), and specificity was 99.8% (95% CI 99.4-99.9) (Figures 1 and 2 [available at www.jpeds.com]). The statistical power was 52% for sensitivity calculation and 99% for specificity. The positive predictive value of the Panbio COVID-19 Ag Rapid Test for this 4.8% SARS-CoV-2 prevalence was 92.5% (95% CI 78.6-97.4). The negative predictive value of the Panbio COVID-19 Ag Rapid Test was 97.3 % (95% CI 96.8-97.8). Positive likelihood ratio (PLR) was 233.8 (95% CI 73.5-743.3), and negative likelihood ratio (NLR) was 0.54 (95% CI 0.44-0.67).

Figure 2.

Representation of sensitivity, specificity, and predictive values for rapid antigenic test Panbio using RT-PCR as the gold-standard reference.

Discussion

Compared with RT-PCR testing, our results show a low sensitivity of the Panbio COVID-19 Ag Rapid Test (45.4%) with high specificity (99.8%). The concordance between RT-PCR and Ag test was only moderate (k = 0.6). The high proportion of false-negative Ag tests (54.5%) may have public health implications. If these patients with false-negative results are contagious, they will not be isolated and may continue to spread infection. A false-negative result may have implications for the treatment received by the patient. However, RT-PCR positivity can persist for some weeks after a SARS-CoV-2 infection.⁶ We cannot exclude the possibility that discordant results are due to the persistence of a previous SARS-CoV-2 infection in children with symptoms due to another pathogen. In these cases, the Ag test could better reflect the contagiousness of patient than would RT-PCT, but the concordance between contagiousness and RT-PCR or Ag positivity is not clearly known.

The predictive values were good, beyond 95%. The predictive value of a test depends on the pretest probability. In this study, the prevalence of SARS-CoV-2 infection was almost 5%. This prevalence was the current real situation among children attended in emergency departments in several Spanish cities, where the test is being used as a point-of-care technique. To assess the implication of these results in different prevalence settings, we calculated likelihood ratios. The high PLR suggests that the test is good when the result is positive, and the low NLR indicates that the test has only limited value when the result is negative. In this scenario, with a prevalence of 5%, a positive result will identify the infection in 92.5% (95% CI 79.5%-97.5%) of symptomatic cases. A negative result we will be wrong in 2.8% (95% CI 2.3%-3.4%), making it necessary to perform 35 tests to misdiagnose a patient. An increase in prevalence would lead to an increase in the chances of having the infection with a negative test result.

The low sensitivity and NLR of the Ag test may call into question its value as a diagnostic tool. Nevertheless, in a pandemic situation, a cheap, rapid, and widely distributed test with a good PLR may be worthy as a first screening test.⁷ Still, pediatricians should be aware of the actual accuracy and establish sequential strategies of diagnosis if needed.

Previous studies in adults have evaluated the validity of the Panbio COVID-19 Ag Rapid Test compared with RT-PCR testing in symptomatic adults with fewer than 7 days of symptoms. The authors found a greater Ag test sensitivity in 327 adults (82.6%) compared with sensitivity in 85 pediatric patients (62.5%).^{3 , 4} The difference between both populations could be related to differences in SARS-CoV-2 viral load in the upper respiratory tract between children and adults. Another similar study showed that the sensitivity of other Ag Rapid Test (Bioeasy 2019-Novel Coronavirus Fluorescence Ag Rapid Test Kit) was significantly greater in samples with high viral loads.⁸ However, previous studies found no age-related differences in SARS-CoV-2 nasopharyngeal viral load.^{9 , 10}

This study has several limitations. Sample size calculation was done with an assumption of 90% sensitivity based on the manufacturer's information, so the final statistical power was lower than expected. Still, the sample size is large. We did not record the specific symptoms, sampling day after onset, coinfections, or the cycle threshold (CT) values of RT-PCR, so we do not know if a high CT value could be related to a lower sensitivity of the comparative Ag test. Equipment for RT-PCR test was not uniform, and samples with CT close to the threshold may represent equivocal results. Viral culture was not performed, so the relationship between contagiousness, RT-PCR and Ag is unclear. Ongoing research of the study team will include some of this information. In the current pandemic situation, our data should be considered by clinicians and policymakers to understand the limited accuracy of the Panbio COVID-19 Ag Rapid Test as performed on NPS samples of children with symptoms consistent with COVID-19 of less than 5 days of duration.

Appendix

List of additional members of the EPICO-AEP Working Group:

Cristina Calvo, MD, PhD, Pediatric Department, Hospital Universitario La Paz, Madrid, Spain.

M^a José Mellado, MD, PhD, Pediatric Department, Hospital Universitario La Paz, Madrid, Spain.

Paula Rodríguez-Molino, MD, Pediatric Department, Hospital Universitario La Paz, Madrid, Spain.

Teresa del Rosal, MD, PhD, Pediatric Department, Hospital Universitario La Paz, Madrid, Spain.

Mar Santos, MD, PhD, Pediatric Department, Hospital Universitario Gregorio Marañón, Madrid, Spain.

Marisa Navarro, MD, PhD, Pediatric Department, Hospital Universitario Gregorio Marañón, Madrid, Spain.

Elena Rincón, MD, Pediatric Department, Hospital Universitario Gregorio Marañón, Madrid, Spain.

Begoña Santiago, MD, PhD, Pediatric Department, Hospital Universitario Gregorio Marañón, Madrid, Spain.

Jesús Saavedra-Lozano, MD, PhD, Pediatric Department, Hospital Universitario Gregorio Marañón, Madrid, Spain.

David Aguilera-Alonso, MD, Pediatric Department, Hospital Universitario Gregorio Marañón, Madrid, Spain.

Cristina Epalza, MD, Pediatric Department, Hospital 12 de Octubre, Madrid, Spain.

Daniel Blázquez-Gamero, MD, PhD, Pediatric Department, Hospital 12 de Octubre, Madrid, Spain.

Sara Villanueva, MD, Pediatric Department, Hospital 12 de Octubre, Madrid, Spain.

Pablo Rojo, MD, PhD, Pediatric Department, Hospital 12 de Octubre, Madrid, Spain.

Gero Calleja, MD, Pediatric Department, Hospital Universitario Niño Jesús, Madrid, Spain.

J. A. Alonso, MD, Pediatric Department, Hospital Universitario Niño Jesús, Madrid, Spain.

Mercedes de la Torre, MD, Pediatric Department, Hospital Universitario Niño Jesús, Madrid, Spain.

F. J. Sanz-Santaeufemia, MD, Pediatric Department, Hospital Universitario Niño Jesús, Madrid, Spain.

M. I. Iglesias, MD, Pediatric Department, Hospital Universitario Niño Jesús, Madrid, Spain.

B. Herrero, MD, Pediatric Department, Hospital Universitario Niño Jesús, Madrid, Spain.

M. Alonso, MD, Pediatric Department, Hospital Universitario Niño Jesús, Madrid, Spain.

Toni Soriano-Arandes, MD, Pediatric Department, Hospital Universitari Vall d'Hebron, Barcelona, Spain.

J. M. Pujol, MD, Pediatric Department, Hospital Universitari Vall d'Hebron, Barcelona, Spain.

Susana Melendo, MD, Pediatric Department, Hospital Universitari Vall d'Hebron, Barcelona, Spain.

Pere Soler-Palacin, MD, PhD, Pediatric Department, Hospital Universitari Vall d'Hebron, Barcelona, Spain.

Silvia Simó, MD, Pediatric Department, Hospital Universitario Sant Joan de Deu, Barcelona, Spain.

Victoria Fumadó, MD, PhD, Pediatric Department, Hospital Universitario Sant Joan de Deu, Barcelona, Spain.

Miguel Lanaspa, MD, Pediatric Department, Hospital Universitario Sant Joan de Deu, Barcelona, Spain.

M. Urretavizcaya, MD, Pediatric Department, Complejo Hospitalario de Navarra, Pamplona, Spain.

Mercedes Herranz, MD, Pediatric Department, Complejo Hospitalario de Navarra, Pamplona, Spain.

Marta Pareja, MD, Pediatric Department, Complejo Hospitalario Universitario de Albacete, Albacete, Spain.

Fatima Ara, MD, Pediatric Department, Hospital Universitario Quirónsalud Madrid, Madrid, Spain.

Fernando Cabañas, MD, Pediatric Department, Hospital Universitario Quirónsalud Madrid, Madrid, Spain.

Rut del Valle, MD, PhD, Pediatric Department, Hospital Infanta Sofia, SanSebastián de los Reyes, Spain.

Ana Barrios, MD, Pediatric Department, Hospital Infanta Sofia, SanSebastián de los Reyes, Spain.

Enrique Otheo, MD, Pediatric Department, Hospital Ramón y Cajal, Madrid, Spain.

José Luis Vázquez, MD, Microbiology Department, Hospital Ramón y Cajal, Madrid, Spain.

Lola Falcón, MD, PhD, Pediatric Department, Hospital Universitario Virgen del Rocío, Sevilla, Spain.

Olaf Neth, MD, PhD, Pediatric Department, Hospital Universitario Virgen del Rocío, Sevilla, Spain.

Peter Olbrich, MD, PhD, Pediatric Department, Hospital Universitario Virgen del Rocío, Sevilla, Spain.

Walter Goicoechea, MD, PhD, Pediatric Department, Hospital Universitario Virgen del Rocío, Sevilla, Spain.

Laura Martín, MD, Pediatric Department, Hospital Universitario Regional de Málaga. Málaga, Spain.

Lucía Figueroa, MD, Pediatric Department, Hospital de Villalba, Villalba, Spain.

María Llorente, MD, Pediatric Department, Hospital Universitario del Sureste, Arganda del Rey, Spain.

María Penin, MD, Pediatric Department, Hospital Príncipe de Asturias, Alcalá de Henares, Spain.

Claudia García, MD, Pediatric Department, Hospital Príncipe de Asturias, Alcalá de Henares, Spain.

María García, MD, Pediatric Department, Hospital Príncipe de Asturias, Alcalá de Henares, Spain.

Teresa Alvaredo, MD, Pediatric Department, Hospital Príncipe de Asturias, Alcalá de Henares, Spain.

M^a Inmaculada Olmedo, MD, Pediatric Department, Hospital Puerta de Hierro, Majadahonda, Spain.

Agustín López, MD, Pediatric Department, Hospital Puerta de Hierro, Majadahonda, Spain.

Elvira Cobo, MD, Pediatric Department, Hospital Fundación Alcorcón, Alcorcón, Spain.

Mariam Tovizi, MD, Pediatric Department, Hospital del Tajo, Aranjuez, Spain.

Pilar Galán, MD, Pediatric Department, Hospital Fundación Fuenlabrada, Fuenlabrada, Spain.

Sara Guillén, MD, Pediatric Department, Hospital de Getafe, Getafe, Spain.

Adriana Navas, MD, Pediatric Department, Hospital Infanta Leonor, Madrid, Spain.

M. Luz García, MD, PhD, Pediatric Department, Hospital de Leganés, Leganés, Spain.

Sara Pérez, MD, Pediatric Department, Hospital de Torrejón, Torrejón de Ardoz, Spain.

María José Hernández, MD, Pediatric Department, Hospital Central de la Defensa, Madrid, Spain.

Arantxa Berzosa, MD, Pediatric Department, Hospital Clínico San Carlos, Madrid, Spain.

Nerea Gallego, MD, Pediatric Department, Hospital Universitari Son Espases, Palma de Mallorca, Spain.

Ana López, MD, Pediatric Department, Hospital Universitari Son Espases, Palma de Mallorca, Spain.

Beatriz Ruiz, MD, Pediatric Department, Hospital Universitario Reina Sofía, Córdoba, Spain.

Santiago Alfayate, MD, Pediatric Department, Hospital Clínico Universitario Virgen de la Arrixaca, Murcia, Spain.

Ana Menasalvas, MD, Pediatric Department, Hospital Clínico Universitario Virgen de la Arrixaca, Murcia, Spain.

Eloísa Cervantes, MD, Pediatric Department, Hospital Clínico Universitario Virgen de la Arrixaca, Murcia, Spain.

María Méndez, MD, Institut d'Investigació en Ciències de la Salut Germans Trias i Pujol, Barcelona, Spain.

Ángela Hurtado, MD, Pediatric Department, Instituto Hispalense de Pediatría, Sevilla, Spain.

Yolanda Ruiz, MD, Pediatric Department, Hospital San Pedro, Logroño, Spain.

Cristina García, MD, Pediatric Department, Hospital San Pedro, Logroño, Spain.

Inés Amich, MD, Pediatric Department, Hospital San Pedro, Logroño, Spain.

Manuel Oltra, MD, Pediatric Department, Hospital Universitari i Politècnic La Fe, Valencia, Spain.

Álvaro Villaroya, MD, Pediatric Department, Hospital Universitari i Politècnic La Fe, Valencia, Spain.

Angustias Ocaña, MD, Pediatric Department, Hospital La Moraleja, Madrid, Spain.

Isabel Romero, MD, Pediatric Department, Hospitales Madrid, Madrid, Spain.

María Fernanda Guzmán, MD, Pediatric Department, Hospitales Madrid, Madrid, Spain.

M. J. Pascual, MD, Pediatric Department, Hospital Nisa, Madrid, Spain.

María Sánchez-Códez, MD, Pediatric Department, Hospital Universitario Puerta del Mar, Cádiz, Spain.

Elena Montesinos, MD, Pediatric Department, Consorci Hospital General Universitari de València, Valencia, Spain.

Julia Jensen, MD, Pediatric Department, Hospital Infanta Cristina, Parla, Spain.

María Rodríguez, MD, Pediatric Department, Hospital Infanta Cristina, Parla, Spain.

Gloria Caro, MD, Pediatric Department, Hospital Infanta Elena, Valdemoro, Spain.

Neus Rius, MD, Pediatric Department, Hospital Universitari Sant Joan de Reus, Reus, Spain.

Alba Gómez, MD, Pediatric Department, Hospital Universitari Sant Joan de Reus, Reus, Spain.

Rafael Bretón, MD, Pediatric Department, Hospital Clínico Universitario de Valencia, Valencia, Spain.

Margarita Rodríguez, MD, Pediatric Department, Hospital Universitario Virgen de las Nieves, Granada, Spain.

Julio Romero, MD, Pediatric Department, Hospital Universitario Virgen de las Nieves, Granada, Spain.

Ana Campos, MD, Pediatric Department, Hospital Universitario Sanitas La Zarzuela, Madrid, Spain.

Mercedes García, MD, Pediatric Department, Hospital de Mérida, Mérida, Spain.

Rosa María Velasco, MD, Pediatric Department, Complejo Hospitalario de Toledo, Toledo, Spain.

Zulema Lobato, MD, Pediatric Department, Althaia, Xarxa Assistencial Universitària de Manresa, Manresa, Spain.

Fernando Centeno, MD, Pediatric Department, Hospital Universitario Río Hortega, Valladolid, Spain.

Elena Pérez, MD, Pediatric Department, Hospital Universitario Río Hortega, Valladolid, Spain.

Paula Vidal, MD, Pediatric Department, Hospital Clínico Universitario Lozano Blesa, Zaragoza, Spain.

Corsino Rey, MD, Pediatric Department, Hospital Universitario Central de Asturias, Oviedo, Spain.

Ana Vivanco, MD, Pediatric Department, Hospital Universitario Central de Asturias, Oviedo, Spain.

Maruchi Alonso, MD, Pediatric Department, Hospital Universitario Central de Asturias, Oviedo, Spain.

Pedro Alcalá, MD, Pediatric Department, Hospital General Universitario de Alicante, Alicante, Spain.

Javier González de Dios, MD, Pediatric Department, Hospital General Universitario de Alicante, Alicante, Spain.

Eduard Solé, MD, Pediatric Department, Hospital Universitari Arnau de Vilanova, Lleida, Spain.

Laura Minguell, MD, Pediatric Department, Hospital Universitari Arnau de Vilanova, Lleida, Spain.

Itziar Astigarraga, MD, Pediatric Department, Hospital Universitario de Cruces, Bilbao, Spain.

M^a Ángeles Vázquez, MD, Pediatric Department, Hospital Universitario Torrecárdenas, Almería, Spain.

Miguel Sánchez, MD, Pediatric Department, Hospital Universitario Torrecárdenas, Almería, Spain.

Elena Díaz, MD, Pediatric Department, Hospital Virgen de la Luz, Cuenca, Spain.

Eduardo Consuegra, MD, Pediatric Department, Hospital Universitario de Salamanca, Salamanca, Spain.

María Cabanillas, MD, Pediatric Department, Complejo Asistencial Universitario de Palencia, Palencia, Spain.

Luis Peña, MD, Pediatric Department, Hospital Universitario Materno Infantil de las Palmas, Las Palmas de Gran Canaria, Spain.

Elisa Garrote, MD, Pediatric Department, Hospital Universitario de Basurto, Bilbao, Spain.

Maite Goicoechea, MD, Pediatric Department, Hospital Universitario de Basurto, Bilbao, Spain.

Irene Centelles, MD, Pediatric Department, Hospital General Universitari de Castelló, Castellón, Spain.

Santiago Lapeña, MD, Pediatric Department, Complejo Asistencial Universitario de León, León, Spain.

Sara Gutiérrez, MD, Pediatric Department, Complejo Asistencial Universitario de León, León, Spain.

Soraya Gutiérrez, MD, Pediatric Department, Complejo Asistencial Universitario de León, León, Spain.

Amparo Cavalle, MD, Pediatric Department, PIUS Hospital de Valls, Tarragona, Spain.

José María Olmos, MD, Pediatric Department, Hospital Mare de Déu dels Lliris, Alicante, Spain.

Alejandro Cobo, MD, Pediatric Department, Hospital Universitario de Canarias, Santa Cruz de Tenerife, Spain.

Sara Díaz, MD, Pediatric Department, Hospital Universitario de Canarias, Santa Cruz de Tenerife, Spain.

Beatriz Jiménez, MD, Pediatric Department, Hospital Universitario Marqués de Valdecilla, Santander, Spain.

Raúl González, MD, Pediatric Department, Hospital Sant Joan d'Alacant, Alicante, Spain.

Miguel Lafuente, MD, Pediatric Department, Hospital Infantil de Zaragoza, Zaragoza, Spain.

Matilde Bustillo, MD, Pediatric Department, Hospital Infantil de Zaragoza, Zaragoza, Spain.

Natividad Pons, MD, Pediatric Department, Hospital Lluís Alcanyis, Xátiva, Spain.

Julia Morata, MD, Pediatric Department, Hospital Lluís Alcanyis, Xátiva, Spain.

Elsa Segura, MD, Pediatric Department, Hospital Universitario Son Llatzer de Palma de Mallorca, Palma de Mallorca, Spain.