Abstract
To assist in the clinical management of patients and to support infection control, we tested the use of the severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2) point‐of‐care antigen test (AgPOC) for unplanned hospitalization, coupled with a nucleic acid amplification test (NAAT) using specimens collected at the same time upon arrival. The aim of this study was to assess the performance of the AgPOC in this specific use compared to NAAT for SARS‐CoV‐2 diagnosis, in the context of the low prevalence of infection. For 5 months (between two peaks in France of the SARS‐CoV‐2 pandemic), all patients admitted who undertook the AgPOC/NAAT paired tests were included in the study. AgPOC performances were determined considering the clinical status and the delay of symptoms onset. NAAT and AgPOC results were available for 4425 subjects. AgPOC results showed a homogeneous specificity (>97%) but a low sensitivity at 45.8%. Considering the national guidelines, sensitivity dropped to 32.5% in cases of symptomatic patients with symptoms older than 5 days or more. This study shows the poor performance of AgPOC for entry screening of patients in hospitals. AgPOC may represent a useful tool in the hospital setting only if the use is restricted to patients with consistent symptoms less than 4 days old.
Keywords: hospital screening, point‐of‐care test, SARS‐CoV‐2, SARS‐CoV‐2 antigen
Highlights
-
–
SARS‐CoV‐2 point‐of‐care antigen tests (AgPOC) are poorly evaluated to screen patients for unplanned hospitalization.
-
–
Performance of AgPOC showed low sensitivity in hospital setting.
-
–
Use of AgPOC in hospital should be restricted to patients with consistent symptoms less than 4 days old.
1. INTRODUCTION
The nucleic acid amplification test (NAAT) is currently the gold standard for diagnosing the presence of coronavirus disease 2019 (COVID‐19) infection and managing the spread of severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2). 1 It offers accurate and highly specific results with a high throughput of samples. However, this costly tool requires a medical laboratory having both specialized instruments and skilled staff with molecular expertise. By contrast, the Rapid SARS‐CoV‐2 antigen point‐of‐care test (AgPOC) is quite inexpensive and easy to perform; it gives a rapid result (in less than 20 min). However, a large quantity of published studies raises concerns about their use. AgPOC reported high specificity, but variable sensitivity was observed ranging from 0% to 98% and depending on the evaluation settings and the study design. 2 , 3 In light of this weakness, AgPOC are mainly used for large‐scale screening in different community settings (schools, nursing homes, and community screening centers).
There is a lack of data on the performance of the AgPOC test in hospital settings, thus raising a critical question: is the AgPOC a useful diagnostic tool in managing the SARS‐CoV‐2 pandemic crisis in the overcrowded hospital and in the context of a low prevalence of SARS‐CoV‐2 infection?
As suggested by the French national health authority, the AgPOC could be used in emergency care units when a rapid result is needed. 2 To help in the clinical management of patients and to support infection control, the use of the PanBio™ COVID‐19 Ag (Abbott diagnostics, France) for unplanned hospitalization, coupled with a NAAT assay (as a gold standard) is now performed at arrival at our hospital. To evaluate the performance of the PanBio™ COVID‐19 Ag in this specific use, we performed a monocentric diagnostic cross‐sectional retrospective study.
2. MATERIALS AND METHODS
2.1. Patients and samples
All patients admitted at the University Hospital of Angers from November 21, 2020 to February 28, 2021 and benefitting from AgPOC/NAAT paired tests were included in the study. During this period, we were between two peaks of the SARS‐CoV‐2 pandemic in France (incidence ranging from 78 to 180,3 confirmed cases/100 000 individuals). 4 Clinical data were collected from the medical records. Symptoms suggestive of COVID‐19 disease were: fever; respiratory signs like a cough and shortness of breath; headaches; muscle ache and unusual tiredness; anosmia and loss of taste; and altered states of consciousness.
Nasopharyngeal paired swabs were obtained from a single nostril of patients (one swab by nostril) by trained healthcare workers using universal transport media (UTM). The first swab was placed into a vial of 3 ml of UTM and immediately transferred to the laboratory of Virology to perform NAAT. The second swab was used to perform the AgPOC immediately in the care unit.
At the time of our study, we considered the national guidelines of the Haute autorité de Santé – the French National Health Authority – on the use of testing in community settings. 2 According to these guidelines, AgPOC should be used for the benefit of all symptomatic patients whatever their profile and within 4 days of the onset of symptoms (DAOS), and for all contact cases for the asymptomatic patient. After 5 DAOS, the NAAT should be used as the first line. We classified the symptomatic patients into two categories: ≤4 DAOS; and >5 DAOS).
2.2. Nucleic acid amplification test
The Aptima® SARS‐CoV‐2 assay on the platform Panther® (Hologic), which targets two virus sequences located on the ORF1ab gene, was performed following the manufacturer's instructions and uses TMA to detect SARS‐CoV‐2. A volume of 500 µl UTM was manually placed in the appropriate specimen lysis tube containing 710 µl of lysis buffer; 360 µl of this mix was then used for the lysis, capture, and amplification of nucleic acids.
2.3. Panbio™ COVID‐19 Ag rapid test
The Panbio™ assay was performed according to the manufacturer's instructions. Briefly, the swab was rotated five times against the nasal wall; the procedure was repeated with the same swab into the second nostril. It was then deposited in the extraction tube. Five drops of each extracted specimen were dispensed into the specimen well and the timer was started. The result was read after 15 min and recorded immediately in the patient's medical file.
2.4. Statistical analysis
Patients with missing clinical data, NAAT, or AgPOC were excluded from the analysis. Analytical performances of the Panbio™ COVID‐19 Ag were determined, using the Aptima® SARS‐CoV‐2 assay as the reference, with Graphpad Prism software 9.1. For each clinical condition (i.e., overall, asymptomatic, symptomatic, national guidelines, and >5 DAOS), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NVP) were determined, and confidence intervals were calculated with the Wilson–Brown method. For each parameter, data are presented with 95% confidence intervals (CI).
2.5. Ethics statement
The study was carried out in accordance with the Declaration of Helsinki. This study was a non‐interventional study, with no alterations of the usual sampling procedures. Biological material and clinical data were obtained only for standard viral diagnosis following physicians’ prescriptions (no specific sampling, no modification of the sampling protocol). Data analyses were carried out using an anonymized database.
3. RESULTS
A total of 4425 patients with results of both AgPOC and NAAT and were included in the study. These paired samples mainly originated from Emergency units, followed by the Medical units, Intensive care unit, Gynecological Emergency Unit, and Surgery Unit, as described in Figure 1. The sex ratio was 0.91 (2318 women and 2107 men). The median and mean ages were 52 and 62 years old, respectively (ranged from 14 to 101).
Figure 1.
Distribution of patients by clinical department
Table 1 shows the performance of AgPOC compared with NAAT by clinical subgroup and by national guidelines.
Table 1.
Performance of Panbio™ COVID‐19 Ag rapid test compared with NAAT gold standard method (Aptima® SARS‐CoV‐2 assay)
| Aptima® SARS‐CoV‐2 assay (NAAT) | |||
|---|---|---|---|
| Positive | Negative | Total | |
| Panbio™ COVID‐19 Ag rapid test results | |||
| Overall | |||
| Positive | 86 (1.9) | 33 (0.75) | 119 (2.7) |
| Negative | 102 (2.3) | 4204 (95) | 4306 (97.3) |
| Total | 188 (4.2) | 4237 (95.8) | 4425 (100) |
| Asymptomatic | |||
| Positive | 8 (0.2) | 16 (0.4) | 24 (0.7) |
| Negative | 20 (0.6) | 3445 (98.8) | 3465 (99.3) |
| Total | 28 (0.8) | 3461 (99.2) | 3489 (100) |
| Symptomatic | |||
| Positive | 78 (8.3) | 17 (1.8) | 95 (10.1) |
| Negative | 82 (8.8) | 759 (81.1) | 841 (89.9) |
| Total | 160 (17.1) | 776 (82.9) | 936 (100) |
| ≤4 DAOS (National guidelines) | |||
| Positive | 52 (7.5) | 13 (1.9) | 65 (9.4) |
| Negative | 28 (4.1) | 597 (86.5) | 625 (90.6) |
| Total | 80 (11.7) | 610 (88.4) | 690 (100) |
| >5 DAOS | |||
| Positive | 26 (10.6) | 4 (1.6) | 30 (12.2) |
| Negative | 54 (21.9) | 162 (65.9) | 216 (87.8) |
| Total | 80 (32.5) | 166 (67.6) | 246 (100) |
| Panbio™ COVID‐19 Ag rapid test performance, % (95% CI) | |||
| Overall (n = 4425) | |||
| Sensitivity | 45.8 (38.8–52.9) | ||
| Specificity | 99.2 (98.9–99.4) | ||
| PPV | 72.3 (63.6–79.5) | ||
| NPV | 97.6 (97.1–98) | ||
| Asymptomatic (n = 3489) | |||
| Sensitivity | 28.6 (15.2–47.1) | ||
| Specificity | 99.5 (99.3–99.8) | ||
| PPV | 33.3 (17.9–53.3) | ||
| NPV | 99.4 (99.1–99.6) | ||
| Symptomatic (n = 936) | |||
| Sensitivity | 48.8 (41.1–56.4) | ||
| Specificity | 97.8 (96.5–98.6) | ||
| PPV | 82.11 (73.2–88.5) | ||
| NPV | 90.3 (88.1–92.1) | ||
| ≤4 DAOS = National guidelines (n = 690) | |||
| Sensitivity | 65.0 (54.1–74.6) | ||
| Specificity | 97.9 (96.4–98.8) | ||
| PPV | 80.0 (68.7–87.9) | ||
| NPV | 95.5 (93.6–96.9) | ||
| >5 DAOS (n = 246) | |||
| Sensitivity | 32.5 (23.2–43.4) | ||
| Specificity | 97.6 (94–99.1) | ||
| PPV | 86.7 (70.3–94.7) | ||
| NPV | 75 (68.8–80.3) | ||
Note: All data are described in absolute (n) and percentage (%) values.
Abbreviations: CI, confidence interval; DAOS, days after the onset of symptoms; PPV, predictive positive value; NPV, negative predictive value.
Out of 4425 patients, 936 (21.2%) were symptomatic and 3489 (78.8%) were asymptomatic. According to NAAT results, 4.6% of patients (188/4425) had a positive result for SARS‐CoV‐2. The prevalence of SARS‐CoV‐2 infection in symptomatic and asymptomatic subgroups were 17.1% and 0.8%, respectively. Overall, 86 patients (1.9%) had a positive result, and 4204 patients (95.0%) had a negative result for both methods, showing discordant results (NAAT+/AgPOC− and NAAT−/AgPOC+) in 102 patients (2.3%) and 33 patients (0.75%), respectively. False‐positive antigen results were observed in symptomatic, asymptomatic, national guidelines and >5 DAOS groups, 1.82%, 0.4%, 1.9%, and 1.6%, respectively. False‐negative antigen results were also observed in symptomatic, asymptomatic, national guidelines and >5 DAOS groups, 8.8%, 0.6%, 4.1%, and 21.9%, respectively.
AgPOC results showed a homogeneous specificity (more than 97%) just above the required performance by the World Health Organization (WHO) 5 as described in Table 1.
Overall, we observed a low sensitivity (45.8%). Sensitivity for symptomatic and asymptomatic subgroups were 48.8% and 28.6%, respectively. Considering the national guidelines, that is, using AgPOC for all symptomatic patients within 4 DAOS, sensitivity increased substantially (65%). In cases of symptomatic patients with symptoms older than 5 days or more, sensitivity was 32.5%. NPV were good for all subgroups (more than 90%, except for >5 DAOS). PPV was also good for symptomatic, national guidelines, and >5 DAOS subgroups. However, PPV for overall patients was 72.3% and dropped to 33.3% in the asymptomatic subgroup.
4. DISCUSSION
The significance of using the SARS‐CoV‐2 AgPOC in hospitals remains unclear. It seems to be a useful tool for quickly identifying infected patients and it may improve the clinical management of the patient and prevent the spread of the infection within the hospital. But our study questioned the place that it has been accorded in the medical diagnostic landscape when performed at hospitals.
The present work is one of the first studies conducted in a hospital setting that focuses on patients who need to be hospitalized but are not restricted to the emergency unit. A major strength of our study is that more than 4400 patients were included, representing one of the largest studies to date in a hospital setting within a low prevalence context. Moreover, the study was performed in real‐life conditions; it involved highly experienced testing personnel, and the well‐functioning logistics and standardized procedures enabled us to adequately evaluate AgPOC performance.
As reported in a number of studies covering a variety of study populations, we also reported a high specificity of the Panbio™ assay when compared with NAAT. In contrast to other studies, however, the overall sensitivity was low just at 45.8%. 6 , 7 It was weaker than reported in most published studies 2 , 3 , 8 , 9 and below the required performance level, 5 which is unacceptable especially in a hospital setting. However, data on the sensitivity were extracted from studies that vary in design, population, sample type, and especially prevalence (ranging from 5% to 100%). 2 , 3 In contrast to our work, in most studies AgPOC tests are collected on nasal swabs by non‐trained people or self‐collection, which perform considerably worse than nasopharyngeal taken by trained healthcare workers as underlined by Lee et al. 10 (around 15% of difference in the percentage positive detection rate between NP vs. nasal swab). This suggests that our sensitivity is even better than expected if the study was carried out with nasopharyngeal molecular methods vs AgPOC with nasal. A similarly low sensitivity was observed regarding symptomatic and asymptomatic subgroups in our study. The low sensitivity in the asymptomatic subgroup was also observed in other studies 3 , 11 , 12 , 13 , 14 and this lack of sensitivity of rapid antigen diagnostic tests has been known for many years for the influenza virus. 15 The high NPV and low PPV observed in the asymptomatic subgroup can be explained by the low prevalence as previously highlighted. 7 , 13 , 16 Thus, in asymptomatic patients, SARS‐CoV‐2 AgPOC does not currently appear sufficient for properly identifying the infected patients due to a significant risk of false positives, resulting in inadequate diagnosis and inappropriate decisions for the patient. A positive antigenic test in a patient with low clinical suspicion should therefore be confirmed by RT‐PCR.
In contrast, a negative AgPOC in patients with low clinical suspicion cannot completely exclude the presence of SARS‐CoV‐2 infection. Due to the high contagiousness of the SARS‐CoV‐2, a single false‐negative result can have serious consequences for the spread of the infection in a hospital unit.
Subsequently, AgPOC's performance appeared to be better in patients with high viral loads (cycle threshold values <25), 3 which usually occurs during the presymptomatic period (<3 DAOS with a sensitivity close to 80%) 11 and the early symptomatic phases of the disease (within the first week after symptom onset), 6 , 17 but not beyond that time because of an unacceptable sensitivity. 11 , 18 We, therefore, considered the French national guidelines to assess whether they improve the performance of AgPOC in hospital settings. Even if sensitivity increased substantially to 65% in symptomatic patients with a symptom duration ≤4 days, this level remains low and below WHO requirements. 5 Moreover, in cases of symptomatic patients with symptoms older than 5 days or more, sensitivity dropped drastically to 32.5% with a significant number of false‐negative results. Caramello et al. also found that false‐negative patients were tested later after the symptom onset with a median of 6 days 16 and a recent meta‐analysis showed the usefulness of these tests when they are used for patients within 5 days of symptom onset. 3 Taken as a whole, these previous studies and our own have shown that AgPOC sensitivity is inversely correlated with symptom duration. Our results support fears about the widespread use of AgPOC, especially in hospitals settings as previously raised by a small number of other authors. 7 , 19 Considering this weakness, and despite their ease of use, their rapidity, and their low cost in comparison with molecular assays, AgPOC use should be reassessed. Use in the hospital setting should be subject to a restrictive algorithm that takes into account the symptoms and the time to onset of clinical symptoms. In this light, we have demonstrated the added value of AgPOC in hospital settings only for unplanned hospitalization in patients who have been symptomatic for less than 5 days. This diagnostic tool should therefore be used carefully in the hospital setting.
CONFLICT OF INTERESTS
The authors declare that there are no conflict of interests.
AUTHOR CONTRIBUTIONS
Élise Bouthry and Hélène Le Guillou‐Guillemette conceptualized the study. Steven Roger, Élise Bouthry, and Dominique Savary collected the data. Steven Roger and Adeline Pivert conducted the data analyses. Steven Roger and Hélène Le Guillou‐Guillemette wrote the first draft of the manuscript. Caroline Lefeuvre, Élise Bouthry, Alexandra Ducancelle, and Dominique Savary revised the manuscript. All authors approved the final version.
Roger S, Lefeuvre C, Pivert A, et al. What is the true place of the SARS‐CoV‐2 rapid point‐of‐care antigen test in the hospital setting? Lessons learned from real life. J Med Virol. 2022;94:1723‐1727. 10.1002/jmv.27505
DATA AVAILABILITY STATEMENT
The data that support the findings of this study are available from the corresponding author upon reasonable request.
REFERENCES
- 1. Zhu N, Zhang D, Wang W, et al. A novel coronavirus from patients with pneumonia in China, 2019. N Engl J Med. 2020;382(8):727‐733. 10.1056/NEJMoa2001017 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Suzie D. Revue rapide sur les tests de détection antigénique du virus SARS‐CoV‐2. Éval Technol Santé. 2020:42. [Google Scholar]
- 3. Lee J, Song J‐U, Shim SR. Comparing the diagnostic accuracy of rapid antigen detection tests to real time polymerase chain reaction in the diagnosis of SARS‐CoV‐2 infection: a systematic review and meta‐analysis. J Clin Virol. 2021;144:104985. 10.1016/j.jcv.2021.104985 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Coronavirus: chiffres clés et évolution de la COVID‐19 en France et dans le Monde. 2021. https://www.santepubliquefrance.fr/dossiers/coronavirus-covid-19/coronavirus-chiffres-cles-et-evolution-de-la-covid-19-en-france-et-dans-le-monde
- 5. World Health Organization . Antigen‐detection in the diagnosis of SARS‐CoV‐2 infection using rapid immunoassays. Published. 2020. Accessed August 16, 2021. https://www.who.int/publications-detail-redirect/antigen-detection-in-the-diagnosis-of-sars-cov-2infection-using-rapid-immunoassays
- 6. Scohy A, Anantharajah A, Bodéus M, Kabamba‐Mukadi B, Verroken A, Rodriguez‐Villalobos H. Low performance of rapid antigen detection test as frontline testing for COVID‐19 diagnosis. J Clin Virol. 2020;129:104455. 10.1016/j.jcv.2020.104455 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Osterman A, Baldauf H‐M, Eletreby M, et al. Evaluation of two rapid antigen tests to detect SARS‐CoV‐2 in a hospital setting. Med Microbiol Immunol (Berl). 2021;210(1):65‐72. 10.1007/s00430-020-00698-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Albert E, Torres I, Bueno F, et al. Field evaluation of a rapid antigen test (Panbio™ COVID‐19 Ag Rapid Test Device) for COVID‐19 diagnosis in primary healthcare centres. Clin Microbiol Infect. 2021;27(3):472.e7‐472.e10. 10.1016/j.cmi.2020.11.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Loconsole D, Centrone F, Morcavallo C, et al. The challenge of using an antigen test as a screening tool for SARS‐CoV‐2 infection in an emergency department: experience of a tertiary care hospital in Southern Italy. BioMed Res Int. 2021;2021:1‐7. 10.1155/2021/3893733 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Lee RA, Herigon JC, Benedetti A, Pollock NR, Denkinger CM. Performance of saliva, oropharyngeal swabs, and nasal swabs for SARS‐CoV‐2 molecular detection: a systematic review and meta‐analysis. J Clin Microbiol. 2021;59(5):e02881‐20. 10.1128/JCM.02881-20 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Fourati S, Audureau É, Chevaliez S, Pawlotsky J‐M. Évaluation de la performance diagnostique des tests rapides d'orientation diagnostique antigéniques COVID‐19. 2020. Accessed July 21, 2021. https://www.aphp.fr/contenu/evaluation-de-la-performance-diagnostique-des-tests-rapides-dorientation-diagnostique
- 12. Ferté T, Ramel V, Cazanave C, et al. Accuracy of COVID‐19 rapid antigenic tests compared to RT‐PCR in a student population: the StudyCov study. J Clin Virol. 2021;141:104878. 10.1016/j.jcv.2021.104878 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Bianco G, Boattini M, Barbui AM, et al. Evaluation of an antigen‐based test for hospital point‐of‐care diagnosis of SARS‐CoV‐2 infection. J Clin Virol. 2021;139:104838. 10.1016/j.jcv.2021.104838 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Turcato G, Zaboli A, Pfeifer N, et al. Clinical application of a rapid antigen test for the detection of SARS‐CoV‐2 infection in symptomatic and asymptomatic patients evaluated in the emergency department: A preliminary report. J Infect. 2021;82(3):e14‐e16. 10.1016/j.jinf.2020.12.012 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Green DA, StGeorge K. Rapid antigen tests for influenza: rationale and significance of the FDA reclassification. J Clin Microbiol. 2018;56(10):e00711‐18. 10.1128/JCM.00711-18 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Caramello V, Boccuzzi A. Basile V, et al. Are antigenic tests useful for detecting SARS‐CoV‐2 infections in patients accessing to emergency departments? Results from a North‐West Italy hospital. J Infect. 2021;83(2):237‐279. 10.1016/j.jinf.2021.05.012 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Mertens P, De Vos N, Martiny D, et al. Development and potential usefulness of the COVID‐19 Ag Respi‐Strip diagnostic assay in a pandemic context. Front Med. 2020;7:225. 10.3389/fmed.2020.00225 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Linares M, Pérez‐Tanoira R, Carrero A, et al. Panbio antigen rapid test is reliable to diagnose SARS‐CoV‐2 infection in the first 7 days after the onset of symptoms. J Clin Virol. 2020;133:104659. 10.1016/j.jcv.2020.104659 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Pettengill MA, McAdam AJ. Can we test our way out of the COVID‐19 pandemic? J Clin Microbiol. 2020;58(11):e02225‐20. 10.1128/JCM.02225-20 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The data that support the findings of this study are available from the corresponding author upon reasonable request.