Abstract
Screening, testing and contact tracing plays a pivotal role in control of the COVID-19 pandemic. To enable this it is necessary to increase the testing capacity. This study compared a SARS-CoV-2 rapid antigen test (RAT) and RT-PCR in 842 asymptomatic individuals from Tarapacá, Chile. A sensitivity of 69.86%, specificity of 99.61%, PPV of 94.44% and NPP of 97.22% with Ct values (Ct > 27) that were significantly higher among individuals with false-negative RAT were reported. These results support the fact that RAT might have a significant impact on the identification of asymptomatic carriers in areas that lack suitable laboratories to perform SARS-CoV-2 real-time RT-PCR diagnostics, or the results take more than 24–48 h, as well as zones with high traffic of individuals such as border/customs, airports, interregional bus, train stations or in any mass testing campaign requiring rapid results.
Keywords: Rapid antigen test, RAT, SARS-CoV-2, RT-PCR, Asymptomatic
Introduction
Given the increase in cases of SARS-CoV-2 infections worldwide, there is a need for a reliable rapid diagnostic test in addition to existing gold standard real-time RT-PCR. Rapid antigen tests (RAT) for SARS-CoV-2 can be performed onsite in mass testing, are inexpensive compared to real-time RT-PCR, do not require specific and expensive equipment, and the results are available within 15 min (CDC, 2021), which could serve to evaluate chains of infection and their interruption. A recent meta-analysis revealed that the average sensitivity and specificity of RAT for SARS-CoV-2 were 56.2% and 99.5%, respectively (Dinnes et al., 2020). To date, most of these validations were carried out in symptomatic individuals or using previously collected samples (Cerutti et al., 2020, Kruttgen et al., 2021, Porte et al., 2020, Weitzel et al., 2020, Yamayoshi et al., 2020). In contrast, onsite test validation studies in asymptomatic individuals, to support the use of RAT in mass testing and epidemiological surveillance, are limited (Jakobsen et al., 2021, Mina et al., 2020, Pollock et al., 2021, Schildgen et al., 2021, Toptan et al., 2021). This study performed a mass comparison of RAT and real-time RT-PCR test in asymptomatic individuals from a Chilean region.
Materials and methods
Sample collection
Sample collection was coordinated by a specialized team from SEREMI de Salud Tarapacá. Two nasopharyngeal swabs (NSS) samples from asymptomatic individuals were collected by healthcare workers at Iquique city, Tarapacá Region, Chile between 14–17 January 2021, where the prevalence of SARS-CoV-2 infection was 11% according to data provided by the Chilean Ministry of Health (Minsal, 2021). Taking into consideration that the expected prevalence of positive cases for COVID-19 in asymptomatic individuals can vary from 8–12% and establishing a sampling error of 1.5% and a type I error of 5%, the minimum sample size required was 864 ± 69.7. All participants completed a questionnaire and provided information on demographic characteristics, current and past (14 days) symptoms, and recent exposure to people with COVID-19 (Table 1 ). One swab was immediately tested at the facility using the SARS-CoV-2 RAT (SD Biosensor, Inc. Republic of Korea) and the result was interpreted according to the manufacturer’s guidelines. The second swab was preserved in specimen transport medium and transported at 4 °C to Laboratorio Médico Bioclinic and Hospital Regional de Iquique for RNA extraction and RT-PCR testing.
Table 1.
Characteristics of individuals providing paired NSS (n = 842) by results for SARS-CoV-2 real-time reverse transcription-polymerase chain reaction (RT-PCR) and Rapid Antigen Test (RAT) – January 2021.
| Characteristic |
True positives (n = 51, %) | False negatives (n = 22, %) | False positives (n = 3, %) | True negatives (n = 7 66, %) | Total (n = 842, %) | ||
|---|---|---|---|---|---|---|---|
| Testing Site | Context | Date | |||||
| A | Workers | January 14 | 1 | 55 | 56 (6.7%) | ||
| B | Sanitary residence | January 14 | 51 | 51 (6.1%) | |||
| C | Sanitary residence | January 14 | 1 | 29 | 30 (3.6%) | ||
| D | General public | January 15 | 13 | 5 | 193 | 211 (25.1%) | |
| E | Sanitary residence | January 15 | 9 | 2 | 2 | 145 | 158 (18.8%) |
| F | General public | January 16 | 23 | 14 | 1 | 210 | 248 (29.5%) |
| G | General public | January 17 | 5 | 83 | 88 (10.5%) | ||
| Total | 51 (6.05%) | 22 (2.61%) | 3 (0.35%) | 766 (91%) | 842 (100%) | ||
| Sex | |||||||
| Male | 27 | 9 | 1 | 392 | 429 (51.0%) | ||
| Female | 23 | 13 | 2 | 313 | 351 (41.7%) | ||
| #N/D | 1 | 61 | 62 (7.4%) | ||||
| Total | 51 (6.05%) | 22 (2.61%) | 3 (0.35%) | 766 (91%) | 842 (100%) | ||
| Nationality | |||||||
| Bolivia | 3 | 18 | 21 (2.5%) | ||||
| Brazil | 1 | 1 (0.1%) | |||||
| Chile | 37 | 19 | 1 | 476 | 533 (63.3%) | ||
| China | 1 | 1 (0.1%) | |||||
| Colombia | 11 | 11 (1.3%) | |||||
| Cuba | 4 | 4 (0.5%) | |||||
| Ecuador | 2 | 2 (0.2%) | |||||
| Paraguay | 1 | 1 (0.1%) | |||||
| Perú | 10 | 10 (1.2%) | |||||
| Venezuela | 10 | 3 | 2 | 171 | 186 (22.1%) | ||
| #N/D | 1 | 71 | 72 (8.6%) | ||||
| Total | 51 (6.05%) | 22 (2.61%) | 3 (0.35%) | 766 (91%) | 842 (100%) | ||
N/D: no data.
Rapid antigen test
The SD Biosensor, Inc. Antigen Test (Republic of Korea, Catalog number 9901-NCOV-01G) is a rapid lateral flow immunoassay for the qualitative detection of SARS-CoV-2-specific antigens present in the human nasopharynx, with sensitivity of 96.52% and specificity of 99.68% (SARS-CoV-2 Rapid Antigen Test Package Insert 2020-08, V 1.0). According to the manufacturer, the results are available within 30 min and all necessary reagents to perform the assay are provided. The assay kits are stable when stored at 2–30 °C.
RT-PCR
Viral RNA was extracted using the Mag-Bind Viral DNA/RNA 96 kit (Omega Bio-Tek, Catalog number M6246) on the Kingfisher Flex Magnetic Particle Processor (Thermo Fisher Scientific). Real-time RT-PCR was performed at Laboratorio Médico Bioclinic and Hospital Regional de Iquique using the GenomeCov19 Detection Kit ABM (Applied Biological Materials Inc, Canada, Catalog number G628.v2), with cycle threshold (Ct) values ≤ 40 considered positive for the N and S viral gene regions.
Statistical analysis
Statistical analysis considered sensitivity, specificity, Positive Predictive Value (PPV) and Negative Predictive Value (NPV), accuracy, Kappa coefficient, and Wilson score Confidence Interval at 95% (GraphPad Prism version 9.0.1).
Results
This study evaluated the performance of the SARS-CoV-2 RAT (SD Biosensor, Inc. Republic of Korea) compared with the real-time RT-PCR for SARS-CoV-2 detection among asymptomatic individuals at Iquique city, Tarapacá Region, Chile. The sampling was carried out in seven testing sites corresponding to i) workers (n = 56; 6.7%), ii) sanitary residence (n = 239; 28.4%) and iii) general public (n = 547; 65%). A total of 854 individuals were included (mean age: 36.67 years; SD: 16.48 years; males: 51%; females: 41.6%; N/A: 7.4%). Two NSS samples from each individual were collected by healthcare workers at testing sites. Among a total of 854 NSS submitted, 12 (1.4%) were excluded for lacking real-time RT-PCR results. Among 842 paired NSS from asymptomatic individuals, 54 (6.17%) were antigen-positive and 73 (8.6%) were real-time RT-PCR-positive. Antigen testing sensitivity was 69.86% (51 of 73), specificity was 99.61% (766 of 769), PPV was 94.44% (51 of 54), and NPV was 97.22% (766 of 789). Three paired (0.35%) NSS were antigen-positive and real-time RT-PCR-negative. Accuracy between the two techniques was 97.04% (Kappa coefficient = 0.78, 95% CI: 0.70–0.86) (Table 2 ). Given that the Ct value is inversely related to the viral load (Rao et al., 2020), the PCR Ct value data of the 73 samples that tested positive for NSS SARS-CoV-2 real-time RT-PCR detection were reviewed: Ct values were significantly higher among individuals with false-negative RAT (viral gene N: 28.07 ± 4.343; viral gene S: 28.81 ± 3.873) compared with true positives (viral gene N: 19.99 ± 4.535; viral gene S: 20.93 ± 4.487) (Figure 1A). The k-means clustering analysis was then reviewed to find clusters in an iterative way using the Ct value of viral gene N (Cerutti et al., 2020). Three clusters were identified: c1 = 16.87 (n = 31, strongly positive), c2 = 24.5 (n = 30, moderately positive) and c3 = 31.67 (n = 12, weakly positive), where 96.77% (30/31) of the samples with low Ct value also tested positive for RAT (Figure 1B, cluster 1), while 66.6% (20/30) of the samples with medium Ct value also tested positive for RAT (Figure 1B, cluster 2). In contrast, 8.33% (1/12) of the samples with high Ct value tested positive for RAT (Figure 1B, cluster 3). Three RATs-positive and SARS CoV-2 real-time RT-PCR-negative samples were identified (Table 1).
Table 2.
Agreement between RT-PCR test results and antigen test results overall.
| January 2021 – Tarapacá Region | |||
|---|---|---|---|
| RT-PCR (+) | RT-PCR (−) | Total | |
| RAT (+) | 51 | 3 | 54 |
| RAT (−) | 22 | 766 | 788 |
| Total | 73 | 769 | 842 |
| Analytic parameters | Value | 95% CI |
|---|---|---|
| Sensitivity | 69.86% | 58.56–9.18% |
| Specificity | 99.61% | 98.86–99.87% |
| Positive Predictive Value | 94.44% | 84.89–98.09% |
| Negative Predictive Value | 97.21% | 95.81–98.15% |
| Accuracy | 97.04% | 95.66–98.08% |
| Kappa | SE of kappa | 95% CI |
| 0.787 | 0.041 | 0.707–0.868 |
Figure 1.
(A) Difference in viral cycle threshold (Ct) value between asymptomatic individuals with positive (black) and negative (red) RAT among real-time RT-PCR-positive (n = 73). Analysis for statistical difference was performed by Wilcoxon test. (B) Plot of K-means clustering results of NSS real-time RT-PCR Ct values used to compare to RAT (black dots: RAT+/red dots: RAT−).
Discussion
Improvements in SARS-CoV-2 diagnosis with easy, rapid and cost-efficient approaches are urgently required to control the COVID-19 pandemic. As of April 2021, more than 170 assays are on the market (FINDdx, 2020) and few have been extensively validated. Recently, the second iteration of a Cochrane living review summarized the accuracy of multiple RATs. The average sensitivity reported in symptomatic individuals from 37 evaluations was 72.0% (95% CI: 63.7–79.0%), while that in asymptomatic individuals from 12 evaluations was 58.1% (95% CI: 40.2–74.1%) (Dinnes et al., 2021). The SD Biosensor RAT (Inc., Republic of Korea) manufacturer reported a higher sensitivity (96.52%; 95% CI: 91.33–99.04%) obtained in prospective, randomized, single blinded studies conducted in Brazil and India in symptomatic and asymptomatic individuals (SARS-CoV-2 Rapid Antigen Test Package Insert 2020-08, V 1.0). In addition, a mass screening campaign in Catalonia (North-East Spain) evaluated four RATs in a cohort of 286 asymptomatic individuals. The sensitivity of each was 38.6% for Abbott assay, 51.5% for Siemens, 45.5% for Lepu and 43.6% for Roche (the same used in this study), with 83.3% of specimens with Ct < 30 (Baro et al., 2021).
This study comprised asymptomatic individuals in three different contexts with an 8.64% prevalence of SARS-CoV-2 infection. In agreement with the Centers for Disease Control and Prevention (CDC, 2021) recommendation on the use of antigen testing, the sensitivity of 69.86% indicates that RAT should not replace real-time RT-PCR in diagnosis and surveillance of SARS-CoV-2 infection (CDC, 2021). However, the PPV of antigen testing was 94.44%, indicating that asymptomatic persons with positive antigen results are infected with SARS-CoV-2 and would not require confirmatory real-time RT-PCR. Likewise, the NPV of antigen testing was 97.21%, indicating that asymptomatic individuals with negative antigen results are unlikely to be infected with SARS-CoV-2. Furthermore, individuals with false-negative results of the RAT had significantly higher Ct values (Ct > 27), which can be related to lower viral loads and less infectiousness in general (Bullard et al., 2020, Singanayagam et al., 2020). Given the high predictive values in asymptomatic individuals and the fast test result implying faster tracing of infected individuals, these results support and provide policymakers with evidence that RAT might have a significant role in COVID-19 screening, testing and contact tracing strategies to control the COVID-19 pandemic in i) areas that lack suitable laboratories to perform SARS-CoV-2 real-time RT-PCR diagnostics, ii) areas where results take more than 24–48 h and iii) areas with high traffic of individuals such as border/customs, airports, interregional bus and train stations or in any mass testing campaign requiring rapid results.
Author contributions
FV-E, MJF, SS, PG, MSV, and JA participated in the study design. MP, MA and CG performed the experiments. RL provided the NSS from COVID-19 asymptomatic individuals. FP performed the statistical analysis. FV-E, AG, SS, and RS-R wrote the manuscript. MSV and JA performed funding acquisition. All authors approved the final version of the manuscript.
Conflict of interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Funding source
We wish to thank Ministerio de Salud de Chile for funding this study. The authors also are supported by ANID Chile through Fondecyt grants Nº 1181656 (AG), 1190156 (RS-R) and 1180798/1211547 (FV-E).
Ethical statement
Ministerio de Salud de Chile instructed this study to implement the use of the RAT in asymptomatic individuals. Therefore, this work is considered as a public health intervention to improve diagnosis. The study described here was approved by the Ethics Committee of the Faculty of Medicine at Universidad de Chile (Project Nº 036-2020). NSS from COVID-19 positive and negative patients were anonymized.
Acknowledgments
We gratefully acknowledge all the health services, hospitals and local health authorities that participated in the study, Manuel Fernández (Seremi de Salud Tarapacá), Dolores Romero (Jefa de Salud Pública, Seremi de Salud), Jorge Galleguillos (Director Servicio de Salud Iquique), Carlos Calvo (Sub-Director Médico Servicio de Salud Iquique), and healthcare workers from Departamento administración de Salud – Municipalidad de Iquique and Municipalidad de Alto Hospicio.
References
- Baro B., Rodo P., Ouchi D., Bordoy A.E., Saya Amaro E.N., Salsench S.V., et al. Performance characteristics of five antigen-detecting rapid diagnostic test (Ag-RDT) for SARS-CoV-2 asymptomatic infection: a head-to-head benchmark comparison. J Infect. 2021;82(6):269–275. doi: 10.1016/j.jinf.2021.04.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bullard J., Dust K., Funk D., Strong J.E., Alexander D., Garnett L., et al. Predicting Infectious Severe Acute Respiratory Syndrome Coronavirus 2 From Diagnostic Samples. Clin Infect Dis. 2020;71(10):2663–2666. doi: 10.1093/cid/ciaa638. [DOI] [PMC free article] [PubMed] [Google Scholar]
- CDC . CDC; 2021. Coronavirus disease 2019 (COVID-19): interim guidance for antigen testing for SARS-CoV-2. Revised April 13, 2021. [Google Scholar]
- Cerutti F., Burdino E., Milia M.G., Allice T., Gregori G., Bruzzone B., et al. Urgent need of rapid tests for SARS CoV-2 antigen detection: evaluation of the SD-biosensor antigen test for SARS-CoV-2. J Clin Virol. 2020;132:104654. doi: 10.1016/j.jcv.2020.104654. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dinnes J., Deeks J.J., Adriano A., Berhane S., Davenport C., Dittrich S., et al. Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection. Cochrane Database Syst Rev. 2020;8(8) doi: 10.1002/14651858.CD013705. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dinnes J., Deeks J.J., Berhane S., Taylor M., Adriano A., Davenport C., et al. Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection. Cochrane Database Syst Rev. 2021;3(3) doi: 10.1002/14651858.CD013705.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- FINDdx . FINDdx; 2020. FIND. SARS-COV-2 diagnostic pipeline. Revised April 13, 2021. [Google Scholar]
- Jakobsen K.K., Jensen J.S., Todsen T., Lippert F., Martel C.J.-M., Klokker M., et al. Detection of SARS-CoV-2 infection by rapid antigen test in comparison with RT-PCR in a public setting. medRxiv. 2021 doi: 10.1101/2021.01.22.21250042. [DOI] [Google Scholar]
- Kruttgen A., Cornelissen C.G., Dreher M., Hornef M.W., Imohl M., Kleines M. Comparison of the SARS-CoV-2 rapid antigen test to the real star Sars-CoV-2 RT PCR kit. J Virol Methods. 2021;288 doi: 10.1016/j.jviromet.2020.114024. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mina M.J., Parker R., Larremore D.B. Rethinking Covid-19 test sensitivity - a strategy for containment. N Engl J Med. 2020;383(22):e120. doi: 10.1056/NEJMp2025631. [DOI] [PubMed] [Google Scholar]
- Minsal D. 2021. Reporte Diario Coronavirus - 23 enero. [Google Scholar]
- Pollock N.R., Jacobs J.R., Tran K., Cranston A., Smith S., O’Kane C., et al. Performance and implementation evaluation of the Abbott BinaxNOW rapid antigen test in a high-throughput drive-through community testing site in Massachusetts. J Clin Microbiol. 2021;59(5):e00083–21. doi: 10.1128/JCM.00083-21. 2021.01.09.21249499. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Porte L., Legarraga P., Vollrath V., Aguilera X., Munita J.M., Araos R., et al. Evaluation of a novel antigen-based rapid detection test for the diagnosis of SARS-CoV-2 in respiratory samples. Int J Infect Dis. 2020;99:328–333. doi: 10.1016/j.ijid.2020.05.098. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rao S.N., Manissero D., Steele V.R., Pareja J. A systematic review of the clinical utility of cycle threshold values in the context of COVID-19. Infect Dis Ther. 2020;9(3):573–586. doi: 10.1007/s40121-020-00324-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schildgen V., Demuth S., Lusebrink J., Schildgen O. Limits and opportunities of SARS-CoV-2 antigen rapid tests: an experienced-based perspective. Pathogens. 2021;10(1):38. doi: 10.3390/pathogens10010038. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Singanayagam A., Patel M., Charlett A., Lopez Bernal J., Saliba V., Ellis J., et al. Duration of infectiousness and correlation with RT-PCR cycle threshold values in cases of COVID-19, England, January to May 2020. Euro Surveill. 2020;25(32):2001483. doi: 10.2807/1560-7917.ES.2020.25.32.2001483. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Toptan T., Eckermann L., Pfeiffer A.E., Hoehl S., Ciesek S., Drosten C., et al. Evaluation of a SARS-CoV-2 rapid antigen test: potential to help reduce community spread? J Clin Virol. 2021;135:1041713. doi: 10.1016/j.jcv.2020.104713. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Weitzel T., Legarraga P., Iruretagoyena M., Pizarro G., Vollrath V., Araos R., et al. Comparative evaluation of four rapid SARS-CoV-2 antigen detection tests using universal transport medium. Travel Med Infect Dis. 2020;39:101942. doi: 10.1016/j.tmaid.2020.101942. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Yamayoshi S., Sakai-Tagawa Y., Koga M., Akasaka O., Nakachi I., Koh H., et al. Comparison of rapid antigen tests for COVID-19. Viruses. 2020;12(12):1420. doi: 10.3390/v12121420. [DOI] [PMC free article] [PubMed] [Google Scholar]