Abstract
The purpose of this study was to assess the clinical performance of STANDARD F COVID-19 Ag FIA (SD Biosensor Inc., Gyeonggi-do, Republic of Korea), a rapid antigen detection test (RADT) for diagnosing SARS-CoV-2, in patients attended at the Emergency Department with signs or symptoms compatible with COVID-19 that had started in the last 5 days. The clinical performance of the antigen test was compared with RT-PCR, the reference standard. We included 663 specimens from non-repetitive patients. Clinical sensitivity and specificity were 84.0% (95% CI 76.1−89.7) and 99.6% (95% CI 98.5−99.9), respectively. The positive and negative predictive values were 98.1% (95% CI 92.7−99.7) and 96.4% (95% CI 94.4−97.7), respectively. The kappa index agreement between RT-PCR and the RADT was 0.89 (95% CI 0.84−0.93). We concluded that STANDARD F COVID-19 Ag FIA is an excellent first-line RADT method to diagnose symptomatic patients in the emergency department.
Keywords: SARS-CoV-2, Rapid antigen test, COVID-19, Emergency Department
1. Introduction
Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has rapidly spread worldwide becoming a great diagnostic challenge for microbiology laboratories [1]. An early and rapid SARS-CoV-2 detection has become mandatory for preventing the virus spread [2]. Although the RT-PCR is the gold-standard method to diagnose the infection, point of care (POC) methods remain a highly useful tool to speed up the diagnostic process [3,4].
Although there are POC systems based on RT-PCR with final results in less than 1 hour [5,6], its use to perform large screenings represent huge economical costs. On the contrary, rapid antigen detection tests (RADT) represent both, economic and fast POC methods to accelerate diagnosis. Nevertheless, these RADT have lower sensitivity and specificity than molecular-based diagnostics [7,8]. This loss in accuracy forces an evaluation in different real-life scenarios in order to assess their accuracy and utility.
The purpose of this study was to assess the clinical performance of STANDARD F COVID-19 Ag FIA (SD Biosensor Inc., Gyeonggi-do, Republic of Korea), an automated RADT based on immunofluorescence detection for diagnosing SARS-CoV-2. The evaluation was performed using nasopharyngeal swabs from patients attended in the Emergency Department with signs or symptoms compatible with COVID-19 that had started in the last 5 days.
2. Materials and methods
This prospective study was conducted from October 14, 2020 to November 18, 2020 in a single tertiary university hospital, Hospital Universitario Marqués de Valdecilla (HUMV), in Santander (Spain). Patients attended in the Emergency Department of HUMV with signs or symptoms compatible with COVID-19 that had started in the last 5 days were recruited. Demographic data were no collected.
Left-over nasopharyngeal swab specimens used firstly in routine RT-PCR-based diagnosis in the hospital were used in antigen test evaluation within 24 hours after collection. Specimens were kept at 4°C until testing. Virus transport and preservation media used for nasopharyngeal testing included Biocomma (Biocomma Limited, ShenZen, China) and DeltaSwab (Deltalab, Rubí, Spain). Both systems are nasopharyngeal flocked swabs with non-inactivated preservation virus transport medium and were employed interchangeably.
The clinical performance of the RADT STANDARD F COVID-19 Ag FIA was compared with RT-PCR, the reference method for SARS-CoV-2 diagnosis. The RT-PCR reagent used was VIASURE SARS-CoV-2 Real Time PCR Detection Kit (Certest Biotec, Zaragoza, Spain), based on amplification and detection of ORF1ab and N genes. The limit of detection of the RT-PCR was ≥10 copies of viral RNA for both genes.
Nucleic acid extraction was performed using automated systems as Biocomma Nucleic Acid Purification Kit (Biocomma Limited, ShenZen, China) and IndiMag Pathogen IM48 Cartridge (Indical Bioscience, Leipzig, Germany). Amplification was performed on a BioRad CFX96 thermocycler (BioRad Laboratories, the Netherlands). The cycle threshold (Ct) was automatically determined by the manufacturer's software. A positive RT-PCR result was defined as amplification of any of the 2 SARS-CoV-2 target genes, following manufacturer's instructions.
RADT was performed following manufacturer's instructions. Reading of the results was performed using a F200 or F2400 analyser (SD Biosensor Inc., Gyeonggi-do, Republic of Korea), which automatically reads the intensity of fluorescence following antibody–antigen complex formation. A sample is considered positive if cutoff index (COI) value is ≥1. The results obtained with the antigen test in this study did not impacted patient management and informed consent was not needed.
3. Results
A total of 663 specimens from non-repetitive patients were included in the study. In 125 specimens (18.9%) the RT-PCR result was positive, while the RADT detected 105 (15.8%). A negative result was obtained in 538 specimens (81.1%) by RT-PCR and in 558 (84.2%) by the RADT.
Clinical sensitivity and specificity of the RADT compared with RT-PCR were 84.0% (95% CI 76.1−89.7) and 99.6% (95% CI 98.5−99.9), respectively. The positive and negative predictive values were 98.1% (95% CI 92.7−99.7) and 96.4% (95% CI 94.4−97.7), respectively. The kappa index agreement between RT-PCR and the RADT was 0.89 (95% CI 0.84−0.93), which means an almost perfect agreement (Table 1 ).
Table 1.
Clinical performance of antigen test compared with RT-PCR.
| Sensitivity | 84.0 % | 95% CI (76.1−89.7) |
| Specificity | 99.6 % | 95% CI (98.5−99.9) |
| Positive predictive value | 98.1 % | 95% CI (92.7−99.7) |
| Negative predictive value | 96.4% | 95% CI (94.4−97.7) |
| Kappa index | 0.89 | 95% CI (0.84−0.93) |
CI = confidence interval.
Two false positive results (FP, 0.3%) were obtained with the RADT. Cutoff index values (COI) of FP were 1.22 and 1.69. Twenty false negatives results (FN, 3.0%) were found with the antigen test. In 16 out of 20 of FN the Ct obtained with the RT-PCR was ≥ 30 (Table 2 ).
Table 2.
Antigen test false negative results compared with RT-PCR (n = 20).
| Ct, Gene N | Ct, Gene ORF1a | COI |
|---|---|---|
| 32 | 34 | 0,07 |
| 36 | 35 | 0,59 |
| 25 | 23 | 0,87 |
| 29 | 30 | 0,08 |
| 37 | 36 | 0,24 |
| 27 | 24 | 0,33 |
| 30 | 29 | 0,66 |
| 35 | 34 | 0,26 |
| 34 | 34 | 0,17 |
| 33 | 34 | 0,06 |
| 36 | 40 | 0,19 |
| 28 | 26 | 0,57 |
| 36 | 35 | 0,02 |
| 34 | 34 | 0,23 |
| 27 | 24 | 0,34 |
| 36 | 38 | 0,25 |
| 32 | 29 | 0,08 |
| 31 | 30 | 0,12 |
| 38 | 37 | 0,31 |
| 33 | 31 | 0,08 |
Ct = cycle threshold. Target genes: N and ORF1a.
COI = cutoff index value.
Sensitivity of RADT according to Ct results of RT-PCR has also been analysed (Table 3 ). Sensitivity of the RADT when testing positive samples for SARS-CoV-2 with Ct ≤30, was 92.9%. In samples with Ct ≥31, sensitivity of the RADT was 7.7%.
Table 3.
Sensitivity of antigen test according to Ct results of RT-PCR.
| Patients, n | Sensitivity | |
|---|---|---|
| Ct ≤15 | 6 | 100% |
| Ct 16−20 | 47 | 100% |
| Ct 21−25 | 41 | 92.7% |
| Ct 26−30 | 18 | 72.2% |
| Ct ≥31 | 13 | 7.7% |
| Total | 125 | 84.0% |
Ct = cycle threshold.
For comparison purposes the lowest Ct value obtained between N or ORF1a genes was considered.
4. Discussion
Rapid detection of infected patients with SARS-CoV-2 is mandatory to interrupt transmission of the virus [2]. Rapid molecular methods based on RT-PCR offer results in less than 1 hour, however the price is not feasible for large-scale screening. For this reason, RADT represent an efficient alternative as first-line diagnostic method combining both advantages: low price and fast results. Notwithstanding, performance of RADT is highly dependent on the time elapsed since the onset of symptoms, since this accuracy is closely linked to the viral load [9]. Clinical evaluations in different scenarios are imperative in order to use the correct diagnostics in the correct time.
In this evaluation the antigen test STANDARD F COVID-19 Ag FIA presented an excellent clinical performance (84.0% sensitivity and 99.6% specificity) and an almost perfect agreement compared with the reference method, the RT-PCR, when used in patients with signs or symptoms compatible with COVID-19 that had started in the last 5 days. This finding is consistent with other RADT evaluations performed in symptomatic patients with similar clinical courses (onset of symptoms ≤5 days) [7,10].
The antigen test STANDARD F COVID-19 Ag FIA shows good clinical sensitivity (92.9%) in symptomatic patients with high viral loads (Ct ≤30). Meaning that this RADT is effective in cases of active viral replication, i.e., in the early acute phases of infection. As expected, sensitivity of the RADT decreases as the viral load in patients becomes lower (7.7% sensitivity; Ct ≥31). Similar findings have been reported previously [11,12]. The main reason of FN of RADT is a lack of sensitivity in cases with high Ct. In this evaluation; in the 80% of FN the Ct was ≥30.
Comparing our findings with other evaluations of the STANDARD F COVID-19 Ag FIA, there are disparities of results. Baccani et al, showed an overall sensitivity of 35.7%, while Orsi et al, of 86.7%. Both evaluations showed 100% sensitivity in samples with Ct <26 [13,14]. As seen in our study, time from the onset of symptoms is determinant for a correct use of RADT.
This evaluation has been done in a high pre-test probability scenario. During the study period (October 14 – November 18, 2020) the cumulative incidence rate per 100,000 inhabitants in the last 14 days, rise from 118 to 547 in the region of Cantabria. Together with a final high positive predictive value of the test, no confirmation of positive samples with RT-PCR was needed. However, in our experience, because of the 2 false positive results with antigen test (COI: 1.22 and 1.69), we decided to include in our internal diagnostic algorithm a confirmatory RT-PCR in samples with antigen test result with COI values between 1 and 2 in order to obtain 100% of specificity. This is consistent with a previous work showing a median COI value of 1.4 in samples performed with the same RADT and not confirmed to be positive by RT-PCR [15].
A further advantage of the STANDARD F COVID-19 Ag FIA is the automatic test interpretation of results, thus avoiding possible misinterpretations of RADT based on visual reading. Additionally, the analyser could automatically transmit the results to the laboratory information system, increasing the efficiency of the RADT. It is a rapid, inexpensive and reliable assay with a great value as a point-of-care-technique especially in settings as an Emergency department, where a short time of results is crucial for a better patient management.
An actual matter of concern is how SARS-CoV-2 new variants and mutations could affect the performance of COVID assays. As most RADT capture the viral nucleocapsid protein, and typically mutations affect in the Spike protein, mutations are not likely to limit the performance of RADT [16]. Nevertheless, since mutations in the nucleocapsid protein affecting performance of RADT have already been reported [17], continuous control of accuracy of diagnostic tests is mandatory.
A limitation of the study is that not information about cellularity of nasopharyngeal swabs was provided. We aware some of discrepant results could be explain with this data.
Based on these data, the antigen test STANDARD F COVID-19 Ag FIA is an excellent first-line method to diagnose symptomatic patients in both emergency departments and primary healthcare centres.
Author contributions
Sergio Garcia-Fernandez: Supervision, Conceptualization, Investigation, data curation, validation, writing original draft. Daniel Pablo-Marcos: Investigation, data curation, validation, review and editing of draft. Silvia Velasco de la Fuente: Investigation, data curation, validation, review and editing of draft. María José Reina Rodríguez: Investigation, data curation, validation, review and editing of draft. Mónica Gozalo: Investigation, data curation, validation, review and editing of draft. Jesús Rodríguez-Lozano: Investigation, data curation, validation, review and editing of draft. José Manuel Méndez-Legaza: Investigation, data curation, validation, review and editing of draft. Jorge Calvo: Supervision, conceptualization, Investigation, data curation, validation, review and editing of draft.
Declaration of competing interest
The authors report no conflicts of interest relevant to this article
Acknowledgments
Acknowledgments
We would like to acknowledge routine laboratory technicians for their assistance in the production of the data.
Funding
This study was supported by Plan Nacional de I+D+i 2013–2016, Instituto de Salud Carlos III, Subdirección General de Redes y Centros de Investigación Cooperativa, Ministerio de Economía y Competitividad, Spanish Network for Research in Infectious Diseases (REIPI RD16/0016/0007).
References
- 1.Wiersinga WJ, Rhodes A, Cheng AC, Peacock SJ, Prescott HC. Pathophysiology, transmission, diagnosis, and treatment of coronavirus disease 2019 (COVID-19): a review. JAMA. 2020;324:782–793. doi: 10.1001/jama.2020.12839. [DOI] [PubMed] [Google Scholar]
- 2.Kevadiya BD, Machhi J, Herskovitz J, Oleynikov MD, Blomberg WR, Bajwa N, et al. Diagnostics for SARS-CoV-2 infections. Nat Mater. 2021;20:593–605. doi: 10.1038/s41563-020-00906-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Corman VM, Landt O, Kaiser M, Molenkamp R, Meijer A, Chu DK, et al. Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR. Euro Surveill. 2020;25 doi: 10.2807/1560-7917.ES.2020.25.3.2000045. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Nelson PP, Rath BA, Fragkou PC, Antalis E, Tsiodras S, Skevaki C. Current and future point-of-care tests for emerging and new respiratory viruses and future perspectives. Front Cell Infect Microbiol. 2020;10:181. doi: 10.3389/fcimb.2020.00181. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Basu A, Zinger T, Inglima K, Woo K-M, Atie O, Yurasits L, et al. Performance of Abbott ID Now COVID-19 rapid nucleic acid amplification test using nasopharyngeal swabs transported in viral transport media and dry nasal swabs in a New York City Academic Institution. J Clin Microbiol. 2020;58 doi: 10.1128/JCM.01136-20. e01136-20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Loeffelholz MJ, Alland D, Butler-Wu SM, Pandey U, Perno CF, Nava A, et al. Multicenter evaluation of the cepheid xpert xpress SARS-CoV-2 test. J Clin Microbiol. 2020;58 doi: 10.1128/JCM.00926-20. e00926-20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Albert E, Torres I, Bueno F, Huntley D, Molla E, Fernández-Fuentes MÁ, et al. Field evaluation of a rapid antigen test (PanbioTM COVID-19 Ag Rapid Test Device) for COVID-19 diagnosis in primary healthcare centres. Clin Microbiol Infect. 2021;27 doi: 10.1016/j.cmi.2020.11.004. 472.e7-472.e10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Scohy A, Anantharajah A, Bodéus M, Kabamba-Mukadi B, Verroken A, Rodriguez-Villalobos H. Low performance of rapid antigen detection test as frontline testing for COVID-19 diagnosis. J Clin Virol. 2020;129 doi: 10.1016/j.jcv.2020.104455. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Dinnes J, Deeks JJ, Berhane S, Taylor M, Adriano A, Davenport C, et al. Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection. Cochrane Database Syst Rev. 2021;3 doi: 10.1002/14651858.CD013705.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Merino P, Guinea J, Muñoz-Gallego I, González-Donapetry P, Galán JC, Antona N, et al. Multicenter evaluation of the PanbioTM COVID-19 rapid antigen-detection test for the diagnosis of SARS-CoV-2 infection. Clin Microbiol Infect. 2021 doi: 10.1016/j.cmi.2021.02.001. S1198-743X(21)00076-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Fenollar F, Bouam A, Ballouche M, Fuster L, Prudent E, Colson P, et al. Evaluation of the panbio COVID-19 rapid antigen detection test device for the screening of patients with COVID-19. J Clin Microbiol. 2021;59 doi: 10.1128/JCM.02589-20. e02589-20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Liotti FM, Menchinelli G, Lalle E, Palucci I, Marchetti S, Colavita F, et al. Performance of a novel diagnostic assay for rapid SARS-CoV-2 antigen detection in nasopharynx samples. Clin Microbiol Infect. 2021;27:487–488. doi: 10.1016/j.cmi.2020.09.030. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Baccani I, Morecchiato F, Chilleri C, Cervini C, Gori E, Matarrese D, et al. Evaluation of three immunoassays for the rapid detection of SARS-CoV-2 antigens. Diagn Microbiol Infect Dis. 2021;101 doi: 10.1016/j.diagmicrobio.2021.115434. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Orsi A, Pennati BM, Bruzzone B, Ricucci V, Ferone D, Barbera P, et al. On-field evaluation of a ultra-rapid fluorescence immunoassay as a frontline test for SARS-CoV-2 diagnostic. J Virol Methods. 2021;295 doi: 10.1016/j.jviromet.2021.114201. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Colavita F, Vairo F, Meschi S, Valli MB, Lalle E, Castilletti C, et al. COVID-19 rapid antigen test as screening strategy at points of entry: experience in lazio region, central Italy, August-October 2020. Biomolecules. 2021;11:425. doi: 10.3390/biom11030425. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Harvey WT, Carabelli AM, Jackson B, Gupta RK, Thomson EC, Harrison EM, et al. SARS-CoV-2 variants, spike mutations and immune escape. Nat Rev Microbiol. 2021;19:409–424. doi: 10.1038/s41579-021-00573-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Bourassa L, Perchetti GA, Phung Q, Lin MJ, Mills MG, Roychoudhury P, et al. A SARS-CoV-2 nucleocapsid variant that affects antigen test performance. J Clin Virol. 2021;141 doi: 10.1016/j.jcv.2021.104900. [DOI] [PMC free article] [PubMed] [Google Scholar]