Abstract
Objectives
This study was undertaken to evaluate the diagnostic performance of the BinaxNOW COVID-19 Ag Card rapid antigen assay (Abbott; Chicago, IL, USA) in the detection of COVID-19 infection compared to the reference standard of PCR testing.
Methods
We evaluated the BinaxNOW COVID-19 Ag Card rapid antigen assay relative to a standard reference PCR test. We tested 3810 nasal swabs from symptomatic and asymptomatic adults undergoing surveillance COVID-19 testing at Howard University using one swab for each nostril. One swab was tested using the rapid antigen assay and the other using the PCR test.
Results
The sensitivity of the BinaxNOW COVID-19 Ag Card rapid antigen assay was 91.84% (95% confidence interval (CI): 80.40–97.73%) and the specificity was 99.95% (95% CI: 99.81–99.99%). The range of Ct values for the N gene was 10.74–34.90 (M = 26.88, SD=4.86). Fourteen (28.6%) samples had an N gene Ct value > 30. The average N gene Ct value for rapid test negative (i.e. false negative) samples was 31.92.
Conclusions
The sensitivity of the test in our symptomatic and asymptomatic cohort was lower than the manufacturer's reported sensitivity in a symptomatic cohort (97.1%). Despite their relatively lower sensitivity (especially in asymptomatic individuals), rapid tests have undeniable benefits (i.e., ease of use and rapid results) that make them a helpful tool in the control of the SARS-CoV-2 pandemic. Given the diagnostic accuracy of these tests as evidenced by this study, rapid tests can be thoughtfully employed in situations where swift results are critical.
Keywords: Coronavirus, COVID-19, RT-PCR, Rapid testing, Diagnostic accuracy, COVID-19 testing
1. Background
Early diagnosis of COVID-19 is critical to reducing person-to-person transmission. Laboratory diagnosis through polymerase chain reaction (PCR) testing, considered the standard reference, may take up to 24 h for the results to be confirmed with additional time for transport, whereas diagnosis through rapid antigen testing can take as few as 15 min and be done at the point of care. Rapid and easy-to-perform diagnostic tests therefore allow for earlier initiation of isolation and more effective contact tracing, thereby decreasing transmission of COVID-19.
As rapid testing becomes more available, characteristics of the diagnostic performance of these tests relative to the standard reference, including sensitivity, specificity, positive and negative predictive values are critical to their impact on public health. False negative COVID-19 test results in particular pose a challenge to public health efforts to reduce the spread of the virus. Individuals with false negative results, believing they do not have COVID-19, may stop practicing measures to reduce the spread of the virus, including mask-wearing, social distancing, and avoiding large gatherings [1]. False negative results also limit health professionals’ ability to conduct contact tracing and testing of exposed individuals.
Abbott, the producers of the BinaxNOW COVID-19 Ag Card rapid antigen assay, evaluated the clinical performance of their assay through a multi-site prospective study including 10 sites in the United States [2]. The analysis of 460 nasal swabs from symptomatic patients with suspected COVID-19 infection revealed a sensitivity of 97.1% and a specificity of 98.5% when compared to a real time Polymerase Chain Reaction (RT-PCR) [3].
2. Objectives
This study was undertaken to evaluate the diagnostic performance of the BinaxNOW COVID-19 Ag Card rapid antigen assay (Abbott; Chicago, IL, USA) in the detection of COVID-19 infection in relation to the reference standard of PCR testing in asymptomatic and symptomatic adults undergoing COVID-19 testing at Howard University.
3. Study design
In this single-center study, the results of 3810 anterior nasal swab samples were analyzed. The samples were collected by a healthcare provider from September to December 2020 from adults undergoing periodic employer-mandated COVID-19 surveillance testing at Howard University. Information on the presence or absence of COVID19 symptoms was not collected in all tested individuals. Two swabs were used to collect samples, one for each nostril. One was used for the BinaxNOW COVID-19 Ag Card rapid antigen assay which was performed at the point of care, and the second swab was analyzed using the Applied Biosystems TaqPath COVID-19 RT-PCR kit (Thermo Fisher Scientific Inc.; Waltham, MA, USA) in an institutional reference laboratory. In the event that the results from each nostril did not match, the result provided by the PCR test was accepted, as it is the reference standard test. The primary outcomes were sensitivity, specificity and overall diagnostic accuracy of the rapid antigen assay measured against the PCR testing. Secondary outcomes were negative predictive value (NPV) and positive predictive value (PPV).
3.1. Test methods
The BinaxNOW COVID-19 Ag Card is an assay that uses antibodies to detect SARS-CoV-2 proteins from nasal swab specimens. The samples were taken from the anterior nares using a sterile nasal swab which was then inserted into a cardboard test card along with 6 drops of an extraction reagent. The card was then closed to allow the sample to come into contact with the reagent and the test strip. After 15 minutes, the provider interpreted the test visually based on the presence of pink/purple lines on the test card. Results were read promptly at 15 min, as they are considered invalid after 30 min [2].
The reference standard PCR test for detection of COVID-19 qualitatively detects nucleic acid from SARS-CoV-2. The sample was taken from the anterior nares using a sterile nasal swab, which was then sent to an institutional reference laboratory for PCR testing. RNA was then extracted from the sample using the Applied Biosystems MagMAX™ Viral/Pathogen II (MVP II) Nucleic Acid Isolation Kit according to manufacturer's instructions. Once the RNA was extracted RT-PCR was performed immediately using the Applied Biosystems TaqPath™ COVID-19 Combo Kit according to manufacturer's instructions. The TaqPath™ COVID-19 Combo Kit RT-PCR test targets primers located between the following genes: ORF1ab, N gene, and S gene. The results were then analyzed using the COVID-19 Interpretive Software (v. 2.5) from Thermo Fisher Scientific, Inc and all positive samples were double checked by the performing technologist for final confirmation of the positive. A sample was determined to be positive if the Ct value of 2 of the 3 viral targets was ≤ 37.
4. Results
A total of 3810 samples were collected. Thirty (0.79%) of the BinaxNOW COVID-19 Ag Card rapid antigen assays had to be repeated due to failure of the first assay (i.e., absence of the control line on the test card). The overall positivity rates of the rapid and PCR tests were 1.23% and 1.29%, respectively. The rates of false negative and false positive results of the rapid test were 8.16% and 0.05%, respectively. The sensitivity of the BinaxNOW COVID-19 Ag Card rapid antigen assay performed at point of care was 91.84% (95% confidence interval (CI): 80.40–97.73%) and the specificity was 99.95% (95% CI: 99.81–99.99%). The overall diagnostic accuracy of the rapid antigen was estimated as 999 (95% CI: 888–999). Results from the rapid test were available in 15 min, whereas results from the PCR test took 8–24 h from when the patient was tested to when they were informed of their result. The positive predictive value of the BinaxNOW rapid antigen assay was 95.74% and the negative predictive value was 99.89%. Table 1
Table 1.
Results of RT-PCR tests versus BinaxNOW COVID-19 Ag Card rapid antigen assays in 3810 nasal samples.
| PCR Test Positive | PCR Test Negative | Total | |
|---|---|---|---|
| Rapid Test Positive | 45 | 2 | 47 |
| Rapid Test Negative | 4 | 3759 | 3763 |
| Total | 49 | 3761 | 3810 |
The range of the Ct values for the ORF1ab gene was 14.15–35.27 (M = 26.16, SD = 4.69). The range of Ct values for the N gene was 10.74–34.90 (M = 26.88, SD = 4.86). The range of Ct values for the S gene was 14.67–37.10 (M = 26.69, SD = 4.45). Fourteen (28.6%) of samples had a N gene Ct value > 30 (Table 2 ). The average N gene Ct value for rapid test negative (i.e. false negative) samples as 31.92.
Table 2.
RT-PCR Ct values (N gene) of all TaqPath RT-PCR positive samples.
| Ct Value (N gene) | Number of Samples |
|---|---|
| <20 | 2 |
| <25 | 17 |
| <30 | 35 |
| <35 | 49 |
5. Discussion
Testing the diagnostic accuracy of rapid antigen assays for the detection of COVID-19 has widespread implications for our efforts to curb the spread of the SARS-CoV-2 virus. Rapid tests are practical and have become increasingly popular as a strategy for hosting large scale events and reopening schools [4]. Previous studies have found rapid tests to be inferior to the gold-standard PCR test for the detection of COVID-19, citing low sensitivity as the major limitation. For example, in real-world studies, the sensitivity of the PanBio™ rapid antigen test (also manufactured by Abbott Laboratories), ranges from 74.4%−86.8% [5,6].
The results of this study of the BinaxNOW rapid antigen test in symptomatic and asymptomatic individuals demonstrate a lower sensitivity (91.84%) than that reported by the manufacturer (97.1%) [2]. However, it is important to note that the sensitivity of this test is higher in symptomatic individuals [2,7]. A limitation of our study is the lack of complete data on the symptomatic vs. asymptomatic status of our cohort. However, a recent study of the same rapid test in a cohort of symptomatic and asymptomatic individuals reported similar results (sensitivity 93.3% and specificity 99.9%) [8]
The relatively lower sensitivity of rapid tests must be weighed against their benefits, (i.e., room temperature storage, easy to use, and quick turnaround time for results). Though imperfect, rapid tests can be thoughtfully employed to screen symptomatic and asymptomatic individuals in situations where swift results are critical.
Funding sources
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
References
- 1.West C.P., Montori V.M., Sampathkumar P. COVID-19 testing: the threat of false-negative results. Mayo Clin. Proc. 2020;95(6):1127–1129. doi: 10.1016/j.mayocp.2020.04.004. Accessed January 6, 2021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Abbott. BinaxNOW COVID-19 AG card (PN 195-000) – instructions for use. 2020 December 12. https://www.fda.gov/media/141570/download. Accessed March 5, 2021.
- 3.Abbott. Upping the ante on COVID-19 antigen testing. 2020 August 26. https://www.abbott.com/corpnewsroom/product-and-innovation/upping-the-ante-on-COVID-19-antigen-testing.html. Accessed January 12, 2021.
- 4.Fitzpatrick M.C., Pandey A., Wells C.R., Sah P., Galvani A.P. Buyer beware: inflated claims of sensitivity for rapid COVID-19 tests. The Lancet. 2020;397(10268):24–25. doi: 10.1016/S0140-6736(20)32635-0. Accessed January 12, 2021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Bulilete O., Lorente P., Leiva A., Carandell E., Oliver A., Rojo E., et al. Panbio™ rapid antigen test for SARS-CoV-2 has acceptable accuracy in symptomatic patients in primary health care. J. Infect. 2021;82(3):391–398. doi: 10.1016/j.jinf.2021.02.014. Mar. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Krüger L.J., Gaeddert M., Tobian F., Lainati F., Klein J.A., Schnitzler P., et al. Evaluation of the accuracy and ease-of-use of Abbott PanBio-A WHO emergency use listed, rapid, antigen-detecting point-of-care diagnostic test for SARS-CoV-2. PLoS ONE. 2021;16(5) doi: 10.1371/journal.pone.0247918. May 27https://doi: 10.1371/journal.pone.0247918.eCollection 2021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Pollock N.R., Jacobs J.R., Tran K., Cranston A.E., Smith S., O'Kane C.Y., et al. Performance and implementation evaluation of the Abbott BinaxNOW rapid antigen test in a high-throughput drive-through community testing site in Massachusetts. J. Clin. Microbiol. 2021;59(5) doi: 10.1128/JCM.00083-21. Apr 20e00083-21. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Pilarowski G., Lebel P., Sunshine S., Liu J., Crawford E., Marquez C., et al. Performance characteristics of a rapid SARS-CoV-2 antigen detection assay at a public plaza testing site in San Francisco. J. Infect. Dis. 2021;223(7):1139–1144. doi: 10.1093/infdis/jiaa802. Apr 8https://doi: 10.109. [DOI] [PMC free article] [PubMed] [Google Scholar]