Abstract
Introduction
Digital immunoassays are generally regarded as superior tests for the detection of infectious disease pathogens, but there have been insufficient data concerning SARS-CoV-2 immunoassays.
Methods
We prospectively evaluated a novel digital immunoassay (RapidTesta SARS-CoV-2). Two nasopharyngeal samples were simultaneously collected for antigen tests and Real-time RT-PCR.
Results
During the study period, 1127 nasopharyngeal samples (symptomatic patients: 802, asymptomatic patients: 325) were evaluated. For digital immunoassay antigen tests, the sensitivity was 78.3% (95% CI: 67.3%–87.1%) and the specificity was 97.6% (95% CI: 96.5%–98.5%). When technicians visually analyzed the antigen test results, the sensitivity was 71.6% (95% CI: 59.9%–81.5%) and the specificity was 99.2% (95% CI: 98.5%–99.7%). Among symptomatic patients, the sensitivity was 89.4% (95% CI; 76.9%–96.5%) with digital immunoassay antigen tests, and 85.1% (95% CI; 71.7%–93.8%) with visually analyzed the antigen test, respectively.
Conclusions
The sensitivity of digital immunoassay antigen tests was superior to that of visually analyzed antigen tests, but the rate of false-positive results increased with the introduction of a digital immunoassay device.
Keywords: RapidTesta SARS-CoV-2, Digital immunoassay, Nasopharyngeal sample, SARS-CoV-2, COVID-19, Asymptomatic patients
1. Introduction
COVID-19 has been the primary global health concern for 2 consecutive years (2020 and 2021) and its diagnosis has been mainly performed by respiratory sampling [1]. RT-PCR has been regarded as the “gold standard” method [1] for the detection of SARS-CoV-2, but a long turnaround time is needed for evaluation of the results.
Immunochromatographic antigen testing is a widely used laboratory technique for the detection of SARS-CoV-2 owing to its ease of handling and rapid acquisition of results [2,3]; moreover, a strong correlation between antigen test results and SARS-CoV-2 infectivity has been reported [4]. In recent years, improved sensitivities for pathogen detection using digital immunoassay devices have been proven, especially in the field of influenza [5], and applications to COVID-19 diagnosis have been expected.
The RapidTesta SARS-CoV-2 is a novel immunochromatographic assay that can detect SARS-CoV-2 in 10 min; it was newly approved in Japan in June 2021. The RapidTesta SARS-CoV-2 can be analyzed using a portal digital immunoassay (DIA) device, with presumably greater sensitivity compared with human-eye judgement [5].
We performed a prospective comparison with a DIA device and RT-PCR using a set of two simultaneously collected nasopharyngeal samples from multiple patients.
2. Patients and methods
The current study was prospectively performed between April 20, 2021, and May 31, 2021, at a “drive-through” PCR center in Tsukuba Medical Center Hospital (TMCH), Tsukuba, Japan. During the study period, patients were mainly referred from 63 clinics and a local public health center for suspected COVID-19 based on their symptoms or contact histories with COVID-19 patients.
Informed consent was obtained from all participating patients before sample collection, and the study was performed with the approval of the TMCH ethical committee (approval number: 2021–021).
2.1. Study process and evaluation of antigen test results with and without the use of a digital immunoassay device
In this study, two nasopharyngeal samples were simultaneously collected with FLOQ Swabs (Copan Italia S.p.A., Brescia, Italy) for RT-PCR and flocked type swabs (contained in the RapidTesta SARS-CoV-2 kit) for antigen tests, as previously described [6]. Two nasopharyngeal samples were obtained, each from different nasal cavities. For antigen tests, each swab was diluted into extraction buffer; three drops of the extracted sample were then added to a test cassette through filtration (Supplementary Fig. 1). The examination time was 10 min; the analysis was performed both visually and automatically with a DIA device (Supplementary Fig. 2) by the same examiners. All antigen testing was performed before in-house RT-PCR, and the antigen testers were not informed of the results of in-house RT-PCR.
For RT-PCR, each swab was diluted in 3 mL of Universal Transport Medium (BD), then transferred to the microbiology department of the TMCH for RNA extraction and in-house RT-PCR. Extraction was performed with magLEAD 6gC (Precision System Science Co., Ltd., Chiba, Japan) with a 200 μL aliquot of each nasopharyngeal sample. In total, 100 μL of purified sample were eluted and subjected to in-house RT-PCR analysis [7] and a reference real-time RT-PCR analysis for SARS-CoV-2, which used a method developed by the National Institute of Infectious Diseases, Japan (NIID method) [8]. The reference RT-PCR analysis was performed at Tsukuba Research Institute of Sekisui Medical Co., Ltd.; eluted samples were transported weekly for evaluation. Before evaluation, all samples were preserved at −80 °C.
2.2. Reference RT-PCR using a method developed by the National Institute of Infectious Diseases, Japan
The NIID method has been regarded as the “gold standard” RT-PCR method for SARS-CoV-2 detection and was used as the reference RT-PCR method in this study. The NIID method was performed with duplicate N and N2 assay analysis, as previous described [8]. The CFX96 Touch Real-Time PCR Detection System (Bio-Rad Laboratories, Hercules, CA, USA) and Reliance One-Step Multiplex Supermix (Bio-Rad Laboratories) were used for analysis. In the event of discordance between the reference RT-PCR and in-house RT-PCR results, the GeneXpert® system and an Xpert® Xpress SARS-CoV-2 assay (Cepheid Inc., Sunnyvale, CA, USA) [9] were used for additional evaluation.
2.3. Statistical analyses
The sensitivity, specificity, positive predictive value, and negative predictive value of antigen test results were calculated using the Clopper and Pearson method, with 95% confidence intervals (CIs). McNemar's chi-squared test was used to compare the sensitivities between antigen testing with and without a DIA device. The sensitivities of antigen testing stratified by Ct values (Ct < 20, Ct 20–29, Ct ≥ 30) were compared using Fisher's exact test. P values < 0.05 were considered statistically significant in this study. All calculations were conducted using the R software program, version 4.1.0 (www.r-project.org).
3. Results
During the study period, 1127 nasopharyngeal samples were collected for evaluation. In total, 802 samples were collected from symptomatic patients and 325 samples were collected from asymptomatic patients.
Of the 1127 samples, 74 (6.6%) were positive according to RT-PCR with the NIID method. There were no instances of discordance between RT-PCR with the NIID method and in-house RT-PCR; thus, no additional evaluations with the GeneXpert® system were performed.
The results of the RapidTesta SARS-CoV-2 without the use of a DIA device are shown in Table 1 . The sensitivity was 71.6% (95% CI: 59.9%–81.5%), and the specificity was 99.2% (95% CI: 98.5%–99.7%). The positive predictive value was 86.9% (95% CI: 75.8%–94.2%), and the negative predictive value was 98.0% (95% CI: 97.0%–98.8%).
Table 1.
Sensitivity and specificity of the RapidTesta SARS-CoV-2 without the use of a digital immunoassay device among all patients in the study.
| real-time RT-PCR |
|||
|---|---|---|---|
| Positive | Negative | ||
| Antigen test | Positive | 53 | 8 |
| Negative |
21 |
1045 |
|
| Sensitivity (%) | 71.6(59.9–81.5) | ||
| Specificity (%) | 99.2(98.5–99.7) | ||
Reverse transcription-polymerase chain reaction findings were used as reference results.
RT-PCR, reverse transcription-polymerase chain reaction.
Data in parentheses indicate 95% confidence intervals.
The results of the RapidTesta SARS-CoV-2 with the use of a DIA device are shown in Table 2 . The sensitivity was 78.4% (95% CI: 67.3%–87.1%), and the specificity was 97.6% (95% CI: 96.5%–98.5%). The sensitivity when using a DIA device was significantly higher than that with human-eye judgement (p = 0.03). The positive predictive value was 69.9% (95% CI: 58.8%–79.5%), and the negative predictive value was 98.5% (95% CI: 97.5%–99.1%). All of the cases that were positive by human-eye judgement were also judged to be positive by a DIA device.
Table 2.
Sensitivity and specificity of the RapidTesta SARS-CoV-2 with the use of a digital immunoassay device among all patients in the study.
| real-time RT-PCR |
|||
|---|---|---|---|
| Positive | Negative | ||
| Antigen test | Positive | 58 | 25 |
| Negative |
16 |
1028 |
|
| Sensitivity (%) | 78.4(67.3–87.1) | ||
| Specificity (%) | 97.6(96.5–98.5) | ||
Reverse transcription-polymerase chain reaction findings were used as reference results.
RT-PCR, reverse transcription-polymerase chain reaction.
Data in parentheses indicate 95% confidence intervals.
The results of the RapidTesta SARS-CoV-2 with/without the use of a DIA device among symptomatic patients are shown in Table 3 and Table 4 . For the RapidTesta SARS-CoV-2 without the use of a DIA device, the sensitivity was 85.1% (95% CI: 71.7%–93.8%), and the specificity was 99.3% (95% CI: 98.5%–99.8%). For the RapidTesta SARS-CoV-2 with the use of a DIA device, the sensitivity was 89.4% (95% CI: 76.9%–96.5%), and the specificity was 97.7% (95% CI: 96.4%–98.7%).
Table 3.
Sensitivity and specificity of the RapidTesta SARS-CoV-2 without the use of a digital immunoassay device among symptomatic patients.
| real-time RT-PCR |
|||
|---|---|---|---|
| Positive | Negative | ||
| Antigen test | Positive | 40 | 5 |
| Negative | 7 | 750 | |
| Sensitivity (%) | 85.1(71.7–93.8) | ||
| Specificity (%) | 99.3(98.5–99.8) | ||
Table 4.
Sensitivity and specificity of the RapidTesta SARS-CoV-2 with the use of a digital immunoassay device among symptomatic patients.
| real-time RT-PCR |
|||
|---|---|---|---|
| Positive | Negative | ||
| Antigen test | Positive | 42 | 17 |
| Negative |
5 |
738 |
|
| Sensitivity (%) | 89.4(76.9–96.5) | ||
| Specificity (%) | 97.7(96.4–98.7) | ||
The sensitivities stratified according to Ct value are shown in Table 5 . Without the use of a DIA device, the sensitivity for Ct < 20 was 96.4% (95% CI: 81.7%–99.9%); the sensitivity was 81.3% (95% CI: 63.6%–92.8%) for Ct 20–29. No positive samples (Ct ≥ 30) were detected without the use of a DIA device. When using a DIA device, the sensitivity for Ct < 20 was 100.0% (95% CI: 87.7%–100%); the sensitivities were 84.4% (95% CI: 67.2%–94.7%) for Ct 20–29 and 21.4% (95% CI: 4.7%–50.8%) for (Ct ≥ 30), respectively. The sensitivity of antigen testing significantly declined according to the increase in Ct values, regardless of the use of DIA devices (p < 0.001).
Table 5.
Sensitivities of the RapidTesta SARS-CoV-2 without the use of a digital immunoassay device according to Ct value.
| Ct value(N2) | Sensitivity (%) | Positive | Negative | p value |
|---|---|---|---|---|
| <20 | 96.4(81.7–99.9) | 27 | 1 | <0.001 |
| 20–29 | 81.3(59.0–91.7) | 26 | 6 | |
| ≥30 |
0(0–19.5) |
0 |
14 |
|
| Sensitivities of the RapidTesta SARS-CoV-2 with the use of a digital immunoassay device according to Ct value. |
||||
| Ct value(N2) |
Sensitivity (%) |
Positive |
Negative |
p value |
| <20 | 100(87.7–100) | 28 | 0 | <0.001 |
| 20–29 | 84.4(63.1–93.9) | 27 | 5 | |
| ≥30 | 21.4(3.8–43.4) | 3 | 11 | |
The sensitivities stratified by the duration since the symptom onset are shown in Supplementary Figs. 4 and 5. The sensitivities for visually analyzed antigen tests were 85.7% for 0–2 days, 93.3% for 3–5 days and 66.7% for 6–10 days. The sensitivities for the DIA device were 90.5% for 0–2 days, 93.3% for 3–5 days, and 66.7% for 6–10 days.
4. Discussion
This prospective study revealed that the sensitivity of the RapidTesta SARS-CoV-2 was 78.4% and the specificity was 97.6%. For symptomatic patients, the sensitivities were 85.1% for antigen tests without the use of a DIA device and 89.4% for antigen tests with the use of a DIA device. The sensitivity of DIA tests was superior to the sensitivity of visually analyzed antigen tests, but the rate of false-positive results increased with introduction of a DIA device.
A recent systematic review described overall antigen test sensitivities of 72.0% (95% CI: 63.7%–79.0%) for symptomatic patients and 58.1% (95% CI: 40.2%–74.1%) for asymptomatic patients [10]. For quantitative antigen tests, the overall sensitivity and specificity were 84.8% and 97.9%, respectively [11]. In the current study, the sensitivity for symptomatic patients of RapidTesta SARS-CoV-2 was over 80% and it is considered to meet the acceptable criteria of antigen test recommended by the World Health Organization [12].
To our knowledge, there have been few evaluations of DIA devices for the detection of SARS-CoV-2. Most recently, a prospective study of a DIA device (QuickChaser Auto SARS-CoV-2) with 1401 nasopharyngeal samples showed that the sensitivity and specificity were 74.7% (95% CI: 64.0%–83.6%) and 99.8% (95% CI: 99.5%–100%), respectively [13]. The sensitivity for positive samples (Ct < 30) was 98.2% (56/57). In our study, the overall sensitivity of the RapidTesta SARS-CoV-2 with a DIA device was comparable with a previous DIA device (QuickChaser Auto SARS-CoV-2). This rapid analysis could be completed within 10 min, although its sensitivity for positive samples (Ct < 30) (91.7%; 55/60) was slightly inferior to the previously described DIA device.
False-positive results that indicate the presence of COVID-19 cause substantial harm in clinical practice [14] and the emergence of false-positive results has been repeatedly reported in Japan [[15], [16], [17]]. In the current study, the increased sensitivity when using the DIA device was negatively associated with the decrease of specificity. False-positive results were frequently observed (17/25; 68.0%) via DIA inspection for samples with RT-PCR-negative results in this study. The current study suggested that a DIA device for RapidTesta SARS-CoV-2 detected slight, non-specific color changes, which resulted in an increase in false-positives. When positive RapidTesta SARS-CoV-2 results with a DIA device are recorded in the absence of a visually recognizable positive line, additional SARS-CoV-2 testing with RT-PCR should be performed due to a possible false-positive (Supplementary Fig. 3).
There were some limitations to this study. First, study samples were collected in a local district in Japan in the spring of 2021. The results of lateral flow devices are strongly dependent on the individual judging the outcome [18]. Second, differences among variant strains were not analyzed. Third, study samples were collected from the nasopharyngeal tract and saliva samples were not analyzed. Fourth, improvements of DIA specificity were not examined and further investigations are necessary.
In conclusion, the sensitivity of DIA tests was superior to that of visually analyzed antigen tests, but the rate of false-positive results increased with the introduction of a DIA device. When positive DIA results are recorded without a visually recognizable red line at the positive line location on the test cassette, additional RT-PCR evaluation should be performed due to possible false-positives.
Declaration of competing interest
Sekisui Medical Co., Ltd. provided fees for research expenses and provided the RapidTesta SARS-CoV-2 without charge. Yuta Maehara, Yasushi Ochiai, and Shinya Okuyama belong to Sekisui Medical Co., Ltd., the developer of the RapidTesta SARS-CoV-2.
Acknowledgements
We thank Mrs. Yoko Ueda, Mrs. Mio Matsumoto, Mr. Masaomi Matsubayashi, Mrs. Mika Yaguchi, Mrs. Yumiko Tanaka, and the staff of the Department of Clinical Laboratory of Tsukuba Medical Center Hospital for their support of this study. We thank all participating medical institutions for providing their patients' clinical information. Mrs. Yoko Ueda and Mrs. Mio Matsumoto substantially contributed to creating the database for this study.
Footnotes
Supplementary data to this article can be found online at https://doi.org/10.1016/j.jiac.2021.10.024.
Authorship statement
All authors meet the ICMJE authorship criteria.
Contributors
All authors performed the investigation. Hiromichi Suzuki supervised the project and drafted the manuscript. All authors contributed to the writing of the final manuscript.
Appendix A. Supplementary data
The following are the Supplementary data to this article:
Test flow diagram of the RapidTesta SARS-CoV-2 without the use of a digital immunoassay device.
Test flow diagram of the RapidTesta SARS-CoV-2 with the use of a digital immunoassay device.
Strategy for determining the result of RapidTesta SARS-CoV-2 with a digital immunoassay device.
Differences in the sensitivity of RapidTesta SARS-CoV-2 without a digital immunoassay device stratified by the duration since the symptom onset.
Differences in the sensitivity of RapidTesta SARS-CoV-2 with a digital immunoassay device stratified by the duration since the symptom onset.
References
- 1.World Health Organization. Coronavirus disease (COVID-19) Weekly Epidemiological Update and Weekly Operational Update. https://www.who.int/emergencies/diseases/novel-coronavirus-2019/situation-reports [accessed July 10, 2021].
- 2.Safiabadi Tali S.H., LeBlanc J.J., Sadiq Z., Oyewunmi O.D., Camargo C., Nikpour B., et al. Tools and techniques for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2)/COVID-19 detection. Clin Microbiol Rev. 2021;34 doi: 10.1128/CMR.00228-20. e00228-20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Hanson K.E., Altayar O., Caliendo A.M., Arias C.A., Englund J.A., Hayden M.K., et al. The infectious diseases society of America guidelines on the diagnosis of COVID-19: antigen testing. Clin Infect Dis. 2021 doi: 10.1093/cid/ciab557. (Ahead of print) [DOI] [PubMed] [Google Scholar]
- 4.Korenkov M., Poopalasingam N., Madler M., Vanshylla K., Eggeling R., Wirtz M., et al. Evaluation of a rapid antigen test to detect SARS-CoV-2 infection and identify potentially infectious individuals. J Clin Microbiol. 2021;59:e0089621. doi: 10.1128/JCM.00896-21. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Merckx J., Wali R., Schiller I., Caya C., Gore G.C., Chartrand C., et al. Diagnostic accuracy of novel and traditional rapid tests for influenza infection compared with reverse transcriptase polymerase chain reaction: a systematic review and meta-analysis. Ann Intern Med. 2017;167:394–409. doi: 10.7326/M17-0848. [DOI] [PubMed] [Google Scholar]
- 6.Marty F.M., Chen K., Verrill K.A. How to obtain a nasopharyngeal swab specimen. N Engl J Med. 2020;382:e76. doi: 10.1056/NEJMvcm2010260. [DOI] [PubMed] [Google Scholar]
- 7.Kiyasu Y., Akashi Y., Sugiyama A., Takeuchi Y., Notake S., Naito A., et al. A Prospective evaluation of the analytical performance of GENECUBE((R)) HQ SARS-CoV-2 and GENECUBE((R)) FLU A/B. Mol Diagn Ther. 2021;25:495–504. doi: 10.1007/s40291-021-00535-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Shirato K., Nao N., Katano H., Takayama I., Saito S., Kato F., et al. Development of genetic diagnostic methods for detection for novel coronavirus 2019(nCoV-2019) in Japan. Jpn J Infect Dis. 2020;73:304–307. doi: 10.7883/yoken.JJID.2020.061. [DOI] [PubMed] [Google Scholar]
- 9.Lee J., Song J.U. Diagnostic accuracy of the Cepheid Xpert Xpress and the Abbott ID NOW assay for rapid detection of SARS-CoV-2: a systematic review and meta-analysis. J Med Virol. 2021;93:4523–4531. doi: 10.1002/jmv.26994. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Dinnes J., Deeks J.J., Berhane S., Taylor M., Adriano A., Davenport C., et al. Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection. Cochrane Database Syst Rev. 2021;3:CD013705. doi: 10.1002/14651858.CD013705.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Hirotsu Y., Sugiura H., Maejima M., Hayakawa M., Mochizuki H., Tsutsui T., et al. Comparison of Roche and Lumipulse quantitative SARS-CoV-2 antigen test performance using automated systems for the diagnosis of COVID-19. Int J Infect Dis. 2021;108:263–269. doi: 10.1016/j.ijid.2021.05.067. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.World Health Organization COVID-19 Target product profiles for priority diagnostics to support response to the COVID-19 pandemic v.1.0. 2020. https://www.who.int/publications/m/item/covid-19-target-product-profiles-for-priority-diagnostics-to-support-response-to-the-covid-19-pandemic-v.0.1 1-38.
- 13.Kurihara Y., Kiyasu Y., Akashi Y., Takeuchi Y., Narahara K., Mori S., et al. The evaluation of a novel digital immunochromatographic assay with silver amplification to detect SARS-CoV-2. J Infect Chemother. 2021;27:1493–1497. doi: 10.1016/j.jiac.2021.07.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Healy B., Khan A., Metezai H., Blyth I., Asad H. The impact of false positive COVID-19 results in an area of low prevalence. Clin Med. 2021;21:e54–e56. doi: 10.7861/clinmed.2020-0839. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Itoh K., Kawamitsu T., Osaka Y., Sato K., Suzuki Y., Kiriba C., et al. False positive results in severe acute respiratory coronavirus 2 (SARS-CoV-2) rapid antigen tests for inpatients. J Infect Chemother. 2021;27:1089–1091. doi: 10.1016/j.jiac.2021.03.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Otake S., Miyamoto S., Mori A., Iwamoto T., Kasai M. False-positive results in SARS-CoV-2 antigen test with rhinovirus-A infection. Pediatr Int. 2021;9:1135–1137. doi: 10.1111/ped.14582. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Yamaniha K., Kinjo T., Akamine M., Setoguchi M., Tateyama M., Fujita J. False-positive for SARS-CoV-2 antigen test in a man with acute HIV infection. J Infect Chemother. 2021;27:1112–1114. doi: 10.1016/j.jiac.2021.04.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Peto T., Team U.C.-L.F.O. COVID-19: rapid antigen detection for SARS-CoV-2 by lateral flow assay: a national systematic evaluation of sensitivity and specificity for mass-testing. EClinicalMedicine. 2021;36:100924. doi: 10.1016/j.eclinm.2021.100924. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Test flow diagram of the RapidTesta SARS-CoV-2 without the use of a digital immunoassay device.
Test flow diagram of the RapidTesta SARS-CoV-2 with the use of a digital immunoassay device.
Strategy for determining the result of RapidTesta SARS-CoV-2 with a digital immunoassay device.
Differences in the sensitivity of RapidTesta SARS-CoV-2 without a digital immunoassay device stratified by the duration since the symptom onset.
Differences in the sensitivity of RapidTesta SARS-CoV-2 with a digital immunoassay device stratified by the duration since the symptom onset.