Abstract
The SARS-CoV-2 pandemic led to an urgent need for rapid diagnostic testing in order to inform timely patients’ management. This study aimed to assess the performance of the STANDARD™ M10 SARS-CoV-2 assay as a diagnostic tool for COVID-19. A total of 400 nasopharyngeal or oropharyngeal swabs were tested against a reference real-time RT-PCR, including 200 positive samples spanning the full range of observed Ct values. The sensitivity of the STANDARD™ M10 SARS-CoV-2 assay was 98.00% (95% CI 94.96% to 99.45%, 196/200), while the specificity was also estimated at 97.50% (95% CI 94.26% to 99.18%, 195/200). The assay proved highly efficient for the detection of SARS-CoV-2, even in samples with low viral load (Ct>25), presenting lower Ct values compared to the reference method. We concluded that the STANDARD™ M10 SARS-CoV-2 assay has a similar performance compared to the reference method and other molecular point-of-care assays and can be a valuable tool for rapid and accurate diagnosis.
Keywords: SARS-CoV-2, STANDARDTM M10 SARS-CoV-2 assay, molecular point-of-care test, COVID-19
1. Introduction
The severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) led to an outbreak of the Coronavirus Disease 2019 (COVID-19) [1]. Within a short period of time, SARS-CoV-2 has rapidly spread all over the world, therefore the COVID-19 outbreak was declared by World Health Organization (WHO) as a “pandemic” on March 11, 2020 [2]. The SARS-CoV-2 genome is composed of a positive-sense, single-stranded RNA of around 30 kb in size [3]. This genome consists of two segments. The first segment is formed by two open reading frames (ORF1a and ORF1b), which are translated to two polyproteins, resulting in 16 non-structural proteins, one of which is the RNA-dependent RNA polymerase (RdRp). The second segment encodes for structural proteins, for instance the spike (S), the envelope (E), the membrane (M) and the nucleocapsid (N) [4]. Many of the above genes have been used as diagnostic targets in a variety of nucleic acid amplification-based tests, such as reverse-transcription polymerase chain reaction (RT-PCR) [5].
Currently, COVID-19 can be diagnosed by various methods, such as reverse transcription polymerase chain reaction (RT-PCR), virus culture and antigen or antibody tests [6]. In many cases, presence of the virus is confirmed by real-time RT-PCR, as it is still considered the gold-standard method for diagnosing both symptomatic and asymptomatic cases [7,8]. Regardless of its diagnostic value, real-time RT-PCR is not easily applicable at all cases, mainly due to its long turnaround times and the need for heavy equipment and skilled personnel to perform it [9,10]. Therefore, a great number of point-of-care (POC) methods have been developed, which include fluorescent rapid antigen-detecting (RADT) and rapid nucleic acid amplification (NAAT) tests [11,12]. RADTs are vastly used, since they are inexpensive and able to give the result in a few minutes, however they give a great percentage of false negative results, especially in cases where the viral load is low [11,13].
It is obvious that developing fast, accurate, molecular point-of-care tools for the detection of SARS-CoV-2 RNA is essential to control the pandemic. Various rapid NAATs have been developed and assessed for their use at point-of-care sites. The more frequently assessed rapid NAATs include the Xpert Xpress SARS-CoV-2 (Cepheid,Sunnyvale, CA, USA) and the ID Now SARS-CoV-2 (Abbott, Chicago, IL, USA). Overall, the above tests exhibited a sensitivity of 95.1% (95% CI 90.5% to 97.6%) and specificity of 98.8% (95% CI 98.3% to 99.2%), respectively [11]. Here, we report an evaluation of the STANDARD™ M10 SARS-CoV-2 assay (SD BIOSENSOR), which is a multiplex real-time RT-PCR test designed to detect SARS-CoV-2 RNA in upper respiratory specimens, such as nasopharyngeal or oropharyngeal swabs.
2. Materials and methods
The under evaluation STANDARD™ M10 SARS-CoV-2 assay is designed for use with the STANDARD™ M10 system (SD BIOSENSOR), which conducts sample preparation, nucleic acid extraction and amplification and detection of the target sequences, using the real-time RT-PCR as the test method. The system comprises of the M10 Console, which allows the user to import information about the sample, and the M10 module, which performs the assay. The assay requires single-use disposable cartridges that contain the RT-PCR reagents. Cross-contamination between samples is greatly limited, since the cartridges are self-contained. The assay is targeting two genes of the SARS-CoV-2, the ORF1ab gene and the E gene, and the human RNAse P as an internal control. The procedure lasts 60 min. At the end of each assay the result is shown with the corresponding Cycle Threshold (Ct) values on the M10 Console. The limit of detection (LoD) is 100 copies/ml, as reported by the manufacturer.
The reference method used to assess STANDARD™ M10 SARS-CoV-2 performance was TaqPath™ COVID CE 19-IVD RT PCR kit (Applied Biosystems™), which targets three genes of SARS-CoV-2, the ORF1ab gene, the N gene and the S gene. PCR was carried out in a QuantStudio™ 5 Real-Time PCR System (Applied Biosystems™). According to the manufacturer, the LoD is 250 copies/ml. Ct values were collected from the accompanying software. RNA was extracted by 200μl sample with the KingFisher™ Flex Purification System (Thermo Scientific™) using the MagMAX™ Viral/Pathogen II (MVP II) Nucleic Acid Isolation Kit (Applied Biosystems™), according to manufacturer's instructions.
Totally, 200 negative and 200 positive for SARS-CoV-2 samples were chosen for analysis with the STANDARD™ M10 SARS-CoV-2 assay. More specifically, approximately half of the positive samples chosen, had a Ct value higher than 30, in order to test the assay in samples with low viral load, 25% had a Ct value ≥25 and ≤30 while the rest had a Ct value lower than 25. The assay was performed on 600μl sample, according to manufacturer's instructions. Besides positive and negative outcome of the test, there was also the possibility of presumptive positive as an outcome, in cases where only the E gene was detected. In case of discrepancy in comparison to the reference method, the assay was repeated two more times. The result detected more than 50% (two out of three times) was adopted as the final result and kept for the statistical analyses.
The swab samples included in this study were collected from individuals suspected of COVID-19 disease by various health-care workers across the Region of Crete, Greece, between January and March 2022. All specimens were collected in two commonly used types of viral transport media, biocomma® Virus Transport and Preservation Medium (Biocomma Limited) and BioSci® Virus Transport Medium (Dakewe Biotech Co.). The specimens were tested within the daily routine for the presence of SARS-CoV-2 RNA at the Laboratory of Clinical Virology (Medical School, University of Crete). The study was approved by the Ethics Committee of the General University Hospital of Heraklion, Crete, Greece (Reference number 560/2021). All experiments were performed in accordance with relevant named guidelines and regulations. Informed consent was obtained from all participants or their legal guardians participants or their legal guardians.
Estimates of the sensitivity and specificity of the STANDARD™ M10 SARS-CoV-2 assay were made according to Altman and Bland (1994) [14]. A two-sided alpha value of 0.05 was defined as the significance cut-off. The Ct values for the ORF1ab gene from the reference method were compared with the corresponding Ct values from the STANDARD™ M10 SARS-CoV-2 assay. Shapiro-Wilk test was used to test for normal distributions. Since the normality assumption was not met, Spearman's rank correlations and Wilcoxon signed-rank tests for paired data were used to analyze the above variables. Cabilio & Masaro symmetry tests were also conducted to check for the basic assumption of the Wilcoxon signed-rank test, which confirmed the null hypothesis. A p value less than 0.05 was considered statistically significant. The positive samples were also categorized in 3 groups according to the Ct values obtained from the reference method (<25, ≥25 and ≤30, >30) and Wilcoxon signed-rank tests were also performed in the groups. All statistical analyses and plots were performed in RStudio (v. 1.2.5042, RStudio Team, 2020) [15].
3. Results
Among the 400 samples examined, 12 gave discrepant results compared to the reference method and therefore, they were tested two more times with the STANDARD™ M10 SARS-CoV-2 assay (Supplementary Table 1). Of those, 7 were positive whereas 5 were negative by our reference method. After retesting the positive samples with the STANDARD™ M10 SARS-CoV-2 assay, 3 samples came out positive, 2 came out negative while 2 were presumptive positive. Retesting of the negative samples, resulted in 4 presumptive positive samples, respectively. The fifth negative sample gave all possible outcomes with the STANDARD™ M10 SARS-CoV-2 assay.
Conclusively, of the 200 positive specimens tested, 196 were identified as true positive, 2 as false negative and 2 as presumptive positive. The two presumptive samples had an ORF1ab Ct value by our reference method over 35. Regarding the 200 negative samples, 195 were identified as true negative, 0 as false positive and 4 as presumptive positive. For the calculation of specificity and sensitivity, we considered the 2 positive samples that were identified as presumptive positive from the STANDARD™ M10 SARS-CoV-2 assay as false negative and the 5 presumptive positive from the negative samples as false positive. The negative sample that gave all the possible outcomes was considered as false positive in downstream analyses. Sensitivity of the STANDARD™ M10 SARS-CoV-2 assay was estimated at 98.00% (95% CI 94.96% to 99.45%, 196/200), while specificity was estimated at 97.50% (95% CI 94.26% to 99.18%, 195/200).
Spearman's rank correlation was computed to assess the relationship between ORF1ab Ct values from the STANDARD™ M10 SARS-CoV-2 assay and the reference method. There was a very strong positive correlation between the two variables (r = 0.954, p value < 0.001, Fig. 1 ). The Ct values obtained from the STANDARD™ M10 SARS-CoV-2 assay (median= 28.45, IQR (Interquartile range)= 8.96225) were significantly lower than those obtained from the reference method (median= 29.33, IQR= 8.1875, p value < 0.001).
Fig. 1.
Spearman's rank correlation between ORF1ab Ct values from the STANDARD™ M10 SARS-CoV-2 assay and the reference method. The variables displayed a very strong positive correlation between the two variables (r = 0.954, p value < 0.001).
As previously mentioned, the positive samples were subcategorized according to their ORF1ab Ct values. The Ct values from the two methods did not differ significantly (p value= 0.778), meaning there is no difference in Ct values due to the method in samples with high viral load (Fig. 2 a). In samples with moderate viral load (Ct values ≥25 and ≤30), the Ct values from the STANDARD™ M10 SARS-CoV-2 assay (median= 26.89, IQR= 2.76) were found to be significantly smaller than the reference method (median= 27.89, IQR= 2.473, p value= 0.001408, Fig. 2b).
Fig. 2.
Ct values comparison between the STANDARD™ M10 SARS-CoV-2 assay and the reference method. (a) In samples with high viral load (Ct value <25), no differences between the two methods were detected. (b) In samples with moderate viral load (Ct value ≤25 and ≥30), the STANDARD™ M10 SARS-CoV-2 assay had a better performance than the reference method (p value= 0.001408). (c) In samples with low viral load (Ct value >30), the STANDARD™ M10 SARS-CoV-2 assay had also a better performance than the reference method (p value < 0.001).
Regarding the samples with low viral load (Ct values >30), the Ct values from the STANDARD™ M10 SARS-CoV-2 assay (median= 32.41, IQR= 2.645) were also significantly smaller than those from the reference method (median= 33.29, IQR= 2.93625, p value < 0.001, Fig. 2c).
4. Discussion
From the beginning of the SARS-CoV-2 pandemic to date, real-time RT-PCR is considered the gold standard for diagnosing the COVID-19 [7,16]. This method is highly efficient and reliable, however heavy lab equipment and trained personnel are required to perform it. Furthermore, depending on the workload, results may take several hours to receive. An increasing number of rapid point-of-care (POC) platforms receive FDA (Food and Drug Administration) EUA (Emergency Use Authorization), which allow time-efficient diagnosing. The majority of these POC platforms are basically antigen tests [16], however many other POC platforms that base upon nucleic acid amplification techniques, such as real-time RT-PCR and isothermal nucleic acid amplification (RT-LAMP), are also being used. Such platforms are the Cue COVID-19 test, Abbott ID NOW, Cepheid Xpert Xpress SARS-CoV-2 test, Roche Cobas SARS-CoV-2 & Influenza A/B on the Cobas Liat System, Mesa BioTech Accula SARS-CoV-2, and BioFire Respiratory Panel 2.1-EZ [17].
In the current study, we assessed the performance of the STANDARD™ M10 SARS-CoV-2 assay in comparison to the TaqPath™ COVID-19 CE-IVD RT-PCR kit (Applied Biosystems™) as the reference method. The test showed high overall agreement with the reference method, with 97.75% accuracy (95% CI 95.77% to 98.97%). The accuracy is similar to estimated accuracies of other corresponding POC PCR-based assays that have received FDA EUA. Specifically, the overall agreement for the Roche Cobas SARS-CoV-2 & Influenza A/B on the Cobas Liat System has been estimated at 98.6% [18], while for the Cepheid Xpert Xpress SARS-CoV-2 test an accuracy of 99% has been reported [19]. Finally, the STANDARD™ M10 SARS-CoV-2 assay displayed lower Ct values compared to the reference method for samples with a Ct value above 25, since it was able to detect the ORF1ab target one cycle earlier, on average. This difference confirms the lower reported LoD of the index method compared to the reference method.
Two other studies have also evaluated the STANDARD™ M10 SARS-CoV-2 assay for its use at POC sites. The first evaluation study estimated the sensitivity and specificity of the assay for the ORF1ab target at 95.5% (95% CI 91.7% to 97.6%) and 100% (95% CI 98.7% to 100%), respectively [20]. Furthermore, they also confirmed the lower LoD of the STANDARD™ M10 SARS-CoV-2 assay, compared to the reference methods they used. The second evaluation study followed a more qualitative approach to evaluate the assay and estimated both sensitivity and specificity of 100% [6]. Our estimate of sensitivity (98%) is close to what is reported to the previous two evaluations, however the specificity we report here is lower (97.5%). The reason for this may be the more conservative way we handled the presumptive positive results, since we handled them as false positive results.
Regarding the discordant results between the reference test and the index test, we did not need to repeat the reference test. This decision was taken because all discordant samples came from people that were hospitalized. Therefore, we know that all positive samples that were discordant with the reference test came from diagnosed with COVID-19 people, within the last 15 days. Respectively, the negative samples came from people that were hospitalized for irrelevant reasons and had been tested at least once more within the last week. Thus, we used the result from the reference test as the basis for comparisons, and this is also the reason why we chose to treat the presumptive positive results from the index test as false negative or false positive, depending on the case.
At the time period we conducted the study, between January 2022 and March 2022, three SARS-CoV-2 variants were predominant across Crete, Delta variant and two Omicron variants (BA.1 και BA.2). Furthermore, the Omicron BA.1 variant was present throughout the whole time period of the study, while the Delta variant was present until middle of February. The Omicron BA.2 variant emerged at the end of February. The TaqPath™ COVID-19 CE-IVD RT-PCR kit (Applied Biosystems™) used as the reference PCR, could indicate prevalence of the variants through S gene drop out, that was characteristic of the Omicron BA.1 variant, therefore depending on the time period we could assume the possible variant present by the S gene drop out. The STANDARD™ M10 SARS-CoV-2 assay successfully detected the SARS-CoV-2 RNA, regardless the variants’ prevalence. Therefore, the efficacy of the assay is not mutation-dependent.
In conclusion, the evaluation of the STANDARD™ M10 SARS-CoV-2 assay demonstrated similar performance to a high-throughput laboratory real-time RT-PCR reference method and it is capable of accurate and rapid diagnosis of COVID-19 at the point of care. In particular, it has a run-time less than 60 min, depending on the viral load of the sample, with hands on time 2 to 3 min, with no need for external viral RNA purification process, while minimum lab equipment is required. Thus, we conclude that this assay can be a valuable tool in cases where immediate decision making is required for treatment or infection control policies.
Funding
The reagents for the STANDARD™ M10 SARS-CoV-2 assay were provided by SD BIOSENSOR (Republic of Korea). SD BIOSENSOR did not interfere with the evaluation of the test, the interpretation of the results or the preparation of the manuscript.
CRediT authorship contribution statement
Maria-Eleni Parakatselaki: Investigation, Formal analysis, Writing – original draft. Georgia Alexi: Investigation, Writing – review & editing. Alexandros Zafiropoulos: Writing – review & editing, Supervision. George Sourvinos: Writing – review & editing, Supervision, Conceptualization.
Conflicts of interest
The authors declare no conflict of interest.
Footnotes
Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.jcvp.2022.100129.
Appendix. Supplementary materials
References
- 1.Wang C., Horby P.W., Hayden F.G., Gao G.F. A novel coronavirus outbreak of global health concern. Lancet. 2020;395:470–473. doi: 10.1016/S0140-6736(20)30185-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.WHO . 2020. WHO Director-General's opening Remarks At the Media Briefing On COVID-19 - 11 March 2020.https://www.who.int/director-general/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19-11-march-2020 accessed April 1, 2022. [Google Scholar]
- 3.Khailany R.A., Safdar M., Ozaslan M. Genomic characterization of a novel SARS-CoV-2. Gene. Reports. 2020;19 doi: 10.1016/J.GENREP.2020.100682. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Ramírez J.D., Muñoz M., Hernández C., Flórez C., Gomez S., Rico A., et al. Genetic diversity among SARS-CoV2 strains in South America may impact performance of molecular detection. Pathog. 2020;9:580. doi: 10.3390/PATHOGENS9070580. Page2020;9:580. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Udugama B., Kadhiresan P., Kozlowski H.N., Malekjahani A., Osborne M., Li V.Y.C., et al. Diagnosing COVID-19: the Disease and Tools for Detection 2020. https://doi.org/10.1021/acsnano.0c02624. [DOI] [PubMed]
- 6.Ham S.Y., Jeong H., Jung J., Kim E.S., Park K.U., Bin Kim H, et al. Performance of STANDARD™ M10 SARS-CoV-2 Assay for the Diagnosis of COVID-19 from a Nasopharyngeal Swab. Infect. Chemother. 2022;54:360–363. doi: 10.3947/ic.2022.0047. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Kevadiya B.D., Machhi J., Herskovitz J., Oleynikov M.D., Blomberg W.R., Bajwa N., et al. Diagnostics for SARS-CoV-2 infections. Nat. Mater. 2021;20:593–605. doi: 10.1038/s41563-020-00906-z. 2021 205. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Hong K.H., Lee S.W., Kim T.S., Huh H.J., Lee J., Kim S.Y., et al. Guidelines for Laboratory Diagnosis of Coronavirus Disease 2019 (COVID-19) in Korea. Ann Lab Med. 2020;40:351–360. doi: 10.3343/ALM.2020.40.5.351. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Domnich A., De Pace V., Pennati B.M., Caligiuri P., Varesano S., Bruzzone B., et al. Evaluation of extraction-free RT-qPCR methods for SARS-CoV-2 diagnostics. Arch. Virol. 2021;166:2825–2828. doi: 10.1007/s00705-021-05165-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Ramdas K., Darzi A., Jain S. Test, re-test, re-test’: using inaccurate tests to greatly increase the accuracy of COVID-19 testing. Nat. Med. 2020;26:810–811. doi: 10.1038/s41591-020-0891-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Dinnes J., Sharma P., Berhane S., van Wyk S.S., Nyaaba N., Domen J., et al. Rapid, point-of-care antigen tests for diagnosis of SARS-CoV-2 infection. Cochrane Database Syst. Rev. 2022 doi: 10.1002/14651858.CD013705.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Pu R., Liu S., Ren X., Shi D., Ba Y., Huo Y., et al. The screening value of RT-LAMP and RT-PCR in the diagnosis of COVID-19: systematic review and meta-analysis. J. Virol. Methods. 2022;300 doi: 10.1016/j.jviromet.2021.114392. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Fujita-Rohwerder N., Beckmann L., Zens Y., Verma A. Diagnostic accuracy of rapid point-of-care tests for diagnosis of current SARS-CoV-2 infections in children: a systematic review and meta-analysis. BMJ Evidence-Based Med. 2022 doi: 10.1136/bmjebm-2021-111828. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Altman D.G., Bland J.M. Diagnostic tests. 1: sensitivity and specificity. BMJ Br. Med. J. 1994;308:1552. doi: 10.1136/BMJ.308.6943.1552. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.RStudio Team. RStudio: integrated development environment for R 2020.
- 16.Hart C.R., McLendon P.M., Naik R.R. Dealing with a pandemic: emerging tools, solutions, and challenges. https://HomeLiebertpubCom/Hs 2022. https://doi.org/10.1089/HS.2021.0145. [DOI] [PubMed]
- 17.Donato L.J., Trivedi V.A., Stransky A.M., Misra A., Pritt B.S., Binnicker M.J., et al. Evaluation of the Cue Health point-of-care COVID-19 (SARS-CoV-2 nucleic acid amplification) test at a community drive through collection center. Diagn. Microbiol. Infect. Dis. 2021;100 doi: 10.1016/J.DIAGMICROBIO.2020.115307. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Hansen G., Marino J., Wang Z.-X., Beavis K.G., Rodrigo J., Labog K., et al. Clinical performance of the point-of-care cobas liat for detection of SARS-CoV-2 in 20 Min: a Multicenter Study. 2021. [DOI] [PMC free article] [PubMed]
- 19.Stevens B., Hogan C.A., Sahoo M.K., Huang C.H., Garamani N., Zehnder J., et al. Comparison of a point-of-care assay and a high-complexity assay for detection of SARS-CoV-2 RNA. J. Appl. Lab. Med. 2020;5:1307–1312. doi: 10.1093/JALM/JFAA135. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Domnich A., Orsi A., Trombetta C.-S., Costa E., Guarona G., Lucente M., et al. Clinical medicine comparative diagnostic accuracy of the STANDARD M10 Assay for the Molecular Diagnosis of SARS-CoV-2 in the Point-of-Care and Critical Care Settings. J. Clin. Med. 2022;2022:2465. doi: 10.3390/jcm11092465. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.