Abstract
This research describes the development of a new multiplex real-time RT-PCR test for detection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), with primers designed to amplify a 108 bp target on the spike surface glycoprotein (S gene) and a hydrolysis TaqMan probe designed to specifically detect SARS-CoV-2. The limit of detection (LOD) and clinical performance of this new assay were evaluated. A LOD study with inactivated virus exhibited performance equal to the modified CDC assay, with a final LOD of 1301 ± 13 genome equivalents/mL for the Northwell Health Laboratories laboratory-developed test (NWHL LDT) versus 1249 ± 14 genome equivalents/mL for the modified CDC assay. In addition, a clinical evaluation with 270 nasopharyngeal swab specimens exhibited 98.5% positive percent agreement and 99.3% negative percent agreement compared with the modified CDC assay. The NWHL LDT multiplex design allows testing of 91 patients per plate, versus a maximum of 29 patients per plate on the modified CDC assay, providing the benefit of testing significantly more patients per run and saving reagents, during a time when both of these parameters are critical. The results show that the NWHL LDT multiplex assay performs as well as the modified CDC assay but is more efficient and cost-effective and can be used as a diagnostic assay and for epidemiologic surveillance and clinical management of SARS-CoV-2.
Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) was first discovered as an etiologic agent of coronavirus disease 2019 (COVID-19) in the city of Wuhan, Hubei Province, China, by the end of December 2019.1 It is the seventh coronavirus known to infect humans and also to be transmitted from human to human. The four seasonal coronaviruses (HKU1, NL63, OC43, and 229E) are associated with mild symptoms, whereas SARS-CoV, Middle East respiratory syndrome-related coronavirus, and SARS-CoV-2 can cause severe acute respiratory disease.2 , 3 SARS-CoV-2 belongs to the Betacoronavirus genus and is an enveloped, single-strand RNA virus with an approximately 29.8 kb genome, which can cause a wide range of clinical presentations from asymptomatic or mild illness to fatal outcomes.4 , 5 Thus far, COVID-19 has resulted in a dramatic loss of lives globally; infections and deaths are being tracked by the COVID-19 Dashboard by the Center for Systems Science and Engineering at Johns Hopkins University (https://www.arcgis.com/apps/opsdashboard/index.html#/bda7594740fd40299423467b48e9ecf6). Furthermore, the symptoms of patients with COVID-19 can be similar to those of patients with other seasonal respiratory infections. Presently, there are no available specific therapeutics or vaccinations against COVID-19, making early and accurate diagnosis for this very contagious disease the key mitigation strategy.
Nucleic acid amplification test–based assays for detection of SARS-CoV-2 in respiratory specimens have been the standard diagnostic method. To date, the US Food and Drug Administration has issued 107 laboratory-developed COVID-19 molecular assays Emergency Use Authorization (US Food and Drug Administration, https://www.fda.gov/medical-devices/coronavirus-disease-2019-covid-19-emergency-use-authorizations-medical-devices/vitro-diagnostics-euas, last accessed July 13, 2020). The US CDC developed a SARS-CoV-2 assay that was initially the most widely used. This assay includes primers and probes to detect the N1 and N2 regions of the nucleocapsid gene and also the human RNase P gene to monitor RNA extraction and ensure specimen quality. The modified CDC assay has been shown to have high analytical sensitivity and ideal clinical performance compared with three commercially available COVID-19 diagnostic platforms issued Emergency Use Authorization status by the US Food and Drug Administration.6
The aim of the current study was to develop and evaluate the analytical sensitivity and clinical performance of an efficient and cost-effective test on the Applied Biosystems 7500 Fast Dx Real-Time PCR Instrument (Thermo Fisher Scientific, Waltham, MA). To that end, the Northwell Health Laboratories laboratory-developed test (NWHL LDT) was developed to target the S gene of SARS-CoV-2, and its clinical performance compared with that of the modified CDC assay for the detection of SARS-CoV-2 in nasopharyngeal (NP) specimens from individuals suspected of having COVID-19.
Materials and Methods
Primers and Probe Design
Available whole-genome sequence of SARS-CoV-2 (as of February 27, 2020) retrieved from the National Center for Biotechnology Information GenBank database (https://www.ncbi.nlm.nih.gov/nuccore/?term=SARS-CoV-2) and the Global Initiative on Sharing All Influenza Database (https://www.gisaid.org) were aligned by using Clustal Omega software from EMBL-EBI (Wellcome Trust Genome Campus, Hinxton, Cambridgeshire, UK). The designed primers and probe exhibited 100% similarity with the targeted region of the S gene of all SARS-CoV-2 genome sequences only (Table 1 ). The primers and probe were designed by using Primer Express software version 3.0 (Thermo Fisher Scientific) in the S gene of SARS-CoV-2 and were synthesized by Integrated DNA Technologies, Inc. (Coralville, IA). In addition, the primers and probe of the human RNase P gene used for the assay internal control were also synthesized by Integrated DNA Technologies, Inc., and were the same sequences used in the CDC assay (v3) for this gene target (CDC, https://www.cdc.gov/coronavirus/2019-nCoV/lab/index.html, last accessed July 13, 2020). Here, the 5′ base of the probe of the RNase P gene was modified and labeled with Cy5, and the 3′ base of the probe was labeled with Black Hole Quencher 2 to allow multiplexing of the assay.
Table 1.
Real-Time RT-PCR Primer/Probe Set of S Gene Target for Specific Detection of SARS-CoV-2 and Primer/Probe Set of RNase P Gene
| Target | Name | Sequence∗ | Nucleotide position |
|---|---|---|---|
| S | S Gene-Fwd | 5′-TCAACTCAGGACTTGTTCTTAC-3′ | 21,710∼21,731 |
| S Gene-Rev | 5′-TGGTAGGACAGGGTTATCAAAC-3′ | 21,796∼21,817 | |
| S Gene-Probe | 5′-FAM-TGGTCCCAGAGACATGTATAGCAT-BHQ1-3′† | 21,759∼21,782 | |
| RNase P | RP-Fwd | 5′-AGATTTGGACCTGCGAGCG-3′ | |
| RP-Rev | 5′-GAGCGGCTGTCTCCACAAGT-3′ | ||
| RP-Probe | 5′-Cy5-TTCTGACCTGAAGGCTCTGCGCG-BHQ2-3′‡ |
The nucleotide numbering is based on that of the curated severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) whole-genome sequence (GenBank, https://www.ncbi.nlm.nih.gov/nuccore; accession number MN908947.3).
The 5′ base of the S gene probe was labeled with FAM, and the 3′ base of the probe was labeled with Black Hole Quencher 1.
The 5′ base of the probe of the RNase P gene was labeled with Cy5, and the 3′ base of the probe was labeled with Black Hole Quencher 2.
RNA Extraction
Total RNA was extracted from 110 μL of patient NP specimen collected in 3 mL of viral transport medium or 110 μL of contrived specimen from the serial dilutions panel by the NucliSENS easyMag platform (BioMérieux, Durham, NC) according to the manufacturer's instructions; the final elution volume was 110 μL. To monitor the extraction process, a negative extraction control was included in each extraction run; the RNase P target must be detected from this control.
NWHL LDT
In this one-step, real-time qualitative RT-PCR assay, a TaqPath 1-step real-time quantitative PCR kit (catalog number A15299; Thermo Fisher Scientific) was used to perform cDNA synthesis and PCR amplification on the 96-well plate at a 20 μL final reaction volume. After the PCR condition was optimized, the PCR reaction mix contained 5 μL of 4× RT-PCR Master Mix, 0.72 μL of S gene forward primer at 25 μmol/L, 0.72 μL of S gene reverse primer at 25 μmol/L, 0.16 μL of S gene probe at 25 μmol/L, 0.64 μL of RNase P gene forward primer at 25 μmol/L, 0.64 μL of RNase P gene reverse primer at 25 μmol/L, 0.16 μL of RNase P probe at 25 μmol/L, 6.96 μL of nuclease-free water, and 5 μL of extracted RNA. The thermal cycler profile consisted of 25°C for 2 minutes, 50°C for 15 minutes, and 95°C for 2 minutes, followed by 45 cycles at 95°C for 3 seconds and 55°C for 30 seconds; this was conducted on the 7500 Fast Dx Real-Time PCR Instrument. Each run included a No Template Control, a Negative Extraction Control, and a SARS-CoV-2–positive control. In this study, a Negative Extraction Control was prepared by suspending a noninfectious human HEL cell line (provided by the New York State Department of Health) into a pre-aliquoted 2 mL lysis tube; a SARS-CoV-2 RNA transcript (SARS-CoV-2 standard) containing five genes (E, N, S, ORF1ab, and RdRP), with a titer of 200,000 copies/mL for each gene from Exact Diagnostics (catalog number COVID19; Fort Worth, TX), was diluted with extracted nucleic acid from human specimens to make a SARS-CoV-2–positive control. The final concentration of SARS-CoV-2 with 50,000 copies/mL was aliquoted into 30 μL of a nuclease-free tube and stored at −70°C to −80°C for use. The NWHL LDT used the same RT-PCR reagents and conditions as the modified CDC assay, except for the target primer and probe.
Analytical Sensitivity (Limit of Detection)
Limit of detection (LOD) was determined by extracting and testing serial dilutions of quantified inactivated SARS-CoV-2 from Isolate USA-WA1/2020 (NR-52287, BEI Resources, Manassas, VA). SARS-CoV-2 viral material was provided at a concentration of 4.1 × 109 genome equivalents (GE)/mL, from which the following serial dilutions were prepared in GE/mL: 4000, 2000, 1000, 500, 250, and 125. Ambion RNA Storage Solution (catalog number AM7001; Thermo Fisher Scientific) was used to prevent the potential RNA degradation, and replicates ranging from 4 to 10 at each dilution went through nucleic acid extraction on different days and were tested on both the modified CDC assay and the NWHL LDT. LOD was defined as the concentration of the lowest dilution that can be detected with >95% probability and was determined by using probit analysis.
Analytical Specificity
The specificity of the NWHL LDT primers and probe for SARS-CoV-2 detection was evaluated by using in silico analysis and by testing a SARS-CoV control with a concentration of 20,000 RNA transcript copies/mL (GenBank, https://www.ncbi.nlm.nih.gov/nuccorea; accession number MG772933.1) (catalog number 10006624; Integrated DNA Technologies, Inc.), Middle East respiratory syndrome–related coronavirus control with concentration of 20,000 RNA transcript copies/mL (GenBank, https://www.ncbi.nlm.nih.gov/nuccore; accession number MK796425.1) (catalog number 10006623; Integrated DNA Technologies, Inc.), and clinical specimens positive for respiratory pathogens (n = 41) covering coronavirus (229E, NL63, OC43, and HKU1), influenza A H3, influenza B, 2009 H1N1, respiratory syncytial virus A, respiratory syncytial virus B, parainfluenza virus type 1 to 4, human metapneumovirus, adenovirus, Mycoplasma pneumoniae, and Chlamydia pneumoniae; these were initially identified by using a multiplex respiratory panel between October 2018 and December 2019 at NWHL.
Study Design
A modified version (v3) of the CDC assay was used as a reference method in this study. A total of 270 NP specimens (130 positive and 140 negative specimens) originally submitted for SARS-CoV-2 testing at NWHL between March and April 2020 were selected for this study. The 270 specimens were initially tested by using the modified CDC assay, and extracted RNA was stored at −80°C until testing with the NWHL LDT was performed; all NP specimens after the CDC assay were aliquoted and kept in a freezer at −80°C. The specimens were selected as any consecutive specimen that was performed on the modified CDC assay and represented the true positivity rate (approximately 50%) during the time frame of this study, comprising positive specimens spanning the range of positivity levels. These specimens included 24 with low viral load characterized by a high cycle threshold (Ct) value in the range of 31.7 to 39.7 according to the modified CDC assay. For discordant results, a new RNA extraction was performed from stored, frozen (−80°C) specimens, and testing was repeated on both assays. This study was performed to validate the NWHL LDT for clinical use.
Statistical Methods
The final result interpretation algorithm for reporting a positive specimen requires both N1 and N2 targets to be detected in the modified CDC assay, and both the modified CDC assay and the NWHL LDT use a Ct <40 as the criterion for positivity. Percent positive agreement, percent negative agreement, kappa analysis, and two-sided (upper/lower) 95% CI were calculated by using Microsoft Office Excel 365 MSO Professional Plus 2013 software (Microsoft, Redmond, WA). As a measure of overall agreement, Cohen’s kappa values were calculated, with values categorized as follows: >0.90 = almost perfect, 0.90 to 0.80 = strong, 0.79 to 0.60 = moderate, 0.59 to 0.40 = weak, 0.39 to 0.21 = minimal, and 0.20 to 0 = none.7 , 8
Results
Design of Primers and Probe for SARS-CoV-2 Detection
Using a primer and probe design tool, the assay primers and probe specifically targeting the S gene of SARS-CoV-2 (Table 1) were designed to amplify a 108 bp target on the conserved S gene based on multiple sequence alignments and in silico analysis for the NWHL LDT primers and probe set. The primers and probe designed for S gene detection were conceived by using all available SARS-CoV-2 whole-genome sequences. The analysis of sequence alignments revealed that the region of the S gene of SARS-CoV-2 targeted by the designed primer and probe set had 100% similarity with all available SARS-CoV-2 whole-genome sequences from the GenBank database and the Global Initiative on Sharing All Influenza Database at the time of development. When the S gene of other coronaviruses was analyzed, including SARS-CoV, Middle East respiratory syndrome–related coronavirus, the four seasonal coronaviruses (HKU1, OC43, NL63, and 229E), and bat RaTG13, the closest match to SARS-CoV-2 (which was bat RaTG13) had a total of eight nucleotide mismatches with the NWHL primer/probe set, indicating that this design is highly specific for SARS-CoV-2.
Analytical Sensitivity
The LOD of the NWHL LDT was 1301 ± 13 GE/mL for the S gene target. For the modified CDC assay, the LOD was 1249 ± 14 GE/mL for the N1 target and 946 ± 11 GE/mL for the N2 target (Table 2 ). The final LOD of the modified CDC assay was 1249 ± 14 GE/mL in accordance with the result interpretation algorithm.
Table 2.
Summary of LOD Results
| Molecular assay | Target | No. of replicates detected/total no. of replicates at each dilution expressed by GE/mL (% positive rate) |
Probit (±95% CI),∗ GE/mL | Final LOD,† GE/mL | |||||
|---|---|---|---|---|---|---|---|---|---|
| 4000 | 2000 | 1000 | 500 | 250 | 125 | ||||
| NWHL LDT | S | 4/4 | 6/6 (100) | 9/10 (90) | 5/10 (50) | 2/8 (25) | 1/8 (25) | 1301 ± 13 | 1301 ± 13 |
| Modified CDC assay | N1 | 4/4 | 6/6 (100) | 9/10 (90) | 7/10 (70) | 6/8 (75) | 1/8 (25) | 1249 ± 14 | 1249 ± 14 |
| N2 | 4/4 | 6/6 (100) | 10/10 (100) | 5/10 (50) | 2/8 (25) | 1/8 (25) | 946 ± 11 | ||
GE/mL, genome equivalents/mL; LOD, limit of detection; NWHL LDT, Northwell Health Laboratories laboratory-developed test.
Upper/lower (±) 95% CI.
The final LODs were based on both assays’ results interpretation algorithm and determined by using probit analysis.
Analytical Specificity
In silico analysis and blastn analysis were performed against the standard and betacoronavirus database of the National Center for Biotechnology Information, and results showed no cross-reactivity with other respiratory pathogens. Clinical specimens positive for respiratory pathogens were also tested to further evaluate the specificity of the NWHL LDT. According to the results shown in Table 3 , a specificity of 100% was achieved, with the NWHL LDT exhibiting no cross-reaction with any of the pathogens tested.
Table 3.
Respiratory Pathogens Included in the Analytical Specificity Assessment
| Respiratory pathogens tested | SARS-CoV-2 result |
|---|---|
| SARS-CoV (MG772933.1) | Not detected |
| MERS-CoV (MK796425.1) | Not detected |
| Human coronavirus 229E (n = 4) | Not detected |
| Human coronavirus HKU-1 (n = 4) | Not detected |
| Human coronavirus NL63 (n = 4) | Not detected |
| Human coronavirus OC43 (n = 4) | Not detected |
| Adenovirus (n = 2) | Not detected |
| Enterovirus/rhinovirus (n = 2) | Not detected |
| Human metapneumovirus (n = 2) | Not detected |
| Influenza A H1-2009 (n = 2) | Not detected |
| Influenza A H3 (n = 2) | Not detected |
| Influenza B (n = 2) | Not detected |
| Parainfluenza virus 1 (n = 1) | Not detected |
| Parainfluenza virus 2 (n = 1) | Not detected |
| Parainfluenza virus 3 (n = 1) | Not detected |
| Parainfluenza virus 4 (n = 2) | Not detected |
| Respiratory syncytial virus A (n = 2) | Not detected |
| Respiratory syncytial virus B (n = 2) | Not detected |
| Chlamydia pneumoniae (n = 2) | Not detected |
| Mycoplasma pneumoniae (n = 2) | Not detected |
MERS-CoV, Middle East respiratory syndrome-related coronavirus; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.
Clinical Performance
A total of 270 clinical NP specimens were tested with the NWHL LDT and the modified CDC assay at NWHL. Overall, the NWHL LDT showed a percent positive agreement of 98.5% (95% CI, 0.946–0.996) and a percent negative agreement of 99.3% (95% CI, 0.961–0.999); the overall percent agreement between both assays was 98.9%. A kappa value of 0.978 (95% CI, 0.953–1.0) indicated perfect agreement (Table 4 ). For discordant results from three specimens, two specimens were detected by the modified CDC assay and were not detected by the NWHL LDT; they had a Ct value of 37.3 for N1 and 39.6 for N2, and 38.2 for N1 and 39.2 for N2, respectively. After a fresh RNA extraction and retesting, the specimen with 37.3 N1 and 39.6 N2 Ct values was detected by both assays, and the other specimen had a “not detected” result by both assays. One specimen was initially detected by the NWHL LDT and not by the modified CDC assay, exhibiting a Ct value of 32.0 on the LDT. After retesting, both the modified CDC assay and the NWHL LDT assay results were positive.
Table 4.
Clinical Performance Comparison between the NWHL LDT and Modified CDC Assay for the Detection of SARS-CoV-2 RNA (n = 270)
| NWHL LDT | Modified CDC assay |
Kappa (±95% CI)∗ | PPA (±95% CI)∗ | NPA (±95% CI)∗ | |
|---|---|---|---|---|---|
| Detected | Not detected | ||||
| Detected | 128 | 1 | 0.978 (0.953–1.0) | 98.5% (0.946–0.996) | 99.3% (0.961–0.999) |
| Not detected | 2 | 139 | |||
NPA, percent negative agreement; NWHL LDT, Northwell Health Laboratories laboratory-developed test; PPA, percent positive agreement; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.
Upper/lower (±) 95% CI.
Discussion
In the current study, a multiplex real-time RT-PCR assay was developed and validated for SARS-CoV-2–specific detection in NP specimens on the 7500 Fast Dx Real-Time PCR Instrument. The findings show that the NWHL LDT has comparable clinical performance for the specific detection of SARS-CoV-2 RNA in NP specimens and is more efficient and cost-effective compared with the modified CDC assay. Because the modified CDC assay has been compared with multiple other commercially available diagnostic assays and has equal or better performance, the performance of the NWHL LDT would also show equivalent performance.6 The design also revealed significant advantages over the modified CDC assay, as only one set of primer and probe Master Mix is required to prepare and dispense per specimen, in contrast to three sets of Master Mix preparation and the use of three wells for each patient specimen with the modified CDC assay. The multiplex design of the NWHL LDT allows testing of 91 patients per plate, versus a maximum of 29 patients per plate on the modified CDC assay. Overall, this design allows laboratories to run more than three times as many patients per run and also adds to the ease of setting up each run. In addition, the saving of hands-on time, reagents, and consumables is another advantage at a stage where there are currently global shortages of reagents and major assay supply chain issues.
The design of the primers and probe for the NWHL LDT is based on multiple sequence alignments of all SARS-CoV-2 genome sequences that were available between January 11, 2020, and February 27, 2020. Because RNA viruses are well known for their high mutation and recombination rates,9 it was important to confirm that there were no significant new mutations of the region of the S gene of SARS-CoV-2 targeted by the NWHL LDT that may affect assay performance. Therefore, an additional 140 SARS-CoV-2 genome sequences uploaded after February 27, 2020, to the GenBank database and the Global Initiative on Sharing All Influenza Database from different countries were analyzed, and an alignment was performed with Clustal Omega. This alignment showed that the forward primer and probe are conserved (with 100% homology) to the S gene target regions of the SARS-CoV-2 sequences. One exception was seen with the reverse primer, a single base mismatch of S gene target region in one sequence (GenBank, https://www.ncbi.nlm.nih.gov/nuccore; accession number MT385417.1) from a total of 240 SARS-CoV-2 sequences analyzed both before and after late February. This one mismatch is questionable, because it is not in keeping with the other sequences available; the databases are not curated and therefore occasionally contain errors. The Simplexa COVID-19 Direct (DiaSorin Molecular LLC, Cypress, CA), which has US Food and Drug Administration Emergency Use Authorization approval for SARS-CoV-2, also targets the S gene and shows a slightly lower LOD than the NWHL LDT in their Emergency Use Authorization instructions for use; however, the material used for the Simplexa LOD study differed, making a clear comparison difficult.
Limitations of the current study include that the NWHL LDT is a single-site evaluation at NWHL, and only a single target gene for SARS-CoV-2 detection was used. Although there has been a trend toward dual-target design in commercial assays for the detection of this highly contagious pathogen,6 , 10 , 11 occasional monitoring of SARS-CoV-2 sequences to verify that mutations have not developed in the region targeted by the NWHL LDT primers and probe is an adequate quality monitor to ensure continued consistent analytical performance. In addition, other targets may be added to this assay in the future to further increase sensitivity and specificity. As an additional future plan, the analytical sensitivity of the NWHL LDT may be improved further by increasing the input volume and decreasing the elution volume during the nucleic acid extraction step.
In summary, the NWHL LDT has comparable analytical sensitivity and accuracy for specific detection of SARS-CoV-2 RNA and also exhibits superior efficiency and cost-effectiveness compared with the modified CDC assay. Other laboratories may also be able to establish this assay with ease for diagnostic use.
Acknowledgment
We thank Dr. Kirsten St. George and Daryl Lamson of the New York State Department of Health for providing the noninfectious human HEL cell line.
Footnotes
Supported by the Northwell Health Department of Pathology and Laboratory Medicine.
Disclosures: None declared.
References
- 1.Zhou P., Yang X.L., Wang X.G., Hu B., Zhang L., Zhang W., Si H.R., Zhu Y., Li B., Huang C.L., Chen H.D., Chen J., Luo Y., Guo H., Jiang R.D., Liu M.Q., Chen Y., Shen X.R., Wang X., Zheng X.S., Zhao K., Chen Q.J., Deng F., Liu L.L., Yan B., Zhan F.X., Wang Y.Y., Xiao G.F., Shi Z.L. A pneumonia outbreak associated with a new coronavirus of probable bat origin. Nature. 2020;579:270–273. doi: 10.1038/s41586-020-2012-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Corman V.M., Muth D., Niemeyer D., Drosten C. Hosts and sources of endemic human coronavirus. Adv Virus Res. 2018;100:163–188. doi: 10.1016/bs.aivir.2018.01.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Fauci A.S., Lane H.C., Redfield R.R. Covid-19—navigating the uncharted. N Engl J Med. 2020;382:1268–1269. doi: 10.1056/NEJMe2002387. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Wu A., Peng Y., Huang B., Ding X., Wang X., Niu P., Meng J., Zhu Z., Zhang Z., Wang J.Y., Sheng J., Quan L.J., Xia Z.X., Tan W.J., Cheng G.H., Jiang T.J. Genome composition and divergence of the novel coronavirus (2019-nCoV) originating in China. Cell Host Microbe. 2020;27:325–328. doi: 10.1016/j.chom.2020.02.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Yuen K.S., Ye Z.W., Fung S.Y., Chan C.P., Jin D.Y. SARS-CoV-2 and COVID-19: the most important research questions. Cell Biosci. 2020;10:40. doi: 10.1186/s13578-020-00404-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Zhen W., Manji R., Smith E., Berry G.J. Comparison of four molecular in vitro diagnostic assays for the detection of SARS-CoV-2 in nasopharyngeal specimens. J Clin Microbiol. 2020;58 doi: 10.1128/JCM.00743-20. e00743-20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Landis J.R., Koch G.G. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–174. [PubMed] [Google Scholar]
- 8.McHugh M.L. Interrater reliability: the kappa statistic. Biochem Med (Zagreb) 2012;22:276–282. [PMC free article] [PubMed] [Google Scholar]
- 9.Domingo E., Holland J.J. RNA virus mutations and fitness for survival. Annu Rev Microbiol. 1997;51:151–178. doi: 10.1146/annurev.micro.51.1.151. [DOI] [PubMed] [Google Scholar]
- 10.Zhen W., Smith E., Manji R., Schron D., Berry G.J. Clinical evaluation of three sample-to-answer platforms for the detection of SARS-CoV-2. J Clin Microbiol. 2020;58 doi: 10.1128/JCM.00783-20. e00783-20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Uhteg K., Jarrett J., Richards M., Howard C., Morehead E., Geahr M., Gluck L., Hanlon A., Ellis B., Kaur H., Simner P., Karen C., Carroll K.C., Mostafa H.H. Comparing the analytical performance of three SARS-CoV-2 molecular diagnostic assays. J Clin Virol. 2020;127:104384. doi: 10.1016/j.jcv.2020.104384. [DOI] [PMC free article] [PubMed] [Google Scholar]