Abstract
Introduction
To evaluate the automated cartridge-based PCR approach ARIES SARS-CoV-2 Assay targeting the ORF-sequence and the N-gene of SARS-CoV-2.
Methods
In line with the suggestions by Rabenau and colleagues, the automated ARIES SARS-CoV-2 Assay was challenged with strongly positive samples, weakly positive samples and negative samples. Further, intra-assay and inter-assay precision as well as the limit-of-detection (lod) were defined with quantified target RNA and DNA. The Cepheid Xpert Xpress SARS-Cov-2 Assay was used as gold standard.
Results
Concordance between the ARIES assay and the Cepheid assay was 100% for strongly positive samples and for negative samples, respectively. For weakly positive samples as confirmed applying the Cepheid assay, a relevant minority of 4 out of 15 samples (26.7%) went undetected by the ARIES assay. Intra- and inter-assay precision were satisfactory, while the lod was in the 103 DNA copies/reaction-range, in the 103 virus copies/reaction-range, or in the 103-104 free RNA copies/reaction-range in our hands.
Conclusions
The automated ARIES assay shows comparable test characteristics as the Cepheid assay focusing on strongly positive and negative samples but a slightly reduced sensitivity with weakly positive samples. Decisions on diagnostic use should include considerations on the lod.
KEYWORDS: SARS-CoV-2, COVID-19, point-of-care-testing, automated PCR, molecular rapid testing, testing comparison
INTRODUCTION
The global medical and public health efforts associated with the emergence of the COVID-(Corona Virus Disease-) 19-associated SARS-CoV-2 (Severe Acute Respiratory Syndrome-Corona Virus-2) starting from Wuhan, China, in December 2019 [1] makes the availability of rapid and reliable diagnostic approaches desirable. First polymerase chain reaction (PCR) protocols were rapidly established [2] and made available by organizations like the Center of Disease Control (CDC) [3, 4] or the China CDC [5].
However, traditional in-house real-time PCR is demanding both regarding infrastructural conditions and availability of skilled and well-trained technical assistants [6]. Accordingly, there is demand for rapid and automated diagnostic devices. With positive molecular SARS-CoV-2 test results within less than 10 minutes, the isothermal Abbott ID Now (Abbott Laboratories, Abbott Park, Illinois, USA) approach is presently the most rapid well evaluated molecular point-of-care-testing assay for the diagnosis of COVID-19 but it is outperformed regarding sensitivity by PCR-based assays [7–10]. One well-evaluated, highly sensitive automated PCR-based testing approach is the cartridge-based Xpert Xpress SARS-CoV-2 Assay (Cepheid, Sunnyvale, California, USA) with a limit of detection of less than 100 copies [11-16].
Recently, another automated cartridge-based system ARIES SARS-CoV-2 Assay (Luminex Corporation, Austin, Texas, United States) was cleared by the US Food and Drug Administration (FDA) for in-vitro-diagnostics under emergency use authorization (EUA-IVD), while the system was distributed under a research-use-only (RUO) label in Europa. A validation of the ARIES SARS-CoV-2 Assay against the well described above-mentioned Cepheid assay in line with the recommendations according to Rabenau and colleagues [17, 18] was performed to assess its suitability for in-vitro diagnostic use.
METHODS
Study protocol
The study was conducted as a test validation using the batch number AB0629 of the automated cartridge-based ARIES SARS-CoV-2 Assay (Luminex, referred to as “ARIES assay” in the following) targeting the ORF-sequence and the N-gene with the well-evaluated Xpert Xpress SARS-CoV-2 Assay (Cepheid, referred to as “Cepheid assay” in the following) targeting the E-gene and the N2-region of the N-gene as the gold standard according to the validation protocol as suggested for laboratory-developed real-time PCR assays by Rabenau and colleagues [17, 18]. This implies comparative testing of strongly positive samples (defined as Cepheid cycle threshold (Ct) value ≤34 for all target genes), weakly positive samples (defined as Cepheid Ct value >34 for at least one target gene), negative samples according to the gold standard, inter-assay and intra-assay precision testing as well as the definition of the limit of detection (lod). Here, the lod was determined using patient samples with defined SARS-CoV-2 copy numbers, positive control plasmids and synthesized positive control RNA. Both assays were applied as described by the manufacturer with 200 μL sample used for the ARIES system and 300 μL sample used for the Cepheid approach.
Positive control oligonucleotides
Positive control DNA and RNA was designed based on sequence fragments of the N-gene (N1, N2, N3, China CDC), the RdRp-gene, the E-gene and the ORF-region derived from the genome of the Severe acute respiratory syndrome coronavirus 2 isolate Wuhan-Hu-1 (GenBank accession number NC_045512.2) as identified applying the previously described primers [2-5], prolonged by ≥10 bases at the 5'-end and the 3'-end, and linked by EcoR1 restriction sites (sequence details in Table 1). A pEX-A128 vector backbone was used to construct the positive control DNA plasmid (eurofins Genomics, Ebersberg, Germany). Positive control RNA of 1,005 bases length was transcribed with a T7 RNA polymerase from a linear template generated by PCR (AmpTec GmbH, Hamburg, Germany). Serial dilution was performed with eSwab collection & preservation of Aerobic, Anaerobic & Fastidious Bacteria (COPAN, Brescia, Italia) transport and storage medium spiked with respiratory sample material from patients; for the positive control RNA, it was also attempted with PCR grade water to prevent preterm RNA degradation due to RNAses included in the human patient samples.
Table 1.
Sequence insert fragments used for the positive control DNA plasmid and the positive control RNA as derived from the genome of the Severe acute respiratory syndrome coronavirus 2 isolate Wuhan-Hu-1 (GenBank accession number NC_045512.2)
| RdRp-gene fragment sequence |
| TTCTATAGATTAGCTAATGAGTGTGCTCAAGTATTGAGTG AAATGGTCATGTGTGGCGGTTCACTATATGTTAAACCAGG TGGAACCTCATCAGGAGATGCCACAACTGCTTATGCTAAT AGTGTTTTTAACATTTGTCAAGCTGTCACGGCC |
| E-Gene fragment sequence |
| ATGTACTCATTCGTTTCGGAAGAGACAGGTACGTTAATAG TTAATAGCGTACTTCTTTTTCTTGCTTTCGTGGTATTCTTG CTAGTTACACTAGCCATCCTTACTGCGCTTCGATTGTGTG CGTACTGCTGCAATATTGTTAACGTGAGTCTTGTA |
| N1/N2/N3 fragment sequence (CDC) |
| CAAAGACGGCATCATATGGGTTGCAACTGAGGGAGCCTT GAATACACCAAAAGATCACATTGGCACCCGCAATCCTGC TAACAATGCTGCAATCGTGCTACAACTTCCTCAAGGAACA ACATTGCCAAAAGGCTTCTACGCAGAAGGGAGCAGAGGC GGCAGTCAAGCCTCTTCTCGTTCCTCATCACGTAGTCGCA ACAGTTCAAG |
| N-gene fragment sequence (China-CDC) |
| TTCAACTCCAGGCAGCAGTAGGGGAACTTCTCCTGCTAG AATGGCTGGCAATGGCGGTGATGCTGCTCTTGCTTTGCT GCTGCTTGACAGATTGAACCAGCTTGAGAGCAAAATGTC TGGTAAAGGCCAACAACAACAA |
| ORF fragment sequence (China-CDC) |
| CCTACAACTTGTGCTAATGACCCTGTGGGTTTTACACTTA AAAACACAGTCTGTACCGTCTGCGGTATGTGGAAAGGTT ATGGCTGTAGTTGTGATCAACTCCGCGAACCCATGCTTCA GTCAGCTGATGCACAATCGTTTTTAAACGGGTTTGCGGTG |
Origin of samples
Patient samples were partly derived from the diagnostic routine of the Bundeswehr Hospital Hamburg. Further, patient samples with quantified viral copy numbers were provided by the University Hospital Regensburg, the institution from where external laboratory control assessment schemes for molecular diagnostic tests (“Ringversuche”) are provided in Germany. Finally, well characterized material from the external laboratory control assessment scheme “INSTAND EQA scheme (340) virus genome detection of SARS-CoV-2 - April 2020” by INSTAND e.V. (Düsseldorf, Germany) was applied.
Ethical considerations
As clarified previously (WF-011/19) by the Ethics Committee of the Medical Association of Hamburg, Germany, in line with German National Laws, use of anonymized residual sample materials for test comparison purposes in diagnostic laboratories is neither a “research project involving human beings” in line with § 9 chapter 2 of the “Hamburgisches Kammergesetz für Heilberufe” (Hamburg’s Association of Health Care Professions Act) nor a research project requiring ethical advice according to §15 chapter 1 of the “Berufsordnung für Hamburger Ärzte und Ärztinnen” (Professional Regulations for Physicians in Hamburg). Therefore, ethical clearance was not required for this assessment.
RESULTS AND DISCUSSION
Assessment with strongly positive, weakly positive and negative samples
With the Cepheid SARS-CoV-2 assay applied as gold standard, sensitivity of the automated ARIES assay with the strongly positive samples was 100% (15/15), specificity was 100% (30/30), resulting in excellent agreement according to Landis & Koch [19]. For the weakly positive samples, however, agreement of the ARIES assay with the Cepheid assay was reduced to 26.7% (4/15). Overall agreement about all tested samples between the two assays was 81.7% and Cohen’s Kappa was 0.63 indicating substantial agreement according to Landis & Koch [19]. Thereby, negative reactions of the ARIES-PCR targeting the ORF-sequence were more frequently observed, even in case of the strongly positive samples, compared with the ARIES N-gene-PCR. Details are provided in Tables 2–4.
Table 2.
Sensitivity assessment with strongly positive samples defined by cycle threshold values <34 in the gold standard assay
| Running sample number | Previous diagnostic results from the same patient | SARS CoV-2 target gene copy-number within the sample (if available) | Cepheid assay (gold standard) | ARIES assay (assessed test platform) | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Overall rating | Ct-value of the E-gene | Ct-value of the N-gene | Ct-value of the internal control (RNAse-P-gene) | Overall rating | Ct-value of the ORF sequence | Ct-value of the N-gene | Ct-value of the internal control (RNAse-P-gene) | |||
| #1 | positive | n.a. | positive | 30.4 | 33.1 | 27.9 | positive | negative | 36.3 | negative |
| #2 | positive | n.a. | positive | 30.1 | 32.3 | 27.9 | positive | negative | 37.5 | negative |
| #3 | n.a. | n.a. | positive | 23.3 | 25.5 | 27.7 | positive | 24.0 | 25.8 | negative |
| #4 | n.a. | n.a. | positive | 26.3 | 28.9 | 27.7 | positive | 26.2 | 29.2 | negative |
| #5 | n.a. | n.a. | positive | 30.2 | 32.7 | 28.1 | positive | negative | 31.3 | negative |
| #6 | n.a. | n.a. | positive | 23.9 | 26.3 | 28.0 | positive | negative | 28.8 | negative |
| #7 | n.a. | n.a. | positive | 27.2 | 29.3 | 28.1 | positive | 28.4 | 30.4 | negative |
| #8 | n.a. | n.a. | positive | 30.7 | 33.1 | 28.3 | positive | negative | 34.6 | negative |
| #9 | positive | = 5.8 × 104 | positive | 28.9 | 32.7 | 28.2 | positive | 28.3 | 36.9 | negative |
| #10 | positive | =2.6 × 103 | positive | 27.7 | 29.3 | 28.4 | positive | negative | 33.3 | negative |
| #11 | positive | = 1.0 × 104 | positive | 30.2 | 33.5 | 28.1 | positive | 29.4 | 37.2 | negative |
| #12 | positive | = 1.9 × 103 | positive | 30.2 | 32.9 | 28.1 | positive | negative | 38.3 | negative |
| #13 | positive | =9.2 × 104 | positive | 27.0 | 28.8 | 28.1 | positive | negative | 29.7 | negative |
| #14 | positive | = 7.2 × 104 | positive | 29.0 | 31.4 | 27.6 | positive | 30.5 | 33.3 | negative |
| #15 | positive | n.a. | positive | 26.4 | 28.8 | 28.7 | positive | negative | 30.3 | negative |
Ct = cycle threshold. n.a. = not applicable.
Table 3.
Sensitivity assessment with weakly positive samples defined by cycle threshold values >34 for at least one target gene in the gold standard assay
| Running sample number | Previous diagnostic results from the same patient | SARS-CoV-2 target gene copy-number within the sample (if available) | Cepheid assay (gold standard) | ARIES assay (assessed test platform) | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Overall rating | Ct-value of the E-gene | Ct-value of the N-gene | Ct-value of the internal control (RNAse-P-gene) | Overall rating | Ct-value of the ORF sequence | Ct-value of the N-gene | Ct-value of the internal control (RNAse-P-gene) | |||
| #1 | negative | n.a. | positive | 35.9 | 38.8 | 27.5 | negative | negative | negative | 28.4 |
| #2 | positive | n.a. | positive | 33.2 | 35.8 | 28.1 | negative | negative | negative | 27.7 |
| #3 | positive | n.a. | positive | 35.7 | 38.7 | 30.5 | negative | negative | negative | 23.5 |
| #4 | positive | n.a. | positive | 35.6 | 37.2 | 27.7 | negative | negative | negative | 25.9 |
| #5 | n.a. | n.a. | positive | 32.8 | 35.6 | 27.4 | negative | negative | negative | 24.2 |
| #6 | positive | n.a. | positive | 33.6 | 36.4 | 27.7 | negative | negative | negative | 26.0 |
| #7 | n.a. | n.a. | positive | 33.5 | 36.9 | 28.4 | positive | negative | 36.0 | negative |
| #8 | positive | n.a. | positive | 34.9 | 37.4 | 28.1 | negative | negative | negative | 26.5 |
| #9 | positive | =9.4 × 101 | positive | 37.9 | 39.8 | 28.3 | negative | negative | negative | 30.1 |
| #10 | positive | = 3.6 × 102 | positive | 34.4 | 36.2 | 28.1 | negative | negative | negative | 24.8 |
| #11 | positive | = 8.0 × 102 | positive | 37.0 | 40.5 | 28.0 | negative | negative | negative | 35.2 |
| #12 | positive | = 3.6 × 103 | positive | 31.3 | 34.2 | 28.1 | positive | 29.2 | negative | negative |
| #13 | positive | n.a. | positive | 36.0 | 36.6 | 28.0 | positive | negative | 37.4 | negative |
| #14 | positive | n.a. | positive | 34.8 | 36.1 | 27.9 | positive | negative | 37.7 | negative |
| #15 | positive | n.a. | positive | negative | 39.5 | 28.3 | negative | negative | negative | 27.6 |
Ct = cycle threshold, n.a. = not applicable.
Table 4.
Specificity assessment with samples negative in the Cepheid assay
| Running sample number | Previous diagnostic results from the same patient | SARS-CoV-2 target gene copy-number within the sample (if available) | Cepheid assay (gold standard) | ARIES assay (assessed test platform) | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Overall rating | Ct-value of the E-gene | Ct-value of the N-gene | Ct-value of the internal control (RNAse-P-gene) | Overall rating | Ct-value of the ORF sequence | Ct-value of the N-gene | Ct-value of the internal control (RNAse-P-gene) | |||
| #1 | negative | n.a. | negative | negative | negative | 27.5 | negative | negative | negative | 27.3 |
| #2 | negative | n.a. | negative | negative | negative | 27.3 | negative | negative | negative | 27.1 |
| #3 | negative | n.a. | negative | negative | negative | 27.7 | negative | negative | negative | 25.6 |
| #4 | negative | n.a. | negative | negative | negative | 27.8 | negative | negative | negative | 25.0 |
| #5 | negative | n.a. | negative | negative | negative | 27.8 | negative | negative | negative | 25.2 |
| #6 | negative | n.a. | negative | negative | negative | 28.9 | negative | negative | negative | 27.2 |
| #7 | negative | n.a. | negative | negative | negative | 28.2 | negative | negative | negative | 28.4 |
| #8 | positive | n.a. | negative | negative | negative | 28.3 | negative | negative | negative | 26.6 |
| #9 | negative | n.a. | negative | negative | negative | 28.3 | negative | negative | negative | 28.6 |
| #10 | negative | n.a. | negative | negative | negative | 27.3 | negative | negative | negative | 25.5 |
| #11 | positive | n.a. | negative | negative | negative | 28.4 | negative | negative | negative | 29.1 |
| #12 | negative | n.a. | negative | negative | negative | 27.6 | negative | negative | negative | 28.7 |
| #13 | positive | n.a. | negative | negative | negative | 27.5 | negative | negative | negative | 25.1 |
| #14 | positive | n.a. | negative | negative | negative | 28.1 | negative | negative | negative | 26.6 |
| #15 | negative | n.a. | negative | negative | negative | 27.7 | negative | negative | negative | 27.6 |
| #16 | negative | n.a. | negative | negative | negative | 28.1 | negative | negative | negative | 23.1 |
| #17 | positive | n.a. | negative | negative | negative | 28.3 | negative | negative | negative | 24.8 |
| #18 | positive | n.a. | negative | negative | negative | 28.2 | negative | negative | negative | 27.7 |
| #19 | positive | n.a. | negative | negative | negative | 28.0 | negative | negative | negative | 25.9 |
| #20 | positive | n.a. | negative | negative | negative | 28.2 | negative | negative | negative | 31.6 |
| #21 | positive | n.a. | negative | negative | negative | 27.9 | negative | negative | negative | 32.4 |
| #22 | negative | n.a. | negative | negative | negative | 27.5 | negative | negative | negative | 25.2 |
| #23 | negative | n.a. | negative | negative | negative | 28.1 | negative | negative | negative | 29.1 |
| #24 | negative | n.a. | negative | negative | negative | 28.3 | negative | negative | negative | 25.7 |
| #25 | negative | n.a. | negative | negative | negative | 28.1 | negative | negative | negative | 27.8 |
| #26 | negative | n.a. | negative | negative | negative | 28.1 | negative | negative | negative | 27.8 |
| #27 | negative | n.a. | negative | negative | negative | 28.1 | negative | negative | negative | 24.6 |
| #28 | positive | n.a. | negative | negative | negative | 30.5 | negative | negative | negative | 31.0 |
| #29 | positive | n.a. | negative | negative | negative | 29.9 | negative | negative | negative | 26.1 |
| #30 | negative | n.a. | negative | negative | negative | 25.0 | negative | negative | negative | 28.7 |
Ct = cycle threshold, n.a. = not applicable.
Inter- and intra-assay variation
For the assessment of inter- and intra-assay variation, excellent concordance was observed for the strongly positive samples, while slightly more variation was seen for the weakly positive samples in the high ct-value range (Tables 5 and 6).
Table 5.
Intra-assay precision testing with a strongly positive sample (Cepheid E-Gene Ct-value 23.7; Cepheid N-Gene Ct-value 26.3), a weakly positive sample (Cepheid E-Gene Ct-value 31.0; Cepheid N-Gene Ct-value 34.4), and a negative sample
| Running sample number | ARIES assay | |||
|---|---|---|---|---|
| Overall rating | Ct-value of the ORF-sequence | Ct-value of the N-gene | Ct-value of the internal control (RNAse-P-gene) | |
| Strongly positive sample | ||||
| #1 | positive | 25.5 | 27.5 | 0.0 |
| #2 | positive | 25.5 | 27.8 | 0.0 |
| #3 | positive | 25.5 | 27.1 | 0.0 |
| Weakly positive sample | ||||
| #1 | positive | 0.0 | 34.7 | 0.0 |
| #2 | positive | 0.0 | 36.7 | 0.0 |
| #3 | positive | 0.0 | 33.4 | 0.0 |
| Negative sample | ||||
| #1 | negative | 0.0 | 0.0 | 23.6 |
| #2 | negative | 0.0 | 0.0 | 23.6 |
| #3 | negative | 0.0 | 0.0 | 24.3 |
Ct = cycle threshold.
Table 6.
Inter-assay precision testing with a strongly positive sample (Cepheid E-Gene Ct-value 23.4; Cepheid N-Gene Ct-value 25.8), a weakly positive sample (Cepheid E-Gene Ct-value 35.3; Cepheid N-Gene Ct-value 38.4), and a negative sample
| Running sample number | ARIES assay | |||
|---|---|---|---|---|
| Overall rating | Ct-value of the ORF-sequence | Ct-value of the N-gene | Ct-value of the internal control (RNAse-P-gene) | |
| Strongly positive sample | ||||
| #1 | positive | 23.3 | 25.4 | 0.0 |
| #2 | positive | 23.6 | 26.0 | 0.0 |
| #3 | positive | 22.9 | 25.9 | 0.0 |
| Weakly positive sample | ||||
| #1 | negative | 0.0 | 0.0 | 31.6 |
| #2 | negative | 0.0 | 0.0 | 31.3 |
| #3 | negative | 0.0 | 0.0 | 27.5 |
| Negative sample | ||||
| #1 | negative | 0.0 | 0.0 | 25.5 |
| #2 | negative | 0.0 | 0.0 | 25.8 |
| #3 | negative | 0.0 | 0.0 | 25.3 |
Ct = cycle threshold.
Limit of detection assessment with well characterized samples, DNA plasmids and positive control RNA
With plasmid DNA, the lod of the ARIES system with a positive result indicated by at least one positive PCR target as defined by the manufacturer was in the 103 copies/reaction-range. Within the 102 copies/reaction-range, target DNA in 50% of the repeated runs would have gone undetected. Focusing on the ORF-sequence target, 25% of the runs were positive in the 103 copies/reaction-range, 75% in the 104 copies/reaction-range, and 100% in the 105 copies/reaction-range. Focusing on the N-gene target, 50% of the runs were positive in the 102 copies/reaction range and 100% in the 103 copies/reaction range (Table 7).
Table 7.
Assessment of the limits-of-detection (lod) applying DNA plasmids as positive control in quadruplicate
| Positive control DNA copy number | Ct-values of the ARIES ORF-sequence | Ct-values of the ARIES N-gene | Ct-values of the ARIES the internal control (RNAse-P-gene) | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Run 1 | Run 2 | Run 3 | Run 4 | Run 1 | Run 2 | Run 3 | Run 4 | Run 1 | Run 2 | Run 3 | Run 4 | |
| 1.4 × 106 | 25.3 | 21.7 | 21.7 | 21.9 | 27.2 | 23.7 | 23.9 | 24.1 | negative | negative | negative | negative |
| 1.4 × 105 | 28.6 | 24.1 | 25.5 | 24.2 | 30.7 | 26.6 | 27.7 | 27.1 | negative | negative | negative | negative |
| 1.4 × 104 | 29.6 | 25.0 | 27.3 | negative | 31.5 | 30.1 | 30.4 | 32.5 | negative | negative | negative | negative |
| 1.4 × 103 | 32.3 | negative | negative | negative | 34.4 | 36.2 | 36.5 | 37.2 | negative | negative | negative | negative |
| 1.4 × 102 | negative | negative | negative | negative | negative | negative | 38.3 | 38.8 | 39.1 | 25.3 | negative | negative |
| 1.4 × 101 | negative | negative | negative | negative | negative | negative | negative | negative | negative | 25.7 | 26.4 | 26.8 |
| 1.4 | negative | negative | negative | negative | negative | negative | negative | negative | negative | 27.0 | 27.0 | 26.6 |
| <1 | negative | negative | negative | negative | negative | negative | negative | negative | negative | 27.1 | 27.1 | 26.9 |
| <1 | negative | negative | negative | negative | negative | negative | negative | negative | negative | 27.1 | 26.1 | 27.1 |
Ct = cycle threshold.
With free single-stranded RNA, the limit of detection of the ARIES system was in the 104 copies/reaction-range. Within the 103 copies/reaction-range, only 50% of the runs would have led to positive results. Focusing on the ORF-sequence target, 25% of the runs were positive in the 103 copies/reaction-range, and 100% in the 104 copies/reaction-range. The reason for the single failed amplification of the ORF-sequence in the 105 copies/reaction range remains unclear. Focusing on the N-gene target, 50% of the runs were positive in the 103 copies/reaction range and 100% in the 104 copies/reaction range (Table 8). Due to poor stability of free RNA molecules, their integration in the evaluation has been a particular challenge to the diagnostic assay assessed.
Table 8.
Assessment of the limits-of-detection (lod) applying free RNA as positive control in quadruplicate
| Positive control RNA copy number | Ct-values of the ARIES ORF-sequence | Ct-values of the ARIES N-gene | Ct-values of the ARIES the internal control (RNAse-P-gene) | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Run 1 | Run 2 | Run 3 | Run 4 | Run 1 | Run 2 | Run 3 | Run 4 | Run 1 | Run 2 | Run 3 | Run 4 | |
| 3.8 × 107 | 20.9 | 21.2 | 20.4 | 21.7 | 22.3 | 22.7 | 22.2 | 23.1 | negative | negative | negative | negative |
| 3.8 × 106 | 24.5 | 24.7 | 25.2 | 25.7 | 26.1 | 26.4 | 26.8 | 27.3 | negative | negative | negative | negative |
| 3.8 × 105 | 28.0 | 27.3 | negative | 27.8 | 29.7 | 29.1 | 30.8 | 29.5 | negative | negative | negative | negative |
| 3.8 × 104 | 31.2 | 30.7 | 28.9 | 32.6 | 32.7 | 32.7 | 30.5 | 34.2 | negative | negative | negative | negative |
| 3.8 × 103 | 34.1 | negative | negative | negative | 35.7 | 37.4 | negative | negative | negative | negative | 36.4 | negative |
| 3.8 × 102 | negative | negative | negative | negative | negative | negative | negative | negative | 38.8 | negative | negative | negative |
| 3.8 × 101 | negative | negative | negative | negative | negative | negative | negative | negative | negative | negative | negative | negative |
| 3.8 | negative | negative | negative | negative | negative | negative | negative | negative | 36.5 | negative | negative | negative |
| <1 | negative | negative | negative | negative | negative | negative | negative | negative | negative | 37.0 | negative | negative |
Ct = cycle threshold.
With quantified virus particles in patient samples, the lod of the ARIES assay was in the 103 copies/reaction-range. While all samples containing 103 and more virus copies were detected positive by the ARIES assay, virus RNA in the 102 copies/reaction-range and less went undetected.
DISCUSSION
In summary, the ARIES SARS-CoV-2 Assay did not show any non-specific reactions in our hands in the course of the performed validation procedure. Focusing on diagnostic sensitivity, however, target sequences in a considerable proportion of weakly positive samples which were still detected positive with the automated assay by Cepheid went undetected. With a limit of detection >103 virus copies, stages of the infection with lower viral copy number may be missed. Based on previous estimates [9, 20], respective high copy numbers are only detectable for few days of the infection. In contrast, highly sensitive PCR facilitates the detection of prolonged shedding of viral RNA sequences irrespective of their infectious potential [21–24].
Anyway, clinical or hygiene-related decisions based on positive results of the ARIES SARS-CoV-2 Assay should consider its limits of detection which are considerably higher than the ones of the automated Cepheid approach [11–16]. This is of particular importance in situations when maximum sensitivity is desired, both for clinical decision making or surveillance purposes.
The assessment has a number of limitations. First of all, there was only a limited number of positive samples available for evaluation purposes, so samples had to be taken from different sources. Secondly, the definition of the Cepheid assay as the gold standard is a methodical limitation by itself, because this decision does not consider potential failure of the gold standard itself. For example, it cannot be excluded that there might have been individual SARS-CoV-2-containing samples being positively detected by the ARIES system but missed by the Cepheid approach in broader evaluations including higher sample numbers. However, in spite of such theoretical assumptions, the data provide a preliminary estimation of the diagnostic performance of the validated diagnostic test in line with validation standards as suggested previously [17, 18].
CONCLUSIONS
The automated cartridge-based ARIES SARS-CoV-2 Assay specifically detected SARS-CoV-2 sequences in our validation approach. Compared with the automated Cepheid approach, however, the assay was considerably less sensitive for cases with low target RNA loads, a fact that should be considered in case of decisions for its diagnostic use.
ACKNOWLEDGMENTS
Simone Priesnitz is gratefully acknowledged for excellent technical assistance. The colleagues from the University of Regensburg are acknowledged for providing the precharacterized patient samples for the evaluation.
Footnotes
Funding sources: No financial support was received for this study.
Authors’ contribution: KT, LK, and HF assessed the data and prepared the manuscript. All authors have jointly planned the study, optimized and reviewed the manuscript.
Conflict of interest: Nothing to declare.
REFERENCES
- 1.Zhou P, Yang XL, Wang XG, Hu B, Zhang L, Zhang W, et al. A pneumonia outbreak associated with a new coronavirus of probable bat origin. Nature 2020;579:270–3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Corman VM, Landt O, Kaiser M, Molenkamp R, Meijer A, Chu DK, et al. Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR. Euro Surveill 2020;25:2000045. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.CDC 2019-Novel Coronavirus (2019-nCoV) Real-Time RT-PCR Diagnostic Panel. Cited 2020 June. Available from: ivdc.chinacdc.cn/kyjz/202001/t20200121_211337.html. [DOI] [PMC free article] [PubMed]
- 4.Real-time RT-PCR panel for detection 2019-novel coronavirus. Centers for Disease Control and Prevention (Internet). Cited 2020 June. Available from: https://www.fda.gov/media/134922/download.
- 5.2019-novel coronavirus (2019-nCoV) real-time rRT-PCR panel primers and probes. Centers for Disease Control and Prevention (Internet). Cited 2020 June. Available from: https://www.cdc.gov/coronavirus/2019-ncov/lab/rt-pcr-panel-primer-probes.html.
- 6.Reischl U, Drosten C, Geiβdörfer W, Göbel U, Hoffmann KS, Mauch H, et al. MiQ 01: Nukleinsäure-Amplifikationstechniken. Qualitätsstandard in der mikrobiologischen Diagnostik. 3. Auflag. Urban & Fischer Verlag/Elsevier GmbH 2011, pp. 1-84. [Google Scholar]
- 7.Harrington A, Cox B, Snowdon J, Bakst J, Ley E, Grajales P, et al. Comparison of Abbott ID Now and Abbott m2000 methods for the detection of SARS-CoV-2 from nasopharyngeal and nasal swabs from symptomatic patients. J Clin Microbiol 2020;58:e00798–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Rhoads DD, Cherian SS, Roman K, Stempak LM, Schmotzer CL, Sadri N. Comparison of Abbott ID Now, Diasorin Simplexa, and CDC FDA Emergency Use Authorization methods for the detection of SARS-CoV-2 from nasopharyngeal and nasal swabs from individuals diagnosed with COVID-19. J Clin Microbiol 2020;58: e00760–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Zhen W, Smith E, Manji R, Schron D, Berry GJ. Clinical evaluation of three sample-to-answer platforms for the detection of SARS-CoV-2. J Clin Microbiol 2020;58:e00783–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Basu A, Zinger T, Inglima K, Woo KM, Atie O, Yurasits L, et al. Performance of Abbott ID NOW COVID-19 rapid nucleic acid amplification test using nasopharyngeal swabs transported in viral media and dry nasal swabs, in a New York City academic institution. J Clin Microbiol 2020;58:e01136–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Moran A, Beavis KG, Matushek SM, Ciaglia C, Francois N, Tesic V, et al. Detection of SARS-CoV-2 by use of the Cepheid Xpert Xpress SARS-CoV-2 and Roche cobas SARS-CoV-2 Assays. J Clin Microbiol 2020;58:e00772–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Broder K, Babiker A, Myers C, White T, Jones H, Cardella J, et al. Test agreement between Roche Cobas 6800 and Cepheid GeneXpert Xpress SARS-CoV-2 Assays at high cycle threshold ranges. J Clin Microbiol 2020;58:e01187–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Loeffelholz MJ, Alland D, Butler-Wu SM, Pandey U, Perno CF, Nava A, et al. Multicenter evaluation of the Cepheid Xpert Xpress SARS-CoV-2 test. J Clin Microbiol 2020;58:e00926-20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Wolters F, van de Bovenkamp J, van den Bosch B, van den Brink S, Broeders M, Chung NH, et al. Multi-center evaluation of cepheid xpert® xpress SARS-CoV-2 point-of-care test during the SARS-CoV-2 pandemic. J Clin Virol 2020;128:104426. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Lowe CF, Matic N, Ritchie G, Lawson T, Stefanovic A, Champagne S, et al. Detection of low levels of SARS-CoV-2 RNA from nasopharyngeal swabs using three commercial molecular assays. J Clin Virol 2020;128:104387. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Smithgall MC, Scherberkova I, Whittier S, Green DA. Comparison of Cepheid Xpert Xpress and Abbott ID now to Roche cobas for the rapid detection of SARS-CoV-2. J Clin Virol 2020; 128:104428. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Rabenau HF, Kessler HH, Kortenbusch M, Steinhorst A, Raggam RB, Berger A. Verification and validation of diagnostic laboratory tests in clinical virology. J Clin Virol 2007;40:93–8. [DOI] [PubMed] [Google Scholar]
- 18.Rabenau HF, Kortenbuschen M, Berger A, Steinhorst A. Validierung von Untersuchungsverfahren im Bereich der Virusdiagnostik. J Lab Med 2007;31:41-7. [Google Scholar]
- 19.Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159–74. [PubMed] [Google Scholar]
- 20.Park A, Iwasaki A. Type I and Type III interferons – induction, signaling, evasion, and application to Combat COVID-19. Cell Host Microbe 2020;27:870–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Li N, Wang X, Lv T. Prolonged SARS-CoV-2 RNA shedding: Not a rare phenomenon. J Med Virol 2020; [Epub ahead of print]: 10.1002/jmv.25952. [DOI] [PMC free article] [PubMed]
- 22.Xu K, Chen Y, Yuan J, Yi P, Ding C, Wu W, et al. Factors associated with prolonged viral RNA shedding in patients with coronavirus disease 2019 (COVID-19). Clin Infect Dis 2020;71:799–806. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Wang K, Zhang X, Sun J, Ye J, Wang F, Hua J, et al. Differences of SARS-CoV-2 shedding duration in sputum and nasopharyngeal swab specimens among adult inpatients with COVID-19. Chest 2020; [Epub ahead of print]: 10.1016/j.chest.2020.06.015. [DOI] [PMC free article] [PubMed]
- 24.Qi L, Yang Y, Jiang D, Tu C, Wan L, Chen X, et al. Factors associated with duration of viral shedding in adults with COVID-19 outside of Wuhan, China: A retrospective cohort study. Int J Infect Dis 2020;96:531–7. [DOI] [PMC free article] [PubMed] [Google Scholar]