Direct Comparison of SARS-CoV-2 Analytical Limits of Detection across Seven Molecular Assays

Analytical sensitivity for SARS-CoV-2 detection is a key performance metric for the evaluation of viral detection assays. We determined analytical limits of detection for seven SARS-CoV-2 assays using serial dilutions of pooled patient material quantified with droplet digital PCR. Limits of detection ranged from ≤10 to 74 copies/ml for commercial high-throughput laboratory analyzers (Roche Cobas, Abbott m2000, and Hologic Panther Fusion) and 167 to 511 copies/ml for sample-to-answer (DiaSorin Simplexa, GenMark ePlex) and point-of-care instruments (Abbott ID NOW).

ABSTRACT

Analytical sensitivity for SARS-CoV-2 detection is a key performance metric for the evaluation of viral detection assays. We determined analytical limits of detection for seven SARS-CoV-2 assays using serial dilutions of pooled patient material quantified with droplet digital PCR. Limits of detection ranged from ≤10 to 74 copies/ml for commercial high-throughput laboratory analyzers (Roche Cobas, Abbott m2000, and Hologic Panther Fusion) and 167 to 511 copies/ml for sample-to-answer (DiaSorin Simplexa, GenMark ePlex) and point-of-care instruments (Abbott ID NOW). The CDC assay yielded limits of detection ranging from 85 to 499 copies/ml, depending on the extraction method and thermocycler used. These results can help to inform the assay choice for testing approaches to manage the current COVID-19 outbreak.

INTRODUCTION

Molecular testing for SARS-CoV-2 viral shedding is a critical part of the response to the COVID-19 outbreak. These tests are applied for symptomatic patients with expected high levels of virus present in respiratory samples (1) but are increasingly being used for screening of asymptomatic individuals for whom the kinetics of viral shedding are less well documented (2). An accurate understanding of test performance is important to evaluate the utility of molecular viral detection in a variety of clinical settings, and the analytical limit of detection (LOD) is a key characteristic for method comparison. Due to variability in the reference standard material and methods of quantitation used during the evaluation of limits of detection, it has been challenging to compare performance among test methods.

This study was performed to directly compare analytical limits of detection using a pool of positive patient material quantified with droplet digital PCR (ddPCR). The use of ddPCR allows for a standard independent assessment of viral load based on the number and volume of droplets measured in a droplet emulsion PCR assay, which has been shown to be a more sensitive approach for SARS-CoV-2 detection in low-viral load samples (3).

MATERIALS AND METHODS

Ten residual nasopharyngeal (NP) swab or combined NP/oropharyngeal (OP) swab raw patient samples containing high-titer SARS-CoV-2 virus in universal transport media (UTM) were combined to form patient pooled material (PPM). Nine of these were initially positive using the CDC assay with LightCycler 480 II amplification, and one was positive using Hologic Panther Fusion assay. The PPM was extracted over 3 dilutions in triplicate using EZ1 Virus 2.0 kit (Qiagen) with 400 μl sample input and 90 μl elution, and each extract was quantified using novel coronavirus (2019 nCoV) digital PCR detection kit (ApexBio Technology, Houston TX) per the manufacturer instructions. The viral concentration in stock PPM was calculated based on the average concentration over the 3 dilutions, assuming 100% extraction efficiency, and was stored at −80°C in individual aliquots to ensure only one freeze-thaw cycle. For testing, stock PPM was diluted with UTM or viral transport media (VTM) in serial 1:10 dilutions to 3.5-log copies/ml (3,160 copies/ml) and then in serial 0.5-log (1:3.16) dilutions to calculated concentrations between 1.0- and 3.5-log copies/ml (10 to 3,160 copies/ml) to span the expected range of concentrations needed to determine LOD. Sufficient volume was prepared for individual testing of at least 20 replicate samples at each concentration, and samples were held refrigerated until they were tested within 72 h. Preliminary LOD was determined with 5 replicates over multiple concentrations, and at least 15 additional replicates were performed at concentrations spanning the level with 95% detection. Experimental LOD was determined as the lowest concentration tested with 95% detection, and Probit LOD was determined using Probit analysis (4).

Each commercial assay was performed according to the manufacturer’s instructions as follows: Abbott m2000, RealTime SARS-CoV-2 assay (emergency use authorization [EUA]), high-throughput dual-target assay for RdRp and N genes (Des Plaines, IL). Roche Cobas, SARS-CoV-2 test (EUA), high-throughput dual-target assay for Orf1ab and E genes (Indianapolis, IN); Hologic Panther Fusion, SARS-CoV-2 assay (EUA), high-throughput dual-target assay for ORF1ab region 1 and ORF1ab region 2 (Marlborough, MA); DiaSorin Simplexa, COVID-19 direct kit (EUA), sample-to-answer dual-target assay for S gene and ORF1ab gene (Cypress, CA); GenMark ePlex: SARS-CoV-2 test (EUA), sample-to-answer dual-target assay for two different regions of the N gene (Fremont, CA); and Abbott ID NOW, COVID-19 assay (EUA), sample-to-answer, point-of-care single-target assay for the RdRp gene (Scarborough MA). To mimic the introduction of viral material from respiratory swab samples on the ID NOW platform, quantified samples in UTM were added 1:100 to achieve the stated final concentration in ID NOW elution buffer and immediately tested. This allows for direct comparison between the viral concentration in UTM/VTM and the viral concentration in elution buffer after addition of the swab material.

The CDC assay (EUA) is a dual target assay for N1 and N2 genes and requires separate extractions; these were performed using the EZ1 Virus 2.0 kit (Qiagen, Germantown MD) with 400 μl sample input and 90 μl elution volume or the MagNA Pure MP24 kit (Roche) with 200 μl sample input and 100 μl elution volume. CDC assay reagents were obtained from IDT Technologies, and the CDC assay was performed on the ABI 7500 Fast thermocycler (Applied Biosystems, Foster City CA) or the LightCycler 480 II thermocycler (Roche Diagnostics, Indianapolis IN). Results with one of two viral gene targets detected (indeterminate interpretation) were considered positive on the CDC assay.

RESULTS

Results of replicate testing of dilutions of PPM quantified using ddPCR are shown in Table 1. For commonly available SARS-CoV-2 methods, limits of detection determined by Probit analysis ranged from ≤ 10 copies/ml for the Roche Cobas assay to 511 copies/ml for Abbott ID NOW, which is a point-of-care (POC) assay yielding results in 13 min. There was a trend with more rapid sample-to-answer assays yielding slightly higher LOD (167 to 511 copies/ml) compared to high-throughput laboratory analyzers. The CDC assay showed LOD between 85 and 499 copies/ml, depending on the extraction instrument and thermocycler used. The effective sample volume tested for each assay was calculated based on the amount of sample processed, elution volume, and the amount of eluate added to the amplification reaction (Table 2). The Roche Cobas instrument processes 400 μl per sample, with 50 μl elution volume and all 50 μl of that used for PCR, for an effective tested sample volume of 400 μl. Abbott m2000 processes 500 μl sample, with 80 μl elution and 40 μl used for PCR, for an effective tested sample volume of 250 μl. Hologic Panther Fusion processes 360 μl of sample after 500 μl is diluted into 710 μl lysis buffer, with 50 μl elution and 5 μl used for PCR, for an effective tested sample volume of 25.4 μl. For the CDC assay, 5 μl of elution was added to PCR after EZ1 extraction with 400 μl sample input and 90 μl elution volume (effective tested volume of 22.2 μl) or MP24 extraction with 200 μl sample input and 100 μl elution volume (effective tested volume of 10 μl).

TABLE 1.

Analytical limits of detection for SARS-CoV-2

Assay	No. of SARS-CoV-2 detected/no. tested at a viral concn (cp/ml) of:						Exp. LODa	Probit LOD
	3,160	1,000	316	100	32	10
Abbott m2000	ND	ND	20/20	19/20	20/20	16/20	32	53
Roche Cobas	ND	ND	20/20	20/20	20/20	20/20	≤10	≤10
Panther Fusion	5/5	5/5	5/5	20/20	15/20	1/20	100	74
DiaSorin Simplexa	5/5	5/5	20/20	19/20	14/20	ND	100	167
GenMark ePlex	5/5	5/5	5/5	19/20	8/20	ND	100	190
Abbott ID NOW	5/5	5/5	24/25	14/25	9/20	0/5	316	511
CDC ABI 7500 (MP24)	5/5	5/5	20/20	15/20	1/5	ND	316	237
CDC LC 480 (MP24)	5/5	5/5	19/20	14/20	1/5	ND	316	499
CDC ABI 7500 (EZ1)	5/5	5/5	20/20	20/20	13/20	ND	100	85
CDC LC 480 (EZ1)	5/5	5/5	20/20	19/20	10/20	ND	100	185

^aExp. LOD, experimental limit of detection; Probit LOD, Probit limit of detection; ND, not done; ABI 7500, Applied Biosystems 7500 Fast real-time PCR system; LC 480, Roche LightCycler 480 II real-time PCR system; MP24, Roche MagaPure 24 extractor; EZ1, Qiagen EZ1 extractor.

TABLE 2.

Sample volumes tested for SARS-CoV-2

Assay	Minimum input vol (μl)	Sample processed (μl)	Elution vol (μl)	Eluate added to reaction (μl)	Effective sample vol(s) tested (μl)
Abbott m2000	760	500	80	40	250
Roche Cobas	600	400	50	50	400
Panther Fusion	500a	360	50	5	25.4
DiaSorin Simplexa	—b	—	—	—	50
GenMark ePlex	—	—	—	—	200
Abbott ID NOW	—	—	—	—	100 (target), 100 (internal control)
CDC ABI 7500 (MP24)	350	200	100	5	10
CDC LC 480 (MP24)	350	200	100	5	10
CDC ABI 7500 (EZ1)	400	400	90	5	22.2
CDC LC 480 (EZ1)	400	400	90	5	22.2

^aA 500-μl sample + 710-μl lysis buffer = 1,210 μl input volume.

^bSample-to-answer tests.

DISCUSSION

Accurate assessment of analytical sensitivity for SARS-CoV-2 detection is needed to assess the comparative ability of various assays to diagnose infection and to understand the relative risk for infection in patients with negative test results. Previous comparisons have shown varied results, with laboratory-based methods generally performing better than sample-to-answer and point-of-care methods (5–8). The standard material used and methods of calibration differ for most available studies, making it difficult to assess the true limits of detection in patient samples. Here, we used the standard independent calibration method of ddPCR to quantify a pool of positive patient material and assess limits of detection across seven different assays. The inclusion of multiple positive patient samples in the pool reduces but does not eliminate the possibility of viral mutations present that might adversely affect specific assay detection. Our initial quantification method used extraction prior to ddPCR quantification, assuming 100% extraction efficiency to calculate the viral concentration in the original pooled patient sample. If the extraction efficiency were lower than 100%, the actual viral concentration would be somewhat higher than the calculated value used in this study.

Overall assay sensitivity appears to be primarily related to the effective sample volume tested in the amplification reaction with some influence from assay design and detection instrumentation. The effective volume tested is dependent on the sample input volume and extraction and reaction parameters and can be directly compared for assays having separate extraction and amplification reactions. This relationship is less consistent for sample-to-answer platforms, where the nucleic acid purification, processing, and chemistry used have a substantial effect on assay LOD.

The CDC SARS-CoV-2 assay is a commonly used method, but there are several variations possible, including the extraction method and real-time thermocycler used. We show that the most sensitive detection is achieved using EZ1 extracted material with the ABI 7500 Fast thermocycler, with slight decreases in sensitivity using the Roche MP24 extractor or the Roche LightCycler 480 II thermocycler. Other modifications may have somewhat varying performance.

While the high-throughput laboratory-based instruments (Roche Cobas, Abbott m2000, and Hologic Panther Fusion) achieved the best analytical sensitivity, other sample-to-answer methods (DiaSorin Simplexa and GenMark ePlex) were only approximately 2- to 4-fold less sensitive, indicating good performance with more rapid results. In this study, the Abbott ID NOW POC instrument achieved a sensitivity of approximately 511 copies/ml, 1 log lower than high-throughput laboratory instruments. This can be compared to other studies showing 20,000 copies/ml using Exact Diagnostics synthetic RNA quantified control with a sample dilution protocol (9) and 125 genome equivalents/ml based on the manufacturer package insert (10). Here, we mimicked the use of a dry swab put directly into elution buffer, eliminating the dilution from using VTM in comparison studies (a method that is no longer recommended in the ID NOW package insert).

Analytical sensitivity for virus detection is a significant factor in the choice of assay to use but is not the only consideration. This study did not evaluate the impact of specimen matrix on the ability to detect virus or compatibility with different media types, which may be particularly relevant for methods that bypass the extraction step and permit analysis directly from original swab samples (e.g., Abbott ID NOW system). Our findings are not generalizable to other sample types, including lower respiratory sources.

Using pooled patient material quantified with ddPCR, it is reassuring that measured limits of detection were consistent in the tens of copies per milliliter for high-throughput laboratory instruments and in the hundreds of copies per milliliter for sample-to-answer and POC instruments. Less sensitive methods will fail to detect virus in some cases, primarily during the long tail of viral shedding seen in some patients. However, this may be of lower clinical significance since many of these patients would likely have developed detectable antibody responses by this time, and samples with quantitative PCR (qPCR) cycle threshold (C_T) values > 33 tend not to be recoverable by viral culture, indicating a potential lack of infectivity (11). Further studies are needed to clarify the relationship of viral detection to transmissibility, which will help guide the choice of testing strategies.