How many are we missing with ID NOW COVID-19 assay using direct nasopharyngeal swabs? Findings from a mid-sized academic hospital clinical microbiology laboratory

Abstract

Here, we retrospectively analyzed the comparative results of 182 paired dry nasopharyngeal swabs tested by Abbott ID NOW and nasopharyngeal swabs in viral transport medium by real-time RT-PCR methods. While the overall agreement was 96.2%, we found that of 15 samples that were tested positive with RT-PCR methods, 7 were missed by ID NOW, resulting in a false-negative rate of 47%.

Introduction

Since the emergence of coronavirus disease 2019 (COVID-19), caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), diagnostic molecular assays have been developed by clinical and public health laboratories as well as commercial companies. Production of commercial SARS-CoV-2 nucleic acid detection assays, approved for Emergency Use Authorization (EUA) by the Food and Drug Administration (FDA), has slowly relieved the testing burden faced in many clinical laboratories. Recent publications have shown the comparative performance of several platforms using nasopharyngeal swab (NPS) samples collected in universal viral transport medium (VTM) (Craney et al., 2020; Lieberman et al., 2020; Moran et al., 2020; Rhoads et al., 2020; Zhen et al., 2020a, Zhen et al., 2020b). Harrington et al compared the performance of paired nasal swabs on a simple sample-to-answer platform ID NOW with NPS in VTM (NPS-VTM) tested by Abbott RealTime SARS-CoV-2 assay on the Abbott m2000sp platform (Harrington et al., 2020). However, the characteristics of using direct NPS vs. NPS-VTM has not been demonstrated. Additionally, concerns about the potentially inferior performance of ID NOW have been raised in the scientific community even after the FDA recall of NPS-VTM usage in mid-April. Therefore, the overall goal of this study was to examine the false-negative rate detected by ID NOW by comparing the results of direct dry NPS (dNPS) on ID NOW, which utilizes an isothermal nucleic acid amplification technology, with NPS-VTM tested by real-time RT-PCR methods. The real-time RT-PCR platforms described here included 3 commercial assays (Abbott RealTime SARS-CoV-2 (Abbott Park, IL, USA), Panther Fusion® SARS-COV-2 (San Diego, CA, USA), and Cepheid Xpert® Xpress SARS-CoV-2 (Sunnyvale, CA, USA)) and a laboratory-developed test (LDT), that has been validated and submitted for FDA EUA approval.

Materials and methods

We performed a retrospective data review of SARS-CoV-2 tests from 182 paired dNPS on ID NOW and NPS-VTM on real-time RT-PCR platforms, which were collected between April and May 2020 (4 weeks data) in in-patient and the emergency department settings from symptomatic patients on the same day. dNPS were transported in plain untreated sterile urine collection tubes to the ID NOW testing area within 2 hours of collection. NPS-VTMs tested on m2000sp were heat-inactivated for 30 min at 56 °C followed by swab removal from VTM tubes whereas those tested on Fusion were directly aliquoted into Aptima® Lysis tubes without heat inactivation. For samples tested on LDT, swabs were removed prior to testing. For Xpert Xpress, NPS-VTM (300 μL) was transferred to an assay cartridge per manufacturer's instructions. Samples were loaded onto the respective instrument platforms per manufacturers' instructions. RT-PCR testing platforms were located in three divisions at our institute - Microbiology Division performs Fusion and Xpert Xpress, Molecular Diagnostic Division uses m2000sp, and Galveston National Laboratory utilizes LDT platform. Our LDT system utilizes MagNA Pure 96 system for nucleic acid extraction and BioRad CFX96 for RT-PCR and detects SARS-CoV-2 specific E and ORF8 targets.

Statistical analysis was performed using Medcalc stats (https://www.medcalc.org/calc/diagnostic_test.php), and Cohen's kappa coefficient (κ) was calculated using the Graphpad Prism QuickCalcs website (https://www.graphpad.com/quickcalcs/).

Results

By using a collective data set from all RT-PCR platforms compared against ID NOW, the overall agreement was 96.2% (95% CI: 92.2–98.4%). The positive percent agreement (PPA) was 53.3% (95% CI: 26.6–78.7%) and the negative percent agreement (NPA) was 100% (95% CI: 97.8–100%). The κ value of 0.677 indicated that the agreement was substantial. The overall false-negative rate by ID NOW was 47% (7/15). Since the sample size was limited on m2000sp, Xpert Xpress, and LDT platforms, the statistical analysis was performed only on ID NOW against Fusion (Table 1 ). The overall percent agreement (OPA) was 96.9% (95% CI: 92.3–99.2%) with κ value of 0.735 (95% CI: 0.488–0.981). PPA and NPA were 60.0% (95%CI: 26.2–87.8%) and 100% (95% CI: 97.0–100%), respectively, between ID NOW and Fusion (Table 1). The false-negative rate by ID NOW against Fusion was 40% (4/10).

Table 1.

Multiple 2 × 2 table formats – ID NOW vs. each RT-PCR platform tested.

		Fusion		m2000sp		Xpert Xpress		LDT		Total
		Pos	Neg	Pos	Neg	Pos	Neg	Pos	Neg
ID NOW	Pos⁎	6	0	1	0	0	0	1	0	8
	Neg&	4	119	1	20	1	20	1	8	174
Total		10	119	2	20	1	20	2	8	182
PPA+ (95% CI)		60% (26.2–87.8%)		ND#		ND		ND		53.3% (26.6–78.7%)
NPA$ (95% CI)		100% (97.0–100%)		ND		ND		ND		100 (97.8–100%)
OPA= (95% CI)		96.9% (92.3–99.2%)		ND		ND		ND		96.2% (92.2–98.4%)
κ (95% CI)		0.735 (0.488–0.981)		ND		ND		ND		0.677 (0.455–0.899)

^⁎Pos: Positive.

^&Neg: Negative.

^#ND: Not done.

⁺PPA: Positive percent agreement.

^$NPA: Negative percent agreement.

⁼OPA: Overall percent agreement.

Discussion

Harrington et al. showed a PPA of 75% between ID NOW and m2000sp whereas Zhen et al. showed an 87.7% PPA between ID NOW and Fusion. In contrast, our data demonstrated a much lower PPA between ID NOW and all RT-PCR platforms (53.3%) as well as between ID NOW and Fusion alone (60.0%). A major difference between our study and the formers is the sample type. Harrington et al. and Zhen et al. utilized nasal swabs and NPS-VTM, respectively, whereas we used dNPS. This might raise a question of what specimen type is optimal for reliable results. About a week before the FDA recall of NPS-VTM on ID NOW, we switched from using NPS-VTM to dNPS for ID NOW testing due to a concern of potentially high number of false-negative results (Communication with technical support from Abbott).

The weakness of our study is limited paired positive samples. Specifically, paired testing was not performed when a dNPS was initially tested positive by ID NOW since clinicians were confident on the positive testing results by ID NOW. Additionally, discrepant results between ID NOW and each platform were not repeated due to limited reagent availability and the complex workflow coordination among three testing divisions.

Here, we focused on identifying the overall percentage of non-agreement (3.8%) and the false-negative rate of the rapid ID NOW assay (47% in comparison with all platforms and 40% with Fusion only). The rate of false negativity observed in this study substantiated the findings from other studies that used different sample types on the ID NOW platform (Harrington et al., 2020; Rhoads et al., 2020; Zhen et al., 2020b). Despite the shortest turnaround time provided by the ID NOW platform, a much more accurate assay performance is highly expected due to severe clinical manifestations/complications of COVID-19 and the likelihood of further spreading infection by those tested false-negatives. Therefore, our institute generated an ordering algorithm on SARS-CoV-2 testing to make sure clinicians order RT-PCR testing if results from ID NOW do not fit in the clinical indications. Meanwhile, the NPA of ID NOW from dNPS was high which is relatively easy to achieve in low prevalence settings. This finding was similar to others that used different sample types (Harrington et al., 2020; Rhoads et al., 2020; Zhen et al., 2020b).

Conclusions

In this study, we showed the overall agreement of 96.2% and the false-negative rate of 47% using dNPS on ID NOW. More studies that simultaneously compare different sample types – nasal swabs, dNPS, throat, oropharyngeal swabs, saliva, etc. – would shed more insights into a better understanding of the sample type-specific performance characteristics of the rapid ID NOW platform.

Funding

This project did not receive any funding support from any agencies in the public, commercial, or not-for-profit sectors.

Declaration of conflict of interest

All authors have no conflict of interest.