Comparison of an ID NOW COVID-19 Assay Used at the Point of Care to Laboratory-Based Nucleic Acid Amplification Tests

ABSTRACT

The emergence of a novel coronavirus, namely, SARS-CoV-2, necessitated the use of rapid, accurate diagnostics to quickly diagnose COVID-19. This need has increased with the emergence of new variants and continued waves of COVID-19 cases. The ID NOW COVID-19 assay is a rapid nucleic acid amplification test (NAAT) that is used by hospitals, urgent care facilities, medical clinics, and public health laboratories for rapid molecular SARS-CoV-2 testing at the point of care. The District of Columbia Department of Forensic Sciences Public Health Laboratory Division (DC DFS PHL) implemented ID NOW COVID-19 testing in nontraditional laboratory settings, including a mobile testing unit, health clinic, and emergency department, to assist with rapid identification and isolation for populations at high risk of SARS-CoV-2 transmission in the District of Columbia. The DC DFS PHL provided these nontraditional laboratories with safety risk assessment, assay training, competency assessment, and quality control monitoring as parts of a comprehensive quality management system (QMS). We assessed the accuracy of the ID NOW COVID-19 assay when operated in the context of these trainings and systems. This was done by comparing results from 9,518 paired tests, and strong agreement (κ = 0.88, OPA = 98.3%) was found between the ID NOW COVID-19 assay and laboratory-based NAATs. These findings indicate that the ID NOW COVID-19 assay can be used to detect SARS-CoV-2 in nontraditional laboratory settings when used within the context of a comprehensive QMS.

INTRODUCTION

COVID-19, which is the disease caused by SARS-CoV-2, emerged in late 2019 in Wuhan, China, and became a global pandemic. The first diagnostic assay to receive U.S. Food and Drug Administration (FDA) emergency use authorization (EUA) for detecting SARS-CoV-2 was the Centers for Disease Control and Prevention (CDC) 2019-Novel Coronavirus (2019-nCoV) Real-Time Reverse Transcriptase (RT)-PCR Diagnostic Panel, which could only be run in high-complexity diagnostic testing laboratories (1). Outbreaks in health care facilities (2, 3) and congregate living settings (4–7) brought increased attention to the necessity for rapid point-of-care (POC) diagnostic tests (8–10).

Abbott Diagnostics (Scarborough, ME) received FDA EUA for its ID NOW COVID-19 assay on March 27, 2020 (11). This test uses an isothermal reaction to amplify and detect the SARS-CoV-2 RNA-dependent RNA polymerase (RdRp) gene and has a manufacturer-reported analytical sensitivity of 125 genome equivalents/mL (12). The EUA includes authorization for use at the point of care, outside traditional diagnostic laboratories, at Clinical Laboratory Improvements Amendments (CLIA) waived sites. This allows the assay to be performed at doctors’ offices, clinics, and nontraditional laboratory settings, such as homeless shelters and long-term-care facilities.

Multiple studies reported false negative rates of as high as 45% in hospital settings as laboratories across the country implemented ID NOW COVID-19 testing (13–15). However, other studies found a high degree of agreement between the ID NOW COVID-19 assay and other diagnostic assays (16, 17). None of these studies mention training or any other components of laboratory quality management. The goal of this study was to determine the accuracy of the ID NOW COVID-19 assay when used in nontraditional laboratory settings with the implementation of a robust quality management system (QMS).

MATERIALS AND METHODS

ID NOW instrument deployment and personnel training.

The District of Columbia Department of Forensic Sciences Public Health Laboratory Division (DC DFS PHL) implemented POC testing in three nontraditional settings: a mobile testing unit (MTU), a community medical clinic, and a hospital emergency department.

The MTU was equipped with ID NOW instruments to provide rapid on-site testing for facilities that were at high risk for SARS-CoV-2 spread, including long-term-care facilities, intermediate care facilities, homeless shelters, and transitional housing. The MTU was staffed by a rotating roster of 15 medical technologists, including DC DFS PHL laboratorians and contractors who were specifically hired to work on the MTU. All of the MTU testing staff had fewer than 2 years of molecular diagnostic testing experience (median of 1 week). The staff were trained and competency assessed before testing began. The competency assessment included all six areas of competency that were required for moderate-complexity and high-complexity diagnostic testing (18).

The DC DFS PHL also placed ID NOW instruments at a community medical clinic (Facility A) and a hospital emergency department (Facility B). Each facility performed a comprehensive site assessment and a safety risk assessment with DC DFS PHL support before testing began. Measures were put in place to mitigate the safety risks that were associated with testing at each facility. DC DFS PHL laboratorians trained the testing staff at each facility and performed competency assessments before testing began. DC DFS PHL provided refresher training as needed.

Proficiency testing is not required for CLIA-waived tests, including the ID NOW COVID-19 assay. However, as a means of demonstrating continued competency, testing staff from the MTU and both facilities participated in semiannual proficiency testing through a nationally recognized proficiency testing program (Wisconsin State Laboratory of Hygiene, Madison, WI).

Sample collection.

Paired respiratory swabs were collected from all patients at all MTU-deployed facilities, Facility A, and Facility B. Paired swabs are defined as a preliminary respiratory swab to be used for screening with the ID NOW COVID-19 assay and a supplemental respiratory swab that was collected at the same time from the same collection site to be tested using a laboratory-based NAAT for comparison. The swab type varied by facility because of supply chain limitations and differences in facility policies. Only swab types approved by the test manufacturer were used. The swab types included nasal midturbinate, nasopharyngeal, and oropharyngeal swabs. Further descriptions of the swab types and patient populations are detailed below.

Testing algorithms for different sites.

The ID NOW COVID-19 assay is approved for emergency use, and the instructions for use (IFU) were followed without any modifications (19). The first swab from each patient was returned to its original packaging (not placed in viral transport medium) for transfer to the MTU that was parked directly outside the facility or to the ID NOW COVID-19 testing areas within Facilities A and B. Initially, the samples were stored for <2 h at room temperature or <24 h at 2°C to 8°C before testing. Following an update to the IFU (12), samples collected after October 1, 2020, were stored for <1 h at room temperature before testing. The following sections describe the specific algorithm details of each facility.

Mobile testing unit.

The MTU is a 38-foot truck that was retrofitted to contain a complete laboratory space, including an electrical generator, climate control, a Class II Type A/B3 biological safety cabinet, and sufficient bench space for multiple ID NOW instruments. Testing was performed on board by at least two trained medical technologists using three to five ID NOW instruments.

The DC DFS PHL used the MTU to provide routine SARS-CoV-2 testing to residents and staff at high-risk facilities that met one of the following two requirements that were set by the DC Department of Health (DC Health). First, the facility had ≥3 residents or staff with a positive SARS-CoV-2 test or, second, the facility had ≥1 resident or staff with a positive SARS-CoV-2 test who had not been isolated soon enough to prevent multiple close contacts. All consenting facility residents and staff were tested, regardless of symptom status or past test results. The MTU was deployed 27 times to 12 different facilities between May 21, 2020, and February 12, 2021. Two nasal midturbinate swabs were collected by swabbing both nares from each patient (n = 1,027). The first swab was tested on the MTU by using an ID NOW COVID-19 assay. The second swab from each patient was placed in viral transport media (VTM) (Remel, Lenexa, Kansas, or Hardy Diagnostics, Santa Maria, CA) upon collection and refrigerated (2°C to 8°C) during transport to the DC DFS PHL for testing. Comparator NAAT testing at the DC DFS PHL was performed on the same day as the collection by using either a Panther Fusion SARS-CoV-2 Assay (Hologic, San Diego, CA) or a GeneXpert SARS-CoV-2 Assay (Cepheid, Sunnyvale, CA), following the IFU (20, 21).

Facility A.

Facility A is a community health center serving a culturally and economically diverse patient population in Ward 4 of Washington, DC. The testing pool for this study was primarily symptomatic patients seeking care at Facility A’s respiratory clinic. Certain asymptomatic patients were also tested when their health care provider had other reasons to suspect infection with SARS-CoV-2, such as an epidemiological link to other patients with known SARS-CoV-2 infections. Two oropharyngeal swabs were collected from each patient (n = 1,094) between May 6, 2020, and June 1, 2021. The first swab was tested at the point of care by using an ID NOW COVID-19 assay. The second swab was placed in VTM upon collection, refrigerated (2°C to 8°C), and shipped overnight on ice packs to a commercial laboratory (Laboratory Corporation of America) for testing using a proprietary assay. Cycle threshold (Ct) values are not reported for this assay.

Facility B.

Facility B is a 330-bed, tertiary care hospital in Ward 8 of Washington, DC. At Facility B, both asymptomatic and symptomatic patients were tested upon arrival at the emergency department. Two nasopharyngeal swabs were collected from each patient (n = 7,397) between May 22, 2020, and May 21, 2021. The first swab was tested at the point of care by using an ID NOW COVID-19 assay. The second swab was placed in VTM upon collection and transported on ice packs (2°C to 8°C) to Facility B’s clinical laboratory for testing using an Aptima SARS-CoV-2 Assay (Hologic, San Diego, CA), following the IFU (22). This assay does not generate Ct values.

Data analysis.

Paired swabs, rather than patients, were used as data points because some individual patients were tested on more than one occasion during the study period. The paired test results from the MTU and each facility were collected and anonymized. A pair of tests was excluded from the analysis if the result from either test was invalid. The data from the MTU results were analyzed after each deployment. The results from Facility A were analyzed retrospectively after the study period. The results from Facility B were collected in weekly reports and analyzed on a continual basis.

The overall percent agreement (OPA), positive percent agreement (PPA), and negative percent agreement (NPA) were calculated by using the FDA guidelines for comparing a diagnostic assay to a nonreference standard (23). Briefly, OPA reflects the number of paired swabs for which both assays return the same result (i.e., positive or negative), divided by the total number of paired swabs tested. PPA is analogous to the positive predictive value and reflects the number of paired swabs that tested positive with both assays, divided by the number of paired swabs that tested positive with the ID NOW COVID-19 assay. NPA is analogous to the negative predictive value and reflects the number of paired swabs that tested negative with both assays, divided by the number of paired swabs that tested negative with the ID NOW COVID-19 assay.

Positivity was calculated as the number of paired swabs for which either or both of the test results were positive, divided by the total number of paired swabs.

Cohen’s kappa (κ) was calculated as an estimate of agreement (24). The values of κ were interpreted as follows: 0 ≤ κ < 0.2, no agreement; 0.2 ≤ κ < 0.4, minimal agreement; 0.4 ≤ κ < 0.6, weak agreement; 0.6 ≤ κ < 0.8, moderate agreement; 0.8 ≤ κ < 0.9, strong agreement; 0.9 ≤ κ < 1.0, almost perfect agreement; κ = 1.0, perfect agreement (25).

Ethical review.

This activity was reviewed by the CDC and was conducted in accordance with applicable federal law and CDC policy.^§

RESULTS

Overall test results.

In total, 9,518 paired swabs were tested for SARS-CoV-2 by using the ID NOW COVID-19 assay and a comparator NAAT from the MTU, Facility A, and Facility B (Table 1). The positivity was 8.2% (779/9,518). The PPA was 88.2% (621/704). The NPA was 99.1% (8,739/8,814). The OPA was 98.3% (9,360/9,518). Because the populations being tested were mostly SARS-CoV-2 negative, it is possible for the high NPA to skew OPA. To account for this, we calculated Cohen’s kappa (κ). κ was 0.88, indicating strong agreement between the ID NOW COVID-19 assay and other NAATs.

TABLE 1.

Concordance between ID NOW COVID-19 assay and laboratory-based nucleic acid amplification tests (NAATs) across three testing sites^a

	ID NOW
Laboratory-based NAAT	Positive	Negative	Positivity	PPA	NPA	OPA	Kappa (κ)
Mobile testing unit (n = 1,027)
Panther Fusion, Cepheid GeneXpert
Positive	21	4	2.8% (29/1,027)	84.0%	99.6%	99.2%	0.84
Negative	4	998
Total	25	1002
Facility A (n = 1,094) LabCorp
Positive	129	8	15.1% (165/1,094)	82.2%	99.1%	96.7%	0.86
Negative	28	929
Total	157	937
Facility B (n = 7,397) Panther Aptima
Positive	471	63	7.9% (585/7397)	90.2%	99.1%	98.5%	0.88
Negative	51	6812
Total	522	6875
Total (n = 9,518)
Positive	621	75	8.2% (779/9518)	88.2%	99.1%	98.3%	0.88
Negative	83	8739
Total	704	8814

^aThe data represent numbers of paired swabs.

Mobile testing unit.

On the MTU, 1,027 paired swabs were tested for SARS-CoV-2 during 27 deployments to 12 high-risk facilities. The positivity was 2.8% (29/1027). The PPA was 84.0% (21/25). The NPA was 99.6% (998/1,002). The OPA was 99.2% (1,019/1,027). The κ was 0.84, indicating strong agreement (Table 1). The ID NOW COVID-19 assay reports only a positive or negative result, whereas the laboratory-based NAATs also report a cycle threshold (Ct) value. The test results for the 29 paired swabs that tested positive using one or both assays are summarized in Table 2. A small percentage (4/1,027; 0.4%) of the MTU specimens tested positive with the ID NOW COVID-19 assay and negative with a comparator laboratory-based NAAT. 25 MTU specimens tested positive with the comparator laboratory-based NAAT (25/1,027; 2.4%). Of these, 84% (21/25) also tested positive with the ID NOW COVID-19 assay; 16% (4/25) were discordant. Fig. 1 shows the Ct values of these positive tests, grouped by their result with the ID NOW COVID-19 assay. The mean Ct value of the concordant results (n = 21) was 26.7. The specimens (n = 4) that tested negative with the ID NOW COVID-19 assay had a mean Ct value of 38.

TABLE 2.

Positive results from specimens collected on the MTU, including specimens that tested positive on an ID NOW COVID-19 assay on the MTU, laboratory-based NAATs, or both^a

Result	Date	ID NOW result	Lab NAAT result	Lab NAAT Ct value	Lab NAAT used
Concordant Positive	1/15/2021	POS	POS	36.3	Panther Fusion
	5/28/2020	POS	POS	34.4	Panther Fusion
	5/28/2020	POS	POS	34.4	Panther Fusion
	5/21/2020	POS	POS	33.0	GeneXpert
	5/28/2020	POS	POS	32.2	Panther Fusion
	5/28/2020	POS	POS	31.1	Panther Fusion
	1/28/2021	POS	POS	30.9	Panther Fusion
	7/17/2020	POS	POS	30.0	GeneXpert
	12/1/2020	POS	POS	29.1	Panther Fusion
	12/1/2020	POS	POS	26.7	Panther Fusion
	2/2/2021	POS	POS	25.5	Panther Fusion
	12/11/2020	POS	POS	25.0	GeneXpert
	2/2/2021	POS	POS	25.0	GeneXpert
	12/1/2020	POS	POS	23.1	Panther Fusion
	2/2/2021	POS	POS	22.5	Panther Fusion
	2/2/2021	POS	POS	22.2	Panther Fusion
	12/29/2020	POS	POS	22.0	GeneXpert
	5/21/2020	POS	POS	21.0	GeneXpert
	1/28/2021	POS	POS	20.1	Panther Fusion
	5/28/2020	POS	POS	19.1	Panther Fusion
	1/28/2021	POS	POS	18.1	Panther Fusion
Discordant Negative	7/17/2020	NEG	POS	44.0	GeneXpert
	8/25/2020	NEG	POS	37.9	Panther Fusion
	12/4/2020	NEG	POS	37.1	Panther Fusion
	7/17/2020	NEG	POS	33.0	GeneXpert
Discordant Positive	5/28/2020	POS	NEG	-b	Panther Fusion
	5/28/2020	POS	NEG	-	Panther Fusion
	5/28/2020	POS	NEG	-	Panther Fusion
	2/2/2021	POS	NEG	-	Panther Fusion

^aThe cycle threshold values that are given are from laboratory-based NAATs.

^bNegative results from the laboratory-based NAATs generated no Ct value and are indicated with “-”.

FIG 1.

Cycle threshold (Ct) values for all specimens collected on the mobile testing unit (MTU) that tested positive on a laboratory-based nucleic acid amplification test (NAAT). Specimens that also tested positive on an ID NOW COVID-19 assay are shown in blue. Specimens that tested negative on an ID NOW COVID-19 assay are shown in red. Horizontal bars represent the mean Ct values.

Fig. 2 chronologically depicts the number of positive ID NOW COVID-19 test results from each MTU deployment. Discordant positive results (i.e., patients who tested positive with the ID NOW COVID-19 assay and negative with a laboratory-based NAAT) are indicated. During the MTU deployment on May 28, 2020, 8 out of 49 patient specimens tested positive with ID NOW, but only 5 of the 8 paired swabs were also positive when tested with the laboratory-based NAAT. Further investigation of these discrepancies revealed that each of the three discordant positive specimens was tested immediately after a concordant positive patient specimen or a positive control sample was tested using the same instrument.

FIG 2.

Positive ID NOW COVID-19 results on the mobile testing unit (MTU). Concordant positive results (i.e., those that were also positive on the lab-based nucleic acid amplification test [NAAT]) are shown in blue. Discordant positive results (i.e., those that were negative on the laboratory-based NAAT) are shown in red.

Facility A.

Facility A tested 1,094 paired swabs for SARS-CoV-2. The positivity for this population was 15.1% (165/1,094). The positive percent agreement was 82.2% (129/157). The NPA was 99.1% (929/937). The OPA was 96.7% (1,058/1,094). The κ was 0.86, indicating strong agreement (Table 1). Monthly performance metrics (PPA, NPA, and OPA) were calculated for the tests that were performed at Facility A (Fig. 3). The NPA and OPA remained mostly consistent throughout the study. However, decreases in the PPA occurred during July of 2020 (66.7%) and April of 2021 (12.5%). Because the performance metrics for Facility A were obtained and analyzed after the study period, no investigation into these decreases was performed in real-time. A retrospective analysis is described in the Discussion.

FIG 3.

Monthly quality metrics from Facility A. Quality metrics were calculated retrospectively. The positive percent agreement (PPA), negative percent agreement (NPA), and overall percent agreement (OPA) are shown.

Facility B.

From May 22, 2020, to May 21, 2021, Facility B tested 7,397 paired swabs for SARS-CoV-2. The positivity was 7.9% (585/7,397). The PPA was 90.2% (471/522). The NPA was 99.1% (6,812/6,875). The OPA was 98.5% (7,283/7,397). The κ was 0.88, which again indicated strong agreement (Table 1). The quality metrics from Facility B were monitored throughout the study period by using weekly diagnostic testing results reports. Before October 17, 2020, positive test results at Facility B were sporadic; several weeks had no positive test results. This inconsistency made weekly quality trends difficult to interpret (data not shown). The weekly OPA, PPA, and NPA values from October 17, 2020, through May 21, 2021, are shown in Fig. 4. The OPA and NPA values remained stable throughout the study period. The PPA had a notable drop, beginning the week of December 18, 2020, reaching a low of 77.8% during the week of February 19, 2021. DC DFS PHL provided refresher training to the Facility B testing staff on March 2, 2021 (indicated by an asterisk in Fig. 4). The training focused on avoiding cross-contamination and included how and when to clean the instrument, when to change gloves, and general quality assurance. After the training refresher, the PPA rose to 100%. The PPA dropped again to 75% 8 weeks later.

FIG 4.

Quality assurance and training reduce discordant ID NOW positives at Facility B. Quality metrics were observed in real time through weekly reports. X axis date reflects the first day of the week represented. Weekly positive percent agreement (PPA), negative percent agreement (NPA), and overall percent agreement (OPA) are shown. *Refresher training on 3/2/2021.

DISCUSSION

In December of 2021, President Biden committed to placing rapid diagnostic testing at the forefront of U.S. plans for combating the COVID-19 pandemic, particularly against the emerging Omicron variant (26). Diagnostic assays that target the spike (S) gene may not detect the SARS-CoV-2 Omicron variant because mutations in its S gene prevent primer binding (27). The ID NOW COVID-19 assay targets the RdRp gene and robustly detects variants with highly mutated S genes, including the Omicron variant. This study assessed the accuracy of this rapid POC testing by comparing results from 9,518 ID NOW COVID-19 tests with results from paired swabs that were tested with laboratory-based NAATs.

At two different facilities (representing a tertiary care emergency department and a local health clinic) and on the MTU, Cohen’s kappa ranged from 0.84 to 0.88, indicating strong agreement between the ID NOW COVID-19 assay and the different comparator laboratory-based NAAT assays. This strong agreement is in stark contrast to the previous reports of poor clinical performance (13) and low analytical sensitivity, compared with other assays (28). Notably, the NPA, which estimates the reliability of a negative result from the ID NOW COVID-19 assay, was 99.1% overall, among all study sites, which is higher than was expected, given the previously reported high false negative rate.

Low pretest probability (caused by testing a largely SARS-CoV-2 negative population) can contribute to a high NPA, but it cannot completely explain the observed differences in the reliability of the ID NOW COVID-19 assay between our study and previous studies. Patients tested at Facility A were primarily people with respiratory symptoms, whereas those tested on the MTU were primarily asymptomatic residents of high-risk facilities with ≥1 known COVID-19 cases. Unsurprisingly, Facility A’s patient population had a much higher overall SARS-CoV-2 positivity rate (15.1%) than the population that was tested on the MTU (2.8%). Despite this higher positivity rate, which increases pretest probability, Facility A did not have a higher NPA than the MTU (99.1% versus 99.6%).

We attribute the improved reliability of the ID NOW COVID-19 assay to our focus on training and quality assurance. The DC DFS PHL sent a laboratory team, including representatives from quality, biosafety, and operations, to perform multiple site visits prior to implementing the ID NOW COVID-19 assay at each facility. During these visits, the laboratory team assessed the location where testing would be conducted, evaluated potential contamination events, discussed biosafety, and performed a full competency assessment. The DFS DC PHL ensured that each facility had an appropriate CLIA certificate to conduct waived clinical testing. Constant and consistent lines of communication were key to ensuring quality implementation of the test at each facility.

The ongoing monitoring of testing quality metrics on the MTU and at Facility B supports the importance of a robust QMS for maintaining reliable and accurate results. We tracked quality metrics to find areas for improvement, implemented actions to improve the quality of test performance, and observed improvements in quality metrics.

Three discordant positive results were observed on the MTU in one day (May 28, 2020) (Fig. 2). Each of these discordant results was immediately preceded by a concordant positive or a positive control sample, strongly suggesting that amplicon contamination from the previous test caused false positive results. Our protocol at this time included wiping each instrument with 70% ethanol after each test to avoid cross-contamination. Based on our observations, we supplemented this protocol with a second wipe after any positive test. After the implementation of this more stringent procedure for instrument cleaning between tests, we observed only one additional discordant positive result from 930 tests that were run during the subsequent 25 MTU deployments. If these three results were omitted from the data set, the PPA would rise to 95.5%, the OPA would rise to 99.5%, and the NPA would remain unchanged.

Comparative laboratory-based NAAT testing for specimens collected on the MTU was performed by using a Panther Fusion SARS-CoV-2 Assay or a Cepheid GeneXpert SARS-CoV-2 Assay, both of which report a Ct value for each positive test result. The mean Ct values were higher (38.0) for the patients who received a negative result with the ID NOW COVID-19 assay and a positive result with the laboratory-based NAAT than for those who were positive with both assays (26.7) (Fig. 1). This difference in mean Ct values suggests that the patients who tested negative by using the ID NOW COVID-19 assay and positive by using the laboratory-based NAAT might have had lower viral loads, on average, than did those who tested positive with both assays. Previous studies have reported the inability to culture SARS-CoV-2 virus from specimens with Ct values above 34 (29, 30). Other studies have reported that patients with lower viral loads tended to have lower incidence of severe disease and mortality (31, 32). Taken together, these data suggest that SARS-CoV-2 infections that are not detected by an ID NOW COVID-19 assay might be less severe or less infectious. Infections that are missed by ID NOW may also be late stage or early stage infections, which have lower average viral loads than midstage disease. A larger data set is required to conclusively demonstrate this.

At Facility B, we observed a period of poor test performance that was marked by a low PPA beginning the week of December 18, 2020 (Fig. 4). DC DFS PHL staff provided refresher training on instrument and workspace cleaning to avoid cross-contamination and reduce false positive test results. For several weeks after this refresher training, the PPA at Facility B rose, peaking at 100% during the week of March 20, 2021. However, the improvement in PPA was temporary. PPA dropped several weeks after the refresher training was provided. This drop highlights the importance of maintaining a consistent focus on continuous quality improvement. No further refresher training was provided because of staffing limitations.

Facility A demonstrated stable quality metrics for the majority of the study period (Fig. 3). However, the PPA at Facility A dropped from 100% (March of 2021) to 12.5% (April of 2021), and it then increased back to 100% (May of 2021). In May of 2021, Facility A’s ID NOW COVID-19 testing group received a perfect score in a nationally recognized proficiency testing program. Investigation into the poor performance observed during April of 2021 revealed that each positive result obtained at Facility A by using the ID NOW COVID-19 assay between April 6, 2021, and April 27, 2021, (n = 7) disagreed with the paired result from the commercial testing laboratory to which Facility A sent samples for laboratory-based NAAT testing.

A definitive cause for this three-week period of discrepant tests could not be identified. Facility A used a single shipment of a single lot of unexpired test reagents from February 25, 2021, through April 27, 2021, before changing to a new lot on April 28, 2021. The fact that this change coincides with the end of the period of discrepant test results suggests that this lot of test reagents may have been defective in a way that caused their performance to decline before their expiration date. However, one would expect this type of defect to cause a gradual decline in test performance rather than the sudden drop that was observed here. The possibility also exists that discrepant results might have been attributable to issues with the specimens that were shipped off-site for laboratory-based NAAT testing and not with the ID NOW COVID-19 assay that was performed at Facility A. Such issues could include incorrect storage temperatures before shipping, breaks in the cold chain during shipping, or extended time between sample collection and off-site testing.

LIMITATIONS

Certain limitations in study design must be acknowledged when interpreting these data. We did not have access to patient medical chart data (i.e., demographics, symptoms, and other relevant information), which precluded the correlation of discordant results with patient factors, such as disease status.

We acknowledge that specimen quality can affect test performance. Because paired tests were performed on two sequentially collected specimens, rather than on a single split specimen, it is possible that some discordant results could be attributable to one specimen having a lower viral load due to degradation or errors in specimen collection. For Facility A, the added step of specimen transport to a commercial laboratory for testing introduced opportunities (e.g., a break in the cold chain) for sample degradation. Such degradation might explain Facility A having the lowest PPA observed in this study.

Each facility used a unique testing algorithm, including different laboratory-based NAATs with different analytical sensitivities. This might explain some of the differences that were observed between the comparative results of the facilities.

CONCLUSIONS

Previous studies have disagreed about the accuracy of the ID NOW COVID-19 assay, with some reporting it to be inaccurate and others reporting a high degree of comparability to laboratory-based tests. Based on our data, the ID NOW COVID-19 assay can successfully be used at the point of care to produce accurate and reliable results in individuals with high viral burdens (Ct value of <37). These results were comparable to those that were obtained by using more technically complex methodologies in a high-complexity diagnostic testing laboratory. The ID NOW COVID-19 assay was highly reliable, whether testing was performed by public health laboratorians on the mobile testing unit or by nonlaboratory staff at one of the two medical facilities. The ID NOW COVID-19 assay had an overall percent agreement of 98.3% and a value for κ that indicated strong agreement with the comparator NAATs. Based on our data, the ID NOW assay can be used to test for and detect SARS-CoV-2 in nontraditional laboratory settings when comprehensive training, competency assessments, and other elements of a robust quality system are prioritized.