Prospective population-level validation of the Abbott ID NOW severe acute respiratory syndrome coronavirus 2 device implemented in multiple settings for testing asymptomatic and symptomatic individuals

Abstract

Objective

Diagnostic evaluation of the ID NOW coronavirus disease 2019 (COVID-19) assay in various real-world settings among symptomatic and asymptomatic individuals.

Methods

Depending on the setting, the ID NOW testing was performed using oropharyngeal swabs (OPSs) taken from patients with symptoms suggestive of COVID-19, asymptomatic close contacts, or asymptomatic individuals as part of outbreak point prevalence screening. From January to April 2021, a select number of sites switched from using OPS to combined oropharyngeal and nasal swab (O + NS) for ID NOW testing. For every individual tested, two swabs were collected by a health care worker: one swab (OPS or O + NS) for ID NOW testing and a separate swab (OPS or nasopharyngeal swab) for RT-PCR.

Results

A total of 129 112 paired samples were analysed (16 061 RT-PCR positive). Of these, 81 697 samples were from 42 COVID-19 community collection sites, 16 924 samples were from 69 rural hospitals, 1927 samples were from nine emergency shelters and addiction treatment facilities, 23 802 samples were from six mobile units that responded to 356 community outbreaks, and 4762 O + NS swabs were collected from three community collection sites and one emergency shelter. The ID NOW assay sensitivity was the highest among symptomatic individuals presenting to community collection sites (92.5%; 95% CI, 92.0–93.0%) and the lowest for asymptomatic individuals associated with community outbreaks (73.9%; 95% CI, 69.8–77.7%). Specificity was >99% in all populations tested.

Discussion

The sensitivity of ID NOW severe acute respiratory syndrome coronavirus 2 testing is the highest when used in symptomatic community populations not seeking medical care. Sensitivity and positive predictive value drop by approximately 10% when tested on asymptomatic populations. Using combined oropharyngeal and nasal swabs did not improve the performance of ID NOW assay.

Introduction

The ID NOW coronavirus disease 2019 (COVID-19) assay (Abbott, Chicago, IL, United States) is approved by the U.S. Food and Drug Administration Emergency Use Authorization for the point-of-care, rapid detection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in individuals who are within the first 7 days of symptom onset [1].

The ID NOW assay's limit of detection approximates 250–500 copies/mL and is higher than other lab-based real-time RT-PCR platforms, which are typically under 200 copies/mL [2,3]. The pooled clinical sensitivity and specificity of the ID NOW assay from 13 studies, compared with RT-PCR, were 73.0% (95% CI, 66.8–78.4%) and 99.7% (95% CI, 98.7–99.4%), respectively [4]. However, these studies were limited by sample size and heterogeneous design, such as using different specimen types for testing. There is also a paucity of data on ID NOW performance in asymptomatic individuals, which may be negatively affected given the higher cycle threshold (Ct) values observed in this population [5].

This study had two aims. The first was to prospectively evaluate the clinical performance of the ID NOW compared with RT-PCR in various settings serviced by the sole provincial health authority in the province of Alberta, Canada (4.4 million people). We evaluated the clinical sensitivity and specificity in these four settings:

• 1.Symptomatic individuals or asymptomatic close contacts presenting to community COVID-19 assessment/swab centres.
• 2.Symptomatic inpatients or patients in the emergency department (ED) (i.e. hospital).
• 3.Symptomatic individuals in emergency shelters and addiction treatment facilities.
• 4.Symptomatic or asymptomatic individuals associated with community outbreaks. Most outbreaks were in continuing care centres (including long-term care and designated supportive living environments), followed by industry settings and congregate care.

The second aim of this study was to compare the differences in sensitivity/specificity between the use of oropharyngeal swabs (OPSs) and the use of combined oropharyngeal and nasal swabs (O + NS) for ID NOW testing in two different settings: assessment centres and emergency shelters and addiction treatment facilities. Although OPSs have slightly lower sensitivity than nasopharyngeal swab (NPSs) for SARS-CoV-2 detection using RT-PCR, several studies have demonstrated improved sensitivity with O + NS [[6], [7], [8]].

Methods

Since 4 December 2020, the ID NOW assay was gradually implemented across all regions of Alberta in the following sites:

• 1.Fourty-two COVID-19 Alberta Health Services Public Health assessment/swabbing centres. These are the primary locations for symptomatic and asymptomatic close contact community individuals not needing medical attention to get tested for COVID-19 in Alberta. Testing and swabbing were performed by the assessment centre staff (e.g. nurses).
• 2.Sixty-nine rural hospitals located for the testing of symptomatic inpatients or patients in the ED. Swabbing was performed by physicians, nurses, or respiratory therapists.
• 3.Nine urban emergency shelters and addiction treatment facilities for testing of symptomatic residents. Testing and swabbing were performed on-site by nurses.
• 4.Six mobile units for testing of symptomatic or asymptomatic residents or staff at community outbreak sites. Swabbing and testing were performed by mobile nursing team staff on-site in retrofitted vans.

Individuals at assessment centres were labelled as ‘symptomatic’ if they had one or more of the following COVID-19 symptoms: fevers or chills, runny or stuffy nose, sore throat, cough, difficulty breathing or shortness of breath, nausea or diarrhoea, or loss or altered sense of taste/smell. Individuals at the non-assessment centre locations were also considered symptomatic at the health care provider's discretion. All individuals tested with the ID NOW assay had two parallel swabs collected by trained health care professionals. The first swab collected was either an NPS or OPS, which was placed in universal transport media (UTM) (Yocon Biology, Beijing, China or GDL Korea Co. Ltd, Seoul, Korea) for RT-PCR and transported to an accredited laboratory at room temperature and stored at 4 °C until processing or storage within 72 hours. The second swab was an OPS for ID NOW testing (using the swab provided in the ID NOW kits). The OPS for ID NOW testing was always collected second to ensure that all individuals had a sample available for RT-PCR (i.e. in case the individual refused the second NPS or OPS). After swabbing, the ID NOW swabs were placed in the swab sterile package or a sterile container and tested on-site and within 1 hour as per manufacturer's instructions. If the ID NOW test was negative, the second swab was sent for confirmatory RT-PCR testing using the Alberta Provincial Public Health Laboratory (ProvLab) E gene RT-PCR or on a Health Canada and Food and Drug Administration–approved commercial assay (see "Assays used for RT-PCR testing" in the Supplementary material) [9]. If the ID NOW test was positive, the second swab was either sent for storage (if collected before 1 February 2021) or for variants of concern (VoC) screening. Samples sent for storage (−70 °C) were tested at a later date for SARS-CoV-2, whereas samples sent for VoC testing were evaluated within approximately 72 hours from the time of collection. The ProvLab RT-PCR was exclusively used for RT-PCR testing on stored samples and samples for VoC testing.

Between 26 Jan 2021, and 12 April 2021, three assessment centres and one emergency shelter and addiction treatment facilities switched from using OPS to O + NS for ID NOW testing and OPS or NPS for reference testing. The swab used remained the same (swab provided in Abbott ID NOW kits), and instructions for swabbing were provided to individual sites (see "Details on COVID-19 swabbing and ID NOW testing in the Supplementary material). All other protocols and collectors were the same as above.

All ID NOW samples with parallel RT-PCR results documented were included in our study. Results without proper documentation of testing location or without either confirmatory RT-PCR or VoC testing were excluded.

Data were pulled from our provincial laboratory's centralized electronic database containing SARS-CoV-2 results for all publicly funded testing in the province, except for border testing. The sensitivity and specificity of the ID NOW assay were calculated using the Clopper-Pearson 95% CIs. Statistical analysis was performed using the Pearson chi-square test for categorical variables and a t test using STATA (version 14.1) for continuous variables.

The University of Alberta Research Ethics board approved this study (Pro00111835).

Results

A total of 133 919 results were identified between 4 December 2020, and 24 November 2021. A total of 4807 samples were excluded: 190 did not have testing location recorded, 375 did not have an ID NOW result recorded, three had invalid ID NOW results, 2417 ID NOW results did not have parallel RT-PCR results recorded, and 1822 positive ID NOW samples were not tested for VoC. The remaining 129 112 paired samples were analysed.

ID NOW testing using OPSs

A total of 124 350 samples (15 649 RT-PCR positive) were analysed; 81 697 samples were from 42 assessment centres, 16 924 samples were collected from 69 rural hospitals, 1927 samples were from nine shelters and addiction treatment facilities, and 23 802 samples were from six mobile units that responded to 356 outbreaks. The baseline characteristics of these samples have been provided in Table 1 . Overall, the ID NOW assay sensitivity was the highest among symptomatic individuals presenting to assessment centres (92.5%; 95% CI, 92.0–93.0%; n = 10 633 RT-PCR positive) and the lowest among asymptomatic individuals associated with community outbreaks (73.9%; 95% CI, 69.8–77.7%; n = 494 RT-PCR positive) (Table 2, Table 3 ). The sensitivity was approximately 10% less for the asymptomatic versus symptomatic populations. The specificity was >99% in all populations tested.

Table 1.

Characteristics between individuals tested with ID NOW SARS-CoV-2 using oropharyngeal swabs or combined oropharyngeal and nasal swabs

Site	Symptoms	Mean age (median; range) (y)	Male sex (%)	Nasopharyngeal swab used for parallel RT-PCR testinga (%)
Oropharyngeal swab for ID NOW testing
Assessment centre	Symptomatic (n = 70 879)	34.7 (34.4; 0.04–101)	40.5	49.9
	Asymptomaticb (n = 10 818)	34.5 (32.4; 0.1–102)	46.8	40.7
Hospital	Symptomatic (n = 16 924)	50.7 (53.6; 0.0007–107.9)	48.0	90.2
Emergency shelters and addiction treatment facilities	Symptomatic (n = 1927)	42.2 (39.2; 2.7–101.2)	60.8	14.2
Mobile unit	Symptomatic (n = 4224)	31.8 (31.1; 0.001–103.9)	38.9	15.5
	Asymptomatic (n = 19 578)	51.4 (47.8; 0.5–107.2)	43.5	56.0
Combined oropharyngeal and nasal swab FOR ID NOW testing
Assessment centre	Symptomatic (n = 3890)	35.6 (35.9; 0.7–98.3)	40.8	68.7
	Asymptomaticb (n = 681)	31.2 (29.1; 3.4–83.1)	48.6	76.8
Emergency shelters and addiction treatment facilities	Symptomatic (n = 191)	40.6 (39.4; 19.1–68.2)	62.8	1.1

^aEither nasopharyngeal nor oropharyngeal swab was used for parallel RT-PCR testing. In hospitalized patients, a minority of samples were other specimen types such as endotracheal tube aspirates.

^bAsymptomatic close contacts only.

Table 2.

Performance of ID NOW compared with RT-PCR (nasopharyngeal or oropharyngeal swab), using oropharyngeal swabs or combined oropharyngeal and nasal swabs for ID NOW testing.

Oropharyngeal swab (OPS) for ID NOW testing				Combined oropharyngeal and nasal swab (O+NS) for ID NOW testing
Assessment centre: Symptomatic				Assessment centre: Symptomatic
		RT-PCR				RT-PCR
		Positive	Negative			Positive	Negative
ID NOW	Positive	9833	315	ID NOW	Positive	335	18
	Negative	800	59 931		Negative	28	3509
Assessment centre: asymptomatic				Assessment centre: asymptomatic
		RT-PCR				RT-PCR
		Positive	Negative			Positive	Negative
ID NOW	Positive	768	73	ID NOW	Positive	35	5
	Negative	140	9837		Negative	10	631
Emergency shelters and addiction treatment facilities				Emergency shelters and addiction treatment facilities
		RT-PCR				RT-PCR
		Positive	Negative			Positive	Negative
ID NOW	Positive	62	8	ID NOW	Positive	3	0
	Negative	19	1838		Negative	1	187
Hospital
		RT-PCR
		Positive	Negative
ID NOW	Positive	2624	97
	Negative	308	13 895
Mobile: symptomatic
		RT-PCR
		Positive	Negative
ID NOW	Positive	529	22
	Negative	72	3601
Mobile: asymptomatic
		RT-PCR
		Positive	Negative
ID NOW	Positive	365	78
	Negative	129	19 006

Table 3.

Summary performance of ID NOW (OPS vs. O + NS) compared with RT-PCR (OPS or NPS)

Oropharyngeal swab (OPS) for ID NOW testing
	Sensitivity (%) (95% CI)	Specificity (%) (95% CI)	COVID-19 prevalence (%)	PPV (%) (95% CI)	NPV (%) (95% CI)
Assessment centre: symptomatic	92.5 (92.0–93.0)	99.5 (99.4–99.5)	15.0	96.9 (96.5–97.2)	98.7 (98.6–98.8)
Assessment centre: asymptomatic	84.6 (82.0–86.8)	99.3 (99.1–99.4)	8.4	91.3 (89.2–93.1)	98.6 (98.3–98.8)
Hospital	89.5 (88.3–90.6)	99.3 (99.2–99.4)	17.3	96.4 (95.7–97.1)	97.8 (97.6–98.0)
Mobile: symptomatic	88.0 (85.2–90.5)	99.4 (99.1–99.6)	14.2	96.0 (94.1–97.3)	98.0 (97.6–98.4)
Mobile: asymptomatic	73.9 (69.8–77.7)	99.6 (99.5–99.7)	2.5	82.4 (78.8–85.5)	99.3 (98.8–99.1)
Emergency shelters and addiction treatment facilities	76.5 (65.8–85.3)	99.6 (99.2–99.8)	4.2	88.6 (79.3–94.0)	99.0 (98.5–99.3)
Combined oropharyngeal and nasal swab (O+NS) for ID NOW testing
	Sensitivity (%) (95% CI)	Specificity (%) (95% CI)	COVID-19 prevalence (%)	PPV (%) (95% CI)	NPV (%) (95% CI)
Assessment centre: symptomatic	92.3 (89.0–94.8)	99.5 (99.2–99.7)	9.3	94.9 (92.1–96.7)	99.2 (98.9–99.4)
Assessment centre: asymptomatic	77.8 (62.9–88.8)	99.2 (98.2–99.7)	6.6	87.5 (74.3–94.4)	98.4 (97.3–99.1)
Emergency shelters and addiction treatment facilities	75.0 (19.4–99.4)	100 (98.1–100)	2.1	100	99.5 (97.2–99.9)

COVID-19, coronavirus disease 2019; NPS, nasopharyngeal swab; NPV, negative predictive value; O + NS, combined oropharyngeal + nasal swab; OPS, oropharyngeal swab; PPV, positive predictive value.

The positive predictive value (PPV) was the highest for symptomatic individuals presenting at assessment centres (96.9%; 95% CI, 96.5–97.2%; prevalence 15.0%) and the lowest for asymptomatic patients tested via mobile units (82.4%; 95% CI, 78.8–85.5%; prevalence 2.5%) (Table 3). The negative predictive value (NPV) was the highest for asymptomatic patients tested via mobile units (99.3%; 95% CI 98.8–99.1%) and the lowest for symptomatic inpatients or patients in the ED (97.8%; 95% CI, 97.6–98.0%; prevalence 17.3%).

There was no significant difference in the sensitivity and specificity of the ID NOW assay between symptomatic and asymptomatic outpatients presenting to COVID-19 assessment centres with respect to age, sex, or COVID-19 variants (Tables S2 and S3). There were trends towards improved sensitivity and specificity when an OPS was used over an NPS for RT-PCR reference testing (Tables S4–S7). However, the comparisons in ID NOW performance between each population remained similar regardless of the swab used for RT-PCR (Fig. 1, Fig. 2 ). Some comparisons were limited by sample size (e.g. individuals in emergency shelters).

Fig. 1.

Sensitivity and specificity of ID NOW (oropharyngeal or O + N swab) compared with RT-PCR (nasopharyngeal swab). Error bars represent 95% CIs. AC, assessment centre; O + N, oropharyngeal and nasal.

Fig. 2.

Sensitivity and specificity of ID NOW (oropharyngeal or O + N swab) compared with RT-PCR (oropharyngeal swab). Error bars represent 95% CIs. AC, assessment centre; O + N, oropharyngeal and nasal.

ID NOW testing using combined O + NSs

A total of 4762 paired samples were analysed (412 RT-PCR positive): 4571 samples were from three assessment centres and 191 samples were collected from one urban emergency shelter. The baseline characteristics of these samples are provided in Table 1. The ID NOW results, compared with RT-PCR results, are provided in Table 2, Table 3. There were no statistically significant differences in sensitivity or specificity noted between the results from ID NOW using OPSs and those from ID NOW using O + NSs.

Discussion

In our large, multicentre, population-based study, we observed differing ID NOW sensitivities, compared with RT-PCR, based on the population being tested. The sensitivity was the highest (92.5%) when tested among symptomatic individuals presenting to community COVID-19 assessment centres and the lowest for asymptomatic individuals associated with community outbreaks (73.9%). These results highlight that the ID NOW performance is not standard across the populations tested, likely because of its poorer performance with lower viral loads, as can be observed among asymptomatic individuals or individuals who are further out from their symptom onset [5]. These results can direct where ID NOW device is the most suitable.

The ID NOW sensitivity was slightly lower in the symptomatic populations tested using a mobile service or an in-hospital laboratory. There may be various factors that account for this. First, ID NOW testing is performed immediately after sample collection at assessment centres, whereas hospitals and mobile services often require transportation to the on-site laboratory or mobile unit before ID NOW testing. Although ID NOW testing was mandated to be performed within 1 hour from the collection, short periods of time from transportation may potentially affect performance [10]. Second, a higher proportion of individuals with lower viral loads may be tested in hospitals and via mobile testing. For mobile testing, individuals with symptoms were possibly early in their symptom onset at the time of testing, whereas those presenting to hospitals were possibly late in their symptom onset, both of which can be associated with higher Ct values and, therefore, lower viral loads [5]. Symptomatic individuals presenting to assessment centres, in comparison, are often within the first few days of symptom onset, but because it generally takes approximately 24 hours to identify the need to arrange a booking, are not at the very beginning of their symptom onset.

There are clear differences in ID NOW sensitivity with respect to symptoms, with asymptomatic populations having an approximately 10% decrease in sensitivity. There were significantly higher mean E gene Ct values observed when testing asymptomatic versus symptomatic individuals in assessment centres (Table S8). This has important implications when using the ID NOW for screening of asymptomatic populations, where the risks of a substantial decrease in sensitivity compared to other testing methods (RT-PCR) need to be considered. However, the sensitivity of the ID NOW compared with other point-of-care options, including rapid antigen tests, is far superior, with the sensitivity of rapid antigen tests ranging from 30–50% in asymptomatic populations [4].

The ID NOW sensitivity in our study is underestimated because of the large number (1822; 11.3% of positives) of ID NOW–positive results that were excluded from our analysis, 93% of which came from symptomatic individuals from assessment centres. These were excluded because of the lack of parallel RT-PCR testing. If these excluded samples were included as true positives, the sensitivity of symptomatic individuals presenting to assessment centres would increase to 93.5% (95% CI, 93.0–93.9%).

The ID NOW specificity was high in all populations tested (>99%) and is consistent with that reported in the literature [4]. There are, however, some factors within our study that might have impacted the specificity. When comparing ID NOW with RT-PCR from individuals with confirmed COVID-19, we previously demonstrated many ID NOW–positive, RT-PCR–negative discrepant results [10]. This suggests that other factors may account for the discrepancy, such as sampling error. It is noted that multiple individuals within our dataset had an ID NOW–positive result followed by subsequent negative RT-PCR result, but then on repeat RT-PCR, the next day the patient was found to be positive. These patients were considered ID NOW false positives in our study, although in reality would be considered true positives based on this information, thus increasing the ID NOW specificity. With that being said, our calculated specificity may have decreased if we included all excluded ID NOW–positive samples, as it is unknown how many of those may have been false positives.

As expected, PPV and NPV were the highest among the populations with the highest and lowest COVID-19 prevalence, respectively. PPV was the highest for symptomatic patients in mobile units, hospitals, and assessment centres. NPV was the lowest for asymptomatic patients in mobile units and symptomatic patients in emergency shelters and addiction treatment facilities. COVID-19 prevalence was significantly higher for asymptomatic patients presenting at assessment centres compared with asymptomatic patients from mobile units because asymptomatic testing at the assessment centres was restricted to close contacts. PPV was lower than expected for symptomatic patients at emergency shelters and addiction treatment facilities, which we suspect is due to various reasons. In these settings, it can be challenging to differentiate COVID-19 symptoms from common symptoms related to inadequate housing or drug use, such as rhinorrhea from frequent exposure to the outdoors or coughing from smoking. For similar reasons, it can also be difficult to determine the duration of symptoms, possibly leading to people with symptoms for >7 days to be tested. In addition, it was difficult to control for other factors such as eating or smoking immediately before or during swab collection.

Using an OPS or O + NS for ID NOW testing yielded similar results among both symptomatic and asymptomatic populations, regardless of the swab used for reference RT-PCR. Given the higher inconvenience with O + NS and lack of advantages demonstrated over typical specimens, we recommend against using O + NS for ID NOW COVID-19 testing.

The strengths of this study include a large sample size, stratifying ID NOW performance among various population groups, and incorporating data from real-world sources.

Our study had several limitations. Because of the heterogeneity of our tested populations, it is difficult to exclude the confounders that may have contributed to the ID NOW performance. No statistically significant differences were observed on the basis of age, sex, and VoC detected (Tables S2 and S3). There was, however, an overall trend towards improved sensitivity and specificity when OPSs were used for RT-PCR, particularly when stratifying among the various populations tested (Table S6). The assays used for reference RT-PCR also varied widely (Table S1).

Furthermore, the exclusion of ID NOW–positive results from our study, because of no parallel RT-PCR testing, might have impacted the ID NOW performance. The reasons behind missing parallel RT-PCR are multifactorial and include samples lost or discarded before testing, testing sites going against the guidelines and not obtaining a second swab for RT-PCR confirmation, and patient demographic mismatches resulting in test cancellation or inability to match ID NOW and RT-PCR tests together in our electronic database.

Lastly, we were unable to assess the ID NOW performance concerning the duration of symptom onset, as these data were not captured in our analysis. However, this variability in symptom onset was minimized by excluding patients whose symptom onset was >7 days.

In conclusion, the performance of the ID NOW differs across populations tested for SARS-CoV-2. The ID NOW performance can be maximized if testing is limited to symptomatic individuals presenting at community testing sites (assessment centres). The sensitivity drops by approximately 10% when the testing population is asymptomatic. The ID NOW performance is not affected by age, sex, or VoC (Alpha, Delta). ID NOW testing with O + NS had no effect on the performance.

Author contributions

WS contributed to the conceptualization, methodology, formal analysis, writing of the original draft, supervision, and visualization of the study. AAV and BMB contributed to the conceptualization, methodology, supervision, and review/editing of the manuscript. EB contributed to the data curation and review/editing of the manuscript. GT contributed to the conceptualization, supervision, and review/editing of the manuscript.

Transparency declaration

Supported by internal funding from Alberta Precision Laboratories and Alberta Health Services. Test kits and instruments were paid for by the Public Health Agency of Canada. The manufacturer had no role to play in the study. The authors declare that they have no conflicts of interest.

Editor: L. Kaiser

Appendix A. Supplementary data

The following is the Supplementary data to this article: