The PRONTO study: Clinical performance of ID NOW in individuals with compatible SARS-CoV-2 symptoms in walk-in centres—accelerated turnaround time for contact tracing

Abstract

Background

This PRONTO study investigated the clinical performance of the Abbott ID NOW^TM (IDN) COVID-19 diagnostic assay used at point of care and its impact on turnaround time for divulgation of test results.

Methods

Prospective study conducted from December 2020 to February 2021 in acute symptomatic participants presenting in three walk-in centres in the province of Québec.

Results

Valid paired samples were obtained from 2,372 participants. A positive result on either the IDN or the standard-of-care nucleic acid amplification test (SOC-NAAT) was obtained in 423 participants (prevalence of 17.8%). Overall sensitivity of IDN and SOC-NAAT were 96.4% (95% CI: 94.2–98.0%) and 99.1% (95% CI: 97.6–99.8), respectively; negative predictive values were 99.2% (95% CI: 98.7–99.6%) and 99.8% (95% CI: 99.5–100%), respectively. Turnaround time for positive results was significantly faster on IDN.

Conclusion

In our experience, IDN use in symptomatic individuals in walk-in centres is a reliable sensitive alternative to SOC-NAAT without the need for subsequent confirmation of negative results. Such deployment can accelerate contact tracing, reduce the burden on laboratories and increase access to testing.

Introduction

Currently, the most reliable methodologies for coronavirus disease 2019 (COVID-19) testing are standard laboratory-based nucleic acid amplification tests (NAAT). However, over the first waves of the pandemic, reagent shortages and high demand have challenged our public health capacity and reactivity (1–4). The long turnaround time (TAT) required to produce a test result has also compromised search and contact tracing strategies (5–7). Stand alone rapid tests in specific settings are expected to accelerate case and contact tracing, along with improving public health actions (8–10).

The Abbott ID NOW^TM (IDN) COVID-19 assay, an isothermal NAAT targeting a RdRp segment of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), was granted Health Canada emergency use authorization on September 30, 2020. It is authorized as a lab-based and point-of-care diagnostic assay for the detection of SARS-CoV-2 in individuals with COVID-19 symptoms for fewer than or equal to seven days at time of testing. Early published studies established a lower analytical sensitivity compared with many laboratory-based NAAT assays (11–15). According to the product insert, negative results are to be treated as presumptive and be confirmed with a cleared NAAT. The Canadian Public Health Laboratory Network and the Canadian Society of Clinical Chemist subsequently recommended certain clinical use scenarios to balance expected limited sensitivity with other considerations (16).

Published literature demonstrated that the clinical sensitivity of IDN was linked to corresponding viral loads, with false negative results tending to occur when the standard laboratory-based NAAT cycle thresholds (Ct) are 32 or higher, reflecting lower viral loads (12,13,17). As shown by others, the highest viral loads were found in symptomatic participants presenting in community walk-in centres (9–11). The present study aimed to assess whether IDN could be used as a reliable stand-alone test (without subsequent confirmation) as a means to intervene more quickly on transmission chains, relieve laboratory human and material resources and give more autonomy to front-line healthcare providers. As such, we are reporting the agreement and clinical performance of the IDN, compared to a standard-of-care NAAT (SOC-NAAT) assay, among prospectively recruited symptomatic individuals presenting in community walk-in centres in the province of Québec, Canada.

Methods

In December 2020, IDN instruments were implemented in three walk-in centres in the province of Québec. Volunteer participants were asked to confirm that symptom onset was fewer than or equal to seven days prior to testing and to provide two samples simultaneously, as detailed in Table 1.

Table 1. Characteristics of the participating centres: Type of clinic, sampling and testing methodologies.

Characteristics	Québec City and Montréal	Lévis
Type of centre	Walk-in clinic	Drive-thru clinica
SOC-NAAT sampling	ONPS	Gargle ONPS (when gargle not feasible)
SOC-NAAT method	Laboratory-developed PCR	AllplexTM 2019-nCoV (Seegene) direct PCR
Sampling sequence	SOC-NAAT followed by IDN	IDN followed by SOC-NAAT
IDN sampling	OBNS	OBNS

Abbreviations: IDN, ID NOW^TM; OBNS, oropharyngeal and bilateral nasal swab; ONPS, oral and nasopharyngeal swab; PCR, polymerase chain reaction; SOC-NAAT, standard of care-nucleic acid amplification testing

^a For text simplification, all three centres were considered as walk-in clinics

The oropharyngeal and bilateral nasal swab (OBNS) for the IDN assay was collected with the foam swab provided with the Abbott ID NOW COVID-19 kit as follows: after swabbing the posterior pharynx, tonsils and other inflamed areas for a few seconds each, the swab was inserted in one nostril until a resistance was met at the level of the turbinates (approximatively 2 cm), rotated five times against the nasal wall and slowly removed from the nostril; the same swab was then used for the other nostril. The OBNS for IDN was collected after the oral and nasopharyngeal swab (ONPS) for SOC-NAAT in Québec City and Montréal (18), but performed prior to the gargle for SOC-NAAT in Lévis (19), since the gargle procedure could dilute any virus present when swabbing for IDN.

The IDN test was performed on-site, within one hour of collection, by professionals from diverse training and experience backgrounds who were trained by our teams on using the IDN instrument as per the package insert.

The SOC-NAAT in Montréal (Hôpital Maisonneuve-Rosemont; HMR) and Québec City (CHU de Québec) was a real-time polymerase chain reaction (PCR) assay targeting the structural protein envelope E gene (18,20). Inactivation and thermal lysis, rather than chemical extraction, were performed prior to PCR testing, as previously described (18). The SOC-NAAT in Lévis (Centre intégré de santé et de services sociaux [CISSS] de Chaudière-Appalaches) was based on Seegene Allplex^TM technology as previously described (19).

No personal data were collected outside of the information available on the standard COVID-19 laboratory form (gender, age, duration of symptoms, COVID-19 contact history). The duration of symptoms and contact history, combined with supplemental NAAT when applicable, were used to classify infection stages of participants for whom discordant results were obtained. Acute infection was defined as at least having one symptom among fever, cough, runny nose, dyspnea, sore throat, anosmia and ageusia, or a combination of two of the following: headache, fatigue, muscle pain, anorexia, nausea or vomiting, abdominal cramps or diarrhea within seven days of onset. When the collected data revealed misclassification, erroneous data collected by staff or by participant mistake, the case remained included in the study since representing a real-life situation.

For each study site, TAT was defined as the time between sample collection and the availability of the laboratory report for concordantly positive pairs (both the IDN and the SOC-NAAT results were reported). In Lévis, the time between sample collection and completion of public health questionnaire with the case and household contacts was also calculated. The TAT for negative results was not monitored since negative IDN results were not reported during the study period.

This PRONTO study was undertaken in the midst of the second wave of the COVID-19 in Québec, with thousands of samples being received on a daily basis. There was a context of emergency (with public, administrative and media pressure) to implement rapid testing. Formal Ethical Review Board approval was lifted since the study was mandated by the directeur national de santé publique as part of the Public Health response during the sanitary emergency state. Explicit verbal consent was obtained from all participants after receiving a verbal description of the project.

Statistical analysis

Samples producing invalid results in either arm were excluded from the calculations.

Data were analyzed using a contingency table. In the absence of a gold standard for SARS-CoV-2 ribonucleic acid (RNA) detection, the reference method used for positive percent agreement and negative percent agreement was the SOC-NAAT. In addition to computing the overall rates of agreement, the level of agreement was assessed using kappa statistics (STATA V16.1). By definition, kappa values above 0.75 indicate excellent agreement, values between 0.40 and 0.75 indicate fair to good agreement, and values below 0.40 represent poor agreement beyond chance (21). To evaluate the clinical sensitivity and negative predictive value of IDN and SOC-NAAT, a participant was considered infected if at least one result from the paired samples was positive, assuming 100% specificity of both assays. The 95% confidence intervals (95% CI) were obtained with STATA V16.1.

Outcomes

Between December 6 and February 22, 2020, paired samples were obtained from 2,395 individuals. After exclusion of 23 pairs associated with an invalid result with either method, the performance analysis was based on 2,372 participants (Table 2).

Table 2. Participant characteristics and number of valid pairs included (N=2,395).

Participant characteristics	Québec City		Lévis		Montréal		Total
	n	%	n	%	n	%	n	%
Symptomatic participants recruited	1,246	N/A	790	N/A	359	N/A	2,395	N/A
Invalid results	12	1.0	9	1.1	2	0.6	23a	1.0
Valid paired samples	1,234	99.0	781	98.9	357	99.4	2,372	99.0
Male gender	544	44.1	370	47.4	154	43.1	1,068	45.0
Mean age	40	N/A	32	N/A	38	N/A	37	N/A
Age range (years)	1–88	N/A	1–83	N/A	1–80	N/A	1–88	N/A
Younger than 18 years of age	118	9.6	109	14.0	33	9.2	260	11.0

Abbreviation: N/A, not applicable

^a Among the 23 excluded pairs, 22 invalid results were obtained with Abbott ID NOW^TM and one with standard-of-care nucleic acid amplification test

As shown in Table 3, a total of 423 participants (17.8%) were considered infected (at least one positive result by IDN or SOC-NAAT). Positive concordant results were obtained on 404 pairs (95.5%); among the 19 discordant pairs, four were positive with IDN only and 15 with SOC-NAAT only. Agreement was excellent, as reflected by a kappa coefficient value of 0.97. Overall, IDN sensitivity and negative predictive value were respectively estimated at 96.4% (95% CI 94.2–98.0) and 99.2% (95% CI 98.7–99.6), with little (not statistically significant) variation across centres (Table 4).

Table 3. Prevalence of SARS-CoV-2 infection and distribution of Abbott ID NOW^TM and standard-of-care nucleic acid amplification test results in symptomatic individuals (n=2,372).

Location	Prevalencea		Results
	n/N	%	IDN	SOC-NAAT
				POS	NEG
Québec City	193/1,234	15.6	POS	187	2
			NEG	4	1,041
Lévis	114/781	14.6	POS	109	1
			NEG	4	667
Montréal	116/357	32.5	POS	108	1
			NEG	7	241
Total	423/2,372	17.8	POS	404	4
			NEG	15	1,949

Abbreviations: IDN, ID NOW^TM; NEG, negative; POS, positive; SOC-NAAT, standard of care-nucleic acid amplification testing

^a A participant was considered infected if at least one result from the paired samples was positive, assuming 100% specificity of IDN and SOC-NAAT

Table 4. Agreement between Abbott ID NOW^TM and standard-of-care nucleic acid amplification testing results and clinical performance (n=2,372).

Test	Statistics	Assessment center
		Québec City	Lévis	Montréal	Total
Agreement
PPAa	%	98.9	99.1	99.1	99.0
	95% CI	96.2–99.9	95.0–100	95.0–100	97.5–99.7
NPAa	%	99.6	99.4	97.2	99.2
	95% CI	99.0–100	98.5–99.8	94.3–98.9	98.7–99.6
ORA	%	99.5	99.4	97.8	99.2
	95% CI	98.9–99.8	98.5–99.8	95.6–99.0	98.8–99.5
Cohen’s kappa	Κ	0.98	0.97	0.95	0.97
	95% CI	0.97–1.00	0.95–1.00	0.91–0.98	0.96–0.98
Clinical performanceb
IDN sensitivity	%	97.9	96.5	94.0	96.4
	95% CI	94.8–99.4	91.3–99.0	88.0–97.5	94.2–98.0
SOC-NAAT sensitivity	%	99.0	99.1	99.1	99.1
	95% CI	96.3–99.9	95.2–100	95.3–100	97.6–99.7
IDN NPV	%	99.6	99.4	97.1	99.2
	95% CI	99.0–99.9	98.5–99.8	94.1–98.8	98.7–99.6
SOC-NAAT NPV	%	99.8	99.9	99.6	99.8
	95% CI	99.3–100	99.2–100	97.7–100	99.5–100

Abbreviations: CI, Confidence Interval; IDN, ID NOW^TM; NPA, negative percent agreement; NPV, negative predictive value; ORA, overall rates of agreement; PPA, positive percent agreement; SOC-NAAT, standard of care-nucleic acid amplification test

^a PPA and NPA were computed by considering the SOC-NAAT as the reference method

^b A participant was considered infected if at least one result from the paired samples was positive, assuming 100% specificity of IDN and SOC-NAAT

Characteristics of the 19 participants for whom discordant results were obtained are presented in Table 5. For the 15 negative IDN, the mean Ct value of the corresponding positive SOC-NAAT was 33.5 (range 30.9–35.0). The mean Ct values for the concordantly positive pairs, available for the Québec City site (26.0) and the Montréal site (23.5), were clearly lower, reflecting a higher viral load. Among the 15 participants for whom the discordant profile was SOC-NAAT positive/IDN negative, two were asymptomatic, four were considered as late presentation and nine as acutely infected. Among the four participants for whom the discordant profile was SOC-NAAT negative/IDN positive, two had an acute infection and two could not be staged nor confirmed by supplementary testing.

Table 5. Laboratory and clinical information of participants in whom discrepant results were obtained (n=19).

Assessment center	SOC-NAATa Ct value	Symptoms durationb,c	Contact with a known caseb	Supplementary testingd	Clinical stage
IDN negative and SOC-NAAT positive (IDN false negative), n=15
Québec City	34.2	Symptoms resolved 6 days earlier	Unknown	Initial SOC-NAAT sample retested after chemical extraction: positive result with Ct value of 33.4 Resampled 72 hours later and tested by IDN and SOC-NAAT with a Ct value of 35	Late presentatione (post-symptomatic)
	34.8	N/A	Yes, but not detailed	Initial SOC-NAAT sample retested after chemical extraction: positive result with Ct value of 32.4	Asymptomatic
	34.0	Less than 24 hours	Unknown	Initial SOC-NAAT sample retested after chemical extraction: positive result with Ct value of 32.9	Acute presentation
	31.5	More than 7 days	Unknown	ND	Late presentatione
Lévis	34.0 (2/3 genes)	N/A	Yes, but not detailed	Resampled 2 days later: negative on IDN and SOC-NAAT IDN swabf retested by two other assaysf: negative results	Asymptomatic
	32.0 (3/3 genes)	2 days	Home	ND	Acute presentation
	30.9 (3/3 genes)	1 day	Workplace	IDN swabf retested by two other assaysg: weakly positive with one assay	Acute presentation
	34.4 (3/3 genes)	1 day	Home	IDN swabf retested by two other assaysg: weakly positive with one assay	Acute presentation
Montréal	34.2	More than 7 days	Home	ND	Late presentatione
	33.5	1 day	Workplace	ND	Acute presentation
	31.6	3 days	Home	ND	Acute presentation
	35.0	7 days	Unknown	ND	Late presentatione
	34.2	2 days	No	ND	Acute presentation
	34.9	4 days	Unknown	ND	Acute presentation
	33.3	Less than 24 hours	School	Initial SOC-NAAT sample retested after chemical extraction: positive with Ct value of 33.7	Acute presentation
IDN positive and SOC-NAAT negative (SOC-NAAT false negative), n=4
Québec City	N/A	2 hours	School	IDN swabf tested by NAAT after chemical extraction: positive result with a Ct value of 25.5 Initial SOC-NAAT sample retested after chemical extraction: positive result with a Ct value of 33.8	Acute presentation
		Unknown	Unknown	IDN swabf tested by NAAT after chemical extraction: positive result with a Ct value of 30.8 Initial SOC-NAAT sample retested after chemical extraction: positive result with a Ct value of 35.2	Unknown
Lévis		1 day	Unknown	IDN swabf tested by two other assays: negative results Initial SOC-NAAT sample retested by two commercial assaysg: negative results	Acute presentation; possible false-positive IDN
Montréal		5 days	Home	ND	Acute presentation vs. possible false-positive IDN

Abbreviations: Ct, cycle threshold; IDN, ID NOW^TM; N/A, not applicable; ND, not done; SOC-NAAT, standard of care-nucleic acid amplification test

^a In Québec City and Montréal, the SOC-NAAT was a laboratory-developed test targeting the E gene. In Lévis, the Allplex™ 2019-nCoV assay (Seegene) includes three gene targets (E, RdRp and N); the Ct values shown are the mean of the two or three positive results obtained

^b The duration of symptoms before testing and COVID-19 contact history were obtained through the standard routine questionnaire form. Missing information occurs frequently

^c Some individuals were included in this study based on the assertion that they were symptomatic. The questionnaire form—revised only for discordant pairs—revealed that some participants were asymptomatic. It was decided not to exclude the latter a posteriori

^d The alternate NAAT was the laboratory-developed test preceded by chemical RNA extraction using the NucliSens easyMAG platform (bioMérieux; Saint-Laurent, Canada)

^e Presentation was considered late when symptoms started more than seven days before sampling as IDN is currently Health Canada-approved for participants tested within the first seven days of symptoms

^f In Québec City and Lévis, after elution in the IDN Sample Receiver buffer, the swab sample was transported into a dry 15 mL Falcon tube and frozen for possible subsequent testing by NAAT to resolve discrepancies between IDN and SOC-NAAT results or for retesting of the SOC-NAAT sample with a more sensitive laboratory platform

^g Simplexa COVID-19 (DiaSorin) and FilmArray RP 2.0 (bioMérieux)

The TAT between sampling and availability of laboratory report of positive results was on average 20.1 hours for SOC-NAAT and 1.2 hours for IDN. In Lévis, TAT between sampling and end of public health tracing was on average 36.0 hours for the symptomatic individuals who either had SOC-NAAT positive/IDN negative results or did not participate in this study but were assessed at the same drive-through clinic during the same period, and for whom testing was performed by SOC-NAAT (n=283); it was 13.6 hours for the 110 participants for whom the IDN was positive, representing a difference of 22.4 hours (95% CI 18.8–26.1, p<0.0001).

Discussion

In this PRONTO study, the clinical performance of IDN was compared to SOC-NAAT among a large number of symptomatic individuals in community-based walk-in centres. Agreement between the two testing strategies was nearly perfect. Although the sensitivity of IDN (96.4%) was slightly lower than for SOC-NAAT (99.1%), the difference was not statistically significant. Very few false negative results were observed in both arms, resulting in excellent negative predictive value of 99.5% and 99.8% for IDN and SOC-NAAT, respectively. Thus, our results differ from earlier studies that demonstrated lower sensitivity (55%–84%) (22,23). Some recent studies suggest a better performance (86%–100%), although the 95% CI in these latter studies were wider, due to a smaller sample size (22–28). This discrepancy in sensitivity might be explained by variation in pre-test probability in the target population (29) and by our optimized swabbing methodology (30). The current study was performed in a group with probable higher viral titers and higher pre-test probability, during a high prevalence wave. A multi-compartment swabbing protocol was also used herein, which included three throat areas and both nostrils, which has been previously shown to be a sensitive alternative to nasopharyngeal swabbing (31). Another possible explanation is that the SOC-NAAT comparators used in our study are associated with lower analytical sensitivity than other commercial NAATs currently used for the detection of SARS-CoV-2 (18). Indeed, at the Montréal site (data not shown), during the same period, 127 similar individuals (with COVID-19 compatible symptoms) had their ONPS tested by a commercial NAAT: 38 had concordant positive results; 85 had concordant negative results; and four had negative IDN but positive commercial NAAT results (sensitivity of the IDN 90.5%; 95% CI 77.4–97.3).

The discrepant pairs were classified according to their probable clinical stage since later infections with higher Ct values might not represent contagiousness (32–34). We presumed, as a hypothesis for our study, that false negative results would be associated with a lower viral load, with the infected individual being less infectious. Although the timing of the test is important to monitor dynamic viral load, our data confirmed discordant results to be associated with higher Ct, an indirect indicator of viral load (35,36).

The risk of not detecting all cases (or risk of false negative results) can be mitigated by appropriate counselling: automated messages sent with negative results invite people to get retested and seek medical attention if symptoms do not resolve by themselves after 48 hours (37,38). It could also be counterbalanced by the timeliness of the results and the possibility of increasing access to testing by increasing overall laboratory capacity. Although lower IDN sensitivity and missed cases could be deemed obstacles for promoting the technology, we believe otherwise, especially in the context of high vaccination uptake. Clinical sensitivity of a strategy should include analytical sensitivity but also TAT and access to testing. IDN use accelerated contact tracing, and we feel it increased access to testing by offering a less intrusive OBNS sampling and by delocalizing to the point-of-care. In fact, a Québec survey poll showed that half of the eligible population with COVID-19 compatible symptoms did not get tested during the study period (39). Rapid testing or more comfortable sampling methods could represent a valuable solution (18,19).

The optimal approach for the diagnosis of COVID-19 remains under debate. Some experts focus on test sensitivity and neglect the public health and population impacts of accelerated contact tracing (7,8). Although SOC-NAAT processes are now optimised for high testing volume, laboratory resources are profoundly stretched, particularly with the return to “normal” of healthcare activities. An attractive scenario would be to supply IDN directly to first-line clinics, with clear guidance on whom to test with this strategy (for example, symptomatic individuals and close contacts of positive cases). Cost-effective analysis should be undertaken to better guide Canadian public health specialists, microbiologists, administrators and clinicians.

In our study, results were available faster if samples were tested with IDN vs. SOC-NAAT in all assessment centres, with a faster public health inquiry in Lévis for IDN compared to SOC-NAAT. Although representing different indicators, both are proxies for public health intervention, and congruent in showing a net advantage for IDN. Current public health recommendations are that people with COVID-19 symptoms (and their household contacts in certain high-prevalence regions) should self-isolate from the onset of symptoms. However, no interventions have been made to possible contacts until symptomatic participants have a confirmed diagnosis of COVID-19. Without rapid results, public health loses a valuable window of opportunity, particularly if these contacts do not express a typical disease presentation. We can also postulate that adherence to self-isolation is increased when the diagnosis is confirmed.

Strengths and limitations

Among all the similar studies published to date, this PRONTO study has the largest number of participants, even exceeding the total number of participants included in the systematic review by Tu et al. (24). Being a multi-site study and performed in a real-life setting (e.g. the personnel performing the IDN testing stemmed from diverse training and experience backgrounds), external validity is increased. We were able to collect comparative data as part of the implementation process in overwhelmed walk-in centres and laboratories. We also aimed to document, in two of the sites, the impact of rapid testing on public health. Although a cause-and-effect relationship between IDN use and the impact on transmission to contacts cannot be established, we postulate that faster tracing will benefit public health containment strategies (9,10).

Our study has certain limitations. First, SOC-NAAT differed between laboratories, although adhered to the same validation panels provided by the provincial Public Health Laboratory. Second, very little participant-level data were collected from participating institutions. As such, IDN could not be correlated with the indications for testing, the appropriateness of the test, and the clinical evolution of participants with positive test results. Third, differences in practices within and between walk-in centres (for example different personnel, rapidly changing recommendations over time) may represent confounding variables; for example, by including some asymptomatic participants. Fourth, our diagnostic definition (at least one positive result from the paired samples), which implies 100% specificity of both assays, may have lead to slight overestimation of the sensitivity for both assays. While false positive IDN results are considered unlikely (28) compared with the well described false positive laboratory PCR results (40), we suspect two false positive results in our study (Table 5), and we witnessed some infrequent confirmed false positive IDN results in routine care after the end of the study.

Conclusion

Based on our large experience, IDN use in walk-in centres with an optimized sampling method in acute symptomatic participants can be achieved safely without the need for laboratory confirmation of negative results. In this context, IDN can be considered a stand-alone testing option. Such deployment accelerates contact tracing of positive cases and reduces the burden on laboratories, while increasing access to testing.