Abstract
Objective
We sought to evaluate the test characteristics of Abbott ID‐Now as a screening tool compared to polymerase chain reaction (PCR) testing for identification of COVID in an asymptomatic emergency department population.
Methods
We performed a prospective study enrolling a convenience sample of asymptomatic patients presenting to a single academic emergency department (ED) who received simultaneous testing with ID‐Now and PCR per standardized ED protocols. Sensitivity, specificity, and positive and negative predictive value (PPV, NPV) of ID‐Now were calculated compared to PCR. Stratified analysis by cycle threshold (Ct) values was also performed, defined as high viral load (Ct < 33) and low viral load (Ct ≥ 33).
Results
A total of 3121 patients were enrolled, of whom 2895 had valid results for ID‐Now and PCR. COVID prevalence was 2.6%. ID‐Now had a sensitivity of 85.1% (95% CI 75.9% to 92.7%) and a specificity of 99.7% (99.5% to 99.9%). PPV and NPV were high at 87.5% (83.1% to 96.1%) and 99.6% (99.3% to 99.8%). Stratified analysis by low and high Ct values demonstrated reduction in sensitivity in patients with low viral loads: 91.7% (81.6% to 97.2%) in low Ct value patients versus 58.3% (27.7% to 84.8%) in high Ct value patients.
Conclusions
ID‐Now had excellent performance in asymptomatic ED patients with a low rate of false positives. Cycle threshold analysis suggests a relationship between viral load and ID‐Now sensitivity. Given its speed and performance in this population, ID‐Now should be considered an excellent tool to support clinical decision‐making in ED populations.
Keywords: COVID‐19, diagnostic testing, point‐of‐care testing, triage, validation
1. INTRODUCTION
1.1. Background
The emergence and spread of severe acute respiratory syndrome coronavirus‐2 (COVID) represent a challenge for clinical laboratories and emergency departments, where timely and reliable diagnostic information is crucial for safe and high‐quality care. To augment testing capabilities, a number of diagnostic tests have been approved under Emergency Use Authorization by the Food and Drug Administration. The current standard of care testing involves reverse transcription‐polymerase chain reaction (RT‐PCR) and although improvements have been made in availability and speed of COVID testing, the turnaround time of 4 hours or more is too long for timely decision support in a variety of clinical situations in the ED, including patient cohorting in shared spaces, periprocedural testing for emergent operations, and the timely discharge of patients to congregate living settings. The Abbott ID‐Now COVID assay uses isothermal nucleic acid amplification of the RNA polymerase viral target with a manufacturer published limit of detection of 125 genome equivalents/ml. It is designed as a point‐of‐care test (POCT) with a turnaround time of 5–13 minutes and can be performed by testing operators deemed competent after a brief training.
1.2. Importance
Use of the ID‐Now offers an important opportunity for timely diagnosis, with the potential to mitigate infectious risk to others by identifying and isolating asymptomatic patients from shared waiting and treatment areas. A few studies have examined the performance of ID‐Now compared to standard‐of‐care testing modalities and reported sensitivities from 75% to 93%. 1 , 2 , 3 , 4 However, existing literature focuses on symptomatic patients with clinical concern for COVID infection and uses swabs in viral transport media, whereas the currently approved protocol for the ID‐Now is a dry nasal swab. A recent study also raised concerns about the sensitivity of the ID‐Now for patients with low viral loads; 33%–45% of patients with low viral loads were considered falsely negative. 5
Thus, reevaluation of the ID‐Now in an asymptomatic patient population, using the manufacturer‐indicated dry nasal swab, was performed to improve our understanding of this assay and justify its use in clinical settings, particularly in the ED.
1.3. Goals of this investigation
The objective of this study was to determine the diagnostic performance of the ID‐Now COVID‐19 assay for the evaluation of asymptomatic ED patients compared to the current gold standard RT‐PCR testing. Specifically, we determined the sensitivity, specificity, likelihood ratios (LR), and positive and negative predictive values (PPV, NPV) of the ID‐Now. We also determined these metrics in those with low and high viral loads as defined by cycle threshold (Ct) values above and below 33.
2. METHODS
2.1. Study design
The Bottom Line.
With the advent of new technology, isothermal nucleic acid amplification for ribonucleic acid (RNA) polymerase, detection of SARS‐CoV‐2 infection can be done at the bedside. This study looked at performance of the Abbott ID‐Now in 2895 asymptomatic adult emergency department patients and, compared to routine lab reverse transcription‐polymerase chain reaction, showed the test had a sensitivity of 85% (95% CI 76% to 93%) and specificity of 99.7% (99.5% to 99.9%).
We conducted a retrospective study of the ID‐Now in patients presenting to a single ED at a large, tertiary care hospital. We collected data from our electronic medical record via convenience sampling during a 5‐month period from July 23, 2020 to December 14, 2020. During this time period, citywide mask mandates and closure of non‐essential businesses were in place, and the overall prevalence of COVID infection in the city was low. Our study was approved by institutional review board and we adhered to Standards for Reporting Diagnostic accuracy studies (STARD) reporting guidelines. The study took place at University of California at San Francisco (UCSF) Medical Center, an urban academic ED caring for approximately 45,000 patients annually.
2.2. Selection of participants
Adult participants (over age 18 years) were all patients presenting to the ED during the specified dates who were tested with both ID‐Now and RT‐PCR as part of standard ED protocol. According to this protocol, the triage nurse or physician screened all patients for COVID symptoms (defined as fever, respiratory symptoms, diarrhea, or myalgias) or for a known COVID‐positive contact. Those without COVID symptoms or a known COVID‐positive contact were determined to be low risk. Patients with low concern for COVID infection and those meeting the following criteria received both ID‐Now and RT‐PCR testing, with swabbing performed simultaneously: patients triaged to shared rooms (including the observation unit or hallway beds), patients unable to self‐isolate (congregate living, experiencing homelessness, dialysis) and patients whose estimated waiting room time exceeded 1 hour. We excluded patients who did not obtain both tests.
2.3. Test methods
ID‐Now (Index test) was performed by ED nurses or patient care technicians (PCTs) who had completed standardized training in accordance with manufacturer recommendations. Samples were collected via mid‐turbinate dry nasal swab and tested immediately using the ID‐Now instrument, located in the ED.
RT‐PCR (reference test) was performed by ED nurses or PCTs who used nasopharyngeal swabs placed in viral transport medium. Samples were processed at 1 of 2 USCF clinical laboratories using one of two RT‐PCR platforms, Diasorin Simplexa or Genmark ePlex assays, considered equivalent for the purposes of clinical practice and this analysis. Samples were processed for RT‐PCR testing using standard procedures and reported in the patient record as the reference test result.
For each test, three outcomes were possible: positive, negative, or indeterminate. Patients with indeterminate ID‐Now results underwent repeat testing after cleaning the machine, and if indeterminate a second time were reported as such—all POCT indeterminate results underwent confirmatory RT‐PCR testing. Those performing the ID‐Now index test had no knowledge of the patient's RT‐PCR result, and staff performing RT‐PCR testing did not have access to results of POCT.
2.4. Analysis
Diagnostic accuracy of the ID‐Now index text was determined by calculating sensitivity, specificity, PPV, NPV, and LRs, along with 95% confidence intervals (CIs), using RT‐PCR as the reference standard for COVID infection. The MedCalc statistical calculator was used for all test statistic calculations. 6
In a secondary analysis, chart review was conducted on all patients with discordant results (RT‐PCR positive and ID‐Now negative or vice versa) to evaluate test characteristics of ID‐Now using clinical review as gold standard for COVID infection. Chart review was conducted by a single study investigator, who evaluated the electronic medical record of the encounter in which the patient received COVID testing, searching for either a documented note by an infectious disease consultation team who evaluated the patient, or for a discharge summary from the inpatient hospital medicine team. If either the infectious disease consult note or the discharge summary listed COVID as the clinical diagnosis, it was considered a positive gold standard result.
Given concern about performance of ID‐Now in patients with a low viral load, subgroup analysis was performed. Cycle threshold values were obtained from patients with positive RT‐PCR results and categorized into low viral load (defined as Ct ≥ 33) or high viral load (Ct < 33). Selection of 33 as the threshold value was made in consultation with colleagues from the UCSF microbiology laboratories and based on available prior literature. 5 Test characteristics of ID‐Now, including sensitivity, specificity, PPV and NPV, and LRs were recalculated stratified by low versus high viral load.
Before data collection, sample size calculation was performed estimating 70 positive patients necessary to estimate a sensitivity of 80% with a 95% CI 70%–90%. We chose 80% sensitivity based on estimates from prior studies. 3 , 7 As our prevalence of positive participants was low, we estimated that we would have sufficient numbers of negative participants for the specificity calculation. 2 , 4 , 7
3. RESULTS
3.1. Participants
Over the study period, there were 3121 eligible patients, of whom 2895 had valid results for both the index and reference tests—226 patients had POCT results but not RT‐PCR results because of a change in ED protocol during data collection and were excluded. Seventy‐four patients tested positive by RT‐PCR for an overall COVID prevalence of 2.6%. Participant flow per STARD criteria is outlined in Figure 1. Baseline patient characteristics include a mean age of 52.7 (SD 19.7), 47.6% female, and ED disposition including 26.9% admitted, 61.3% discharged, and 1% sent to the operating room, summarized in Table 1.
FIGURE 1.
STARD flowchart of participants receiving ID‐Now and RT‐PCR testing. Abbreviations: RT‐PCR, reverse transcription‐polymerase chain reaction; STARD, Standards for Reporting Diagnostic accuracy studies
TABLE 1.
Baseline patient demographics
| Characteristic | N = 2895 | |
|---|---|---|
| Age (y) (Mean SD) | 52.7 (19.7) | |
| Female (N, %) | 1380 | 47.6 |
| Disposition (N, %) | ||
| Admit | 780 | 26.9% |
| Discharge | 1775 | 61.3% |
| Operating room | 28 | 1.0% |
| Other | 312 | 10.8% |
3.2. Test results
Test characteristics of ID‐Now are summarized in Table 2A. Of the 74 positive RT‐PCR cases, 63 were detected by ID‐Now and 11 were not. Of the 72 patients with positive ID‐Now results, 63 were positive by RT‐PCR and 9 were negative. ID‐Now had an overall sensitivity of 85.1% (95% CI 75.0% to 92.3%) and a specificity of 99.7% (99.4% to 99.9%). The positive predictive value of ID‐Now was 87.5% (78.4% to 93.14%) and the negative predictive value was 99.6% (99.3% to 99.8%). Positive and negative LRs for ID‐Now were 267 (138 to 515) and 0.15 (0.09 to 0.26).
TABLE 2.
Performance of ID‐Now in diagnosis of COVID
| A. Using RT‐PCR as gold standard | |||
|---|---|---|---|
| Disease present | Disease absent | Total | |
| ID‐Now positive | 63 | 9 | 72 |
| ID‐Now negative | 11 | 2812 | 2823 |
| Total | 74 | 2821 | 2895 |
| Value | 95% CL | |
|---|---|---|
| Sensitivity | 85.1% | 75.0% to 92.3% |
| Specificity | 99.7% | 99.4% to 99.9% |
| PPV | 87.5% | 78.4% to 93.1% |
| NPV | 99.6% | 99.3% to 99.8% |
| LR positive | 267 | 138 to 515 |
| LR negative | 0.15 | 1.%2to 0.26 |
| B. Using Clinical Review as gold standard | |||
|---|---|---|---|
| Disease present | Disease absent | Total | |
| ID‐Now positive | 66 | 6 | 72 |
| ID‐Now negative | 11 | 2812 | 2823 |
| Total | 77 | 2818 | 2895 |
| Value | 95% CL | |
|---|---|---|
| Sensitivity | 85.7% | 75.9% to 92.7% |
| Specificity | 99.8% | 99.5% to 99.9% |
| PPV | 91.7% | 83.1% to 96.1% |
| NPV | 99.6% | 99.3% to 99.8% |
| LR positive | 403 | 180 to 900 |
| LR negative | 0.14 | 0.08 to 0.25 |
Abbreviations: CI, confidence interval; LR, likelihood ratio; NPV, negative predictive value; PPV, positive predictive value; RT‐PCR, reverse transcription‐polymerase chain reaction.
3.3. Secondary analysis
Table 2B displays test characteristics using clinical review as the gold standard of COVID diagnosis. Of the 9 patients with positive ID‐Now and negative RT‐PCR, all received a formal infectious disease consultation, and 3 of the 9 were considered true positives—2 patients received COVID treatment. Of the 11 patients with negative ID‐Now and positive RT‐PCR results, all were considered true positives, and 3 received treatment. Test characteristics of ID‐Now using clinical review as the gold standard were similar to those using RT‐PCR as gold standard.
Of the 74 PCR‐positive patients, 67 had Ct values available, of whom only 7 were classified as low viral load (Ct ≥ 33). Analysis of test characteristics stratified by Ct value is summarized in Table 3. Among the PCR‐positive patients with low Ct values, 8.3% were negative by Abbot ID‐Now, compared to 71.4% of patients with high Ct values. Sensitivity differed between the 2 groups, with a sensitivity of 91.7% (81.6% to 97.2%) among low Ct value patients, compared to 58.3% (27.7% to 84.8%) in high Ct value patients. Figure 2 shows the Ct values of all RT‐PCR positive patients, demonstrating the relationship between false‐negative ID‐Now testing and higher Ct values.
TABLE 3.
Test characteristics of ID‐Now, stratified by cycle‐threshold value
| High viral load (Ct < 33) | Low viral load (Ct ≥ 33) | |
|---|---|---|
| n | 60 | 7 |
|
Negative ID‐Now n (%) |
5 (8.3%) | 5 (71.4%) |
|
Sensitivity (95% CI) |
91.67% (81.61% to 97.24%) |
58.33% (27.67% to 84.83%) |
|
Specificity (95% CI) |
99.68% (99.40% to 99.85%) |
99.68% (99.40% to 99.85%) |
|
PPV (95% CI) |
85.94% (76.01% to 92.18%) |
43.75% (25.73% to 63.59%) |
|
NPV (95% CI) |
99.82% (99.59% to 99.92%) |
99.82% (99.65% to 99.91%) |
Abbreviations: CI, confidence interval; NPV, negative predictive value; PPV, positive predictive value.
FIGURE 2.
RT‐PCR positives by cycle threshold value and ID‐Now result. Cycle threshold values for all patients with positive RT‐PCR results are plotted in ascending order. Blue diamonds indicate positive ID‐Now results (considered true positives) and red diamonds indicate negative ID‐Now results (considered false negatives). Abbreviations: Ct, cycle threshold; RT‐PCR, reverse transcription‐polymerase chain reaction
4. LIMITATIONS
Our study has a few important limitations. It is a single‐center study using a convenience sample, which limits its generalizability. The evolving nature of the pandemic meant that ED protocols were not consistent throughout the study period. For example, COVID testing policy changed during the last 3 weeks of data collection to allow ID‐Now testing without confirmatory RT‐PCR, resulting in 226 patients with POCT results without RT‐PCR results. Although the dual‐testing ED protocol was intended for low‐risk, asymptomatic patients, owing to screening errors some patients receiving both tests may have been at risk for COVID infection, which would not have been captured by our analysis. Our study sample had an overall low prevalence of COVID infection at 2.6%, which would result in a lower PPV and higher NPV and may not reflect clinical circumstances with higher rates of asymptomatic infection. A sensitivity analysis evaluating PPV and NPV at varying levels of COVID is available as an addendum.
Misclassification bias may have occurred as the gold standard PCR test has imperfect sensitivity (but excellent specificity). This may have resulted in shifting true positive cases as false negatives. We sought to mitigate the effect of imperfect gold standard by conducting a sensitivity analysis using expert clinical review in discordant results, which revealed cases of false negative determinations by the gold standard PCR. The test characteristics of ID‐Now using clinical review and PCR as the gold standard were similar to those using RT‐PCR alone as gold standard.
5. DISCUSSION
This study demonstrates the performance of ID‐Now compared to RT‐PCR for diagnosis of COVID in a population of asymptomatic ED patients. COVID prevalence during the study period was 2.6%. ID‐Now performed well in this population, with a sensitivity of 85.1% and a specificity of 99.7%. PPV and NPV were high at 87.5% and 99.6%. In chart review of patients with discordant results, all those positive by RT‐PCR were considered true positives, whereas 6 of the ID‐Now positives were considered false positives. Test characteristics demonstrated a notable reduction in sensitivity at low viral loads (91.7% in low Ct value patients, 58.3% in high Ct value patients), though these results are limited by the small sample size of high Ct value patients (n = 7).
To our knowledge, this is the first study of ID‐Now performance in an asymptomatic ED population. The sensitivity and specificity fall within previously reported ranges from prior studies using viral transport medium and in patients in whom COVID infection was suspected, which is reassuring. Previous studies have raised concerns about performance of ID‐Now in patients with low viral loads (presumably less likely to display symptoms), and a reduction in sensitivity was seen in this population as well. Patients shedding low amounts of virus appear to be less infectious, and the benefit of a rapid result with high sensitivity to identify infectious patients is of considerable use in the initial patient management decisions in the ED. 8
Our results suggest that ID‐Now is a reliable tool for COVID screening for asymptomatic patients in the ED setting. Although our Ct analysis suggests a relationship between viral load and sensitivity of ID‐Now, the overall sensitivity of ID‐Now remained high. The poor sensitivity of ID‐Now in the small number of patients with high Ct values suggests more research is needed to evaluate the performance of ID‐Now in patients with low viral loads.
The fact that ID‐Now can be performed with a rapid turnaround time using machines within the ED offers several benefits, most important, cohorting of patients by COVID status. Rapid testing allows more precise risk assessment before aerosolizing procedures in the ED or the operating room. Discharging patients experiencing homelessness or to congregate living situations can also be expedited. Rapid testing within the ED setting has also been associated with less time in isolation and reduced use of personal protective equipment. 8
In summary, rapid and reliable COVID testing allows for safer and more effective care of ED patients. Our study demonstrates that ID‐Now demonstrates excellent sensitivity and specificity in a cohort of ED patients. Given the numerous benefits of rapid turnaround time and strong performance in this population, ID‐Now should be considered an excellent screening tool to support clinical decision‐making in ED populations.
CONFLICT OF INTEREST
The authors have no conflicts of interest to disclose.
AUTHOR CONTRIBUTIONS
Anu Ramachandran, Jeanne Noble, and Ralph Wang conceived and designed the study. Jeanne Noble, Anu Ramachandran, Ralph C Wang, and AD oversaw clinical data collection— Anne Deucher, Steve Miller and Patrick Wai Tang procured microbiological data. Anu Ramachandran performed the statistical analysis with oversight by Jeanne Noble and Ralph C Wang. Anu Ramachandran drafted the manuscript, and all authors contributed substantially to its revision.
Biography
Anu Ramachandran, MD, is an emergency physician at the University of California San Francisco in San Francisco, California.
Addendum: Positive and negative predictive values at different levels of disease prevalence
| Prevalence (%) | PPV | 95% CI | NPV | 95% CI |
|---|---|---|---|---|
| 0.5 | 59.49% | 43.18% to 73.95% | 99.93% | 99.88% to 99.96% |
| 1 | 74.70% | 60.44% to 85.09% | 99.86% | 99.75% to 99.92% |
| 3 | 90.04% | 82.39% to 94.59% | 99.58% | 99.25% to 99.76% |
| 5 | 93.90% | 88.84% to 96.75% | 99.28% | 98.73% to 99.60% |
| 10 | 97.01% | 94.38% to 98.43% | 98.50% | 97.36% to 99.15% |
| 20 | 98.65% | 97.42% to 99.30% | 96.68% | 94.24% to 98.11% |
Abbreviations: CI, confidence interval; NPV, negative predictive value; PPV, positive predictive value.
Ramachandran A, Noble J, Deucher A, Miller S, Tang PW, Wang RC. Performance of Abbott ID‐Now rapid nucleic amplification test for laboratory identification of COVID‐19 in asymptomatic emergency department patients. JACEP Open. 2021;2:e12592. 10.1002/emp2.12592
Supervising Editor: Christian Tomaszewski, MD, MS
Funding and support: By JACEP Open policy, all authors are required to disclose any and all commercial, financial, and other relationships in any way related to the subject of this article as per ICMJE conflict of interest guidelines (see www.icmje.org). The authors have stated that no such relationships exist.
REFERENCES
- 1. Zhen W, Smith E, Manji R, Schron D, Berry GJ. Clinical evaluation of three sample‐to‐answer platforms for detection of SARS‐CoV‐2. J Clin Microbiol. 2020;58(8):e00783‐20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Poljak M, Korva M, Knap Gašper N, et al. Clinical evaluation of the cobas SARS‐CoV‐2 test and a diagnostic platform switch during 48 hours in the midst of the COVID‐19 pandemic. J Clin Microbiol. 2020;58(6):e00599‐20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Harrington A, Cox B, Snowdon J, et al. Comparison of Abbott ID Now and abbott m2000 methods for the detection of SARS‐CoV‐2 from nasopharyngeal and nasal swabs from symptomatic patients. J Clin Microbiol. 2020;58(8): e00798‐20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Ravi N, Cortade DL, Ng E, Wang SX. Diagnostics for SARS‐CoV‐2 detection: a comprehensive review of the FDA‐EUA COVID‐19 testing landscape. Biosens Bioelectron. 2020;165:112454. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Basu A, Zinger T, Inglima K, et al. Performance of Abbott ID Now COVID‐19 rapid nucleic acid amplification test using nasopharyngeal swabs transported in viral media and dry nasal swabs in a New York City academic institution. J Clin Microbiol. 2020;58(8):1136–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. MedCalc . Accessed January 24, 2021. https://www.medcalc.org/calc/diagnostic_test.php
- 7. Rhoads DD, Cherian SS, Roman K, Stempak LM, Schmotzer CL, Sadri N. Comparison of abbott ID now, diaSorin simplexa, and cdc fda emergency use authorization methods for the detection of SARS‐CoV‐2 from nasopharyngeal and nasal swabs from individuals diagnosed with COVID‐19. J Clin Microbiol. 2020;58(8): e00760‐20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Hinson JS, Rothman RE, Carroll K, et al. Targeted rapid testing for SARS‐CoV‐2 in the emergency department is associated with large reductions in uninfected patient exposure time. J Hosp Infect. 2021;107:35‐39. [DOI] [PMC free article] [PubMed] [Google Scholar]