Evaluation of the COVID19 ID NOW EUA assay

Highlights

• •ID NOW EUA SARS-CoV-2 assay had an overall agreement of 78.7 % when compared to the standard of care reference methods.
• •ID NOW had a sensitivity of 71.7 % and specificity of 100 %.
• •All the false-negative results occurred with weakly positive samples, with reference method C_T values ≥35.

Abstract

Background

The SARS-CoV-2 pandemic caused a major surge in needed diagnostic capacity. In response, many EUA assays have become available for clinical laboratories, and more recently, the point of care device, Abbott ID NOW.

Objectives

To determine the analytical performance of the ID NOW assay for detecting SARS-CoV-2.

Study design

Residual NP samples collected in viral transport media were tested by the ID NOW platform in two independent laboratories. Results were compared to either the CDC or New York EUA assays, which served as reference methods.

Results

Overall agreement of ID NOW was 78.7%. Sensitivity was 71.7% and specificity was 100%. Notably, all false-negative results correlated to those samples that were weakly positive.

Conclusions

ID NOW performs well for strong and moderately positive samples but has reduced sensitivity for weakly positive samples. This sensitivity, among other concerns, should be taken into consideration when using this test for patients with a low suspicion for COVID-19 disease.

1. Background

The SARS-CoV-2 pandemic caused a major surge in the diagnostic capacity needed for adequate response efforts. Many commercial companies have developed diagnostic assays that are available for clinical testing in the USA, if authorized through the FDA’s emergency use authorization (EUA) process. While most of the SARS-CoV-2 EUA assays for molecular detection must be performed in moderate- to high-complexity clinical laboratories, a few are authorized as point-of-care devices, such as the Abbott ID NOW. In addition to clinical laboratories, this assay can be performed by trained non-laboratory personnel in patient care settings such an Emergency Departments, physician’s offices or pharmacies, potentially bringing diagnostic testing for SARS-CoV-2 closer to the patient [1]. Among the marketing information for this new assay, potential advantages include its reported sensitivity (stated limit of detection (LOD) of 125 genome equivalents/mL) and run time (detection of SARS-CoV-2 RNA as early as five minutes and a negative result in thirteen minutes). Also, the ID NOW may provide rapid molecular results either from direct testing of nasopharyngeal, nasal, or oropharyngeal swabs or testing of the viral transport media (VTM) from swabs placed in this fluid after collection [2]. To date, reported performance of the ID NOW SARS-CoV-2 assay in the peer-reviewed literature has been variable. While Rhoads et al. showed 94 % positive agreement between the ID NOW and a modified CDC laboratory developed test (LDT), other evaluations showed lower positive percent agreements for the ID NOW as compared to an LDT reference method, ranging from 75 to 87% [[3], [4], [5], [6]].

2. Objective and methods

Given the potential advantages of this device over more traditional format molecular tests such as real-time RT-PCR, a small evaluation of the ID NOW COVID-19 test was conducted at two laboratories to assess its performance. Residual positive and negative nasopharyngeal patient samples collected in VTM were tested using the ID NOW EUA assay. Samples had been stored at -80℃ prior to testing. Results from the ID NOW assay were compared to the original results from either the CDC EUA or the New York EUA assays, which served as the reference methods. Positive and negative samples were alternated to assess for potential carry-over contamination of either the patient sample or amplicon. Precision was assessed by running one strongly positive, one moderately positive and one negative sample in triplicate. At both facilities, the work was conducted inside a Class II biosafety cabinet (BSC) by certified laboratory personnel.

3. Results

In total 46 positive and 15 negatives were tested for a total of 61 samples. Overall agreement of the ID NOW with the reference method was 48/61 (78.7 %). Specificity was 100 % (15/15). However, sensitivity was 71.7 % (33/46) with the ID NOW producing false negative results in 13 of the 46 positive samples tested. Notably, all false-negative results corresponded to those samples that were weakly positive, with a cycle threshold (C_T) values between 35–40 for all targets (Fig. 1 ). This suggests that the ID NOW has acceptable performance for strongly or moderately positive samples but may lack sensitivity when the sample contains low amount of viral RNA. Importantly, in a review of more than 5000 positive SARS-CoV-2 results at Wadsworth, 18 % of the tests had Ct values in the 35–39 range (data not shown), which would suggest that the ID NOW would have failed to detect approximately 1 in 7 to 1 in 8 of all the positive samples tested. An alternative explanation for the lack of detection of these weakly positive samples could be the degradation of viral RNA during either storage or the single freeze-thaw step. Precisions studies testing 2 positive samples in triplicate revealed that the ID NOW missed one replicate of the moderately positive sample, resulting in a precision of 83.3 % (5/6).

Fig. 1.

False-negative Results. False-negative samples are stratified by C_T value by reference method. FN; false-negative.

4. Discussion

After this evaluation was performed, a notice was issued by the manufacturer, stating that samples collected in VTM were no longer acceptable for the ID NOW COVID-19 EUA assay, citing lower sensitivity for these specimens [7], which was also observed in our study. While removing this specimen type is an important step to reducing false-negative results, this now presents a challenge to both laboratories and patient care settings in verifying assay performance prior to implementation [8]. Given that only the direct swab is acceptable, verification cannot be performed using residual or contrived samples that are readily available and commonly used for this purpose. Notably, the direct swab positive control included with the assay does not contain SARS-CoV-2 material and therefore cannot be used for either a target amplification control or for assay verification. There were other challenges and considerations observed during the evaluation period. The test procedure requires multiple cartridge transfers and manipulations, which may be challenging for personnel not familiar with or accustomed to adhering to molecular techniques. Manipulations also have the potential to cause contamination of the device and the operation environment. While no cross-contamination was observed during the evaluation period, frequent decontamination of the instrument and testing area was performed by wiping with 20 % bleach followed by 70 % ethanol to prevent both instrument and specimen contamination. Assay manipulation, with regards to biosafety, was also an important consideration. One procedural step, swirling the specimen in an open shallow well, poses a biohazard risk during test set up, thus all testing was performed inside a BSC. This would be challenging or impossible for most non-laboratory settings. Additionally, hands-on time to complete the assay was on average 7−9 min for positive samples and 15−16 min for negative samples. Given that a single device can only run one specimen at a time and the number of manipulations required, this assay would be limited in throughput for high volume testing, averaging only 3–4 test per hour.

Assessment of the ID NOW using residual samples in VTM demonstrated that the assay performs well for strong and moderately positive samples but has a dramatic reduction in sensitivity for weakly positive samples, which supports findings by other published studies. The impact on sensitivity achieved by restricting the assay’s use to direct swabs is not possible to assess in the laboratory setting. Therefore, sensitivity concerns, together with device workflow and biosafety mitigation, should be taken into consideration when planning for deployment of the test, and when testing is performed, in the non-laboratory setting, especially on patients with a low suspicion for COVID-19 disease.

CRediT authorship contribution statement

Stephanie L. Mitchell: Conceptualization, Methodology, Validation, Formal analysis, Investigation, Resources, Writing - original draft, Writing - review & editing, Supervision. Kirsten St. George: Conceptualization, Methodology, Validation, Formal analysis, Investigation, Resources, Writing - review & editing, Supervision.