Performance of a Novel Point-of-Care Molecular Assay for Detection of Influenza A and B Viruses and Respiratory Syncytial Virus (Enigma MiniLab) in Children with Acute Respiratory Infection

Abstract

The performance of the Enigma MiniLab assay for influenza A and B viruses and respiratory syncytial virus (RSV) was compared to a centralized laboratory respiratory virus panel. The positive and negative percent agreement for influenza A virus, influenza B virus, and RSV were 79.2% (95% confidence interval [95% CI], 57.8 to 92.9%) and 99.4% (95% CI, 98.4 to 99.9), 100% (95% CI, 47.8 to 100%) and 100% (95% CI, 99.3 to 100%), 98.5% (95% CI, 94.6 to 99.8%) and 94.5% (95% CI, 91.9 to 96.4%), respectively.

TEXT

Influenza virus and respiratory syncytial virus (RSV) infections are common causes of acute respiratory illness resulting in high rates of hospitalizations and lost work or school days (1, 2). Rapid testing has an important role in clinical decision making and could facilitate the rational prescription of antivirals, reduce unnecessary pathology testing and antimicrobial therapy, and allow appropriate institution of infection control and public health interventions.

Previously commercially available rapid tests were based on lateral flow or fluorescent immunochromatographic technology detecting the presence of viral nucleoproteins. However, the performance of such tests is disappointing, with relatively poor sensitivities compared with PCR (3). Consequently, negative samples cannot confidently exclude infection and must be further tested with more-sensitive assays. Additionally, a recent randomized controlled trial evaluating the impact of these rapid tests on prescribing and clinical outcomes found little evidence to support their use (4).

Highly sensitive molecular assays for the detection of influenza virus and RSV have become commercially available and include the GeneXpert (5), Simplexa (6), and Prodesse (7) assays, which all detect influenza A and B viruses and RSV. Commercial assays particularly suited to near patient or the point-of-care setting have also been introduced, including the Alere I Influenza A&B (Alere, Stockport, United Kingdom) (8–10) and Cobas Liat Influenza A/B (Roche, Pleasanton, CA) (11) assays. These have both obtained CLIA (Clinical Laboratory Improvement Amendments) waiver, allowing use of the assay in nontraditional laboratory sites, such as emergency departments, clinic rooms, and pharmacy clinics (12).

The Enigma MiniLab influenza A/B & RSV assay obtained CE-IVD (in vitro diagnostics) designation in January 2014 and is available across Europe. The assay is fully automated, accepting nasopharyngeal swabs directly into a disposable cartridge. Magnetic bead purification and concentration are followed by fluorogenic reverse transcriptase PCR for the qualitative detection of influenza A virus (matrix gene), influenza B virus (nonstructural gene), and RSV (fusion gene). The assay has a running time of 90 min. The system requires less than 2 min of hands-on time and is scalable, with each module operating independently and on demand, meaning that up to six tests can be run concurrently (12). Figure 1 shows the platform in a single-module configuration.

FIG 1.

The Enigma MiniLab platform in a single-module configuration.

We investigated the diagnostic accuracy, turnaround time, acceptability, and ease of use of the Enigma MiniLab FluAB/RSV assay when operated in a near-patient setting by nonlaboratory staff, compared with a centralized laboratory-based molecular assay.

An Enigma MiniLab machine was placed in a treatment room on a 42-bed ward of Evelina London Children's Hospital, a 154-bed pediatric hospital in central London, United Kingdom. The ward has six high-dependency beds and accepts patients up to the age of 16 years and most respiratory admissions.

Clinicians were advised to test children with signs or symptoms of respiratory tract infection, including bronchiolitis, pneumonia, or influenza-like illness (ILI). Duplicate nasopharyngeal swabs were obtained for testing; however, no instruction was provided on the order in which to obtain the samples. One sample was tested in the hospital centralized virology laboratory using the xTAG Respiratory Virus Panel (RVP) Fast version 2 (Luminex Corp., Austin, TX, USA). The other sample was tested using the ward-based Enigma MiniLab platform by a trained operator (nurse or doctor). Both assays were performed in accordance with the manufacturers' instructions. For the Enigma MiniLab assay, the sample was collected into 1 ml Sigma-Virocult (Medical Wire & Equipment, Corsham, United Kingdom), and for the xTAG RVP, the sample was collected into 1 ml universal transport medium (UTM-RT) system (Copan Diagnostics, Corona, CA, USA), according to the manufacturer's instructions.

Discrepant results for samples with sufficient residual material were resolved with the Xpert Flu/RSV XC assay (Cepheid, Sunnyvale, CA, USA) according to the manufacturer's instructions. The study was considered to be an evaluation of a new service, and patient consent was not required.

A total of 56 staff were trained and assessed as competent to use the Enigma MiniLab platform. Each operator tested a median of six samples each during the course of the study (range, 1 to 59). Twenty-three (41%) operators tested three or fewer samples each, and 11 operators tested 25 or more samples each, comprising 58% of all samples tested.

The study ran for a 20-week period from 1 October 2014 to 31 March 2015 with a total of 698 nasopharyngeal swabs tested. Of these, 39 (5.6%) failed, giving no result; on three occasions (0.4% of all tests), this was due to operator error; however, the other tests failed due to unknown machine faults. These were spurious and self-correcting without the need for intervention from the manufacturer.

After excluding the failed tests and cases where no parallel xTAG RVP test was sent (n = 92), there were a total of 567 samples with evaluable results. The positive percent agreement (PPA) and negative percent agreement (NPA) are shown in Table 1.

TABLE 1.

Performance of the Enigma MiniLab assay compared with the xTAG RVP assay for the detection of influenza A and B viruses and RSV

Virus	No. of samples with the following results detected (Enigma MiniLab/xTAG RVP)				PPAa (95% CI)	NPAa (95% CI)
	+/+	+/−	−/+	−/−
Influenza A	18	4b	6c	539	81.8 (59.7–94.8)	98.9 (97.6–99.6)
Influenza B	5	1d	0	561	100 (47.8–100)	99.8 (99–100)
RSV	125	29e	3f	410	97.7 (93.3–99.5)	93.4 (90.7–95.5)

^aThe positive percent agreement (PPA) and negative percent agreement (NPA) and 95% confidence interval (95% CI) are shown.

^bOne sample was resolved as positive and three were resolved as negative by GeneXpert (GX) testing.

^cFour samples were resolved as positive and one was resolved as negative by GX testing; one sample was not available for further testing.

^dOne sample was not available for further testing.

^eFive samples were resolved as positive and 16 as negative by GX testing; eight samples were not available for further testing.

^fTwo samples were resolved as negative by GX testing; one sample was not available for further testing.

In total, there were 40 discrepant results. Residual material was available for 30 of these, which were further tested with the Xpert Flu/RSV XC assay. Ten discrepant samples had no residual material available, these samples could not be resolved, and the xTAG RVP result was assumed to be the correct result. Table 2 shows the performance characteristics following resolution of discrepant results.

TABLE 2.

Performance of the Enigma MiniLab assay following discrepant resolution for the detection of influenza A and B virus and RSV^a

Virus	No. of samples with the following results detected (Enigma MiniLab/resolved results)				PPA (95% CI)	NPA (95% CI)
	+/+	+/−	−/+	−/−
Influenza A	19	3	5	540	79.2 (57.8–92.9)	99.4 (98.4–99.9)
Influenza B	5	0	0	562	100 (47.8.9–100)	100 (99.3–100)
RSV	130	24	2	411	98.5 (94.6–99.8)	94.5 (91.9–96.4)

^aThirty discrepant samples were tested with the Xpert FLU/RSV XC assay, and concordant results in any two of the three molecular assays were taken to be the correct result. Residual material was unavailable for 10 discrepant samples; the xTAG RVP result was used as the correct result for these 10 samples. PPA, positive percent agreement; NPA, negative percent agreement.

Although the majority of samples were tested with the Enigma MiniLab assay during daytime hours (36% tested between 11 a.m. and 7 p.m.), significant numbers of samples were tested outside these times (18% between 11 p.m. and 3 a.m., 14% between 3 a.m. and 7 a.m., 16% between 7 a.m. and 11 a.m., and 16% between 7 p.m. and 11 p.m.). The xTAG RVP assay was run daily throughout the study period with the exception of a small number of weekends, dependent on staff availability. Overall, the assay was performed daily for 93% of the study period. The median turnaround time for the xTAG RVP assay was 24.1 h (range, 4.7 to 96.9 h) including transportation to the central laboratory. The turnaround time for the Enigma MiniLab assay was not measured but is likely to be around taking 90 min, assuming samples are tested immediately.

This is the first clinical evaluation of this novel assay for the detection of influenza virus and RSV.

The PPA for detection of influenza A virus was lower than expected at 79.2%. The only other published data of this assay reported a sensitivity of 100% for influenza A virus (13). The PPA is comparable to that of another molecular device designed for use at the point of care. For example, the Alere I assay was shown to have a sensitivity of 82.3% in one study (combined for influenza A and B viruses), although the majority of these samples came from adults (14) and 73.2% in another study (10) compared with commercially available multiplex PCR. This lower than expected PPA may have implications for how the assay is used at the point of care; for example, a negative test may require secondary testing with a more sensitive assay. Other point-of-care assays have demonstrated very high sensitivity, for example, the Cobas Liat which was found to have a sensitivity for influenza A virus of 99.2% (11). Presumably, clinicians would be more confident to accept a negative result in the knowledge that the assay would result in very few false-negative results.

One potential reason for the slightly reduced PPA for influenza A virus could be a partial mismatch between the primers and the currently circulating strains. The current version of the Enigma MiniLab assay was designed and validated against strains circulating in the Northern Hemisphere during the 2013-2014 season. This could explain the different sensitivity value of 100% reported in a previous study (13) that was conducted with samples predating the 2014-2015 season. This issue highlights the importance of manufacturers' conducting post-market surveillance studies and responding to changes in circulating strains.

Although the PPA for RSV was excellent, the NPA was slightly lower at 94.5% with 24 potential false-positive results. Residual sample was not available in eight of these cases, and the xTAG RVP result was taken to be the correct result; however, this may not be the case, and this presumption may have resulted in an underestimation of the NPA. Nevertheless, false-positive results can be problematic for patient management, since patients with false-positive results may be housed in the same area as patients with true positive results.

The failure rate of 5.6% was higher than expected, although this was due to user error on only three occasions. The manufacturer has since made some modifications to the platform so the failure rate would be expected to be lower than this figure. As the cartridge is designed to be a closed system, thus minimizing any contamination, once loaded, the sample cannot be extricated from the cartridge. This prevents retesting in the event of a sample failure, and a new specimen must be obtained. The failure rate is comparable to that seen with the xTAG RVP assay, with a total of 317 out of 3,921 (8.1%) of samples needing to be retested due to various reasons.

The centralized laboratory test was performed according to demand (at least daily increasing to twice daily if there were sufficient samples). This, together with the multistep, hands-on nature of the xTAG RVP test, resulted in a median turnaround time of just over 24 h. This could potentially result in delay in prescribing antiviral therapy and complicates the allocation of respiratory isolation rooms. The Enigma MiniLab assay takes just 90 min to provide a result; however, this may be longer than some clinicians and patients are willing to wait, particularly in high-patient-throughput settings such as the emergency department. These turnaround times are similar to those reported by Chu and colleagues in a retrospective analysis of the impact of introducing a rapid PCR for influenza virus and RSV (15). They found that the turnaround time was reduced from 25.2 h using a laboratory-developed reverse transcription-PCR (RT-PCR) to 1.7 h for the rapid PCR assay. The authors also noted a significant reduction in the length of time of oseltamivir prescription.

A further consideration is the ability of the centralized laboratory test to detect a broad range of pathogens in addition to influenza virus and RSV. In a setting where clinicians have become accustomed to receiving results for a panel of targets, this may lead to the point-of-care test functioning as an accompanying screening assay rather than as a replacement test for the broad panel. Clearly, use in this scenario will have important implications for the assay's cost-effectiveness. While modeling data suggest that the identification of influenza virus in the emergency department and other settings using rapid multiplex PCR testing is a cost-effective strategy (16, 17), there are few studies that have investigated this prospectively.

Limitations of this study include the inability to resolve 10 of the discrepant samples due to insufficient residual sample; in these cases, the true result was taken as the xTAG RVP result; however, this might not be the case, and it may have led to an underestimation of the sensitivity for the Enigma MiniLab assay. Additionally, this is a single-center experience and would benefit from collaboration with other centers. Clinicians were not given any instruction on how to obtain the samples and in what order, this could have affected the performance characteristics of either of the assays.

As with all molecular assays, there is a risk that these tests might miss the identification of novel strains, which are not targeted by the primers and probes.

The current model of managing most presumptive infectious diseases involves the administration of empirical antimicrobials rather than pathogen-directed use (18). Rapid assays, such as the Enigma MiniLab assay, may have the potential to influence clinical decision making at the point of need and may result in a reduction in unnecessary antibiotic use, targeted use of antiviral drugs for influenza virus (and potentially also for RSV in the future), and rational use of isolation facilities.