The Aries Bordetella assay (Aries BA) (Luminex Corporation) recently received FDA clearance for the detection and differentiation of Bordetella pertussis and Bordetella parapertussis nucleic acids in nasopharyngeal swab (NPS) samples. The objective of this study was to evaluate the performance of the Aries BA in comparison to that of the BioFire FilmArray respiratory panel (RP).
KEYWORDS: Bordetella, Bordetella pertussis, whooping cough, molecular methods, rapid tests, sample to answer
ABSTRACT
The Aries Bordetella assay (Aries BA) (Luminex Corporation) recently received FDA clearance for the detection and differentiation of Bordetella pertussis and Bordetella parapertussis nucleic acids in nasopharyngeal swab (NPS) samples. The objective of this study was to evaluate the performance of the Aries BA in comparison to that of the BioFire FilmArray respiratory panel (RP). The Aries BA was evaluated using retrospective, remnant nasopharyngeal swabs (NPS), previously tested by FilmArray RP. Performance characteristics evaluated included positive percent agreement (PPA) and negative percent agreement (NPA) with the FilmArray RP. Discordant analysis was performed using bidirectional sequencing. A time and motion study was performed to compare the laboratory workflow of the two tests. Three hundred samples were included in the study. There were no samples positive for B. parapertussis. The PPA and NPA of the Aries BA were 61.1% (95% confidence interval [CI], 35.8 to 82.7%) and 100% (95% CI, 98.7 to 100%). Discordant results included five Bordetella bronchiseptica results that were incorrectly identified as B. pertussis by the FilmArray RP and one false-negative result for both the Aries BA and the FilmArray RP. The overall agreement between the Aries BA and FilmArray RP for the detection of B. pertussis was considered good at 97.7% with a kappa value of 0.71 (95% CI, 0.51 to 0.9). The Aries BA offers a new diagnostic option for the rapid and targeted approach to the diagnosis of pertussis. Unlike the FilmArray RP, the Aries BA did not cross-react with B. bronchiseptica in our study, although a larger sample set should be tested to confirm this finding.
INTRODUCTION
Pertussis, commonly known as whooping cough, is an acute, contagious respiratory disease caused by the bacterium Bordetella pertussis. B. pertussis can lead to life-threatening complications in infants and young children and was documented as one of the major causes of childhood mortality in the 20th century (1). Since the availability and widespread use of the pertussis vaccine in the 1940s, the incidence has decreased by more than 75% as compared with that of the prevaccine era. In spite of widespread childhood vaccination, over 100,000 cases were reported in the United States between 2012 and 2014 (2). Studies have estimated the number of B. pertussis-related cough illnesses to actually be between 800,000 and 3.3 million per year in the United States (3). A recent update from the WHO estimated that in 2014 there were 24.1 million pertussis cases worldwide, with 160,700 estimated deaths in children under 5 years of age (4). Another species of Bordetella, Bordetella parapertussis, also causes respiratory disease in humans. B. parapertussis infections present with symptoms similar to whooping cough and may be responsible for up to 20% of pertussis-like disease, mainly in young children (5). There is no cross-reactivity between the pertussis vaccine and B. parapertussis; therefore, all populations are susceptible to B. parapertussis, regardless of vaccination state. B. pertussis and B. parapertussis infections present with a biphasic display of symptoms. The communicable stage occurs from disease onset to approximately 2 weeks post paroxysmal cough stage (6). Early and accurate diagnosis is critical for positive patient outcome, as treatment is most effective within the first 2 weeks of infection.
The Aries Bordetella assay (Aries BA) uses Luminex Corporation’s MultiCode real-time PCR chemistry in combination with the Aries systems. The system is capable of automated nucleic acid extraction and purification, real-time PCR detection of nucleic acid sequences, and data analysis. The Aries Bordetella assay can directly detect and differentiate B. pertussis and B. parapertussis from nucleic acid in nasopharyngeal swab (NPS) samples from patients with signs and symptoms of whooping cough. The Aries Bordetella assay targets the pertussis toxin (ptxA) promoter and IS1001 repeat sequence in the genomes of B. pertussis and B. parapertussis, respectively. The promoter toxin gene targeted by the Aries Bordetella assay confers a high degree of specificity and is designed to detect only B. pertussis, unlike other PCR-based assays, such as those that target the IS481 region. In contrast, IS481, a multicopy insertion element, is present in multiple Bordetella species, including B. pertussis, Bordetella holmesii, and Bordetella bronchiseptica (7–9).
The objective of this study was to evaluate the performance of the Aries Bordetella assay (Aries BA) in comparison to that of the current test method in use at our institution, the FilmArray respiratory panel (RP) on the BioFire FilmArray 2.0. (bioMérieux/BioFire Diagnostics). The following respiratory pathogens are included on the FilmArray RP: adenovirus, coronaviruses (HKU1, NL63, 229E, OC43), human metapneumovirus, human rhinovirus/enterovirus, influenza A, influenza A (H1, H3, H1-2009), influenza B, parainfluenza viruses (1–4), respiratory syncytial virus, Bordetella pertussis, Chlamydia pneumoniae, and Mycoplasma pneumoniae. The evaluation of the Aries BA included assay performance (positive percent agreement [PPA] and negative percent agreement [NPA]) and workflow parameters, including hands-on time (HOT) and turnaround time (TAT) to results.
(This study was presented in part at the 2018 Association for Molecular Pathology Annual Meeting in San Antonio, Texas.)
MATERIALS AND METHODS
Study design and settings.
This was a retrospective study performed at Memorial Sloan Kettering Cancer Center (MSKCC), a 478-bed tertiary care cancer center in New York City with an annual volume of respiratory testing ranging from 6,000 to 10,000 tests/year depending on the severity of the influenza season. All nasopharyngeal swabs (NPSs) and lower respiratory tract samples collected for respiratory pathogens testing since 2011 were tested using the FilmArray RP.
Samples.
This study evaluated retrospectively collected NPS samples. All samples were previously tested using the FilmArray RP according to the manufacturer’s instructions and as previously described (10). Residual samples were stored at 2 to 8°C for up to 7 days or frozen at −80°C prior to testing. All NPS samples that were positive for B. pertussis between 2014 and 2018 were included in the study. Additionally, fresh FilmArray RP samples that were negative for B. pertussis were randomly selected and collected daily over a 2-week period to cover several days of testing and provide a minimum of 200 negative samples for inclusion in the study. Technologists performing the Aries BA testing were blinded to the results of the FilmArray RP testing. Aries BA testing was performed on 200 μl of each residual sample according to the manufacturer’s instructions and as previously described (11). Residual samples were stored at −80°C for discordant testing when applicable.
Statistical and discordant analysis.
Results were analyzed to determine the positive percent agreement (true positive/[true positive + false negative]) and negative percent agreement (true negative/[true negative + false positive]) of the Aries Bordetella versus those of the FilmArray RP platform. Discordant samples underwent repeat testing on both the Aries and FilmArray platforms and with bidirectional sequencing using analytically validated primers targeting the ptxA promoter in genomic regions distinct from that of the Aries BA. All concordant samples were also tested by bidirectional sequencing. The final call for the presence or absence of B. pertussis on discordant samples was made based on the result of bidirectional sequencing. Data analysis was performed in GraphPad Prism 7.04 (GraphPad Software, Inc., La Jolla, CA).
Workflow assessment.
A time and motion study was performed to compare the workflow of the Aries Bordetella assay (1 instrument, 12 cassettes) to that of the BioFire FilmArray RP on the FilmArray 2.0 (8 instruments). Data were collected from two independent observations of the two methods from sample acquisition to result. Data collected included the following: description of each step, number of touch points, hands-on time (HOT), automation (hands-off) time for each step, and overall HOT and turnaround time (TAT).
RESULTS
Accuracy.
A total of 300 NPS samples from 297 patients, previously tested by the FilmArray RP, were selected for testing in this study and included 18 samples that were positive for B. pertussis and 282 samples that were negative for B. pertussis. Of the 282 samples that were negative for B. pertussis, 73.4% (207/282) were negative for all pathogens detected by the FilmArray RP and 26.6% (75/282) were positive for at least one pathogen other than B. pertussis (Table 1). There were no known samples that were positive for B. parapertussis. The Aries BA was positive for 11/18 NPS samples for a PPA of 61.1% (95% confidence interval [CI], 35.8 to 82.7%) with the FilmArray RP. All samples that were negative for B. pertussis by the FilmArray RP were concordantly negative by the Aries BA for a NPA of 100% (95% CI, 98.7 to 100%). The overall agreement between the Aries BA and the FilmArray RP for the detection of B. pertussis was 97.7% with a kappa value of 0.75 (95% CI, 0.57 to 0.93) (Table 2).
TABLE 1.
Samples included in this study
| FilmArray RP results | Total no. of samples |
|---|---|
| Negative | 207 |
| Influenza B | 2 |
| Influenza A H3 | 1 |
| Human metapneumovirus | 6 |
| Parainfluenza virus 3 | 11 |
| Parainfluenza virus 1 | 3 |
| Respiratory syncytial virus | 1 |
| Rhinovirus/enterovirus | 47 |
| Adenovirus | 1 |
| Bordetella pertussis | 10 |
| Bordetella pertussis coinfections (7 rhinovirus/enterovirus; 1 adenovirus) | 8 |
| Other coinfections (coronavirus OC43/parainfluenza virus 3, coronavirus HKU1/parainfluenza virus 3, rhinovirus/enterovirus/parainfluenza virus 3) | 3 |
| Total | 300 |
TABLE 2.
Performance characteristics of the Aries BAa
| Aries Bordetella assay result | FilmArray respiratory panel result |
||
|---|---|---|---|
| No. detected | No. not detected | Total no. | |
| No. detected | 11 | 0 | 11 |
| No. not detected | 7 | 282 | 289 |
| Total no. | 18 | 282 | 300 |
Positive percent agreement, 61.1% (95% CI, 35.8 to 82.7%); negative percent agreement, 100% (95% CI, 98.7 to 100%); overall percent agreement, 97.7%, kappa = 0.75 (95% CI, 0.57 to 0.93).
Discordant analysis.
A total of 7 samples from 4 patients had discordant results between the FilmArray RP and the Aries BA test (Table 3). All 7 samples were positive for B. pertussis by FilmArray RP and negative by the Aries BA. Upon repeat testing by FilmArray RP, 85.7% (6/7) of samples repeated as positive for B. pertussis. The one NPS sample that was negative on repeat by FilmArray RP was originally positive for rhinovirus/enterovirus and B. pertussis. The rhinovirus/enterovirus target was detected on repeat testing. Similarly, repeat testing was performed for all 7 samples on the Aries BA with all samples remaining negative on repeat testing. Bidirectional sequencing of all 7 samples showed that 5 of the 7 samples were positive for Bordetella bronchiseptica, 1 sample was positive for B. pertussis, and 1 sample was negative for all Bordetella species. Sequence alignment and phylogenic analysis of the discordant samples sequenced are shown in Fig. 1. Following review of discordant results, 6/7 samples were called negative for B. pertussis, and one sample was called positive for B. pertussis. Bidirectional sequencing was also performed on the 11 B. pertussis-positive concordant samples, and they were confirmed as positive for B. pertussis (data not shown).
TABLE 3.
Discordant analysis results
| Sample no. | FilmArray RP initial result | FilmArray RP repeat result | Aries BA initial result | Aries BA repeat result | Bidirectional sequencing result | Final call |
|---|---|---|---|---|---|---|
| 229a | B. pertussis | B. pertussis | Negative | Negative | B. bronchiseptica | Negative |
| 230 | B. pertussis | B. pertussis | Negative | Negative | B. bronchiseptica | Negative |
| 233a | B. pertussis | B. pertussis | Negative | Negative | B. bronchiseptica | Negative |
| 234a | B. pertussis | B. pertussis | Negative | Negative | B. bronchiseptica | Negative |
| 237a | B. pertussis | B. pertussis | Negative | Negative | B. bronchiseptica | Negative |
| 240 | B. pertussis rhinovirus/enterovirus | Rhinovirus/enterovirus | Negative | Negative | Negative | Negative |
| 250 | B. pertussis | B. pertussis | Negative | Negative | B. pertussis | B. pertussis |
Samples from 1 patient collected over 2 months.
FIG 1.
Sequence alignment of discordant FilmArray RP and Aries BA results. Bidirectional sequencing of a region of the ptxA gene was performed on all samples with discordant results between the FilmArray RP and Aries BA. Sequences were aligned using ClustalW2 including B. pertussis and B. bronchiseptica reference strains. Nucleotide differences are highlighted within the blue rectangles.
Workflow assessment.
HOT and overall TAT to results were measured for all preanalytical, analytical, and postanalytical steps for 1, 6, and 12 samples. Results of the workflow assessment are shown in Fig. 2. The HOT for the Aries BA ranged from 2 min and 20 s for 1 sample to 18 min and 59 s when setting up 12 samples. For the FilmArray RP, the HOT ranged from 2 min and 45 s for 1 sample to 33 min and 03 s for 12 samples. The run time (automation time) for the Aries BA was approximately 1 h and 55 min, for an overall TAT of 1 h and 57 min for one sample and 2 h and 12 min for 12 samples. The run time for the FilmArray was 1 h and 08 min, for an overall TAT of 1 h and 11 min for one sample and 2 h and 48 min for 12 samples.
FIG 2.
Comparison of workflow between the FilmArray RP and Aries BA results. A time and motion study was performed to compare the workflow of the Aries Bordetella assay (1 instrument, 12 cassettes) to that of the BioFire FilmArray RP on the FilmArray 2.0 (8 instruments) for 1, 6, and 12 samples. HOT, hands-on time; AuT, automation time. Different touch points in the workflow are denoted by successive number (i.e., HOT1, HOT2).
DISCUSSION
In this study, we evaluated the clinical performance of the Aries Bordetella assay for the detection of B. pertussis in an oncology patient population. Our study had two main findings. First, we showed that the overall agreement between the FilmArray RP and the Aries BA was high at 97.7%. Both the FilmArray RP and Aries BA target the single-copy gene ptxA for the detection of B. pertussis and, thus, high agreement was expected (12, 13). In contrast to the FilmArray RP and Aries BA, other commercial assays, including the Focus Bordetella test (Focus Diagnostics), the illumigene pertussis DNA (Meridian Biosciences), and the AmpliVue Bordetella assay (Quidel), target the multicopy insertion sequence IS481 gene, a gene target that is also present in low copies in other Bordetella species, including B. holmesii and B. bronchiseptica. In one study, the sensitivity of the FilmArray RP for B. pertussis was compared to that of the Focus assay and shown to be approximately 30% less sensitive, highlighting the potential for false-negative results when using the FilmArray RP (14). However, while the use of the multicopy IS481 gene increases the sensitivity of molecular tests, it also has the potential to cause false-positive results due to cross-reactivity with other Bordetella species (8, 14).
Our second finding was that the FilmArray RP can yield false-positive results by misidentifying B. bronchiseptica as B. pertussis. This result is in agreement with a recent case report (15). McNulty and colleagues reported two cases of immunocompromised patients diagnosed with B. pertussis by FilmArray RP testing performed on NPS samples. Both patients were treated with azithromycin but did not improve. Subsequent bacterial cultures performed on lower respiratory tract samples grew B. bronchiseptica. Isolates recovered from both patients were tested on the FilmArray RP and resulted in false-positive results, confirming the inaccurate results (15). Our study, performed in an oncology patient population, further highlights this significant issue with the FilmArray RP, which is now acknowledged in the test package insert, to occur at high concentration of B. bronchiseptica. In one of the two patients positive for B. bronchiseptica, testing with the FilmArray RP was performed multiple times over a 2-month period. The patient, a posttransplant lymphoma patient, presented with respiratory symptoms including a productive cough but no classical evidence of whooping cough. He completed treatment with azithromycin with recommendations for prophylaxis of all close contacts. B. pertussis direct fluorescence antibody assay and a bacterial culture were performed on a NPS sample and resulted as negative. Infection with B. bronchiseptica was never considered in the setting of a positive B. pertussis result. The second patient, a multiple myeloma patient with recurrent disease, presented with fever and a dry cough, but workup for other pathogens was also pursued given the atypical presentation for B. pertussis. This patient was similarly treated with azithromycin but empirically treated with imipenem as well. Given the potential for delays in appropriate antimicrobial therapy, laboratories and clinicians should be aware of this limitation of the FilmArray RP in patients who do not present with clinical symptoms consistent with whooping cough. The Aries BA did not cross-react with the few B. bronchiseptica-positive specimens in our study. According to the manufacturer, in silico analysis showed that Aries B. pertussis primers should not cross-react with B. bronchiseptica. However, a larger sample set should be tested to confirm this finding.
In terms of laboratory workflow, the hands-on time of the Aries BA was comparable to that of the FilmArray RP. The Aries BA has a longer run time of 1 h 50 min, almost twice the run time of the FilmArray RP, resulting in a longer TAT per sample. However, as the number of samples increased, the overall TAT to results for the FilmArray RP increased due in part to additional HOT in the preanalytical setup and the wait for an instrument to become available when capacity was exceeded on the FilmArray RP. This result highlights a limitation of many sample-to-answer platforms where the throughput is low to moderate and, thus, requires purchase of several units to maintain a rapid TAT in laboratories with higher test volumes.
Our study has some limitations. First, this was a single-center, retrospective study performed in a primarily adult, oncology patient population with a relatively low prevalence of B. pertussis. Second, in addition to the ptxA gene, the Aries BA also targets the insertion sequence IS1001 for the detection of B. parapertussis (12), but no cases of B. parapertussis were identified in our study, limiting the evaluation of the second component of the Aries BA.
In conclusion, while the multiplexed, syndromic approach to testing for respiratory infections has great advantages, especially in immunocompromised hosts, our study shows that the Aries BA offers a new diagnostic option for the rapid and targeted approach to the diagnosis of pertussis.
ACKNOWLEDGMENTS
This work was supported by an industry-sponsored research grant from Luminex to N.E.B. S.A.D. and S.D. are employees of Luminex.
This work was supported in part through National Institutes of Health/National Cancer Institute Cancer Center Support (grant P30 CA008748).
REFERENCES
- 1.Centers for Disease Control and Prevention. 2017. Pertussis (whooping cough)—disease specifics. Centers for Disease Control and Prevention, Atlanta, GA: https://www.cdc.gov/pertussis/clinical/disease-specifics.html. [Google Scholar]
- 2.Centers for Disease Control and Prevention. 2017. Pertussis (whooping cough)—pertussis cases by year (1922–2015). Centers for Disease Control and Prevention, Atlanta, GA: http://www.cdc.gov/pertussis/surv-reporting/cases-by-year.html. [Google Scholar]
- 3.Cherry JD. 2005. The epidemiology of pertussis: a comparison of the epidemiology of the disease pertussis with the epidemiology of Bordetella pertussis infection. Pediatrics 115:1422–1427. doi: 10.1542/peds.2004-2648. [DOI] [PubMed] [Google Scholar]
- 4.Yeung KHT, Duclos P, Nelson EAS, Hutubessy RCW. 2017. An update of the global burden of pertussis in children younger than 5 years: a modelling study. Lancet Infect Dis 17:974–980. doi: 10.1016/S1473-3099(17)30390-0. [DOI] [PubMed] [Google Scholar]
- 5.Mattoo S, Cherry JD. 2005. Molecular pathogenesis, epidemiology, and clinical manifestations of respiratory infections due to Bordetella pertussis and other Bordetella subspecies. Clin Microbiol Rev 18:326–382. doi: 10.1128/CMR.18.2.326-382.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Centers for Disease Control and Prevention. 2017. Pertussis (whooping cough)—clinical features. Centers for Disease Control and Prevention, Atlanta, GA: http://www.cdc.gov/pertussis/clinical/features.html. [Google Scholar]
- 7.Glare EM, Paton JC, Premier RR, Lawrence AJ, Nisbet IT. 1990. Analysis of a repetitive DNA sequence from Bordetella pertussis and its application to the diagnosis of pertussis using the polymerase chain reaction. J Clin Microbiol 28:1982–1987. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Reischl U, Lehn N, Sanden GN, Loeffelholz MJ. 2001. Real-time PCR assay targeting IS481 of Bordetella pertussis and molecular basis for detecting Bordetella holmesii. J Clin Microbiol 39:1963–1966. doi: 10.1128/JCM.39.5.1963-1966.2001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Woolfrey BF, Moody JA. 1991. Human infections associated with Bordetella bronchiseptica. Clin Microbiol Rev 4:243–255. doi: 10.1128/CMR.4.3.243. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Babady NE, Mead P, Stiles J, Brennan C, Li H, Shuptar S, Stratton CW, Tang YW, Kamboj M. 2012. Comparison of the Luminex xTAG RVP fast assay and the Idaho Technology FilmArray RP assay for detection of respiratory viruses in pediatric patients at a cancer hospital. J Clin Microbiol 50:2282–2288. doi: 10.1128/JCM.06186-11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Juretschko S, Mahony J, Buller RS, Manji R, Dunbar S, Walker K, Rao A. 2017. Multicenter clinical evaluation of the Luminex Aries flu A/B & RSV assay for pediatric and adult respiratory tract specimens. J Clin Microbiol 55:2431–2438. doi: 10.1128/JCM.00318-17. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Babady NE, Stiles J, Ruggiero P, Khosa P, Huang D, Shuptar S, Kamboj M, Kiehn TE. 2010. Evaluation of the Cepheid Xpert Clostridium difficile Epi assay for diagnosis of Clostridium difficile infection and typing of the NAP1 strain at a cancer hospital. J Clin Microbiol 48:4519–4524. doi: 10.1128/JCM.01648-10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Jerris RC, Williams SR, MacDonald HJ, Ingebrigtsen DR, Westblade LF, Rogers BB. 2015. Testing implications of varying targets for Bordetella pertussis: comparison of the FilmArray respiratory panel and the Focus B. pertussis PCR assay. J Clin Pathol 68:394–396. doi: 10.1136/jclinpath-2014-202833. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Register KB, Sanden GN. 2006. Prevalence and sequence variants of IS481 in Bordetella bronchiseptica: implications for IS481-based detection of Bordetella pertussis. J Clin Microbiol 44:4577–4583. doi: 10.1128/JCM.01295-06. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.McNulty MC, Shibib DR, Steinbeck JL, Mullane K, Pisano J, Matushek S, Beavis KG, Tesic V, Pitrak D. 2018. Misdiagnosis of Bordetella bronchiseptica respiratory infection as B. pertussis by multiplex molecular assay. Clin Infect Dis 67:1919–1921. doi: 10.1093/cid/ciy469. [DOI] [PubMed] [Google Scholar]