Multicenter Evaluation of BioFire FilmArray Meningitis/Encephalitis Panel for Detection of Bacteria, Viruses, and Yeast in Cerebrospinal Fluid Specimens

Abstract

Rapid diagnosis and treatment of infectious meningitis and encephalitis are critical to minimize morbidity and mortality. Comprehensive testing of cerebrospinal fluid (CSF) often includes Gram stain, culture, antigen detection, and molecular methods, paired with chemical and cellular analyses. These methods may lack sensitivity or specificity, can take several days, and require significant volume for complete analysis. The FilmArray Meningitis/Encephalitis (ME) Panel is a multiplexed in vitro diagnostic test for the simultaneous, rapid (∼1-h) detection of 14 pathogens directly from CSF specimens: Escherichia coli K1, Haemophilus influenzae, Listeria monocytogenes, Neisseria meningitidis, Streptococcus pneumoniae, Streptococcus agalactiae, cytomegalovirus, enterovirus, herpes simplex virus 1 and 2, human herpesvirus 6, human parechovirus, varicella-zoster virus, and Cryptococcus neoformans/Cryptococcus gattii. We describe a multicenter evaluation of 1,560 prospectively collected CSF specimens with performance compared to culture (bacterial analytes) and PCR (all other analytes). The FilmArray ME Panel demonstrated a sensitivity or positive percentage of agreement of 100% for 9 of 14 analytes. Enterovirus and human herpesvirus type 6 had agreements of 95.7% and 85.7%, and L. monocytogenes and N. meningitidis were not observed in the study. For S. agalactiae, there was a single false-positive and false-negative result each, for a sensitivity and specificity of 0 and 99.9%, respectively. The specificity or negative percentage of agreement was 99.2% or greater for all other analytes. The FilmArray ME Panel is a sensitive and specific test to aid in diagnosis of ME. With use of this comprehensive and rapid test, improved patient outcomes and antimicrobial stewardship are anticipated.

INTRODUCTION

Infectious meningitis and encephalitis are clinical conditions that can have very serious consequences. The morbidity and mortality of these infections can be high, particularly with bacterial meningitis, which can be rapidly life threatening, and the best outcomes are achieved with rapid initiation of the appropriate antimicrobial therapy (1). Those surviving infection may have significant long-term sequelae, such as loss of limbs, problems with vision and hearing, seizures, and cognitive deficits (2). In addition, the costs associated with these infections are significant, both in the short term related to hospitalization and treatment and in the long-term related to lost contributions to society (3, 4).

In the United States, there are approximately 4,100 cases of bacterial meningitis, including 500 deaths, every year (5). The most common pathogens of acute infections are Streptococcus pneumoniae, Streptococcus agalactiae (group B Streptococcus), Neisseria meningitidis, Haemophilus influenzae, Escherichia coli (particularly the K1 serotype), and Listeria monocytogenes, which together account for over 80% of infections. Viruses are the major cause of aseptic meningitis, a relatively common and often benign infection, with up to 85% being caused by non-polio enteroviruses (EV). Viral meningitis is more common than bacterial meningitis and often is much less severe. There are approximately 20,000 encephalitis-related hospitalizations per year in the United States, with an average of 1,400 deaths per year (6, 7). There is a vast array of pathogens that cause encephalitis, the majority of which are viruses. A number of noninfectious processes can also cause encephalitis, and the etiology remains unknown in up to 70% of cases (7).

There are a number of diagnostic challenges for identification of patients with meningitis/encephalitis (ME). The clinical presentation may be varied, and symptoms such as fever, headache, neck stiffness, altered consciousness, seizures, and focal neurological findings often overlap various infectious agents. An etiology is not always identified; this is due to a lack of targeted testing, the vast number of possible infectious causes, and the fact that approximately 10% of suspected cases are ultimately determined to have a noninfectious cause for their clinical symptoms. Routine testing such as cellular and chemistry parameters in the cerebrospinal fluid (CSF) may suggest the type of infection (e.g., bacterial versus viral or fungal); however, these parameters are not specific (8). For bacterial meningitis, culture is useful but takes 2 to 5 days and may be falsely negative if the organism is fastidious, pretreatment with antibiotics or incorrect specimen handling has rendered the sample sterile, or the causative organism does not grow in conventional culture. For nonbacterial infectious causes of ME, clinical suspicion is needed for the correct testing to be ordered, particularly for viral agents. Also, with current protocols, a comprehensive analysis may require significant CSF specimen volumes, which are not always available. Presently, many laboratories, particularly those in smaller and more rural settings, rely on reference laboratories to perform CSF testing for infectious agents. This leads to delays, which can have significant clinical consequences if therapy is delayed or incorrectly administered (9–11). It may also have an impact on appropriate infection prevention measures, in some cases including postexposure prophylaxis of close contacts.

A rapid, sensitive, and comprehensive test for some of the most common agents of infectious ME, requiring a small volume of CSF, would be very useful and help to overcome some of the challenges for conventional laboratory-based diagnosis. The FilmArray ME Panel was designed to identify 14 common agents of community-acquired ME in about 1 h: Escherichia coli K1, Haemophilus influenzae, Listeria monocytogenes, Neisseria meningitidis, Streptococcus agalactiae, cytomegalovirus (CMV), enterovirus (EV), herpes simplex virus 1 (HSV-1), HSV-2, human herpesvirus 6 (HHV-6), human parechovirus (HPeV), varicella-zoster virus (VZV), and Cryptococcus neoformans/Cryptococcus gattii. In this multicenter prospective study, residual CSF specimens were collected and tested at 11 sites across the United States. The performance of the FilmArray ME Panel was evaluated by comparison to conventional culture for bacteria and PCR with sequencing for viral and yeast targets.

MATERIALS AND METHODS

Clinical specimens.

The study was conducted at 11 geographically distinct U.S. sites over a period of approximately 8 months (February through September 2014). Specimens meeting the following inclusion criteria were selected: a CSF specimen was collected by lumbar puncture (LP) with adequate residual volume of uncentrifuged CSF (≥500 μl) left over from standard care testing for bacterial culture, and the specimen was able to be enrolled within 7 days of collection for testing (or frozen for later testing). Repeat specimens from the same subject were excluded. A waiver of the informed consent requirement was obtained from the Institutional Review Boards (IRBs) at each study site for the use of residual CSF specimens. Each residual specimen collected for the study was assigned a study code number (SCN) to deidentify the specimen aliquots used for FilmArray testing and comparator PCR testing and to provide select clinical and demographic data, including subject hospitalization status at the time of specimen collection, CSF chemistry results (white blood cell [WBC] count and differential [if performed], protein, and glucose), any additional CSF test requests and results, and the final CSF bacterial culture results. Additionally, the immune status of the subject, antimicrobial therapies administered within 24 h prior to specimen collection, and final clinician diagnosis were recorded for specimens that were positive for ME pathogens via standard laboratory testing of CSF, FilmArray, or PCR comparator testing or if CSF glucose, protein, and WBC were indicative of infection (i.e., glucose level of ≤45 mg/dl, protein level of ≥100 mg/dl, and WBC count of ≥5 cells/μl) (12). Immune status was determined and categorized according to 2013 Infectious Diseases Society of America guidelines (13).

Specimen aliquoting.

Aliquots of CSF specimens were prepared for testing at each site following deidentification. The aliquot for FilmArray testing was either tested fresh or archived at ≤−70°C within 7 days of the date of specimen collection. Frozen specimens were collected in the initial phase of enrollment (February through June) before the final “investigational use only” (IUO) version of the FilmArray ME Panel was available; specimens were then thawed and tested along with fresh specimens. All FilmArray testing was performed at the study site. Aliquots for comparator PCR testing were immediately frozen at ≤−70°C and shipped to BioFire on a weekly basis.

FilmArray ME Panel testing.

Approximately 200 μl of specimen was subject to FilmArray ME Panel testing according to the manufacturer's instructions. The FilmArray ME Panel test consists of automated nucleic acid extraction, reverse transcription, nucleic acid amplification, and results analysis in approximately 1 h per run (i.e., per specimen). This study was conducted with an IUO version of the FilmArray ME Panel that is identical to the final FDA-cleared/CE-marked in vitro diagnostic (IVD) version, with the exception that Epstein-Barr virus (EBV) is not available in the commercial product. (Note that EBV results are reported in this article for informational purposes only and are not included in the main data analysis.)

Each FilmArray pouch contains two internal controls: an RNA process control that targets an RNA transcript from the yeast Schizosaccharomyces pombe (freeze-dried into the pouch and rehydrated upon addition of sample, which was carried through all stages of the test process) and a PCR2 control that detects a DNA target dried into the wells of the array (which ensures that the second-stage PCR is successful). FilmArray ME Panel runs were considered valid if the run completed normally and both internal controls passed. The FilmArray software performs automated result analysis with each target in a valid run reported as “detected” or “not detected.” If either internal control fails, the software automatically provides a result of “invalid” for all panel analytes.

Five different frozen external control mixes (ECMs), covering all 14 targets, were provided to the study sites for daily testing. Four ECMs consisted of artificial CSF media that contained both whole organisms and nucleic acid template present at approximately 10× the limit of detection (LoD), and one ECM was negative for all analytes. FilmArray operators were required to complete a valid ECM run (correct results obtained) on each day of specimen testing, or else the results of that day of testing were considered invalid. The use of ECMs for daily testing of all analytes was specific to the clinical evaluation in order to comply with the FDA-requested study design.

CSF bacterial culture comparator testing.

Bacterial cultures were performed on every specimen enrolled. Testing was performed at each study site using the laboratories' standard procedures. Solid media (sheep blood, chocolate, and in some cases MacConkey agar) and, at some sites, a thioglycolate broth were incubated at 35 to 37°C in 5% CO₂ for 2 to 5 days. Bacterial isolates were identified using standard procedures validated and utilized by each study site (e.g., biochemical, phenotypic, or matrix-assisted laser desorption ionization–time of flight mass spectroscopy [MALDI-TOF MS] analysis).

Gram stains were also performed at each site using cytocentrifuged CSF. All sites had a policy in place to review the results of Gram stains with organisms isolated in culture and cultures that did not yield any growth. Results of the staining were reported for patient care and collected for this trial as part of data used for discrepancy investigation.

CSF viral and fungal PCR and sequencing comparator testing.

Nucleic acid was extracted from specimens using a MagNA Pure LC 2.0 automated system with the Total nucleic acid isolation—high performance kit (Roche Diagnostics, Indianapolis, IN). Extract from each specimen was then subject to PCR testing consisting of two well-validated assays for each analyte followed by bidirectional sequencing. All real-time PCR comparator assays were designed and validated by BioFire Diagnostics. Validation testing demonstrated that nearly all assays had an LoD that was within at least 10-fold of the FilmArray assay, and this was considered “equivalent sensitivity.” For HPeV only, both comparator assays were more sensitive than FilmArray. The PCR comparator assay targets, the validated LoD level, and the corresponding FilmArray assay LoD are listed in Table S1 in the supplemental material. Testing was performed by BioFire personnel in a blind manner. Comparator assays were only considered positive when a bidirectional sequencing result of adequate quality was found to match a sequence for the expected analyte. A specimen was considered positive for a given analyte if either of the two assays for that analyte was positive.

Results and discrepant analysis.

A FilmArray result was considered true positive (TP) or true negative (TN) only when it agreed with the result from the comparator method. Discrepant analysis ensued when results were discordant: i.e., false positive (FP) or false negative (FN). When sufficient specimen volume was available, discordant specimens were investigated using a combination of retesting with the FilmArray ME Panel (if the native specimen was available) or comparator methods (if extract was available) as well as testing with additional, independent molecular assays (see Table S2 in the supplemental material). Additionally, the discrepancy investigation for this study relied heavily on additional clinical information about the subjects whose specimens were tested in this evaluation. If clinical data were unavailable or if the available data did not support a diagnosis of meningitis or encephalitis, the FilmArray ME result was considered unconfirmed for the discordant investigation. Final adjudication of discrepant analyses was determined by one investigator (A.L.L.) but reviewed by all authors (except those from BioFire) for accuracy. Note that the performance data for sensitivity and specificity or positive and negative percentage of agreement (PPA and NPA, respectively) presented in this article consist of unresolved data only as presented in the package insert for the FDA-cleared test; discrepancy investigation is provided but was not used to recalculate performance data.

Calculations and statistical analysis.

Sensitivity and PPA were calculated as 100 × [TP/(TP + FN)], and specificity and NPA were calculated as 100 × [TN/(TN + FP)]. As described, PPA and NPA were calculated in the same manner as sensitivity and specificity, respectively. The terms “PPA” and “NPA” are used instead of “sensitivity” and “specificity” to indicate that a non-“gold standard” assay (e.g., PCR) was used for the original comparator analysis. The exact binomial two-sided 95% confidence intervals (CI) were calculated for both performance measures according to the Wilson score method (14). Because the final clinical diagnosis was collected for positive specimens only (positive by FilmArray ME, comparator testing, or with CSF parameters suggestive of infection) but not all specimens, it was not possible to calculate positive or negative predictive values.

RESULTS

Demographics.

A total of 1,643 prospective CSF specimens were acquired for the clinical study; 83 of these were excluded from further testing because the specimen was found to not meet the inclusion criteria after enrollment (e.g., insufficient volume for aliquoting, outside the 7-day window, the subject had been previously enrolled, or no data were obtained due to an invalid FilmArray test). Thus, a total of 1,560 specimens met all enrollment criteria, were prospectively tested successfully, and were included in the final analysis. A total of 1,015 (65%) specimens were tested fresh, and 545 (35%) had been frozen prior to testing on FilmArray ME Panel. The demographic characteristics collected on all enrolled patients were sex, age group, and patient location (emergency department [ED], hospitalized, or outpatient) at time of LP. There were nearly equal numbers of males (797 [51%]) and females (763 [49%]) enrolled. The majority of specimens (93%) were obtained from hospitalized patients or patients presenting to the emergency department (920/1,560 [59%] and 528/1,560 [34%], respectively). Outpatient collections constituted 7% (112/1,560) of the total enrollment and included patients being examined for various neurologic evaluations for which an LP and bacterial culture were standard of care. The age distribution included 921 (59%) adults 18 years of age or older and 639 (41%) pediatric patients of <18 years of age.

Summary of FilmArray ME Panel findings.

The FilmArray ME Panel detected at least one potential pathogen in 136 of the 1,560 specimens that were tested, yielding an overall positivity rate of 8.7%, as shown in Table 1. The highest detection rates were in the pediatric age groups. The relative prevalence of each analyte among the positive specimens detected by the FilmArray ME Panel is presented in Table 2. The most prevalent organisms detected during this study were EV, HHV-6, S. pneumoniae, HSV-2, and HPeV, which were found in 51 (37.5%), 22 (16.2%), 16 (11.8%), 12 (8.8%), and 12 (8.8%) specimens, respectively. All other assay targets were detected in 7 (5%) or fewer of the specimens. L. monocytogenes and N. meningitidis were the two targets with no FilmArray detections in this study.

TABLE 1.

Positivity rate for the FilmArray ME Panel for all samples and by age group

Parameter	Positivity
	No. of samples	% of total
All samples (n = 1,560)
Negative samples	1,424	91.3
Positive samples	136	8.7
Single detections	131	8.4
Codetections	5	0.3
Age group (n)
<2 mo (299)	58	19.4
2–23 mo (143)	17	11.9
2–17 yr (197)	15	7.6
18–34 yr (224)	15	6.7
35–64 yr (522)	23	4.4
≥65 yr (175)	8	4.6

TABLE 2.

Total number of FilmArray ME analyte detections by total positive samples detected and age group

Analyte	FilmArray result
	No. detected	% of positive samples	No. of positive detections by age group
			<2 mo	2–23 mo	2–17 yr	18–34 yr	35–64 yr	>65 yr
Bacteria
E. coli K1	3	2.2	0	1	0	0	2	0
H. influenzae	2	1.5	0	1	0	0	1	0
L. monocytogenes	0	0.0	0	0	0	0	0	0
N. meningitidis	0	0.0	0	0	0	0	0	0
S. agalactiae	1	0.7	0	0	0	0	0	1
S. pneumoniae	16	11.8	2	2	2	3	4	3
Total	22	16.2	2	4	2	3	7	4
Viruses
CMV	6	4.4	4	0	1	1	0	0
EV	51	37.5	31	5	11	4	0	0
HSV-1	4	2.9	0	2	0	0	2	0
HSV-2	12	8.8	0	0	0	1	8	3
HHV-6	22	16.2	9	7	2	3	1	0
HPeV	12	8.8	12	0	0	0	0	0
VZV	7	5.1	0	0	0	3	3	1
Total	114	83.8	56	14	14	12	14	4
Yeast
C. neoformans/C. gattii	5	3.7	1	0	0	1	2	1

Codetections were observed in five specimens, representing 0.3% of all specimens and 3.7% of the positive specimens (5/136) (Table 1). Codetections were as follows: S. pneumoniae and CMV, S. pneumoniae and varicella-zoster virus (VZV), S. agalactiae and HSV-2, HHV-6 and HSV-1, and EV and HPeV. All five of these specimens were found to have at least one false-positive detection as defined by comparator testing.

FilmArray ME Panel performance.

FilmArray ME Panel testing was successful on the first attempt for 1,560 of 1,577 specimens that meet all other enrollment criteria and were tested, representing a success rate of 98.9%. Seventeen specimens were unsuccessfully tested: 11 due to incomplete tests (software or instrument errors) and 6 due to internal pouch control failures. These six samples were clear in appearance, and all had a protein level of <100 mg/dl and a WBC count of <5 cells/μl. The exact reason for the six control failures cannot be discerned. The low percentage of failures (6/1,577 [0.38%]) is consistent with other published data on different FilmArray panels and indicates there was not a significant amount of inhibition of PCR in the samples (15). The final data set consisted of 1,560 specimens as described above.

The summary of performance characteristics for individual FilmArray ME Panel targets is presented in Table 3. Sensitivity/PPA and specificity/NPA were calculated with respect to the comparator methods of culture for bacterial pathogens and PCR and sequencing for all other pathogens. In a subanalysis of fresh and frozen samples, no difference was seen under the study conditions (freezing immediately after enrollment at <−70°C and thawing once prior to testing [data not shown]); therefore, performance data are shown in aggregate. More details on the effects of freezing are provided in the FilmArray ME Panel package insert (16). The FilmArray ME Panel demonstrated a sensitivity/PPA of 100% for 9 of 14 analytes: E. coli K1, H. influenzae, S. pneumoniae, CMV, HSV-1, HSV-2, HPeV, VZV, and C. gattii. Two analytes had lower sensitivities: 95.7% for EV and 85.7% for HHV-6. Two analytes, L. monocytogenes and N. meningitidis, were not detected by FilmArray or culture during the trial, and therefore no sensitivity could be calculated. For S. agalactiae, the ME Panel had one positive detection that was not confirmed by culture. Ninety-five percent confidence intervals (95% CI) for the sensitivity/PPA could be calculated for 9 of 14 targets (i.e., 5 targets had too few observations). Due to the low number of detections for many analytes, only one target had a lower bound of the 95% CI higher than 80% (EV). The specificity/NPA of the FilmArray ME Panel was 99.2% or greater for all analytes. The lower boundaries of the 95% CI for specificity were 98.7% or greater for all targets.

TABLE 3.

Performance summary and characteristics of the FilmArray ME Panel versus those of the comparator assays^a

Analyte	Sensitivity/PPAb			Specificity/NPAb
	TP/(TP + FN)c	%	95% CI	TN/(TN + FP)c	%	95% CI
Bacteria
E. coli K1	2/2	100	34.2–100	1,557/1,558	99.9	99.6–100
H. influenzae	1/1	100		1,558/1,559	99.9	99.6–100
L. monocytogenes	0/0			1,560/1,560	100	99.8–100
N. meningitidis	0/0			1,560/1,560	100	99.8–100
S. agalactiae	0/1	0.0		1,558/1,559	99.9	99.6–100
S. pneumoniae	4/4	100	51.0–100	1,544/1,556	99.2	98.7–99.6
Viruses
CMV	3/3	100	43.9–100	1,554/1,557	99.8	99.4–99.9
EV	44/46	95.7	85.5–98.8	1,507/1,514	99.5	99.0–99.8
HSV-1	2/2	100	34.2–100	1,556/1,558	99.9	99.5–100
HSV-2	10/10	100	72.2–100	1,548/1,550	99.9	99.5–100
HHV-6	18/21	85.7	65.4–95.0	1,532/1,536	99.7	99.3–99.9
HPeV	9/9	100	70.1–100	1,548/1,551	99.8	99.4–99.9
VZV	4/4	100	51.0–100	1,553/1,556	99.8	99.4–99.9
Yeast
C. neoformans/C. gattii	1/1	100		1,555/1,559	99.7	99.3–99.9

^aThese data are based on comparator assay only and do not reflect any discordant analysis.

^bThe performance measures of sensitivity and specificity only refer to bacterial analytes for which the gold standard of CSF bacterial culture was used as the reference method. Performance measures of positive percentage of agreement (PPA) and negative percentage of agreement (NPA) refer to all other analytes, for which PCR/sequencing assays were used as comparator methods.

^cA FilmArray ME result (detected or not detected) was considered true positive (TP) or true negative (TN) only when it agreed with the result (positive or negative) from the comparator method. When the results were discordant, they were considered to be either false negative (FN) or false positive (FP) relative to the comparator method.

Comparator analysis and discrepancy investigation.

There were a total of 21,840 individual FilmArray ME analyte tests performed for the 1,560 samples. The overall percentage of agreement between FilmArray and the comparator testing was 99.8% (21,791/21,840). There were 141 positive pathogen detections with the FilmArray ME Panel; the comparator methods had 104 pathogen detections. The initial overall percentage of agreement for positive targets was 69.5% (98/141). There were 21,699 negative pathogen detections with the FilmArray ME Panel; the comparator assays had 21,693 negative detections. The initial percentage of agreement for the negative targets was >99.9% (21,693/21,699). Additional data for all the TP results with associated laboratory data and final clinical diagnosis are presented in Table S3 in the supplemental material.

Using comparator testing as “truth,” there were 43 FP detections and 6 FN detections. Clinical and laboratory data for these 49 samples were analyzed along with the results of any additional discrepancy testing that was performed (e.g., repeating FilmArray, comparator assay, or additional molecular testing when specimen volume was available). In a total of 21 FP cases (43%), there was supportive evidence for the FilmArray result, bringing the overall concordance for the positive and negative results to 84.4% and >99.9%, respectively. For some patients, access to the clinical chart was limited or limited information was documented. A summary of the discrepancy investigation is presented in Table 4. Figure 1 shows the results as they were resolved after discrepant investigation. More extensive data on all discrepant samples are presented in Table S2 in the supplemental material.

TABLE 4.

Results of discrepant investigation for FilmArray ME Panel

FilmArray ME result disposition based on comparator testing	No. or type of resulta
	Total	Discrepant investigation outcome
		FA confirmed	FA unconfirmed
FN		TN	FN
S. agalactiae	1		1
EV	2		2
HHV-6	3	1	2
Total	6	1	5
FP		TP	FP
E. coli K1	1		1
H. influenzae	1	1
S. agalactiae	1		1
S. pneumoniae	12	5	7
CMV	3	1	2
EV	7	5	2
HSV-1	2		2
HSV-2	2	1	1
HHV-6	4	1	3
HPeV	3	3
VZV	3	2	1
C. neoformans/C. gattii	4	2	2
Total	43	21	22

^a“FA confirmed” indicates the results of discrepant analysis supported the original FilmArray ME result as true negative (TN) or true positive (TP). “FA unconfirmed” indicates the results of discrepant analysis did not support the original FilmArray ME result, and the result was considered false negative (FN) or false positive (FP).

FIG 1.

Relative performance of the FilmArray ME Panel versus comparator assays after additional discrepancy investigation. Additional discrepancy investigation included other laboratory testing and clinical data (see Table S2 in the supplemental material). FA, FilmArray ME Panel result; FP, false positive, FN, false negative.

Bacterial targets.

For the bacterial analytes, which used culture as the comparator, the FilmArray ME Panel detected 22 targets: 7 (32%) were concordant with culture, 15 (68%) were considered false positive, and 1 was false negative. The majority of the false-positive results were S. pneumoniae (80% [12/15]). A single false-negative result was noted with S. agalactiae. Discrepant analysis showed that there was additional testing or clinical data to support the FilmArray result for six of the false-positive results: five for S. pneumoniae and one for H. influenzae (Table 4).

In addition, there were 5 bacteria reported from culture that are not targeted by the FilmArray ME Panel; 1 Staphylococcus epidermidis isolate, 1 Salmonella sp. isolate, 2 Propionibacterium sp. isolates, and 1 Nocardia sp. isolate.

Viral targets.

For the viral targets, which used PCR and sequencing as comparator, the FilmArray ME Panel detected 114 targets, and 21.1% (24/114) were considered false positives; false positives were observed for all of the viral targets. A total of five false-negative results were also observed (two for EV and three for HHV-6). Discrepant analysis with one or more tests confirmed 13 FP FilmArray results (10 positive by a secondary PCR target, 5 confirmed by standard of care molecular testing, and 2 confirmed by clinical diagnosis). Accounting for resolution of the 13 FPs, there was a 9.6% (11/114) false-positive rate for the viral targets.

Yeast target.

For the cryptococcal target, which used PCR and sequencing as the comparator, the FilmArray ME Panel reported five positives, and 80% (4/5) were considered false positives; there were no false-negative results. For two of the FP, discrepancy investigations supported the FilmArray result, leaving 40% (2/5) of Cryptococcus results as unconfirmed FP.

Epstein-Barr virus.

The IUO version of the FilmArray ME Panel utilized for this study included an assay for Epstein-Barr virus (EBV) in addition to the other 14 pathogens discussed above. There were a total of 41 EBV detections with the FilmArray ME Panel, representing a prevalence of 2.6% in the patient population tested. The majority of positives were in adults (97.6% [40/41]). The comparator method of PCR followed by sequencing detected 22 specimens positive for EBV, of which 18 were also positive by FilmArray ME Panel test. The resulting performance characteristics were a PPA of 81.8% (95% CI, 61.5 to 92.7) with two false-negative results and an NPA of 98.5% (95% CI, 97.8 to 99.0) with 23 false-positive results.

DISCUSSION

Laboratory testing is essential for definitive diagnosis of infectious ME as the presenting signs and symptoms lack predictive value, particularly in young infants (17, 18). While culture of the CSF is considered the gold standard for diagnosis of bacterial meningitis (19, 20), final results of culture are often not available until 48 or more hours after specimen collection. These delays can have significant clinical consequences for patients: unnecessary broad antimicrobial coverage may be given pending results, or conversely, definitive therapy may be delayed. Such delays may also have an impact on close contacts that might require prophylactic treatment.

Rapid methods to identify those at risk of central nervous system (CNS) infections have been researched extensively. Gram stain has been assessed in cases of suspected meningitis and reported to have sensitivity ranging from 10 to 93%, depending on the organism and population being tested (1), and a near 100% specificity (21). While pleocytosis in the CSF is a sensitive marker of inflammation, several studies have shown cell counts may be normal in both adult and pediatric patients despite a diagnosis of bacterial meningitis (21, 22). As for the detection of viruses, molecular testing has improved sensitivity and is faster than culture, becoming the standard of care for many viral CNS infections, such as for the herpesviruses, EV, and HPeV (23). Cryptococcal meningitis has been diagnosed by a combination of testing, including CSF direct staining, culture, and antigen testing. Using these tests, most cases of cryptococcal meningitis are able to be diagnosed with good sensitivity and specificity (24, 25). More recently, molecular methods for detection of both C. neoformans and C. gattii have been reported that may improve laboratory diagnosis (26, 27).

This evaluation of the FilmArray ME Panel demonstrated the performance of a multiplex test for diagnosis of infectious ME in a large prospective study of 1,560 remnant CSF specimens with greater than 21,000 results generated. To our knowledge, this is the largest clinical trial for a commercial in vitro diagnostic test for detection of multiple agents of ME in CSF specimens. The number of positive detections was relatively low, but this is expected in light of the relative rarity of some of these infections and the fact that not all of the patients that underwent LP would ultimately have been diagnosed with ME attributable to any cause. The FilmArray testing system was shown to be reliable, with few failures, and rapid, with results available in approximately 1 h. The data presented here along with testing of archived positive CSF specimens and contrived samples (not shown) (16) were used as part of the regulatory submissions for the BioFire meningitis/encephalitis panel, which received de novo clearance in the United States in 2015.

FilmArray ME Panel bacterial targets.

There were relatively low numbers of bacterial detections with the FilmArray ME Panel. The definitive diagnosis of bacterial meningitis has been historically based on isolation in culture, which has a sensitivity of, on average, approximately 80% or greater (1). Culture was therefore used as the comparator assay for this study. The sensitivity of culture does vary by pathogen, and pretreatment with antibiotics lowers sensitivity (20). There were no detections by either test of N. meningitidis or L. monocytogenes, so sensitivity calculations were not possible, but there were no false-positive results with the FilmArray ME Panel for these two targets. For the other bacterial targets, a total of 22 positives were detected by FilmArray ME Panel. Seven specimens (32%) were also positive with culture, and the remaining specimens were culture negative. The final diagnosis in the medical record indicated bacterial meningitis consistent with the FilmArray detection. For these seven samples, the FilmArray ME results would have been available much sooner than culture and could have resulted in a timelier narrowing of therapy. As detailed in Table S2 in the supplemental material, for 15 specimens that were FP for a bacterial target (FilmArray positive, culture negative), discrepancy investigations supported the FilmArray diagnosis of bacterial meningitis in 6 cases (5 specimens of S. pneumoniae and 1 of H. influenzae). For three of these cases (one specimen of H. influenzae and two of S. pneumoniae), Gram stain was also positive with an organism consistent with the FilmArray finding. For one of these three patients, a >65-year-old female with a Gram stain detected Gram-positive diplococci but a negative culture: pretreatment with broad-coverage antimicrobials likely sterilized the CSF, but nucleic acid was still detectable (patient 7 in Table S2). This is of clinical significance because definitive identification, in addition to the Gram stain, may allow more targeted antimicrobial therapy. The increased sensitivity of an amplified molecular method that can detect nonviable and lower levels of bacteria in the CSF can increase the sensitivity of laboratory diagnosis of ME. Other studies looking at single organism-specific PCR or multiplex assays for a variety of organisms, such as H. influenzae, S. pneumoniae, and N. meningitidis, have demonstrated increased sensitivity compared to conventional methods (28–30).

For the remaining nine FP bacterial results, discrepancy investigations did not support the FilmArray results. There were seven FP S. pneumoniae detections from five study sites in various age groups. These patients had final diagnoses, including other infectious and noninfectious processes, none of which includes S. pneumoniae infection. There was one FP for E. coli K1, as well as one FP for S. agalactiae. The patient with an FP E. coli K1 detection was a 35- to 64-year-old female who had normal CSF parameters and a final diagnosis of a noninfectious CNS disease (hyperviscosity syndrome in patient 1 in Table S2 in the supplemental material). The patient with the S. agalactiae detection was a >65-year-old male with normal CSF parameters and a final diagnosis of sepsis secondary to prostatitis (patient 3 in Table S2).

There was a single FN bacterial result with FilmArray ME Panel (culture positive with S. agalactiae but FilmArray negative). The specimen was from a >65-year-old male patient with an intrathecal device for drug delivery (baclofen pump); the medical records indicated an infection in a pocket site, and the CSF parameters were all normal, with a single colony noted on the culture plates, which was not saved for further analysis. Infections related to CNS hardware can include a wide variety of bacteria and may not always be associated with infectious meningitis (31). It is possible that this patient did not have true infectious meningitis but a device-related soft tissue infection. Caution should be used when interpreting testing with the FilmArray ME Panel, particularly negative results, when testing CSF collected by LP in patients with CNS-related hardware (32).

While the bacterial targets encompass the most common agents for infants, older children and adults, the FilmArray ME Panel does not cover all possible agents of bacterial CNS infections (see the Results section), nor does it provide any information on antibiotic susceptibility. Therefore, culture continues to be necessary for all CSF specimens, regardless of the FilmArray ME Panel result. Likewise it seems prudent to consider a Gram stain for confirmation of FilmArray results and for better understanding of the difference in sensitivities of the two methods.

FilmArray ME Panel viral targets.

There were a relatively large number of viral detections with the FilmArray ME Panel in the viral target cohort. The comparator method of PCR and sequencing confirmed the FilmArray findings in 90 (79%) of these cases, leaving 24 FP viral results. The calculated PPA and NPA were very good for all targets (>95%), with the exception of HHV-6, which had a PPA of 85.7% and an NPA of 99.7%. The results of discrepancy investigation confirmed the FilmArray finding in 13 cases (5 of EV, 3 of HPeV, 2 of VZV, 1 of HSV-2, 1 of CMV, and 1 of HHV-6 [see Table S2 in the supplemental material]). There were five FN viral results (negative by FilmArray and positive by comparator PCR and sequencing), and in one of these (HHV-6), the results supported the negative result by FilmArray.

It is important to remember that while the comparator PCR methods were expected to match or exceed the sensitivity of FilmArray, there are differences in analytical performance that contribute to the discrepancies noted between the two tests. The median real-time PCR quantitation cycle (C_q; analogous to cycle threshold [C_T]) observed at the LoD for each comparator assay is shown in Table S1 in the supplemental material, along with the median and observed range of C_q values measured in clinical specimens during the study. While these nested-PCR assays are not quantitative, an examination of C_q values is helpful in assessing relative analyte levels between specimens. A comparison of the median C_q value observed at LoD to the median value observed in clinical specimens indicates that the level of analyte observed in many specimens was below the assay LoD. For example, the CMV UL123 assay has an LoD of 100 tissue culture infective doses (TCID₅₀)/ml. Contrived specimens spiked at this level have a median C_q value of 17.5. In the study, there were three CMV detections with the UL123 assay in clinical specimens with a median C_q value of 20.2. This suggests that the median level of CMV in clinical specimens is approximately 10-fold lower (three cycles) than the assay LoD. The median C_q values observed in clinical specimens were later than those observed at the LoD for all assays, with the exception of those for EV, HSV-1, VZV, and one of two HPeV assays. This indicates that the levels of viruses and Cryptococcus in many of the specimens tested in the study were very low and may explain some of the observed discordant results. Testing repeatedly at or near the LoD can lead to variation in results between and within a given method due to sampling variability.

EV and HPeV.

EV was the most common positive pathogen in this trial, with 51 detections; for HPeV, there were 12 detections. As many of the samples were collected in the late summer/early fall, this finding is consistent with the known epidemiology of EV and HPeV. The FilmArray ME Panel demonstrated excellent positive and negative percentages of agreement for both viruses (>95% and >99%, respectively). For the EV TP specimens, 64% (28/44) had a final diagnosis of viral/aseptic meningitis due to EV in the medical record. The other 16 cases had final diagnoses of viral/aseptic ME with no agent specified, viral syndrome, or other nonspecific syndrome. Of the nine TP HPeV detections, three had a final diagnosis of viral/aseptic meningitis due to HPeV, and the other six had diagnoses of viral syndrome or fever in infancy (see Table S3 in the supplemental material). It appears that the FilmArray may have provided a more definitive diagnosis in these 22 patients (16 with EV and 6 with HPeV). Of importance, the FilmArray has the significant advantage over the one currently available, FDA-cleared molecular test (33) for detection of enterovirus in CSF, in that the FilmArray can also detect HPeV in addition to the other 12 pathogens.

HSV.

There were a total of 16 positive HSV detections in this trial: 4 with HSV-1 and 12 with HSV-2. For 12 (75%) specimens, the comparator PCR and sequencing were also positive, confirming the FilmArray findings (see Table S3 in the supplemental material). For the two TP HSV-1 detections, one patient had a final diagnosis of aseptic meningitis due to HSV, and the other had a final diagnosis of ME with an unspecified cause. For the 10 TP HSV-2 detections, 3 patients had a final diagnosis of HSV meningitis and the other 7 had various presentations—none linked to HSV.

For the four FP results (two of HSV-1 and two of HSV-2), additional discrepancy investigation confirmed one of the HSV-2 results. This was a specimen from a 35- to 64-year-old male with elevated protein and WBC count in the CSF, a positive HSV laboratory-developed test (LDT) PCR result from CSF performed as standard of care, and a final diagnosis of HSV encephalitis (patient 28 in Table S2 in the supplemental material). The remaining HSV-2 FP sample was from a 35- to 64-year-old female and had normal CSF parameters; however no additional clinical information was available (patient 29 in Table S2). For the two FP HSV-1 results, both patients were in the 2- to 23-month age range and had normal CSF parameters, and the final diagnoses were not HSV encephalitis (febrile seizure and fever of unknown origin). One of the patients had a negative HSV LDT PCR on the CSF done at the testing site (patient 26 in Table S2). The other was a TP for HHV-6 using the FilmArray (patient 27 in Table S2). As both of these samples originated from one enrollment site, there exists the possibility of contamination during FilmArray testing or sample aliquoting. These results are concerning as they seem most likely to represent FPs due to contamination.

Other herpesviruses.

Detection of the other herpesviruses (CMV, VZV, and HHV-6) in the CNS is not routinely performed in most evaluations of acute infectious ME. They are critical pathogens in certain populations, including immunosuppressed patients (34), and are best detected in the CNS by molecular methods (23). Furthermore, all of the herpesviruses included in the FilmArray ME Panel are known to establish latency so detection may represent a recent primary infection, reactivation with disease, reactivation without disease, or latent detection in cells present in the CSF. This might be particularly important with evidence of a bloody, traumatic tap and contamination of the CSF with peripheral blood cells. Therefore, careful correlation of the clinical presentation would be needed for interpretation of a positive result with any of these viruses.

VZV.

For this trial, all seven VZV detections were in adults (>18 years of age), and comparator testing confirmed four. For the four TP results, one patient had a final diagnosis of VZV encephalitis and the other three had varied presentations (two with headache as a primary complaint) (see Table S3 in the supplemental material). The discrepancy investigations confirmed the FilmArray results in two of three FP specimens; both patients had a final diagnosis of herpes zoster (patients 37 and 38 in Table S2 in the supplemental material). The remaining unconfirmed FP was a male with HIV infection whose spinal fluid was also positive for S. pneumoniae by FilmArray (patient 39 in Table S2). These results suggest that the FilmArray ME Panel was able to detect VZV and that the comparator testing was less sensitive. The findings also indicate that the level of VZV in the CSF was relatively low in these clinical samples (<0.10 TCID₅₀/ml [see Table S1 in the supplemental material]).

CMV.

A total of six CMV-positive specimens were detected by the FilmArray ME Panel, with three (50%) being confirmed by comparator. The positives were predominantly in pediatric age groups (four <2 months, one 2 to 17 years, and one 18 to 34 years of age); none were bloody in appearance. For the three TP CMV detections, two had known CMV infections (one with CMV encephalitis and one with CMV viremia), and one had a fever of unknown origin (see Table S3 in the supplemental material). Using discrepancy analysis, one of 3 FP specimens was confirmed as TP. This patient was a 2- to 17-year-old female with altered CSF parameters who also had S. pneumoniae bacterial meningitis confirmed by culture and FilmArray (patient 16 in Table S2 in the supplemental material). The finding of CMV by CSF PCR in the presence of a bacterial meningitis has been reported and may represent a false-positive finding possibly related to latent virus in the WBC in an immunocompetent CMV-seropositive host (35). The two remaining CMV FP specimens were both in patients <2 months of age. Both had normal CSF parameters and a final diagnosis not specifically related to CMV (febrile illness with pyelonephritis and hyperbilirubinemia) (patients 17 and 18 in Table S2). Both the two FP CMV results and the two TP CMV results in this age group must be considered carefully as they would trigger concern for unsuspected congenital infection. Others have reported the utility of testing for CMV using CSF PCR in cases of suspected congenital infection; however, detection in control patients was also noted (36). Due to limitations of the clinical information available in this study, we are not able to determine clinical implications of the four CMV detections (two TP and two FP results) in these young infants.

HHV-6.

The FilmArray assay can detect both variants A and B; however, it does not discriminate between them (16). In this trial, there were 22 detections of HHV-6 with the FilmArray ME Panel, making it the second most prevalent target after EV. The majority of detections (82%) were in pediatric patients; two detections were in CSF that had blood present. For the 18 TP HHV-6 detections, one had a diagnosis of HHV-6 infection, and one had a diagnosis of unspecified aseptic meningitis. The rest of the TPs had a variety of infectious and noninfectious final diagnoses (see Table S3 in the supplemental material). There were four FPs and three FNs for this analyte, and discrepancy investigation resolved one FP and one FN (see Table S2). Based on the known latency of the virus and its relatively high prevalence in brain and other tissues of the normal host, careful consideration of a positive result with the FilmArray ME Panel is needed before attributing CNS disease to this virus (37–39).

EBV.

A review of clinical data for subjects with a positive FilmArray ME Panel EBV result indicated that the vast majority did not have other clinical data indicative of ME disease (data not shown). It appeared that the detection of EBV in these subjects was likely explained by the presence of latent viral nucleic acid (e.g., from B cells) or reactivation in response to other medical conditions. While latency/reactivation is characteristic of all herpesviruses (including HSV-1/2, CMV, HHV-6, and VZV), the subjects with positive EBV results, in particular, appeared to have other medical conditions as the underlying cause of their illness. For the patient population tested in this study (CSF specimens submitted to the laboratory with a request for bacterial culture), a positive result for EBV could be misleading to clinicians and negatively affect patient care. This, combined with the observed discordance between the FilmArray ME Panel EBV assay and the reference method, resulted in the target being excluded from the final commercial version of the ME Panel, and no further analysis will be presented in this article.

Cryptococcus.

While the calculated PPA and NPA were good (100% and 99.7%, respectively, with no CI for PPA), there was only one TP result, as well as four FP results. For the single TP, the patient had a final diagnosis of cryptococcal meningitis (see Table S3 in the supplemental material). Two of the four FP patients were positive by cryptococcal antigen testing (latex agglutination or lateral flow) and had been diagnosed with cryptococcal meningitis (patients 40 and 41 in Table S2 in the supplemental material). Also of note, an additional eight specimens in the trial that were negative for C. neoformans/C. gattii when tested with FilmArray and comparator assays were also tested by cryptococcal antigen (data not shown). Seven of these specimens tested positive for antigen; review of medical records indicated that the patients were on antifungal therapy for cryptococcal infection at the time of specimen collection or had history of Cryptococcus infection. Therefore, positive antigen results for these patients in the absence of a positive PCR or isolation in culture are likely due to antigen persistence rather than the presence of a live organism. It has been shown that cryptococcal antigen can remain positive for long periods of time after therapy, even when culture and smears are negative (40). In a recent study of HIV-infected individuals with cryptococcal meningitis, the relative sensitivity of the FilmArray ME Panel for detection of Cryptococcus in CSF has been demonstrated to be 96% for detection of ≥100 Cryptococcus CFU/ml and also demonstrated that the antigen test may be positive while the FilmArray is negative (41).

For the remaining two FP cryptococcal detections, there was no additional testing for Cryptococcus: (i) a <2-month-old female with E. coli urinary tract infection (UTI) and rhinovirus respiratory infection (patient 42 in Table S2 in the supplemental material) and (ii) an 18- to 34-year-old male with cauda equine syndrome (patient 43 in Table S2). Interpretation of a positive cryptococcal result with the FilmArray ME Panel should be done in the context of all testing, and consideration of inclusion of antigen testing in suspected cases of cryptococcal meningitis could help to confirm the diagnosis.

Concern for discrepant results.

There were a significant number of unresolved FP results in this study (n = 22 [Table 4]). The exact nature of these cannot be determined, but some may have resulted from contamination of the specimens during aliquoting and testing at the trial sites (e.g., carryover from ECM material or contamination from common oral flora [i.e., S. pneumoniae and HSV-1] of study personnel) or some other aspect of the assay configuration or testing process. Based on the serious consequences of false-positive results in a CSF specimen, laboratories should follow strict procedural steps to minimize external contamination of samples or the testing environment (16). Of critical importance are the analytical and postanalytical steps that can help mitigate the potential for FP results. The FilmArray system itself is self-contained, and the testing pouch is not opened at any time after sample/buffer addition. This should minimize the risks related to amplicon contamination. For the testing personnel, consideration may be given to use of a mask or face shield, particularly if setup is done outside a biological safety cabinet; this additional step could lessen the risk of contamination with respiratory flora from the operator. While the testing is simple to perform, a level of understanding of amplified testing is needed to prevent unintentional lapses of protocol that could risk contamination of the testing. Results from testing should be scrutinized closely in the context of all available medical history to correlate the known epidemiology of the analyte detected with the presentation of the patient. Daily review of the positive results and the percentage of positives for an individual analyte may be useful to identify unusual clusters that could indicate possible contamination. Due to the relative rarity of these diagnoses, the expectation of multiple positives in any given day or week would be low for most analytes; clinical assessments should be performed before releasing results, and repeat testing should be considered in these situations. Additionally, a prior historical knowledge of local disease prevalence and seasonality would be useful for interpretation. The user must remember that despite a good collective performance for the FilmArray ME Panel, the individual false-positive and -negative results using any diagnostic test for ME can have extremely serious consequences.

In this trial, there were relatively few FN results, although the prevalence of positive analytes was low. With any test, there exists the possibility of false-negative results due to various reasons, but it is important to remember testing with the FilmArray ME Panel does not eliminate the need for culture and Gram stain of CSF due to the possibility of other infectious agents not covered in the panel. Nor does it eliminate the need for testing other sample types. In a large study of bacterial meningitis in pediatric patients, 7% of cases were identified in blood culture alone (42). Likewise, HSV infections are not ruled out by testing negative in the CSF; additional specimen types such as blood and surface swabs are often necessary for a complete evaluation of HSV, particularly in the newborn (43, 44). For E coli, only the K1 capsular type is detected by the FilmArray ME Panel. While this is the most common capsular type reported in newborns, causing up to 80% of cases of meningitis, other E. coli types are known to cause infections of the CNS (45). The narrowed specificity of the FilmArray ME Panel for this organism was a necessary design consideration in order to prevent detection of contaminating E. coli nucleic acid, which is common in PCR reagents (e.g., DNA polymerase and reverse transcriptase) (46, 47).

There are several limitations for this study. The trial was designed to determine the performance of the testing relative to another laboratory test, with the analysis of clinical data done retrospectively. Full clinical data were not available on all patients, and while additional data were accessible for the positive samples, there was no access to records for determination of readmissions, follow-up treatment, or additional testing that might have been performed after the LP. Differences in standard testing on CSF specimens at each test site could have influenced the comprehensiveness of the discrepancy investigation data. The pretest probability for the presence of an infectious cause for ME was not controlled for in the study design as enrollment was based solely on a sample being submitted for CSF culture with sufficient volume left over for study testing.

The FilmArray ME Panel represents a significant paradigm shift. It is the first test system allowing close-to-patient, rapid assessment of a broad range of infectious agents associated with CNS infections. The data presented in this study show that the FilmArray ME Panel is able to detect a broad range of pathogens directly in the CSF with good performance relative to culture and molecular reference methods. The panel may have significant utility in several patient populations, and the simplicity of the testing process means it can be offered in a variety of care settings, including community and rural hospitals. For young infants, the ability to test for both bacterial and viral agents will be tremendously helpful and may potentially allow more targeted use of both antibacterial and antiviral drugs. For the immunocompromised patient, this test may be of critical importance. While this study did not directly assess the utility of testing in the immunocompromised, it seems logical that these patients will also benefit greatly from a comprehensive test for ME in the CSF. It is estimated that 5 to 10% of transplant patients will have an infection involving the CNS—most often manifesting as meningitis, encephalitis, or brain abscess (48). However, each laboratory must assess the best populations or clinical scenarios where the FilmArray ME Panel could be applied for testing. As this test is more widely adopted, its clinical performance with associated patient care impact can be more fully and systematically evaluated.