Multicenter Evaluation of NeuMoDx Group B Streptococcus Assay on the NeuMoDx 288 Molecular System

Group B Streptococcus (GBS) is the leading cause of neonatal sepsis and meningitis in developed countries. Recommendations for antepartum GBS detection include enriched culture with several options for identifying GBS, some of which are time-consuming.

ABSTRACT

Group B Streptococcus (GBS) is the leading cause of neonatal sepsis and meningitis in developed countries. Recommendations for antepartum GBS detection include enriched culture with several options for identifying GBS, some of which are time-consuming. To reduce the time for identification and determination of the maternal GBS colonization status, rapid nucleic acid amplification technologies have been developed and commercialized. For rapid detection of GBS, a three-site clinical study was conducted to evaluate the NeuMoDx GBS assay, a real-time PCR test performed for vaginal/rectal swab specimens in Lim broth enrichment culture on the NeuMoDx 288 molecular system (NeuMoDx system); these data were used to a support 510(k) submission. A total of 1,250 eligible remnant samples were prospectively enrolled and tested during the study. The results of the PCR assay were compared to the results of the Centers for Disease Control and Prevention (CDC)-recommended enriched-culture method, which served as the gold standard reference method for the study. The NeuMoDx GBS assay results yielded a sensitivity of 96.9% (95% confidence interval [CI] = 94.1 to 98.4), specificity of 96.0% (95% CI = 94.6 to 97.1), and a total agreement with the reference method of 96.2% (95% CI = 93.8 to 98.3). NeuMoDx GBS assay results were also compared to results obtained using the BD MAX GBS assay on the BD MAX system. The two systems demonstrated a total percent agreement of 98.0% (95% CI = 95.5 to 100.0). The performance of the NeuMoDx GBS assay implemented on the NeuMoDx system compared favorably to the CDC enriched-culture method and to the BD MAX GBS assay.

INTRODUCTION

Infection with Streptococcus agalactiae, the Lancefield group B Streptococcus (GBS), is a major cause of infantile disease during the neonatal and perinatal periods (1, 2). GBS colonizes the maternal gastrointestinal and genitourinary tracts in 10 to 37% of pregnant women, and most carry GBS intermittently (2). Vertical transmission of GBS from an asymptomatic mother to a baby, either in utero or during delivery, results in transient colonization of skin and/or mucous membranes in 30 to 70% of neonates (1, 2). Systemic disease develops in 1 to 4% of colonized neonates, making GBS the leading cause of neonatal sepsis and meningitis in the United States (1–4).

Centers for Disease Control and Prevention (CDC) guidelines for preventing neonatal GBS colonization and infection recommend screening pregnant women at 35 to 37 weeks of gestation to predict who will be colonized during labor and who should receive intrapartum antimicrobial prophylaxis to prevent GBS transmission to the neonate during delivery (3). For screening, vaginal/rectal swab specimens are collected from the lower third of the vagina and from the anal canal (a single swab from the vagina first then the anal canal, or one swab collected from each site).

The CDC laboratory guidelines published in 2010 for detection of GBS currently remain the gold standard for the identification of GBS from clinical specimens processed for routine screening of antepartum women. These culture processing methods are time-consuming and labor-intensive since they recommend inoculating a selective broth media (e.g., Lim broth) to be incubated for 18 to 24 h (3). Next, a subculture of the incubated broth to standard solid medium, including sheep blood agar, or chromogenic agar is required, and solid medium requires reincubation for an additional 18 to 24 h if GBS growth is not detected. Colonies can then be identified as GBS based on colony morphology, biochemical tests, and latex agglutination testing. The CDC also recommends “optional” direct broth testing using antigen detection (i.e., if chromogenic broth is used, then GBS may be confirmed by latex agglutination) or nucleic acid amplification tests (NAATs). Several U.S. Food and Drug Administration (FDA)-cleared tests have been commercialized and provide results at least 1 day earlier than culture (5).

The NeuMoDx 288 molecular system (NeuMoDx Molecular, Inc., Ann Arbor, MI) is a “sample-to-result” fully automated real-time PCR-based diagnostic platform (Fig. 1), which integrates reagent storage, specimen preparation, nucleic acid extraction, PCR setup, amplification, and detection, as well as results analysis and reporting, on a single completely automated and random-access testing platform. All reagents required for nucleic acid extraction and amplification are provided in room temperature-stable, unit-dose sizes, completely eliminating the need for any manual preparation. Upon loading the specimens and requesting the appropriate test, the system performs all operations required to obtain a test result without any further user intervention. The NeuMoDx GBS assay implemented on the NeuMoDx system provides rapid in vitro qualitative results for detection of GBS from vaginal/rectal specimens after inoculation and enrichment in Lim broth. To perform the assay, an aliquot of the broth is mixed with NeuMoDx lysis buffer and processed automatically on the NeuMoDx system. The extracted GBS DNA, if present, is amplified using the NeuMoDx GBS PCR reagents that target a section of the pcsB gene sequence of the GBS chromosome. The NeuMoDx GBS test reagents include a DNA sample processing control (SPC1) to monitor the presence of potential inhibitory substances and/or system or reagent failures that may be encountered during the process. The NeuMoDx GBS assay can provide first results in approximately 60 min, similar to other DNA assays performed on the NeuMoDx system.

FIG 1.

NeuMoDx 288 system.

The purpose of our multicenter method comparison study was to determine the clinical sensitivity and specificity of the NeuMoDx GBS assay, as implemented on the NeuMoDx system. Inoculated, enriched Lim broth samples were tested, and the NeuMoDx GBS assay results were compared to results obtained by the procedures for processing clinical specimens for culture of GBS recommended by the CDC (the gold standard reference method) and to results obtained by the BD MAX GBS assay (Becton Dickinson, Franklin Lakes, NJ). The BD MAX GBS assay is FDA cleared and is one of the standard-of-care tests used by laboratories for GBS detection from Lim broth-enriched vaginal/rectal swab specimens. A diagram of the workflow for the method comparison study is shown in Fig. 2.

FIG 2.

Diagram of workflow for NeuMoDx method comparison study.

MATERIALS AND METHODS

Clinical samples.

A total of 1,250 deidentified remnant and previously inoculated Lim broth samples were enrolled and tested during the study. Samples were collected in a blinded prospective manner at three different CLIA-certified clinical laboratories (sites A, B, and C). All samples were enrolled according to institutional review board-approved procedures. Vaginal/rectal swab specimens were collected from pregnant women at 35 to 37 weeks gestation for routine standard-of-care screening purposes. Site A used the ESwab in liquid Amies medium (Copan Diagnostics, Murrieta, CA), site B used the M40 Transystem Amies agar gel transport swab (Copan Diagnostics), and site C used the BD double CultureSwab in liquid Stuart medium (Becton Dickinson). Vaginal/rectal swab specimens were cultured in Lim broth from 18 to 24 h at 35 to 37°C according to established CDC guidelines. Sample inclusion criteria included: (i) remnant samples of Lim broth inoculated with vaginal/rectal swabs collected per CDC guidelines from antepartum adult women at 35 to 37 weeks gestation; (ii) swab specimens collected in appropriate transport media prior to Lim broth inoculation stored at room temperature for ≤24 h or stored at 2 to 8°C for ≤6 days; (iii) Lim broth inoculated directly from swab(s) or an aliquot of transport media that was then aerobically incubated for 18 to 24 h at 35 to 37°C; and (iv) enriched Lim broth samples stored at 2 to 8°C for ≤7 days. Sample exclusion criteria included: (i) swab specimens not meeting the inclusion criteria for collection, transport, and/or storage, or inoculated Lim broth not meeting CDC incubation requirements; (ii) specimens from women in active labor; and (iii) any sample yielding inconclusive CDC culture isolate identification results or not tested according to the complete testing algorithm defined below.

Sample deidentification.

All enrolled samples were assigned a unique study number without any relationship to a patient identifier (e.g., name, birth date, medical record number, etc.).

Testing algorithm.

Lim broth samples were inoculated, incubated, and tested using the standard-of-care PCR test at each laboratory (i.e., the FDA-cleared BD MAX GBS assay) according to the manufacturer’s instructions, and the results were recorded. Two 1-ml aliquots were prepared from remnant samples of each enrolled and enriched Lim broth tube (one for NeuMoDx GBS testing and one for CDC enriched-culture method for GBS detection). Testing was performed on the NeuMoDx 288 molecular system according to manufacturer’s instructions by qualified, trained personnel. The CDC culture identification method was performed using a remnant aliquot of each Lim broth sample (3). Specifically, remnant Lim broth was plated to tryptic soy agar with 5% sheep blood at all three clinical sites, as outlined in the CDC guidelines. Colonies consistent with GBS morphology were confirmed via Gram stain, and Gram-positive cocci were evaluated for lack of catalase production. The final identification of GBS was confirmed using latex agglutination (Remel PathoDX Strep Grouping; Thermo Fisher Scientific, Waltham, MA). In addition, site A and site C performed the following additional processes based on internal standard procedures as a supplement to the CDC recommendations. For example, in addition to the blood agar plate, site A inoculated each Lim broth sample onto Granada chromogenic media (Hardy Diagnostics, Santa Maria, CA) as an additional indicator of the presence of any beta-hemolytic GBS and performed a PYR enzyme test (Hardy Diagnostics) on any gamma-hemolytic colonies with typical GBS morphology; those that tested PYR negative were then Gram stained, and confirmatory testing was performed as needed. Site C also plated the enriched Lim broth onto GBS Detect media (Hardy Diagnostics), which is used for the isolation and detection of nonhemolytic GBS. All sites performed the required testing and incubated the inoculated media according to CDC guidelines.

Quality control.

One external positive control and one external negative control were processed on the NeuMoDx system each day of testing during the study per CLIA requirements. Valid quality control (QC) results were confirmed as a prerequisite for including sample data from a given day in the performance analysis. External QC samples were prepared, divided into aliquots, and qualified in-house at NeuMoDx Molecular, Inc. (Ann Arbor, MI), and then provided to each testing site. Negative-control samples were comprised of Lim broth only. Positive-control samples used negative clinical Lim broth as the matrix and were spiked with an Acrometrix GBS positive control (Thermo Fisher Scientific) at a concentration to yield an approximate cycle threshold (C_T) value of 28 ± 1 during control acceptance testing prior to distribution to each site.

Invalid results.

If a QC test yielded an invalid result, a second test was performed according to the type of invalid result obtained (i.e., indeterminate or unresolved). For an indeterminate result, which generally indicates an instrument error, the test was repeated from the same control aliquot. For an unresolved result, which can indicate a control (or sample) processing error, a new control aliquot was used, and repeat testing was performed. If a patient Lim broth sample yielded an invalid result, the result was categorized as either indeterminate or unresolved, and repeat testing was performed on the sample (up to two times if needed) to obtain a valid result.

Discrepancy testing.

Discrepant results were defined as NeuMoDx GBS assay results that differed from the CDC enriched-culture method for GBS detection (reference method). A glycerol stock of each identified GBS culture isolate was prepared upon completion of all CDC reference method confirmatory testing and frozen at ≤70°C for additional testing, if needed. For performance analysis, discrepancies were handled as follows: if the NeuMoDx GBS assay result was negative and the CDC reference method was positive, a false-negative result was assumed and included in the performance analysis. If the NeuMoDx GBS assay was positive and the CDC reference method was negative, a false-positive result was assumed and included in the performance analysis.

Data analysis.

The sensitivity, specificity, and total agreement of the NeuMoDx GBS assay compared to the CDC reference method (enriched culture) were determined. The upper and lower limits of the 95% confidence interval (CI) were calculated using the 95% score CI method according to the CLSI-approved guideline EP12-A2 (6).

Interlaboratory reproducibility study.

The NeuMoDx 288 molecular system was evaluated by testing five replicates of a four-member GBS panel on three instrument systems (one system at each laboratory site) across 5 different days of testing. Panels consisted of moderate-positive (4× limit of detection [LOD]), low-positive (1.5× LOD), low-negative (1:100 dilution of 1× LOD), and negative samples that were prepared in-house by NeuMoDx Molecular, Inc., and kept frozen until time of use. Five replicates of each of the four sample concentrations were tested, along with one positive and one negative external control, on each day of study testing. Testing across the 5 different days was not necessarily consecutive.

RESULTS

Clinical samples.

Of the 1,250 samples collected and tested during the study, 57 samples were excluded from the final data analysis due to protocol noncompliance, including 48 samples that had handling errors due to frozen Lim broth and 9 samples that had equivocal culture results due to overgrowth of Proteus spp. Of the remaining 1,193 valid specimens tested across the three laboratory sites by the NeuMoDx GBS assay, there were 253 true-positive results, 895 true-negative results, 37 false-positive results, and 8 false-negative results compared to the CDC enriched-culture method (Table 1). All results yielded a sensitivity of 96.9% (253/261) (95% CI = 94.1 to 98.4), specificity of 96.0% (895/932) (95% CI = 94.6 to 97.1), and a total agreement of 96.2% (95% CI = 93.8 to 98.3) compared to the CDC reference method. Site-specific performance data are listed in Table 1. Three specimen cultures grew nonhemolytic GBS (NHGBS), and all three of the original Lim broth specimens tested positive by the NeuMoDx GBS assay and the BD MAX GBS assay.

TABLE 1.

Results of prospective sample testing at each laboratory site

Site	No. of samples that were:				Total	% (95% CI)
	True positive	False positive	True negative	False negative		Sensitivity	Specificity	Total agreement
A	73	9	263	6	351	92.4 (84.4–96.5)	96.7 (93.8–98.3)	95.7 (91.1–99.4)
B	62	19	318	1	400	98.4 (91.5–99.7)	94.4 (91.4–96.4)	95.0 (91.1–98.0)
C	118	9	314	1	442	99.2 (95.4–99.9)	97.2 (94.8–98.5)	97.7 (93.3–101.4)
Total	253	37	895	8	1,193	96.9 (94.1–98.4)	96.0 (94.6–97.1)	96.2 (93.8–98.3)

Eight negative and 37 positive NeuMoDx GBS assay results were discrepant compared to the CDC reference method (Table 1). Of the 37 false-positive samples, 20 (54%) also tested positive by the BD MAX GBS assay. Of the eight false-negative samples, seven (88%) also tested negative by the FDA-cleared BD MAX GBS assay. Six of the false-negative results were obtained on the same day from the same test run at site A (Table 1). In an attempt to determine a potential cause, all six of the original Lim broth samples were repeat tested by both molecular assays; five of six were again reported as a negative result by the NeuMoDx GBS assay, and six of six were again reported as negative by the BD Max GBS assay. One GBS glycerol stock isolate was unavailable for follow up testing. Matrix-assisted laser desorption ionization–time of flight analyses identified all five isolates from each of the five glycerol stocks as GBS. To rule out molecular assay primer/probe annealing concerns for the identification and detection of the cultivated GBS organisms, a 0.5 McFarland standard and two 10-fold dilutions of the glycerol stock were created using saline. The samples were run with valid external controls; all dilutions of the organisms were detected and called positive on the system using the NeuMoDx GBS assay.

A comparison was made of NeuMoDx GBS assay results to results obtained from the BD MAX GBS assay performed as the standard of care at each of the clinical laboratories. The result agreement between the two molecular methods demonstrated a total percent agreement of 98%. The percent agreements between the two systems are given in Table 2.

TABLE 2.

Result agreement between NeuMoDx and BD MAX systems^a

Parameter	No. of samples tested	Agreement (%)
PPA	270/274	98.5
NPA	899/919	97.8
Total	1,169/1,193	98.0

^aNeuMoDx GBS assay results compared to results obtained from the BD MAX system. PPA, positive percent agreement; NPA, negative percent agreement.

Invalid results.

There were 1,250 samples enrolled and 32 sets of controls (64 individual control samples) tested during the study. Combined, 1,314 initial tests were performed using the NeuMoDx system.

There were ten samples and one external control repeat tested due to invalid results. Specifically, six samples and one positive external control yielded an indeterminate result due to a system error, and four samples yielded an unresolved result due to sample or processing control errors. All eleven of the invalid results reported a valid result upon repeat testing. The ten repeat sample results are included in the final data analysis: the repeat control result occurred on the day when all data were excluded due to suspected handling errors and is therefore not included in the performance analysis.

Combined, the initial rate of invalid results (indeterminate plus unresolved) was 0.84% (11/1,314). Of this 0.84% invalid rate, the initial indeterminate rate was 0.5% (7/1,314), and the initial unresolved rate was 0.3% (4/1314). The final indeterminate and unresolved rates were both 0% since all of the invalid results were resolved upon retesting.

Performance of controls.

To fulfill daily QC requirements, 32 sets of external control samples were tested across all three sites during the study. The only invalid result from a QC sample occurred at site A when the positive-control sample yielded an indeterminate result and was resolved upon repeat testing. All other QC results were valid and correct upon initial testing.

Interlaboratory reproducibility.

Combined results from data obtained during the interlaboratory reproducibility study testing are presented in Table 3. The rate of initial invalid results obtained during the reproducibility study was 1.3% across three sites. There were no panel members that yielded an unresolved result, which provided a 0% unresolved rate. However, on day 2 of testing at site B, there were four samples that yielded an indeterminate result due to system error, which led to an indeterminate rate of 1.3% (4/300). Upon repeat testing, two of the four panel members obtained a valid result, whereas the remaining two samples yielded an indeterminate result a second time, then obtained a valid result upon the third test attempt. There were 15 sets of GBS external control samples tested during the study, and all 30 controls produced valid and correct results.

TABLE 3.

Combined results of interlaboratory reproducibility study^a

GBS sample panel concentration	Agreement type	Performance requirement (%)	NeuMoDx
			% agreement	95% CI
Moderate-positive samples (4× LOD, 95% CI upper limit)	Positive (PPA)	>99	100 (75/75)	95.1–100
Low-positive samples (1.5× LOD, 95% CI upper limit)	Positive (PPA)	>95	97.3 (73/75)	90.8–99.3
Low-negative samples (1:100 dilution of 1× LOD, 95% CI upper limit)	Negative (NPA)	>95	98.7 (74/75)	92.8–99.8)
Negative samples (no target present)	Negative (NPA)	∼100	100 (75/75)	95.1–100

^aThe lower and upper limits of the 95% CI were calculated using 95% CI method according to CLSI-approved guideline EP12-A2. LLOD, lower limit of detection; PPA, positive percent agreement; NPA, negative percent agreement.

DISCUSSION

The incidence in the United States of GBS neonatal early-onset disease, often presenting as sepsis and pneumonia during the first 7 days of life, and late-onset disease), often presenting as sepsis and meningitis between day 7 and 3 months, is reported to be 0.7 to 3.7 cases per 1,000 live births and 0.5 to 1.8 cases per 1,000 live births, respectively (2). A CDC Active Bacterial Core Surveillance (ABCs) report, which surveys select geographical regions in the United States, reported similar incidences in 2014 (4). Despite nearly universal GBS screening, GBS early-onset disease continues to occur, and the GBS burden remains a significant public health issue. Efforts to improve screening for GBS status are therefore important and may be able to take advantage of new rapid diagnostic technologies (1).

The traditional CDC enriched-culture method requires incubating vaginal/rectal swabs in enrichment broth for ≥18 h at 35 to 37°C prior to GBS testing (3). Available GBS testing includes culture, with or without the aid of a chromogenic medium, and NAATs. After enrichment, culture-based methods can take from 24 to 48 h for a final result, in contrast to molecular-based tests that are faster and less labor-intensive than culture (5).

The NeuMoDx 288 molecular system is a new, real-time PCR-based diagnostic laboratory instrument. The first FDA-approved, commercially available assay on the instrument, the NeuMoDx GBS assay, provides in vitro qualitative results for detection of GBS from enriched Lim broth culture of vaginal/rectal specimens, and it provides results at least 1 day earlier than culture of Lim broth.

The assay represents a powerful molecular tool to aid in faster determination of maternal GBS colonization status compared to culture and offers a simple and easy-to-use testing alternative to working up culture isolates, especially when highly skilled laboratory staff can be difficult to procure.

The NeuMoDx GBS assay on the NeuMoDx 288 molecular system demonstrated excellent performance in comparison to the CDC reference method in our multicenter study. The assay yielded a sensitivity of 96.9% (95% CI = 94.1 to 98.4), specificity of 96.0% (95% CI = 94.6 to 97.1), and a total agreement with the reference method of 96.2% (95% CI = 93.8 to 98.3), and an agreement of 98.0% (95% CI = 95.5 to 100.0) was demonstrated with the BD MAX GBS assay. Six false-negative results were obtained on the same day on the same test run at site A (Table 1) using the NeuMoDx assay. It is suspected that these results were most likely caused by sample handling/human errors after considering the follow-up testing results and contributed to the lower test sensitivity at site A.

The initial rate of invalid results (indeterminate plus unresolved) was very low at 0.84% (11/1314), and final indeterminate and unresolved rates were both 0% since all invalid results were resolved upon retesting. External controls also performed very well during testing. Only a single invalid result (i.e., when a positive control yielded an indeterminate result that was subsequently resolved upon repeat testing) occurred at site A. The interlaboratory reproducibility of the NeuMoDx GBS assay on the NeuMoDx system met previously set performance requirements (Table 3). The initial invalid result rate was low (1.3%), and the unresolved rate was 0%. All external controls performed as expected during the reproducibility study and produced results.

Investigators have reported a slightly increased GBS detection rate by real-time PCR compared to the CDC broth-enriched culture reference method (7). Proposed causes include (i) enhanced test sensitivity, (ii) nonviable GBS detection, and (iii) suboptimal detection by culture due to obscuring genitourinary flora or the presence of NHGBS. In our study, three samples from site C contained NHGBS that were all detected by the NeuMoDx GBS and BD MAX GBS assays. The NHGBS cases were readily detected by culture by site C, since this site used chromogenic GBS Detect media that can detect NHGBS (in contrast to site A, which used Granada agar that cannot detect NHGBS). In general, culture-based detection of NHGBS presents a significant challenge for most clinical laboratories. NHGBS colonies morphologically resemble enterococci and require supplemental identification protocols or use of special media which are not currently listed in the CDC culture reference method.

When the NeuMoDx GBS assay results were compared to results obtained with the FDA-cleared BD MAX GBS assay on the BD MAX system (Table 2), the positive percent agreement and negative percent agreement were 98.5 and 97.8%, respectively. Result agreement between the two automated systems demonstrated a total percent agreement of 98.0%. Such a high level of agreement indicates that the performance of both systems and assays is substantially equivalent for the detection of GBS.

The NeuMoDx 288 system has several advantages: (i) it is fully automated, (ii) it offers random access, (iii) it is easy to use, (iv) it provides high throughput, and (v) it has room temperature on board reagent storage. The system, when loaded to capacity, can process 288 specimens without any user intervention, with a time to first result of ∼60 min for DNA assays, such as the NeuMoDx GBS assay. The NeuMoDx GBS assay and system join an expanding list of NAAT technologies and instrumentation available for GBS detection (5, 8). Indeed, many systems have been recently FDA cleared and commercialized, generating results at least 1 day earlier than the traditional CDC culture method (5). Miller et al. found that molecular testing (e.g., real-time PCR) following Lim broth enrichment provides rapid identification and the highest level of sensitivity for GBS detection (8).

In conclusion, our multicenter evaluation determined the NeuMoDx GBS assay implemented on the NeuMoDx 288 molecular system performed well compared to the CDC enriched-culture method, as well as the BD MAX GBS assay.