Multicenter Evaluation of the New Vitek 2 Neisseria-Haemophilus Identification Card

Abstract

The new Neisseria-Haemophilus identification (NH) card for Vitek 2 was compared with 16S rRNA gene sequencing (16S) as the reference method for accurate identification of Neisseria spp., Haemophilus spp., and other fastidious gram-negative bacteria. Testing was performed on the Vitek 2 XL system with modified software at three clinical trial laboratories. Reproducibility was determined with nine ATCC quality control strains tested 20 times over a minimum of 10 days at all three sites. A challenge set of 30 strains with known identifications and 371 recent fresh and frozen clinical isolates were also tested. Expected positive and negative biochemical reactions were also evaluated for substrate reproducibility. All microorganisms were tested on the NH card, and all clinical and stock isolates were saved for 16S testing. All reproducibility tests yielded expected results within a 95% confidence interval. For challenge microorganisms, there was 98% overall correct identification, including 8% low discrimination, 2% incorrect identification, and 0% unidentified. For clinical strains, there was 96.5% overall correct identification, including 10.2% low discrimination, 2.7% incorrect identification, and 0.8% unidentified. The 2.7% (10/371) of clinical isolates that gave an incorrect identification consisted of 7 isolates correct to genus and 3 strains incorrect to genus. There were an additional 27 strains (primarily Neisseria species) for which the 16S identification result was different from the NH card result. These were all unclaimed species by the system. The new NH card met all performance criteria within a 95% confidence interval compared to identification of clinical isolates by 16S.

In the diagnostic clinical microbiology laboratory, rapid and accurate identification of bacterial pathogens is essential for prompt and appropriate management of infected patients. This is important for bacterial pathogens within the genera Haemophilus and Neisseria.

The science has evolved over many years. Classical methods gave way to substrate-based identification of these microorganisms involving overnight and then shorter incubation. More recently molecularly based identification has started to change the landscape (2, 5, 7, 8, 12), but for most laboratories that lack such sophisticated technical capabilities, the ability to use either a manual or automated system that will give a high level of correct identifications is sufficient for most purposes.

For identification of Haemophilus and Neisseria species, a variety of commercial methods have been available on the market for many years. For Neisseria species these have included Neisseria-Kwik, Gonogen, Gonochek II, RIM-N, API QuadFERM-Plus, Minitek, Identicult-Neisseria, and several others (1, 6, 16, 18). All of these are based on colorimetric changes in miniaturized substrates, with either enzymatic or growth end points. The ability to identify and separate Haemophilus species from nonpathogenic Neisseria species, particularly those from the respiratory tract, then led to the addition of Haemophilus identification in these systems. RapID NH, API NH strip, and Haemophilus Identification Test Kit were developed to separate Haemophilus species (3, 7, 9-11, 13-15, 17). These systems all perform with reasonable accuracy in the clinical laboratory. Depending on the method used and on the number and variety of strains tested, correct identifications varied between 73 and 99% for species in either genus.

The aforementioned systems are designed primarily for smaller laboratories. With changes and consolidations of laboratories, the use of automated systems for the identification of clinical isolates has become commonplace, and commercial manufacturers have now attempted to expand the capabilities of their identification systems to optimize identification for better patient care.

bioMérieux, Inc., has developed a new Neisseria-Haemophilus (NH) identification card for the Vitek 2 system. The NH card is based on colorimetric technology utilizing dehydrated media containing chromogenic substrates. The card has a database that includes 27 taxa of gram-negative fastidious bacteria and that maintains the predominance of Haemophilus and Neisseria species but also includes Actinobacillus, Campylobacter, Capnocytophaga, Cardiobacterium, Eikenella, Gardnerella, Kingella, Moraxella, Oligella, and Suttonella species. In a recently published investigation from one laboratory, 91% of strains included in the database were identified correctly without the need for additional tests (17). In the present investigation, the quality, reproducibility, and accuracy of this NH card were assessed in three large tertiary care clinical laboratories. Confirmation of identifications was made by 16S rRNA gene sequencing as a quality assurance step to assess the performance of the NH card.

MATERIALS AND METHODS

Test methodology.

Strains were isolated in pure culture on blood agar, chocolate agar, or other media that supported the growth of the individual strains for testing. For inoculation into the 64-well NH card, an overnight or 48-h culture of the strain was suspended in 0.45% aqueous NaCl to a turbidity equivalent to a 3.0 McFarland standard. The inoculated card was loaded into the Vitek 2 XL automated identification system according to the manufacturer's instructions. The inoculated card was incubated for 6 h, and the substrate reactions were read by one of two optical heads in the Vitek 2 system. A computer-assisted algorithm, based on data collected from known strains of the claimed species, was used to interpret the data for a final identification. Sufficient data have been collected from known strains to estimate the typical reactions of the claimed species to a set of discriminating biochemicals. An identification was obtained by evaluating the reactions in the NH card and comparing those results to expected reactions for the species.

Each clinical isolate at all three study sites was stored as a heavy suspension at 4°C in sterile physiological saline. These strains were then shipped directly to MIDI Labs (Newark, DE) for 16S rRNA gene sequencing.

Quality control and reproducibility testing.

Nine microorganisms comprised the quality control and reproducibility set of organisms. The strains tested were Eikenella corrodens ATCC BAA-1152, Campylobacter jejuni subsp. jejuni ATCC BAA-1153, Neisseria lactamica ATCC 23970, Gardnerella vaginalis ATCC BAA-1154, Haemophilus (Actinobacillus) actinomycetemcomitans ATCC BAA-1155, Neisseria gonorrhoeae ATCC 19424, Haemophilus influenzae ATCC 9007, Oligella urethralis ATCC 17960, and Haemophilus aphrophilus ATCC 33389. The set was tested 20 times over a minimum of 10 days at each of the three clinical trial sites. The results were used to evaluate system and substrate reproducibility. One manufacturing lot of NH cards was used for the entire study.

Evaluation of system reproducibility was based on the number of correct identifications. Correct identification was defined as accurate identification of a microorganism by the system as the only choice (with any level of confidence—excellent, very good, good, or acceptable) or as one of the choices within a multichoice (i.e., low-discrimination) result. The expected system reproducibility performance criterion was set at 95% or greater correct identification within a 95% confidence interval.

Evaluation of substrate reproducibility was based on microorganism/biochemical well combinations with a distinct expected reaction of positive or negative. For individual well reactions with distinct positive or negative reactions, the expected substrate reproducibility performance criterion was agreement of at least 95% within a 95% confidence interval.

Challenge testing.

Each clinical study site tested a panel of 30 well-characterized isolates once on the NH card. The strains were supplied to each study site by bioMérieux. The isolates were not among those used to create the NH database and were selected to represent the identification claims of the NH product. Definitions for the levels of correct identification were as follows: overall correct identification, the isolate was accurately identified by the system as the only choice (with any level of confidence) or as one of the choices within a low-discrimination result; low discrimination, the identification contained two or three single or multichoice identification results in contrast to the correct identification with a single choice; incorrect identification, a final identification in which the genus or species was incorrect; unidentified microorganisms, a final identification of “unidentified organism”, “inconclusive identification,” or “nonreactive biopattern.”

The minimum performance requirements for the clinical trial were at least 95% overall correct identification, less than 25% low discrimination, less than 2% incorrect identification, and less than 5% unidentified organisms for the NH card within a 95% confidence interval.

Clinical isolates.

A total of 371 recently isolated clinical strains, either fresh or recently frozen (within the past 6 months) and previously identified as one of the “claimed” species, were tested at the three clinical trial sites. All isolates were tested once on the NH card and by 16S rRNA gene sequencing utilizing approximately 500 bp. 16S rRNA gene sequencing was considered the reference method. MIDI Labs (Newark, DE) performed the 16S sequencing and utilized the Applied Biosystems MicroSeq microbial analysis software and database to evaluate genetic similarity (19). When a genetic match was not found in the MicroSeq library, MIDI performed a BLAST search of the GenBank public database (4). Sequences from isolates with a genus level 16S sequencing identification from MIDI or GenBank were also compared to the bioMérieux, Inc., proprietary 16S database. For the purposes of this investigation the 16S sequencing results were interpreted in the context of the NH identification database. A 16S sequencing result consisting of a single-choice, species level genetic match to the NH card identification was a complete, correct result. A 16S sequencing result with no genetic difference between multiple species was confirmed through supplemental biochemical testing if one of the species listed matched the NH identification. A 16S sequencing result with a species level identification that was discrepant with the NH identification was evaluated to determine if the microorganism listed was part of the NH claim list. If it was, the discrepant result was confirmed or resolved by retesting the isolate. If it was not listed on the NH claim list, the isolate was excluded from the data set but the results were reviewed to evaluate issues related to all unclaimed species. A 16S sequencing result reported by MIDI Labs as a genus level identification and/or match to a sequence in the GenBank database was sent to the bioMérieux R&D Microbiology group in La Balme, France, for review to determine the relationship of the sequence result to microorganisms in the database. Definitions for the level of correct identification and the performance criteria were the same as those used for the challenge testing.

Excluded/unclaimed genera/species.

Strains that were not included as claimed genera or species by NH card identification were recorded but necessarily excluded from the overall data set compilations. The 16S RNA sequence was used to arbitrate the NH card results for these strains. The criterion for an excluded or unclaimed strain was an identification from the NH card that was not in the database and that was supported by 16S RNA sequencing, no identification given by the NH card and a 16S sequence identification that was not one of the claimed species, or no 16S rRNA gene sequencing identification.

RESULTS

Quality control and reproducibility testing.

The performance requirements for quality control and reproducibility were met, within a 95% confidence interval, both cumulatively and by the individual trial site with two exceptions. C. jejuni subsp. jejuni ATCC BAA-1153 failed at one trial site but passed cumulatively and H. actinomycetemcomitans ATCC BAA-1155 failed at one trial site and cumulatively (Table 1). There were four substrates that had less than 100% accuracy and reproducibility in this set. These were l-glutamine, arginine arylamidase (ArgA), lysine arylamidase (LysA), and alanine-phenylalanine-proline arylamidase, although only H. actinomycetemcomitans failed the overall 95% confidence interval test and has since been removed from the NH quality control specifications. The ArgA and LysA tests for this strain were correct only one-third of the time. The other ATCC strains having distinct predictable reactions tested, as expected, correctly 100% of the time by site and cumulatively across sites.

TABLE 1.

NH card quality control and reproducibility results for all sites combined

ATCC no.	Species or subsp.	% Correct identification (within 95% CIb)	Discrepant test result(s)a (% correct reaction)
BAA-1153	Campylobacter jejuni subsp. jejuni	89 (yes)	1GLM (85)
BAA-1152	Eikenella corrodens	100 (yes)	None
BAA-1154	Gardnerella vaginalis	100 (yes)	None
BAA-1155	Haemophilus actinomycetemcomitans	87 (no)	ArgA (33), LysA (37)
33389	Haemophilus aphrophilus	98 (yes)	None
9007	Haemophilus influenzae	100 (yes)	None
19424	Neisseria gonorrhoeae	100 (yes)	None
23970	Neisseria lactamica	100 (yes)	None
17960	Oligella urethralis	97 (yes)	APPA (84)

^a1GLM, l-glutamine; APPA, alanine-phenylalanine-proline arylamidase.

^bCI, confidence interval.

Challenge testing.

For the 90 cumulative results from all three trial sites, the challenge set was identified correctly 98% of the time and met all performance criteria set by the clinical trial protocol. The results also fell within the established confidence interval. There were three strains (10%) at one trial site and two strains (7%) at the other two sites that gave low-discrimination results, but these results included the correct identification. There was one strain (3%) at two trial sites that was not identified correctly. At one site Actinobacillus ureae was misidentified as H. influenzae; at the other site Moraxella catarrhalis was misidentified as Neisseria elongata.

Clinical isolate testing.

The clinical isolates tested comprised all 27 of the taxa contained in the NH identification card database. Compared to 16S rRNA gene sequencing as the reference method, 96.5% (358/371) of the clinical isolates were identified correctly by the NH card and met all performance criteria set by the clinical trial protocol across all three sites (Table 2). The cumulative performance by species for all the clinical isolates tested at all three sites is shown in Table 3. The overall correct-identification performance for the new NH card was 96.5%, with only 10.2% low-discrimination results and 2.7% incorrect results. The results also fell within the established confidence intervals.

TABLE 2.

NH card overall performance summary by site for clinical isolate testing

Testing laboratory	Total no. tested by site	% Overall correct identification	% Low discriminationa	% Incorrect identification	% Unidentified
Site A	128	97	11	2	1
Site B	113	98	8	2	0
Site C	130	95	12	4	1
Total	371	96.5	10.2	2.7	0.8

^aData are included as part of percent overall correct identification data in column 3.

TABLE 3.

NH card performance summary by species for the clinical isolate test set for all sites combined

Species or subsp.	Total no. of isolates tested	% Overall correct	% Low discrimination	% Incorrect identification	% Unidentified
Actinobacillus ureae	2	100	0	0	0
Campylobacter coli	4	75	25	25	0
Campylobacter fetus subsp. fetus	2	100	100	0	0
Campylobacter jejuni subsp. jejuni	18	94	6	6	0
Capnocytophaga sp.	5	100	0	0	0
Cardiobacterium hominis	3	100	0	0	0
Eikenella corrodens	13	85	8	0	15
Gardnerella vaginalis	2	100	0	0	0
Haemophilus actinomycetemcomitans	6	100	0	0	0
Haemophilus aphrophilus	13	92	15	8	0
Haemophilus haemolyticus	2	0	0	50	50
Haemophilus influenzae	90	99	6	1	0
Haemophilus parahaemolyticus	7	86	43	14	0
Haemophilus parainfluenzae	41	95	12	5	0
Haemophilus paraphrophilus	4	100	25	0	0
Haemophilus segnis	2	100	100	0	0
Kingella dentrificans	4	100	100	0	0
Kingella kingae	6	100	0	0	0
Moraxella catarrhalis	38	97	5	3	0
Neisseria cinerea	11	91	35	9	0
Neisseria elongata	6	100	17	0	0
Neisseria gonorrhoeae	51	100	6	0	0
Neisseria lactamica	8	100	0	0	0
Neisseria meningitidis	22	100	0	0	0
Neisseria sicca	6	100	17	0	0
Oligella urethralis	4	100	0	0	0
Suttonella indologenes	1	100	0	0	0
Total	371	96.5	10.2	2.7	0.8

In addition to the strains identified in Table 3, a total of 27 isolates in the data set were excluded as unclaimed by the NH card; testing for 25 of these gave an incorrect species result but a correct genus. Of those claimed as Neisseria sicca, 16 were Neisseria flavescens, six were Neisseria macacae/Neisseria mucosa, and two were Neisseria subflava. One isolate with a low-discrimination Campylobacter sp. result (Campylobacter coli/Campylobacter jejuni) was an unclaimed Campylobacter upsaliensis. One strain claimed as H. actinomycetemcomitans was Pasteurella bettyae, and one unidentified strain was an unclaimed Brevundimonas species.

Incorrect and discrepant identifications of claimed species as judged by comparison with 16S rRNA gene sequencing are listed in Table 4. The overall performance was excellent. Clinically only one result might be considered problematic. An isolate identified as Neisseria cinerea with an acceptable Vitek 2 confidence was identified by gene sequencing as C. coli.

TABLE 4.

Incorrect or discrepant identifications of the clinical isolate test set for all sites

NH card identification	Vitek 2 confidence	16S sequence identification
Unidentified	NAa	Haemophilus haemolyticus
Haemophilus parainfluenzae	Excellent	Haemophilus influenzae
Actinobacillus ureae	Very good	Haemophilus parahaemolyticus
Neisseria cinerea	Acceptable	Campylobacter coli
Campylobacter coli	Excellent	Campylobacter jejuni subsp. jejuni
Haemophilus influenzae	Excellent	Haemophilus haemolyticus
Neisseria sicca	Acceptable	Neisseria cinerea
Haemophilus aphrophilus/H. paraphrophilus/H. segnis	Low discrimination	Haemophilus parainfluenzae
Unidentified	NA	Eikenella corrodens
Unidentified	NA	Eikenella corrodens
Haemophilus haemolyticus/H. influenzae	Low discrimination	Haemophilus parainfluenzae
Neisseria elongata	Good	Moraxella catarrhalis
Haemophilus segnis	Excellent	Haemophilus aphrophilus

^aNA, not available in the database.

DISCUSSION

This study was designed to accurately validate the performance of the latest Vitek 2 NH identification card in high-volume tertiary clinical laboratories that perform routine identification of pathogens on automated platforms. The performance of the NH identification card on Vitek 2 met the validation requirements for the three components tested, as described in the study protocol. For the claimed species, there was a high level of performance. With a 95% confidence interval the Vitek 2 NH card gave correct results over 95% of the time at all three laboratory test sites.

With respect to the Neisseria species, the NH card had greater limitations for nonpathogenic Neisseria species than 16S sequence identification. N. sicca is a claimed species but would often be identified as N. flavescens if the gold standard 16S rRNA gene sequencing identification was used. However, in most cases separation as a nonpathogenic Neisseria species would be sufficient to exclude pathogenic strains. This issue is not unique to any of the automated or conventional systems. At the technologist level, the new NH card would misidentify organisms approximately 3% of the time, and currently about 7% of isolates would be unclaimed in the test system.

The NH card was designed to improve significantly the capabilities of the Vitek 2 system to identify a variety of fastidious clinical microorganisms, but particularly Haemophilus and Neisseria species. Clinical trials such as presented here are important to adequately test such new identification systems. The data support the claims of the manufacturer for this new automated identification card.

All microorganism and biochemical combinations having distinct expected reactions of positive or negative in which quality control was evaluated met the performance criteria when measured cumulatively across sites with the exception of four biochemical reactions for three quality control species. As a result of these trials H. actinomycetemcomitans has been removed from the quality control list, leaving only two discrepancies that passed statistical analysis. For the clinical isolates, only a few isolates of some species were included in the evaluation. Haemophilus haemolyticus may be misidentified as H. influenzae or Haemophilus parainfluenzae, unless hemolysis is observed. Also, all four isolates of Kingella denitrificans were identified with low discrimination, although a correct identification was made. Additional strains of some of these species should be evaluated to confirm the conclusions of this investigation.

Overall, the data for which the card has a claim indicate that the NH card meets all performance criteria at greater than 95% confidence. The results from this clinical trial in three large tertiary clinical laboratories indicate that the Vitek 2 NH card is acceptable for routine use in a clinical microbiology laboratory.