Assessment of Reproducibility of Matrix-Assisted Laser Desorption Ionization–Time of Flight Mass Spectrometry for Bacterial and Yeast Identification

Abstract

Matrix-assisted laser desorption ionization–time of flight (MALDI-TOF) mass spectrometry (MS) has revolutionized the identification of clinical bacterial and yeast isolates. However, data describing the reproducibility of MALDI-TOF MS for microbial identification are scarce. In this study, we show that MALDI-TOF MS-based microbial identification is highly reproducible and can tolerate numerous variables, including differences in testing environments, instruments, operators, reagent lots, and sample positioning patterns. Finally, we reveal that samples of bacterial and yeast isolates prepared for MALDI-TOF MS identification can be repeatedly analyzed without compromising organism identification.

TEXT

Matrix-assisted laser desorption ionization–time of flight (MALDI-TOF) mass spectrometry (MS) has revolutionized the identification of microorganisms in the clinical microbiology laboratory, and numerous studies have documented the superior analytical performance characteristics of MALDI-TOF MS compared to those of manual and automated biochemical methods for microorganism identification (1–4). Compared to biochemical-based identification, MALDI-TOF MS-based identification significantly reduces the cost and time to microbial identification while simultaneously enhancing favorable patient outcomes and reducing the cost and length of hospitalization (5–8).

Despite the implementation of MALDI-TOF MS in clinical microbiology laboratories, studies specifically designed to assess the reproducibility of MALDI-TOF MS platforms for microorganism identification are limited. As part of a multicenter study, using the manufacturer's default unmodified settings, we assessed the ability of a MALDI-TOF MS platform, the Vitek MS version 2.0 system (bioMérieux, Durham, NC, USA), to reproducibly identify bacterial and yeast isolates at three different sites. We probed variations in operators, reagent lots, and sample positioning patterns. Finally, the reproducibility of microbial identification after initial analysis (initial read) of the samples compared to repeat analysis (reread) of the same samples was investigated. The rereading of samples may be of importance in instances when the definition of additional microbiological characteristics is required, e.g., in the detection of antimicrobial resistance determinants, or when the initial read failed due to operator or instrument error.

Over five different days, reproducibility studies were performed with three independent Vitek MS version 2.0 systems located at three geographically distinct sites in the United States: site 1 was in Missouri, site 2 was in New York, and site 3 was in California. At each site, two different operators conducted reproducibility testing. All isolates were applied to target slides (bioMérieux) using disposable loops (Sarstedt, Newton, NC, USA). For bacterial isolates, 1 μl of matrix solution (α-cyano-4-hydroxycinnamic acid; bioMérieux) was overlaid and allowed to dry at room temperature before analysis. Prior to the application of 1 μl of matrix solution, yeast isolates were lysed on the target slide by direct application of 0.5 μl of formic acid (25% [vol/vol]; bioMérieux) according to the manufacturer's recommendations. The testing of samples was conducted using a defined algorithm: if an identification result was recorded and data acquisition was acceptable, the identification was accepted. If no identification was recorded (i.e., the organism was not identified) and data acquisition was acceptable, the result was accepted as not identified. Finally, if no identification was recorded due to acquisition of a bad spectrum (typically due to inadequate sample preparation), the sample was reapplied in the same position on a new target slide and the analysis repeated once.

Bacterial isolates were cultured aerobically on Trypticase soy agar with 5% (vol/vol) sheep blood (Thermo Scientific [Remel], Lenexa, KS, USA) for 18 to 24 h at 35 to 37°C, while yeast isolates were cultured aerobically on Sabouraud's dextrose agar, Emmons' modification (Thermo Scientific [Remel]) for 24 to 72 h at 25 to 37°C. A collection of 10 blinded isolates (labeled R1 to R10) was analyzed at all sites on each day of testing. The collection was composed of well-characterized organisms to ensure that organism identification in response to variations in testing environments, instruments, operators, reagents, sample positioning patterns, and initial and repeat reading of samples could be accurately assessed. Therefore, the collection included nine isolates obtained from the American Type Culture Collection (ATCC) (Manassas, VA, USA): Enterobacter aerogenes ATCC 13048 (R1), Escherichia coli ATCC 8739 (R2), Klebsiella pneumoniae (specifically, K. pneumoniae subsp. ozaenae) ATCC 11296 (R3), Proteus mirabilis ATCC 29906 (R4), Pseudomonas aeruginosa ATCC 10145 (R5), Staphylococcus aureus ATCC 29213 (R6), Streptococcus agalactiae ATCC 13813 (R7), Klebsiella oxytoca ATCC 13182 (R8), and Candida glabrata ATCC MYA-2950 (R10); and an isolate from the Centraalbureau voor Schimmelcultures (CBS) (Utrecht, The Netherlands), Candida albicans CBS 5703 (R9). Of the 10 isolates, only two of the isolates were not used to generate the organism database, K. pneumoniae ATCC 11296 and C. albicans CBS 5703, which is a possible limitation of the study.

At each site, two independent testing runs were performed daily (for a total of 10 testing runs over 5 days at each site). In the first run, isolates were applied to the target slide in duplicate (thus, two replicates of each isolate) in sequential order by sample number (e.g., R1 to R10) by one operator, while in a second separate run, the isolates were applied to the target slide, again in duplicate, in random order by a second operator. Hence, a single isolate was applied to a target slide four times a day at each site and 12 times a day at all three sites. For every run the instruments were calibrated with a calibrant strain, E. coli ATCC 8739. Quality control organisms (E. aerogenes ATCC 13048, K. oxytoca ATCC 13182, P. aeruginosa ATCC 10145, S. aureus ATCC 29213, and C. glabrata ATCC MYA-2950) and a negative control (matrix only) were also tested by each technologist every day of testing.

Three different lots of matrix solution and formic acid (reagents) and target slides were used by both operators at each site for sample preparation throughout the 5 days of testing. Reagents and target slides belonging to lot 1 were used a total of four times at each site, while lots 2 and 3 were used at each site a total of three times each. The prepared isolates were analyzed once and data recorded to generate initial read identification results. Subsequently, the isolates were reanalyzed to produce reread results. In situations in which a sample was repeated due to the initial acquisition of a bad spectrum, the repeat result served as the initial and only identification result. In theory, each day a single isolate could be analyzed a total of eight times (four initial reads and four rereads) at an individual site and 24 times (12 initial reads and 12 rereads) at all three sites. Therefore, over 5 days each isolate could be analyzed a total of 40 times (20 initial reads and 20 rereads) at each site and a total of 120 times (60 initial reads and 60 rereads) at all three sites. Collectively, for the 10 isolates analyzed at all sites over the five days of testing, a possible total of 1,200 reads (600 initial reads and 600 rereads) could be recorded. Only species-level identifications were accepted. All statistical calculations were performed using SAS 9.3 (SAS, Cary, NC, USA).

For all sites, 598/600 (99.7%) initial read identification results were correctly identified to the species level. In one instance each, C. albicans and C. glabrata were not identified, but importantly, they were not misidentified (Table 1). Upon initial read, two samples (K. pneumoniae and K. oxytoca) assayed during day three of testing recorded no identification due to acquisition of a bad spectrum. Per our algorithm, the K. pneumoniae and K. oxytoca isolates were reapplied in the same position on another target slide and analyzed, and the results were recorded as the initial and only results. Therefore, of 598 possible reread results, 595 (99.5%) were correctly identified to the species level (Table 2), while one sample of P. mirabilis and two samples of P. aeruginosa were unidentified but not misidentified. For the 1,193 combined initial and reread results that generated an identification, the mean confidence value for organism identification was 99.84% (standard deviation, 1.24%), thus indicating that the confidence values for organism identification were reproducibly high. These data clearly indicate that initial and reread organism identification using the Vitek MS version 2.0 system is highly reproducible.

TABLE 1.

Reproducibility of organism identification at all sites after initial read of samples

Sample no.	Organism	No. identified/total no. (%) on day:					No. identified/total no. (%)	95% confidence interval (%)
		1	2	3	4	5
R1	E. aerogenes	12/12	12/12	12/12	12/12	12/12	60/60 (100)	94.0–100
R2	E. coli	12/12	12/12	12/12	12/12	12/12	60/60 (100)	94.0–100
R3	K. pneumoniae	12/12	12/12	12/12	12/12	12/12	60/60 (100)	94.0–100
R4	P. mirabilis	12/12	12/12	12/12	12/12	12/12	60/60 (100)	94.0–100
R5	P. aeruginosa	12/12	12/12	12/12	12/12	12/12	60/60 (100)	94.0–100
R6	S. aureus	12/12	12/12	12/12	12/12	12/12	60/60 (100)	94.0–100
R7	S. agalactiae	12/12	12/12	12/12	12/12	12/12	60/60 (100)	94.0–100
R8	K. oxytoca	12/12	12/12	12/12	12/12	12/12	60/60 (100)	94.0–100
R9	C. albicans	12/12	12/12	11/12	12/12	12/12	59/60 (98.3)	91.1–99.9
R10	C. glabrata	12/12	12/12	11/12	12/12	12/12	59/60 (98.3)	91.1–99.9
Total		120/120 (100)	120/120 (100)	118/120 (98.3)	120/120 (100)	120/120 (100)	598/600 (99.7)	98.8–99.9

TABLE 2.

Reproducibility of organism identification at all sites after reread of samples

Sample no.	Organism	No. identified/total no. (%) on day:					No. identified/total no. (%)	95% confidence interval (%)
		1	2	3	4	5
R1	E. aerogenes	12/12	12/12	12/12	12/12	12/12	60/60 (100)	94.0–100
R2	E. coli	12/12	12/12	12/12	12/12	12/12	60/60 (100)	94.0–100
R3	K. pneumoniae	12/12	12/12	11/11a	12/12	12/12	59/59 (100)	93.9–100
R4	P. mirabilis	12/12	12/12	12/12	11/12	12/12	59/60 (98.3)	91.1–99.9
R5	P. aeruginosa	11/12	12/12	12/12	11/12	12/12	58/60 (96.7)	88.5–99.6
R6	S. aureus	12/12	12/12	12/12	12/12	12/12	60/60 (100)	94.0–100
R7	S. agalactiae	12/12	12/12	12/12	12/12	12/12	60/60 (100)	94.0–100
R8	K. oxytoca	12/12	12/12	11/11a	12/12	12/12	59/59 (100)	93.9–100
R9	C. albicans	12/12	12/12	12/12	12/12	12/12	60/60 (100)	94.0–100
R10	C. glabrata	12/12	12/12	12/12	12/12	12/12	60/60 (100)	94.0–100
Total		119/120 (99.2)	120/120 (100)	118/118 (100)	118/120 (98.3)	120/120 (100)	595/598 (99.5)	98.5–99.9

^aFor one of these samples, the initial read recorded no identification due to acquisition of a bad spectrum. The sample was reapplied in the same position on a new target slide and analyzed. The repeat result was recorded as the initial and only identification.

To ensure there were no obvious differences in reproducibility testing between the sites, the results of testing at each site were analyzed separately. For site 1, 199/200 (99.5%) and 200/200 (100%) of the identification results were correct to the species level after initial and reread testing, respectively. At site 2, 200/200 (100%) of the initial and 195/198 (98.5%) of the reread results were correct to the species level. Finally, 199/200 (99.5%) and 200/200 (100%) of the identification results were correct to the species level at site 3 after initial and reread testing, respectively. Using Fisher's exact test to analyze the combined initial and reread results, no significant difference in organism identification between the testing sites was observed at the 0.05 level of significance (P = 0.463).

Finally, given that reproducible quality control testing forms the foundation of all testing in clinical microbiology, we sought to understand the reproducibility of the quality control testing at all sites. Upon initial reading, all quality control organisms were correctly identified to the species level (Table 3), while after rereading, with the exception of one K. oxytoca sample and two P. aeruginosa samples, which were unidentified rather than misidentified, all quality control organisms were correctly identified to the species level (Table 3). When analyzed initially, 30/31 (96.8%) of the negative control (matrix only) samples were not identified, i.e., the expected result was achieved. For one of the samples, a Vitek MS version 2.0 result of mixed genera (Corynebacterium pseudodiphtheriticum, Tatumella ptyseos, Bacillus megaterium, and Streptococcus salivarius subsp. thermophilus), which is considered unidentified, was obtained. The reread result for this sample was no identification. The initial discordant result may have been due to extraneous debris on the spot. To unequivocally demonstrate the absence of contaminating material, the initial testing operator applied matrix from the same vial to a target slide. The expected result of no identification after both the initial read and reread of the sample was obtained, confirming the matrix was not contaminated. Unfortunately, due to operator error one of the 31 negative control samples was not reread; consequently, of 30 possible negative control reread results, 30 (100%) were not identified (expected result). For the 301 combined initial read and reread results that generated an organism identification, the mean confidence value for organism identification was 99.9% (standard deviation, 0.008%), indicating that the confidence values for quality control organism identification were reproducibly high. Using Fisher's exact test to analyze the combined initial read and reread results, no significant difference in quality control testing between the testing sites was observed at the 0.05 level of significance (P = 0.175). Based on these data, it can be concluded that quality control testing on the Vitek MS version 2.0 system is highly reproducible.

TABLE 3.

Reproducibility of quality control testing at all sites after initial read and reread of samples

Organism	Initial reads		Rereads
	No. identified/total no. (%)	95% confidence interval (%)	No. identified/total no. (%)	95% confidence interval (%)
S. aureus	31/31 (100)	88.8–100	31/31 (100)	88.8–100
K. oxytoca	30/30 (100)	88.4–100	29/30 (96.7)	82.8–99.9
P. aeruginosa	30/30 (100)	88.4–100	28/30 (93.3)	77.9–99.2
E. aerogenes	30/30 (100)	88.4–100	30/30 (100)	88.4–100
C. glabrata	31/31 (100)	88.8–100	31/31 (100)	88.8–100
Negative control (matrix only)	30/31 (96.8)a	83.3–99.9	30/30 (100)	88.4–100

^aFor one of these samples, a result of mixed genera (C. pseudodiphtheriticum, T. ptyseos, B. megaterium, and S. salivarius subsp. thermophilus), which is considered unidentified, was obtained.

Prior to this report, data describing the reproducibility of microbial identification using MALDI-TOF MS were limited. To the best of our knowledge, only a single multicenter study focused on the reproducibility of microbial identification using MALDI-TOF MS has been described (9). The study concentrated on the ability of different Bruker MALDI-TOF MS platforms (Bruker Daltonics, Billerica, MA, USA) to identify Gram-negative nonfermentative bacterial isolates at different testing sites. Despite the importance of the work, yeast isolates were not assayed. In addition, differences in reagent lots, target slides, sample positioning patterns, and initial read and reread of samples do not appear to have been tested.

Here, we describe the reproducibility of organism identification using the Vitek MS version 2.0 system. We unambiguously demonstrate that bacterial and yeast identification using the Vitek MS version 2.0 system is highly reproducible and can tolerate numerous variables, including different testing environments, instruments, operators, reagent and target slide lots, and sample positioning patterns. Furthermore, we show that both initial and repeat testing of samples is not only possible but highly reproducible within and between laboratories. This has important practical implications in a diagnostic microbiology laboratory where subsequent analysis of an isolate may be required to further define key microbiological characteristics, e.g., in the detection of antimicrobial resistance determinants, or as a result of operator or instrument error. In conclusion, MALDI-TOF mass spectrometric-based microbial identification is highly reproducible in the setting of the routine clinical microbiology laboratory.