Accuracy of Matrix-Assisted Laser Desorption Ionization–Time of Flight Mass Spectrometry for Identification of Clinical Pathogenic Fungi: a Meta-Analysis

Abstract

Fungal infections in the clinic have become increasingly serious. In many cases, the identification of clinically relevant fungi remains time-consuming and may also be unreliable. Matrix-assisted laser desorption ionization–time of flight mass spectroscopy (MALDI-TOF MS) is a newly developed diagnostic tool that is increasingly being employed to rapidly and accurately identify clinical pathogenic microorganisms. The present meta-analysis aimed to systematically evaluate the accuracy of MALDI-TOF MS for the identification of clinical pathogenic fungi. After a rigorous selection process, 33 articles, involving 38 trials and a total of 9,977 fungal isolates, were included in the meta-analysis. The random-effects pooled identification accuracy of MALDI-TOF MS increased from 0.955 (95% confidence interval [CI], 0.939 to 0.969) at the species level to 0.977 (95% CI, 0.955 to 0.993) at the genus level (P < 0.001; χ² = 15.452). Subgroup analyses were performed at the species level for several categories, including strain, source of strain, system, system database, and modified outcomes, to calculate the accuracy and to investigate heterogeneity. These analyses revealed significant differences between the overall meta-analysis and some of the subanalyses. In parallel, significant differences in heterogeneity among different systems and among different methods for calculating the identification ratios were found by multivariate metaregression, but none of the factors, except for the moderator of outcome, was significantly associated with heterogeneity by univariate metaregression. In summary, the MALDI-TOF MS method is highly accurate for the identification of clinically pathogenic fungi; future studies should analyze the comprehensive capability of this technology for clinical diagnostic microbiology.

INTRODUCTION

Pathogenic fungi have been increasingly detected in clinical microbiological laboratories in recent years. Invasive fungi have received growing attention for their potentially life-threatening pathogenicity (1), especially in patients undergoing transplants or receiving treatment for malignancies (2). Therefore, the development of precise, rapid, and cost-effective methods to identify clinically relevant fungi appears crucial. Unfortunately, the identification of yeast and yeast-like fungi, as well as filamentous fungi, in many clinical laboratories still mainly depends on phenotypic or molecular methods that are time-consuming, labor-intensive, and often inconclusive (3–5). As an alternative to these standard identification methods, the rapid, cost-effective, and accurate (6, 7) method matrix-assisted laser desorption ionization–time of flight mass spectroscopy (MALDI-TOF MS) has been widely used in recent years in European clinical microbiology laboratories for the identification of bacteria, mycobacteria, and fungi (8, 9). The technology is now being adopted in clinical laboratories worldwide (10) due to its superiority over traditional methods for some tasks. The clearance of the Vitek MS, a product of bioMérieux, for clinical use by the State Food and Drug Administration (SFDA) of China in 2012 and by the U.S. FDA in 2013 (11) paves the way for many more clinical laboratories in these two countries to adopt MALDI-TOF MS to identify clinical pathogens. Because of its promise, some experts even describe MALDI-TOF MS technology as “a revolution in clinical microbiology” (12). Currently, four commercial systems are in use worldwide: the MALDI Biotyper (Bruker Daltonics, Bremen, Germany), the Saramis (AnagnosTec, Potsdam, Germany), and, more recently, the Andromas (Andromas, Paris, France) and Vitek MS (bioMérieux, Marcy l'Etoile, France) systems (13). Each of the four systems is equipped with a MALDI-TOF MS instrument from either Bruker Daltonics or Shimadzu (9). Among them, the MALDI Biotyper is commonly used in conjunction with an instrument from the same manufacturer. The Saramis, which is mainly combined with an Axima instrument (a product of Shimadzu), was purchased by bioMérieux in 2010 to be incorporated into the Vitek MS series (10). Finally, the Andromas system, including three distinct databases, which can be used in conjunction with either Bruker or Shimadzu instruments, was primarily used in France (8, 10). Although many articles have reported bacterial identification by MALDI-TOF MS and have compared available systems (14–16), studies comparing or using MALDI-TOF MS systems for the identification of clinically related fungi have been relatively rare. In addition, many studies have included only a few strains, and some results have been inconsistent (17, 18). Therefore, the present work aimed to analyze the gross accuracy of MALDI-TOF MS using different systems to identify clinically pathogenic fungi by performing a meta-analysis that synthesizes large amounts of data to improve the reliability of the results.

MATERIALS AND METHODS

Search strategy.

We queried PubMed (up to 18 October 2013) with the string “(maldi-ms [MeSH Terms] AND fungi [MeSH Terms]) AND (identification [Title/Abstract] OR detection [Title/Abstract])” to identify relevant articles. The Embase database was also searched (before 23 October 2013) with the words “maldi,” “fungi,” “identification,” and “detection.” The language, publication status, and geographical distribution of the publications were not restricted. In addition, we scanned the references of the eligible studies and reviews that were identified. The authors of the original studies were contacted for detailed information if the full text could not be obtained from the database. The meta-analysis was performed by referring to (when appropriate) the PRISMA guidelines (19). EndNote X4 (Thomson Reuters) was used for literature management.

Two investigators (H.L. and J.S.) independently performed the literature search and data extraction. Disagreements were resolved by discussion and/or consultation with a third researcher (Z.W.).

Study selection criteria and data extraction.

Studies evaluating the accuracy of MALDI-TOF MS for identification of clinical fungi by comparison with reference methods were considered eligible for the meta-analysis. Articles relating to the validation of the MALDI-TOF MS fungal database expanded by researchers or to the evaluation of commercial databases were included. Data on the identification accuracy of MALDI-TOF MS for fresh/frozen clinical isolates and isolates previously confirmed by gold standard methods were all included.

Studies or data were excluded if they fell into the following categories: studies with no abstract; studies on technological innovations, such as modification of the preanalytical steps of MALDI-TOF MS identification; studies applying MALDI-TOF MS to the identification of industrial/environmental microorganisms or plant- or animal-pathogenic microbes; studies using MALDI-TOF MS for the identification of clinical nonfungal pathogens or studies identifying clinical pathogens by mass spectroscopy methods other than MALDI-TOF; studies with fewer than 40 specimens, with reference methods that did not include molecular biology, or lacking a comparator method or gold standard; studies on drug resistance; and case reports or reviews.

Data abstraction was conducted using the numbers of total isolates and of isolates correctly identified at the genus and species levels in comparison to reference methods, when pertinent data were available. Data were separately collected according to the category of strain, the MS system used (Andromas, Biotyper, Saramis, Vitek MS), and the calculation method used for identification ratios (whether those data were excluded in the absence of referential mass spectral entries). In addition, the following information was abstracted: the first author, publication year, study design (prospective or retrospective), source of strains (clinical isolates only or reference strains added), database of system (commercial database only or self-established database added), threshold, geographical distribution of strains, blinded status, and reference methods. When two or more thresholds were included in a study, the one by which more fungi could be correctly identified, compared to the reference method, was abstracted.

Quality assessment.

The following modified criteria, referring to the quality assessment for studies of diagnostic accuracy (QUADAS) (20), were used to assess the quality of original studies: study design, category and geographical distribution of strains, blinded status, reference methods, threshold, strain source, and system database.

Statistical analysis.

The major effect-size index consisted of the correct identification ratios. Specifically, the identification ratios obtained after MALDI-TOF MS identification were compared to results obtained with the reference method used in each of the studies, as some researchers have found that using a lower threshold can in many cases provide a higher identification ratio, compared to the reference method, than that specified by the manufacturer (21, 22). The identification ratio was calculated as the number of correctly identified isolates divided by the total number of isolates. Before the synthesis, the data were preprocessed. To obtain the overall performance of MALDI-TOF MS for accurate identification of clinical fungal isolates, we averaged the both identification ratios when two identification systems were employed to identify the same strains in a study. We then split the identification ratios on different systems to analyze the identification performance of an individual system. Given that the data to be analyzed were nonnormally distributed, double arcsine transformation (23) was implemented before data synthesis. The transformation results in a roughly normally distributed variable (24, 25) and makes the variance more stable than the “canonical” logit transformation for ratios (24, 26).

The double arcsine-transformed ratios were subsequently pooled in fixed-effect and random-effects models (27). To better understand the results, pooled transformed estimate formulas were back-transformed into the “original units” of ratios (25). The random-effects model pooled ratios were adopted when significant heterogeneity was present. Otherwise, fixed-effect model pooled ratios were adopted. Subgroup analyses at the species level were performed on the following categories: strain, source of strain, system, and system database. In addition, the highest correct identification ratios (data were excluded in the absence of referential mass spectral entries) performed on the four systems and on the Biotyper were also analyzed separately, as most uncommon fungal species can in principle be correctly identified by the enrichment of reference spectra in the library. Some exceptions include certain species for which satisfactory spectra are inherently difficult to obtain, such as Candida guilliermondii or Cryptococcus neoformans. Comparison of identification accuracy at the genus and species levels, as well as comparisons between subanalyses with various situations and the overall species-level analysis, was performed using a Pearson chi-square test.

To analyze possible sources of high-level heterogeneity and to validate the effects of subanalyses, multivariate and univariate metaregressions were performed (27, 28) with the default DerSimonian-Laird method (29). Interpretive parameters of univariate metaregressions were provided to enable comparisons across identifications. An influence (sensitivity) analysis (29) with the random-effects model for the enrolled articles at the species level was undertaken to inspect the influence of individual research on the overall identification accuracy.

Heterogeneity between studies was estimated with Cochran's Q statistic and the I² measure. Publication bias was evaluated by using the rank correlation method of Begg and Egger's regression method (29). Statistical significance was defined as a P value of <0.1 and an I² value of <50% for the qualitative Cochran's Q and quantitative I² measures, respectively. For metaregression, for the Pearson chi-square test and for publication bias interpretations, statistical significance was set as a P value of <0.05. All analyses were performed with R statistical software (version 3.0.2; R Foundation for Statistical Computing, Vienna, Austria) and with the analysis packages meta (version 3.1-2) and metafor (version 1.9-2).

RESULTS

Eligible studies.

A total of 282 items were obtained by searching PubMed with defined retrieval strings. After additional Embase retrieval and duplicate removal, 2 other citations were included. A total of 242 articles were excluded after title and/or abstract review. Among the excluded articles, 2 were excluded because no abstract was provided; 1 case report and 1 report with fewer than 40 strains were excluded; 4 were excluded due to their relating to drug resistance; 15 were excluded due to their focus on technological innovation; 130 were excluded because they reported the use of MALDI-TOF MS in non-clinically related research; 35 were rejected due to a focus on the application of MALDI-TOF to clinically related research other than the identification of clinical fungi; and 27 were discarded because they reported the use of a mass spectrometry technique other than MALDI-TOF or because they had no relationship with both MALDI-TOF and the identification of clinical fungi. In addition, 11 were excluded because they did not focus on identification accuracy despite being related to the identification of clinical fungi by MALDI-TOF, and 1 was rejected because it was not feasible to translate the article into English. After the papers were screened, 42 citations remained for full-text examination. Another 9 studies were excluded after full-text scanning, as they did not meet the inclusion criteria as described (e.g., the absence of a reference method or the absence of molecular biology). As a result, 33 articles involving 38 trials were included in this meta-analysis. Among these articles, 4 (7, 8, 13, 18) reported on the identification performance of the Andromas system, 24 (1, 4, 13, 17, 21, 22, 30–47) reported on the identification performance of the Biotyper system, 6 (21, 30, 42, 48–50) reported on the identification performance of the Saramis system, and 4 (36, 51–53) reported on the identification performance of the Vitek MS system. Notably, 5 were comparison studies between two systems (three were between the Biotyper and the Saramis, one was between the Biotyper and the Andromas, and another was between the Biotyper and the Vitek MS).

Quality of studies.

The major characteristics of the enrolled eligible studies are listed in Table 1. Among the 33 enrolled studies, 16 (48.48%) were prospective, 12 (36.36%) were retrospective, and 5 (15.15%) included both types of data. Four (12.12%) studies (22, 39, 41, 49) included reference strains, while the remaining studies (87.88%) used all clinical isolates. Eight (24.24%) studies (4, 7, 18, 31, 34, 35, 41, 44) expanded the database by establishing reference spectra for clinical fungi, while others (75.76%) used the databases provided by the instrument suppliers. The strains analyzed in most of the studies (66.67%, or 22/33) were isolated from Europe. Among them, 8 studies used strains isolated from France and 2 used isolates from multiple European countries; 5 (15.15%) used isolates from America and 4 (12.12%) used isolates from Asia; among the rest, one used isolates from 3 countries on different continents and another one used isolates from the ARTEMIS and SENTRY programs involving 22 countries. Only 8 (24.24%) articles clearly stated that a blind method was used throughout the studies; one indicated that a blind method was used for the prospective part of their investigation; the others did not indicate whether a blind method was used for their studies. Twenty-seven studies were focused on the identification of yeasts, and 8 focused on mold identification. Five out of 33 articles (31, 32, 39, 48, 51) did not report thresholds for identification. The majority of the included studies (81.82%, or 27/33) employed two or more methods that acted as references. DNA sequencing of all isolates was performed in only 3 of the 5 studies employing molecular methods as a reference (22, 35, 52), despite the fact that molecular methods are considered the gold standard for the identification of pathogens.

TABLE 1.

Characteristics of the 33 reports enrolled in this meta-analysis

Study	Study design	Organism(s)	Geographical distribution of strains	Blinded statusa	Reference method(s)b
Marinach-Patrice et al., 2009 (37)	Retrospective	Molds	France and Belgium	NR	MO and MB
Marklein et al., 2009 (38)	Retrospective plus prospective	Yeasts	Germany	NR	MO, BI, and MB
Stevenson et al., 2010 (44)	Retrospective	Yeasts	USA	NR	BI and MB
van Veen et al, 2010 (45)	Retrospective plus prospective	Yeasts	Netherlands	NR	BI and MB
Alanio et al., 2011 (7)	Prospective	Aspergillus	France	Yes	MO and MB
Bader et al., 2011 (30)	Prospective	Yeasts	Germany	Yes	MO, BI, and MB
Cassagne et al., 2011 (31)	Prospective	Molds	France	NR	MO and MB
Dhiman et al., 2011 (33)	Retrospective plus prospective	Yeasts	USA	Yes	BI and MB
Martinez-Lamas et al., 2011 (50)	Retrospective	Candida	Spain	NR	BI and MB
McTaggart et al., 2011 (22)	Retrospective	Cryptococcus and non-Cryptococcus yeasts	Canada	NR	MB
Pinto et al., 2011 (40)	Retrospective plus prospective	Candida and non-Candida yeasts	Australia	Yes for prospective part	BI and MB
Putignani et al., 2011 (1)	Retrospective plus prospective	Candida and non-Candida yeasts	Italy	NR	BI and MB
Yan et al., 2011 (47)	Prospective	Yeasts	USA	Yes	MO, BI, and MB
Alshawa et al., 2012 (18)	Prospective	Molds	France	Yes	MO and MB
Bille et al., 2012 (8)	Prospective	Aspergillus and yeasts	France	NR	BI and MB
De Carolis et al., 2012 (4)	Prospective	Molds	Italy	Yes	MO and MB
Firacative et al., 2012 (34)	Prospective	Cryptococcus	Australia	NR	MB
Iriart et al., 2012 (51)	Prospective	Aspergillus and yeasts	France	NR	MO, BI, and MB
Posteraro et al., 2012 (41)	Retrospective	Cryptococcus	Italy	NR	MB
Quiles-Melero et al., 2012 (39)	Retrospective	Candida	Spain	NR	BI and MB
Seyfarth et al., 2012 (49)	Retrospective	Yeasts	Germany	NR	BI and MB
Yaman et al., 2012 (46)	Prospective	Candida	Turkey	NR	BI and MB
Castanheira et al., 2013 (32)	Retrospective	Candida	22 countries	NR	BI and MB
Ferreira et al., 2013 (17)	Retrospective	Yeasts	Spain	NR	MO, BI, and MB
Kolecka et al., 2014 (35)	Prospective	Malassezia yeasts	Greece, Italy and Sweden	NR	MB
Lacroix et al., 2014 (13)	Prospective	Candida	France	Yes	BI and MB
Lohmann et al., 2013 (21)	Prospective	Yeasts	France	NR	MO, BI, and MB
Mancini et al., 2013 (36)	Retrospective	Candida, Candida-related and non-Candida yeasts	Italy	Yes	BI and MB
Nenoff et al., 2013 (48)	Retrospective	Molds	Germany, Uganda and China	NR	MO and MB
Rosenvinge et al., 2013 (42)	Retrospective	Yeasts	Denmark	NR	BI and MB
Sendid et al., 2013 (43)	Prospective	Yeasts	France	NR	MO, BI, and MB
Westblade et al., 2013 (52)	Prospective	Candida and non-Candida yeasts	North America	NR	MB
Won et al., 2013 (53)	Prospective	Yeasts	South Korea	NR	BI and MB

^aNR, no report.

^bMO, morphology; MB, molecular biology; BI, biochemistry.

Overall meta-analysis.

In the 33 enrolled studies, a total of 9,977 fungal isolates (8,842 yeast isolates and 1,135 mold isolates) were included. Forest plots of random-effects and fixed-effect models were used to summarize the overall statistical results of the meta-analysis at the genus and species levels (Fig. 1 and 2).

FIG 1.

Forest plot for the meta-analysis of the overall identification ratio at the genus level. CI, confidence interval; W, weight; fixed, fixed-effect model; random, random-effects model; events, number of correct identifications; total, total number of identifications. Gray squares represent the weight of individual studies with the fixed-effect model; horizontal lines through the squares represent 95% confidence intervals; gray diamonds represent the overall estimate and its confidence interval; dotted vertical lines represent the fixed-effect model; and dashed vertical lines represent the random-effects model.

FIG 2.

Forest plot for the meta-analysis of the overall identification ratio at the species level. CI, confidence interval; W, weight; fixed, fixed-effect model; random, random-effects model; events, number of correct identifications; total, total number of identifications.

The overall correct identification ratios of MALDI-TOF MS for clinical pathogenic fungi ranged from 0.91 to 1.00 at the genus level and from 0.81 to 1.00 at the species level. Moderate heterogeneity was found at the genus level (Q = 18.10 [P = 0.003]; I² = 72.4% [95% CI = 36.3% to 88.0%]), and significant heterogeneity was found at the species level (Q = 352.36 [P < 0.0001]; I² = 90.9% [95% CI = 88.3% to 92.9%]). The random-effects pooled identification accuracy of MALDI-TOF MS increased from 0.955 with a 95% CI of 0.939 to 0.969 at the species level to 0.977 with a 95% CI of 0.955 to 0.993 at the genus level (P < 0.001 and χ² = 15.452).

Subgroup meta-analyses and investigation of heterogeneity.

The heterogeneity and random-effects or fixed-effect pooled ratios of subgroup analyses performed at the species level on categories including strain (see Fig. S1 in the supplemental material), strain source (clinical isolates only or reference strains also) (see Fig. S2 in the supplemental material), system database (commercial database only or self-established database also) (see Fig. S3 in the supplemental material), system (see Fig. S4 in the supplemental material,) and modified outcomes (namely, the highest correct identification ratios defined in the statistical analysis) from all systems and the Biotyper (Fig. 3 and 4) are listed in Table 2.

FIG 3.

Forest plot for the subanalysis of modified outcomes from all systems at the species level. CI, confidence interval; W, weight; fixed, fixed-effect model; random, random-effects model; events, number of correct identifications; total, total number of identifications.

FIG 4.

Forest plot for the subanalysis of modified outcomes from the Biotyper system at the species level. CI, confidence interval; W, weight; fixed, fixed-effect model; random, random-effects model; events, number of correct identifications; total, total number of identifications.

TABLE 2.

The heterogeneity and pooled correct identification ratios by subgroup analyses

Subanalysis	No. of isolates (no. of studies)	Within-group heterogeneity		Correct identification ratio (95% CI)a	P comparison with overall ratio (χ2 value)
		P (Q value)	I2 (95% CI [%])
Category of strains
Yeasts	8,842 (27)	<0.0001 (291.38)	91.1 (88.2–93.2)	0.959 (0.943–0.973)	0.129 (2.309)
Molds	1,135 (8)	<0.0001 (45.71)	84.7 (71.6–91.7)	0.934 (0.888–0.969)	0.002 (9.961)
Systems
Andromas	2,130 (4)	<0.0001 (33.68)	91.1 (80.3–96.0)	0.972 (0.936–0.994)	<0.001 (13.202)
Biotyper	7,289 (24)	<0.0001 (303.10)	92.4 (89.9–94.3)	0.954 (0.933–0.971)	0.725 (0.124)
Saramis	1,926 (6)	<0.0001 (63.71)	92.2 (85.7–95.7)	0.938 (0.881–0.978)	0.001 (10.477)
Vitek MS	1,818 (4)	<0.0001 (31.79)	90.6 (78.9–95.8)	0.933 (0.887–0.968)	<0.001 (16.923)
Source of strains
Clinical isolates only	9,549 (29)	<0.0001 (332.42)	91.6 (89.0–93.5)	0.950 (0.932–0.965)	0.097 (2.755)
Clinical isolates plus reference strains	428 (4)	0.0304 (8.92)	66.4 (1.5–88.5)	0.988 (0.960–1.000)	0.001 (10.962)
System database
Commercial database only	8,527 (25)	<0.0001 (282.37)	91.5 (88.7–93.6)	0.955 (0.937–0.970)	0.988 (0)
Commercial database plus self-established database	1,450 (8)	<0.0001 (63.44)	89.0 (80.6–93.7)	0.955 (0.916–0.983)	0.962 (0.002)
Modified outcomes
All systems	3,366 (11)	0.0003 (32.41)	69.1 (42.4–83.5)	0.987 (0.978–0.994)	<0.001 (73.927)
Biotyper	2,501 (8)	0.1269 (11.28)	37.9 (0–72.6)	0.992b (0.987–0.995)	<0.001 (74.183)

^aRandom-effects pooled ratios, except where noted otherwise.

^bFixed-effect pooled ratio.

The heterogeneity was not obviously decreased in subgroup meta-analyses, except in subanalyses on isolates with reference strains added and on modified outcomes performed on all systems and on the Biotyper. The heterogeneity decreased from a significant level (Q = 352.36 [P < 0.0001] and I² = 90.9% [95% CI = 88.3% to 92.9%] for the overall meta-analysis) to a moderate level (Q = 8.92 [P = 0.0304] and I² = 66.4% [95% CI = 1.5% to 88.5%] for subanalysis of isolates with reference strains added; Q = 32.41 [P = 0.0003] and I² = 69.1% [95% CI = 42.4% to 83.5%] for subanalysis of modified outcomes performed on all systems) or a low level (Q = 11.28 [P = 0.1269] and I² = 37.9% [95% CI = 0 to 72.6%] for subanalysis of modified outcomes performed on the Biotyper system).

There were significant differences in the correct identification ratios at the species level (P < 0.05 for each comparison) between the overall meta-analysis and some of the subanalyses, including mold isolates, the identification system (Andromas, Saramis, and Vitek MS systems), isolates identified in studies with reference strains added, and modified outcomes from all systems and of the Biotyper system (Table 2). No significant differences (P > 0.05 for each comparison) were observed between the overall meta-analysis and the other subanalyses. The correct identification performances of the subanalyses on the Andromas system, on isolates with reference strains added, and on modified outcomes were superior to the overall ratio, while those of the subanalyses on mold identification and on the Saramis and Vitek MS systems were inferior to the overall ratio in this meta-analysis.

Slight significant differences (P = 0.049) in heterogeneity were found among different systems through multivariate metaregression with 4 moderators: strain (yeasts versus molds), system, source (source of strain), and database (database of system). Still, with 5 moderators (strain, system, source, database, and outcome [namely, the calculation method for identification ratios represented in the study selection criteria and data extraction]), significant differences in heterogeneity among different systems (P = 0.026) and among different calculation methods (P = 0.004) were discovered. The results of multivariate metaregression confirmed the univariate associations in all cases except for the moderator of system, which showed a significant difference by multivariate metaregression but no significant difference (P = 0.07) by univariate metaregression. None of the factors, except for the moderator of outcome, was significantly associated with heterogeneity by univariate metaregression (Table 3). Influence analysis (Fig. 5) showed that no individual study had any obvious influence on the combined overall ratio at the species level; this analysis was performed by inspecting pooled estimates that were calculated by omitting one study at a time (29).

TABLE 3.

Univariate metaregression for ratios of correct identification

Moderator	Metaregression coefficient	95% CI	P
Source	0.1231	−0.0693 to 0.3156	0.2099
Strain	−0.0737	−0.2091 to 0.0617	0.2863
Database	0.0214	−0.1115 to 0.1543	0.7522
System	−0.0589	−0.1225 to 0.0048	0.0700
Outcome	0.1493	0.0382 to 0.2604	0.0084

FIG 5.

Influence analysis with a random-effects model for the enrolled articles at the species level. CI, confidence interval.

Assessment of publication bias.

Begg rank correlation (with continuity correction) and Egger's linear regression test of funnel plot asymmetry at the species level showed that little publication bias was detected in this meta-analysis (z = −0.341 and P = 0.733 for Begg; t = −1.576 and P = 0.125 for Egger's).

DISCUSSION

As a newly developed technology for the identification and antimicrobial resistance analysis of clinical pathogens (54, 55), MALDI-TOF MS has shown many merits, as described above. This meta-analysis highlights the performance of the four available MALDI-TOF MS systems for the accurate identification of clinical pathogenic fungi at both the genus and species levels. The evaluation results demonstrate that MALDI-TOF MS is a highly accurate technology for clinical fungus identification, with a correct identification proportion of 0.955 (95% CI = 0.939 to 0.969) at the species level and 0.977 (95% CI = 0.955 to 0.993) at the genus level. Among individual systems, the Andromas system yielded the highest performance in the four synthesized studies enrolled in this meta-analysis, showing an accuracy of 0.972 (95% CI = 0.936 to 0.994) at the species level. However, it is still uncertain whether the accuracy of the Andromas system for the identification of clinical fungi is generally superior to those of the other three systems. This uncertainty remains because there are currently no sufficiently direct comparisons between the Andromas system and other systems (13), and the identification ratios can be affected by many factors (for example, the category of strain, the proportion of common and unusual species, or the reference library version) that were revealed by the severe heterogeneity in the pooled ratios. It comes as no surprise that the identification accuracy of the Vitek MS was similar to that of the Saramis (0.933 [95% CI, 0.887 to 0.968] for the Vitek MS versus 0.938 [95% CI, 0.881 to 0.978] for the Saramis), as the former was developed on the basis of the latter (36).

From a professional standpoint, molds are more difficult to identify than yeasts in clinical practice (37). Still, an identification accuracy of 0.934 with a 95% CI of 0.888 to 0.969 revealed by MALDI-TOF MS in 8 pooled studies with 1,135 various mold isolates indicates that MALDI-TOF MS is a good method for the accurate and rapid identification of pathogenic molds, despite the fact that it was still a relatively low correct identification ratio compared to that for yeasts. At the same time, we noticed that the identification capability of MALDI-TOF MS against common and unusual fungal isolate species was variable. This observation is due not only to the inherent difficulty of obtaining satisfactory spectra from some species, such as Candida guilliermondii or Cryptococcus neoformans, but also to the insufficient numbers of spectra for uncommon fungal species in commercial reference libraries (at least in their previous versions). Thus, it will be increasingly important to update these libraries to continuously enrich them with fungal strains and species that are absent or poorly represented in their current versions, although there was no significant difference (P = 0.962) in the correct identification ratios at the species level between the overall meta-analysis and the subanalysis with researcher-expanded databases. Fortunately, commercial databases are continuously improved and updated at approximately 3- to 6-month intervals (12).

Subanalysis of studies in which the clinical strains were supplemented with reference strains decreased the heterogeneity to 66.4% with a 95% CI of 1.5% to 88.5% and improved the accuracy to 0.988 with a 95% CI of 0.960 to 1.000 in the four enrolled studies (428 isolates), implying that strains with substantive and distinct properties can be more easily identified by MALDI-TOF MS than those with inherently ambiguous properties. For example, only 66% of Candida guilliermondii and 50% of Cryptococcus neoformans isolates can be correctly identified by MALDI-TOF MS (33, 44). The pooled correct identification ratios were raised to 0.987 with a 95% CI of 0.978 to 0.994 and to 0.992 with a 95% CI of 0.987 to 0.995 by the subanalyses with modified outcomes from the four systems and the Biotyper system, respectively, while the heterogeneity was reduced to 69.1% with a 95% CI of 42.4% to 83.5% and 37.9% with a 95% CI of 0 to 72.6%, respectively. These results indicate that the gross identification capability of MALDI-TOF MS for clinical pathogenic fungi can be further improved by updating the databases with more mass spectra for unusual species.

Clinically, it is often not feasible to apply more elaborate, bivariate meta-analysis models to the evaluation of identification instruments for clinical pathogens due to a lack of direct comparisons; thus, the synthesis of single ratios is unavoidably adopted (27, 56). Still, the stability in a meta-analysis of single ratios is different from that of bivariate meta-analysis, and in many cases, the heterogeneity among individual ratios is very severe, as has been demonstrated in many previously published articles on single ratios (27, 28, 57). Such seemingly inherent and significant heterogeneity also appeared in this meta-analysis; nevertheless, subanalyses from several clinical aspects were performed.

Many fungal diseases have a worldwide distribution, whereas others are endemic to specific geographical regions (2). The isolates in the present enrolled studies mainly came from Europe, where MALDI-TOF MS has been used for several years, and only a few strains in one study came from Africa (48); such situations will inevitably call the identification capability of MALDI-TOF MS into question. Similarly, nearly half of the enrolled studies were retrospective, and 72.73% of studies did not clearly report a blinding method, both of which lessen the quality of the original articles. As for reference methods, although sequencing is currently the gold standard, less than 10% of studies sequenced all isolates, and most of the studies compared MALDI-TOF MS with commonly used biochemical methods and/or morphology unless a discrepancy occurred. Thus, many studies likely overestimated the accuracy of MALDI-TOF MS when the reference phenotypic system and the system under evaluation both made mistakes.

Finally, some other meaningful limitations of the present work should be acknowledged. First, with regard to proportion, the predominant fungal isolates investigated in most of the enrolled studies were commonly detected species, which are much easier to identify than unusual species (32), and this leads to overestimations of accuracy. Second, because the number of investigations directly comparing two different systems was less than 4 (for example, there was only 1 comparing the Biotyper and Vitek MS systems), sizeable heterogeneity emerged, making comparative evaluation of the identification performance between different systems unfeasible. Third, the databases are updated frequently, and newly developed databases generally have more reference spectra than older ones; this may be another cause of the severe heterogeneity. The use of more databases and terms for literature retrieval might retrieve more articles; nevertheless, little publication bias was found in the present meta-analysis.

As an entirely new technology for the clinical diagnosis of pathogenic diseases, MALDI-TOF MS has many advantages over other current methods for pathogen identification. For example, the turnaround times for the identification of clinical fungi from colony or culture to the identification result have been routinely shortened to less than 10 min with the Andromas software (8) and approximately 13 min with the Biotyper software (46), and the detection (reagent) cost has declined to $0.50 per isolate using the Biotyper system (33). Moreover, other applications of MALDI-TOF MS have been developed, such as strain typing (58), assessing drug susceptibility (59), and detecting virulence factors (60). On the other hand, MALDI-TOF MS currently has some limitations. The high cost of purchasing and maintaining the instrument is the major limitation, and it is difficult for the method to detect antimicrobial resistance or to discriminate closely related species (10).

In summary, MALDI-TOF MS showed high accuracy for the identification of clinical pathogenic fungi in the present meta-analysis. Therefore, future studies to analyze the comprehensive capability of this technology for clinical microbiology diagnostics are warranted.