Accuracy of Molecular Amplification Assays for Diagnosis of Staphylococcal Pneumonia: a Systematic Review and Meta-analysis

ABSTRACT

Rapid and accurate identification of staphylococcal pneumonia is crucial for effective antimicrobial stewardship. We performed a meta-analysis to evaluate the diagnostic value of nucleic acid amplification tests (NAAT) from lower respiratory tract (LRT) samples from suspected pneumonia patients to avoid superfluous empirical methicillin-resistant Staphylococcus aureus (MRSA) treatment. PubMed, Scopus, Embase, Web of Science, and the Cochrane Library Database were searched from inception to 2 September 2020. Data analysis was carried out using a bivariate random-effects model to estimate pooled sensitivity, specificity, positive likelihood ratio (PLR), and negative likelihood ratio (NLR). Of 1,808 citations, 24 publications comprising 32 data sets met our inclusion criteria. Twenty-two studies (n = 4,630) assessed the accuracy of the NAAT for methicillin-sensitive S. aureus (MSSA) detection, while 10 studies (n = 2,996) demonstrated the accuracy of the NAAT for MRSA detection. The pooled NAAT sensitivity and specificity (with 95% confidence interval [CI]) for all MSSA detection were higher (sensitivity of 0.91 [95% CI, 0.89 to 0.94], specificity of 0.94 [95% CI, 0.94 to 0.95]) than those of MRSA (sensitivity of 0.75 [95% CI, 0.69 to 0.80], specificity of 0.88 [95% CI, 0.86 to 0.89]) in lower respiratory tract (LRT) samples. NAAT pooled sensitivities differed marginally among different LRT samples, including sputum, endotracheal aspirate (ETA), and bronchoalveolar lavage (BAL) fluid. Noticeably, NAAT pooled specificity against microbiological culture was consistently ≥88% across various types of LRT samples. A meta-regression and subgroup analysis of study design, sample condition, and patient selection method could not explain the heterogeneity (P > 0.05) in the diagnostic efficiency. This meta-analysis has demonstrated that the NAAT can be applied as the preferred initial test for timely diagnosis of staphylococcal pneumonia in LRT samples for successful antimicrobial therapy.

INTRODUCTION

Pneumonia due to Staphylococcus aureus, more specifically, methicillin-resistant S. aureus (MRSA), is associated with extended hospital stays, high costs, severe morbidity, and premature mortality worldwide. S. aureus accounts for 1 to 10% of community-acquired pneumonia (CAP) cases and up to 16% of nosocomial pneumonia cases (1). However, in recent decades, the majority of large medical centers in the United States, Europe, and Australia have seen a dramatic rise in the percentage of staphylococcal infections induced by MRSA (2–4). Indeed, MRSA now accounts for 20 to 40% of all nosocomial pneumonia (2). The increased prevalence of MRSA in the health care setting has led to the formation of guidelines for the administration of potentially toxic and pricey anti-MRSA antibiotics, such as vancomycin or linezolid, as empirical treatment for all ventilator-associated pneumonia (VAP), hospital-acquired pneumonia (HAP), and health care-associated pneumonia (HCAP), along with awareness of delays (5, 6). While antibiotics may protect from MRSA infection, broad-spectrum antibiotics can cause harm in other ways (7). Even short courses of anti-MRSA agents can alter host flora, expose patients to drug-induced toxicity, increase multidrug-resistant pathogens, increase treatment-related side effects, and increase hospitalization costs (8, 9). If initial antibiotics are ineffective and are subsequently altered after diagnostic test results become available, mortality does not improve significantly (10, 11). It is, therefore, becoming highly important to balance these two conflicting interests, namely, the need for full coverage while avoiding unnecessary antibiotics.

In the diagnosis of staphylococcal pneumonia, the clinical laboratory plays a crucial role. However, it faces various obstacles due to the complexity of the quality and variety of specimens from suspected patients. Since staphylococcal strains are typically associated with the upper respiratory tract (12), the efficacy of the nasopharyngeal swab (NPS) in the diagnosis of clinical pneumonia is likely to be restricted. Sputum testing has recently been commonly used to determine the causative pathogens of bacterial pneumonia. However, due to the potential for inevitable oral bacterial contamination of the upper respiratory tract (13), the use of traditional sputum sample cultivation is occasionally deemed inappropriate. In terms of assessing causative pathogens, the quantitative cultivation of samples obtained directly from affected lesions through bronchoalveolar lavage (BAL) or safe specimen brushing with bronchoscopy is more reliable (14, 15). However, this technique is considered either too invasive or practically inapplicable for the majority of children. In a patient who is intubated or unable to provide an appropriate sputum sample, endotracheal aspirates (ETA) have recently been reported as an alternative to BAL samples.

Identification of staphylococcal strains from respiratory samples relies on the Gram stain and conventional culture method upon sample availability. While respiratory secretion Gram stains are fast and relatively inexpensive, they are not sufficiently safe to be used when determining whether or not to initiate antimicrobial treatment (16). In critically ill patients with suspected pneumonia, conventional culture-based investigation of the respiratory sample remains the preferred method for detecting bacterial pathogens (8, 17). A positive microbiological diagnosis in respiratory secretions can only be achieved in approximately 30% of cases, with a relatively high false-negative (FN) rate as commensal and colonizing microorganisms complicate the analysis, and testing requires a certain amount of samples to start cultivation (18). Conventional phenotypic diagnosis usually takes 48 to 72 h for respiratory samples, leading to an undesirable delay in the detection of pathogens and initiation of antimicrobial treatment. Individuals with pneumonia infections who receive antibiotics can also have adverse outcomes from the staphylococcal culture (19). As a result, patients would likely lose the optimal chance of accessing care, and disease control facilities would not be in a position to initiate immediate management measures. As a consequence of these drawbacks, novel culture-independent approaches have been suggested in patients with suspected pneumonia as promising alternatives to classify pathogens and guide antibiotic choices.

Unlike traditional microbiological culture, nucleic acid amplification tests (NAAT) rely on the detection of bacterial DNA rather than the recovery of viable bacteria and thus are less affected by the prior administration of antibiotics. The molecular amplification tests can detect both methicillin-sensitive Staphylococcus aureus (MSSA) and MRSA in respiratory secretions within a short period (20). Using a rapid diagnostic test, a previous study reported a reduction in the length of anti-MRSA antibiotic use in patients with suspected pneumonia (21). Several studies have assessed the accuracy of molecular techniques with different gene targets like PCR (22), real-time PCR (23), GeneXpert (24), and loop-mediated isothermal amplification (LAMP) (25); however, the literature on the importance of these tests for timely pneumonia management is too widely scattered for any meaningful interpretation. Given the need to make a medical decision on the diagnosis of staphylococcal pneumonia, we systematically reviewed and analyzed available data from studies showing a variety of different clinical presentations, including CAP, VAP, HAP, and lower respiratory tract infection (LRTI), to determine NAAT pooled summary estimates compared to microbiological culture.

METHODS

Search strategy.

This research was carried out by adopting the diagnostic accuracy requirements of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) (26). A computer-aided literature search without any limitations, for related studies published until 2 September 2020, has been performed systematically through PubMed, Scopus, Embase, Web of Science, and the Cochrane Library. The search terms in conjunction with Boolean operators “OR” and “AND” included the following terms: “Staphylococcus aureus,” “S. aureus,” “methicillin-resistant S. aureus,” “MRSA,” “NAAT,” “NAA,” “nucleic acid amplification,” “molecular assay,” “LAMP,” “loop-mediated isothermal amplification,” “PCR,” “polymerase chain reaction,” “LCR,” “ligase chain reaction,” “LCx,” “qPCR,” “real-time PCR,” “RT-PCR,” “Amplicor,” “ProbeTec,” “Gen-Probe,” “Roche,” “Abbott,” “Cepheid,” “pneumonia” OR “staphylococcal pneumonia” OR “lower respiratory tract” OR “respiratory symptom.” Citations of the reports and the included studies were also searched to find potentially relevant studies.

Study selection.

The search terms were used to scan all relevant citations via electronic databases, and duplicates were carefully removed using the EndNote X9 program (Thomson Reuters, New York, NY, USA). The retrieved records were initially examined by reviewing titles and abstracts, and unrelated studies were excluded from subsequent analysis. The full text of potentially relevant studies for accuracy data was retrieved and carefully examined. Any disagreements between the two reviewing analysts (K. Chen and S. Ahmed) were resolved by consensus.

Inclusion criteria.

We included full-text, peer-reviewed, cross-sectional studies, cohort studies, randomized controlled trials, and case-control studies that used an NAAT to detect staphylococcal infections in LRT samples and compared it to the culture reference standard. The studies explicitly provided true-positive (TP), true-negative (TN), false-positive (FP), and false-negative (FN) values for the index test or included sufficient information to construct 2-by-2 contingency tables for sensitivity and specificity determination. All studies followed the standard definition for staphylococcal pneumonia diagnosis and were considered for inclusion if they included evidence of a pulmonary infiltrate on chest radiography or clinically suspected based on two of the following features: fever, sputum production, leukocytosis or leukopenia, pleuritic chest pain, coughing, and LRTI.

Exclusion criteria.

Conference proceedings, poster abstracts, commentaries, reviews, editorials, case reports, animal experimentation, mechanism studies, and meta-analysis were excluded. Studies that did not include sufficient data for computation of sensitivity and specificity and comprised a reduced sample size (≤10 samples) were ineligible. Noninterpretable test results by both index test and microbiological reference standard were also not included.

Data extraction.

Two analysts (K. Chen and S. Ahmed) independently assessed all related studies with prespecified eligibility criteria to ensure the reproducibility of the study selection. Studies that did not comply were resolved by consultation with the third investigator (S. C. Ojha). The data recorded from eligible studies were author(s), country, study type, publication year, sample type, samples per patient, sensitivity, specificity, reference standard, and other related attributes. For missing details, unclear reference standards, and sample process conditions, the authors were consulted individually. Based on available information from qualified studies, 2-by-2 NAAT performance contingency tables were developed in comparison with the microbiological reference standards. Studies dealing with different index tests versus a specific reference standard were regarded as independent studies.

Quality assessment.

The Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) was applied to evaluate the methodological quality of the studies (27). QUADAS-2 consisted of four domains: (i) patient selection, (ii) index test, (iii) reference standard, and (iv) flow and timing. The review authors (K. Chen and S. Ahmed) analyzed all four domains for the possible risk of bias and the first three domains for applicability concerns. In order to allow assessors to examine the potential risk of bias, signaling questions were asked. Disagreements were resolved by consensus among the reviewing authors.

Statistical analysis.

The following program was used for statistical analysis: RevMan 5.4 (Nordic Cochrane Centre, Copenhagen, Denmark) for the overall methodological quality evaluation of the included studies and the generation of summary plots (28). Meta-DiSc 1.4 (Cochrane Colloquium, Barcelona, Spain) was used for analysis of pooled sensitivity, specificity, likelihood ratios, diagnostic odds ratio (DOR), and data heterogeneity computation (29). NAAT diagnostic accuracy with a 95% confidence interval (CI) was calculated against microbiological culture using a random-effects model. Additionally, the I-square (I²) statistics were applied to measure the heterogeneity among the included studies, where I² values of <40% indicate low heterogeneity, 40 to 60% moderate heterogeneity, 60 to 90% substantial heterogeneity, and >90% considerable heterogeneity (30). Sample conditions, study design, and digestion/decontamination were analyzed as potential sources of heterogeneity using subgroup analysis. Deek's funnel plot was used to determine publication bias (31). Ethical approval was not applicable to this study.

RESULTS

Literature selection.

A total of 1,808 unique citations were identified (PubMed, 724; Embase, 332; Scopus, 343; Web of Science, 370; and the Cochrane Library, 39) (Fig. 1). Of these, 542 citations were omitted due to redundancy in databases. After screening titles and abstracts of 1,266 articles, 295 studies deemed potentially relevant were subjected to full-text revision. Table S1 in the supplemental material provides a summary of reviewed studies, including explanations why those studies have been omitted. Eventually, 24 studies meeting all inclusion requirements were included in subsequent analyses (24, 25, 32–53).

FIG 1.

Study selection flow diagram.

Characteristics of the included studies.

The baseline features of the studies included are shown in Table 1. Seventeen studies were carried out in high-income countries and seven in a lower-middle-income country. Studies documenting the use of two or more index tests against microbiological reference standard were regarded as separate studies. Based on this principle, 24 publications comprising 32 data sets were included in this meta-analysis. Twenty-two studies (n = 4,630) assessed the accuracy of NAAT for MSSA detection, while the other 10 studies (n = 2,996) demonstrated the accuracy of NAAT for MRSA detection. The median number of samples per study for the diagnosis of MSSA was 106 (interquartile range, 76 to 200), while for MRSA, the median number of samples per study was 198 (interquartile range, 99 to 200). Of 24 studies, 3 studies (n = 363) solely assessed the accuracy of NAAT for diagnosis of staphylococcal pneumonia in sputum samples, while two studies (n = 284) were evaluated for MRSA. Five studies (n = 426) evaluated the accuracy of NAAT for the detection of MSSA, while two studies (n = 181) evaluated the detection of MRSA in ETA samples. Nine studies (n = 908) evaluated the accuracy of NAAT for MSSA detection, while two studies were evaluated for MRSA detection in BAL samples. The remainder of the studies contained indistinguishable LRT sample types of various percentages. All analyses were performed either at tertiary care hospitals or university research laboratory facilities. Only studies conducted in English before 2 September 2020 were included.

TABLE 1.

Baseline characteristics of studies included in the meta-analysis^a

Reference	Yr	Country	Setting	Pros enroll	Patient selection	Specimen condition	Digest/Decont	Sample size	Sample type(s)	NAAT	Potential risk(s)
Boattini et al. (32)	2020	Italy	UHL	Yes	Convenience	Fresh	No	113	LRT	ELITe MGB	LRTI
Cercenado et al. (24)	2012	Spain	TCC	No	Consecutive	Fresh	No	135	ETA	Xpert	VAP
Clavel et al. (33)	2016	France	UHL/TCC	Yes	Consecutive	Fresh/frozen	Yes	120	BAL, ETA	qPCR	VAP
Collins et al. (34)	2020	USA	UHL	No	Convenience	Fresh/frozen	No	175	BAL	Unyvero LRT	LRTI
Coppens et al. (35)	2019	Belgium	UHL	Yes	Consecutive	Fresh/frozen	Yes	79	ETA	Xpert/qPCR	VAP
Gadsby et al. (36)	2015	UK	TCC	No	Consecutive	Fresh/frozen	Yes	323	Sputum, ETA	PCR	CAP
Ghodousi et al. (37)	2020	Iran	UHL	Yes	Convenience	Fresh/frozen	Yes	269	LRT	qPCR	HAP
Hou et al. (25)	2018	China	UHL/TCC	Yes	Consecutive	Fresh	Yes	1,855	Sputum, BAL	LAMP	LRTI
Hu et al. (38)	2020	China	TCC	No	Consecutive	Fresh	Yes	157	Sputum	qPCR	RI
Huang et al. (39)	2015	China	UHL	Yes	Consecutive	Fresh	Yes	154	Sputum	qPCR	HAP
Lee et al. (40)	2019	Taiwan	UHL	No	Consecutive	Fresh	No	59	ETA, BAL	FA-PP	LRTI
Mansour and Albendary (41)	2018	Egypt	UHL	Yes	Consecutive	Fresh	No	96	ETA	m-PCR	VAP
Oh et al. (42)	2013	South Korea	TCC	No	Convenience	Fresh	Yes	129	ETA, BAL	Xpert MRSA	RTI
Paonessa et al. (43)	2019	USA	TCC	Yes	Consecutive	Fresh	Yes	247	BAL	Xpert	Pneumonia
Papan et al. (44)	2018	Germany	TCC	Yes	Consecutive	Fresh	No	79	BAL, ETA, PF	Unyvero	Pneumonia
Peiffer-Smadja et al. (45)	2020	France	UHL	Yes	Consecutive	Fresh	No	95	BAL	Unyvero	HAP, VAP
Roisin et al. (46)	2018	Belgium	TCC	Yes	Consecutive	Fresh	Yes	85	LRT	VAPchip	Pneumonia
Sansot et al. (47)	2019	France	TCC	Yes	Convenience	Fresh	No	50	BAL	FA-BCID	VAP
Tchatchouang et al. (48)	2019	Cameroon	TCC	Yes	Convenience	Fresh/frozen	No	40	BAL	RT-PCR	LRTI
Trevino et al. (49)	2017	USA	TCC	No	Consecutive	Fresh/frozen	No	100	LRT	Xpert	VAP
Tschiedel et al. (50)	2019	Germany	TCC	Yes	Convenience	Fresh	No	70	BAL	SeptiFast	RTI
Wang et al. (51)	2016	China	UHL	Yes	Consecutive	Fresh	Yes	76	Sputum	qLAMP	HAP
Webber et al. (52)	2020	USA	TCC	No	Consecutive	Frozen	No	200	LRT	FA-PP	RTI
Yoo et al. (53)	2020	South Korea	TCC	No	Convenience	Fresh/frozen	No	100	Sputum, ETA	FA-PP	RTI

^aAbbreviations: BAL, bronchoalveolar lavage; BCID, blood culture identification; Digest/Decont, digestion/decontamination; ETA, endotracheal aspirates; FA-PP, FilmArray pneumonia panel; HAP, health care-associated pneumonia; LRTI, lower respiratory tract infections; Pros enroll, prospective enrollment; qPCR, quantitative (real-time) PCR; LAMP, loop-mediated isothermal amplification; TCC, tertiary care center (hospital); UHL, university hospital laboratory; VAP, ventilator-associated pneumonia.

Quality appraisal.

Figure 2 illustrates the cumulative risk of bias and applicability concerns of the 24 studies included in our meta-analysis (24, 25, 32–53). Additional details on the quality assessment of independent studies are provided in the supplemental material (see Fig. S1). In the patient selection domain (Fig. 2), the majority of studies (21 [87.5%]) were at low risk of bias, while three studies (29, 40, 49) were at high risk of bias due to the improper exclusions of certain patients. With regard to applicability, the majority of studies (22 [91.7%]) included patients with clinical features suggestive of pneumonia (Table 1). However, two studies (38, 53) that did not reflect a typical clinical scenario were considered to have high applicability concerns. The methodological quality’s main limitations pertained to the following: blinding the index test results to those who interpreted the reference test and the reference test results to those who interpreted the index test. The majority of studies (23 [95.8%]) were at unclear risk in the index test and the reference standard domain due to lack of information on whether their test results were interpreted blind. Only one study (51) reported a blind interpretation of the index test and the reference standard, which was considered to be of low concern. Concerns regarding the applicability of the index test were unclear in all studies (Fig. 2), since there is no globally standardized operating protocol for processing and storage of samples. In all the studies, the investigators used microbiological culture as a reference standard and performed culture in either a hospital or university-affiliated reference laboratory, so we regarded the applicability concern about the reference standard bias to be of low concern. Subsequently, there was no doubt about the potential risk of bias in the flow and timing domain, as both index and reference standards were applied to the same samples. The overall methodological quality was acceptable for the studies included in our meta-analysis.

FIG 2.

Methodological quality and risk of bias assessment of the included studies against a microbiological culture reference standard.

Summary estimates.

The studies were highly heterogeneous: thus, pooled NAAT summary estimates for combined staphylococcal pathogen detection were not considered meaningful for antimicrobial treatment. We initially focused on MRSA or MSSA identification in LRT samples (i.e., sputum, ETA, and BAL), as most studies did not distinguish sample types. Studies measuring NAAT precision in independent sample types were also tested for pooled summary estimates. Consequently, we assessed the accuracy of index tests and commercial tests, as well as a potential source of heterogeneity among predefined subgroup studies.

Detection of staphylococcal pneumonia in LRT specimen.

For 22 studies (24, 25, 32–58) that met the inclusion criteria for comparing NAAT with the microbiological culture, a total of 4,630 LRT samples (sputum, ETA, and BAL) for the detection of MSSA in suspected patients with pneumonia were used. NAAT's sensitivity and specificity for MSSA detection ranged from 0.0 (95% CI, 0.0 to 0.24) to 1.0 (95% CI, 0.94 to 1.0) and from 0.50 (95% CI, 0.34 to 0.66) to 1.00 (95% CI, 0.98 to 1.0), respectively (Fig. 3A). The pooled NAAT summary estimates for MSSA detection in LRT samples were sensitivity of 0.91 (95% CI, 0.89 to 0.94), specificity of 0.94 (95% CI, 0.94 to 0.95), positive likelihood ratio (PLR) of 17.2 (95% CI, 9.66 to 30.63), negative likelihood ratio (NLR) of 0.11 (95% CI, 0.06 to 0.22), and DOR of 207.54 (95% CI, 81.03 to 531.6). The I² statistical sensitivity and specificity values were 82.5 and 87.6%, respectively, indicating substantial heterogeneity. The area under the concentration-time curve (AUC) of hierarchical summary receiver operating characteristics (HSROC) was 0.98 (95% CI, 0.97 to 0.99), implying excellent overall diagnostic validity (Fig. 4A).

FIG 3.

Forest plot displaying NAAT pooled sensitivity and specificity for detection of MSSA as well as MRSA in LRT samples. Each square represents the sensitivity and specificity of a particular study; the black line represents its confidence interval. Abbreviations: TP, true positive; FP, false positive; FN, false negative; TN, true negative; CI, confidence interval.

FIG 4.

HSROC plot of NAAT for (A) MSSA and (B) MRSA detection in LRT samples. Blue circles indicate the data points from each of the investigations, and the solid blue line represents the HSROC curve.

Similarly, 10 studies (24, 25, 39, 41–44, 49, 52, 53) comprising 2,996 samples met the criteria for the diagnosis of MRSA in suspected pneumonia patients. The NAAT sensitivity and specificity ranged from 0.0 (95% CI, 0.0 to 0.26) to 1.0 (95% CI, 0.9 to 1.0) and from 0.85 (95% CI, 0.83 to 0.86) to 1.0 (95% CI, 0.96 to 1.0), respectively (Fig. 3B). The pooled summary estimates of NAAT for diagnosis of MRSA were sensitivity of 0.75 (95% CI, 0.69 to 0.80), specificity of 0.88 (95% CI, 0.86 to 0.89), PLR of 10.1 (95% CI, 5.44 to 18.72), NLR of 0.12 (95% CI, 0.04 to 0.37), and DOR of 135.41 (95% CI, 14.83 to 1,236.3). The I² values for sensitivity and specificity were 94.4 and 86.9%, respectively, indicating considerable heterogeneity among studies. The AUC of HSROC was 0.98 (95% CI, 0.96 to 0.99), suggesting overall strong diagnostic validity (Fig. 4B).

Detection of staphylococcal pneumonia in sputum.

Three studies (38, 51, 52) involving 363 sputum samples measured the accuracy of NAAT for MSSA detection, while the other two studies (39, 52) (n = 284) demonstrated the accuracy of NAAT for MRSA detection. The sensitivity and specificity of NAAT for MSSA detection ranged from 0.75 (95% CI, 0.35 to 0.97) to 1.0 (95% CI, 0.75 to 1.00) and from 0.90 (95% CI, 0.83 to 0.95) to 1.00 (95% CI, 0.97 to 1.00), respectively (see Fig. S2A in the supplemental material), while the NAAT sensitivity and specificity for MRSA detection ranged from 1.0 (95% CI, 0.29 to 1.00) to 1.0 (95% CI, 0.90 to 1.00) and from 0.90 (95% CI, 0.83 to 0.95) to 1.00 (95% CI, 0.86 to 0.96), respectively (Fig. S2B). For MSSA detection, pooled summary estimates of NAAT were lower (sensitivity of 0.94 [95% CI, 0.79 to 0.99], specificity of 0.95 [95% CI, 0.92 to 0.97], PLR of 16.89 [95% CI, 4.18 to 68.22], NLR of 0.11 [95% CI, 0.02 to 0.57], and DOR of 243.77 [95% CI, 14.85 to 4,001.3]) than those of MRSA (sensitivity of 1.0 [95% CI, 0.91 to 1.0], specificity of 0.91 [95% CI, 0.87 to 0.94], PLR of 9.84 [95% CI, 6.48 to 14.95], NLR of 0.05 [95% CI, 0.01 to 0.50], and DOR of 233.72 [95% CI, 26.56 to 2,056.9]). The I² statistical sensitivity and specificity values were 0.0% for MRSA detection, indicating low heterogeneity, while NAAT sensitivity and specificity for MSSA detection were 65.7 and 90.2%, respectively, indicating significant heterogeneity. The AUC of HSROC for MSSA was 0.98 (95% CI, 0.93 to 1.0), indicating perfect overall diagnostic validity (Fig. S2C). In MRSA, due to the small number of available studies (39, 52), the AUC was not estimable (Fig. S2D).

Detection of staphylococcal pneumonia in ETA.

Five studies (24, 33, 35, 44), consisting of 426 samples, tested the accuracy of NAAT for MSSA detection in ETA samples, while the other two studies (24, 41) (n = 181) demonstrated the accuracy of NAAT for MRSA detection. The NAAT sensitivity and specificity for MSSA detection ranged from 0.63 (95% CI, 0.24 to 0.91) to 1.0 (95% CI, 0.94 to 1.00) and from 0.50 (95% CI, 0.34 to 0.66) to 0.99 (95% CI, 0.93 to 1.00), respectively (see Fig. S3A in the supplemental material), while the NAAT sensitivity and specificity for MRSA detection ranged from 0.0 (95% CI, 0.0 to 0.26) to 0.98 (95% CI, 0.89 to 1.00) and from 1.00 (95% CI, 0.90 to 1.00) to 1.00 (95% CI, 0.96 to 1.00), respectively (Fig. S3B). NAAT demonstrated greater pooled diagnostic accuracy for MSSA detection (sensitivity of 0.95 [95% CI, 0.91 to 0.98], specificity of 0.88 [95% CI, 0.83 to 0.92], PLR of 9.33 [95% CI, 1.81 to 48.19], NLR of 0.07 [95% CI, 0.01 to 0.39], and DOR of 149.48 [95% CI, 19.94 to 1,120.6]) than for MRSA (sensitivity of 0.78 [95% CI, 0.66 to 0.88], specificity of 0.99 [95% CI, 0.96 to 1.00], PLR of 26.48 [95% CI, 0.33 to 2,146.8], NLR of 0.18 [95% CI, 0.0 to 1,112,238.3], and DOR of 136.86 [95% CI, 0.082 to 229,269.3]). The I² values of NAAT sensitivity and specificity for MSSA detection were 86.3 and 92.4%, respectively, indicating considerable heterogeneity, while the I² MRSA sensitivity and specificity statistical values were 97.9 and 21.4%, respectively, suggesting substantial to low heterogeneity. The AUC for MSSA was 0.97 (95% CI, 0.94 to 1.00), which indicated excellent overall diagnostic validity, but it was not estimated for MRSA due to the limited number of included studies (24, 41) (Fig. S3C and D).

Detection of staphylococcal pneumonia in BAL samples.

The accuracy of NAAT for MSSA detection in BAL samples was evaluated by nine studies (33, 34, 43–45, 47, 48, 50, 52) consisting of 908 samples, while another two studies (43, 52) (n = 288) demonstrated MRSA detection. The NAAT sensitivity and specificity for MSSA detection ranged from 0.0 (95% CI, 0.0 to 0.26) to 1.0 (95% CI, 0.4 to 1.00) and from 0.83 (95% CI, 0.71 to 0.91) to 1.00 (95% CI, 0.96 to 1.00), respectively (see Fig. S4A in the supplemental material). On the flip side, the sensitivity and specificity of NAAT for MRSA detection ranged from 0.83 (95% CI, 0.36 to 1.00) to 0.96 (95% CI, 0.80 to 1.00) and from 0.90 (95% CI, 0.84 to 0.94) to 1.00 (95% CI, 0.94 to 1.00), respectively (Fig. S4B). NAAT revealed lower diagnostic accuracy for MSSA detection (sensitivity of 0.88 [95% CI, 0.80 to 0.93], specificity of 0.97 [95% CI, 0.95 to 0.98], PLR of 22.78 [95% CI, 8.93 to 58.15], NLR of 0.21 [95% CI, 0.09 to 0.47], and DOR of 156.47 [95% CI, 38.19 to 649.29]) compared to MRSA (sensitivity of 0.94 [95% CI, 0.79 to 0.99], specificity of 0.92 [95% CI, 0.88 to 0.95], PLR of 21.46 [95% CI, 1.95 to 236.71], NLR of 0.11 [95% CI, 0.02 to 0.60], and DOR of 260.8 [95% CI, 45.51 to 1,494.5]). For MSSA detection, the I² statistical values for sensitivity and specificity were 52.8 and 76.5%, respectively, suggesting low to substantial heterogeneity, whereas the NAAT sensitivity and specificity for MRSA detection were 2.7 and 91.7%, respectively, indicating low to considerable heterogeneity. The AUC for MSSA was 0.98 (95% CI, 0.97 to 1.00), reflecting very strong overall diagnostic validity, but due to the small number of included studies (43, 52), it was not estimated for MRSA (Fig. S4C and D).

Diagnostic accuracy of in-house versus commercial tests.

Table 2 summarizes the diagnostic accuracy of studies based on different NAAT. For identification of both MSSA and MRSA, the pooled summary estimates of the in-house NAAT were consistently higher (sensitivity of ∼95%, specificity of ∼90%). The pooled summary estimates of the commercial tests against culture were higher for MSSA (sensitivity of 0.89 [95% CI, 0.86 to 0.92], specificity of 0.94 [95% CI, 0.93 to 0.94], PLR of 14.02 [95% CI, 7.44 to 26.42], NLR of 0.15 [95% CI, 0.08 to 0.29], DOR of 128.3 [95% CI, 46.3 to 355.4], and AUC of 0.98 [95% CI, 96.0 to 99.0]) compared to MRSA (sensitivity of 0.71 [95% CI, 0.65 to 0.77], specificity of 0.87 [95% CI, 0.86 to 0.89], PLR of 10.34 [95% CI, 5.11 to 20.92], NLR of 0.15 [95% CI, 0.05 to 0.44], DOR of 114.2 [95% CI, 11.58 to 1,125.3], and AUC of 0.98 [95% CI, 0.96 to 0.99]). The FilmArray pneumonia panel (FA-PP) reported consistently higher summary estimates for identification of both MSSA (sensitivity of 1.0 [95% CI, 0.9 to 1.0], specificity of 0.90 [95% CI, 0.87 to 0.93], PLR of 9.6 [95% CI, 4.92 to 18.56], NLR of 0.05 [95% CI, 0.01 to 0.25], DOR of 215.4 [95% CI, 38.93 to 1,191.8], and AUC of 0.98 [95% CI, 0.96 to 0.99]) and MRSA (sensitivity of 1.0 [95% CI, 0.82 to 1.00], specificity of 0.93 [95% CI, 0.89 to 0.95], PLR of 11.34 [95% CI, 5.76 to 22.33], NLR of 0.05 [95% CI, 0.01 to 0.36], and DOR of 214.63 [95% CI, 27.44 to 1,678.6]) among the commercial tests.

TABLE 2.

Subgroup analysis of eligible studies focused on various types of NAAT^a

Diagnostic target with microbiological culture reference standard	Subgroup	NAAT	No. of data	% sensitivity (95% CI)	% specificity (95% CI)	PLR (95% CI)	NLR (95% CI)	DOR (95% CI)	AUC (95% CI)
S. aureus	In house	qPCR	5	97 (93–99)	97 (96–98)	29.0 (11.46–73.16)	0.04 (0.01–0.15)	1,254.4 (159.0–9,899.5)	99 (98–100)
	Commercial	GeneXpert	4	97 (93–100)	93 (90–95)	15.6 (1.88–129.1)	0.05 (0.01–0.28)	358.0 (43.2–2,968.3)	99 (97–100)
		FA-PP	3	100 (90–100)	90 (87–93)	9.6 (4.92–18.56)	0.05 (0.01–0.25)	215.4 (38.93–1,191.8)	98 (96–99)
		Unyvero PP	3	81 (69–90)	98 (96–99)	32.6 (5.07–209.43)	0.24 (0.08–0.74)	158.6 (10.61–2,371.3)	97 (96–98)
		LAMP	2	53 (36–69)	94 (93–95)	7.88 (5.36–11.57)	0.49 (0.26–0.90)	15.82 (7.13–35.10)	−
		FA-BCID	1	77 (46–95)	100 (91–100)	−	0.23 (0.09–0.62)	−	−
		RT-qPCR	1	−	93 (80–98)	−	−	−	−
		SeptiFast	1	60 (15–95)	83 (71–91)	3.44 (1.40–8.41)	0.48 (0.16–1.43)	−	−
		ELITE MGB	1	99 (94–100)	100 (88–100)	−	0.01 (0.0–0.08)	−	−
		VAPchip	1	86 (64–97)	97 (89–100)	27.43 (6.93–108.51)	0.15 (0.05–0.42)	−	−
MRSA	In house	qPCR	1	100 (90–100)	90 (83–95)	9.83 (5.75–16.81)	0.0
	Commercial	GeneXpert	5	89 (82–94)	93 (91–95)	13.51 (6.39–28.56)	0.1 (0.0–1,990.93)	279.0 (45.24–1,720.5)	99 (98–100)
		FA-PP	2	100 (82–100)	93 (89–95)	11.34 (5.76–22.33)	0.05 (0.01–0.36)	214.63 (27.44–1,678.6)	−
		LAMP	1	42 (31–52)	85 (83–86)	2.69 (2.06–3.5)	0.69 (0.58–0.82)	−	−
		Unyvero PP	1	0 (0–98)	100 (88–100)	−	1.0 (1.0–1.0)	−	−

^aAbbreviations: −, not estimable; AUC, area under the curve; DOR, diagnostic odds ratio; FA-BCID, FilmArray blood culture identification; FA-PP, FilmArray pneumonia panel; LAMP, loop-mediated isothermal amplification; NAAT, nucleic acid amplification tests; NLR, negative likelihood ratio; PLR, positive likelihood ratio; qPCR, quantitative PCR (real-time PCR); RT-qPCR, reverse transcriptase quantitative PCR.

Meta-regression and subgroup analyses.

Since significant heterogeneity between studies was observed, a meta-regression analysis investigated the potential source of heterogeneity in predefined subgroups. Meta-regression indicated that study design (prospective/others), country (developed/developing), patient selection (consecutive/convenience), sample condition (fresh/frozen), and digestion/decontamination (yes/no) were not significant sources of heterogeneity (meta-regression P = 0.07, P = 0.58, P = 0.54, P = 0.8, and P = 0.79, respectively).

Publication bias.

Publication bias was assessed using a Deek’s funnel plot asymmetry test. In this study, no apparent bias in publication was found (P > 0.05).

DISCUSSION

Rapid identification of staphylococcal strains and their resistance markers in patients with suspected pneumonia is crucial, as timely action with appropriate antimicrobial therapy can lead to better clinical outcomes. However, the different case definitions and the multiple samples used in the different studies make the comparison of research findings difficult and hinder the management of the disease. The majority of studies in the past have focused on nasal screening to predict staphylococcal pneumonia (54–56); however, it remains controversial whether the bacteria isolated from upper airways are indeed causative pathogens. The use of the NPS is convenient, as lower airway specimens typically require invasive procedures (e.g., bronchoscopy or lung punctures). Nevertheless, the NPS screened at hospitalization may not reflect clinical symptoms as staphylococcal strains are common commensals of the upper respiratory tract (12), and detection of organisms by molecular tests at any single point of time does not necessarily equate to active infection (57). In order to assess the diagnostic performance of NAAT across LRT sample types for detection of MSSA and MRSA in patients with suspected pneumonia, we, therefore, performed a systematic review and meta-analysis.

Based on the results obtained from our study, we identified that NAAT had higher overall summary estimates for detection of MSSA (sensitivity of 0.91 [95% CI, 0.89 to 0.94], specificity of 0.94 [95% CI, 0.94 to 0.95], PLR of 17.2 [95% CI, 9.66 to 30.63], NLR of 0.11 [95% CI, 0.06 to 0.22], DOR of 207.54 [95% CI, 81.03 to 531.6], and AUC of 0.98 [95% CI, 0.97 to 0.99]) compared to MRSA (sensitivity of 0.75 [95% CI, 0.69 to 0.80], specificity of 0.88 [95% CI, 0.86 to 0.89], PLR of 10.1 [95% CI, 5.44 to 18.72], NLR of 0.12 [95% CI, 0.04 to 0.37], DOR of 135.41 [95% CI, 14.83 to 1,236.3], and AUC of 0.98 [95% CI, 0.97 to 0.99]). It should be noted that there was considerable variation in the molecular assays’ accuracy for independent MSSA or MRSA detection. In 3 of the 22 data sets, sensitivity was low (≤60%) for the detection of MSSA (25, 44, 50). Correspondingly, 3 out of 10 data sets reported low sensitivity (≤50%) for MRSA detection (25, 41, 44). Low sensitivity in the study by Hou et al. (25) for detection of both MSSA or MRSA could be explained due to a resource problem where the results were not investigated in parallel by PCR. Patient characteristics, including personal error and pediatric population sampling error, may also have contributed to low sensitivity in these studies (25, 41, 44, 50). In contrast, specificity was persistently higher and consistent across studies, and none of the included studies had specificity less than 83%, except for the study by Coppens et al. (35). Conjointly, among the factors that led to variable NAAT accuracy for MSSA or MRSA detection could be relatively smaller sample size, different DNA extraction protocols, the target genes adopted, and the quality of reaction materials. Figure 5 shows the pooled sensitivity and specificity of NAAT for detection of MSSA and MRSA in LRT samples. The high sensitivity of NAAT across sample types and relatively low number of noninterpretable findings in principle endorse the use of the test in respiratory samples. NAAT demonstrated consistently greater diagnostic accuracy for both MSSA (sensitivity of 0.94 [95% CI, 0.79 to 0.99], specificity of 0.95 [95% CI, 0.92 to 0.97]) and MRSA (sensitivity of 1.0 [95% CI, 0.91 to 1.0], specificity of 0.91 [95% CI, 0.87 to 0.94]) in sputum compared to other forms of LRT samples. As sputum collection is noninvasive, this is of substantial interest for the medical evaluation of patients with suspected pneumonia.

FIG 5.

NAAT pooled sensitivity and specificity estimates across LRT sample types.

Nevertheless, some of the constraints of current evidence demonstrate the need for further research and greater standardization of testing before the formulation of policies. Despite certain limitations, national CAP guidelines recommend sputum investigation for patients hospitalized with suspected pneumonia (17), both to facilitate microbiological diagnosis for therapy guidance and for surveillance purposes. Alternative lower respiratory specimen forms such as BAL fluid or ETA can be assumed to be less vulnerable to contamination. However, these require a semi-invasive procedure that is certainly not routine in nonintubated patients. About the comparison of this analysis with previous studies, a meta-analysis investigating the usefulness of sputum Gram stain in the detection of causative pathogens in CAP patients has been identified (58, 59). Albeit sputum Gram stain is a rapid, low-cost, and straightforward procedure, its role is still controversial in the initial assessment of CAP patients. Similarly, a meta-analysis by Parente et al. primarily summarized the importance of MRSA nasal screening to rule out MRSA pneumonia (54). Both of these articles were not LRT specific for the definitive diagnosis of staphylococcal pneumonia.

In addition, the NAAT subgroup study showed that in-house MSSA detection tests (sensitivity of 0.97 [95% CI, 0.93 to 0.99], specificity of 0.97 [95% CI, 0.96 to 0.98]) were comparable to those for MRSA (sensitivity of 1.0 [95% CI, 0.9 to 1.0], specificity of 0.94 [95% CI, 0.83 to 0.95]) in LRT samples (Table 2), whereas the overall pooled sensitivity and specificity of the commercial tests were higher for MSSA detection (sensitivity of 0.89 [95% CI, 0.86 to 0.92], specificity of 0.94 [95% CI, 0.93 to 0.94]) with respect to MRSA (sensitivity of 0.71 [95% CI, 0.65 to 0.77], specificity of 0.87 [95% CI, 0.86 to 0.89]). The overall sensitivity was relatively higher for the in-house tests than for the commercial NAAT. The DNA extraction protocol, the adopted target genes, the decontamination process, the presence of PCR inhibitors, and the quality of reaction materials may be among the factors contributing to bias. Aiming for more in-depth details among the commercial tests, we observed that FA-PP consistently provided higher summary estimates for identification of both MSSA (sensitivity of 1.0 [95% CI, 0.9 to 1.0], specificity of 0.90 [95% CI, 0.87 to 0.93]) and MRSA (sensitivity of 1.0 [95% CI, 0.82 to 1.00], specificity of 0.93 [95% CI, 0.89 to 0.95]) compared to all other tests (Table 2). The PLR for commercial tests was 9.6, indicating that patients with staphylococcal pneumonia are ∼10 times more likely to have a positive NAAT result than patients without staphylococcal pneumonia. Our meta-analysis suggested that a positive NAAT could be potentially useful as a rule-in test for diagnosing staphylococcal pneumonia, particularly qPCR (in-house) and FA-PP (commercial).

The key strengths of this research include a systematic search strategy, impartial selection criteria, and independent analyst assessment. The usefulness of the systemic guidelines, the precise reference standard, the bivariate random-effects model for data manipulation, and the prespecified heterogeneous subgroups between data sets were taken into account. As assessed by the QUADAS-2 tool, studies with a high risk of bias and high applicability concerns were omitted during screening. Studies that did not comply with the specified criteria for the diagnosis of staphylococcal pneumonia were excluded from the subsequent analysis. Besides, studies that identified pleural fluid or blood in the LRT specimen, which may appear to overestimate the diagnostic performance of the index test, were not included.

Some limitations of this research should be taken into consideration. Despite an exhaustive review of literature across repositories, we might have overlooked a few important studies. The impact of factors such as nonstandardized sample preparation, sample volume, amplification procedures, NAAT experience, and laboratory facilities on the accuracy of NAAT could not be discussed due to the high degree of reporting variability in the included studies. It should also be noted that the gene targets and NAAT standards for the studies were too different, which could be a probable cause for heterogeneity. Although in the meta-regression analysis, the study design, sample condition, and decontamination method were not significant sources of heterogeneity, these variables may potentially increase heterogeneity and restrict the generalizability of the overall diagnostic accuracy of the NAAT. Furthermore, the meta-analysis was limited due to an inadequate number of MRSA studies evaluating NAAT accuracy in various LRT sample forms and should be interpreted with caution. Finally, publication bias was a concern.

To the best of our knowledge, this is the first meta-analysis evaluating the diagnostic accuracy of NAAT across LRT specimens to rule out staphylococcal pneumonia. Our findings indicate that continued empirical use of vancomycin in patients suspected of staphylococcal pneumonia is unlikely to be justified. In order to prevent unnecessary and expensive anti-MRSA treatment, NAAT, specifically FA-PP, can be used as the preferred initial point of diagnosis test for staphylococcal pneumonia. With new guidelines under future development, consideration should be given to molecular amplification data compatible with local clinical data, including chest X-ray or computed tomography (CT) image findings, in order to diagnose and select appropriate antistaphylococcal agents. Given the limited data of NAAT across LRT sample types, a detailed investigation using a larger number of prospective studies would be of interest in elucidating the best sample form for future implication. Furthermore, future research should strive to continue thoroughly validating the diagnostic accuracy of NAAT in samples other than those evaluated in this meta-analysis, such as blood and pleural fluid, as well as its impact on the prognosis and clinical outcomes of patients.