Diagnostic Accuracy of a Molecular Drug Susceptibility Testing Method for the Antituberculosis Drug Ethambutol: a Systematic Review and Meta-Analysis

Abstract

Ethambutol (EMB) is a first-line antituberculosis drug; however, drug resistance to EMB has been increasing. Molecular drug susceptibility testing (DST), based on the embB gene, has recently been used for rapid identification of EMB resistance. The aim of this meta-analysis was to establish the accuracy of molecular assay for detecting drug resistance to EMB. PubMed, Embase, and Web of Science were searched according to a written protocol and explicit study selection criteria. Measures of diagnostic accuracy were pooled using a random effects model. A total of 34 studies were included in the meta-analysis. The respective pooled sensitivities and specificities were 0.57 and 0.93 for PCR-DNA sequencing that targeted the embB 306 codon, 0.76 and 0.89 for PCR-DNA sequencing that targeted the embB 306, 406, and 497 codons, 0.64 and 0.70 for detecting Mycobacterium tuberculosis isolates, 0.55 and 0.78 for detecting M. tuberculosis sputum specimens using the GenoType MTBDRsl test, 0.57 and 0.87 for pyrosequencing, and 0.35 and 0.98 for PCR-restriction fragment length polymorphism. The respective pooled sensitivities and specificities were 0.55 and 0.92 when using a lower EMB concentration as the reference standard, 0.67 and 0.73 when using a higher EMB concentration as the reference standard, and 0.60 and 1.0 when using multiple reference standards. PCR-DNA sequencing using multiple sites of the embB gene as detection targets, including embB 306, 406, and 497, can be a rapid method for preliminarily screening for EMB resistance, but it does not fully replace phenotypic DST. Of the reference DST methods examined, the agreement rates were the best using MGIT 960 for molecular DST and using the proportion method on Middlebrook 7H10 media.

INTRODUCTION

Tuberculosis (TB) is one of the most serious infectious diseases in the world. According to the 2013 Global Tuberculosis report by the World Health Organization (WHO), in 2012 an estimated 450,000 people developed multidrug-resistant TB (MDR-TB), and there were approximately 170,000 deaths due to MDR-TB worldwide (1). MDR-TB and extensively drug-resistant TB (XDR-TB) are among the greatest threats to the success of TB control in the world (2, 3). Ethambutol (EMB) is one of the first-line drugs included in the directly observed, treatment short-course antitubercular regimen recommended by the WHO (3). EMB is commonly used in combination with isoniazid (INH), rifampin (RIF), and pyrazinamide to treat TB, particularly when treating MDR-TB and XDR-TB (3). EMB has also been found to protect companion drugs against resistance, particularly INH (4). Initially, EMB was effective for preventing treatment failures caused by M. tuberculosis isolates resistant to other anti-TB drugs; however, the resistance rate of EMB has gradually increased in some regions and is close to 50% in TB patients that are retreated (5–7). In China, the resistance rate for EMB increased from 6.52% in 2007 to 17.18% in 2010 (8). Therefore, rapid and effective methods of drug susceptibility testing (DST) for M. tuberculosis resistance to EMB are vital so that clinicians can make appropriate, rational decisions regarding drugs that will be most effective for treatment. Conventional, phenotypic DST of EMB is the most commonly used approach in many countries. The WHO describes phenotypic DST as the gold standard testing method; however, phenotypic DST is not efficient when used clinically, due to the long turnaround time. Recently, the development of molecular technology has allowed molecular assay testing methods based on the detection of the embB gene to be more widely used for diagnosing TB drug resistance. These methods are attractive, since they can shorten the turnaround time for testing to less than 1 day (9). Many previous studies have examined the performance of molecular assays when testing for EMB resistance based on the embB gene; however, the sensitivity and specificity results have been inconsistent. In the present study, a systematic review and meta-analysis was performed to evaluate the overall accuracy of using molecular assays to test for EMB resistance in M. tuberculosis isolates and sputum samples. Factors associated with the heterogeneity of findings between studies were also identified, and the effects of study and test characteristics on diagnostic accuracy were assessed.

MATERIALS AND METHODS

Systematic review.

This systematic review was performed according to the guidelines of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PrismA) set by the PrismA Group (10).

Search strategy and selection criteria.

A search for biomedical articles in English, which had been published between January 1990 and September 2013, was conducted using the electronic databases PubMed, Embase, and Web of Science. The search terms used were as follows: ethambutol, embB, and tuberculosis.

Selected studies met the following inclusion criteria: the studies (i) used molecular assays for testing the susceptibility of M. tuberculosis to EMB, (ii) used the embB gene as the detection target of molecular detection assays in clinical TB specimens or M. tuberculosis isolates, (iii) evaluated the accuracy (sensitivity and specificity) of the molecular assays, and (iv) had one or more reference standards that were recommended by the WHO. The reference standards included the proportion method (PM) on Lowenstein-Jensen (LJ) media (EMB critical concentration, 2 μg/ml), Middlebrook 7H10 media (EMB critical concentration, 5 μg/ml), Middlebrook 7H11 media (EMB critical concentration, 7.5 μg/ml), radiometric Bactec 460 media (EMB critical concentration, 2.5 μg/ml), and MGIT 960 media (EMB critical concentration, 5 μg/ml).

Studies were excluded if they met the following predetermined criteria: (i) the study was a review or the sensitivity and specificity data were grouped for meta-analysis by assay category, and/or (ii) the full-text of the study was not available in English. Studies with fewer than 20 samples were also excluded in order to reduce selection bias.

Data extraction and quality assessment.

Two reviewers independently assessed the final set of articles and extracted the data using a pilot data extraction form. Initially, both reviewers read the titles and abstracts of all studies. The two reviewers then evaluated the studies that were considered possibly eligible. The full-text of each study was carefully read, according to the inclusion criteria, to assess whether it should be included. Disagreements were resolved by consensus, and the authors of any studies in question were contacted to obtain more detailed information. The data extracted from the articles included the first author, the year of publication, the sample size, the specimen type, and the numbers of true-positive, false-positive, false-negative, and true-negative results. In addition, because embB 306 was the main mutation codon, data targeting the embB 306 codon was primarily extracted; however, data were also extracted from some studies that also targeted embB 406 and embB 497 by PCR-DNA sequencing. Two blinded reviewers assessed the quality of the studies using QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies), the revised tool for QUADAS. QUADAS-2 is used in systematic reviews to evaluate the risk of bias in, and the applicability of, diagnostic accuracy studies. It is comprised of four domains: patient selection, index test, reference standard, and flow and timing. Each domain is assessed for risk of bias, and the first three domains are also assessed for applicability. Signaling questions are included to help judge the risk of bias (11). The risk of bias is judged as “low,” “high,” or “unclear.” If the answers to all signaling questions for a domain are “yes,” then the risk of bias is judged to be low. If any signaling question is answered “no,” the potential for bias exists. The unclear category should only be used when insufficient data are reported to make a judgment (11). Applicability was judged as low, high, or unclear using similar criteria.

Statistical analysis.

Analyses were performed using two software programs: the Meta-Disc, version 1.4 (XI Cochrane Colloquium, Barcelona, Spain) and Cochrane RevMan 5.2. For each study, measures of test accuracy were computed using standard methods as follows: sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), positive predictive value (PPV), negative predictive value (NPV), and diagnostic odds ratio (DOR); these measures were pooled using the random effects model (12, 13). The area under the summary receiver operating characteristic (SROC) curve is a global measure of overall performance; therefore, the SROC curve was used to evaluate the effect of the assay, with an area under the curve of 1 indicating perfect discriminatory ability (14). Heterogeneity was analyzed by using chi-square (χ²) and I-square (I²) tests (12).

RESULTS

Figure 1 outlines the study selection process. A total of 339 potentially relevant citations were identified from all searches, and a final total of 28 (9, 15–41) eligible articles were included in the meta-analysis. Since some articles used more than one detection technique or more than one sample type, 34 independent studies were defined in the meta-analysis. Both PCR-DNA sequencing and GenoType MTBDRsl were used in two articles (24, 27), and both PCR-DNA sequencing and GenoType MTBDRsl were used to detect clinical isolated strains and sputum specimens, respectively, in another two articles (23, 29). An additional 311 studies were excluded for the following reasons: the study was a duplicate among the PubMed, Embase, and Web of Science databases, the reference testing of the study did not meet eligibility criteria, the study was a review, or the diagnostic 2×2 table could not be extracted.

FIG 1.

Flow chart of study selection.

Study characteristics.

The overall sample size from the 34 selected studies was 5,212, which included 2404 EMB-resistant isolates and 2808 EMB-susceptible isolates (Table 1). The molecular assays included PCR-DNA sequencing (n = 16), GenoType MTBDRsl (n = 11), pyrosequencing (n = 3), PCR and restriction fragment length polymorphism (PCR-RFLP) (n = 2), and other methods (n = 2). The other methods included a one-step amplification refractory mutation system and variable-number tandem repeats of mycobacterial interspersed repetitive units. Among the PCR-DNA sequencing group (n = 16), thirteen studies used embB 306 as the detection target, and three studies used embB codons 306, 406, and 497 as the detection targets. Seven studies used the LJ PM for the reference test, two studies used Bactec MGIT 960, one study used Bactec 460, and six studies used other reference standards, including Bactec 460 or MGIT 960. In the GenoType MTBDRsl group (n = 11), eight studies involved detection of clinical isolated strains, and three studies involved clinical sputum specimens.

TABLE 1.

Summary of the studies included in the meta-analysis

Method and referencea	Yr	Study settingb	Country	Specimen type	Reference test(s)c	Size (no. of strains)
PCR-DNA sequencing
Escalante (15)	1998	UN	Peru	Isolate	LJ PM, Bactec 460	29
Lee (16)	2004	UN	Singapore	Isolate	Bactec 460	45
Ramaswamy (17)	2004	UN	Mexico	Isolate	LJ PM	50
Zhang (18)	2007	UN	China	Isolate	LJ PM	66
Sekiguchi (9)	2007	UN	Japan, Poland	Isolate	AgarPM, LJ PM, MGIT 960	138
Guo (19)	2008	UN	China	Isolate	LJ PM	66
Jadaun (20)	2008	UN	India	Isolate	LJ PM	30
Perdigao (21)	2009	UN	Lisbon	Isolate	MGIT 960	109
Plinke (20)	2009	UN	Karakalpakstan	Isolate	LJ PM, MGIT 960	197
Hillemann (23)	2009	NRL	UN	Isolate	MGIT 960, LJ PM	106
Brossier (24)	2010	French Reference Center for Mycobacteria	France	Isolate	LJ PM	52
Hu (25)	2010	Microbiology laboratory at Fudan University, Shanghai	China	Isolate	LJ PM	351
Shi (26)	2011	Tuberculosis Reference Laboratory at Henan Provincial CDC	China	Isolate	LJ PM	160
Huang (27)	2011	Reference Laboratory of Mycobacteriology, Research and Diagnostic Center, CDC	China	Isolate	AgarPM, MGIT 960	234
Campbell (28)	2011	Mycobacteriology Laboratory Branch	UN	Isolate	AgarPM	314
Miotto (29)	2012	UN	UN	Isolate	MGIT960	175
GenoType MTBDRsl
Hillemann (23)	2009	NRL	UN	Isolate	MGIT 960, LJ PM	106
Brossier (24)	2010	French Reference Center for Mycobacteria	France	Isolate	LJ PM	52
Kiet (30)	2010	Pham Ngoc Thach Hospital	Vietnam	Isolate	LJ PM	62
Huang (27)	2011	Reference Laboratory of Mycobacteriology, Research and Diagnostic Center, CDC	China	Isolate	AgarPM, MGIT 960	234
Said (31)	2012	Diagnostic Microbiology Laboratory at Tshwane Academic Division	Africa	Isolate	AgarPM	316
Zivanovic (32)	2012	UN	Serbia	Isolate	LJ PM, MGIT 960	19
Ignatyeva (33)	2012	SRL	Estonian	Isolate	MGIT 960	195
Miotto (29)	2012	UN	UN	Isolate	MGIT 960	174
Hillemann (27)	2009	NRL	UN	Sputum	MGIT 960, LJ PM	60
Ajban (34)	2012	P. D. Hinduja National Hospital and Medical Research Centre	India	Sputum	MGIT 960	150
Miotto (29)	2012	UN	UN	Sputum	MGIT 960	56
PCR-RFLP
Ahmad (35)	2006	UN	Kuwait	Isolate	Bactec 460	197
Ahmad (36)	2008	UN	Beirut, Dubai	Isolate	Bactec 460	50
Pyrosequencing
Isola (37)	2005	UN	Abkhazia	Isolate	AgarPM	28
Zhao (38)	2004	UN	China	Isolate	Bactec 460	42
Engstrom (39)	2012	SRL	UN	Isolate	LJ PM	272
Other methods
Johnson (40)	2006	Routine diagnostic laboratory	Africa	Isolate	AgarPM, Bactec 460	352
Shen (41)	2007	Tuberculosis Reference Laboratory of the Shanghai	China	Isolate	LJ PM	162

^aReferences are each indicated by the first author's last name, with the corresponding reference number in parentheses.

^bAbbreviations: SRL, Supranational Reference Laboratory; NRL, National Reference Laboratory; UN, unknown; CDC, Centers for Disease Control and Prevention.

^cAbbreviations: AgarPM, PM on agar culture medium; LJ PM, PM on LJ culture medium.

Data extraction and quality assessment.

All extracted data were double-checked by a second author and filled a 2×2 table (Table 1), as shown in the study report. A quality assessment of all of the included studies is shown in Fig. 2. Overall, the quality of the study was satisfactory. As shown in the Fig. 2, 9 (26%) studies were at low risk, 7 studies (20%) were of unclear risk, and 18 studies (52%) were at high risk for patient selection bias due to inconsecutive or nonrandom patient selection. Most of the studies were at low risk for index test (n = 22, 64%) and reference standard (n = 28, 82%) bias. A total of 23 studies (67%) were at high risk for flow and timing bias. One reason for this was the fact that not all selected patients were included in the diagnostic analysis and the other reason was that the patients did not receive the same reference standards. As for applicability, 20 studies (59%) were at high risk of the patient selection; however, all select studies (n = 34, 100%) were at low risk of reference standard and index test.

FIG 2.

Quality assessment of included studies.

Group analysis according to detection methods. (i) PCR-DNA sequencing group.

A total of 16 studies that used a single-site amino acid replacement at position 306 of the embB gene as the molecular marker for the detection of EMB drug susceptibility met the inclusion criteria. The pooled sensitivity and specificity estimates for the 16 studies were 0.57 (95% confidence interval [CI] = 0.54 to 0.60) (Fig. 3A) and 0.93 (95% CI = 0.91 to 0.94) (Fig. 3B), respectively. The PLR and NLR were 10.19 (95% CI = 4.69 to 22.10) and 0.48 (95% CI = 0.42 to 0.55), respectively. The DOR was 21.28 (95% CI = 9.55 to 47.43), the PPV was 0.85 (95% CI = 0.82 to 0.88), and the NPV was 0.75 (95% CI = 0.73 to 0.77) (Table 2). The area under the SROC curve was 0.5643, and Q* (the point where sensitivity and specificity are equal, which is the point closest to the ideal top-left corner of the SROC space) was 0.5483 (Fig. 3E).

FIG 3.

Forest plots of the pooled sensitivity and specificity and SROC curve of PCR-DNA sequencing for detection of EMB drug susceptibility. Each solid circle indicates the point estimate of sensitivity and specificity from each study; the size of the solid circle indicates the size of each study. Both error bars and discontinuous lines indicate the 95% CI values. Diamond indicates the pooled sensitivity and specificity for all of the studies. The curve is the regression line that summarizes the overall diagnostic accuracy. (A) Sensitivity of PCR-DNA sequencing used the embB 306 codon as the target; (B) specificity of PCR-DNA sequencing used the embB 306 codon as the target; (C) sensitivity of PCR-DNA sequencing used the embB 306, 406, and 497 codons as the target; (D) specificity of PCR-DNA sequencing used the embB 306, 406, and 497 codons as the target; (E) SROC curve of PCR-DNA sequencing used the embB 306 codon as the target; (F) SROC curve of PCR-DNA sequencing used the embB 306, 406, and 497 codons as the target.

TABLE 2.

All the pooled accuracy measures in the meta-analysis

Group	No. of studies (n)	Accuracy parameter value (95% CI)a
		Sensitivity	Specificity	PLR	NLR	PPV	NPV	DOR
PCR-DNA sequence (codon 306)	16 (2,122)	0.57 (0.54–0.60)	0.93 (0.91–0.94)	10.19 (4.69–22.10)	0.48 (0.42–0.55)	0.85 (0.82–0.88)	0.75 (0.73–0.77)	21.28 (9.55–47.43)
PCR-DNA sequence (codons 306, 406, and 497)	3 (439)	0.76 (0.70–0.81)	0.89 (0.83–0.93)	10.18 (1.28–80.99)	0.27 (0.22–0.33)	0.92 (0.88–0.95)	0.69 (0.63–0.65)	33.69 (4.53–250.90)
GenoType MTBDRsl (isolates)	8 (1,160)	0.64 (0.60–0.67)	0.70 (0.67–0.74)	5.17 (1.95–13.66)	0.46 (0.34–0.61)	0.69 (0.65–0.73)	0.65 (0.61–0.69)	12.85 (3.52–46.96)
GenoType MTBDRsl (sputum)	3 (266)	0.55 (0.47–0.63)	0.78 (0.69–0.85)	2.86 (0.98–8.29)	0.56 (0.47–0.68)	0.76 (0.67–0.84)	0.57 (0.49–0.65)	5.52 (2.07–14.71)
Pyrosequencing	3 (343)	0.57 (0.49–0.65)	0.87 (0.82–0.92)	4.16 (2.80–6.19)	0.54 (0.40–0.72)	0.80 (0.71–0.87)	0.70 (0.63–0.76)	8.87 (5.14–15.30)
PCR-RFLP	2 (222)	0.35 (0.24–0.46)	0.98 (0.94–1.00)	12.84 (4.29–38.46)	0.68 (0.57–0.80)	0.90 (0.74–0.98)	0.72 (0.65–0.78)	18.53 (5.68–60.45)
Other group	2 (541)	0.85 (0.75–0.92)	0.96 (0.92–0.96)	38.44 (0.17–8,495.37)	0.10 (0.00–9.95)	0.79 (0.69–0.88)	0.97 (0.96–0.99)	378.81 (0.27–522,235.40)

^aAbbreviations: PLR, positive likelihood ratio; NLR, negative likelihood ratio; PPV, positive predictive value; NPV, negative predictive value; DOR, diagnostic odds ratio.

A total of 3 studies used multiple single sites (amino acid replacement at positions 306, 406, and 497) of the embB gene as the molecular markers for the molecular detection of EMB drug susceptibility. The pooled sensitivity and specificity for the 3 studies were 0.76 (95% CI = 0.70 to 0.81) (Fig. 3C) and 0.89 (95% CI = 0.83 to 0.93) (Fig. 3D), respectively. PLR and NLR was 10.18 (95% CI = 1.28 to 80.99) and 0.27 (95% CI = 0.22 to 0.33), respectively. DOR was 33.69 (95% CI = 4.53 to 250.90). PPV was 0.92(95% CI = 0.88 to 0.95) and NPV was 0.69 (95% CI = 0.63 to 0.65) (Table 2). The area under the SROC curve was 0.8293, and Q* was 0.7620 (Fig. 3F).

(ii) GenoType MTBDRsl group: isolate subgroup.

Eight studies detected EMB-resistant isolates using the GenoType MTBDRsl assay. The pooled sensitivity and specificity were 0.64 (95% CI = 0.60 to 0.67) (Fig. 4A) and 0.70 (95% CI = 0.67 to 0.74) (Fig. 4B), respectively. The PLR and NLR were 5.17 (95% CI = 1.95 to 13.66) and 0.46 (95% CI = 0.34 to 0.61), respectively. The PPV and NPV were 0.69 (95% CI = 0.65 to 0.73) and 0.65 (95% CI = 0.61 to 0.69), respectively. The DOR was 12.85 (95% CI = 3.52 to 46.96) (Table 2). The area under the SROC curve was 0.6455, and Q* was 0.6102 (Fig. 4E).

FIG 4.

Forest plots of the pooled sensitivity and specificity and SROC curve of GenoType MTBDRsl for detection of EMB drug susceptibility. Each solid circle indicates the point estimate of sensitivity and specificity from each study; the size of the solid circle indicates the size of each study. Both error bars and discontinuous lines indicate the 95% CI values. A diamond indicates the pooled sensitivity and specificity for all of the studies. The curve is the regression line that summarizes the overall diagnostic accuracy. (A) Sensitivity of GenoType MTBDRsl for the detection of EMB drug susceptibility in isolates; (B) specificity of GenoType MTBDRsl for the detection of EMB drug susceptibility in isolates; (C) sensitivity of GenoType MTBDRsl for the detection of EMB drug susceptibility in sputum specimens; (D) specificity of GenoType MTBDRsl for the detection of EMB drug susceptibility in sputum specimens; (E) SROC curve of GenoType MTBDRsl for the detection of EMB drug susceptibility in isolates; (F) SROC curve of GenoType MTBDRsl for the detection of EMB drug susceptibility in sputum specimens.

(iii) GenoType MTBDRsl group: sputum subgroup.

Three studies used the Genotype MTBDRsl assay to detect EMB resistance on sputum. The pooled sensitivity and specificity were 0.55 (95% CI = 0.47 to 0.63) (Fig. 4C) and 0.78 (95% CI = 0.69 to 0.85) (Fig. 4D), respectively. The PLR and NLR were 2.86 (95% CI = 0.98 to 8.29) and 0.56 (95% CI = 0.47 to 0.68), respectively. The PPV and NPV were 0.76 (95% CI = 0.67 to 0.84) and 0.57 (95% CI = 0.49 to 0.65), respectively. The DOR was 5.52 (95% CI = 2.07 to 14.71) (Table 2). The area under the SROC curve was 0.7078, and Q* was 0.6591 (Fig. 4F).

(iv) Pyrosequencing group.

The pooled sensitivity and specificity for detection of resistance to EMB were 0.57 (95% CI = 0.49 to 0.65) (Fig. 5A) and 0.87 (95% CI = 0.82 to 0.92) (Fig. 5B), respectively, with pyrosequencing. The PLR and NLR were 4.16 (95% CI = 2.80 to 6.19) and 0.54 (95% CI = 0.40 to 0.72), respectively. The DOR was 8.87 (95% CI = 5.14 to 15.30). The PPV and NPV were 0.80 (95% CI = 0.71 to 0.87) and 0.70 (95% CI = 0.63 to 0.76), respectively (Table 2). The area under the SROC curve was 0.7549, and Q*was 0.6975 (Fig. 5C).

FIG 5.

Forest plots of the pooled sensitivity and specificity and SROC curve of pyrosequencing for detection of EMB drug susceptibility. Each solid circle indicates the point estimate of sensitivity and specificity from each study; the size of the solid circle indicates the size of each study. Both error bars and discontinuous lines indicate the 95% CI values. A diamond indicates the pooled sensitivity and specificity for all of the studies. The curve is the regression line that summarizes the overall diagnostic accuracy. (A) Sensitivity; (B) specificity; (C) SROC curve.

(v) PCR-RFLP group.

The pooled sensitivity and specificity for detection of resistance to EMB were 0.35 (95% CI = 0.24 to 0.46) (Fig. 6A) and 0.98 (95% CI = 0.94 to 1.00) (Fig. 6B), respectively, using PCR-RFLP. The PLR and NLR were 12.84 (95% CI = 4.29 to 38.46) and 0.68 (95% CI = 0.57 to 0.80), respectively. The DOR was 18.53 (95% CI = 5.68 to 60.45). The PPV and NPV were 0.90 (95% CI = 0.74 to 0.98) and 0.72 (95% CI = 0.65 to 0.78), respectively (Table 2).

FIG 6.

Forest plots of the pooled sensitivity and specificity of PCR-RFLP for detection of EMB drug susceptibility. Each solid circle indicates the point estimate of sensitivity and specificity from each study; the size of the solid circle indicates the size of each study. Both error bars and discontinuous lines indicate the 95% CI values. A diamond indicates the pooled sensitivity and specificity for all of the studies. The curve is the regression line that summarizes the overall diagnostic accuracy. (A) Sensitivity; (B) specificity.

(vi) “Other methods” group.

The pooled sensitivity and specificity for detection of resistance to EMB were 0.85 (95% CI = 0.75 to 0.92) (Fig. 7A) and 0.96 (95% CI = 0.92 to 0.96) (Fig. 7B), respectively, using other methods. The PLR and NLR were 38.44 (95% CI = 0.17 to 8,495.37) and 0.10 (95% CI = 0.00 to 9.95), respectively. The DOR was 378.81 (0.27 to 522,235.40), and the PPV and NPV were 0.79 (95% CI = 0.69 to 0.88) and 0.97 (95% CI = 0.96 to 0.99), respectively (Table 2).

FIG 7.

Forest plots of the pooled sensitivity and specificity of other methods for detection of EMB drug susceptibility. Each solid circle indicates the point estimate of sensitivity and specificity from each study; the size of the solid circle indicates the size of each study. Both error bars and discontinuous lines indicate the 95% CI values. A diamond indicates the pooled sensitivity and specificity for all of the studies. The curve is the regression line that summarizes the overall diagnostic accuracy. (A) Sensitivity; (B) specificity.

Group analysis according to reference method.

Some studies reported a high agreement between GenoType MTBDRsl and PCR-DNA sequencing (24, 25, 28, 31). Pyrosequencing can provide the same accuracy as the sequencing method (38); thus, we considered the studies that used the three molecular assays for the detection of EMB resistance as a group in order to stratify by type of reference standard. All of the studies were divided into three groups, according to the drug concentration of the reference tests. The low-concentration group included all of the studies that used LJ PM with a drug concentration of 2 μg/ml or on Bactec 460 medium with a drug concentration of 2.5 μg/ml. The high-concentration group included all of the studies that used the MGIT 960 and the PM on Middlebrook 7H10 medium with a drug concentration of 5 μg/ml. The multiple-concentration group included the studies that used more than one reference standard. The pooled accuracy measurements are shown in Table 3.

TABLE 3.

Group analyses for different drug concentrations and different countries

Group	Accuracy parameter value (95% CI)a
	Sensitivity	Specificity	PLR	NLR	DOR
Low concn	0.55 (0.51–0.60)	0.92 (0.89–0.94)	5.38 (4.18–6.92)	0.52 (0.47–0.57)	11.00 (7.87–15.36)
High concn	0.67 (0.64–0.79)	0.73 (0.70–0.76)	2.96 (1.59–5.51)	0.56 (0.41–0.76)	5.42 (2.19–13.43)
Multiple reference stands	0.60 (0.56–0.65)	1.0 (0.99–1.0)	48.83 (16.15–147.65)	0.39 (0.32–0.49)	141.98 (46.56–432.96)
Europe	0.69 (0.63–0.75)	0.87 (0.91–0.96)	4.53 (2.83–7.26)	0.41 (0.27–0.62)	11.84 (5.71–24.55)
Asia	0.57 (0.53–0.61)	0.94 (0.92–0.96)	6.98 (393–12.41)	0.48 (0.41–0.56)	15.92 (7.84–32.29)

^aAbbreviations: PLR, positive likelihood ratio; NLR, negative likelihood ratio; DOR, diagnostic odds ratio.

Group analysis according to specimen resource regions.

All of the studies were divided into two groups according to the region (Asia or Europe) from which the samples originated. The pooled accuracy measures are shown in Table 3.

Heterogeneity.

The heterogeneity test results of pooled accuracy measures are shown in Table 4. When the studies were stratified by type of index test, there was homogeneity in all of the pooled sensitivity data and NLRs, with the exception of the GenoType MTBDRsl assay detecting EMB isolates and the “other methods” group. However, significant heterogeneity was observed in most of pooled specificity data and PLRs. When the studies were analyzed by drug concentration reference standards, homogeneity was present in all of the pooled measures in the low drug concentration group. Homogeneity was observed in samples from Asia when the sensitivity, PLR, NLR and DOR of selected studies were pooled.

TABLE 4.

Heterogeneity test results of pooled accuracy measures

Group	I2 (%), Pa
	Sensitivity	Specificity	PLR	NLR	DOR
PCR-DNA sequencing (codon 306)	35.8, 0.0768	91.0, <0.05	89.1, <0.05	54.8, 0.0035	80.9, <0.05
PCR-DNA sequencing (codons 306, 406, and 497)	0.0, 0.6745	91.6, <0.05	86.4, <0.05	0.0, 0.3940	78.2, <0.05
GenoType MTBDRsl (isolates)	68.7, 0.002	95.1, <0.05	91.9, <0.05	83.0, <0.001	87.9, <0.05
GenoType MTBDRsl (sputum)	61.7, 0.0734	93.1, <0.05	79.5, 0.0075	0.0, 0.7313	37.3, 0.2030
Pyrosequencing	57.9, 0.1349	82.4, <0.05	0.0, 0.5298	47.2, 0.1505	0.0, 0.7133
PCR-RFLP	16.3, 0.2744	0.0, 0.3486	0.0, 0.8218	10.1, 0.2916	0.0, 0.7402
Other group	94.3, <0.05	96.9, <0.05	96.4, <0.05	89.8, <0.05	95.2, <0.05
Low concn	8.8, 0.3576	50.7, 0.0182	0.0, 0.5546	4.0, 0.4062	0.0, 0.8091
High concn	72.8, <0.05	94.2, <0.05	90.1, <0.001	81.9, <0.05	87.5, <0.05
Multiple reference stands	64.2, <0.05	21.5, 0.2587	29.9, 0.1897	69.3, <0.05	21.8, 0.2558
Europe	76.8, <0.05	0.00, 0.8070	0.00, 0.6223	74.7, <0.001	23.0, 0.2727
Asia	49.7, 0.0503	79.3, <0.05	59.5, 0.059	54.4, 0.0156	57.6, 0.0087

^aAbbreviations: PLR, positive likelihood ratio; NLR, negative likelihood ratio; DOR, diagnostic odds ratio.

DISCUSSION

Rapid and effective drug susceptibility testing for M. tuberculosis has been a hot topic in research worldwide. Early detection of drug resistance in TB patients can contribute to TB control and management and reduce the prevalence and transmission of TB. The WHO has called for research into a fast and accurate drug susceptibility testing method to reduce the spread of M. tuberculosis and drug-resistant TB in order to reduce the global TB burden (3). The use of molecular methods has been recommended as an effective way to decrease the turnaround time for the detection of drug resistance in M. tuberculosis (36, 42). Recently, molecular methods for drug susceptibility testing of M. tuberculosis have begun to be more widely used. Molecular assays for drug susceptibility testing to RIF and INH, based on the rpoB gene and katG gene, have become more effective. The turnaround time for PCR–single-strand conformational polymorphism analysis assay and pyrosequencing is less than 48 h, making it substantially faster than conventional drug susceptibility testing methods (43, 44). Earlier studies suggested that mutations in the embB gene, in particular amino acid replacements at position 306, were the major mechanism for the acquisition of resistance to EMB in M. tuberculosis (45–48). Consequently, in recent years, many studies have focused on molecular drug susceptibility testing of EMB based on the embB gene for detection of drug-resistant M. tuberculosis strains. The most common methods have been PCR-DNA sequencing, GenoType MTBDRsl, PCR-RFLP, and pyrosequencing. Therefore, it is important that the overall accuracy of the methods used for the detection of EMB resistance be explored.

embB 306 is the main reason for EMB resistance in M. tuberculosis (45–48); thus, in most of the selected studies in this review, the detection target was only embB 306. Therefore, the diagnostic value of different detection methods, using embB 306 as the detection target, was initially performed here. The specificities of every molecular technique were >0.7, and the specificity of PCR-DNA sequencing and PCR-RFLP were >0.93; these are similar to the specificities reported for RIF and INH (43, 44). This suggests that these techniques are good for detecting M. tuberculosis strains susceptible to EMB. Of all of the detection methods examined, the sensitivity of GenoType MTBDRsl was the highest, whereas the sensitivity of PCR-RFLP was the lowest. These results suggest that GenoType MTBDRsl is best when embB 306 is the only detection target. However, the pooled sensitivity for detection of EMB resistance by GenoType MTBDRsl was still lower than that of RIF and INH, which use rpoB and katG as detection targets, respectively. Therefore, resistant strains can easily be judged as susceptible, when embB 306 is used as the detection target, by mistake. The specificity showed large variations among different studies in genotype GenoType MTBDRsl group and lower pooled specificity than other groups due to the low specificities (0.55, 0.56) of two included studies (29, 31) with large sample sizes. In the study by Said et al. (31), all of the isolates were MDR, whereas Hazbón et al. (49) reported that embB 306 mutations that may be associated with MDR and that MDR isolates phenotypically susceptible to EMB carried mutations at codon 306 of the embB gene. This may explain the low specificity noted in the Said study.

Although the embB 306 mutation is the mutant hot site in EMB-resistant strains (45–48), it was not enough to cover a single detection site for embB 306 (50, 51). Other studies used multiple sites of the embB gene as detection sites when using molecular detection techniques (16, 17, 26–28). Consequently, in the present study, the diagnostic value of using multiple sites of embB as detection targets was compared to using a single site of embB as a detection target. The sensitivity and PPV of using multiple sites as detection target were 19 and 7% higher, respectively, than when using only embB 306 (0.57 versus 0.76 [P < 0.05]; 0.85 versus 0.92 [P < 0.05]), and 12 and 23% higher, respectively, than when using GenoType MTBDRsl that targeted embB 306 (0.64 versus 0.76 [P < 0.05]; 0.69 versus 0.92 [P < 0.05]). As has been reported in some studies, embB 406 and 497 mutations often occur in strains that do not have embB 306 mutations (45, 51, 52); thus, using multiple sites as targets can enhance the sensitivity. Although the pooled sensitivity of multiple sites was still not high enough (0.76), the PPV was high (0.92). This suggests a high accuracy when judging resistant strains using multiple sites as detection targets. Multiple-site detection, using rapid PCR-DNA sequencing, showed better agreement with the reference methods compared to the other molecular DST methods examined and could be a quick method to preliminarily screen for EMB-resistant strains.

Although the sensitivity of multisite (embB 306, 406, and 497) detection was higher than the sensitivity of single site (embB 306) detection, the sensitivity and specificity of multisite detection still did not satisfy the clinical requirement. This depends on EMB resistance mechanisms. EMB resistance is regulated by both gene mutations and gene expression (51). embB 306, 406, and 497 were only shown to be associated with embB mutations and EMB resistance. It has been reported that EMB resistance is regulated by multiple genes, such as embA, embB, embC, and Rv3792 (53). When the EMB concentration of a reference method is <5 μg/ml, the embB mutation rate in EMB-resistant strains is not high enough to produce adequate sensitivity. In addition, although the association between embB 306 mutations and EMB resistance in clinical M. tuberculosis isolates is so strong that it has been proposed as a marker for EMB resistance in diagnostic tests, another study reported that in some M. tuberculosis strains, embB 306 mutations do not cause EMB resistance but instead predispose M. tuberculosis to become resistant to any antibiotic and to become MDR (49). This may explain the reason why embB 306 mutations occur in EMB-susceptible strains and also suggests that embB 306 mutations can serve as a marker for TB cases that are at increased risk for the development of drug resistance.

The MIC range of EMB-susceptible and -resistant strains was narrow (54), and the concentrations of different reference methods were different. It has been reported that the results of the Bactec 460 method and AgarPM were discordant (54, 55). It was reported that false susceptibility to EMB is of little consequence in settings of susceptible M. tuberculosis isolates (55). Thus, the effect of different drug concentrations on the evaluation of molecular detection of EMB resistance was analyzed. For example, when embB 306 was the target, the result showed a lower sensitivity in the group with the low drug concentration in reference methods compared to the other groups. This suggests that different reference methods with different drug concentrations affect the evaluation of the molecular detection of EMB resistance based on embB. The agreement between molecular DST and MGIT 960 and between molecular DST and PM on Middlebrook 7H10 media was better compared to all other reference DST methods examined. This result also supports the speculation of Parsons et al. (51) that the mutation rate of embB 306 is higher in more-resistant strains.

The rates of gene mutations related to drug resistance in M. tuberculosis were varied depending on the region. The meta-analysis of pyrosequencing for the rapid detection of RIF resistance in M. tuberculosis showed that the sensitivity of molecular detection for INH resistance, based on katG, was lower in Asia than in Europe (43). The present result showed that the sensitivity and specificity of using molecular techniques to identify EMB resistance, based on embB 306 as the detection target, was higher in Europe than in Asia. This suggests that using molecular detection to identify EMB resistance, based on embB 306, works better in Europe.

Some summary measures were significantly heterogeneous in the present study. Therefore, group analyses and subgroup analyses were used to examine the reasons for heterogeneity. The results suggested that the variability in the reference and detection methods among studies could partly explain the heterogeneity. In the low-reference-drug-concentration group, all of the studies used a consistent LJ PM reference, with the exception of two studies that used Bactec 460 and one study that used LJ PM and Bactec 460. Even so, heterogeneity persisted in some of the summary estimates. Since any factor can affect the heterogeneity in a diagnostic meta-analysis, other factors—such as variations in the study, population (e.g., severity of disease and comorbidities), the design method, and/or sample collection method (consecutive or random collection of samples)—likely resulted in variations in the accuracy estimates in the present study (56).

There were some limitations to our review. First, since only abstracts and only three databases were searched, some studies were not included in this review. Second, due to the linguistic abilities of our team, only studies that were published in English were included. Third, some studies with missing data were excluded, since the authors could not be contacted. The effects of factors such as laboratory infrastructure and expertise with molecular detection of EMB resistance could not be analyzed since this information was not available.

Conclusion.

Molecular drug susceptibility testing methods, using embB 306 as a single detection target, are not good for the detection of EMB resistance due to low sensitivity. PCR-DNA sequencing, using multiple sites of the embB gene, including embB 306, 406, and 497 as detection targets, could be a rapid method for preliminarily screening for EMB-resistant strains. However, the drug susceptibility results using PCR-sequencing were not strictly accurate. Therefore, the molecular DST cannot fully replace phenotypic DST. Molecular DST with MGIT 960 and the PM on Middlebrook 7H10 media provided the best agreement rates.