Pyrazinamide Susceptibility Testing in Mycobacterium tuberculosis: a Systematic Review with Meta-Analyses^,

Abstract

Standard culture-based testing of the susceptibility of Mycobacterium tuberculosis to pyrazinamide is difficult to perform. This systematic review with meta-analyses evaluated the roles of molecular assays targeting pncA and of pyrazinamidase assays. PubMed and Embase were searched for relevant publications in English. Sensitivity and specificity were estimated in bivariate random-effects models. Of 128 articles identified, 73 sets of data involving culture isolates were initially included in meta-analyses. Summary estimates of sensitivity and specificity, respectively, were 87% and 93% for PCR-DNA sequencing (n = 29), 75% and 95% for PCR-single-stranded conformation polymorphism (SSCP) (n = 5), 96% and 97% for a mixture of other molecular assays (n = 6), and 89% and 97% for pyrazinamidase assays using the Wayne method (n = 33). The median prevalence (range) of pyrazinamide resistance was 51% (31% to 89%) in multidrug-resistant M. tuberculosis isolates and 5% (0% to 9%) in non-multidrug-resistant isolates. Excluding studies with possibly considerable false resistance in the reference assay gave the following estimates of sensitivity and specificity, respectively: 92% and 93% for PCR-DNA sequencing (n = 20), 98% and 96% for other molecular assays (n = 5), and 91% and 97% for the Wayne assay (n = 27). The Wayne assay had significant funnel plot asymmetry, so the test performance might have been overestimated. Considering the prevalence of pyrazinamide resistance in different clinical settings, PCR-DNA sequencing, and possibly other molecular assays targeting pncA, can detect pyrazinamide resistance in multidrug-resistant M. tuberculosis isolates, with predictive values largely exceeding 90%, and rule out pyrazinamide resistance in non-multidrug-resistant isolates, with predictive values exceeding 99%. Molecular assays are probably the way forward for detecting pyrazinamide resistance.

INTRODUCTION

Multidrug-resistant tuberculosis (MDR-TB), including extensively drug-resistant tuberculosis (XDR-TB), has emerged as a global health threat (44, 73). In 2008, there were an estimated 390,000 to 510,000 cases of MDR-TB globally (73). The proportion of MDR-TB among all incident TB cases is estimated to be 3.6% worldwide (73). Pyrazinamide, a first-line drug with remarkable sterilizing activity (39, 40, 79), has an important role in the treatment of both drug-susceptible and drug-resistant tuberculosis (74). MDR-TB with bacillary resistance to pyrazinamide would further worsen prognoses (43), and the rising prevalence of such MDR-TB is noted with concern (1). Thus, delineation of pyrazinamide resistance in isolates of MDR Mycobacterium tuberculosis has potential prognostic value. Standard culture-based pyrazinamide susceptibility tests are difficult to perform as a result of poor buffering of test media (79), the use of acidic medium pH that inhibits growth (21, 41, 67), and excessively large inocula that reduce the activity of pyrazinamide (15, 41, 77), thereby leading to false resistance. Pyrazinamide resistance is largely caused by pncA mutations (58). Alternative drug susceptibility testing (DST) methods based on the detection of pncA mutations or on pyrazinamidase activity may help determine susceptibility to pyrazinamide. Molecular tests indirectly applied to culture isolates can shorten the turnaround time of drug susceptibility testing to less than 1 day (46, 59, 68, 72). Initially developed to differentiate Mycobacterium bovis from M. tuberculosis and the Mycobacterium avium complex and also Mycobacterium marinum from Mycobacterium kansasii (28), pyrazinamidase assays are relatively easy to perform and can often be done in parallel with culture-based DST to independently measure enzyme activity. A literature search through MEDLINE and Embase with key phrases did not show any previous systematic review on the diagnostic performance of DST methods based on pncA mutations or pyrazinamidase activity. This review aims to clarify the role of these assays by evaluating their test performance in different clinical settings.

METHODS USED FOR META-ANALYSIS

PubMed and OvidSP were used to search in MEDLINE, life science journals, and Embase through 27 April 2011 for biomedical articles from publications in English containing the following key words in titles or abstracts, with the help of Boolean operators (“and” or “or”): (i) pyrazinamidase; (ii) pncA; (iii) susceptibility, susceptible, sensitiv*, or resistan*; and (iv) tuberculosis. The asterisk denotes a wild card. The search algorithm used in PubMed is shown in an appendix in the supplemental material.

Inclusion criteria.

A study was included only in the presence of concurrent sensitivity and specificity, with reference to standard phenotypic DST of M. tuberculosis to pyrazinamide. Standard DST methods included the agar or Lowenstein-Jensen proportion methods, a liquid medium method using Middlebrook 7H9 medium, the Bactec radiometric method, and the Bactec mycobacterial growth indicator tube 960 (MGIT). Data were extracted by the first author in duplicate.

Exclusion criteria.

Data on sensitivity and specificity were grouped for meta-analysis by assay category and by the nature of the samples (culture isolates versus clinical specimens). A study or a set of data within a study was excluded from meta-analysis in the presence of selection bias or when there were fewer than three sets of data in a group.

Test characteristics.

Sensitivity refers to the proportion of pyrazinamide-resistant strains detected by the evaluated assay among strains defined as pyrazinamide resistant by the reference phenotypic assay. Specificity refers to the proportion of pyrazinamide-susceptible strains detected by the evaluated assay among strains defined as pyrazinamide susceptible by the reference phenotypic assay.

Summary estimates of sensitivity, specificity, positive likelihood ratio (LR), and negative LR were estimated by meta-analysis using a bivariate random-effects model via the SAS Proc Mixed procedure (53).

Meta-regression.

Heterogeneity within each group of assays was explored by unweighted meta-regression analysis using the Moses-Shapiro-Littenberg method, which involves regression of the log diagnostic odds ratio against a measure of diagnostic threshold (25, 32, 48). Significant heterogeneity for a covariate was considered present with P values of ≤0.05. Covariates included the reference DST method, whether the study cohort or sampling method was well defined, whether the putative regulatory region of pncA was included, whether the Wayne method was considerably modified, and whether there could be considerable false resistance in the reference phenotypic assay. The last covariate was assumed to be present when >10% of pyrazinamide-resistant strains had pyrazinamidase activity or no pncA mutations and no mention of retesting these strains. Bivariate meta-analysis was repeated after excluding data sets with significant heterogeneity.

Funnel plot asymmetry.

Funnel plot asymmetry was examined by plotting the natural log diagnostic odds ratio against the reciprocal of the square root of the effective sample size for each test method (14). Significant asymmetry was denoted by P values of ≤0.05. Significant funnel plot asymmetry suggests either publication bias for studies with positive findings or bias due to exaggerated estimates from smaller studies or studies of lower quality.

Prevalence of pyrazinamide resistance and predictive values.

Each included study was also examined for the respective prevalence levels of pyrazinamide resistance in samples of MDR and non-MDR M. tuberculosis isolates. The critical prevalence levels required of each test method to attain predictive values of ≥85% under different clinical settings were then calculated using the following standard formulae: (i) pretest odds = prevalence/(1− prevalence); (ii) positive predictive values (PPV) = posttest odds/(1 + posttest odds), where posttest odds = pretest odds × positive LR; (iii) negative predictive values (NPV) = 1 − posttest odds/(1 + posttest odds), where posttest odds = pretest odds × negative LR; (iv) positive LR = sensitivity/(1 − specificity); and (v) negative LR = (1 − sensitivity)/specificity.

MetaDiSc version 1.4 (76), SAS Enterprise Guide 3.0, OpenOffice.org 3.0, and SPSS version 10 (Chicago, IL) were used for statistical analysis.

FINDINGS AND CONCLUSIONS

A literature search initially identified 128 articles. Figure 1 shows how articles were selected for meta-analyses. All included data involved culture isolates of M. tuberculosis. Only one identified article contained concurrent data on sensitivity and specificity involving direct examination in specimens (62).

Fig. 1.

Flow diagram of reviewed articles.

Tables S1 and S2 in the supplemental material summarize major findings of 48 publications (1–4, 6–8, 10, 12, 13, 17, 18, 22–24, 26, 27, 29–31, 33–37, 42, 45–47, 49, 51, 52, 56–66, 68–70, 71, 75) included in the initial meta-analysis, which comprised 40 sets of data on molecular assays targeting pncA and 33 sets of data on pyrazinamidase assays using the Wayne method. All molecular assays covered the entire open reading frame of pncA. Except for two studies (13, 58), all mentioned sequencing upstream of pncA. Data for pyrazinamidase assays were inadequate for evaluating methods other than the Wayne method.

Meta-regression showed that the possible presence of considerable false resistance was the only major source of heterogeneity for both PCR-DNA sequencing (P = 0.006) and the Wayne assay (P = 0.022).

Table 1 shows results of the initial meta-analysis. Results of standard phenotypic DST were based on repeat findings, when validation of pyrazinamide-resistant isolates without pncA mutations or with pyrazinamidase activity was made. Data pertaining to members of the M. tuberculosis complex other than M. tuberculosis (such as M. bovis, M. bovis BCG, Mycobacterium africanum, and Mycobacterium microti) were excluded whenever possible. Molecular assays by line probe, branch migration inhibition, denaturing gradient gel electrophoresis, microarray, and temperature-mediated heteroduplex analysis using denaturing high-performance liquid chromatography were considered in one group. Compared with the results of the Wayne assay, the summary estimate of sensitivity was nonsignificantly lower for PCR-DNA sequencing (89% versus 87%, P = 0.60), whereas the summary estimate of specificity was significantly lower (97% versus 93%, P = 0.04). The choice of reference culture-based DST methods for evaluating test performance did not significantly change the summary estimates of sensitivity and specificity for evaluated assays. For PCR-DNA sequencing targeted at pncA, summary estimates of sensitivity and specificity, respectively, were 87% and 95%, 85% and 88%, and 90% and 94% when the Bactec radiometric method, MGIT, and Lowenstein-Jensen or Middlebrook 7H10/11 proportion methods were used as predominant reference assays, respectively. For the Wayne assay, summary estimates of sensitivity and specificity, respectively, were 89% and 96%, 89% and 98%, 91% and 97%, and 86% and 97% when the Bactec radiometric method, MGIT, the Lowenstein Jensen proportion method, and the Middlebrook 7H10/11 proportion method were used as predominant reference assays, respectively. Funnel plot asymmetry was significant for both the Wayne assay (P = 0.006) and PCR-DNA sequencing targeted at pncA (P = 0.04).

Table 1.

Initial meta-analyses of pyrazinamide susceptibility testing methods in culture isolates of M. tuberculosis^a

Assay category	No. of data sets	No. of resistant strainsb	No. of susceptible strainsb	Mean value (95% CI)					Funnel plot asymmetry (P value)
				Sensitivity (%)	Specificity (%)	Positive LR	Negative LR	DOR
PCR-DNA sequencing	29	998	1,207	87 (82–91)	93 (89–96)	12.9 (7.6–22.0)	0.14 (0.09–0.21)	93 (48–181)	0.04
PCR-SSCPc	5	228	261	75 (53–88)	95 (83–98)	14.1 (3.9–51.0)	0.27 (0.11–0.66)	53 (11–254)	0.56
Molecular assaysd	6	198	416	96 (89–99)	97 (89–99)	28.9 (8.3–100.8)	0.04 (0.01–0.12)	756 (138–4,140)	0.18
Wayne assay	33	1,141	2,052	89 (84–92)	97 (95–98)	27.8 (16.0–48.3)	0.12 (0.08–0.17)	237 (121–465)	0.006

^aCI, confidence interval; DOR, diagnostic odds ratio; LR, likelihood ratio; SSCP, single-stranded conformation polymorphism.

^bBased on standard phenotypic drug susceptibility testing.

^cWith or without multiplex PCR amplimer conformation analysis.

^dComprises line probe assay, microarray, branch migration inhibition, denaturing gradient gel electrophoresis, and temperature-mediated heteroduplex analysis using denaturing high-performance liquid chromatography.

Table 2 shows results of the final meta-analysis, which was restricted to studies that either showed no discrepancy between reference and evaluated assays or retested pyrazinamide-resistant isolates in case of discrepancy. No meta-analysis was performed for PCR-SSCP, as there were only two sets of data after applying the restriction. Summary estimates (95% confidence interval) of sensitivity and specificity, respectively, were 92% (87% to 95%) and 93% (88% to 97%) for PCR-DNA sequencing, 98% (93% to 100%) and 96% (87% to 99%) for other molecular assays (line probe assay, microarray, branch migration inhibition, and denaturing gradient gel electrophoresis), and 91% (87% to 94%) and 97% (94% to 98%) for the Wayne assay. There was no significant difference in the summary estimates of sensitivity and specificity between PCR-DNA sequencing and the Wayne assay (P = 0.78 and P = 0.13, respectively). Funnel plot asymmetry was persistently significant (P = 0.03) for the Wayne assay but nonsignificant for PCR-DNA sequencing (P = 0.07) and other molecular assays (P = 0.27).

Table 2.

Final meta-analyses, excluding studies with possibly considerable false resistance in the reference assay^a

Assay category	No. of data sets	No. of resistant strainsb	No. of susceptible strainsb	Mean value (95% CI)					Funnel plot asymmetry (P value)
				Sensitivity (%)	Specificity (%)	Positive LR	Negative LR	DOR
PCR-DNA sequencing	20	715	877	92 (87–95)	93 (88–97)	13.8 (7.0–27.3)	0.09 (0.05–0.14)	158 (68–364)	0.07
Molecular assaysc	5	185	381	98 (93–100)	96 (87–99)	27.7 (6.6–116.4)	0.02 (0.01–0.07)	1,438 (202–10,230)	0.27
Wayne assay	27	912	1,802	91 (87–94)	97 (94–98)	27.3 (14.4–51.7)	0.09 (0.06–0.14)	296 (138–635)	0.03

^aCI, confidence interval; DOR, diagnostic odds ratio; LR, likelihood ratio.

^bBased on standard phenotypic drug susceptibility testing.

^cComprises line probe assay, microarray, branch migration inhibition, and denaturing gradient gel electrophoresis.

The median prevalence of pyrazinamide resistance in culture isolates of MDR M. tuberculosis, based on 14 studies included in the initial meta-analysis, was 51% (range, 31 to 89%) (1, 2, 4, 6, 8, 17, 24, 26, 42, 49, 59, 63, 69, 75). The median prevalence of pyrazinamide resistance in culture isolates of M. tuberculosis susceptible to both isoniazid and rifampin, based on eight studies, was 5% (range, 0% to 9%) (6, 17, 24, 26, 34, 49, 59, 63).

Using summary estimates of sensitivity and specificity in the final meta-analysis for calculation, Table 3 shows the critical prevalence levels of pyrazinamide resistance required of three categories of pyrazinamide DST assays to attain different predictive values. Considering the prevalence range of pyrazinamide resistance in different clinical settings, both molecular assays targeting pncA mutations and the Wayne assay can rule in pyrazinamide resistance in MDR M. tuberculosis isolates, with PPV largely exceeding 90%, and rule out pyrazinamide resistance in non-MDR M. tuberculosis isolates, with NPV exceeding 99%.

Table 3.

Prevalence levels of pyrazinamide resistance in culture isolates of M. tuberculosis for attaining different predictive values^a

Assay category	Critical prevalence level (%) required to attain:
	PPV of:				NPV of:
	≥85%	≥90%	≥95%	≥99%	≥85%	≥90%	≥95%	≥99%
PCR-DNA sequencing targeted at pncA mutations	≥29	≥39	≥58	≥88	≤67	≤56	≤38	≤10
Other molecular assays targeting pncA mutationsb	≥17	≥25	≥41	≥78	≤90	≤85	≤73	≤34
Wayne assay	≥17	≥25	≥41	≥78	≤66	≤55	≤36	≤10

^aBased on estimates of sensitivity and specificity obtained in the final meta-analysis. CI, confidence interval; DOR, diagnostic odds ratio; LR, likelihood ratio; NPV, negative predictive values; PPV, positive predictive value.

^bComprises line probe assay, microarray, branch migration inhibition, and denaturing gradient gel electrophoresis.

To our knowledge, this is the first systematic review and meta-analysis of molecular and pyrazinamidase assays for testing the susceptibility of M. tuberculosis in culture isolates to pyrazinamide. Data were insufficient for evaluating these assays when directly applied to clinical specimens and insufficient for evaluating pyrazinamidase assays based on the method described by McClatchy et al. (38) and on Russell's phenol-hypochlorite method (5, 54). Assuming that the prevalence of pyrazinamide resistance is 31% to 89% in MDR M. tuberculosis culture isolates and 0% to 9% in non-MDR isolates, this systematic review shows that molecular assays targeted at pncA and the putative regulatory region, especially PCR-DNA sequencing, may reliably detect pyrazinamide resistance in MDR M. tuberculosis strains and rule out pyrazinamide resistance in non-MDR stains. With significant funnel plot asymmetry in the final meta-analysis, the current review may have overestimated the actual test performance of the Wayne assay. Given the relatively few published data included in the current review, it remains to be confirmed whether other molecular assays, such as line probe assay, microarray, branch migration inhibition, and denaturing gradient gel electrophoresis, may attain higher predictive values than PCR-DNA sequencing.

Instead of estimating sensitivity and specificity separately by pooling in a univariate model without considering the threshold effect or by using the summary receiver operating characteristic curve that removes the effect of a possible threshold by comparing diagnostic odds ratios rather than estimates of sensitivity and specificity, we have used a bivariate random-effects model that takes into account the correlation between sensitivity and specificity within studies without masking important differences in these test performance estimates (25, 53, 72). The use of a random-effects model allows for the presence of heterogeneity between studies.

As there may be concern about the performance of DST of M. tuberculosis to pyrazinamide in Middlebrook 7H10/11 medium (47) and MGIT (9, 50), it is important to examine whether the choice of reference DST medium may affect the evaluation of test performance of molecular assays and pyrazinamidase assays. Among standard culture-based reference DST methods, the Bactec radiometric method is probably the most reliable (79) and is currently the reference method of choice for pyrazinamide susceptibility testing (11), although its reliability has also been questioned (16). MGIT has widely replaced the Bactec radiometric method, owing to the concern about disposal of radioactive substances. However, MGIT may overreport pyrazinamide resistance (9, 50), possibly because of several differences between the inoculum used in MGIT and that used in the Bactec radiometric method (9). Lowenstein-Jensen medium is considered acceptable, provided that a good technique is used, whereas Middlebrook 7H10 medium is the least reliable (79). The bivariate random-effects model used in the current review, which is capable of comparing sensitivities and specificities of different candidates, suggests no significant differences in test performance between included studies that used the Bactec radiometric method, MGIT, or the Middlebrook 7H10/11 and Lowenstein-Jensen proportion methods as reference DST assays.

In the initial meta-analysis, the sensitivity of PCR-DNA sequencing targeted at pncA may have been underestimated for two reasons. First, among 29 included studies on PCR-DNA sequencing targeted at pncA, only nine mentioned retesting pyrazinamide-resistant isolates that contained no pncA mutations (10, 18, 23, 24, 36, 47, 49, 57, 58). The lack of validation by retesting may leave false resistance unidentified, thereby reducing sensitivity. This might partly account for the relatively low sensitivity estimates in the range of 67% to 85% from nine studies included in the initial meta-analysis (3, 4, 7, 26, 30, 31, 66, 69, 75). The same problem might also apply to the Wayne assay; only 16 out of 33 included studies on the Wayne assay mentioned retesting pyrazinamide-resistant isolates that had pyrazinamidase activity in the presence of a discrepancy between reference and tested assays (2, 10, 12, 13, 24, 29, 36, 42, 45, 47, 49, 52, 57, 58, 61, 64). Second, the use of a relatively low resistance breakpoint in standard culture-based DST may also lead to misinterpretation of susceptible isolates as resistant (false resistance). Heifets has suggested that 300 mg/liter (19–21) might be more appropriate than 100 mg/liter (55) as the resistance breakpoint in the Bactec radiometric method, while Zhang et al. have suggested a compromised cutoff at 200 mg/liter after estimating the MIC using the Henderson-Hasselbach equation (77, 79). Again, the same problem might also apply to the Wayne assay.

The possible presence of false resistance in the reference assay was identified as the only major source of heterogeneity. Excluding data sets with possibly considerable false resistance resulted in an increase in the summary estimate of sensitivity from 87% (95% confidence interval, 82% to 91%) in the initial analysis to 92% (87% to 95%) for PCR-DNA sequencing and from 89% (84% to 92%) in the initial analysis to 91% (87% to 94%) for the Wayne assay. Furthermore, funnel plot asymmetry became nonsignificant for PCR-DNA sequencing in the final meta-analysis.

Although the current review suggests that the Wayne test might be sensitive and specific for detecting pyrazinamide resistance, molecular assays are probably the way forward. Notwithstanding the nonclustered distribution of pncA mutations (57) and a relatively high cost of molecular assays, recent advances in DNA sequencing technology will make molecular detection of pyrazinamide resistance more affordable and rapid in the near future. The requirement of a sufficient inoculum with a considerable number of bacilli for detecting pyrazinamidase activity makes the Wayne assay prone to misinterpretation of some susceptible isolates as resistant (false resistance) and incurs a delayed turnaround time.

A lower cost and a shorter turnaround time give pncA sequencing an edge over the Bactec radiometric method based on MICs. Although alternative mechanisms of pyrazinamide resistance, such as unknown target alteration, uptake of pyrazinamide, and efflux of pyrazinoic acid, may reduce the sensitivity of pncA sequencing (45, 65, 78, 79), most pyrazinamide-resistant strains, including those with low-level resistance (an increase in the MIC by 2- to 3-fold), have pncA mutations (10, 57).

Besides the possibility that the sensitivity of tested assays will be underestimated due to unidentified false resistance in the reference assay, other limitations in the current review include the following. First, the failure of the search algorithm in identifying all eligible studies, as well as the exclusion of publications in other languages from the current review, could have introduced publication bias. Second, heterogeneity exists between the included studies (data not shown). Although the random-effects model allows for the presence of heterogeneity, there may still be some controversy about combining study estimates in its presence. The estimate obtained from the random-effects model refers to a mean effect, around which true study effects vary, rather than a universally true study effect that varies between studies due to sampling error. Third, although restrictions applied in the final meta-analyses may minimize the underestimation of sensitivity by reducing false resistance, technical problems inherent with standard culture-based pyrazinamide susceptibility testing methods and the choice of breakpoint concentration may still lead to misinterpretation of some susceptible isolates as resistant, thereby underestimating the sensitivity of evaluated assays. Lastly, the current review has evaluated test performance with no reference to clinical treatment outcomes or assurance of unique isolates by genotyping. This could have introduced bias in validation.

In conclusion, in most epidemiological settings, PCR-DNA sequencing, and possibly other molecular assays targeting pncA mutations, can reliably detect pyrazinamide resistance in MDR M. tuberculosis isolates and rule out pyrazinamide resistance in non-MDR isolates. Molecular assays are probably the way forward for detecting pyrazinamide resistance.