High-Resolution Melting Curve Analysis for Rapid Detection of Rifampin Resistance in Mycobacterium tuberculosis: a Meta-Analysis

Abstract

A rapid, simple, accurate, and affordable method for the detection of drug-resistant tuberculosis is very critical for the selection of antimicrobial therapy and management of patient treatment. High-resolution melting curve analysis has been used for the detection of rifampin resistance in Mycobacterium tuberculosis and has shown promise. We did a systematic review and meta-analysis of published studies to evaluate the accuracy of high-resolution melting curve analysis for the detection of rifampin resistance in clinical M. tuberculosis isolates. We searched the PubMed, BIOSIS Previews, and Web of Science databases to identify studies and included them according to predetermined criteria. We used the DerSimonian-Laird random-effects model to calculate pooled measures and applied Moses' constant for linear models to fit the summary receiver operating characteristic curve. According to the selection criteria, most of the identified studies were excluded, and only seven studies were included in the final analysis. The overall sensitivity of the high-resolution melting curve analysis was 94% (95% confidence interval [CI], 92% to 96%), and the overall specificity was very high at 99% (95% CI, 98% to 100%). The values for the pooled positive likelihood ratio, negative likelihood ratio, and diagnostic odds ratio were 63.39 (95% CI, 30.21 to 133.00), 0.06 (95% CI, 0.04 to 0.09), and 892.70 (95% CI, 385.50 to 2,067.24), respectively. There was no significant heterogeneity across all included studies for the measurements we evaluated. The summary receiver operating characteristic curve for the same data shows an area of 0.99 and a Q* value of 0.97. High-resolution melting curve analysis has high sensitivity and specificity for the detection of rifampin resistance in clinical M. tuberculosis isolates. This method might be a good alternative to conventional drug susceptibility tests in clinical practice.

INTRODUCTION

Tuberculosis (TB) remains the leading infectious cause of morbidity and mortality, particularly in developing countries. In 2010, 8.8 million new cases of TB and 1.4 million TB-related deaths were reported worldwide. The emergence of multidrug-resistant TB (MDR-TB) and extensively drug-resistant TB (XDR-TB) and even totally drug-resistant TB (TDR-TB) has significantly threatened TB control efforts, although the global incidence of TB appears to be decreasing (1, 2). MDR-TB is defined as TB that is resistant to both isoniazid (INH) and rifampin (RIF), with or without resistance to other anti-TB drugs. XDR-TB is defined as MDR-TB with resistance to any fluoroquinolone and to at least one of three injectable second-line drugs (kanamycin, capreomycin, or amikacin) (3). TDR-TB is defined as TB that is resistant to five first-line drugs and seven second-line drugs (2). Therefore, rapid diagnosis is needed not only to reduce the spread of drug-resistant strains but also to monitor and limit the emergence of newly resistant strains because of the limited number of antibiotics available to treat TB (4, 5).

Conventional culture methods for the detection of drug-resistant TB include the absolute concentration method, the proportion method, and the resistance ratio method (6). These methods are still time-consuming, cumbersome, and laborious, although some commercial drug susceptibility tests (DSTs) with liquid culture decrease turnaround time by using expensive automatic equipment. Other phenotypic methods, such as the colorimetric redox-indicator method, nitrate reductase assay, microscopic-observation susceptibility assay, and thin-layer agar assay, do not require additional equipment but do require approximately 1 week to produce definitive results (7–10). Bacteriophage-based assay, a special phenotypic method, is rapid (the turnaround time is 2 to 3 days) and sensitive, but it appears to have variable and slightly lower specificity (6, 10). Some genotypic methods for rapidly detecting drug resistance-conferring mutations have been developed, and these include DNA sequencing, line-probe assay, GenoType MTBDR assay, real-time PCR, DNA microarrays, denaturing high-performance liquid chromatography, and single-strand conformation polymorphism (SSCP) analysis (6, 10–13). Among these molecular assays, real-time PCR is relatively rapid and simple and easily accessible. Moreover, the main advantage of real-time PCR is the closed-tube system, which safeguards against cross-contamination from amplified DNA because downstream processing of PCR products is not required (10).

In recent years, real-time PCR coupled with melting curve analysis, especially with high-resolution melting curve analysis (HRMA), has shown promise as a tool for the detection of DNA sequence variation because this novel method detects more mutations with few probes or without probes. HRMA is a rapid, simple, cost-effective, and closed-tube method, and it has been applied to a variety of diseases, such as inherited, infectious, and oncological diseases (14). This PCR-based method was successfully used for rapid identification and susceptibility testing of Mycobacterium tuberculosis in previous studies (2, 4, 15, 16). On the other hand, previous studies have demonstrated that RIF resistance is a good surrogate marker for MDR-TB, especially when molecular methods are used for rapid detection of RIF-resistant M. tuberculosis isolates, as 90% of RIF-resistant M. tuberculosis isolates, or an even greater proportion, are also resistant to INH (17–20). Thus, HRMA might be a good alternative method for rapid detection of RIF-resistant TB or even MDR-TB, and this has been demonstrated in some studies (2, 4, 21–25). Nonetheless, the accuracy of HRMA has not been systematically evaluated. Thus, we conducted a systematic review of published reports and a meta-analysis of included studies to evaluate the overall accuracy of HRMA for rapid detection of RIF resistance in M. tuberculosis.

MATERIALS AND METHODS

Search strategy and selection criteria.

We searched the PubMed, BIOSIS Previews, and Web of Science databases for articles published between January 1999 and December 2012. The search terms used were tuberculosis, Mycobacterium tuberculosis, TB, MTB, RNA polymerase B, rpoB, rifampin, rifampicin, multidrug-resistant tuberculosis, drug resistance, high-resolution melting curve, HRM, high-resolution melting curve analysis, and HRMA. There was no language restriction. We also searched the reference lists of some primary studies and several previously published reviews on DSTs of M. tuberculosis. Conference abstracts, case reports, editorials, letters, and reviews were excluded because of the limited data presented in these articles.

Studies were included in the meta-analysis if they (i) presented original data and provided sufficient information to judge the methodological quality of the study; (ii) included at least one accepted reference standard for the detection of drug-resistant M. tuberculosis; (iii) provided enough information to describe true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN); (iv) were performed on clinical M. tuberculosis isolates or sputum samples; and (v) used saturated dye, not unsaturated dye. Reasons for study exclusion were (i) use of labeled or unlabeled probes for the indexed test; (ii) use of HRMA for M. tuberculosis detection; (iii) application of HRMA for typing of strains; and (iv) application of HRMA for drug resistance detection of only nontuberculous mycobacteria (NTM).

Two reviewers (X.Y. and Q.L.) independently screened the studies for inclusion and abstracted relevant data. Any citations identified by either reviewer were further evaluated by review of full-text reports. Disagreements between two reviewers were reconciled by consensus. The initial “unrevised” results before discrepant analysis were used for reporting sensitivity and specificity. When multiple studies were published by the same authors, only the study performed on the largest number of samples was included.

Data extraction and quality assessment.

One reviewer (X.Y.) reviewed the final set of included articles and abstracted data regarding test characteristics and study quality using a piloted data extraction form. A second reviewer (Q.L.) independently assessed a subset of the included articles to determine concordance in the assessment of data quality and the accuracy of extracted data. Disagreements were discussed and resolved. Data retrieved from the reports included publication year of the study, reference method, methodological quality, total number of samples, sensitivity, and specificity.

We assessed the methodological quality of the included studies by using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS) criteria (26). The study quality criteria included study design (case-control or cross-sectional), specimen collection method (convenience or consecutively or randomly), interpretation of determination and reference standard results (unblended, single blind, or double blind), and verification of the test results with the reference standard (partial, differential, or complete).

Data synthesis and meta-analysis.

The numbers of TP, FP, TN, and FN were taken directly from the source reports or calculated from the data provided in the article. We used standard methods recommended for meta-analyses of diagnostic test evaluations and used Meta-DiSc software (version 1.4) to analyze data (27). The data were pooled using the Mantel-Haenszel fixed-effects model, which is more suitable for data with a low degree of variability. The measures of test accuracy we calculated for each study were sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), and diagnostic odds ratio (DOR), along with their 95% confidence intervals (CIs). We established summary receiver operating characteristic (SROC) curves using Moses' constant for linear models. The area under the SROC curve (AUC) was used as a measure of the overall test accuracy, and the Q* value was used to represent the point where sensitivity and specificity are equal. These accuracy measures of indexed tests were displayed graphically on forest plots or SROC curves. A forest plot is a graph which displays the results of all included studies in a systematic review (28). Forest plots are used to summarize and facilitate the visual interpretation of results of multiple pooled studies in a meta-analysis (29). The SROC curve is intended to represent the relationship between sensitivity and specificity across all included studies, and each data point in the SROC space represents an individual study. The SROC curve has been recommended for evaluation of the performance of a diagnostic test in a meta-analysis (30, 31).

Heterogeneity was explored to find the factors that influence accuracy estimates and to evaluate the appropriateness of statistical pooling of accuracy estimates from included studies. Heterogeneity might be due to the variability of variations in index tests and reference standard methods and differences in study quality, threshold, prevalence of RIF-resistant TB, and types of dyes and instruments used for HRMA. We used the chi-squared and I² tests included in the Meta-DiSc program to evaluate statistical heterogeneity. Further reasons were investigated by subgroup analyses if there was statistically significant heterogeneity across the included studies.

RESULTS

Study characteristics.

As shown in Fig. 1, a total of 63 potentially relevant citations were identified by searching multiple databases and sources. Fifteen articles were selected for full-text review after duplicate articles were excluded and titles and abstracts were screened. Then one patent report and one review were excluded and six articles were also excluded because unsaturated dye or common melting curve analysis was used in the study. Finally, a total of seven studies published between 1998 and 2012 were included in the final analysis.

Fig 1.

Flow chart for study selection.

The main characteristics of the 7 included studies, which were conducted in 7 different countries, are summarized in Table 1. A total of 1,027 strains were analyzed in the meta-analysis and the average sample number was 147 (range, 49 to 287). All studies evaluated the performance of HRMA on clinical isolates. Most studies (5 out of 7, 71.4%) used the agar proportion method as the reference method, except for one study that used the absolute concentration method and one study that used the manual MGIT 960 commercial broth system. Genomic DNAs were extracted from M. tuberculosis isolates and used for real-time PCR in all studies. Five of seven studies applied a Rotor-Gene 6000 instrument (Corbett Research, Sydney, Australia) to perform real-time PCR and HRMA. Of these five studies, two studies used SYTO9 dye (Molecular Probes, Eugene, OR) and the other three studies used EvaGreen dye (Quantace, London, United Kingdom), SYBR GreenER dye (Invitrogen, Carlsbad, CA), or LCGreen Plus dye (Idaho Technology, Inc., Salt Lake City, UT). Two of seven studies included in the meta-analysis applied both a LightCycler LC480 instrument (Roche Diagnostics, Penzberg, Germany) and ResoLight dye (Roche Diagnostics, Penzberg, Germany) to perform real-time PCR and HRMA. Of all seven included studies, two studies adopted two real-time PCRs for DNA amplification and HRMA (22, 23), and one study used an additional primer set to distinguish M. tuberculosis from NTM (4).

Table 1.

Description of studies included in the meta-analysis

Reference no.	Year	Country	Total no. of samples	Reference method	Instrument	Dye	Study design	Sample selection	Blinding	Verification
21	2008	South Africa	287	Agar proportion	Rotor-Gene 6000	SYTO9	Cross-sectional	Unclear	Yes	Complete
22	2009	Austria	68	MGIT 960	LightCycler 480	ResoLight	Case-control	Unclear	Unclear	Complete
25	2010	Singapore	59	Agar proportion	Rotor-Gene 6000	SYTO9	Cross-sectional	Unclear	Yes	Complete
23	2010	South Korea	197	Agar proportion	Rotor-Gene 6000	EvaGreen	Case-control	Convenience	Unclear	Complete
4	2010	Georgia	252	Agar proportion	Rotor-Gene 6000	SYBR GreenER	Case-control	Unclear	Unclear	Complete
24	2011	Australia	115	Absolute concentration	Rotor-Gene 6000	LCGreen Plus	Case-control	Unclear	Unclear	Complete
2	2012	India	49	Agar proportion	LightCycler 480	ResoLight	Case-control	Unclear	Unclear	Complete

The quality of individual studies was relatively high, as all studies met 8 or more of the QUADAS criteria. Five studies were case-control in design and the other two studies were cross-sectional. Only one study used convenience samples, while sampling methods of other studies were unknown. Two out of seven studies reported blinded interpretation of the indexed test independent of the reference standard. For all studies, complete verification of RIF resistance results was reported, with conventional DSTs as the reference standards.

Meta-analysis.

Figure 2 shows the forest plot of sensitivity and specificity based on results of the 7 included studies. Of the 7 included studies, 6 studies reported sensitivity of >90% and 5 studies reported specificity of 100%. The overall sensitivity was 94% (95% CI, 92% to 96%), and the overall specificity was very high at 99% (95% CI, 98% to 100%). Pooled PLR, NLR, and DOR values were 63.39 (95% CI, 30.21 to 133.00), 0.06 (95% CI, 0.04 to 0.09), and 892.70 (95% CI, 385.50 to 2,067.24), respectively (Fig. 3 and Fig. 4A). The chi-squared and I² tests for heterogeneity in the summary results suggested that there was no significant heterogeneity across all included studies for all the statistical measures mentioned above. The SROC curve (Fig. 4B) for the same data shows an AUC of 0.99 and a Q* value of 0.97, indicating a high level of overall accuracy.

Fig 2.

Forest plots estimates of sensitivity (A) and specificity (B) for high-resolution melting curve analysis. Each solid circle represents an individual study. Error bars represent 95% confidence intervals (CI).

Fig 4.

Forest plots estimates of diagnostic odds ratio (DOR) (A) and summary receiver operating characteristic (SROC) curves (B) for high-resolution melting curve analysis. (A) The DOR of each study was more than 100.0, so solid circles representing individual studies are not seen in the forest plots. Error bars represent 95% confidence intervals (CI). (B) Each solid circle represents an individual study. AUC, area under the curve; SE (AUC), standard error of AUC; Q*, an index defined by the point on the SROC curve where the sensitivity and specificity are equal; SE (Q*), standard error of the Q* index.

Fig 3.

Forest plots estimates of positive likelihood ratio (PLR) (A) and negative likelihood ratio (NLR) (B) for high-resolution melting curve analysis. Each solid circle represents an individual study. Error bars represent 95% confidence intervals (CI).

DISCUSSION

Our meta-analysis of 7 studies on HRMA for the detection of RIF resistance shows that HRMA has high sensitivity and specificity when applied to M. tuberculosis culture isolates. The majority of studies indicated sensitivities of >90% and specificities of ≥98% (2, 4, 21–24). The overall sensitivity and specificity were 94% and 99%, respectively. The sensitivity of HRMA for RIF resistance detection is slightly higher than that of PCR-SSCP analysis and slightly lower than the sensitivities of phenotypic methods and other genotypic methods which were systematically evaluated in previous reviews (6–9, 12, 32, 33). This may be attributable mainly to the limitations of molecular methods for the detection of RIF resistance. RIF resistance in M. tuberculosis is due to rpoB gene mutations. Since more than 95%, but not 100%, of RIF resistance-associated mutations are located within the RIF resistance-determining region (RRDR), approximately 5% of cases were not detected by these molecular methods, which detect only the RRDR and not the whole sequence of the rpoB gene (34). In addition, some mutations were not detected because C/G and A/T transversions have very little influence on overall thermal denaturation profiles (21, 22). The specificity of HRMA for RIF resistance detection is higher than the specificities of PCR-SSCP analysis and bacteriophage-based assays and as high the specificities of phenotypic methods and other genotypic methods (6–9, 12, 32, 33). This meta-analysis also showed that HRMA had a very high PLR and a very low NLR for the detection of RIF resistance, indicating an excellent ability to both confirm and exclude the presence of RIF resistance. There was no statistically significant heterogeneity for the above-mentioned estimates. This suggested that there was a low level of variability across all included studies and that HRMA for the detection of RIF resistance was not strongly influenced by different reference methods, instruments, or dyes. Thus, we did not perform subgroup analysis by reference method, instrument, and dye. Moreover, the high mean DOR and large AUC values calculated in our meta-analysis also indicate a high level of overall accuracy for the detection of RIF resistance. Nevertheless, the confidence intervals for the PLR and the DOR are wide for all included studies. This may be attributable to the small sample size and high sample variation of each included study.

When culture isolates are tested, the turnaround time of HRMA for the detection of RIF resistance is 3 to 4 h, whereas it is several days or even several weeks for phenotypic methods (21). Therefore, the main advantage of HRMA over phenotypic methods is the rapidity of this method. The Xpert MTB/RIF test, a genotypic assay which is currently recommended by the WHO, has a rapid turnaround time of less than 2 h (33). However, the Xpert MTB/RIF test can detect only RIF resistance, whereas HRMA can also detect INH, fluoroquinolone, and streptomycin resistance in M. tuberculosis (35). Moreover, HRMA for the detection of RIF resistance uses no labeled or unlabeled probes and requires no downstream processing of PCR products, while cross-contamination from amplified DNA can be avoided by the use of a closed-tube system. Thus, HRMA is technologically simpler and more cost-effective than other genotypic methods. Nonetheless, there are also some disadvantages associated with the application of HRMA. First, HRMA was performed only on culture isolates in all included studies. If this method could be efficiently and routinely used for the detection of RIF resistance in M. tuberculosis from clinical specimens, the diagnostic interval would be further shortened. HRMA may be adapted to clinical specimens by improving the yield and purity of extracted DNA, enhancing PCR amplification, and reducing the impact of template concentration. Second, the accuracy of HRMA critically depends on the quality of the template DNA, the instrument, and the dye. This may limit the extensive application of HRMA to some degree. Third, uniform standard operation procedures must be established before HRMA is used in clinical laboratories worldwide. Finally, studies to determine how to detect mutations outside the RRDR at the same time as mutations within the RRDR are also needed.

Our meta-analysis had several strengths. First, to identify studies, we performed a very comprehensive search of electronic databases and other sources according to a written protocol. Moreover, study selection and data extraction were conducted independently by two reviewers and disagreements were resolved by consensus. Lastly, we used established criteria to assess the quality of included studies and used rigorous statistical methods for assessment of diagnostic accuracy. However, our systematic review also had several limitations. First, blinding strategies and sampling methods were not stated in most of the included studies, although the overall quality of the included studies was good. For example, some inappropriate sampling methods can create selection bias which may result in high levels of sample variation and wide confidence intervals. Second, there were few studies available for some assessments, such as meta-regression. The small number of studies also meant that this meta-analysis might provide very powerful evidence for these assessments. Third, cost-effectiveness, reliability, and patient outcomes were not systematically analyzed and evaluated because none of the studies in this meta-analysis reported enough information on these issues. Fourth, we did not apply funnel plots and regression asymmetry tests to evaluate publication bias, as statistical and graphical approaches for publication bias are not recommended for diagnostic meta-analyses (9, 36). However, we reduced the publication bias by not restricting the search to any particular language.

Despite these limitations, this study has shown that HRMA for the detection of RIF resistance has high levels of sensitivity and specificity when this method is performed on culture isolates. HRMA may be a good alternative to conventional DSTs in clinical practice. Nevertheless, there are several concerns, such as standardization of the assay procedure and evaluation of the performance on clinical specimens, that must be addressed.