The Use of an Automated Quantitative Polymerase Chain Reaction (Xpert MTB/RIF) to Predict the Sputum Smear Status of Tuberculosis Patients

Abstract

(See the Editorial Commentary by Fennelly, on pages 389–91.)

Xpert MTB/RIF–generated cycle-threshold (C_T) values have poor clinical utility as a rule-in test for smear positivity (cut-point ≤20.2; sensitivity 32.3%, specificity 97.1%) but moderately good rule-out value (cut-point >31.8; negative predictive value 80.0%). Thus, 20% of individuals with C_T values >31.8 were erroneously ruled out as smear-negative. This group had a significantly lower sputum bacillary load relative to correctly classified smear-positive patients (C_T ≤ 31.8; P < .001). These data inform on public health and contact tracing strategies.

Individuals who test smear-positive for acid-fast bacilli are more likely to transmit tuberculosis [1, 2]. Xpert MTB/RIF (MTB/RIF; Cepheid) is a new molecular test endorsed by the World Health Organization (WHO) for the frontline diagnosis of tuberculosis [3, 4, 5]. Thus, MTB/RIF will be used to diagnose patients who have not undergone smear microscopy testing. Existing infection control and contact tracing guidelines rely on the smear status of patients to inform patient management [6, 7]. The diagnostic accuracy of MTB/RIF for predicting the smear status of individuals remains undescribed. Thus, it is unclear how patients who undergo only MTB/RIF testing should be managed. Furthermore, few tools are available to identify tuberculosis patients who are at risk of delayed sputum conversion and relapse due to a high baseline bacillary load [8, 9]. Further study of measures of bacterial burden, including smear status and grade, liquid culture time to positivity (TTP), and quantitative polymerase chain reaction cycle-threshold (C_T) values, are thus warranted. C_T values are continuous variables inversely correlated with the concentration of starting material (in this case, the number of copies of the MTB complex rpoB gene) in the sputum.

METHODS

We recently evaluated MTB/RIF performance in a high human immunodeficiency virus (HIV) prevalence setting in Cape Town, South Africa [10, 11]. Here, we present the performance of C_T values in 496 pretreatment patients with suspected tuberculosis, alone or in combination with other demographic and clinical factors, for the detection of smear-positive individuals and, of those who are smear-positive, those with the highest smear grade. This was evaluated using receiver operating characteristic (ROC) curve analysis, and cut-points were selected for their rule-in and rule-out values. In an attempt to improve predictive outcome, we performed a multivariable regression analysis using clinical and chest radiographic data (scored by at least 2 readers using the standardized chest radiograph scoring system [12]). Furthermore, we performed an analysis of the relationship between C_T values and the TTP of liquid cultures grown using the MGIT 960 system. Demographic details of the cohort and definitions of diagnostic categories are published elsewhere [10].

RESULTS

Of the 496 patients with suspected tuberculosis, 141 had culture-confirmed definite tuberculosis. One hundred and eleven of this definite tuberculosis group were MTB/RIF-positive, whereas an additional 19 patients were culture-negative and MTB/RIF-positive. We showed this latter group to likely represent true tuberculosis cases [10]. Of the 130 MTB/RIF-positive cases, 34 were smear-negative and 96 were smear-positive for acid-fast bacilli (11 scanty, 24 one-plus positive [P+], 29 two-plus positive [P++], and 32 three-plus positive [P+++] cases classified according to the WHO/International Union Against Tuberculosis and Lung Disease method) [13]. Concentrated fluorescent smear microscopy was performed on sputa digested with N-acetyl-L-cysteine-sodium hydroxide. Samples from smear-negative individuals had a significantly increased average C_T value relative to those from smear-positive individuals (median 31.3 [interquartile range {IQR}, 27.3–34.25] vs 21.7 [13.0–25.3]; P < .001). The average C_T values, shown as median (IQR) for each smear grade (in order of increasing grade), were 26.3 (21.4–28.1), 24.3 (21.8–27.4), 22.6 (20.2–24.1), and 18.7 (17.0–21.2), respectively (P < .001, 1-way analysis of variance). Thus, average C_T values decrease with increasing smear positivity.

The performance (sensitivity, specificity, positive predictive value [PPV], and negative predictive value [NPV]) of C_T values at specific cut-points, corresponding to use as a rule-in or rule-out test, or at Youden's index [14], for the detection of smear-positive (cut-points of ≤20.2, >31.8, and ≤27.1, respectively) or P+++-graded individuals (cut-points of ≤17.7, >29.7, and ≤21.8, respectively) is shown in Table 1. The relevant ROC curves are shown in Supplementary figures 1 and 2.

Table 1.

Performance of Xpert MTB/RIF-Generated Average C_T Values and a Clinical Prediction Rule for the Detection of Smear-Positive Individuals and Smear-Positive, P+++ Individuals at Different C_T Value or Score Cut-points

	Performance of Average CT Values
	For Smear Positivity Detection					For the Detection of Smear-Positive, P+++-Graded Individuals
Test use	Suggested Average CT Value Cut-point	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)	Suggested Average CT Value Cut-point	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)
Rule-in	≤20.2	32.3% (23.8–42.2), 31/96	97.1% (85.1–99.5), 33/34	96.9% (84.3–99.5), 31/32	33.7% (25.1–43.5), 33/98	≤17.7	40.6% (25.5–57.7), 13/32	95.9% (90.0–98.4), 94/98	76.5% (52.7–90.5), 13/17	83.2% (75.23–89.0), 94/113
Rule-out	>31.8	95.8% (89.8–98.4), 92/96	47.1% (31.5–63.3), 16/34	83.6% (75.6–89.4), 92/109	80.0% (58.4–91.9), 16/20	>29.7	96.9% (84.3–99.5), 31/32	25.0% (17.1–35.0), 22/88	32.0% (23.5–41.8), 31/97	95.7% (79.0–99.2), 22/23
Youden indexa	≤27.1	82.3% (73.5–88.6), 79/86	79.4% (63.2–89.7), 27/34	91.9% (84.1–96.0), 79/86	61.4% (46.6–74.3), 27/44	≤21.8	87.5% (71.9–95.0), 28/32	75.5% (66.1–83.0), 74/98	53.9% (40.5–66.7), 28/52	94.9% (87.5–98.0), 74/78
	Performance of the clinical prediction scoreb Score = 0.3 × (average CT value) – 3 × (1 if self-reported weight loss occurred; 0 if no self-reported weight loss occurred)
	For Smear Positivity Detection					For the Detection of Smear-Positive, P+++ Graded Individuals
Test use	Suggested Score Cut-point	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)	Suggested Score Cut-point	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)
Rule-in	≤3.9	49.5% (39.4–59.5), 45/91 P = .02c	97.0% (84.7–99.5), 32/33	97.9% (88.7–99.6), 45/46	41.0% (30.8–52.1), 32/78	≤2.5	34.5% (19.9–52.7), 10/19	95.8% (89.7–98.4), 91/95	71.4% (45.4–88.3), 10/14	82.7% (74.6–88.7), 91/110
Rule-out	>6.9	95.6% (89.2–98.3), 87/91	60.6% (43.7–75.3), 20/33	87.0% (79.0–92.2), 87/100	83.3% (64.2–93.3), 20/24	>6.6	96.6% (82.8–99.4), 28/29	31.6% (23.1–41.45), 20/95	30.1% (21.7–40.1), 28/93	96.8% (83.8–99.4), 30/31
Youden indexa	≤5.6	83.5% (74.6–89.8), 76/91	84.9% (69.1–93.4), 28/33	93.8% (86.4–97.3), 76/81	65.1% (50.2–77.6), 28/43	≤4.2	86.2% (69.4–94.5), 25/29	67.1% (56.5–76.1), 57/85	47.2% (34.3–60.3), 25/53	93.3% (84.3–97.4), 57/61

Abbreviations: C_T, cycle threshold; CI, confidence interval; PPV, positive predictive value; NPV, negative predictive value.

^aDefined as the best compromise between sensitivity and specificity assuming equal weighting (13).

^bSix patients were missing weight loss data. These patients were excluded from the C_T alone versus clinical prediction score receiver operating characteristic curve comparisons.

^cP values ≤ .05 are shown for the clinical prediction score versus average C_T values alone. Nonsignificant P values are not shown.

At a cut-point of ≤20.2, C_T values showed good rule-in value for smear positivity (specificity 97.1% [95% confidence interval {CI}, 85.1%–99.5%]) and PPV 96.9% (95% CI, 84.3%–99.5%) but suboptimal clinical utility (sensitivity 32.3% [95% CI, 23.8–42.2]). A similar pattern was seen in smear-positive individuals graded P+++ (Table 1).

At a cut-point of >31.8, C_T values possessed good rule-out value for smear positivity (sensitivity of 95.8% [95% CI, 89.8–98.4] and NPV of 80.0% [95% CI, 58.4–91.9]) and moderate clinical utility (specificity of 47.1% [95% CI, 31.5–63.3]). Thus, 80% of smear-negatives were correctly classified. The remaining 20% were misclassified as smear-positive. Nevertheless, this latter group (the misclassified smear-positives) had significantly higher C_T values relative to individuals who were correctly classified as smear-positive (C_T values <31.8) (median, 33.6 [IQR, 32.2–35.4] vs 21.5 [IQR, 18.4–24.6]; P < .0001). Thus, individuals misclassified as smear-positive likely possess a lower bacillary load than correctly classified smear-positive individuals. Similarly, smear-negative individuals below a cut-point of 31.8 (and hence misclassified as smear-positive) had a higher bacterial load than the smear-negative individuals who were correctly classified (ie, above the cut-point of 31.8) (median average CT value of 27.8 [IQR, 23.0–29.8] vs 35.0 [IQR, 32.9–36.2]; P < .0001).

In an attempt to improve predictive capability of the C_T values, we derived a clinical prediction score (CPS) that incorporated demographic and clinical variables and the volume of sputum used for MTB/RIF testing (Table 2; detailed CPS methodology in Supplementary data). The presence of self-reported weight loss (P = .05) and average C_T values (P < .001) remained significant in the multivariate analysis (Table 2) and were weighted according to their β-coefficients in the final model for CPS derivation. In contrast to C_T values, HIV status was not a significant predictor of smear status when included in the multivariate analysis; this is likely because C_T values are a better proxy marker of bacterial load and have finer discriminative ability.

Table 2.

Univariate and Multivariate Analyses of Known Smear-Positive Associates and Derivation of a Clinical Prediction Score

	Univariate Analysis
	Odds Ratio (95% CI)		P Value
Demographic factors
Age	0.97 (.94–1.00)		.17
Male sex	2.67 (1.18–6.03)		.02
HIV-infected	0.39 (.16–.91)		.03
Previous Tuberculosis	0.51 (1.19–1.34)		.17
Smoker (past or current)	1.21 (.51–2.92)		.67
Symptoms
Hemoptysis	0.53 (.19–1.51)		.23
Weight loss	10.88 (3.16–37.4)		<.01
Appetite loss	2.00 (.89–4.48)		.09
Chest radiography
Compatible with active Tuberculosis	0.22 (.05–.01)		.05
Presence of cavities	13.42 (2.92–61.54)		<.01
Xpert MTB/RIF–specific factors
Average CT value	0.75 (.68–.83)		<.01
Sample volume used (mL)	1.00 (.99–1.00)		.30
	Multivariate Analysis
	Odds Ratio (95% CI)	P Value	β-Coefficient (95% CI)	Scoreb
Male sex	2.16 (.43–10.74)	.35	.77 (−.83 to 2.37)	N/Aa
HIV-infected	0.85 (.24–2.92)	.79	−.17 (−1.41 to 1.07)	N/Aa
Weight loss	9.51 (1.05–90.07)	.05	2.25 (.00–4.5)	−3
Presence of cavities	4.07 (.70–23.53)	.12	1.4 (−0.35 to 3.16)	N/Aa
Average CT value	0.72 (.60–.85)	<.001	−.33 (−.51 to −.16)	.3
Clinical prediction score formula	0.3 × (average CT value) – 3 × (1 if self-reported weight loss occurred; 0 if no self-reported weight loss occurred)

Abbreviations: CI, confidence interval; C_T, cycle-threshold; HIV, human immunodeficiency virus.

^aNot included in final model as not significant (P ≥ .05).

^bBased on the β-coefficient in the final model.

Use of the CPS resulted in a significant improvement in smear-positivity detection at a cut-point corresponding to use as a rule-in test (cut-point of ≤20.2; sensitivity of 49.5% vs 32.3% [P = .02]; Table 1). However, at the rule-out cut-point (≤20.2), neither specificity (P = .28) or NPV significantly improved (P = .79). The area under the ROC curves for smear positivity or P+++ detection also did not improve significantly (P = .16 and P = .33, respectively; Supplementary Figures 1 and 2).

Finally, average C_T values strongly correlated with liquid culture TTP (r_s = 0.56; P < .0001; linear regression formula: average C_T = 0.65 × [TTP in days] + 17.95). Thus, it is possible to equate MTB/RIF-generated C_T values with liquid culture TTP.

DISCUSSION

MTB/RIF will now be performed instead of smear microscopy, given its endorsement by the WHO as a frontline technology for the diagnosis of tuberculosis [3, 4]. The key findings of our study were as follows: (1) MTB/RIF-generated average C_T values have poor clinical utility as a rule-in test for smear positivity (sensitivity of 32.3% and specificity of 97.1% at a cut-point of ≤20.2), indicating that a positive result (ie, samples with C_T values below the cut-point) could be interpreted with high confidence but when negative did not exclude the patient from being smear-positive (almost 70% of smear-positive individuals were erroneously classified as smear-negative); (2) MTB/RIF had a moderately high rule-out value for smear positivity (sensitivity of 95% and an NPV of 80.0% at a cut-point of >31.8), the clinical utility of which is detailed below; and (3) average C_T values correlated well with TTP, a validated surrogate of mycobacterial burden and response to treatment [9].

The moderately high NPV of MTB/RIF when used as a rule-out test for smear positivity is likely to be clinically useful, because above this cut-point of 31.8, 80% of smear-negative cases are correctly identified. Although 20% of individuals above this cut-point are smear-positive cases misclassified as smear-negative, these misclassified smear-positives are, of all the smear-positives, those with the lowest mycobacterial burden and may thus be less likely to transmit disease. From a contact tracing viewpoint, 95% of all smear-positive cases fall below this cut-point, but so do approximately 50% of all smear-negative cases. However, of all smear-negatives, this latter group has the highest burden of disease and hence likely the highest risk of disease transmission.

Thus, from a public health perspective, smear-positive cases with the least risk of transmission will be misclassified. However, the precise relationship between MTB/RIF C_T values and disease transmission remains to be determined and will require prospective molecular epidemiological transmission-based studies. The poor rule-in value of MTB/RIF for both smear positivity and P+++-graded individuals relates to the overlapping average C_T values for each smear grade [10]. Thus, given this overlap, attainment of high specificity occurs at the expense of considerable sensitivity.

TTP is a proxy marker of bacterial viability, whereas MTB/RIF and smear microscopy quantify both viable and nonviable organisms; however, we have shown a good correlation between TTP and average C_T values and provided a method for equating these variables. Our data nonetheless have implications for clinical trials, which use longitudinal measurements of mycobacterial burden as surrogate markers of drug efficacy [8].

There are several limitations of our study. The conclusiveness of our findings is limited by our small sample size and wide CIs, and thus can only be regarded as preliminary. The use of different spot sputum specimens from the same patient for smear microscopy and MTB/RIF might be a source of inaccuracy.

In summary our preliminary findings, which require confirmation in larger studies, indicate that MTB/RIF-generated average C_T values of >31.8 provide moderately good rule-out value for smear positivity. Whether individuals falling above this cut-point, compared with those below, will display reduced disease transmission requires prospective validation. Our data have public health implications for the roll-out of MTB/RIF.

Supplementary Data

Supplementary materials are available at Clinical Infectious Diseases online (http://www.oxfordjournals.org/our_journals/cid/). Supplementary materials consist of data provided by the author that are published to benefit the reader. The posted materials are not copyedited. The contents of all supplementary data are the sole responsibility of the authors. Questions or messages regarding errors should be addressed to the author.