To Test or Not? Xpert MTB/RIF as an Alternative to Smear Microscopy to Guide Line Probe Assay Testing for Drug-Resistant Tuberculosis

ABSTRACT

Xpert MTB/RIF (Xpert) revolutionized tuberculosis (TB) diagnosis. Laboratory decision making on whether widely-used reflex drug susceptibility assays (MTBDRplus, first-line resistance; MTBDRsl, second-line) are conducted is based on smear status, with smear-negative specimens often excluded. We performed receiver operator characteristic (ROC) curve analyses using bacterial load information (smear microscopy grade, Xpert-generated semi-quantitation categories and minimum cycle threshold [C_Tmin] values) from Xpert rifampicin-resistant sputum for the prediction of downstream line probe assay results as “likely non-actionable” (no resistance or susceptible results generated). We evaluated actionable-to-non-actionable result ratios and pay-offs with missed resistance versus LPAs done universally. Smear-negatives were more likely than smear-positive specimens to generate a non-actionable MTBDRplus (23% [133/559] versus 4% [15/381]) or MTBDRsl (39% [220/559] versus 12% [47/381]) result. However, excluding smear-negatives would result in missed rapid diagnoses (e.g., only 49% [264/537] of LPA-diagnosable isoniazid resistance would be detected if smear-negatives were omitted). Testing smear-negatives with a semi-quantitation category ≥ “medium” had a high ratio of actionable-to-non-actionable results (12.8 or a 4-fold improvement versus testing all using MTBDRplus, 4.5 or 3-fold improvement for MTBDRsl), which would still capture 64% (168/264) and 77% (34/44) of LPA-detectable smear-negative resistance, respectively. Use of C_Tmins permitted optimization of this ratio with higher specificity for non-actionable results but decreased resistance detected. Xpert quantitative information permits identification of a smear-negative subset in whom the payoffs of the ratio of actionable-to-non-actionable LPA results with missed resistance may prove acceptable to laboratories, depending on context. Our findings permit the rational expansion of direct DST to certain smear-negative sputum specimens.

INTRODUCTION

Reflex drug susceptibility testing (DST) should be done in all rifampicin-resistant tuberculosis (TB) cases to enable rapid effective treatment. Achieving this depends doing DST directly, including on smear-negative specimens. However, the widely-used WHO-endorsed line probe assays (LPAs) MTBDRplus (which detects first-line resistance to rifampicin and isoniazid) and especially MTBDRsl (both Bruker, Germany; second-line drug resistance to fluoroquinolones and second-line injectable drugs) perform sub-optimally on paucibacillary specimens (1) and can fail to generate an actionable (resistance or susceptible) result. Culture is hence often required to generate high concentrations of bacilli for testing; however, culture is costly and slow.

Non-actionable MTBDRplus and MTBDRsl results in our setting of a large TB reference laboratory in Cape Town, South Africa occur in ~24 and ~40% of Xpert MTB/RIF (Xpert)-positive smear-negative patients, respectively, and are a larger cause of missed resistance than LPA false-susceptible results (2). Laboratories typically use smear status to guide whether LPA testing is done on a specimen and may choose to not directly test smear-negative specimens (in which case LPAs are done on isolates). If laboratories universally do LPAs on smear-negative specimens it can, due to frequent non-actionable results, result in wasteful expenditure (consumables and labor for MTBDRsl are ~ $50 [3]), care cascade loss (additional specimen collection is only triggered once the LPA is known to be non-actionable and is itself often unsuccessful), and reduced user confidence. Therefore, despite the WHO recommendation that MTBDRsl is done on smear-negative specimens, direct testing is often in reality limited to smear-positive specimens, even in well-resourced settings, as shown a recent survey (4, 5). This underutilization undermines LPAs’ potential impact, which remain the only molecular DSTs deployed widely for first- and second-line resistance. LPAs may indeed work well on some smear-negatives; however, as smear microscopy is a crude and insensitive categorical measure of bacterial load (6, 7), it is unable to identify this subset of smear-negatives upfront.

We hypothesized that, in situations where Xpert is a frontline TB test, its already-available molecular quantitative information could be used to a priori exclude certain specimens from unnecessary LPA testing; thereby permitting LPAs to be applied more efficiently (i.e., on specimens with a reduced non-actionable result risk) and, if laboratories do not directly test smear-negative specimens, this information would permit LPA testing to be expanded to some smear-negatives. In other words, preexisting quantitative information routinely generated by Xpert could be used to improve LPA-based laboratory decision making and the drug-resistant TB care cascade. We also evaluated if smear grade would be more useful than smear status (positive or negative) for situations where Xpert quantitative information is not available (due to, for example, limitations in laboratory information systems). This framework would also be useful for new DSTs expected to succeed LPAs that still perform sub-optimally on paucibacillary sputum.

MATERIALS AND METHODS

Microbiology.

We analyzed Auramine smear microscopy, Xpert MTB/RIF (v4.3), MTBDRplus and MTBDRsl (both v2) results. The cohort consisted of 951 people programmatically-diagnosed with Xpert rifampicin-resistant TB from 01/06/2016 to 30/09/2019 at a high-volume laboratory in a cohort where LPA sensitivity and specificity for resistance was previously-described (2). All people had sputum tested directly with both LPAs irrespective of smear status.

Analyses.

(i) AUROCs. We did area under the receiver operator characteristic (AUROC) curve analyses (GraphPad v6, USA) using different sputum bacterial load measures (smear microscopy, Xpert semi-quantitation category and C_Tmin; AUROCs that accounted for PCR inhibition [C_T SPC probe - C_Tmin rpoB] were also derived) to classify if MTBDRplus or MTBDRsl were non-actionable, defined as an absence of a resistant or susceptible result (hence not providing actionable information to health workers). We identified thresholds corresponding to rule-out (≥95% sensitivity; almost all non-actionables correctly identified) and rule-in (≥95% specificity; almost all actionables correctly identified) scenarios, as well as Youden’s index (8), but primarily present data from the rule-in scenario because it is the most appropriate clinical use case as it would minimize the incorrect exclusion of people from the potential benefits of rapid LPAs.

(ii) Definitions. LPA non-actionable results occur when both the amplification control and/or TB detection bands are absent or, if both are present, at least 1 drug class locus control band is absent. Invalid results reflect absence of either control bands (amplification and/or conjugate band), and TB detection bands are absent or indeterminate for 1 or both gene locus control bands. Smear microscopy grading (SC, P+, P++, P++) was done per guidelines (9). Xpert semi-quantitation category (“very low,” “low,” “medium,” “high”) is generated automatically by the GeneXpert software based on the C_T value of the first positive rpoB probe (10) and the minimum cycle threshold value (C_Tmin) which, in a positive Xpert, is the lowest C_T value (rounded to nearest integer) of the rpoB probes.

(iii) Diagnostic accuracy metrics. Sensitivity and specificity (95% binomial confidence intervals) of bacterial load for non-actionable results were evaluated overall and in smear-positive versus -negative specimens. We calculated, at each bacterial load threshold, the number of actionable results that are generated before a non-actionable is encountered (ratio of actionable-to-non-actionable results) and how maximizing this ratio was offset against missed LPA-based isoniazid (MTBDRplus) and fluoroquinolone (MTBDRsl) diagnoses. We also calculated PPV as, of the proportion of people with a positive result (i.e., classified as “likely non-actionable,”) those that were truly non-actionable. Similarly, NPV was, of those with a negative result (i.e., not classified as “likely non-actionable) those truly actionable.” We conducted analyses overall and restricted them to smear-negative people.

Ethics.

This study was approved by the Stellenbosch University Health Research Ethics Committee (N16/04/045) and Western Province Department of Health (2016/RP18/637).

RESULTS

Non-actionable LPAs and missed resistance diagnoses stratified by smear status and grade.

MTBDRplus and MTBDRsl non-actionable result rates irrespective of smear status were 19% (148/792) and 40% (267/673) (actionable-to-non-actionable results ratios of 5.4 and 2.5, respectively). Smear-negative specimens were, compared to smear-positives, more likely to generate a non-actionable MTBDRplus (23% [133/559] versus 4% [15/381]; P = 0.001) or MTBDRsl (39% [220/559] versus 12% [47/381]; P < 0.001) result (ratios of 3.2 and 24.4 for MTBDRplus, 1.5 and 7.1 for MTBDRsl, respectively). Non-actionable results, a receiver operating characteristic (ROC) curve of smear grade to detect non-actionable results, and the balance between the number of actionable results per non-actionable result and missed rapid drug resistance diagnoses are summarized in Fig. 1 (positive and negative predictive values in Fig. S1).

FIG 1.

Smear grade’s ability to discriminate “likely non-actionable” from “likely actionable” results (if ≤ each grade) and associated pay-offs between the ratio of actionable-to-non-actionable results with missed resistance. (A) Non-actionable results were more frequent at lower than higher grades and more so for MTBDRsl than MTBDRplus. In-column percentages reflect the proportion patients with a non-actionable result. (B) Smear grade had moderate AUC for identifying “likely non-actionable” results (dashed lines 95% CIs) but no grade approached 95% specificity (where most people with an actionable result would be correctly identified as actionable). However, at most thresholds, sensitivity (the proportion of people with a non-actionable result classified as “likely non-actionable) remained high.” (C and D) show the ratio of actionable-to-non-actionable results (solid lines, left y-axes) and the proportion of LPA-detectable resistance successfully included or detected using that threshold (dashed lines, right y-axes), and how these change as specimens with a certain smear grade (or greater) are tested by MTBDRplus or MTBDRsl, respectively. Abbreviations: AUCs-area under curve, CI-confidence intervals, FQ^R-fluroquinolone resistance, INH^R-isoniazid resistance, LPA-line probe assay, P-positive, SC-scanty, Xpert-Xpert MTB/RIF.

MTBDRplus.

Smear status (smear-negativity) to identify non-actionable results had a sensitivity and specificity of 90% (133/148) and 54% (426/792), respectively. While more non-actionable results occurred in smear-negatives rather than smear-positives (Fig. 1A), smear grade had suboptimal area under the curve (AUC) for predicting non-actionable results (Fig. 1B). The actionable-to-non-actionable ratio improved as increasing smear grades are used to exclude specimens from testing (if they are less or equal to that grade); however, this is offset against substantial missed resistance (Fig. 1C). For example, to improve this ratio to 21.5 (threshold of scanty-positive and above, in other words, any smear-positive tested or smear-negatives to be omitted), 51% (273/537) of LPA-diagnosable isoniazid resistance would be detected (Table S1).

MTBDRsl.

Smear-negativity had a sensitivity and specificity of 83% (220/266) and 54% (339/674) for non-actionable results. The actionable-to-non-actionable ratio was less than MTBDRplus’s, driven by more frequent non-actionable results in smear-negatives (39% [220/559] versus 23% [133/559] for MTBDRplus, P < 0.001). For example, MTBDRsl’s highest ratio was 16 (Fig. 1D) whereas for MTBDRplus it was 109 (~7-fold higher). If smear-negative specimens were excluded from MTBDRsl, only 58% (60/104) of LPA-diagnosable fluroquinolone resistance would be detected (Table S1).

Xpert MTB/RIF semi-quantitation category.

(i) All patients. (a) MTBDRplus. Non-actionable results were more frequent at lower semi-quantitation categories (Fig. 2A) and, in the “very low” category, higher than in smear-negative people overall (49% [62/126] versus 23% [133/559]; P < 0.001). Semi-quantitation category had higher AUC than smear grade. No semi-quantitation category threshold approached the rule-in specificity criterion where ~95% of all actionables would be correctly identified (Fig. 2B). The largest improvement in the ratio of actionables-to-non-actionables occurred when specimens in the lowest 2 semi-quantitation categories (“very low,” “low”) were excluded (5.4 when all tested versus 24.2 if ≥ “medium” included, meaning that only specimens with this semi-quantitation category or greater would get MTBDRplus) and this was accompanied by a moderate reduction in detected resistance of 22% (118/537) (Fig. 2C). In other words, if ≥ “medium” was used, ~5-fold fewer non-actionables would occur and 78% (419/537) of potentially detectable resistance would still be detected.

FIG 2.

Xpert semi-quantitation category, non-actionable LPA results, and pay-offs with missed resistance as specimens with bacterial loads less than or equal specific semi-quantitation categories are excluded due to being flagged as “likely non-actionable”. (A) Trends for semi-quantitation category mirrored those for smear grade. (B) This translated into moderate AUCs for discriminating “likely non-actionable results” (dashed lines 95% CIs) but no optimal rule-in threshold was identifiable (AUCs were slightly diminished in smear-negatives, Fig. S2). (C and D) show the ratio of actionable-to-non-actionable results (solid lines, left y-axes) and the corresponding proportion of LPA resistance captured using varying thresholds (dashed, right y-axes) and how this improves as specimens with higher semi-quantitation categories are tested by MTBDRplus or MTBDRsl, respectively. (E and F) are limited to smear-negative specimens. Abbreviations: AUCs-area under curve, CI-confidence intervals, FQ^R-fluoroquinolone resistance, H- “high,” INH^R-isoniazid resistance, LPA-line probe assay, L- “low,” M- “medium,” NPV-negative predictive value, P-positive, PPV-positive predictive value, VL- “very low,” Xpert-Xpert MTB/RIF.

(b) MTBDRsl. MTBDRsl non-actionable rates in the “very low” and “low” semi-quantitation categories were higher than in smear-negative specimens (71% [90/126] and 49% [102/210] in each category versus 39% [220/559; P < 0.001, P = 0.021]). MTBDRsl never obtained similar actionable-to-non-actionable ratios to MTBDRplus, even when specimens with the same semi-quantitation category were compared. If ≥ “medium” was used, this ratio improved ~3-fold from 2.5 to 7.2 with 88% (92/104) of potentially detectable resistance detected (Fig. 2D).

(ii) Smear-negatives. (a) MTBDRplus. Semi-quantitation category AUCROCs in smear-negatives for both LPAs were less than those seen overall (Fig. S2 and Fig. 2). If laboratories that do not test smear-negative people wish to partly expand testing, they may test ≥ “medium” smear-negatives (ratio 12.8 versus 3.2 for the test-all smear-negatives strategy, 4-fold improvement), which would still capture 64% (168/264) of detectable resistance (Fig. 2E). Within smear-negatives, 20% (114/559), 33% (182/559), 36% (200/559), and 11% (63/559) were “very low,” “low,” “medium,” and “high,” respectively; meaning 47% (263/559) of smear-negatives would be ≥ “medium,” showing this rule could significantly expand testing in smear-negatives with dramatically fewer non-actionable result rates.

(b) MTBDRsl. Similarly, if MTBDRsl was done on ≥ “medium” smear-negatives, the ratio would improve from 1.5 for the test-all strategy to 4.5 (3-fold improvement), with 77% (34/44) of detectable resistance detected (Fig. 2F).

Xpert MTB/RIF C_Tmin.

(i) All patients. (a) MTBDRplus. C_Tmin had, compared to smear grade and Xpert semi-quantitation category, higher AUCROCs (Fig. 3A) and was the only readout that met the rule-in ≥ 95% specificity criterion at C_Tmin ≥ 29, which occurred in 11% (86/792) of patients. C_Tmin ≥ 29 had 30% (44/148) sensitivity, which is the proportion of non-actionables correctly classified as “likely non-actionable” (non-actionables hence reduced by a third). This threshold had 95% (750/792) specificity, meaning that, offset against this improvement in non-actionable rate, 5% (42/792) of actionables would be misclassified as “likely non-actionable” and hence erroneously excluded from MTBDRplus (Table S1). NPV was 80% (750/854) meaning that, for every 10 patients with C_Tmin < 29 (and hence classified as “likely actionable”), 8 would indeed be actionable and other 2 non-actionable (false-negative). PPV was 51% (44/86), meaning approximately half of patients with a C_Tmin ≥ 29 (hence classified as non-actionable), would indeed be non-actionable and the others actionable (false-positive) (Fig. S1). Ratios of actionable-to-non-actionable results for C_Tmin were broadly like those for semi-quantitation category, peaking at ~ 77 (C_Tmin ≤ 12; estimates less than this imprecise due to few specimens with extremely low C_Tmins) (Fig. 3B). At C_Tmin ≥ 29, this ratio is 6.4 (improved from the test-all ratio of 5.4) at the cost of missing 5% (25/537) of potentially detectable resistance.

FIG 3.

Xpert C_Tmin’s ability to discriminate “likely non-actionable” from “likely actionable” LPA results. (A) A ROC curve for all specimens showing AUCs (dashed lines 95% CIs, rule-in thresholds shown [≥ 95% specificity; almost all actionables correctly identified]). (B and C) (solid lines, left y axis) show pay-offs between the ratios of actionable-to-non-actionable results and the corresponding proportion of resistance detected (dashed right, y axis). (D and E) are the same but restricted to smear-negative people. Ratios were highest at low C_Tmin and slowly decreased as LPA testing expanded to include specimens with higher C_Tmins, which had the upside of increasing detected resistance. AUCs and these ratios were less for smear-negative people than overall data. Above C_Tmin x axes are Xpert semi-quantitation categories. Abbreviations: AUC-area under curve, CI-confidence intervals, C_Tmin-cycle threshold (minimum), FQ^R-fluoroquinolone resistance, INH^R-isoniazid resistance, LPA-line probe assay, NPV-negative predictive value, P-positive, PPV-positive predictive value, ROC-receiver operator characteristic, Xpert-Xpert MTB/RIF.

(b) MTBDRsl. C_Tmin ≥ 28 had 34% (90/266) sensitivity, permitting an approximate one third reduction in non-actionables, and a rule-in specificity of 95% (638/674), meaning 5% (36/674) of actionables would be misclassified as “likely non-actionable” (Fig. 3A). NPV was like that for MTBDRplus (Fig. S1) but PPV higher (71% [90/126] versus 51% [44/86] for MTBDRplus; P = 0.003), meaning approximately 7/10 people with C_Tmin ≥ 28 (hence classified as non-actionable), would indeed be non-actionable and the other 3/10 actionable (false-positive). Ratios of actionable-to-non-actionables results peaked at ~ 38 (C_Tmin 16), less than half that of the best ratios obtained with MTBDRplus. At C_Tmin ≥ 28, this ratio would be 7.0 (versus the test-all ratio of 5.0, 1.4-fold or 40% improvement) and would result in only 4% (4/104) of potentially detectable resistance missed.

(ii) Smear-negative patients. (a) MTBDRplus. C_Tmin had less AUCROC in smear-negatives than overall with a similar rule-in threshold (Fig. 3D). Even at the same C_Tmins, slightly worse (lower) actionable-to-non-actionable ratios occurred in smear-negatives (Fig. 3E) (for example, 13.8 versus 24 overall at C_Tmin 20). If the rule-in C_Tmin < 29 threshold was used, this ratio was 4.4 (versus 3.2 for the test-all smear-negatives strategy, representing a 38% improvement) and resulted in 91% (241/264) of potentially detectable resistance captured. Furthermore, ratios ≥10 were possible, permitting MTBDRplus to be expanded to at least some smear-negatives (C_Tmin < 23; 37% [156/426] of smear-negatives and 67% [177/264] of LPA-detectable resistance).

(b) MTBDRsl. If the rule-in C_Tmin < 29 threshold was used, the actionable-to-non-actionable result ratio was 1.7 (versus 1.5 for the test-all smear-negatives strategy, a 13% improvement) and resulted in 93% (41/44) of potentially detectable resistance detected. MTBDRsl on specimens with C_Tmin < 19 would have a ratio of 5.4 (Fig. 3F), which may be more acceptable in settings where non-actionable results on smear-negative specimens can be tolerated less. This ratio was better than that in test-all strategy (3.6-fold improvement) and use of the ≥ “medium” semi-quantitation category (ratio of 4.5). A total of 36% (119/332) of smear-negatives were C_Tmin < 19, corresponding to 45% (24/44) of detectable resistance. Predictive values of this approach in smear-negatives, including for MTBDRplus, are in Fig. S1.

Xpert MTB/RIF C_T SPC probe - C_Tmin rpoB strategy.

AUROCs for the C_T SPC probe - C_Tmin rpoB strategy had similar point estimates to C_Tmin alone and were only significantly lower for MTBDRplus overall and in smear-negative people (Table S2, no differences for MTBDRsl).

DISCUSSION

LPAs are WHO-recommended first- and second-line rapid DSTs. However, they are not always performed directly on specimens in which they may provide an actionable resistant or susceptible result, in part due to an elevated non-actionable result risk in smear-negatives. This deprives people of the benefits of early DST that are possible using currently available widely-implemented tests. Although better DSTs, especially for second-line resistance, are doubtlessly required, the use of existing widely-available technologies should be optimized.

Focusing on smear-negative specimens specifically, our key findings are: (i) Performing MTBDRplus testing only on specimens with C_Tmins below specific thresholds (e.g., 29) reduces non-actionable result rates and allows most LPA-detectable isoniazid resistance to be detected, (ii) for MTBDRsl, which usually results in 1.5 actionables per non-actionable result, ratios close to 5 are attainable (C_Tmin < 19), permitting 45% of detectable fluroquinolone resistance to be detected, and (iii) if C_Tmins are unavailable or not captured, use of an Xpert semi-quantitation category ≥ “medium” threshold would expand LPA testing to almost half of smear-negatives without encountering high non-actionable result rates. However, unlike C_Tmin, use of semi-quantitation categories did not achieve high rule-in specificity. Our study provides a framework for how LPA testing (or any form of downstream DST) can be made more efficient.

The precise threshold (and type of readout) used to determine whether LPA testing on smear-negatives should proceed will depend on locally-acceptable ratios of actionable-to-non-actionable results versus the proportion of isoniazid or fluoroquinolone resistance laboratories are comfortable excluding from the potential benefits of direct LPA DST. For example, C_Tmin < 29 improves the ratio of actionable-to-non-actionable MTBDRplus results on smear-negatives specimens by over a third (to 4.4) and detects > 90% of resistance, whereas C_Tmin < 23 improves this ratio to over 10 and detects about two thirds of resistance. For MTBDRsl, more resistance is missed in comparison to MTBDRplus as lower bacillary load specimens are excluded. However, improvements in the ratio for MTBDRsl on smear-negatives (3.6-fold or from 1.7 to 5.4, C_Tmin < 19) still occur.

Our findings also demonstrate that, where WHO-recommended rapid molecular diagnostic tests are available, smear microscopy, which comes at additional expense and is less accurate at informing when “likely actionable” LPA testing should occur, is increasingly redundant for guiding downstream laboratory decision making given the large range of Xpert C_Tmins (and to a lesser extent semi-quantitation categories) within people with smear-negative sputum (6). PCR test quantitative readouts should therefore be used to guide DST rather than smear status.

The principle of applying molecular quantitative information to determine how downstream is completed, DST is agnostic to both the frontline TB test, which could include quantitative nucleic acid amplification tests like Truenat rather than Xpert, as well as the downstream molecular DST, which could include Xpert MTB/XDR (11), FluoroType MTBDR (12) and others (13), all of which will likely be expensive and need to be selectively done. Importantly, frontline tests often target multicopy genes that genotypic DSTs do not, resulting in differences in the limit of detection and increasing the risk of downstream non-actionable results from DSTs. Thus, knowing which TB-positive specimens may proceed onward to downstream DST with high actionable result likelihood is a need that will grow.

A strength and limitation are that our study is from a programmatic context, which permitted large sample size; however, the exact thresholds require validation in other settings or laboratories. Furthermore, as our study used sputum per our local algorithm (14), Xpert was done a different sputum collected on the same day as the one used for smear microscopy, LPA, and culture. Variations in mycobacterial load between sputum may have affected our results. Our study was intended to demonstrate proof-of-concept and illustrate what, purely from a laboratory perspective, such payoffs may look like. Although our findings permit the use of Xpert information to rationally expand the use of existing LPAs to certain paucibacillary specimens ordinarily excluded, we affirm that, resource-permitting, isoniazid and fluoroquinolones DST should be attempted directly on any TB-positive rifampicin-resistant specimen irrespective of smear status (15). Hence, our findings will primarily be of interest to settings where direct MTBDRplus or MTBDRsl testing of smear-negatives is not conducted (1, 16). Lastly, future work should include Ultra as many countries are transitioning from Xpert; however, Xpert and Ultra rpoB C_T values correlate highly, meaning that the thresholds identified here may also have utility for Ultra (17).

In summary, we demonstrated how LPAs may be reliably expanded to a significant proportion of smear-negative patients while detecting resistant cases that would otherwise be missed. Xpert C_Tmins or, failing that, Xpert semi-quantitation category were superior to smear status to decide if reflex LPA should be completed directly. More broadly, the utility of molecular quantitative information already generated as part of the TB diagnostic process for informing decision making regarding downstream DSTs requires better consideration.