Rapid Implementation of New TB Diagnostic Tests: Is It Too Soon for a Global Roll-Out of Xpert MTB/RIF?

Abstract

In 2011 the World Health Organization approved Xpert MTB/RIF for tuberculosis diagnosis and recommended its rapid implementation. Xpert MTB/RIF is accurate: sensitivity is 72.5–98.2% (smear-negative and -positive cases, respectively) and specificity 99.2%. Benefits include same-day diagnosis and simultaneous detection of rifampicin resistance. However, the test has some shortcomings and has not had time for thorough evaluation. Cost-effectiveness studies are difficult to perform and few have been completed. Existing data suggest cost-effectiveness in some, but not all, settings. The urgent need for better diagnostics is evident. Yet, serial implementation of new technologies causes ineffective spending and fragmentation of services. How new tests are incorporated into existing diagnostic algorithms affects both outcomes and costs. More detailed data on performance, effect on patient-important outcomes, and costs when used with adjunct tests are needed for each setting before implementation. While awaiting further clarification it seems prudent to slow its implementation among resource-constrained tuberculosis control programs.

Background

The World Health Organization (WHO) estimates that in 2010, there were 8.8 million incident cases of tuberculosis (TB), and 1.45 million deaths from the disease.¹ Although incidence and mortality are decreasing overall, certain aspects of TB control remain unsatisfactory, including the identification and treatment of patients infected with drug-resistant strains. This is limited in part by a reliance on suboptimal diagnostics. During the last decade there has been a considerable effort to improve diagnostic capacity, leading to the development and implementation of a variety of new diagnostic tools.

In 2011, the WHO endorsed the Xpert MTB/RIF diagnostic test and provided a guide for its rapid implementation.^2–4 This followed recommendations from an expert group meeting and global consultation in late 2010 that examined data from four published works and unpublished data from 12 investigator-driven, single-center studies. Xpert MTB/RIF is already in use in over 40 countries,⁵ which is remarkable given that the first work reporting data using this test was published in 2010. This rapidity has generated questions regarding whether such widespread and speedy implementation is appropriate.^6,7

Xpert MTB/RIF: A promising new technology.

The Xpert MTB/RIF has been hailed as a significant breakthrough in TB diagnostics, and for good reason. The first diagnostic test based on molecular technologies to have reached maturity, it has proven to be accurate and effective, with testing of a single sample giving a sensitivity of 98.2% among smear-positive cases and 72.5% for culture-positive, smear-negative cases and 99.2% specificity.⁸ Results are generated within 90 minutes and, crucially, rifampicin resistance is detected at diagnosis. Whereas isoniazid monoresistance is common, rifampicin monoresistance is not, and resistance to rifampicin is considered a good surrogate for multidrug-resistant tuberculosis (MDR-TB)⁹; this information thus allows appropriate treatment to be instigated from the outset, avoiding an initial period of ineffective therapy, and thereby limiting morbidity and mortality from the infection and the generation and dissemination of increasingly drug-resistant bacteria.

The equipment is simple to use, requires minimal training, and there is no need for a biological safety laboratory environment. These characteristics mean it has the potential to be used at the point of treatment in a microscopy center or in a TB or human immunodeficiency virus (HIV) clinic, thus decentralizing molecular TB diagnosis.¹⁰ Furthermore, the platform can accommodate high sample numbers, it gives standardized results, and there is almost no risk of cross-contamination and a lack of cross-reactivity with other mycobacteria. It is not surprising that the WHO and others are keen to extend its availability as far as possible.

Yet, the test has some limitations. The analytical limit of detection for Xpert MTB/RIF is reported to be higher than that for culture (131 CFU/mL¹¹ and 10–100 CFU/mL of specimen, respectively), and there have been instances of false reporting of resistance in samples with a low bacillary load.¹² This is particularly concerning when considering children and patients with HIV, who characteristically exhibit paucibacillary disease; yet, despite a lack of evidence in pediatric cases,^13,14 the WHO recommendations have been applied to all patients including children. There is also uncertainty regarding the interpretation of culture-negative, Xpert MTB/RIF-positive samples.¹⁵ In addition, although the sensitivity and specificity of the test are high, their usefulness depends on the positive and negative predictive values, which vary according to local prevalence of both TB and of drug resistance. In most low income countries, the prevalence of rifampicin resistance among treatment-naive patients is around 2%, whereas in some locations such as Tajikistan it exceeds 15%.¹⁶ A test with a sensitivity of 95% and specificity of 98% gives a positive predictive value of over 90% in the former setting and only 49% in the latter.¹⁷ These limitations mean that Xpert MTB/RIF may not be appropriate for use in all settings.

What is the cost of implementation?

Xpert MTB/RIF is a high-tech tool and upfront costs initially appear discouragingly high: at subsidized prices, the platform is priced at US$17,000 and each cartridge ∼US$17.¹⁸ However, this does not necessarily translate to high costs when economic data are looked at more closely. Although the WHO has stated that the capital and running costs of Xpert MTB/RIF are similar to the cost of performing culture and drug susceptibility testing,³ this is controversial and expert opinion is polarized. Moreover, given the high cost of TB treatment, in particular of MDR-TB, even an expensive diagnostic test that can reduce treatment costs may prove attractive.¹⁹

The implementation of a new test in a given setting is a big commitment and the impact of its introduction will be felt for years to come. Given resource constraints in high-burden settings, national TB programs need to make important policy decisions about which tests to use, and how to incorporate new tests into existing diagnostic algorithms. Planning, sourcing and setting up equipment, training, and adjustment of local practices to a new technology take time, and ensuring long-term sustainability of servicing and consumables is often more challenging than initial implementation of the equipment.

Recent commentaries,^20,21 and the WHO themselves,² have highlighted the need for detailed data on the performance and cost of Xpert MTB/RIF in combination with adjunct tests to permit informed and effective decisions. New information is constantly emerging, and over the past months evaluations of the cost of using Xpert MTB/RIF alone or in conjunction with adjunct tests have been published.^22,23 However, these studies are still few in number, and such analyses are complex to perform and vary in their methodologies, settings, and conclusions.

To generate meaningful cost-effectiveness data, the quality of the cost and outcome figures used in the calculations must be high. Good quality data is difficult to collect and often unavailable, and calculations rely heavily on estimates. Regional price disparities and fluctuating markets make it difficult to generalize findings and recommendations, and complex and often hidden costs need to be taken into account. These include those pertaining to infrastructure, equipment maintenance and calibration, import and transport of materials, regular supplies because of short reagent shelf-lives, and safe disposal of waste products. Considerations should include the burden to both the health service and the patient and their family,^24,25 factors seldom included in published cost-effectiveness data. Moreover, despite the existence of many available tools for performing cost-effectiveness analyses,²⁶ there is no consensus on how one ought to be performed. Several comprehensive publications discuss the challenges inherent in making such a calculation.^25,27

Sophisticated cost-effectiveness evaluations are being performed as part of the ongoing Xpert MTB/RIF roll-out process and have thus far shown promising results. A recent analysis of the effect of Xpert MTB/RIF on the cost-effectiveness of TB care in low- and middle-income settings compared the introduction of Xpert MTB/RIF to a base case of sputum microscopy and clinical diagnosis.²² The authors drew favorable conclusions, finding an incremental cost-effectiveness ratio (ICER) of US$41–$138 per disability adjusted life year (DALY) averted amongst the different settings. These ICERs fall below the WHO willingness to pay threshold, implying feasibility. However, as this is a purely theoretical threshold based on a country's per capita gross domestic product, such conclusions must be interpreted with caution. Limitations of this analysis include the assumption that a negative Xpert MTB/RIF result will not lead to further TB diagnostic procedures, which is unrealistic especially in HIV-positive individuals.

It is important not to be distracted by attractive ICERs and to remember that not all cost-effective strategies are affordable, and thus cannot necessarily be adopted.²⁴ As many as 60% of TB patients may live in areas where laboratory infrastructure is insufficient to perform even sputum smear microscopy, let alone complex molecular techniques, regardless of their cost-effectiveness.²⁸ A South African modeling exercise²³ found that implementing Xpert MTB/RIF increased the cost per TB case diagnosed by 17%, from $312 to $367, and by 9% ($835 to $912) when treatment costs were taken into account. Although this seems a modest increase, the additional sum—which has to be budgeted over and above the cost of the current diagnostic guidelines—of its implementation was calculated at US$ 301–343 million. In the context of an underfunded health system with numerous compelling needs creating intense competition for resources, if this additional sum cannot be obtained, the strategy simply cannot be applied.

To decide whether an intervention ought to be adopted, a budget impact analysis is needed. This estimates the financial consequences of adoption and diffusion of a new intervention within a specific setting to assess its affordability, given the setting's own resource constraints.²⁹ The Liverpool School of Tropical Medicine and collaborators have recently developed a useful tool to support decision making with regard to the implementation of a given new diagnostic method.³⁰ The Impact Assessment Framework (IAF) comprises analysis of five components: effectiveness, equity, health systems, scale-up, and policy. It includes a critical appraisal of the new intervention against other available or anticipated interventions to address the important risk that the test may be supplanted by newer technology within a short period of time. The IAF is expected to strengthen the evidence base and facilitate decisions about which new diagnostic tests and approaches to adopt, and when and how to implement them.

The impact of a given diagnostic strategy is particularly relevant in the context of the high costs and difficulties of treating versus diagnosing MDR-TB³¹: even under well-managed conditions, treatment success with the current WHO-recommended MDR-TB regimens is only around 60%.³² Regardless of the cost-effectiveness of a new diagnostic technique, unless there is guaranteed provision of appropriate drugs and care for patients with MDR-TB, it is at best pointless and at worst harmful to implement such a technique. There is an ethical obligation to ensure treatment of any patient diagnosed, particularly in the context of extensively drug-resistant (XDR) TB that is generated by the mismanagement of patients with MDR-TB. The WHO lists high-quality MDR-TB treatment as a non-negotiable prerequisite,⁴ making it necessary to first identify settings in which the national program is able to adequately treat patients with drug-resistant TB, before striving for the best methods of performing drug susceptibility testing.

What are the wider consequences of hasty implementation?

Xpert MTB/RIF is the latest in a succession of new diagnostics for TB that have emerged over the years, including the Mycobacteria Growth Indicator Tube (MGIT) system, Griess, the Microscopic-Observation Drug-Susceptibility (MODS) assay, thin-layer agar, colorimetric assays, line probe assays, and more recent molecular detection methods. Several developments during this process have generated an enthusiastic response, each has been validated and implemented to varying degrees, and many subsequently failed to show sustainable benefits and have fallen into disuse. This is often commercially driven to some extent: non-commercial tests such as nitrate reductase assays and MODS have shown excellent results at low cost but have not received enthusiastic endorsements, whilst implementation of commercial devices without full evaluation of their long-term sustainability has sadly led to expensive equipment falling out of use. For example, interferon gamma release assays (IGRAs) were received as a major breakthrough in TB diagnostics, resulting in numerous costly studies and widespread implementation of the test. Ultimately, they have proven incapable of differentiating active TB from latent infection and are not significantly superior to their cheaper predecessor, the tuberculin skin test. An expert panel recently concluded that IGRAs should be used only as part of an overall diagnostic work-up for latent TB and not to diagnose or rule out diagnose active TB.³³ It has been suggested that lessons need to be learned from this experience and that proper evaluation of new diagnostics needs to be conducted.³⁴

Although the urgent need for better diagnostics is evident, each implementation phase has costs associated with requirements for additional material and human resources, parallel tests, and training, monitoring and evaluation. If this is a single and appropriate occurrence, the long-term cost savings and benefits justify this initial disruption. However, the consecutive nature of test development can result in serial installations of different technologies within as little as a decade within the same region. Overlapping implementation phases cause frequent changes in local policy, leading to wasted money and fragmentation of services. It also means that cost comparison studies are often performed after, rather than prior to, this process. The WHO itself stresses that Xpert MTB/RIF is only the first of a new generation of automated molecular diagnostic platforms, and that others are at prototype stage and expected to become available in due course.² If this is the case, then can we expect subsequent recommendations for their implementation? With the promise of other tests on the horizon, it might be prudent to hold back on widespread implementation of Xpert MTB/RIF while waiting to see whether they can offer any additional benefits.

Potential problems with the WHO approach to incorporating the test in TB program algorithms.

Once the decision has been taken to adopt a tool such as Xpert MTB/RIF, a TB program must decide which patients it will be used for and how to incorporate it into its diagnostic pathway. Its position within an algorithm will significantly affect costs. For example, a South African study found that using smear microscopy to prescreen TB suspects before Xpert MTB/RIF testing reduced the cost of detecting one TB case by $115 compared with Xpert MTB/RIF alone ($401 versus $516).³⁵ Costs are also sensitive to local demographic and epidemiological characteristics, such as HIV prevalence. In this study, the cost was much lower in HIV-infected patients than HIV-uninfected patients for both strategies (Xpert MTB/RIF alone: $202 and $1446 per TB case detected in HIV-infected and -uninfected patients, respectively; smear microscopy followed by Xpert MTB/RIF: $200 and $669 in HIV-infected and -uninfected patients, respectively). The test may therefore lend itself particularly well for use in populations with a high HIV prevalence, but on the other hand, costs may be too high for use in populations with low HIV prevalence. Moreover, in the mentioned study, the ranking of cost-effectiveness of different diagnostic strategies was found to be sensitive to baseline Xpert MTB/RIF performance. The authors found that repeating their calculations using data from Boheme and others initial validation study rather than their own data, Xpert MTB/RIF alone was actually found to be cheaper than microscopy plus Xpert MTB/RIF, because of the higher diagnostic accuracy of Xpert MTB/RIF in the larger, rigorously conducted trial. This highlights the danger of using data from meticulously conducted validation studies to direct global policy decisions.

The WHO advocates use of a risk factor-based approach in allocating different testing modalities. Xpert MTB/RIF is recommended as the initial diagnostic test in patients suspected of having MDR-TB or HIV-associated TB, and as a follow-on test to microscopy in all other cases (conditional recommendation, recognizing major resource implications)³; this policy is designed to optimize use of resources. However, although studies have shown that factors such as previous TB treatment and contact with MDR-TB cases are indeed strong risk factors for infection with a drug-resistant strain, association with HIV is more controversial,³⁶ and over 50% of patients who have MDR-TB are thought not to have any such risk factors at all¹⁹ and would not be identified using this approach. Levels of resistance among circulating strains vary in different settings, which affects the success of any algorithm based on presence or absence of risk factors in an individual.

Use of Xpert MTB/RIF as an initial test can be problematic. It tests for resistance to rifampicin only, and without the concurrent use of culture, identification of resistance to other drugs is not possible. With the growing importance of MDR- and XDR-TB, there is increasing interest in patterns of resistance to other medications, with work currently underway to improve detection of resistance to pyrazinamide³⁷ and to second-line drugs such as ciprofloxacin and kanamicin.³⁸ Thus, conventional microscopy, culture, and drug susceptibility testing remain necessary for monitoring of therapy, prevalence surveys and/or surveillance, and for recovering isolates for drug susceptibility testing other than rifampicin including second-line drugs. This means that, for the foreseeable future, laboratories will be required to use all of these technologies in parallel.

The cost-effectiveness of Xpert MTB/RIF compared with other available technologies will therefore depend greatly on the presently available facilities in a given location, such as access to culture, full resistance testing, and other molecular technologies. A recent British study looked at the performance of line-probe assays (LPA) within a national service under non-trial conditions.³⁹ The authors found that the LPA showed high diagnostic accuracy, comparable to Xpert MTB/RIF, and was both rapid and reliable. It requires less (and mostly generic) laboratory equipment than Xpert MTB/RIF, but is more technically demanding. Thus, the LPA may prove an effective and cheaper alternative in centers already equipped with the required technical skills, whereas Xpert MTB/RIF may prove more suitable in centers with sufficient equipment to support the logistic and operational challenges of Xpert MTB/RIF outlined previously, but which do not routinely perform molecular assays.

The advantage of using rapid molecular testing is that it identifies individuals who require full resistance testing. Persons shown to be infected with TB that is sensitive to rifampicin can safely be commenced on first-line treatment, whereas same-day detection of individuals with rifampicin resistance enables collection of further sputum samples for full resistance testing without delaying the initiation of their treatment. Yet, depending on a region's current strategy toward resistance testing, the identification of more rifampicin-resistant TB through use of Xpert MTB/RIF as a screening tool for resistance may paradoxically increase the demand for specialist reference laboratories to undertake further testing.²¹ This needs to be factored into calculations of the cost of implementation of Xpert MTB/RIF.

A more cautious approach.

Xpert MTB/RIF has so far proven to be accurate and cost-effective, and the test has great potential for widespread applicability. However, it is still in its infancy and, despite an abundance of publications, there is insufficient data on which to base sweeping policy documents. There is evidence that both performance and affordability show a certain degree of contextual variability. Moreover, there is very little data comparing it to other available tests. There remains a need for strong evidence that this test genuinely affects patient-important outcomes when applied in clinical practice.

While we await further clarification, it seems reckless to push this expensive high-tech equipment onto existing TB control programs that strive to deal with high burdens of patients and disease using existing human and technological resources. Existing evidence should be independently reviewed in a thorough and controlled manner and the technique properly evaluated before we can say with certainty that this test is superior to all other tests that are available now or in the near future.