The Risk of Falsely Declaring Noninferiority of Novel Latent Tuberculosis Treatment in Large Trials

Addressing the global burden of latent tuberculosis infection (LTBI) is critical to eliminate TB and will require a much-improved diagnostic test, a much shorter treatment, or both. It is an exciting time for research to shorten LTBI treatment, with ongoing and recently completed studies holding promise of ultrashort, safer, more sterilizing regimens that would be easier to deliver to large populations (1–3). As in trials involving other diseases for which there is an existing effective treatment, developers of LTBI trials often opt for a noninferiority design to minimize sample sizes and costs. A particular challenge for investigators in these trials is deciding which subjects to enroll in the absence of a robust “gold-standard” diagnostic test for LTBI. In a modeling analysis presented in this issue of the Journal, Stout and colleagues (pp. 598–605) examined the factors that would lead to a false-positive outcome in a noninferiority trial comparing new versus established treatments for LTBI in the absence of a perfect test (4). After performing sensitivity analyses of key assumptions, the authors concluded that their model findings were valid under certain alternate scenarios.

The authors examined the impact of LTBI prevalence, the sensitivity and specificity of currently available proxy tests for LTBI, and the choice of noninferiority margins and other parameters on the design and interpretation of noninferiority trials. There is much debate about what constitutes “true” latent infection. A particular concern in noninferiority trials relates to the specificity of a test for LTBI and the prevalence of true LTBI in the study population.

A low prevalence of LTBI would mean that many individuals in the trial are not infected, which increases the risk of falsely declaring noninferiority. This modeling analysis suggests that without testing for LTBI, that risk is substantial when the LTBI prevalence is below 45%. When LTBI prevalence is less than 45%, it is still better to “enrich” the trial population for LTBI by enrollment based on LTBI tests. However, as the low specificity of the LTBI test would again result in low prevalence, more specific tests, such as IFN-γ release assays (IGRAs), should be used. Indeed, more broadly, the authors conclude that noninferiority trials evaluating regimens for treating LTBI should enroll participants based on IGRAs rather than on the PPD tuberculin skin test (TST), to decrease the risk of misclassifying ineffective regimens as noninferior. The conclusion that IGRAs are better than TSTs was predetermined by the assumption that the TST has a considerably lower specificity (59–70% for TSTs vs. 93–99% for IGRAs) in “standard practice” than IGRAs. These specificity estimates are derived from studies that used a TST cutoff of 5 mm for children and HIV-infected individuals in accordance with CDC recommendations—a sensible public health decision to ensure that few cases of LTBI are missed by maximizing sensitivity at the expense of specificity (5). However, trials do not have to use standard TST cutoffs. Previous studies suggested that it is possible to achieve very high specificity levels with the TST using higher thresholds (6). Indeed, at cutoffs of ≥15 mm, the TST’s ability to predict subsequent TB is similar to that of IGRAs (7–9).

The authors also used their model to examine the impact of the enrollment strategy used in the BRIEF-TB (Brief Rifapentine-Isoniazid Evaluation for TB Prevention) trial, an important study that is shifting the paradigm of LTBI treatment by demonstrating the noninferiority of a 1-month rifapentine plus isoniazid regimen compared with 9 months of isoniazid alone (3). BRIEF-TB enrolled a high-risk population of individuals with HIV (median CD4 cell count, 470 cells/mm³) who resided in high-burden settings and had been prescribed antiretroviral therapy, including many whose LTBI tests were unreactive (i.e., “negative”) at baseline. Based on Stout and colleagues’ model, inclusion of individuals with that high a proportion of negative LTBI tests would essentially reduce the proportion of subjects with true LTBI in the enrolled population, resulting in fewer subjects with incident TB outcomes, thus increasing the risk of falsely declaring noninferiority. A false-positive noninferiority result would be unhelpful because it may result in assuming, at the very least, the wrong magnitude of effect or, at the worst, the use of an ineffective regimen. There may, however, be other issues that are not sufficiently addressed in the model. The model does not consider the impact of antiretroviral therapy–mediated immune reconstitution; an initial negative LTBI result or presumed estimate of burden may be a poor indicator of who will progress to disease or benefit from TB-preventive therapy (10). Another potential limitation is that in settings with intensive exposure, and with the recognized high probability of progression soon after infection, some of the observed effects of LTBI treatment will be due to a reduction of new infections during the trial. The model places a greater weight on reactivation than on reinfection as a driver of active disease in high-burden areas, and decreases the impact treatment has on progression of incident infections (11, 12). The impact of new infections would be most pronounced in very-high-burden settings, although the absolute effect of this is uncertain and likely modest. The ultrashort regimen does have important advantages, including greater treatment completion rates, which at a population level may justify the use of a less effective regimen, for example, by allowing the use of a larger noninferiority margin in future trials. It is important to conduct further trials of this and other short regimens, particularly in HIV-uninfected individuals and child contacts, where assumptions regarding the performance of LTBI tests and prevalence of LTBI are different.

Other important issues should be considered in the design of LTBI treatment noninferiority trials, such as the power of the study and the risk of falsely failing to demonstrate noninferiority. Small trials that are underpowered may show an intervention arm to be inferior to the standard, resulting in a missed opportunity. The true difference between interventions is often unknown at the time a trial is set up. Although Stout and colleagues used a pragmatic 30% difference, a larger or smaller true difference between the two regimens may be possible. A sensitivity analysis to examine the effect of different scenarios would have been informative for future trials. It is likely that when the true difference is small, it would be even more pertinent to conduct the trial in a high-prevalence setting and enroll participants using the most specific assay available.

The authors also point out a broader issue: efficacy versus effectiveness. Whereas efficacy trials measure the effect of treatment under optimized and controlled conditions, effectiveness trials measure this effect in the real world. Trials investigating approaches that have limited specificity for LTBI but are easily applied are valuable because they show how much TB disease can be prevented in real-world practice. One must be aware, however, that such trials do not establish the efficacy of a new regimen, and that a noninferiority design may provide misleading results. This raises the question as to whether noninferiority designs are at all suitable for effectiveness trials, which often apply imperfect methods for selecting patients or for implementing the intervention.

Stout and colleagues have addressed a key issue in conducting noninferiority trials for LTBI treatment, and their work has implications for future studies. The authors’ conclusions, which raise valid concerns, should not lead us to “throw the baby out with the bath water.” In the future, we need to develop multiple, more robust trial designs that include further evaluation of the ultrashort regimen or an even more ambitious program to demonstrate superiority when the risk/benefit ratio is less clear, rather than take a step back because of a hypothetical concern that the result of this elegant trial may be a false positive.