Reanalysis of Methamphetamine Dependence Treatment Trial

Researchers working in the field of clinical trials for addictive disorders have discussed whether the use of responder analyses (analyses which compare the proportion of patients in each treatment arm who achieve the desired response) in these trials represents “setting the bar too high.” These discussions involve assumptions about the relative ease or difficulty of establishing a treatment effect using group means versus doing so using responder analyses. In the paper by McCann DJ and Li SH [1], the authors have shown that using a responder analysis identified a treatment effect in methamphetamine dependence, which an initial analysis of group means did not. This demonstrates that responder analysis may be a more appropriate approach, and depending on the study design, may demonstrate differences not appreciated by group means. Other authors have also identified situations in which a responder approach demonstrates an effect where group means did not [2], or where a comparison of means yielded equivocal results of uncertain clinical significance [3]. In addition, Falk et al. [4] recently compared the use of a responder analysis to customarily used group mean comparisons in alcoholism trials and found that their responder definition was as sensitive as customary measures in detecting treatment effects.

Addictions are behavioral disorders, characterized by compulsive self‐administration of substances despite physical and psychosocial consequences. Although considerable attention is given in addiction trials to biological measures of drug use as endpoints, cocainuria or methamphetaminuria is only a marker and not the disorder under treatment. The aim of treatment is often expressed as an effort to modifying patients’ drug use behavior, but the desired effect is improvement in physical and psychosocial consequences. Changing the behavior only minimally, without having impact on the consequences, would be pointless. Drug‐taking behavior observed during the brief window of a clinical trial is a surrogate endpoint, as trials intended to show effects on physical or psychosocial consequences of drug use would need to be very long and very large, and may be impractical. When drug‐taking behavior is used as a surrogate endpoint, there should be a demonstration of change in behavior that can be reasonably predictive of improvement, such as avoidance of alcohol‐related health and social consequences.

A common approach to clinical trials in addictive disorders, particularly trials evaluating treatments of stimulant dependence, is to compare the overall drug use between the treatment arms. For example, through some combination of self‐report and biological assays, each patient's amount of drug use is quantified in some way (e.g., days of use, numbers of negative urine tests). These totals are then aggregated for the entire treatment group, and a mean is calculated (such as mean percent of drug tests that do not show drug use) and compared between groups. However, this type of measure is extremely difficult to interpret in regard to clinical relevance as defined earlier. A statistically significant difference between the mean percent “clean” urine tests could be driven by any number of phenomena. Perhaps many of the patients in one group reduced their drug use minimally, whereas fewer did so in the other group. This would be an example where it is doubtful that any improvement in health or social consequences has been achieved. Perhaps there were differential rates of missing data and the finding is spurious. With these thoughts in mind, it can be difficult if not impossible to establish what difference between group means would represent a “clinically significant” result. This is because it cannot be determined whether anyone, in either group, accomplished a change in drug‐taking behavior significant enough to represent meaningful improvement in psychosocial function and physical consequences. A difference in group means could, of course, be driven by a few people in one group reducing their drug use substantially, or stopping using drugs altogether, whereas a smaller number in the other group did so. This would be a favorable result–but the comparison of means cannot distinguish between this possibility and those of lesser clinical significance.

In addition, comparisons of group means do not provide direct information about treatment response in individual patients. They don't help clinicians understand what kind of response their patients might experience, or allow patients or clinicians to readily compare the risks of the treatment, which are typically presented in terms of the proportion of patients who experience a given adverse event, to the potential benefits.

For these reasons, a responder analysis is an approach that illustrates the clinically important effect of a treatment. The choice of a responder definition for analysis is a challenging problem. A responder analysis has to include a definition that predicts clinical benefit. One pattern of use that is generally accepted to be associated with clinical benefit is complete abstinence. In the alcoholism field, the duration of abstinence considered to represent a stable condition, or sustained remission, is often set at 12 months [5]. Once well established, abstinence from alcohol appears, for many patients, to be a stable pattern, sustained over several years of follow up [6]. Validation of other patterns of behavior as surrogates for clinical benefit can be accomplished by examination of data on long‐term functioning of treated individuals comparing use patterns with outcomes. Falk et al. [4] based their choice of responder definition in alcoholism trials on analyses of this type. It would seem appropriate for researchers interested in developing treatments for stimulant dependence to pursue this type of analysis, using data from clinical trials, longitudinal observational studies, health care utilization databases, or other sources of information which can shed light on what duration of drug abstinence predicts ongoing abstinence and/or good psychosocial and physical functioning, or what patterns of stimulant abuse short of abstinence are consistent with good functioning.

A responder analysis approach can incorporate a grace period (a period of time during which patients who use drugs are not considered nonresponders). This allows time for the drug to be titrated to an effective level and for patients to become engaged in treatment. It can incorporate response definitions other than abstinence, if there is a level of use that can be considered nonharmful (as in the case of alcohol use [7]). It can, if appropriate, allow for “slips” that are not full relapses. Analyses can also compare the proportion of partial responders as well as full responders, to address the common concern that some patients who did not meet the responder definition may have nevertheless achieved a clinical response that some would find acceptable.

Addictions are chronic disorders and addiction treatment drugs are maintenance treatments. The short‐term effect observed during the trial should be predictive of an ongoing effect, assuming the medication is continued. Current development trials for chronically administered drugs for chronic conditions are often 12 weeks (after any grace period, titration, etc.), but may be longer if the effect cannot be observed in that time period. In the case of alcoholism treatments, there is data indicating that drinking patterns over 3 months may not be stable or representative of future experience [8]. Because of this, 12‐week alcoholism treatment trials may be too brief to predict ongoing response. It is also known that periods of abstinence are quite common among alcohol dependent individuals: periods of abstinence lasting at least 3 months were reported by 62.3%[9]. This could make it hard to show a treatment effect in a brief alcoholism treatment trial. In addition, abstinence at 6 months has been shown to be a predictor of abstinence at 5‐year follow‐up [10]. For this reason, it is not unreasonable to ask for 6‐month trials for alcoholism treatment trials. Lacking any comparable information arguing for a shorter period, 6 months seems reasonable for other trials of treatments of other addictive disorders as well. A grace period of several weeks incorporated into the 6‐month total duration of the trial may also be a reasonable approach to address the concerns noted above.

Most trials include a period of posttreatment follow‐up. If a drug were intended to be used chronically, and to be effective as long as it is used, one would not necessarily expect that efficacy would be maintained after the drug is discontinued. Therefore, the on‐treatment observation period needs to be long enough to show an effect that is clinically meaningful. If the treatment period is very short, or the drug is expected to be used briefly but affect the patient over a longer period of time (a “cure”), it could be necessary to include posttreatment observations in the analysis. Shorter trials may be appropriate if there are data showing that the relatively brief period of response in a shorter trial was predictive of ongoing response.

Although a responder analysis can be a way of showing clinical benefit, a trial involving 10 weeks of grace and 2 weeks of efficacy ascertainment, as described in the McCann et al. paper, would be unusual. Data supporting health benefits of 2 weeks of abstinence, or of data supporting 2 weeks of abstinence as a predictor of ongoing abstinence, would need to be generated to use this responder definition as an endpoint.