Customizing Treatment to the Patient: Adaptive Treatment Strategies

Much research has documented both the wide heterogeneity in patient response in the treatment of drug dependence (Crits-Christoph et al., 1999) and the wide variety of available treatment options. Because of patient heterogeneity clinicians often must adapt the treatment to the patient to obtain an acute response. In practice adaptation is often achieved by implementing treatments (or variations of one treatment) sequentially in a trial and error fashion until a response is obtained. Even once an acute response has been achieved, there remains a high risk of relapse both during and following treatment (McLellan et al., 2000; McLellan, 2002; McKay et al., 2004), requiring trying more treatment alternatives. Taken together, these realities imply that in both the acute and long term treatment of drug dependence, clinicians must continually adapt and readapt the treatment to the patient.

The confluence of the above disorder characteristics with the availability of multiple treatments and multiple variations of each treatment motivates the construction of adaptive treatment strategies (Lavori and Dawson, 1998, 2004; Lavori, et al. 2000; Murphy and McKay, 2003; Collins, et al. 2004). These strategies individualize treatment via decision rules that recommend when and for whom the treatment should change; frequently they incorporate a sequence of treatments. The decision rules take as input patient characteristics and outcomes collected during treatment, such as patient risk factors, patient response, and patient adherence, and provide as output a treatment recommendation. The adaptive treatment strategy concept underlies a number of treatment approaches used in substance abuse and related fields; examples include the stepped care models in substance abuse (Sobel and Sobel, 2000; Brooner and Kidorf, 2002; McKay, 2005), structured treatment interruptions in AIDS (Lisziewicz and Lori, 2002) and the algorithms used in the treatment of depressive, bipolar and schizophrenicdisorders (Miller et al., 1999; Rush et al., 1999).

The purpose of this supplemental issue is to share with readers of Drug and Alcohol Dependence recent research on both the methodology and implementation of adaptive treatment strategies for treatment of drug abuse and related disorders. The articles in this issue by Marlow et al. and Brooner et al. provide evaluations of promising adaptive treatment strategies. In the intervention evaluated by Marlowe et al., level of judicial supervision is matched to the risk level of the drug offender. In addition to illustrating the promise of this particular adaptive intervention, Marlowe et al. raise an important issue for consideration. Even though a particular risk factor (e.g. young male) may correlate with treatment outcomes, this does not imply that this risk factor is useful for deciding which treatment is best for each individual. Using a risk factor to adapt the treatment to an individual requires that the risk factor interact with the treatments. This interaction must be sufficiently strong so that for some values of the risk factor one treatment produces better outcomes, whereas for other values of the risk factor a different treatment is better.

Brooner et al. compare two adaptive treatment strategies (in addition to a “treatment-as-usual” condition and a contingent voucher condition). Depending on the particular adaptive treatment condition, the intensity of the counseling and the use of both positive and negative behavioral contingencies is adapted and then readapted to the patient’s symptoms as measured by urine tests and adherence (at counseling sessions). A crucial characteristic of the interventions discussed by both Marlowe et al. and Brooner et al. is that the decision rules are explicit and operationalized. Both articles provide their decision rules. The intervention evaluated by Marlowe et al. uses a very simple one-time matching of judicial supervision level to offender risk level. In contrast, Brooner et al. use a sequence of decision rules to decide when and if an individual will move from one treatment step to the next. Each individual is provided a sequence of adapted variations in treatment, adapted mainly via the number and type of assigned counseling sessions.

Despite the activity in evaluating adaptive treatment strategies, the development of data collection and analytic methods that directly inform the construction of adaptive treatment strategies lags behind. Four articles in this supplemental issue address these topics. Murphy et al. discuss a promising clinical trial design: the sequential, multiple assignment, randomized trial (SMART). In SMART, subjects are randomized more than one time. A number of SMART trials have been, or are being, conducted. These include the CATIE trial for antipsychotic medications in patients with Alzheimer’s (Schneider et al., 2001), the STAR*D trial for depression treatment (Rush et al., 2003; Lavori et al, 2001) and trials in cancer (Stone, et al. 1995; Tummarello, et al. 1997). Currently there are a number of ongoing SMART studies including a trial concerning alcohol dependence by David Oslin at the University of Pennsylvania, a trial concerning attention deficit disorder by William Pelham, at the State University of New York, Buffalo and numerous trials in the design stage concerning cancer by Peter Thall at the MD Anderson Cancer Center.

The Rivera et al., Rosenberg et al. and Pineau et al. articles present data analysis methods for constructing adaptive treatment strategies. The methods discussed by Rivera et al. and Rosenberg et al. stem from the field of “control theory.” Not only is control theory is an active area of research in engineering, operations research and applied mathematics, as Rivera et al. pointed out it figures prominently in everyday life in useful inventions such as cruise control in automobiles and insulin pumps. For example, control theory is used to construct the decision rules that ensure that automobiles move at the desired speed accommodating the presence of hills and valleys and that patients are provided the correct amount of insulin tailored to the variation in the individual’s food intake throughout the day.

An important message, emphasized in the Rivera et al. article, is that methods in control theory are designed to transfer variance in outcomes to variance in treatment. Translated to clinical science, this means that these methods seek to reduce the variance in outcomes (e.g. the heterogeneity in response both across patients and within a patient across time) by introducing variance in treatment (e.g. tailoring the treatment to each patient and re-tailoring throughout the course of the disorder). Methods in control theory operationalize this crucial clinical insight and thereby seek to make adaptive interventions work better.

Another important message in both the Rivera et al. and Rosenberg et al. articles concerns the utility of computer simulations. The computer simulations depend on a mechanistic, dynamic model of how patient characteristics and outcomes to past treatment interact with present treatment to result in subsequent patient outcomes. These models incorporate both time-varying outcomes and treatments. Rosenberg et al. provide an overview of a dynamic model of the time-varying relationship between viral load, CD4 T cell count and treatment of HIV. Given a dynamic model one can, via simulation, create a population of virtual patients. A host of different adaptive treatment strategies can then be compared and contrasted using this population. This process results in one or more adaptive treatment strategy proposals that can be evaluated in clinical trials. Used effectively, the simulation enables comparison of many different adaptive treatment strategies without the commitment of real patients and the associated resources. Of course the better the dynamic model mimics the important aspects of the true interrelationship between treatment and patient outcomes, the better the adaptive treatment strategy proposals will be.

The Pineau et al. article uses methods from the field of “reinforcement learning” in computer science to construct adaptive treatment strategies from data. Reinforcement learning can be viewed as an extension of control theory to settings in which either mechanistic models are extremely complex or there is insufficient information to form a mechanistic model (recall the mechanistic model expresses how patient characteristics and outcomes to past treatment interact with present treatment to result in subsequent patient outcomes). These methods are often used in the field of artificial intelligence and in the development of games. In these settings greater reliance is placed on experimental, randomized, studies. The Pineau et al. article illustrates the use of a reinforcement learning method with the STAR*D data obtained from patients with major depressive disorder.

An important message communicated in the Pineau et al. paper is that methods from reinforcement learning are designed to evaluate not only the immediate effects of a treatment but also how this treatment increases the efficacy of prior and subsequent treatments. In other words, when considered as part of a sequence of treatments, a particular treatment is most valuable if it builds upon the successes of the prior treatments and sets the patient up for further and continued success with the subsequent treatments. An additional and easily overlooked effect is the diagnostic effect of a treatment. The diagnostic effect of a treatment is the treatment’s ability to elicit informative patient responses that permit the clinician to better match the subsequent treatment to the patient. Diagnostic effects are unimportant if one is considering a stand-alone treatment, however, in the context of an adaptive treatment strategy diagnostic effects of a treatment may become extremely important. All of these effects are incorporated into construction of adaptive treatment strategies by the reinforcement learning methods.

Trivedi et al. consider the crucial next step in the science of customizing treatment to the individual. Given an efficacious adaptive treatment strategy, how can one ensure that the strategy is used with fidelity in clinical practice? Trivedi et al. discuss the ideas behind the use of computerized decision support systems in improving fidelity. The computerized decision support systems proposed by Trivedi et al. are used during the clinic visit and thus provide critical information at the time treatment decisions are made. Furthermore Trivedi et al. emphasize a crucial ingredient of adaptive treatment strategies, that is, the use of measurement based care. In measurement based care, specific assessments of individual characteristics, symptoms, side effects, functionality, etc. rather than unstructured global judgments are used in the decision rules underlying the adaptive treatment strategy.

Conclusions

The six articles in this supplemental issue illustrate the benefits of adaptive treatment strategies, provide multiple intriguing approaches to constructing adaptive treatment strategies using clinical data and introduce a promising approach for ensuring that evidence-based adaptive treatment strategies can be used with fidelity in clinical practice. As these articles and others show, adaptive treatment strategies have the potential simultaneously to conserve resources and improve intervention efficacy and effectiveness by targeting treatments when and for whom they will do the most good. They deserve continued research attention and a prominent role in drug abuse treatment and related areas.