Primary outcome indices in illicit drug dependence treatment research: systematic approach to selection and measurement of drug use end-points in clinical trials

Abstract

Aims

Clinical trials test the safety and efficacy of behavioral and pharmacological interventions in drug-dependent individuals. However, there is no consensus about the most appropriate outcome(s) to consider in determining treatment efficacy or on the most appropriate methods for assessing selected outcome(s). We summarize the discussion and recommendations of treatment and research experts, convened by the US National Institute on Drug Abuse, to select appropriate primary outcomes for drug dependence treatment clinical trials, and in particular the feasibility of selecting a common outcome to be included in all or most trials.

Methods

A brief history of outcomes employed in prior drug dependence treatment research, incorporating perspectives from tobacco and alcohol research, is included. The relative merits and limitations of focusing on drug-taking behavior, as measured by self-report and qualitative or quantitative biological markers, are evaluated.

Results

Drug-taking behavior, measured ideally by a combination of self-report and biological indicators, is seen as the most appropriate proximal primary outcome in drug dependence treatment clinical trials.

Conclusions

We conclude that the most appropriate outcome will vary as a function of salient variables inherent in the clinical trial, such as the type of intervention, its target, treatment goals (e.g. abstinence or reduction of use) and the perspective being taken (e.g. researcher, clinical program, patient, society). It is recommended that a decision process, based on such trial variables, be developed to guide the selection of primary and secondary outcomes as well as the methods to assess them.

INTRODUCTION

Despite the development of evidence-based behavioral and pharmacological interventions, the use of and dependence upon illicit drugs and prescription medications continue to be major public health concerns. Clinical trials contributed the empirical basis to support these interventions [1]; however, there is a lack of consensus among clinical researchers and practitioners about what outcomes are most important and how best to define the efficacy of drug dependence treatment. Further, given the interdependence between outcomes and methods of assessment, a related question also still lacking consensus is what measures or methods best assess those outcomes. Drug use is measured variably, including self-report, collateral report and qualitative or quantitative assessment of a variety of biological markers, with different measures potentially yielding different results, thus making cross-study comparisons difficult [2,3]. This is the case not only in the United States but also internationally [4,5]. While drug-taking behavior is the cardinal feature of drug dependence and the primary target for interventions, it is not the only outcome of interest, with other important biopsychosocial dimensions to consider [6–11]. This dilemma is long-standing, as exemplified in the seminal paper by Dole & Nyswander [12] on methadone as a treatment for opiate dependence. They pointed to methadone’s positive benefits of relief from ‘drug hunger’ and blockade of heroin’s euphoria, yielding an overall reduction in opiate use, but also noted improved functioning in educational, employment and familial arenas. This is consistent with the three key outcome domains measured in virtually every published evaluation of addiction treatment since the 1960s, namely substance use, employment/self-support and criminal activity [8].

An important factor contributing to this lack of consensus is the differing perspectives of the investigator, practitioner, drug user or policymaker. Researchers may be interested primarily in the impact of interventions on drug use, regardless of how measured. Clinicians and program administrators are typically interested in a broader array of outcomes, defined as clinically meaningful change [13]. From the client’s perspective, the experience of reduced symptom severity and improved quality of life are important [14,15]. Policymakers, third-party payers and society more generally expect treatment to yield improvements in multiple areas of psychosocial function, increased public safety and decreased financial burden [8].

A second factor in determining drug treatment outcomes is whether an intervention focuses on a single or multiple drugs, reflecting the purported specificity versus generalizability of treatment effects [16]. The stated intervention goal with respect to substance use, namely abstinence or reduced use, harm and/or consequences, also has an impact on how a treatment is appraised [17,18]. The European College of Neuropsychopharmacology’s consensus panel on efficacy of interventions [4] distinguishes between trials that focus on either full recovery (‘cure’) versus drug use stabilization and harm minimization (‘care’) as the main treatment goal. Type of intervention, behavioral or pharmacological, may also influence the potential therapeutic target and the associated outcome [19]. This latter factor also may influence outcome assessment timing. Some interventions, directed at the immediate effects on drug use, have a relatively short time-frame, while others focusing on improved psychosocial dimensions of recovery usually have a much longer perspective. Differences in outcome assessment timing result in different periods of relapse risk and resumed use [3], suggesting different trial efficacies.

Continued research focusing on determining appropriate outcomes and the standardization of measurement across drug dependence treatment clinical trials has been recommended [2]. This includes further examination of the properties and benefits of alternative and potentially complementary substance use/drug-taking, medical and psychosocial outcomes; different approaches to their measurement; and methods of evaluating the timing and sequencing of assessments consistent with the natural course of recovery and relapse processes. Several previous expert panels, both in the United States [9,20,21] and Europe [4], were unable to reach consensus on these issues.

This paper summarizes the recommendations of a panel of substance abuse treatment and research experts (see the list of participants in Appendix I), convened by the US National Institute on Drug Abuse (NIDA) in December 2009 to select an appropriate primary outcome measure for drug dependence treatment clinical trials, and in particular to consider the feasibility of selecting a common outcome measure to be included in all or most drug dependence treatment trials. First we provide a brief overview of perspectives concerning outcomes used in treatment trials with tobacco and alcohol; we next summarize the state of the science regarding toxicology and self-report measures of drug use; and finally we summarize the discussion and recommendations from the panel’s deliberation.

PERSPECTIVES ON PRIMARY OUTCOMES FROM TRIALS WITH LEGAL DRUGS: TOBACCO AND ALCOHOL

Outcome data for interventions targeting the two most prevalent legal substances of abuse, tobacco and alcohol, were considered.

Tobacco experience

Work-groups from the Society for Research on Nicotine and Tobacco and others [22–24] have published relevant recommendations on measuring craving and withdrawal [25], self-report criteria for abstinence in clinical trials [26] and biochemical verification of abstinence [27].

Nicotine dependence differs from other drug dependencies in that it does not cause urgent psychosocial problems, yet even low levels of use are harmful [28]. Therefore, neither temporary abstinence nor drug use reduction is an acceptable goal; the focus is on long-term abstinence as the primary treatment outcome. Most trials enter current smokers, establish quit dates and tie outcome assessments to that date. Three processes are measured typically: abstinence initiation within 24 hours, onset of a smoking lapse and transition from lapse to relapse (return to regular smoking), as interventions have different effects on each of these outcomes [29]. The major goal of abstinence is measured typically in one of four ways [26,30]. Continuous abstinence (CA) is abstinence that begins on the quit date and continues till the last assessment. Prolonged abstinence (PA) is continuous abstinence with a short (usually 2 weeks) grace period after the quit date. Point prevalence abstinence (PPA) is a short abstinence (usually 7 days) period immediately preceding an assessment time-point. Repeated point prevalence abstinence (RPPA) is PPA at repeated follow-ups. The major advantages of CA, PA and RPPA are requiring longer abstinence and better prediction of the ultimate goal—life-long abstinence or health benefits. PA, and especially CA, are more closely tied to and influenced by treatment. Furthermore, PPA and PA count the common occurrence of some initial lapses with later complete abstinence as successes, and they can capture this effect even if a treatment has a delayed effect (e.g. a blocking agent). Finally, only PPA can be verified biochemically [27]. These four measures are correlated highly, with one no more valid than the other [31,32].

Falsification of abstinence self-reports is substantial in trials with face-to-face treatments, but minimal in less intensive treatments. Biochemical verification provides additional assurance that the participant’s self-reports are accurate [27,30]. Self-reports of amount of smoking (e.g. number of cigarettes smoked) collected via the time-line follow-back (TLFB) have poor validity [33], due in part to digit bias [34]. Observer verification also appears to add little [35].

Alcohol experience

Unlike the situation described for smoking cessation, alcohol consumption within certain limits is not viewed as harmful, and in some cases moderate alcohol intake may be health-enhancing. The National Institute on Alcohol Abuse and Alcoholism (NIAAA) published guidelines and limits/thresholds that specify low-, moderate- and high-risk drinking based on number of standard drinks consumed per day and per week by men and women [36,37]. While guidelines such as these and those of the British National Health Service [38] and the World Health Organization [39] are helpful clinically, their utility as outcome measures is complicated by differing sizes of beverage containers, varying alcohol concentrations and individuals’ difficulties in judging the size of their drinks accurately [40–43]. This has led to a strong recommendation that a common method for reporting alcohol consumption be adopted internationally [43].

The degree to which an individual’s drinking is considered harmful, based on exceeding thresholds and associated negative consequences, and whether criteria for an alcohol use disorder are met, influence intervention goals. These may range from reduced consumption and harm-reduction for high-risk drinkers or alcohol abuse (e.g. college binge-drinking) to total abstinence for alcohol-dependent individuals [17,18].

Given this range of interventions and goals, there is a lack of consensus about best outcome(s) for clinical trials [44]. Besides abstinence, still important in alcoholism treatment trials, there is an increased array of outcomes, including continuous measures such as percentage of days abstinent, drinks per drinking day, number of heavy drinking days, presence of alcohol-related problems and dependence symptoms and biological markers of drinking or heavy drinking to verify self-reports, or integrated with self-reports for a combined drinking index [45–47].

Some trials employed empirically based composite indicators that integrate aspects of both alcohol consumption and alcohol-related problems, allowing classification of outcomes as abstinent, moderate drinking without problems, heavy drinking or problems or heavy drinking and problems [48]. Such an approach accommodates both abstinence-oriented and harm-reduction interventions. Outcome selection may vary based on the intervention’s mechanism of action [49]. A NIAAA expert panel selected days of heavy drinking (at least four or five drinks per drinking day for women and men, respectively) as the ‘optimal’ outcome for clinical trials, as assessed by drinking estimation methods such as the TLFB procedure [50,51]. More recently, the US Food and Drug Administration (FDA) promoted percentage of subjects with no heavy drinking days as an end-point in pharmacotherapy trials for alcohol dependence; individuals who are abstinent or who engage in low-risk drinking are considered successful treatment responders [45].

While there is a pronounced difference in the relative weight given to biological indicators and self-report of use in tobacco and alcohol research, substance-taking behavior is the primary outcome measure for tobacco and alcohol treatment trials. Researchers acknowledge the importance of psychosocial and physical consequences of smoking and drinking and endorse potential outcomes in these domains, but the primary and more proximal treatment focus is to reduce or eliminate substance use [45]. Issues concerning methods providing the most valid indicator of use and endorsement of substance-taking behavior as the defining outcome in clinical trials apply equally well to drug dependence treatment.

BIOLOGICAL MONITORING OF ILLICIT DRUG USE AS A PRIMARY OUTCOME

Drug exposure detection has traditionally utilized urine testing; alternative matrices including oral fluid, sweat and hair are now available. Advantages of urine testing include adequate specimen volume, on-site and laboratory-based test availability, relatively high drug concentrations, proven accuracy and reliability, easy automation, low-cost, well-established quality control programs and a large scientific database for interpretation. Disadvantages include the need for private collection facilities and same-sex collectors for observed collections, short detection windows and ease of test adulteration (addition of chemicals, or even high fluid consumption, reduce sensitivity). There are multiple advantages for alternative matrix testing, as each offers unique information about the participants’ drug use. Specimen collection may be less invasive and less subject to adulteration, may improve drug stability, may provide opportunities for multiple samples and may lower the risk of disease during handling and analysis. Furthermore, alternative matrices offer a choice in drug detection windows, frequently measure parent drug concentrations that may correlate more effectively with drug effects and permit easier shipment and storage. If parent drug and metabolites are present, test interpretation is frequently improved. There also are disadvantages associated with each matrix, including cost, turnaround time, adsorption of drug to collection devices, extensive specimen preparation and limited controlled drug administration data to aid in the interpretation of test results.

Urine tests reflect drug use ranging from 12 hours for alcohol [52] to typically 2–4 days for other drugs of abuse [53–56]. An important exception is cannabis; chronic daily smoking may result in positive tests for an extended time, creating a situation where it is difficult to differentiate new drug use from residual drug excretion [57]. If urine cannabinoid concentrations are normalized to urine creatinine concentrations, noting the time between urine specimen collections, predictive models can be employed to estimate new cannabis use [57,58]. A recently published model differentiates for the first time new cannabis exposure from residual drug excretion in chronic, daily cannabis smokers, taking into consideration cannabinoid concentrations at the time of treatment admission and variable times between urine collections [59]. Models have also been developed to identify new cocaine exposure [60]. Urine monitoring usually requires a minimum of twice- or thrice-weekly collection to identify new drug use adequately. Alcohol’s detection window in urine may be extended to 5–7 days by quantifying the non-oxidative metabolites ethyl glucuronide and ethyl sulfate [52], although caution in test interpretation is encouraged due to positive results with unintended alcohol exposure [61].

Drug testing is available for three additional matrices, oral fluid (or saliva), sweat and hair, although additional research is needed to document the utility of each of these as an outcome in clinical trials. Each of these has advantages and disadvantages relative to urine testing and to each other. The choice of which biological measure to employ should be determined primarily by the desired time-frame of detection.

Oral fluid testing is increasing rapidly due to interest in drug treatment, work-place drug testing and drugged driving programs. Presence of drug in oral fluid documents drug exposure, particularly recent drug use, and also may correlate more effectively with blood concentrations. A short detection time may be useful if recent drug use is the focus of monitoring, but could require more frequent testing to ensure sustained abstinence. On-site and laboratory-based screening and confirmation tests for oral fluid are becoming more available and assay costs are reasonable due to competition; however, on-site tests to date have not achieved adequate sensitivity and specificity. Unfortunately, many manufacturers have simply tried to modify urine assays for the new matrix. Drug metabolites are found primarily in urine, while oral fluid generally contains both parent drug and metabolites. Drug concentrations may also be much lower in oral fluid than in urine. In many cases, assays have not been modified adequately to target different analyte profiles and lower cut-offs.

On-site oral fluid tests are available for many drug classes and are being developed and improved for many other drugs. The advantages of these tests are that oral fluid is collected non-invasively and under direct observation without the need for specialized facilities. It is preferable to utilize collection devices without chemicals to stimulate saliva flow, because stimulation increases oral fluid pH, changing the distribution of drug between plasma and saliva. Stimulation also increases the amount of fluid, diluting drug concentrations and reducing sensitivity. Many drugs, including cocaine and sympathomimetic amines, reduce saliva production, making specimen collection more difficult. Testing for Δ9-tetrahydrocannabinol (THC) can also be problematic due to passive contamination and contamination of the oral mucosa during smoking, not through diffusion of drug from the blood, and because the highly lipophilic THC may be tightly adsorbed to the specimen collection device, making it difficult to elute and greatly reducing test sensitivity. Oral fluid monitoring requires a minimum of twice- or thrice-weekly collection to adequately identify new drug use. Additional research is needed to exclude the possibility of passive contamination of oral fluid from smoked and/or oral drugs.

Sweat testing is a more recent and less investigated method of monitoring drug use. The primary advantage is that the sweat patch is worn for 1 week, and accumulates drug excreted throughout the week. Drugs in sweat can also reflect use as much as 24–48 hours prior to patch application. Both parent drug and metabolites are excreted. Thrice-weekly urine and weekly sweat specimens from participants in a methadone maintenance treatment program were tested for opiates and cocaine [55,62]. Weekly sweat testing was equivalent or better than thrice-weekly urine tests in identifying cocaine and opiate drug use. Another advantage of sweat testing is that it decreases the opportunity for adulteration. Each sweat patch has a unique identification number and will not adhere to the skin if removed; punctures are also readily visible. Limitations to sweat testing include intra- and inter-subject variability in sweat production, low analyte concentrations, occasional skin sensitivity, failure of the patch to adhere to the skin and possible contamination of the patch from the environment and during specimen handling. Some portion of the drug also may be reabsorbed into the skin, degraded on the patch and/or escape through the semi-permeable membrane. Sweat patch testing identifies drug use but is not quantitative. As is the case for oral fluid, this new technology has few scientific data from controlled drug administration studies to guide the interpretation of test results. There does not appear to be a dose–concentration relationship and the question of residual excretion of drug in heavy chronic users long after last use has not been resolved fully [63]. In addition, only a single laboratory in the United States is offering routine testing of the sweat patch and the cost has been estimated to be 1.7 times more than for urine testing for cocaine [64].

Hair is another alternative matrix that may be helpful in evaluating drug use. The primary advantage of hair testing is the long window of drug detection, although that is dependent upon hair length. For example, several studies found that hair analysis identified more cocaine use than urine tests [65]. Three centimeters of hair can be collected every 3 months to detect drug use efficiently over this time-frame. Although the cost of hair testing is much higher than urine testing, the number of specimens required, visits for hair collection and staff time to collect specimens is greatly reduced. Other advantages are the stability of drug analytes in hair at room temperature and the resultant ease of storage, handling and shipping of specimens. If a repeat specimen is required, a new specimen that reflects the original time of sampling is easy to collect. An individual cannot abstain from drug use for a short period of time prior to hair collection and avoid detection, as can be the case with urine, oral fluid and sweat. Further, adulteration of hair by bleaching, dyeing or straightening is easily apparent.

A limitation of hair testing is the differential incorporation of basic drugs into hair according to its melanin content [66,67]. Darker hair contains more melanin than lighter-colored hair and will most probably contain greater concentrations of basic drugs, such as cocaine or methamphetamine, if exposed to the same amount of drug. This complicates hair test interpretation. In the future, we may find that normalization of basic drug concentrations to melanin concentrations in hair will reduce this apparent discrepancy [68]. Neutral and acidic drugs, i.e. THC, do not appear to bind preferentially to melanin and may have less variable disposition into hair of different colors and with different melanin content. Hair testing is a sensitive technology to detect basic drugs, such as cocaine, but lacks sensitivity to detect cannabis use compared to urine tests. Ethyl glucuronide, unlike alcohol, is incorporated into hair, suggesting that hair could be a valuable matrix for alcohol testing [69].

A second major limitation of hair testing is potential contamination by drugs in the environment [70,71]. Whether it is possible to differentiate contamination from actual drug use remains a controversial subject. Furthermore, highly sensitive tandem mass spectrometry methods are frequently required. Other limitations of hair testing are that recent drug use over the past 10 days may not be detected, there may be a high refusal rate for hair sampling and frequently too little specimen is collected.

SELF-REPORT OF ILLICIT DRUG USE AS A PRIMARY OUTCOME

Monitoring drug use via self-report can be flexible and provide a range of data and outcome measures that are sensitive to changes in patterns or intensity of substance use. Self-reports are not invasive and can be collected remotely (i.e. via interviews, over the telephone, through direct entry on questionnaires or computer screens or over the internet) and in a wide variety of formats. Finally, self-reports may be collected retrospectively over comparatively long periods of time and minimize missing data (as participants who miss assessment visits can provide data later covering the missing time-periods).

The accuracy of self-reported substance use in clinical trials remains highly controversial, with some studies pointing to impressive reliability, validity, sensitivity and consistency with other indicators [72–77] and others suggesting poor accuracy and substantial under-reporting with respect to biological measures [64,78–81]. A key consideration is that the reliability and validity of substance users’ self-reports are not fixed properties of the reports themselves or the data collection instruments; rather, these vary with sample and the method and context of collection.

Self-reports can be accurate given appropriate context and investigators’ use of methods to enhance accuracy. Factors and procedures to enhance accuracy of self-report among drug users in clinical settings and in clinical trials include assurance of confidentiality and absence of adverse consequences, use of appropriate recall cues and ‘bogus pipeline’ techniques (e.g. participants are convinced that their self-reported values will be verified by biomarkers or other types of measures), clarifying how data are used, collection from multiple sources (including biological indicators and collateral information) and standardized, consistent and clear instructions and procedures [82,83]. Factors reducing accuracy include significant consequences (positive or negative) of reporting substance use, lack of confidentiality or collection of self-report data with individuals who are cognitively impaired, have significant psychiatric comorbidity or are under the influence of drugs or alcohol, and collection in a non-clinical or non-research context [84]. Extensive reliability and validity data have been collected on the TLFB technique, a calendar-based interview collecting day-by-day use information over the past 60 days [50,72,85]. Variants of the TLFB approach with substantial psychometric support include the Form 90 [86–90] and the Substance Use Calendar [91] that incorporates a strategy for comparing self-report to urine results.

INTEGRATING SELF-REPORT AND TOXICOLOGY

Self-report methods and toxicology methods for assessing drug-taking behavior each have both strengths and weaknesses. Neither is a direct measure, but each is an indicator of drug-taking behavior. The strengths and weaknesses of each approach are summarized in Table 1. Although monitoring of drug use through urinalysis and other biological indicators is an important strategy of assessing recent drug use and monitoring treatment response in clinical trials, evaluation of treatment outcome is complex, and there are multiple considerations that often lead investigators to use and rely primarily on self-reports in treatment outcome research. These include greater flexibility and range of data and outcome measures than can be obtained via biological indicators which, as described earlier, tend to be limited to detection of relatively recent substance use, and provide limited information regarding anything other than point-prevalence abstinence unless they are collected frequently. Qualitative biological measures are usually insensitive to issues such as changes in patterns or intensity of substance use, as well as detection of significant reductions in drug use over time. Secondly, biological indicators carry with them costs of collection and assays, while collection of self-report data is comparatively less expensive [82]. Collection of biological samples is generally invasive and requires in-person contact with the participant, unlike self-report. Finally, as noted above, self-reports may be collected retrospectively over comparatively long periods of time and minimize missing data, whereas appropriate ‘windows’ for biological data collection are short and fixed.

Table 1.

Summary of strengths and limitations of self-report and toxicology approaches to assessing drug-taking behaviour.

Method of assessment	Strengths	Limitations
Self-report	Convenient, inexpensive Usually good validity Can provide temporal and quantitative detail Missing data potentially retrievable at a subsequent visit	Uncertain validity Risk of willful or accidental distortion
Toxicologya	Objective data Adequate specimen volume On-site and laboratory-based test availability	Inconvenience, expense Poor quantity/frequency resolution Utility depends on frequency of sample collection Poor sensitivity to reduced but continued use Missing data permanently lost

^aEach potential substrate for toxicology testing (urine, saliva, hair, breath, sweat, cuticles, etc.) has its own method-specific strengths and weaknesses, which vary depending upon the drug being assessed.

One important factor in determining the extent of agreement between self-report and biological markers is the level of drug use. Agreement is greatest at the extremes of drug use frequency—either high or low levels of use. In addition, the measures will have better agreement when the time-frame covered by the questions is the same as the biological window for ascertainment. Other factors influencing agreement are the drug and the purpose and timing of biological measurements, for example at treatment intake, when motivation to report accurately is thought to be higher than at follow-up [92]. Disagreement between self-report and biological measures can occur when the specimen is collected outside the window of drug detection, when there is poor recall or deliberate misreporting or when the analytical method is not sensitive enough to detect drug use. There is a strong consensus that multiple assessments, including self-report and biological testing, yield the most accurate drug use information.

The panel proposed an outcome measure that combines results from both self-report and biological testing; a positive self-report or toxicology test indicates drug use during an assessment period. While such a measure can be used to determine a dichotomous abstinence/non-abstinence outcome, it also can accommodate reduction in the number or percent of days of use. Also in this context, it should be noted that periods of brief abstinence detected by toxicology testing or self-report are acknowledged as beneficial and are consistent with ‘improvement’ measures, but not by measures requiring periods of sustained abstinence. Strategies that support accurate self-report and combine self-reports with biological indicators have a number of advantages. Although the field has not yet achieved consensus on an ideal strategy for combining self-report and biological data [75], efforts towards that goal are being made, most notably for cocaine studies. The current algorithm used for this purpose in cocaine clinical trials was developed by the NIDA Division of Pharmacotherapy and Medical Consequences of Substance Abuse in conjunction with the College on Problems of Drug Dependence as a result of a consensus meeting held in 1999 [21]. It combines self-report based on the TLFB, quantitative urine benzoylecgonine levels and an estimate of the concordance between the two to determine the cocaine-use status of each study day. The resulting primary outcome variable, which has been used in a large percentage of cocaine clinical trials funded by this division of NIDA over the past 10 years, is called ‘the weekly fraction of cocaine non-use days’ [93]. As a more recent example, in 2010 NIDA’s National Drug Abuse Treatment Clinical Trials Network (CTN) Treatment Effect and Assessment Measures (TEAM) Task Force (http://ctndisseminationlibrary.org/display/522.htm) recommended to use as primary outcome in CTN trials the number of days of drug use during the last 30 days of the active treatment phase, based on self-report corroborated by qualitative urine drug screening tests. Typically, a positive toxicology result overrules self-reported abstinence for the period covered by the toxicology procedure. Toxicology testing is also sometimes the only outcome, ignoring self-report. However, combining the objectivity of toxicology with the reduced data loss, the wider assessment window and the continuous data associated with self-reports provides a composite measure that has advantages over either of its components [93].

Such an approach works best when both the self-report and toxicology are collected at frequent intervals—for example, self-reports for each day with toxicology two or three times per week. For commonly abused drugs such as cocaine and short-acting opioids for which toxicology is sensitive for only 2–3 days after use, this provides a degree of temporal precision for each indicator that makes it relatively easy to combine them into a single classification. For example, each day can be categorized as positive or negative for drug use based on self-report, with a positive toxicology resulting in each of the past 2 or 3 days being categorized as positive for drug use regardless of the self-report. It is more complicated when toxicology testing is infrequent or when toxicology may remain drug-positive for a long duration after drug use as for cannabis, and there are no established guidelines for integrating self-report and toxicology data in such cases. Nevertheless, the panel recommended that clinical trials of drug dependence treatment should routinely collect both self-report and toxicology data at whatever greatest frequency is practical for the study design, patient participation logistics and available budget. While the requirement of such frequent assessments may provide more accurate measures of substance use, it may also restrict the individuals who are willing and able to enroll and remain in a trial having a rigorous data collection schedule, may result in a measurement effect that reduces the effect of the intervention relative to the comparison group [94] and may have an impact on external validity and generalizability of findings. However, such a limitation may be necessary in an efficacy trial to maximize internal validity and in a randomized trial the effects of repeated assessment should be constant across groups. Subsequent effectiveness trials, with less frequent assessments and greater external validity, would be necessary to determine treatment effectiveness and generalizability.

It is also important that self-report data be collected independently of toxicology data—i.e. that patients provide their self-reports before being informed of toxicology results. It is possible that additional information may be gained by repeating the self-report assessment after the toxicology data are available and presented to the patient, but the value of doing this is uncertain.

NO SINGLE CLINICAL OUTCOME METRIC

The panel reached consensus that the primary outcome measure should be an indicator of drug-taking behavior, and that there is no single clinical metric that is appropriate for inclusion in most drug dependence treatment trials. The panel noted that a wide variety of indicators are available, that the most appropriate one might vary by study methods and goals, that a decision process for determining an appropriate primary outcome would be useful and that there is probably substantial correlation between all the commonly used indicators, although this latter supposition is one on which data are needed for verification.

The panel noted that the array of outcome measures reported in contemporary studies remains as diverse as in reviews by Wells and colleagues [2] from more than 20 years ago. The most common self-report methods are questions about number of days used during the past 30 days from the Addiction Severity Index or the Maudsley Addiction Profile, and the TLFB procedure’s categorization of each past day as a day of use or not.

Outcome measures used commonly in these studies include: percentage of days used, percentage of days abstinent, number of abstinent visits (a composite of abstinence and retention), percentage of positive (or negative) toxicology tests, longest duration of continuous abstinence and percentage of patients achieving abstinence of ‘x’ duration (often 2 or 3 weeks), among others. Studies vary in whether missing toxicology samples are treated as missing (unknown) or imputed to be drug-positive. Imputation of missing samples as drug-positive is quite common practice, but can lead to quite implausible conclusions when used indiscriminately; unfortunately, it is not clear where the boundary lies between appropriate and inappropriate imputation.

FACTORS TO CONSIDER IN A DECISION PROCESS FOR DETERMINING APPROPRIATE OUTCOME MEASURES

A number of questions to be considered as aids in the process of selecting an appropriate primary outcome variable for a specific study were proposed. These could be seen as key elements in a decision process for outcome selection:

• What specific type of drug of abuse is being studied?

• What is known about the pattern of use and temporal course of this particular drug?

• What toxicological options are available for assessing drug-taking?

• What are the strengths, weaknesses and precision of toxicology methods?

• Will patients enter the trial abstinent or as ongoing active users?

• Does the intervention’s mechanism suggest appropriate outcome measures?

• How long are the treatment effects and outcomes expected to last—only during active treatment delivery or beyond? If beyond, what is the length of expected benefit beyond treatment to demonstrate durability?

• To what audience is this trial directed?

• Is reduced use of the target drug an acceptable benefit/outcome, or only abstinence?

• Are alcohol and other drugs being assessed to determine what impact reduction in the target drug has on other substance use, either contributing to a decrease, increase, or substitution effect?

• Are indirect/surrogate indices such as desire/intention/craving appropriate?

• Must the trial’s outcome measure satisfy the FDA criterion of ‘success’?

Figure 1 presents a graphic depiction of this process, indicating that the specific primary outcome for a given trial may be best determined in consideration of factors such as those listed above and, as such, the specific parameters of drug use behavior to be assessed, the specific biological matrix to be employed and the time-frames for assessments will vary across trials. Table 2 presents a number of examples of this trial-to-trial variability in primary outcomes as a function of trial-specific factors in both behavioral and pharmacological interventions, as well as providing an overview of how the factors presented above were operationalized. While these are five representative protocols conducted in the NIDA Clinical Trials Network [95–99], the principles involved in the outcome selection process are applicable to clinical trials conducted in a number of countries. As can be seen, while all five protocols include drug-taking behavior as the primary or a co-primary outcome, there is considerable variation across trials. This reflects the difficulty in recommending a single common outcome for clinical trials.

Figure 1.

Heuristic model of factors to be considered in selecting outcome measures in drug dependence treatment clinical trials. TLFB: timeline follow-back

Table 2.

Examples from clinical trials of factors involved in primary outcome decisions.

	Examples of diversity of primary outcome variables as a function of trial characteristics
Factors to be considered	Opiate detox with buprenorphine/naloxone	Motivational enhancement	Motivational incentives	ADHD and adolescent drug use	Web-based cognitive behavioral therapy
Target drug	Opiates	Any substance use disorder with use within the past 28 days	Stimulants—cocaine, amphetamine or methamphetamine	Multiple drugs of abuse	Primary drug of abuse
Type of intervention	Pharmacotherapy: 13-day buprenorphine/naloxone taper in in-patient setting	Behavioral: 3 sessions of MET at beginning of out-patient treatment	Behavioral: motivational incentives—provision of tangible incentives to clients for drug-free urines	Pharmacotherapy: 16 weeks of methylphenidate in adolescents with ADHD and substance use disorders	Behavioral: web-based version of the Community Reinforcement Approach plus abstinence incentives
Primary outcome focus	Response to taper/detox treatment	Treatment engagement and retention; substance use	Stimulant use reduction	ADHD symptom severity; substance use	Substance use; treatment retention
Operationalized primary outcome variable	Positive treatment response: participant completes 13-day detoxification and last urine specimen is negative for opiates	Weeks in treatment at CTP through 3-month follow up; Days of substance use, validated with urine and breath specimen results	Percentage of stimulant-free urines submitted; longest duration of abstinence from primary target drugs (cocaine, amphetamine, methamphetamine and alcohol)	Number of DSM-IV ADHD symptoms endorsed; number of days of alcohol and drug use during the past 28-day assessment periods	Abstinence from 10 tested drugs and heavy drinking during the 12 weeks of active treatment; number of days patients are in treatment until last face-to-face contact
Rationale for outcome measure	Evaluated detoxification, which is primarily concerned with safe and effective medical management of withdrawal and transfer to ongoing care	Evaluated a brief intervention designed to enhance engagement in treatment, which was expected to improve drug use outcomes	The primary focus of the incentives intervention was establishing and prolonging stimulant drug abstinence	Evaluated the impact of effective pharmacotherapy treatment for ADHD on substance use	The focus of the incentives intervention was on establishing and prolonging drug abstinence
Method of assessment	Urine specimen collected on day 13 or 14 is negative for opiates	Self-report drug and alcohol use based on Substance Use Calendar, as validated by urinalysis and breathe specimens	Full screen urinalysis and alcohol breathalyzer; self-reported drug use based on Addiction Severity Index	DSM-IV ADHD symptom checklist; self-report of use based on time-line follow-back, urine toxicology screens	Self-report of drug use and heavy drinking days based on time-line follow-back and urine toxicology screens
Secondary outcomes	Withdrawal symptoms, treatment retention, patient satisfaction	Cost–benefit, patient satisfaction, patient/staff attitudes	Treatment retention, counseling utilization	Psychosocial function, coping skills acquisition, HIV risk behaviors	Treatment retention, HIV risk behaviors, psychosocial functioning
Follow-up time perspective	Short-term, primarily interested in status at end of 13-day active detoxification period	Intermediate-term evaluated the durability of MET relative to standard treatment through a 3-month follow-up	Intermediate- to longer-term Evaluated outcomes throughout active treatment but also out to 6-month follow-up	Short-term Primary focus on 16-week active treatment phase and 1-month follow-up	Short- to longer-term Test for difference in substance use primary outcome based on last 4 weeks of active treatment but also will follow out to 6 months

ADHD: attention deficit hyperactive disorder; CTP: community treatment program; DSM-IV: Diagnostic and Statistical Manual version IV; HIV: human immunodeficiency virus; MET: motivational enhancement therapy.

THE FDA STANDARD: A ‘SUCCESS’ METRIC

The panel’s consensus that the cardinal feature of drug use disorders is the behavior of drug-taking, and that an indicator of this behavior should be the primary outcome variable in drug dependence treatment clinical trials, is in contrast to the view of the US FDA. The FDA suggests that drug-taking behavior is only a surrogate indicator for risks of health or behavioral problems, and that a clinical metric reflecting sufficient behavior change beyond drug use is needed to reasonably conclude a probable benefit in health and behavior domains. In particular, the metric must define clinical ‘success’ so that results and effect sizes can be expressed as the percentage of patients achieving success. Success must be clinically meaningful. The panel’s consensus was that interventions that reduce drug use by half are clinically meaningful [13] and that a greater acceptance of the value of interventions producing limited benefits may be appropriate, especially in a field where available treatment options are often quite limited.

The appropriate standard for considering an intervention to be of clinical benefit may depend on the treatment(s) already available and on the stage of therapy development for a particular substance abuse problem. Early in therapeutics development, clinical trials may ask the modest question ‘does this intervention have any benefit?’, whereas later, it becomes more relevant to address the question of ‘does this intervention have benefit of clinically meaningful magnitude?’. While we focus here on drug-taking as proximal behavior and primary outcome, a companion paper addresses measures appropriate to broader domains associated with drug-taking [100].

ALTERNATIVES TO SEEKING A SINGLE CLINICAL OUTCOME METRIC

One impetus for selecting a common outcome for clinical trials is facilitating between-study comparisons, although within-study comparisons are often far more valid and informative than between-study comparisons. As separate studies always differ, it may well be that efforts to compare studies on a common clinical metric may be more misleading than informative.

One standard and accepted method for comparing and combining data across studies is that of meta-analysis, in which the magnitudes of within-study effects are compared or combined across different studies, commonly on a standardized metric of the statistical ‘effect size’. This standardized effect size approach may be appropriate for comparison of substance abuse treatment trials that employed different clinical metrics for assessing their outcomes.

The panel concluded that producing appreciable change in drug-taking behavior itself can be sufficient to be considered clinically meaningful, without requiring benefit on more global or distant health or behavior outcomes. At the same time, the panel recognized and endorsed the value of assessing such health and behavior outcomes—especially functional behavioral and psychosocial outcomes as they often precede medical health consequences. These other domains of potential behavioral assessment are discussed in a companion paper [100].

CONCLUSIONS

• An indicator of drug-taking behavior is the appropriate primary outcome variable in most clinical trials of drug abuse treatments.

• An outcome measure that combines information from both self-report and objective toxicology testing is often preferable to either alone.

• There is no single outcome measure recommended as the standard index for incorporation into most clinical trials.

• Selection of a specific primary outcome measure for any specific study will depend upon the study and its goals.

• Use of alcohol and drugs other than the drug targeted specifically by the intervention should be assessed and included as potential secondary outcomes.

• For many trials it will be valuable to include secondary outcomes related to behavioral functioning and/or quality of life.

• Research is needed regarding correlations of drug-taking indicators with one another and with behavioral functioning and/or quality of life in both the short and long term.

• Research is needed regarding the effect of implementing and incorporating research-quality outcome measures into clinical treatment settings.

• Research regarding relationships of clinical trial data to functional behavioral outcomes may guide refinements in selecting primary outcome measures in future clinical trials.

APPENDIX I

Clinically meaningful substance abuse treatment outcome measure for effectiveness trials 15–16 December 2009

Participant list
George Bigelow PhD	David Epstein PhD
Johns Hopkins University School of Medicine	National Institute on Drug Abuse
James Bjork PhD	Daniel J. Feaster PhD
National Institute on Drug Abuse	University of Miami
Michael Bogenschutz MD	Patrick Flynn PhD
University of New Mexico	Texas Christian University
Gregory Brigham PhD	Lawrence Friedman MD
Maryhaven, Inc.	National Institute on Drug Abuse
Rita Cardim	John Gardin PhD
Johns Hopkins Health System	ADAPT, Inc.
Kathleen Carroll PhD	Udi Ghitza PhD
Yale University School of Medicine	National Institute on Drug Abuse
Ling Chen PhD	Shelly Greenfield MD, MPH
US Food and Drug Administration	Harvard University Medical School
Allan Cohen MA, MFT	John Hamilton
Bay Area Addiction Research and Treatment, Inc.	Regional Network of Programs, Inc.
Dennis Daley PhD, LSW	Deborah Hasin PhD
University of Pittsburgh Medical Center	Columbia University
Marta De Santis PhD	Marilyn Huestis PhD
National Institute on Drug Abuse	National Institute on Drug Abuse
Traci Donnelly MS	John Hughes MD
Phoenix House of New York	University of Vermont
Dennis Donovan PhD	Petra Jacobs MD
University of Washington	National Institute on Drug Abuse
Sarah Q. Duffy PhD	Ron Jackson MSW
National Institute on Drug Abuse	Evergreen Treatment Services
Jeffrey Leimberger PhD	Michael Klein PhD
Duke Clinical Research Institute	US Food and Drug Administration
Michael Levy PhD	Carmen Rosa MS
CAB Health and Recovery Services	National Institute on Drug Abuse
Esther Lewis MPA	Jeffrey Selzer MD
Indiana Access to Recovery	Medical Society of the State of New York
Robert Lindblad MD	Eugene Somoza MD, PhD
The EMMES Corporation	University of Cincinnati
Walter Ling MD	Maxine Stitzer PhD
University of California, Los Angeles	Johns Hopkins School of Medicine
G. Alan Marlatt PhD	Michele Straus MS, RPh
University of Washington	National Institute on Drug Abuse
Dennis McCarty PhD	Jose Szapocznik PhD
Oregon Health and Science University	University of Miami
Patrick B. McEneaney MBA	Betty Tai PhD
Phoenix House of New England and Florida	National Institute on Drug Abuse
David Metzger PhD	Stephen Tiffany PhD
University of Pennsylvania	University at Buffalo, The State University Of New York
Mary Ellen Michel PhD	Madhukar Trivedi MD
National Institute on Drug Abuse	University of Texas Southwestern Medical Center
Karen Nielsen MBA, MPA	Paul Wakim PhD
Public Sector Healthcare	National Institute on Drug Abuse
Neal Oden PhD	Elizabeth Wells PhD
The EMMES Corporation	University of Washington
Harold Perl PhD	Roger Weiss MD
National Institute on Drug Abuse	Harvard University Medical School
Kenzie Preston PhD	Celia Winchell MD
National Institute on Drug Abuse	US Food and Drug Administration
Cendrine Robinson (PhD student)	Joan Zweben PhD
Uniformed Services University of Health Sciences	University of California, San Francisco