Beyond Drug Use: A Systematic Consideration of Other Outcomes in Evaluations of Treatments for Substance Use Disorders

Abstract

Across the addictions field, the primary outcome in treatment research has been reduction in drug consumption. A comprehensive view of the impact of substance use disorders on human functioning suggests that effective treatments should address the many consequences and features of addiction beyond drug use, a recommendation forwarded by multiple expert panels and review articles. Despite recurring proposals, and a compelling general rationale for moving beyond drug use as the sole standard for evaluating addiction treatment, the field has yet to adopt any core set of “other” measures that are routinely incorporated into treatment research. Among the many reasons for the limited impact of previous proposals has been the absence of a clear set of guidelines for selecting candidate outcomes. This paper is the result of the deliberations of a panel of substance abuse treatment and research experts convened by the National Institute on Drug Abuse to discuss appropriate outcome measures for clinical trials of substance abuse treatments. This paper provides an overview of previous recommendations and outlines specific guidelines for consideration of candidate outcomes. A list of outcomes meeting those guidelines is described and illustrated in detail with two outcomes: craving and quality of life. The paper concludes with specific recommendations for moving beyond the outcome listing offered in this paper to promote the programmatic incorporation of these outcomes into treatment research.

INTRODUCTION

Most research on the effectiveness of treatments for substance-use disorders focuses on the extent to which interventions reduce drug consumption, with one or more indicators of substance use as the primary outcomes. A more comprehensive view of the appropriate targets of treatment evaluation extends beyond the quantity and frequency of drug use. Highly salient constructs such as craving are experienced by the addicted individual as aversive and disruptive to functioning, while change self-efficacy is a common, important intermediate target of treatment. In addition, the addictive process often affects functioning outside of the immediate realm of drug use. These include consequences in the domains of health, well-being, psychological functioning, relationships, productivity, and criminality. Further, it is the impact of these consequences on the individual user, significant others, and society rather than drug use, per se, that drives personal and societal concerns about addiction. Therefore, comprehensive evaluations of treatment outcomes for substance use disorders should address those consequences.

Addiction treatments will be most effective to the extent that they ameliorate or reduce the panoply of negative consequences of drug use. But levels of drug use are not necessarily tightly coupled to these consequences. Therefore, measures limited to drug use cannot represent all significant sequela of drug dependence. Instead, a comprehensive appraisal of the impact of a treatment on functioning will have to address outcomes beyond use.

There are additional reasons to incorporate a broader array of outcome measures into research on addictions treatment. Importantly, outcomes other than drug use can provide crucial information about the mechanisms responsible for treatment efficacy and effectiveness. These outcomes might also be used as critical markers in adaptive treatment strategies in which interventions are altered systematically as a function of the person's response to treatment (e.g., [1]) Moreover, measures of other outcomes can address critical questions about treatment safety and cost, issues that cannot be assessed simply through considerations of levels of drug use. Furthermore, researchers interested in harm reduction as a treatment goal would target a wider range of outcomes than drug use (e.g., [2]). Finally, many empirically validated interventions have not been widely adopted in the treatment community [3]. Although there are multiple obstacles to dissemination of evidence-based practices, a concern for many providers is that the efficacy and effectiveness of evidence-based treatments have often been marketed solely on the basis of reductions in substance use. Treatments with demonstrable impact on other clinically and socially relevant consequences of addictive disorders would likely make those interventions more appealing to a wider array of critical stakeholders and advance their diffusion across the treatment community.

A panel of substance abuse treatment and research experts was convened by the National Institute on Drug Abuse (NIDA) in December of 2009 to discuss appropriate outcome measures for clinical trials of substance abuse treatments. One of the subgroups formed for that meeting (comprised of the authors of this paper) was charged with formulating recommendations for assessments of treatment outcomes beyond the conventional drug-use measures used in treatment studies. This paper, which is the result of the deliberations of that group, provides an overview of previous recommendations, outlines specific guidelines for consideration of candidate outcomes, describes the application of those guidelines to select outcome domains, and offers explicit recommendations for systematic incorporation of these outcomes into treatment research.

CONSIDERATION OF PREVIOUS RECOMMENDATIONS

The idea that measures of treatment effectiveness should include assessments of factors that go beyond evaluations of drug-use measures has long been advocated in the literature (e.g. [4,5]) For example, a highly cited review by Wells, Hawkins, and Catalano (1988) [6] recommended that the Addiction Severity Index [7], which generates composite scores reflecting problem severity across seven areas of functioning, should be incorporated routinely into treatment assessments.

Over the years, several expert review groups have also identified a variety of domains that should be routinely assessed in addictions treatment research. In 1992, the Clinical Decision Network of Cocaine Addiction Pharmacotherapy, sponsored by the National Institute on Drug Abuse [8], discussed the possible use of psychiatric outcomes, craving, subjective drug effects, and retention in treatments as important adjunct outcomes that might be used to evaluate outcomes from clinical efficacy trials in cocaine addiction pharmacotherapy. Six years later, a meeting co-sponsored by NIDA and the College on Problems of Drug Dependence [9] recommended that measures of drug-related problems/problem severity, craving, withdrawal, psychosocial functioning, and clinician ratings of global improvement be included as important secondary outcomes in treatment trials. More recently, a clinical consensus statement generated by an expert panel convened by the European College of Neuropsychopharmacology [10] suggested that assessments of clinically relevant reduction in drug-related harm, craving, and clinical global assessments be incorporated into studies of treatment efficacy. The most recent set of recommendations regarding the assessment of clinically meaningful outcomes in drug use treatments comes from a task force sponsored by NIDA's Clinical Trials Network (2010) [11]. This group recommended that the Addiction Severity Index and a measure of quality of life be included across CTN trails.

IMPACT OF PREVIOUS RECOMMENDATIONS

Certainly, there are numerous examples of treatment studies that have targeted outcomes other than drug use (such as social functioning, work, and criminal behavior), but there has not been any consistency in ancillary measures across treatment studies, and researchers do not universally include other measures in their studies. Moreover, even when included in treatment research, there is no expectation for actually reporting the results of these measures. That is, despite the recommendations produced by expert committees and comprehensive reviews over the past two decades, the field has yet to adopt any core set of “other” measures that are routinely incorporated into treatment research. The limited impact of these recommendations likely derives from several sources. First, though there is overlap, various committees have forwarded somewhat different sets of suggested domains of measurement. This lack of consensus does not promote sustained momentum for the adoption of a standard battery of assessments. Second, in most reports and summaries, arguments for assessments of selected other outcomes have not been consistently well developed. In the absence of a compelling rationale, treatment researchers have not been persuaded that any particular set of additional outcome assessments is definitive. Third, previous recommendations rarely identify specific measures for selected outcomes. This issue is particularly critical, as some of the concepts (e.g., quality of life, psychosocial functioning) are variously defined and can be approached in a myriad of ways. Without clear guidance on a specific set of measures to address key domains, researchers have been forced make decisions on a study-by-study basis, sometimes using ad hoc measures with unknown psychometric properties. Finally, treatment research designed to systematically address other outcomes may require larger and more expensive studies, with the implication that such studies would require long-term approaches to prevention and treatment. To date, neither researchers nor funding agencies have been convinced that the clinical yield of these extra efforts would offset their cost.

GENERAL GUIDELINES FOR CONSIDERATION OF CANDIDATE OUTCOMES

The potential deleterious consequences of substance use are broad, representing multiple domains across all major areas of human functioning. Table 1 lists the 21 domains that were initially considered by the panel as possible candidates for inclusion in treatment studies. However, the routine assessment of all possible outcomes in every study is not viable – so choices had to be made. We believe the following guidelines offer a principled, reasonable approach for selection of candidate variables:

1. The outcome must be a consequence or a strong, concurrent correlate of excessive drug use. The point of this standard is to distinguish variables that are consequences of drug use or are common features of drug dependence from risk factors that may be causal or clearly antecedent to substance-use disorders (e.g. [12]). As an example of the latter, attention deficit/hyperactivity disorder (ADHD) is a well-documented risk factor for substance-use disorders [13], yet there is little evidence that this condition arises as a consequence of substance use. Certainly, some variables might serve as both causes and consequences of substance-use disorders – as one example, impulsivity may be both a risk factor for drug dependence as well as further exacerbated by chronic exposure to drugs of abuse [14,15]. In this case, impulsivity as a consequence of drug use might warrant attention in treatment studies.

2. The outcome has broad clinical or societal salience and relevance. Numerous outcomes of chronic drug use are associated with substantial distress among users and/or create considerable societal concern. On the other hand, chronic drug use can produce outcomes that generate limited clinical or public attention. For instance, drug tolerance, an indisputable consequence of repeated drug exposure and a core feature of drug dependence [16,17], is rarely cited as a major clinical issue for the recovering addict. Accordingly, there would be little compelling reason to track tolerance as a routine outcome in treatment trials.

3. The outcome is common across abused substances and widespread among people dependent on those substances. In general, the case for routinely including a consequence in treatment research becomes compelling if the outcome is pervasive across people and drugs. In contrast, an outcome unique to a particular drug (e.g., drug-specific withdrawal) or restricted to a select group of users (e.g., medical complications from intravenous drug use [18]) would have limited utility as an outcome variable across all treatment studies.

4. Practical measures with documented, strong psychometric properties are available to assess the outcome. This standard encompasses two critical elements: First, the measure must be feasible within the context of a treatment study - that is, reasonably brief, easy to implement, and applicable across a wide range of drug users. Second, the measure must have psychometric qualities expected of any modern scientific instrument – strong reliability, ample sensitivity and selectivity, and excellent construct validity. In the absence of a measure with these features, no outcome, regardless of its salience, relevance, or pervasiveness, can be plausibly assessed in any treatment study.

5. There is replicable evidence that the outcome can be altered following treatment for addictive behaviors. A treatment that produces sustained reductions in drug-use behaviors should eventually yield corresponding changes in the deleterious consequences of excessive drug use. But changes in “other” outcomes may lag reductions in drug use, so studies may have to use a timeframe capable of documenting any delayed changes. Moreover, some outcomes are more proximal to the putative effect of a treatment program whereas others are more distal. For example, many treatments explicitly target drug use, and those treatments would likely have a greater impact on that variable than on more distal outcomes such as quality of life or psychosocial functioning. This proximal-distal distinction suggests that the beneficial consequences of effective addiction treatment may not only take longer to accrue for non-substance abuse outcomes, they may be smaller as well, as those outcomes can be influenced by a broad range of factors. Regardless, without persuasive evidence that a variable is capable of changing as a function of treatments that target substance dependence, there would be little reason to assess that outcome routinely across treatment studies. Ideally, the evidence would indicate a causal relationship between a treatment and a candidate outcome, but the literature might not be developed to the point where causality is established definitively. In that case, replicable evidence of associations between a candidate outcome and other, clearly established addiction treatment outcomes would suffice.

Table 1.

Listing of potential outcome domains considered by work group.

Arrests/incarceration Change self-efficacy Clinically relevant reduction in drug-related harm Composite indices of problem severity Coping Craving Days in stable housing Days worked/employed or in school Days/time in treatment Decreased days in the hospital or ER Drug withdrawal (acute and protracted) Global assessment of function Health Intensions and plans to abstain from drug Psychiatric outcomes Psychosocial functioning Quality of life Readiness to change/stage of change Social network/Social support Stress Subjective drug effects Successful treatment completion

The five domains indicated by bold typeface met guidelines for inclusion (see text) and were recommended as outcomes in treatment studies.

PRIMARY DOMAINS CONSIDERED FOR INCLUSION

After consideration of the extant treatment literature, review of previous recommendations, discussion among addictions experts, and application of the criteria described above, we identified five candidates for inclusion as primary outcomes in treatment studies. These were change self-efficacy, psychosocial functioning, network support/social support, craving, and quality of life. There is a long history of research on self-efficacy in the addictions field with considerable evidence that people with substance-use disorders have relatively low self-efficacy beliefs with regard to their ability to restrict or control dug use in high-risk situations [19]. Self-efficacy beliefs have emerged as consistent predictors of drug-use outcomes across multiple treatment studies [20–23]. Moreover, there are several validated instruments for self-efficacy assessment across major drugs of abuse [24–27], and there is considerable evidence that self-efficacy can increase as a consequence of substance-use treatments (e.g., [28,29]). Impaired psychosocial functioning is a hallmark of psychiatric conditions in DSM, including substance-use disorders [16]. Problems with functioning in occupational, educational, marital, parental, family, and community roles, which collectively define impaired psychosocial functioning, are associated with a range of substance-use disorders in the general population as well as in clinical samples [30–34]. There are several validated measures of psychosocial functioning [35–37]. Impairment in this domain predicts treatment outcomes, and treatments of substance-use disorders can enhance psychosocial functioning [38]. Social networks and social support are related concepts, with the former describing a person's constellation of social relationships, and the latter referring to the degree to which a person's social needs are met through interaction with others [39–41]. Both are important correlates of substance-use disorders with evidence that drug users are prone to associate with other drug users, and social support for substance use or substance desistence influences levels of drug consumption and abstinence attempts [39]. Social support and social networks can be assessed with a variety of instruments suitable for the drug-abuse field [42,43], and both constructs may be influenced positively by treatments for substance-use disorders [44,45].

We will describe two domains in greater detail – craving and quality of life – to demonstrate the application of the selection guidelines listed in the preceding section. These two outcomes, which reflect very different dimensions and levels of functioning, are particularly illustrative of the range of variables that will enrich our understanding of the impact of treatment on substance-use disorders. Moreover, these outcomes provide clear examples of the consequences of addiction that contribute directly to the impairment and distress associated with substance-use disorders.

Craving

Craving is ubiquitous across all abused substances [46]. In most contemporary conceptualizations of drug disorders, craving plays a central role in addictive processes, serving as both a cause and consequence of chronic drug use (e.g., [47–50]). There have been multiple recommendations that craving be included as a standard outcome across treatment studies (e.g., [8–10,51]), and even a cursory review of the literature shows that craving is one of the more commonly assessed “other outcomes” in treatment research (e.g., [52–65]).

Clinically, craving has substantial diagnostic and predictive relevance. It is included in the International Classification of Diseases–10th edition [66] as a major component of drug dependence and has been proposed for DSM-V as a defining feature of addiction [67]. Though craving and drug use are not necessarily tightly coupled, in the sense that not all instances of drug use or relapse are always preceded by craving [46], there is evidence that craving can be predictive of relapse (e.g., [68–70]). To the user, craving is highly salient - addicts often describe craving as an obstacle to quitting drug use [71], and craving itself is frequently depicted as distressing and disruptive to functioning [50].

There are psychometrically validated craving measures for all major drugs of abuse including alcohol [72], cocaine [73], heroin [74,75], marijuana [76], and nicotine [77,78]. Short forms of these instruments have been developed and are suitable for rapid, valid assessment of general craving levels [72,74,76,77,79]. Finally, there is considerable evidence that treatments for drug use can affect levels of craving [59]. For example, craving is reduced by FDA-approved medications for various forms of drug dependence including buprenorphine and methadone for opioid dependence (e.g., [80,81]), acamprosate and naltrexone for alcohol dependence (e.g., [52–54,82]), and bupropion, nicotine patches, and varenicline for tobacco dependence [55,56,61]. In sum, the domain of craving meets all the guidelines outlined above and should be included routinely as an outcome in studies of treatments for substance abuse.

Quality of Life

Many researchers have argued that studies of biomedical treatments must move beyond a limited focus on disease-specific pathology to a broader appraisal of an individual's quality of life when assessing the impact of interventions for nearly any disorder [83–85]. Indeed, clinical trials across many areas of medicine characteristically include quality of life as an outcome variable [86–89]. The construct of quality of life has received considerable research attention – a Medline search of the term appearing in abstracts from 2001 to the present generated over 65,000 citations. Though there are exceptions, addiction research clearly lags other biomedical fields for inclusion of quality of life evaluations in treatment research [90]. Several researchers have recommended that quality of life be part of any outcome evaluation of substance-use treatment [91,92], a recommendation echoed in the conclusions of many of the expert panels and workshop reviews cited above.

Although it has been defined in multiple ways, quality of life generally refers to an individual's perception of well-being or satisfaction across diverse areas of functioning. The definition adopted for this paper is the one forwarded in a recent set of guidelines for patient-reported outcomes by the Food and Drug Administration [93]: “Health-related quality of life [is a] multi-domain concept that represents the patient's general perception of the effect of illness and treatment on physical, psychological, and social aspects of life.” (page 32) Contained within this definition are three important considerations for addiction treatments. First, the construct is assessed subjectively by the individual; in essence, it is a quintessential patient-reported outcome that is not well captured by proxy reports or other modes of measurement. Second, quality of life refers to a person's appraisal of the impact of both addiction and addiction treatment on functioning. Presumably, addictive disorders attenuate quality of life, and treatments, to the extent they are successful, should enhance quality of life. But treatments could have iatrogenic effects that could diminish quality of life, even though the same treatment might reduce drug use. Finally, as quality of life reflects appraisals across multiple domains of functioning, a comprehensive assessment of the construct will have to address these various domains.

Quality of life, as affected by substance use disorders, is highly salient clinically [90], a fact recognized by the DSM-IV [16] description of substance dependence as “a maladaptive pattern of substance use, leading to clinically significant impairment or distress.” Attenuated quality of life has been associated with a range of substance-use disorders including alcohol dependence (e.g., [94,95]), heroin dependence [96], cocaine dependence [97], and cigarette smoking [98]. There is mounting evidence that quality of life can be enhanced following treatments that generate reductions in drug use across a range of substance-use disorders (e.g., [94,95,99–102].

There is no shortage of assessments of quality of life – reviews of this literature typically identify scores of disease-specific and general measures of this construct (e.g., [103–106]). Some of the more commonly deployed instruments, which assess general, multidimensional aspects of quality of life include the SF-36 [107] and the World Health Organization Quality of Life-BREF [108]. Both of these instruments have reasonable reliability for each of several aspects of quality of life and evidence of construct-related validity. A comprehensive evaluation of the psychometric properties and validities of these or any other questionnaires is well beyond the scope of the present article. Nevertheless, there is a large literature external to the addictions field that has reviewed psychometric and methodological issues relevant to the evaluation of quality of life (e.g., [109–111]). The addictions field would do well to systematically exploit findings from those studies to incorporate validated instruments and sophisticated analyses of quality of life outcomes in evaluations of addiction treatment and to assess the relative strengths and weaknesses of these instruments as outcome measures for addiction treatment. More generally, quality of life clearly meets the guidelines proposed in this paper and should be included as a standard outcome in studies of treatments for substance use disorders.

PRACTICAL CONSIDERATIONS

We recognize all studies cannot be powered to be comparably sensitive to changes in all recommended outcome domains. Practically, measures of these outcomes might be included with the research powered to address changes in only a targeted domain. Other domains could be assessed as secondary or exploratory targets. At the least, however, major clinical trials (e.g., Phase III studies) should include assessments of at least one major domain other than drug use with adequate power to detect meaningful changes in that domain.

We also recognize that some of the outcome domains recommended in this paper are controversial, and there is likely to be considerable dispute regarding the best way to explicitly assess any of these outcomes. The complexity of these issues cannot be fully addressed much less resolved in this paper. We do, however, offer a proposal in the next section regarding procedures for evaluation of candidate outcome domains and implementation of specific assessment plans.

CONCLUSIONS AND RECOMMENDATIONS

A compelling case can be made for routinely including domains of functioning beyond drug use as primary outcomes in treatment studies. However, these recommendations are not particularly new, as similar proposals have been advanced multiple times over the past 20 years. Yet, the momentum for these recommendations typically dissipates with the publication of the proposals. We believe the recommendations described in this paper are an important step in the process necessary for moving the field forward. But, in order to go beyond this point, we suggest explicit action steps that will promote the programmatic incorporation of these outcomes into treatment research. Specifically:

• We urge greater attention from journal editors, journal reviewers, funding agencies, grant reviewers, and leaders of professional societies to the issue of inclusion of outcomes beyond drug-use measures. Any outcome or set of outcomes cannot and should not be imposed on the scientific community by fiat, but a broadened perspective on treatment outcomes cannot be established if critical gatekeepers in the scientific process are not sensitized to the salient issues.

• Given the putative importance and complexity of these outcomes, we recommend that expert committees be established to focus on individual outcome domains. These committees, which could be supported by funding agencies and consortia of professional societies, should articulate a coherent, scientifically based rationale for the inclusion of the outcome in treatment research, identify obstacles to adoption of measures, provide a standardized definition of the outcome domain, propose relevant instruments, generate explicit assessment strategies, suggest comprehensive data-analytic plans, identify the most pressing research questions, publish reports with detailed recommendations, and establish a follow-up process to monitor adoption of recommendations. The most important product of these committees would be the explicit identification of instruments that directly target the outcome domain. In the absence of specific outcome measures, we understand that the field is unlikely to add any outcome domain to a standard assessment protocol for treatment research. It is essential that the membership of these committees be inclusive and broadly representative of the scientific and clinical community. Ideally, committee representation should cross national boundaries. It may be impractical to tackle simultaneously all of the outcome domains forwarded in this article – but one or two domains could be addressed as demonstrations of the feasibility this approach. There are clear precedents for the utility of this strategy in other scientific fields. For example, the European Organization for Research and Treatment of Cancer (EORTC) created the Quality of Life Group in 1980 to promote the design, implementation, and analysis of quality of life studies within selected phase III clinical trials. One of the products of this group was the EORTC Quality of Life Questionnaire-Core 36 [112] an instrument that has been widely used in the oncology field. Finally, for this proposal to succeed it is crucial that the sponsors of these expert committees provide continued assistance for sustained operation of the committees and a commitment to act on recommendations generated by the committees.

• The programmatic incorporation of broader outcome measurements into addiction treatment requires sustained support from funding agencies for psychometric development and evaluation of relevant instruments. Too often, the real costs of measures research, particularly programs that focus on development of self-report instruments, are not given adequate attention, which unnecessarily compromises the quality of the assessment instruments. Moreover, measurement methods within some of these outcome domains are ripe for adoption of innovative assessment approaches, which will demand more resources, especially in their development phase, than required by traditional paper and pencil instruments. For example, computerized adaptive testing (CAP) is a method based on item-response theory 113] whereby the administration of specific items given to the respondent is determined iteratively as a function of the person's response to previously administered items. With CAP, levels of a given construct can be accurately determined with relatively fewer items than traditional “fixed item” assessments, and levels can be reliably established across a wide range of the response scale. Recent applications within the quality of life literature reveal the potential of this approach (e.g., [114–116].

SUMMARY

Across the addictions field, the primary outcome in treatment research has been reduction in drug consumption. A comprehensive view of the impact of substance use disorders on human functioning suggests that effective treatments should address the many features and consequences of addiction beyond drug use, a recommendation forwarded by multiple expert panels and review articles. Despite recurring proposals, and a compelling general rationale for moving beyond drug use as the sole standard for evaluating addiction treatment, the field has yet to adopt any core set of “other” measures that are routinely incorporated into treatment research. Among the many reasons for the limited impact of previous proposals has been the absence of a clear set of guidelines for selecting candidate outcomes. We offer a set of guidelines and present a list of outcomes that we believe should be considered for inclusion in nearly all treatment research. The application of those guidelines is illustrated with two outcomes: craving and quality of life. We conclude with specific recommendations for moving beyond the outcome listing offered in this paper to promote the programmatic incorporation of these outcomes into treatment research.