Alcohol Treatment Research Assessment Exposure: A Critical Review of the Literature

Abstract

Alcohol treatment researchers have speculated about the benefits of research participation (e.g., research follow-up interviews functioning as aftercare) for more than four decades (Gallen, 1974). Alternatively, research participation can decrease study design sensitivity and hamper the interpretability of research findings. To the extent that the typical alcohol treatment trial is characterized by frequent and comprehensive data collection, accounting for potential research assessment related effects is essential for proper interpretation of study findings. Given this background, the purpose of this article is to review the alcohol treatment research literature on assessment exposure resulting in subject reactivity. In addition, interventions that use data collection activities to inform clinical practice are receiving increased attention and such interventions share common characteristics with research assessment related clinical improvements. Therefore, a second purpose of this paper is to compare and contrast these two influences of behavior change. Study findings indicate that during- and post-treatment data collection activities (i.e., both research and clinical data) positively influence clinical outcomes, although there appears to be important differences regarding the mechanisms by which these two data collection activities exert their influence. Understanding of mechanisms of behavior change, effect boundaries, and the conditions under which clinical improvement is most likely to occur is only at a rudimentary level.

Alcohol treatment researchers have been aware of the potential clinical benefits associated with research participation for some time (Gallen, 1974). Two of the more salient components of the typical alcohol treatment research study protocol are the frequency (e.g., quarterly) and duration (e.g., often continuing for 12-months post-treatment or longer) of scheduled participant follow-up assessments, which has been viewed as a continuing care process (Sobell & Sobell, 1981). Study participant reactivity to ongoing research assessments may be defined broadly as the research participant’s reaction to being observed (Kantowitz et al., 2005) and has been addressed by leading behavioral and social scientists for a number of years (Campbell & Stanley, 1963). In this regard, concerns have been raised about methodological problems such as reduced design sensitivity. Alternatively, such reactivity effects have been shown to influence clinical improvements (e.g., reduced alcohol use and related negative consequences; Clifford, Maisto & Davis, 2007). Given that research assessment exposure may contribute to enhanced clinical outcomes and simultaneously, hamper the interpretability of research findings derived from alcohol treatment trials, more in-depth understanding regarding research assessment exposure related subject reactivity effects seems essential. Therefore, the primary purpose of this paper is to review the alcohol treatment research literature on assessment exposure contributing to improved clinical outcomes such as decreased alcohol use.

In addition, clinical interventions that use data collection activities to inform clinical practice are receiving increased attention and show some commonalities with research assessment protocols (e.g., continued assessment, improved clinical outcomes). Consequently, there may be some confusion about distinguishing features of assessment reactivity and the aforementioned clinical interventions. Such confusion could impede progress in understanding the effects of each type of data collection activity and the manner in which behavior is influenced. Therefore, a second purpose of this paper is to compare and contrast studies of research assessment exposure and studies of clinical interventions that use data collection activities to inform clinical practice.

Identification and Selection of Relevant Articles

Relevant articles for this review of the alcohol treatment research literature on assessment exposure resulting in subject reactivity were identified via a computerized literature search using the key words “alcohol assessment reactivity,” “alcohol research subject reactivity,” and “alcohol assessment exposure.” Study inclusion criteria were such that only empirical studies examining assessment exposure reactivity effects among an alcohol use disorders (AUD) clinical sample were retained for review. Potentially relevant articles were selected based on an initial review of identified article titles and abstracts. Retained articles were then read and a final decision was made regarding article retention. In addition, the reference section of each identified article was reviewed for additional articles not identified in the computerized search. Review of selected article references yielded no new articles for study inclusion, which increased confidence that the computerized literature search yielded all relevant articles.

Classification of Studies

The overall alcohol treatment research assessment related subject reactivity literature can be divided into two broad categories: “Informed Speculation/Interpretation” and “Empirical Corroboration.” The category “Informed Speculation/Interpretation” includes articles that indicate study outcomes may have been influenced by research participation (i.e., researcher attributions), but the study was not designed to assess subject reactivity effects. On the other hand, the category “Empirical Corroboration” includes studies that addressed clinical outcomes affected by research assessment. Studies classified within the category “Empirical Corroboration” were further divided into those studies that primarily focused on the pre-treatment assessment period and those that centered on the follow-up assessment period.

Informed Speculation/Interpretation

As early as 1974, Gallen reported that post-hospitalization contact with alcohol patients contributed to low attrition, reliable data, and improved treatment outcomes. He recommended that the follow-up interview be conceptualized as an important treatment component. The Sobell’s (1981) hypothesized that frequent follow-up interviews can serve as continuing care and are associated with beneficial and desirable outcomes. More recently, Bien, Miller, and Tonigan (1993) noted that the frequently observed decrease in alcohol use among control group participants may be an effect of exposure to research assessments.

Along these lines, Heather et al. (1986) tested the efficacy of a self-help manual, with and without telephone contact, for problem drinkers. Subjects in the telephone sample showed a greater reduction of alcohol-related problems as well as a higher proportion of reduced drinking relative to the group that received program materials via the mail. The authors concluded that interactions with a research interviewer may have contributed to the observed clinical improvements. In a subsequent follow-up effort, Heather et al. (1987) reported that the observed clinical improvements at the 6-month follow-up appear to have spread to other domains of functioning to include physical health and well-being as well as degree of social interaction. Thus, it appears that the use of comprehensive research assessments may be linked to improvements in areas other than drinking per se, such as alcohol and other drug related negative consequences, occupational and marital functioning, and psychological well-being.

A number of investigators, many of them leaders in the alcohol treatment outcomes research field, have alluded to subject reactivity to the research assessment interview as a potential contributor to alcohol treatment outcomes (e.g., Bien, Miller, & Tonigan, 1993; Edwards, Orford, Egert, Guthrie, Hawker, Hensman, Mitcheson, Oppenheimer, & Taylor, 1977; Sobell, Brochu, Sobell, Roy & Stevens, 1987; Stout, Rubin, Zwick, Zywiak, & Bellino; 1999; Wilson, 1978; Zweben, Pearlman, & Li, 1988). The most important study to raise serious concerns regarding research assessment subject reactivity effects, however, was Project MATCH (Matching Alcoholism Treatments to Client Heterogeneity; Project MATCH Research Group, 1997). When robust matching effects were not detected, Project MATCH investigators speculated, among other considerations, that the frequent and intensive research assessment protocol may have reduced the ability to detect differential treatment matching effects. More recently, Project COMBINE (2006), the largest multi-site alcohol treatment outcomes randomized clinical trial (RCT) conducted to date, showed that individuals receiving placebo medication combined with intensive medical management reported significant improvement in their drinking behaviors. In addition, an active medication (i.e., acamprosate) was not shown to be efficacious, a finding that is inconsistent with previous research. Project COMBINE, similar to Project MATCH, was characterized by comprehensive research assessments that were administered multiple times across an extended period of evaluation. As a result, subject reactivity to the extensive Project COMBINE research protocol cannot be ruled out. Therefore, both Project COMBINE and Project MATCH have contributed significantly to alcohol treatment researchers’ increased awareness of potential research protocol related subject reactivity effects.

Empirical Corroboration

To date, there have been 11 published empirical research reports that address potential alcohol treatment research assessment subject reactivity effects (Clifford et al., 2000; Clifford et al., 2007; Epstein et al., 2005; Hester et al., 2012; Kaminer, et al., 2008; Kypri et al., 2007; Maisto et al., 1985; Maisto et al. 2007; Morgenstern et al., 2007; Ogborne & Annis, 1988; Worden, et al., 2008). Of these 11 reports, six (Clifford et al., 2007; Hester et al., 2012; Maisto et al., 1985; Maisto et al., 2007; Ogborne and Annis, 1988; Kypri et al., 2007) were based on experimental designs, one focused on adolescents (Kaminer et al., 2008), two involved college students (Hester et al., 2012; Kypri et al., 2007), and two were specific to adult women (Epstein et al., 2005; Worden et al, 2008). Descriptive characteristics associated with each study are presented in Table 1.

Table 1.

Individual assessment reactivity study characteristics.

Author	Pub. Yr.	Design	Sample	N	Data	Empirical Support
Maisto et al.	1985	2 × 2 Factorial	Adult	48	P	No
Ogborne & Annis	1988	2 × 2 Factorial	Adult	76	P	No
Clifford et al.	2000	Correlational	Adult	167	S	Moderate
Epstein et al.	2005	Correlational	Female Adult	92	S	Moderate
Morgenstern et al.	2007	Quasi-Experimental	MSM	198	S	Moderate
Clifford et al.	2007	2 × 2 Factorial	Adult	235	P	Strong
Maisto et al.	2007	2 × 2 Factorial	Adult	235	P	Strong
Kypri et al.	2007	Experimental	Col. Student	293	P	Weak
Kaminer, et al.	2008	Correlational	Adolescent	177	P	Moderate
Worden et al.	2008	Correlational	Female Adult	25	S	Weak
Hester et al.	2012	Experimental	Col. Student	144/82*	P	Strong

Pub. Yr. = Publication Year; S = Secondary; P = Primary; MSM = men who have sex with men;

^*sample sizes associated with two reported experiments

Pre-Treatment Assessment Focus

Of the 11 research reports reviewed, three focused primarily on baseline assessment related effects (Epstein et al., 2005; Morgenstern et al., 2007; Kaminer et al., 2008). Epstein and colleagues (2005) conducted a study to investigate the pre-treatment points at which women either stopped or reduced their alcohol use. Using data (N=102) from a randomized clinical trial contrasting individual and couples cognitive-behavioral therapy (CBT) for women with a diagnosed alcohol use disorder (AUD), secondary data analyses were conducted to examine the following three research questions. One, do study participants modify their drinking behavior prior to treatment initiation and, if they do, when during the baseline assessment process does such change occur? Two, do factors such as treatment motivation and interactions with an interviewer affect assessment reactivity? Three, do pre-treatment changes in alcohol use predict 18-month (i.e., 12-month post-treatment completion) drinking outcomes? To investigate these questions, the pre-treatment assessment period was divided into four distinct intervals: 1) 90 days prior to initial patient contact (i.e., initial telephone screen, which required about 10-minutes to complete); 2) day of the telephone screen to the day before an in-person clinical screen interview, which required 90–120 minutes to complete; 3) day of the in-person clinical screen to the day before the baseline assessment, which required approximately three hours to complete; and 4) day of the baseline interview to the day before the first treatment session. In addition, quarterly follow-up assessments were conducted through 18-months post-baseline assessment (i.e., 12-months post-treatment completion; treatment consisted of 20 sessions scheduled across a 6-month period). Of the six scheduled follow-up interviews, three were conducted by telephone (i.e., 3-month, 9-month, and 15-month) and the remaining three were scheduled as in-person interviews.

Of the 102 subjects who participated in the parent study, 92 subjects provided data through at least the first week of treatment and constitute the study sample for this secondary data analytic study. Repeated measures ANOVA was used for all analyses pertaining to pre-treatment changes in alcohol use and factors that might account for such changes in alcohol use (i.e., treatment motivation, interviewer interactions). If the omnibus test was significant, pair-wise comparisons were conducted for consecutive time periods and a Bonferroni adjustment was made to control for inflated family-wise type I error. In addition, multiple regression analysis was used to investigate the extent to which changes in pre-treatment alcohol use were associated with longer-term alcohol use. Study findings showed reductions in alcohol use (i.e., both frequency and quantity) occurred across the pre-treatment assessment period such that 44% of the study sample was abstinent prior to the first treatment session. In addition, significant pre-treatment reductions in alcohol use were associated with better (i.e., less) alcohol use both during treatment and 12-months post-treatment completion. Finally, neither participant motivation for treatment nor interactions with an interviewer accounted for the observed changes in participant alcohol use. Given that the derived study findings are based on secondary data analyses, study results must be interpreted cautiously. Nevertheless, the study findings are consistent with an assessment reactivity effect.

Morgenstern et al. (2007) conducted a study to examine the efficacy of two behavioral AUD treatments targeting men who have sex with men (MSM) who were at risk for HIV transmission. More specifically, four sessions of motivational interviewing (MI) delivered across a 12-week period was contrasted with 12 weekly sessions of MI + CBT. Study participants were recruited via media and targeted outreach efforts that included advertisement in gay-oriented publications, use of online MSM chat rooms, and active recruitment in locations known to be frequented by gay men (e.g., gay bars, sex clubs, bathhouses, and large gay community events). Study procedures were such that potential study participants received an initial telephone screen that focused on alcohol use and risky sexual behavior (N=411). Eligible participants participated in an intake assessment that was administered by a trained interviewer. Upon completion of the intake interview (N=317), study participants either agreed to participate in treatment (N=89) or refused treatment participation but agreed to participate in study follow-up assessments (N=109). Thus, the study design was such that individuals were either randomly assigned to one of two treatment conditions or classified within a “non-help seeking” group as a consequence of their refusing treatment. No significant differences were detected between those participants retained in the study and those lost to attrition. Study results showed that significant reductions in drinking behavior occurred across treatment groups, including the non-help seeking control group, prior to treatment onset and post baseline assessment. More specifically, alcohol use appeared to be stable during the pre-baseline period across all three research conditions (MI, MI+CBT, control) and sharp reductions in alcohol use were observed immediately after the baseline interview and prior to the start of treatment. Alcohol use trajectories across the three research conditions were such that individuals assigned to the MI condition continued to reduce their alcohol use during the early phase of treatment. In contrast, individuals assigned to the MI+CBT condition continued to show gradual reductions in alcohol use across the 12-week treatment period. Interestingly, reductions in alcohol use among those individuals classified as non-help seeking controls were limited to the immediate post-baseline period. Given the lack of treatment interest among individuals classified as non-help seeking controls, it seems unlikely that participant treatment motivation accounts for these results. It should be noted, however, that six non-help seeking control participants either sought AUD treatment or attended more than 5 self-help group meetings during the 12-week treatment period and were excluded from study analyses. Similar to the Epstein et al. (2005) study, derived findings are consistent with a research assessment reactivity hypothesis. Nevertheless, given the limitations associated with quasi-experimental study designs (e.g., non-equivalent groups), derived findings must be interpreted cautiously.

Kaminer et al. (2008) conducted secondary data analyses to examine the effects of intake assessments on subsequent alcohol and other drug use among adolescents presenting for AUD treatment. All study participants (N=177) reported baseline alcohol use. Of these 177 adolescents, more than half (51.4%, n=91) reported being abstinent at the first treatment session. Thus, more than half the sample reported a shift from using alcohol to becoming abstinent during the post-intake assessment and pre-treatment initiation period. Of the 145 adolescents who screened positive for drug use at baseline, 42 (29%) reported being abstinent at the first treatment session. Furthermore, alcohol and other substance use at the first treatment session were predictive of alcohol and other substance use at the last treatment session. Thus, a substantial portion of the positive behavior change from alcohol and other substance use to abstinence occurred during the post-baseline assessment and pre-treatment period. Similar to the findings obtained by both Epstein et al. (2005) and Morgenstern et al. (2007), study results are consistent with an assessment reactivity effect. Study limitations include post-hoc data analyses, self-reported data, and a correlational study design, which taken together necessitate cautious interpretation of study findings. In this regard, alternative explanations for the observed behavior changes cannot be ruled out. Participant motivation for change, for example, may have influenced both treatment seeking and reductions in alcohol and other substance use.

Irrespective of study design limitations, all three investigations of baseline assessment reactivity effects yielded results that are indicative of beneficial behavior change occurring post-assessment and pre-treatment initiation. Although the research designs used in each of these investigations preclude definitive conclusions, it is noteworthy that the investigators, across all three studies, concluded that the obtained study results were consistent with a research assessment reactivity hypothesis and that further investigation was warranted.

Post-Treatment Assessment Focus

Of the eight remaining studies, three (Clifford et al., 2000; Maisto et al., 1985; Ogborne and Annis, 1988) were previously reviewed (Clifford & Maisto, 2000) and are briefly summarized here to maintain continuity. The Ogborne and Annis (1988) and Maisto et al. (1985) studies were experimental investigations of the reactive effects of frequent follow-up contacts and self-monitoring procedures on treatment outcomes, and treatment length and follow-up style on treatment outcomes, respectively. Each of these studies failed to detect significant effects. However, inspection of the research methodologies used and statistical analyses performed reveals that each of these efforts lacked sufficient statistical power to demonstrate a subject reactivity effect. More specifically, based on an estimated moderate effect size of d = .50, three numerator degrees of freedom (i.e., four experimental conditions in each study), data analyses using analysis of variance (ANOVA) techniques, and sample sizes of approximately 12 and 19 subjects per condition, the resulting statistical power associated with the Maisto et al. (1985) and Ogborne and Annis (1988) studies approximated 25% and 40%, respectively. The more essential problem, however, was that the integrity of the experimental manipulations was not maintained. For example, in the Ogborne and Annis (1988) study only 53% of the original sample completed the 12-month follow-up interview and participant compliance regarding the self-monitoring procedures was very low (i.e., 13%). With respect to the Maisto et al. (1985) study, procedures were not employed to ascertain the fidelity of each of the two follow-up style groups. Thus, group differentiation with respect to experimental condition is unknown.

Clifford et al. (2000) reported alcohol treatment research assessment exposure subject reactivity effects based on secondary data analyses. Study participants were classified into one of three research assessment exposure conditions (i.e., regularly scheduled follow-up interviews, delayed year two follow-up interviews, and missed scheduled follow-up interviews). Study findings showed that subjects exposed to regularly scheduled follow-up interviews reported a significantly (Wilks’ Lambda = .608, p < .0005) greater percentage of abstinent days, lower mean number of drinks per day, and a lower percentage of heavy drinking (i.e., six or more drinks) days across the entire year two follow-up period relative to either the delayed year two or missed scheduled follow-up interviews groups. Alternatively, the delayed year two follow-up group experienced poorer alcohol use outcomes across the entire year two follow-up period compared to year one, and among the three groups contrasted, this group fared the poorest on all drinking measures by the end of the second year. Although study findings must be interpreted cautiously as study participants were not randomly assigned to research conditions, the results are consistent with the interpretation that alcohol treatment research assessment exposure influences alcohol use outcomes.

In a randomized trial conducted by Clifford and colleagues (Clifford et al. 2007; Maisto et al., 2007), 235 subjects meeting DSM-IV criteria for an alcohol use disorder (AUD) were randomized to one of four research assessment exposure conditions: infrequent-brief, frequent-brief, infrequent-comprehensive, or frequent-comprehensive. Study results provided direct experimental support for research assessment exposure related effects. For the criterion of treatment participation, the data showed that the likelihood of an individual presenting for outpatient alcohol treatment varied as a function of the comprehensiveness of the baseline research assessment battery (p < .05), as individuals assigned to the brief assessment conditions were almost twice as likely to be an outpatient alcohol treatment “no-show” as their counterparts assigned to the comprehensive assessment conditions (19.8% vs. 10.5%, respectively).

The comprehensiveness of the baseline assessment battery also affected utilization of intensive (i.e., detoxification, hospital inpatient, residential) alcohol/substance abuse treatment services (p < .05) such that individuals assigned to the brief assessment conditions were more likely than their comprehensive assessment counterparts to use intensive alcohol/substance abuse treatment services during the follow-up period.

With respect to alcohol and other drug use, individuals assigned to the infrequent-brief condition reported drinking more often, in greater quantity, and a greater frequency of heavy drinking than individuals assigned to other conditions (p < .05). Negative consequences resulting from alcohol use also varied as a function of research assessment exposure such that subjects assigned to the infrequent-brief condition reported greater negative consequences than subjects assigned to other conditions (p < .05). In addition, drug-taking behavior varied across the frequency of research assessment exposure factor. Individuals assigned to the infrequent research assessment exposure conditions reported using drugs significantly more often than their counterparts assigned to the frequent conditions (p < .001). Thus, the Clifford et al. (2007) study provides strong empirical evidence for research assessment related behavior change among adults presenting for treatment of an alcohol use disorder. In addition, these assessment-related changes in alcohol were not mediated by treatment participation (Maisto et al., 2007).

Kypri et al. (2007) screened college students attending a primary care clinic for hazardous drinking using the Alcohol Use Disorders Identification Test (AUDIT). Eligible (i.e., AUDIT score > 7) consenting individuals were randomly assigned to either an information only group, or to a group that received information followed by a 10-minute Web-based assessment four weeks later. In addition, both research groups were exposed to two follow-up assessments (i.e., 6-month and 12-month), and follow-up non-responders were sent reminder emails followed by a reminder telephone call. The results indicated that at the 12-month follow-up, but not at the 6-month follow-up, individuals exposed to the Web-based assessment reported significantly less alcohol consumption, to include heavy drinking episodes, and fewer alcohol related problems relative to individuals assigned to the information only condition. Study participant-research interviewer interactions (at recruitment and 2 consent interviews, one of which was for the initial assessment and the other, conducted after the initial assessment, for follow-up assessments), however, appear to have been more than minimal. Therefore, it is not clear to what extent other factors such as personal contact with a research interviewer may have contributed to the study’s findings. In this regard, the authors noted that the study hypotheses were not supported strongly and that the increased differences at 12-months, relative to the 6-month assessment, could not be explained easily. Given the limitations associated with the Kypri et al. (2007) study, reported findings cannot be viewed as strong evidence for assessment reactivity effects. It may be that the relatively brief assessment exposure (i.e., 10-minutes) accounts, at least in part, for the weaker assessment related reactivity effects.

Worden et al. (2008) conducted secondary data analyses to examine post-treatment follow-up data from a sample (N=102) of women receiving individual or couples CBT for AUD. Based on the information provided, it appears that Worden and colleagues used data from the same parent study as Epstein et al. (2005); two primary differences between the two investigations are the time period of interest (i.e., pre-treatment versus post-treatment) and the number of subjects retained for data analyses (i.e., based on reported degrees of freedom, it appears that the Worden et al. study retained only 25 (24.5%) study participants for data analysis). To investigate subject reactivity to the follow-up interview, data from the 9, 12, and 15-month follow-up interviews were grouped into three time periods: two weeks immediately prior to the completion of each follow-up interview, the two weeks immediately following the interview, and the two week period halfway between the follow-up under examination and the subsequent follow-up interview. Statistically significant follow-up reactivity effects were not detected for the 9-month and 15-month follow-up telephone interviews, although a reactivity effect was detected at the 12-month in-person follow-up interview. The authors concluded that more intensive in-person contact may be necessary for assessment related reactivity and that reactivity effects are not sufficient to substantially influence treatment outcomes. This conclusion stands in contrast to the findings that Heather et al. (1986) reported, in which the telephone sample showed greater improvement relative to the postal sample, as well as the findings reported by Clifford and colleagues (2000, 2007), in which research assessment exposure contributed to improved clinical outcomes and greater treatment utilization. These inconsistencies may have been due, at least in part, to a number of limitations associated with the Worden et al. study. Significant among these limitations are: 1) data analyses were based on alcohol use measures derived from the TimeLine Follow-Back (TLFB), which has not been validated for date specific data; 2) reduced design sensitivity to detect an assessment reactivity effect due to the exclusion of subjects who remained abstinent or consumed fewer than three drinks during a follow-up period; and 3) a failure to account for baseline assessment reactivity effects (Epstein et al. reported that 44% of the sample achieved abstinence prior to treatment initiation). Given these limitations, the study’s potential for detecting research assessment related reactivity effects appears minimal.

Hester et al. (2012) as part of their evaluation of a computer based intervention (i.e., College Drinker’s Checkup: CDCU) for heavy drinking college students (N=144) meeting NIAAA’s (2004) criteria for heavy episodic drinking, found that significant reductions in alcohol use and related problems were reported at both study follow-up points (i.e., 1-month and 12-month) not only by those students randomly assigned to the intervention group but also by those individuals assigned to an assessment only condition. In this study, four measures of alcohol use were used to ascertain program effectiveness (standard drinks per week, peak BAC in a typical week, average number of drinks in two heavy drinking episodes in the prior month, and average peak BAC in those two heavy drinking episodes). Heavy drinking was operationalized as four or more drinks on an occasion for females and five or more drinks on an occasion for males. In addition, alcohol related problems were assessed at the baseline, 1-month, and 12-month assessment points using the AUDIT, Brief Drinker’s Profile (BDP), and College Students Alcohol Problems (CSAP) instruments.

The unexpected reductions in alcohol use and alcohol related problems reported by students assigned to the assessment only condition coupled with their comments during follow-up assessments (e.g., “I never realized how much I was drinking” and “I never added it all up before”) led Hester and colleagues to design a second experiment that included a delayed assessment condition with the aim of investigating the effects of assessment reactivity on intervention outcomes and aiding the interpretation of derived findings. This delayed assessment experiment (N=82) contrasted the alcohol use reports of students randomly assigned to a CDCU intervention condition (n=42) with those of students assigned to a delayed assessment condition (n=40) at a 1-month post-intervention assessment point. A 98% follow-up rate was achieved with one subject assigned to the delayed assessment condition lost to follow-up. Based on the same alcohol use measures that were used in the initial CDCU evaluation, study results indicated substantially stronger effects (i.e., an average between group effect size of d=.82) for the CDCU intervention relative to the average between group effect size of d = .35 that was found in the initial evaluation study). The different effect sizes associated with these two studies is considerable and is consistent with an assessment reactivity effect. To further investigate assessment reactivity, secondary analyses were conducted that involved a comparison of the control (i.e., assessment) groups used in these two experiments. The results of these analyses showed that, although these two groups did not differ with respect to alcohol use at baseline, they differed significantly at the 1-month follow-up. In this regard, both the “Drinks per Week” and “Peak BAC in a Typical Week” varied significantly (p < .05) at the 1-month follow-up such that the delayed assessment correlated with the assessment reactivity study was associated with greater alcohol use. The decline in alcohol use associated with the initial assessment group was statistically significant for each of the four alcohol use measures. Alternatively, reductions in alcohol use associated with the delayed assessment condition were limited to measures of heavy use (i.e., average number of drinks in two heavy drinking episodes in the prior month, and average peak BAC in those two heavy drinking episodes). Summarizing their findings, Hester et al. (2012) indicated that the observed differences in ‘typical drinking’ among control group participants at the 1-month assessment, but not at baseline assessment, was most parsimoniously explained by participant assessment reactivity.

Summary of Assessment Reactivity Studies

Studies investigating alcohol treatment research assessment reactivity effects have yielded mixed results. Studies failing to yield statistically significant findings (i.e., Ogborne and Annis, 1988; Maisto et al., 1985) have been hampered by statistical and methodological shortcomings. Nevertheless, the directionality of findings from these studies suggests that an assessment exposure related behavior change hypothesis remains tenable. On the other hand, the Worden et al, 2008 and Kypri et al. 2007 studies provided weak evidence in support of assessment-related behavior change (e.g., alcohol use). Study limitations associated with each of these two studies, however, were sufficient to mask potential reactivity effects. The remaining five studies (Clifford et al. 2000; Clifford et al., 2007; Hester at al., 2012; Kaminer et al. 2008; Maisto et al., 2007) provide moderate-to-strong empirical evidence supporting research assessment exposure related behavior change. Overall, the evidence indicates that research assessment exposure influences clinical outcomes (e.g., reduced alcohol use) and simultaneously, reduces design sensitivity and hampers the interpretability of study findings.

Alcohol Treatments Involving Post-Treatment Patient Contact

Alcohol treatments involving during and post-treatment patient contact collect data that is given to clinicians for clinical purposes. McLellan and colleagues (2005), for example, have recommended that individuals engaged in outpatient addictions treatment be monitored on a regular basis and that the information collected be provided to clinicians immediately in an attempt to reduce attrition and improve outcomes, a process they call “Concurrent Recovery Monitoring” and view as an alternative approach to traditional addiction treatment outcomes evaluation. The Concurrent Recovery Monitoring approach can be described as an enhanced follow-up effort that combines evaluation and clinical monitoring. Along these lines, a number of AUD treatment intervention studies (e.g., Chestnut Health Systems Group) have reported improved clinical outcomes as a result of continued post-treatment therapeutic contact with patients. For example, Scott and Dennis (2002) developed a treatment protocol referred to as ‘recovery management checkups’, which involves quarterly follow-up contact with patients in an attempt to reconnect them, if necessary, with the treatment system. Individuals assessed as requiring further treatment are immediately transferred to a treatment linkage manager who, based on motivational interviewing techniques, assists the person with treatment reintegration (e.g., schedules treatment and arranges for transportation).

A randomized trial comparing individuals receiving Recovery Management Checkups with those receiving quarterly research follow-up assessments showed that the Recovery Management Checkups condition, at the 24-month follow-up, was associated with better patient management (i.e., a higher treatment readmission rate, greater treatment participation, and readmission to treatment earlier in the follow-up period; Dennis et al., 2003; Scott et al. 2005). In addition, patients in the Recovery Management Checkups condition reported better substance use outcomes (Scott et al. 2005). In this regard, study investigators noted that shifts from substance use to recovery that were related to post-treatment Recovery Management Checkups were largely mediated by the extent to which treatment was reinitiated (e.g., treatment readmission and readmission earlier in the follow-up period). Such mediation effects stand in contrast to the research assessment exposure reactivity effects reported by Clifford and colleagues in which research assessment exposure was shown to be causally related to both increased treatment participation and decreased alcohol use (Clifford et al., 2007). Changes in alcohol use related to exposure to research assessment, however, were not mediated by treatment participation (Maisto et al., 2007).

Studies of controlled continuing care intervention, similar to the work of the Chestnut Health Systems Group, have shown that extended post-treatment therapeutic contact with patients is beneficial, although not always in the manner hypothesized. For example, McKay et al. (2005) investigated the effectiveness of a telephone-based continuing care program for alcohol and cocaine dependent individuals. Individuals completing a four-week intensive outpatient program were randomized to one of three 12-week continuing care conditions (i.e., telephone-based monitoring and brief counseling intervention, individual relapse prevention, or standard group counseling) and followed for 24-months. Study hypotheses centered on a comparison between the telephone-based continuing care condition and the two more intensive face to face conditions, which were not expected to differ (i.e., main effect hypothesis). Study results, however, showed that individuals who failed to achieve a number of the initial outpatient treatment goals (e.g., abstinence from alcohol and cocaine, self-help group attendance) reported better outcomes if they received standard group continuing care as opposed to telephone-based continuing care. On the other hand, individuals who had made good progress toward achieving the outpatient treatment goals reported better outcomes when assigned to the telephone continuing care group as opposed to the standard continuing care group (i.e., an interaction effect).

Extended Case Monitoring interventions, which involve continued post-treatment telephone contact conducted on a tapering schedule with patients in an effort to prevent relapse or, if necessary, reintegrate the patient into the treatment system, have been investigated by Stout and colleagues (Stout et al., 1999; Stout et. al, 2001; Zweben et al., 2003). Extended Case Monitoring is a low-intensity, long-term intervention for alcohol problems based on specific counselor strategies, relapse prevention activities, and significant-other involvement (Zweben et al., 2003). To evaluate the effectiveness of Extended Case Monitoring, Stout and colleagues conducted a randomized clinical trial and hypothesized that Extended Case Monitoring would have a significant beneficial effect on treatment reengagement and severe relapse. End of study data analyses (unpublished), in contrast to preliminary data analyses (Stout et al., 2001; Zweben et al., 2003), revealed a significant interaction effect such that socially isolated individuals assigned to Extended Case Monitoring showed significantly greater improvement in alcohol use (i.e., percent days abstinence and mean number of drinks per drinking day), relative to controls (Stout, personal communication). Thus, both the McKay et al. and Stout et al. investigations of post-treatment continuing care interventions yielded findings that differed in significant ways from their original study hypotheses. In this regard, both studies yielded interaction effects suggestive of better clinical outcomes associated with more intensive post-treatment therapeutic contact among poorer prognostic patients (i.e., patients failing to achieve outpatient treatment goals and the socially isolated). Furthermore, McKay et al. reported that better prognosis patients do better when assigned to less intensive post-treatment contact.

It is not intended that the described studies provide a full review of alcohol treatment continuing care research. Rather, the described studies reflect the rigorous experimental investigations of extended post-treatment therapeutic contact with patients. Along these lines, McKay (2005) reviewed the alcohol and other substance use treatment literature to determine whether a case could be made for extended treatments for alcohol and other substance use disorders and it appears that the answer is a tentative yes. Due to the relatively small body of direct and indirect evidence available, however, additional studies still need to be conducted before any definitive conclusions can be made about the utility of extended addictions treatment.

In summary, there is increasing recognition among alcohol treatment researchers and clinicians that continued post-treatment contact with AUD patients is beneficial. Such understanding, however, is only at a general descriptive level, in that there is little understanding of how such beneficial effects occur. Moreover, little research has been conducted to identify the subpopulations of patients most likely to benefit from these interventions, or how such interventions can be manipulated (i.e., controlled) for research and clinical purposes.

Assessment Reactivity and Clinical Interventions Involving Post-Treatment Data

Clinical interventions that use post-treatment data collection to inform clinical decision-making and research assessment interviews share certain characteristics. For example, both involve sustained post-treatment contact and the assessment of patient functioning, which are the more salient aspects of both research assessment interviews and post-treatment clinical intervention related data collection.

Despite these two important similarities, there are essential differences between assessment reactivity and post-treatment data collection to inform clinical practice. First, these two lines of inquiry developed independently. Second, clinical practice rarely involves programmed post-treatment contact with patients, as few programs have the necessary resources to maintain such contact, and relatively few research studies involve post-treatment clinical, as opposed to research, contact with patients, although the number of such studies have increased recently (Dennis et al., 2003; McKay et al., 2005; Scott et al., 2005; Stout et al. 1999; Stout et al., 2001; Zweben et al., 2003). A third difference is that the ongoing use of data collection activities for clinical purposes involves post-treatment interventionist contact, scheduled according to patient risk, aimed at relapse prevention and treatment reintegration, whereas research follow-up assessments involve no direct attempt to vary follow-up contact or influence the course of AUD. Fourth, it is not clear to what extent the same mechanisms of behavior change are responsible for the therapeutic benefits derived from each of these activities. For example, Scott et al. (2005) found that treatment participation mediated the relationship between improved clinical outcomes and post-treatment data collection activities. Maisto et al. (2007), on the other hand, reported that treatment participation did not explain the clinical benefits associated with research assessment exposure. Such contradictory findings suggest different mechanisms of behavior change may be responsible for the clinical improvements associated with each of these two post-treatment data collection activities.

Closing Comments

It seems clear that assessment reactivity and post-treatment data collection for purposes of informing clinical interventions have considerable and potentially important differences to go along with their salient similarities. Both, however, are important because, at least under some conditions, they seem to contribute to improved clinical outcomes. Assessment reactivity also is of methodological importance because it can affect the sensitivity of clinical trials and other research designs. The available research on each topic shows that we have only a rudimentary level of understanding in each of these areas. Research is needed to achieve the goals of identifying underlying behavior change mechanisms and effect boundaries associated with each type of contact as well as the conditions under which such effects are most likely and of significant magnitude. At this time, viewing each area of inquiry as separate from the other seems the most likely way to generate programmatic research across investigators to achieve these goals.