Assessment reactivity to follow-up in a study of women’s treatment for alcohol dependence

Abstract

Little research has been conducted on symptom reductions in response to assessments in clinical trials, despite the impact such reactivity may have on interpretation of outcomes. Reactivity to data collection procedures during post-treatment follow-up may obscure treatment effects. The current study examined whether female participants (n=102) in a trial of cognitive-behavioral treatment for alcohol dependence had lower drinking quantity and frequency immediately after participating in follow-up assessments. Repeated measures ANOVAs were used to compare each participant’s drinking among two-week time periods immediately before the follow-up, directly after the follow-up, and between follow-ups. No assessment reactivity was found for 9 or 15 month follow-up interviews, but was suggested at a 12 month in-person interview.

1. Introduction

In assessment reactivity, patients demonstrate symptom reduction in response to administered measures or evaluations (Epstein et al., 2005). Some researchers (e.g. Clifford & Maisto, 2000; Heather, 1999) have argued that the generalizability of follow-up data becomes questionable in the face of reactivity effects. Whereas research subjects in clinical trials typically participate in several follow-ups, patients in community settings rarely are contacted after treatment has ended.

The presence of reactivity effects would suggest that a control group receiving follow-ups is a group that is receiving an effective intervention, which could obscure differences between the treatment group and the “control” group, or between two different treatment groups. There have only been two studies of possible assessment reactivity in response to follow-up procedures. Both studies compared a condition receiving frequent follow-ups with a condition receiving infrequent follow-ups. The first (Ogborne & Annis 1988) found no differences between the follow-up frequency groups. In the second study (Clifford, Maisto, Franzke, Longabaugh, & Beattie, 2000), the intensive follow-up group had significantly more abstinent days, a lower percentage of heavy drinking days, and a lower number of standard drinks consumed on drinking days than individuals in the nonintensive follow-up group.

The current study had two main objectives. The primary objective was to determine if reactivity effects were occurring in response to assessments at post-treatment follow-ups. The primary hypothesis was that participants would have a lower percentage of days drinking, mean drinks per day, and percent days of heavy drinking in the two weeks directly after the follow-up assessments. However, given that participants were informed of the intervals at which they would be assessed, participants may have become accustomed to follow-up procedures, and therefore might show anticipatory reactivity prior to the assessment. The secondary aim of the study was to test for anticipatory assessment reactivity.

2. Method

2.1. Participants

The data were taken from 102 women participating in a larger clinical trial of cognitive-behavioral treatments for alcohol dependence (McCrady, Epstein, Cook, & Jensen, in preparation). The majority of participants (95%) identified themselves as Caucasian. Participants ranged in age from 28 to 68, with a mean age of 45 (SD=9.18).

2.2. Measures

Analyses for the present study focused on the daily drinking data collected with the Timeline Followback Interview (TLFB; Sobell & Sobell, 1996). The TLFB is an interview that takes approximately 10–30 min to complete and allows the researcher to collect retrospective daily drinking information by asking the drinker to try to recall the quantity and type of drinks he or she consumed each day. Interviewers were trained to administer the TLFB in the manner recommended by Sobell and Sobell (1992). The TLFB has been shown to have high test–retest reliability, and moderate convergent validity (Sobell & Sobell, 1992).

2.2.1. Procedure

Participants in the clinical trial first participated in a phone screening, lasting approximately 10 min. Participants eligible for the study then attended a clinical screen interview, lasting approximately 90 min; and a baseline interview, lasting about 120 min. Participants were then randomized to either a couples or individual condition of up to 20 sessions of outpatient treatment over six months. Participants completed follow-ups every three months for 18 months post-baseline. Follow-ups were collected by phone at 3, 9, and 15 months; in person at 6, 12, and 18 months.

Because of the need for thorough follow-up data in the time windows under examination, data for the current analyses were taken only from women who had complete drinking data during these windows. Because of the potential for obscuring patterns of reactivity among participants who did drink, participants also were excluded if they had less than 3 days of drinking in the follow-up period. To avoid confounding the assessment reactivity with treatment effects, the follow-ups included in the analyses were post-treatment, including follow-ups at 9, 12, and 15 months post-baseline. The 18-month post-baseline follow-up data were not included in the analysis because there were no data to examine after the last follow-up interview.

To examine the hypothesis that better drinking outcomes should be observed in the period of time directly after the follow-up, repeated measures analyses of variance (ANOVAs) were calculated. These ANOVAs compared within-subject drinking in three periods: the two weeks directly before completion of each follow-up, the two weeks directly after the follow-up, and the two weeks that were halfway between the follow-up under examination and the subsequent follow-up. This latter time point was chosen to ensure that the periods of time surrounding the follow-up would be compared with a time in which reactivity should theoretically not be present. Data from the 9, 12, and 15 month follow-up periods were collapsed for each subject. The primary dependent variables were the percentage of days on which the woman drank, mean drinks per day, and percent days of heavy drinking (defined here as greater than 3 drinks/day).

3. Results

Separate repeated-measures ANOVAs were conducted using three dependent variables: percent days drinking (PDD), mean drinks per day (MDD), and percent days heavy drinking (PDH). There were no significant differences between the two weeks before, after, or between the follow-ups on any of the dependent variables (Table 1).

Table 1.

Means and standard deviations of outcome variables

Follow-up	Time window	PDD		MDD		PDH
		Mean	SD	Mean	SD	Mean	SD
Entire FU period	Before	38.49	37.63	1.41	1.56	14.63	24.79
	After	33.63	35.85	1.52	1.96	15.23	26.42
	Between	40.73	38.29	1.48	1.73	14.18	26.88
9 months	Before	34.52	31.65	1.17	1.55	13.24	23.54
	After	38.49	38.87	1.77	2.48	16.60	30.03
	Between	44.81	45.93	1.75	2.13	17.86	32.72
12 months	Before	39.71	37.24	1.50	1.64	12.28	22.64
	After	36.78	32.97	1.05	1.11	7.72	20.40
	Between	40.46	30.71	1.28	1.16	10.00	21.92
15 months	Before	43.15	33.34	1.62	1.78	19.05	28.85
	After	40.48	35.41	1.65	1.50	21.13	25.84
	Between	34.89	33.16	1.31	1.62	13.34	22.33

PDD = Percent Days Drinking; MDD = Mean Drinks Per Day; PDH = Percent Days Heavy Drinking.

Analyses also were broken down by follow-up. When a repeated-measures ANOVA was conducted on the 9-month follow-up alone, there were no significant effects. However, within-subjects contrasts suggested on both PDD and MDD resulted in nonsignificant linear trends (p<.09) at 9 months, suggesting that drinking increased with time.

When a repeated-measures ANOVA was conducted on the 12-month follow-up alone, there was not a significant interaction or significant main effects. Within-subjects contrasts did reveal a significant quadratic trend, F(1, 24)=5.05, p=.03, that was different from the trend at 9 months. Consistent with a hypothesis of reactivity, the quadratic pattern suggested that MDD were lowest immediately after the follow-up, as compared with the two weeks immediately before the follow-up and in the two weeks between the 12 and 15-month follow ups.

When a repeated-measures ANOVA was conducted on the 15-month follow-up period alone, no significant differences were found on PDD, MDD, or PDH.

4. Discussion

The current study examined whether female participants in a clinical trial of alcohol dependence treatment would show assessment reactivity to follow-up procedures. It was predicted that the participants would show reduced drinking quantity and frequency in the two weeks immediately after the follow-up interviews, compared with the two weeks before the follow-up interviews and the two weeks directly between scheduled follow-ups (that is, the time period furthest away from any follow-up interview). Analyses included dependent variables of percent days drinking, mean drinks per day, and percent days of heavy drinking.

Overall, the analyses did not reveal immediate reactivity to follow-up interviews. However, when the 12-month follow-up period was analyzed separately, a possible reactivity effect was seen. In analyses on both untransformed and transformed data, the analyses on the 12-month follow-up period suggested a pattern of reactivity such that the mean drinks per day were highest immediately before the follow-up, lowest immediately after the follow-up, and moderate when between follow-ups. It is interesting that this trend was found at the 12-month follow-up, since follow-ups were conducted in-person at 12 months, rather than by phone as were 9 and 15 month follow-ups. It is possible that more intensive in-person contact is necessary for reactivity to appear.

There are some limitations to the current study. It is possible that participants who did not complete follow-ups were different from those that did. Individuals who were excluded from the current analyses had a higher mean drinks per drinking day at baseline. However, on other drinking variables assessed at baseline, such as percent abstinent days and percent heavy drinking days, included participants did not differ from excluded participants. Additionally, in order not to obscure a pattern of reactivity, individuals who had less than three drinks in a follow-up period were excluded from the analysis. This led to the exclusion of the many participants who had remained abstinent, and who may have found the follow-ups helpful in maintaining abstinence, and also could have shown low-level reactivity during the follow-up.

In conclusion, the current study found evidence for some assessment reactivity after an in-person two hour follow-up conducted 12-months post-baseline, but not for briefer follow-ups conducted by phone at 9 and 15 months post-baseline. When the in-person follow-up was examined alone, a significant quadratic trend resulted, which was suggestive of reactivity. It is possible that additional reactivity effects would be found with a larger sample size. Although these results suggest a small amount of reactivity to follow-up procedures, it is likely that there is not a reactivity effect in response to common follow-up procedures that is pronounced enough to substantially influence treatment outcome reports.