Reactivity to Alcohol Assessment Measures: An Experimental Test

Abstract

Aims

Previous research has suggested that alcohol screening and assessment may affect drinking.

Design

This study was a randomized test of reactivity to alcohol assessment questionnaires among a group of heavy drinking college students.

Setting and Participants

A total of 147 university students completed a screening questionnaire and were randomized to either immediate assessment or delayed assessment. The immediate assessment group completed a set of drinking questionnaires at baseline, 3, 6, and 12 months, while the delayed assessment group completed questionnaires only at 12 months.

Measurements

Primary outcomes included overall volume of drinking, risky drinking, and use of risk reduction behaviors.

Findings

We found a significant effect of assessment on measures of risky drinking and risk reduction behaviors, but not on overall volume of drinking. Specifically, at 12 months, participants who had previously completed drinking assessments had a lower peak BAC (d=−.373), were more likely to report a low score on the Alcohol Use Disorders Identification Test (AUDIT; odds ratio = 2.55), and tended to use more strategies to moderate their alcohol consumption (d=.352). Risk reduction behaviors that were affected tended to be those that limited alcohol consumption, rather than those that minimized consequences.

Conclusions

These results may have implications for the development of brief interventions.

INTRODUCTION

In the field of alcohol treatment research, there has long been interest in whether the assessment of drinking reduces drinking behavior. A number of intervention researchers have speculated that the assessment of drinking contributes to the positive effects of drinking interventions (1–5). For instance, Sobell and Sobell (6) hypothesized that their follow-up interviews might be operating as a de facto treatment, and thus might be contributing to reductions in drinking behavior. Likewise, in light of the failures to find differences between conditions in Project MATCH (1) there was speculation that the substantial assessment procedures, among other features, might have reduced drinking across experimental conditions. In other studies, assessment reactivity is commonly invoked to explain intervention findings when there is a downward trend in both the intervention and control groups (7). Unfortunately, only a handful of investigators have used an experimental design to directly test the hypothesis that assessment of drinking reduces drinking behavior.

One way to test for effects of assessment on drinking is through the use of a delayed (or minimal) assessment control. In this design, participants are screened into a research study, with one group completing only minimal assessments during the research period, and another group completing more extensive assessments. In a recent study, college students who received a score of 8 or more on the Alcohol Use Disorders Identification Test (AUDIT) were randomized to complete or not complete a 10–minute drinking assessment (8). Assessment questions included a 14-day drinking calendar, alcohol problems, and perceived drinking norms. At 12-months, students who completed the assessment battery showed lower overall drinking and heavy episodic drinking. Similarly, McCambridge and Day (9) randomized a group of college students to either complete or not complete the AUDIT as part of a demographic and health survey. At a two-month follow-up, those who had completed the AUDIT reported lower levels of hazardous drinking (but not overall drinking). Additionally, in a study of college students, Carey et al. (10) found that an extensive Timeline Followback (TLFB) interview was more effective in reducing drinking at 1 month than a briefer assessment. However, the TLFB condition did not reduce drinking-related problems more than the shorter assessment. These studies provide experimental evidence that drinking assessment can reduce self-reported drinking behavior, with effect sizes on risky drinking that are similar to those reported in studies of brief interventions (11). However, there are exceptions to these findings, with some investigators failing to find an effect of assessment on drinking (2, 12).

Understanding the effects of assessment on drinking behavior is potentially of great significance. First, identifying the contribution of the assessment to outcome allows us to better estimate the incremental effect of the treatment being tested in intervention trials. If assessment does indeed affect drinking, research results might be routinely overestimating the effect of the treatment when it is delivered without a substantial assessment battery. It might also help explain the large number of treatment trials in which participants in all groups show improvement. Second, if assessment itself has a meaningful effect on drinking, such knowledge might be helpful in developing efficacious and cost-effective brief interventions for drinking. Indeed, there has been a frequent call for brief treatments that can be implemented with minimal time and effort (7).

The present study was designed to build on previous assessment literature. Specifically, this was among the first studies to employ an experimental design to examine assessment effects on three types of drinking outcomes: overall volume of alcohol consumption (i.e., drinks per week), risky drinking (i.e., peak blood alcohol level and AUDIT score), and risk reduction behaviors (e.g., alternating alcoholic and non-alcoholic drinks). This study differs from past studies that have examined assessment effects on drinking (2, 9, 12) in that our assessment condition involved repeated assessments over time. This was also the first study to our knowledge to examine the affect of assessment on risk reduction behaviors.

METHODS

Participants

In Fall 2006 and Spring 2007, we recruited participants from a medium-sized private university in the southern United States. Participants were recruited as part of a larger clinical trial comparing motivational interviewing and feedback interventions (13). The evaluation of assessment effects was a secondary aim that was built into the larger clinical trial. Several methods were used to reach potential participants, including brief announcements in undergraduate psychology and health courses, invitation emails to students enrolled in undergraduate psychology courses, and flyers posted on campus. To be eligible, participants had to be at least 18 years of age and report at least one heavy-drinking “binge” episode (i.e., 5 or more drinks for men, 4 or more drinks for women, in a single episode) in the past 2 weeks. For completing assessments, participants could receive either psychology course extra credit or payment ($20 at baseline, 3 months, and 6 months; $40 at 12 months). For the parent study, 363 eligible students consented to participate; the present analyses used 147 participants allocated to one of the assessment groups. Other study conditions have been described elsewhere (13).

To determine eligibility, students answered two screening questions about age and frequency of heavy drinking in the past two weeks. The screening and all assessments were completed online via a secure website. Eligible participants were randomized using a computerized algorithm (stratified by sex and heavy drinking frequency) to Immediate Assessment (IA; n = 72) or Delayed Assessment (DA; n = 75). At baseline, both groups completed a short set of demographic questions, and the IA group completed an additional set of questions focused on alcohol consumption, alcohol-related problems, protective behaviors, readiness to change, and beliefs about campus drinking norms. The demographic component took approximately 5 minutes and the alcohol-related component took approximately 25 minutes. The IA group also completed the full set of questions (demographic and alcohol-related, taking approximately 30 minutes to complete) at 3, 6 and 12 months, while the DA group did not complete any additional assessments until 12 months. The informed consent and other documents explained the project as a research study looking at college student drinking; participants were told that they might or might not receive an intervention as part of the project. Neither of the groups in the present analyses received any formal intervention as part of their participation in this study. Eighty-seven percent of participants completed the 12 month follow-up (IA, n = 63; DA, n = 66), with no significant differences between groups in terms of follow-up rate. There were no adverse events reported in either group.

Participants were 66.0% female, 85.7% White, and had a mean age of 19.8 years. Freshmen represented 41.5% of the sample, sophomores 24.5%, juniors 21.8%, and seniors 12.2%. The majority of students reported living in dorms/residence halls on campus (47.6%) and off-campus housing (43.5%). On the baseline screener, participants reported an average of 3.1 heavy drinking episodes in the previous two weeks. There were no significant differences between the groups in the number of heavy drinking episodes or any of the measured demographic variables (all p’s>.05).

Measures

Volume of alcohol consumption was estimated using a 7-day drinking calendar modified from the Daily Drinking Questionnaire(14). Participants estimated the number of drinks they consumed on each day in a typical week in the past month. The number of drinks per week was calculated by summing the drinks reported on all seven days.

Risky drinking was measured in two ways. To estimate peak blood alcohol concentration (BAC), participants were asked to think of the time when they drank the most in the past month and then report the number of drinks they had and the number of hours over which they were drinking. The number of drinks and number of hours were used along with the participant’s weight and gender to estimate a peak BAC in the past month. As a second measure of risky drinking, we used the AUDIT questionnaire (15), which has shown good reliability and validity among college drinkers (16–18). Consistent with past research (9) we examined the AUDIT both as a continuous score (ranging from 0–40), as well as a dichotomized score (0–7 vs. 8 or more). Previous research has typically used a cutoff of 8 as a measure of risky drinking (19).

To measure risk reduction behaviors, we used the 15-item Protective Behavioral Strategies Survey (PBSS; 20). We chose this measure because we believed that many of the items would be closely related to risky drinking behaviors. Participants were asked to what extent they had used various protective behaviors while drinking in the past 3 months. Responses were given on a 5-point Likert scale of “Never” to “Always”. Examples of these behaviors include alternating alcoholic and non-alcoholic drinks, avoiding drinking games, and using a designated driver. Martens et al. (20) divided the PBSS scale into three subscales based on factor analysis (limited drinking, changing manner of drinking, using harm reduction strategies), while Walters et al. (21) found that for heavy drinking college students, the limited drinking subscale could be further subdivided (mixing beverages more weakly, planning ahead).

Data Analysis

We used all available cases for analysis (i.e., those who completed the 12-month follow-up). Assessment effects were analyzed by regressing each outcome (logistic regression in the case of the dichotomized AUDIT score) on the two groups of interest, adjusting for the number of binge episodes reported on the baseline screener. Adding gender and age (the other screening variables) as covariates did not change the pattern of results, and were thus not used in analyses. We used Cohen’s d as a measure of effect size (22).

RESULTS

Table 1 shows the means and standard deviations for the binge variable at screening and drinks per week, peak BAC, AUDIT, and PBSS at the 12 month follow-up. Table 2 shows the effect of assessment on drinking and related outcomes. In terms of overall consumption, there were no significant differences between groups on drinks per week, (t=−.673, p=.502). For risky drinking, there were significant differences between groups on peak BAC (t=−2.108, p=0.037, effect size=−.373) and the dichotomized AUDIT score (Wald chi-square=5.457, p=.019). At 12 months, the IA group reported a mean peak BAC of 96 mg/dl, whereas the DA group reported a mean peak BAC of 120 mg/dl, a between-group difference of 28 mg/dl after adjusting for the number of heavy episodes reported on the screener. Fifty six percent of the IA group scored less than 8 on the AUDIT, whereas 39% of the DA group scored less than 8 on the AUDIT, which means that the IA group was 2.55 times more likely to report a low AUDIT score when adjusting for the number of heavy episodes reported on the screener. The effect on the continuous AUDIT score fell just short of the conventional .05 significance level (t=−1.916, p=.058). In terms of risk reduction behaviors, there was a significant difference between the groups on the PBSS (t=1.992, p=0.049, effect size=.352), with assessment increasing the PBSS score by approximately 3.6 units when adjusting for the number of heavy episodes reported on the screener. Assessment had a significant effect on one PBSS subscore (limiting amount of drinking by mixing beverages more weakly) and a nearly significant effect on another subscore (changing manner of drinking). In post hoc analyses, we found that participants who had completed the assessment were more likely to alternate between alcoholic and non-alcoholic beverages, put extra ice in drinks, and avoid drinking games.

Table 1.

Screening and 12 month follow-up data by group.

	Immediate Assessment	Delayed Assessment
	(n=63)	(n=66)
	Mean (SD)	Mean (SD)
Screening question
Heavy drinking (binge) episodes	3.24 (1.85)	2.94 (1.63)
12 month follow-up
Drinks per week	9.52 (9.98)	9.98 (7.97)
Peak BAC	96 mg/dl (75)	120 mg/dl (84)
AUDIT	7.98 (4.63)	9.26 (5.34)
Dichotomized AUDIT score (% < 8)	56%	39%
Protective Behaviors score	50.10 (10.97)	46.92 (9.74)

Table 2.

Effect of assessment on drinking and related outcomes, adjusted for baseline binge drinking.

					95% CI
Measure	b	t	p	d	lower	Upper
Overall Drinking
Drinks per week	-1.007	-.673	.502	-.119	-3.967	1.953
Risky Drinking
Peak BAC	-.028	-2.108	.037	-.373	-.055	-.002
AUDIT (continuous)	-1.585	-1.916	.058	-.339	-3.223	.052
AUDIT (dichotomized, lower category)		5.457a	.019	2.553b	1.163	5.604
Risk Reduction
Protective Behavior score	3.568	1.992	.049	.352	.023	7.114
Subscale 1: Limiting Drinking c	1.708	1.631	.105	.288	-.365	3.780
Part 1a. Mixing Beverages More Weakly:	1.400	2.802	.006	.495	.411	2.389
Item 3: Alternate alcoholic and non-alcoholic drinks	.535	2.491	.014	.440	.110	.960
Item 11: Drink water while drinking alcohol	.308	1.473	.143	.260	-.106	.723
Item 12: Put extra ice in your drink	.557	2.683	.008	.474	.146	.968
Part 1b. Planning Ahead:	.307	.460	.646	.081	-1.014	1.629
Item 2: Determine not to exceed a set number of drinks	.308	1.597	.113	.282	-.074	.691
Item 4: Have a friend let you know when you have had enough to drink	.120	.502	.617	.089	-.354	.595
Item 6: Leave the bar/party at a predetermined time	-.065	-.319	.750	-.056	-.465	.335
Item 10: Stop drinking at a predetermined time	-.057	-.281	.779	-.050	-.457	.343
Subscale 2. Changing Manner of Drinking	1.365	1.940	.055	.343	-.027	2.757
Item 5: Avoid drinking games	.923	4.002	.000	.708	.467	1.379
Item 9: Drink shots of liquor	.034	.197	.844	.035	-.311	.380
Item 13: Avoid mixing different types of alcohol	-.010	-.047	.963	-.008	-.437	.417
Item 14: Drink slowly, rather than gulp or chug	.354	2.023	.045	.358	.008	.699
Item15: Avoid trying to 'keep up' or 'out-drink' others	.064	.339	.735	.060	-.311	.440
Subscale 3. Using Harm Reduction Strategies	.496	1.199	.233	.212	-.322	1.314
Item 1: Use a designated driver	.096	.546	.586	.097	-.253	.445
Item 7: Make sure that you go home with a friend	.195	.957	.341	.169	-.208	.599
Item 8: Know where your drink has been at all times	.204	1.183	.239	.209	-.138	.546

^aWald Chi-square

^bOdds Ratio

^cSubscale 1: Limiting Drinking, divided into subscores 1a and 1b as suggested by Walters et al. (21).

DISCUSSION

This study was a randomized test of effects of the assessment of drinking behavior on heavy drinking college students. We found a significant assessment effect on measures of risky drinking and risk reduction behaviors, but not on overall volume of drinking. Specifically, participants who completed drinking assessments at baseline, 3, and 6 months, reported reduced peak BAC and AUDIT scores, and tended to use more strategies to moderate their alcohol consumption at 12 months, compared with participants in the delayed single assessment group. Specifically, assessment reduced peak BAC by 28 mg/dl, and more than doubled the odds of having a low AUDIT score.

As previously noted, examination of assessment effects was a secondary aim of the parent study, which was designed to detect “medium” effect sizes (d=.5) of interventions. In this arm of the trial, we found significant effects with effect sizes greater than .35. In comparison, in their meta-analysis, Moyer et al. (11) reported an average effect size of .263 for brief intervention studies at 6–12 months post-intervention. In our study, IA had an estimated effect greater than this on four additional outcomes beyond those we reported as significant: continuous AUDIT, two protective behavior subscores, and one protective behavior item. Thus, there may be additional assessment effects beyond those reported here.

Our pattern of results on drinking is consistent with previous research on assessment reactivity, which tends to find more of an effect on risky drinking than on overall quantity (8, 9). Also consistent with previous work is our pattern of findings on the AUDIT questionnaire. Like our study, McCambridge and Day (9) reported more significant effects when the AUDIT was dichotomized. This may be due in part to the scoring of the AUDIT; three of its items pertain to volume and frequency of heavy drinking, such that frequent bingeing virtually guarantees a score of 6 or more, even in the absence of adverse consequences. Thus, it is possible that lower AUDIT scores are more reflective of overall drinking patterns, while higher AUDIT scores are more indicative of dependence or adverse consequences.

To our knowledge, we are the first to test the effect of assessment on measures of risk reduction behaviors. Importantly, the kind of items that showed the most improvement in our study tended to be behaviors that limited alcohol consumption, such as mixing drinks more weakly, alternating beverages, and avoiding drinking games. Other behaviors that were more related to minimizing the consequences of drinking, such as leaving with friends and knowing where your drink had been at all times, tended not to be affected. Indeed, in previous research some risk reduction items (e.g., know where your drink has been at all times) seem to be unrelated to overall drinking or peak BAC, whereas other items (e.g., use a designated driver) may actually increase the amount of drinking (21). However, another explanation may be that we encountered a ceiling effect in some risk reduction questions. For instance, “using a designated driver” and “knowing where your drink had been at all times” tended to be highly endorsed, even at baseline (used frequently by 73.6% and 86.1% of IA participants, respectively). In contrast, items such as alternating alcoholic and non-alcoholic drinks and using extra ice were endorsed much less often at baseline (used frequently by 20.8% and 22.2% of participants, respectively). Thus, it is possible that there was little room for improvement on some items that were already used quite often. At the same time, this may suggest that viewing the less endorsed behaviors served as prompts or reminders for those who were completing the questionnaires and had not yet adopted these behaviors.

Our study was strengthened by the use of a randomized prospective design, a long-term follow-up, and an excellent retention rate. Our study was limited in terms of the selectivity of sample and the self report nature of our questionnaires. Our sample was relatively young and mostly female and White. In addition, because drinking was self-reported, it is impossible to tell whether the effect of the assessment is a reporting bias or a true effect on drinking. Future research might help determine the nature of the effect by comparing self-reports to collateral interviews or biochemical measures. Finally, our IA group included repeated assessments of drinking, consequences, and risk reduction behaviors at baseline, 3 and 6 months. This schedule makes our study somewhat different from past studies that used a single assessment point. It also makes it impossible to determine whether we would have found similar effects as a result of a single assessment point or whether the effects will persist longer than 6 months from the final follow-up point. Future studies might consider varying the content, length and number of assessment points to determine the contribution of each element.

Moos (23) has thoughtfully summarized some of the mechanisms by which assessment might affect drinking. One possible mechanism, based on self-regulation theory, involves a process by which people monitor and compare their own behavior to some personally-held standard or norm; according to this theory, focused attention on one’s drinking may cause an individual to alter his or her behavior to be more in line with a personally held standard or norm (24). This perspective is consistent with social learning theory, which suggests that behavior is influenced by actual consequences of behavior, as well as observations and self-reflection about potential consequences (25). Through reminding people about potential consequences and possible strategies to reduce risk, assessment may help them not only to adopt new behaviors, but also to remember behaviors that they observe in others. In addition, self determination theory (26) suggests that people are more likely to make changes that are perceived as having been freely chosen (autonomy), when they believe they are competent to make changes (competence), and when those changes are supported by others (relatedness). Assessment batteries which facilitate self-reflection, while at the same time avoiding direct appeals to change, would seem to further the perception of autonomy. Many young adults are already optimistic about changes (27), and providing a menu of strategies for reducing heavy drinking may further their perception of competence.

In conclusion, we found that an assessment battery served as a de facto intervention, reducing levels of heavy drinking and increasing risk reduction behaviors. We found between-group effects that were similar to previous studies of assessment (as well as brief interventions). Subsequent intervention studies might consider including delayed or minimal assessment groups in order to determine how generalizable the observed effects will be to settings in which there is not a substantial assessment component. Our results might also have implications for the development of brief cost-effective interventions. That is, it might be possible to create effective interventions for drinking behavior that only involve the assessment or monitoring of drinking behavior. Assessment interventions may be especially well suited for younger drinkers, who tend to be more concerned about normative comparisons and less practiced in strategies for safe drinking.