Unpacking Personalized Feedback: An Exploratory Study of the Impact of Its Components and the Reactions It Elicits Among Problem Drinking Men Who have Sex With Men

Abstract

Personalized feedback (PF) has demonstrated effectiveness in reducing drinking. Few studies have examined its effectiveness with adult problem drinkers or its potential mediators or moderators, including developing discrepancy. This study aimed to identify potential mediators and moderators of PF provided to adult problem drinking men who have sex with men (PDMSM).

Method:

An exploratory analysis of PF provided to PDMSM in the context of modified behavioral self-control therapy (N = 90). The association of individual items of PF, severity of PF, and independently rated, in-session participant reactions to PF with drinking outcomes (mean drinks per drinking day, MDDD) were examined using correlations and logistic and linear regression.

Results:

Significant pre–post differences in MDDD emerged. Other drug risk, family risk, and having an abnormal liver enzyme test result were significantly associated with proxies for developed discrepancy in expected directions; however, no PF item or reaction to PF predicted drinking outcomes. Severity of PF was not associated with participant reactions or drinking outcome.

Conclusions:

PF may be an effective intervention for PDMSM. Further research is needed to identify potential mediators and moderators of PF among adults.

INTRODUCTION

Personalized feedback (PF) is a brief intervention designed to provide individuals with information about their drinking patterns and compare them to general population norms. PF is implemented via a myriad of modalities (e.g., web-based, mailed, in-person; Walters & Neighbors, 2005; Walters & Woodhall, 2003; White, 2006) and studied among a variety of populations, most extensively with college students (Walters & Neighbors, 2005; White, 2006). PF may be imbedded within another therapeutic intervention, such as motivational interviewing (MI, Murphy et al., 2004; Walters & Neighbors, 2005) and psychoeducation (Walters, Bennett, & Miller, 2000), or utilized as a standalone intervention. Its structure varies in complexity. PF ranges from simple feedback about drinking norms (e.g., Agostinelli, Brown, & Miller, 1995) to extensive, detailed feedback that includes information about personal risk factors, family risk, tolerance, and consequences (e.g., Walters & Woodhall, 2003). Only one study, Project MATCH, included biological measures of potential liver damage and disease as a part of PF (Miller, Zweben, Di-Clemente, & Rychtarik, 1992). Across all modalities (e.g., mailed, computerized, or in-person), contexts (e.g., within therapy or alone), and structures (e.g., simple or detailed), PF has demonstrated a preponderance of success in helping individuals reduce their drinking at statistically significant levels (Cunningham, Wild, Bondy, & Lin, 2001; Doumas & Hannah, 2008; Miller, Benefield, & Tonigan, 1993; Walters & Neighbors, 2005; Walters, Vader, Harris, Field, & Jouriles, 2009; White, 2006; White, Mun, Pugh, & Morgan, 2007).

With its efficacy demonstrated, more recent studies of PF have focused on its potential mediators and moderators. In the context of MI, PF is thought to help an individual increase his awareness of the discrepancy between what he imagines he is drinking (self-ideal discrepancy) and its intensity in relation to his peers (normative discrepancy; Miller & Rollnick, 2002; Murphy, Dennhardt, Skidmore, Martens, & McDevitt-Murphy, 2010). Theoretically, heightened awareness of discrepancy creates a threshold of concern or discomfort with the status quo and thus motivates an individual for change. Some studies demonstrate changes in perception of drinking norms as a significant mediator of PF in reducing drinking (Borsari & Carey, 2000; Neighbors, Larimer, & Lewis, 2004; Walters, Vader, & Harris, 2007; Walters et al., 2009), while other studies have not (e.g., Larimer et al., 2007; Neal & Carey, 2004). One study discovered that while change in perceived norms was associated with increased discrepancy and intentions to change alcohol use, it did not necessarily lead to reduced drinking (Neal & Carey, 2004).

Certainty of the veracity or accuracy of PF (e.g., belief that the norms used for drinking comparisons are true) may play a role in PF’s efficacy. An economic theory called decision dilemma theory (DDT) posits that poor decision making is the result of having “equivocal” feedback (Bowen, 1987; Hantula & DeNicolis Bragger, 1999) or “feedback for which multiple (positive or negative) interpretations can be constructed” (Bowen, 1987, p. 56). Amrhein and colleagues (Amrhein, Miller, Yahne, Palmer, & Fulcher, 2003) applied this theory to continued investment in drug use, in which individuals committed more to stable or increased drug use despite receiving PF. The authors noted that drug users who had “high equivocality,” or who saw drug use as neutral and disagreed with the negative feedback, had particularly poor outcomes. Thus, belief in PF may be particularly important to the participant’s experience of PF and subsequent outcomes.

While development of discrepancy or belief in the accuracy of PF may act as mediators of PF’s effectiveness, attributes of individuals receiving PF may moderate the intervention’s efficacy (Collins, Carey, & Smyth, 2005). Some evidence suggests that problem drinkers with the highest risk for developing alcohol dependence respond particularly well to PF (Doumas & Hannah, 2008; Murphy et al., 2001; White et al., 2007). Those with the highest risk for developing alcohol dependence are likely those who receive the most severe PF. Severity of feedback refers to here as the magnitude of difference between one’s drinking and normative comparisons. Average feedback might involve some deviation from quantity norms, almost no current symptoms of alcohol dependence, and no physiological consequences of drinking. Severe feedback might indicate a large deviation from quantity norms and substantial symptoms of alcohol dependence—such as having abnormal liver enzyme test results demonstrating liver damage, a particularly high tolerance to alcohol, or reporting problems from alcohol use. While receipt of severe feedback may be particularly effective in developing discrepancy, no study has examined the effect of severity of feedback on developing discrepancy or drinking outcomes.

Interestingly, all existing research on mediators and moderators was implemented in the context of college students or young adults only. College students may react to PF differently from adults given their particular developmental stage in which their need for peer approval may be heightened (Erikson, 1968). As a result, comparison of drinking norms may unintentionally capitalize on this need. Research is therefore necessary to identify the potential mediators and moderators of PF for other groups of problem drinkers who may benefit from its impact and respond to it via alternative mechanisms.

Problem drinking men who have sex with men (PDMSM) is one subgroup of problem drinkers that may particularly benefit from a brief motivational intervention, such as PF. While men who have sex with men (MSM) have lower rates of alcohol dependence than the general population, compared to their heterosexual counterparts, they are more likely to experience problems at lower levels of alcohol consumption (Bux, 1996; Mackesy-Amiti, Fendrich, & Johnson, 2009; National Institute on Alcohol Abuse and Alcoholism, 2005). Among MSM, heavy drinking is a risk factor for HIV infection (Koblin et al., 2006) and is related to a greater frequency of risky sex (Irwin, Morgenstern, Parsons, Wainberg, & Labouvie, 2006; Ostrow & Stall, 2008). Furthermore, MSM are less likely to accept abstinence as a treatment goal (Bux, 1996), thus limiting their willingness to engage in traditional treatment programs, the majority of which are abstinence-based in the USA (Rosenberg & Davis, 1994). PF is therefore a potentially efficient, effective intervention that can be utilized in non-traditional settings with MSM, thus reducing risk for alcohol use disorders and HIV infection.

To begin identifying factors that contribute to PF’s impact on alcohol outcomes for adult problem drinkers, this study posed several exploratory questions regarding PF implemented with a group of adult PDMSM. We hypothesized: (1) PF would be associated with reduced drinking for this sample of MSM; (2) among all the individual items of PF, norm comparisons of drinking and abnormal liver enzyme results would have the strongest associations with developed discrepancy, perceived inaccuracy of PF, and reduced drinking; (3) individuals with severe PF would be more strongly associated with developed discrepancy, perceived accuracy of feedback, and reduced drinking compared to those with average PF; and (4) participants reporting developed discrepancy and belief in feedback would be associated with reduced drinking post-PF.

METHOD

Data was utilized from a randomized controlled trial (RCT) investigating the efficacy of combined naltrexone (NTX) and modified behavioral self-control therapy (MBSCT) for PDMSM interested in controlled drinking (Morgenstern et al., 2012). Procedures for the RCT are described in detail elsewhere (Morgenstern et al., 2012) but reviewed here briefly. All procedures were approved by an institutional review board.

Participants

Screening

Advertising via media and direct engagement at gay bars and events by community outreach teams resulted in recruitment of 200 MSM interested in controlled drinking. Potential participants were screened initially over the phone and then in-person for formal consenting and in-depth assessment for eligibility.

Study Eligibility

To be eligible for the RCT, men had to: (1) be between ages 18 and 65 years; (2) have an average weekly consumption of ≥24 standard drinks per week over the last 90 days; (3) self-identify as being sexually active with men; and (4) read English at an eighth grade level or higher. Participants were excluded if they: (1) had a lifetime diagnosis of bipolar disorder, schizophrenia, or other psychotic disorder; an untreated current major depressive disorder; or current physiological dependence on alcohol or other drugs (except nicotine or cannabis) demonstrated by current or history of physical withdrawal symptoms; (2) initiated or changed psychotropic medication in the last 90 days; (3) were at risk for serious medication side effects from NTX (i.e., taking contraindicated medications, severe liver abnormalities); (4) were enrolled in concurrent drug- or alcohol-related treatment during the treatment phase of the study.

Sample Description

Of the 200 MSM who participated in the RCT, 101 were randomized to the MBSCT condition that contained PF in one of the initial sessions, described further below. Those in MBSCT are the focus of the present study. There were no significant baseline differences between the MBSCT group and the remainder of the study sample. A detailed description of the sample is available elsewhere (Morgenstern et al., 2012). The typical participant was around age 40 years, Caucasian, HIV negative, had attended at least some college, had a baseline mean weekly consumption of 42.8 standard drinks (SD = 36.8), and drank a mean of 8.4 standard drinks each drinking day (SD = 4.1). Among those who received MBSCT, 93.3% met criteria for alcohol dependence. In the current sample, 51 were randomized to receive NTX and 50 placebo (PBO).

Procedures of the RCT

Eligible participants were enrolled in the study and received medication management via modified Brief Behavioral Compliance Enhancement Treatment (BBCET, Johnson, DiClemente, Ait-Daoud, & Stoks, 2003). Participants were randomly assigned to one of two medication conditions (NTX or PBO) and one of two counseling conditions (BBCET only or BBCET with MBSCT). The treatment phase lasted 12 weeks, with a follow-up assessment a week after treatment termination (referred to as end of treatment). During treatment, participants completed a daily telephone questionnaire about their alcohol intake using an interactive voice recording (IVR) system (TELESAGE, 2005).

Study Interventions

BBCET

BBCET is an intervention designed to enhance medication compliance by reinforcing the benefits of the medication for reduced drinking and managing any adverse effects (Johnson et al., 2003). A psychiatrist administered BBCET via seven, 20-minute sessions over 12 weeks. All BBCET sessions demonstrated adherence to the BBCET protocol.

Medication (NTX or PBO)

Medication assignment was double-blind. Psychiatrists were also blind to participants’ therapy condition. Participants were titrated up from 25 mg to 100 mg of NTX or PBO over three weeks and remained at this level through the treatment phase; 95% of participants received the full 100 mg dose. Just under 90% of participants were medication adherent.

MBSCT

Based on behavioral self-control therapy (Sanchez-Craig, Annis, Bornet, & MacDonald, 1984), MBSCT is a manual-based amalgam of MI and CBT for moderation of drinking designed for PDMSM (Morgenstern et al., 2007). Treatment comprised 12, one-hour sessions of counseling. The first two to three sessions consisted of MI, and the remaining sessions focused on skill development. Fidelity to the protocol was high and described elsewhere (Morgenstern et al., 2012). PF was provided to participants in written form and reviewed by the therapist with the participant in the second or third session of MBSCT, depending on the progress of the initial phase of therapy.

Personalized feedback.

PF was individually tailored and based on results from the battery of tests completed at the screening visit. Participants were provided with their percentile rank in terms of quantity and frequency of drinking compared to adult males in the USA (Pleis & Coles, 2002). Information about risk factors for developing alcohol dependence, including an estimated tolerance of alcohol based on peak blood alcohol concentration (BAC), other drug risk, family risk, and an Alcohol Dependence Scale (ADS, Skinner & Horn, 1984) score were included. Next, participants’ reported negative consequences of drinking, as measured by the Short Inventory of Problems (SIP, Miller, Tonigan, & Longabaugh, 1995), were compared with data on MSM in New York City from the Urban Men’s Health Study (Stall et al., 2001). PF ended with results from four liver enzyme tests performed on a blood sample taken at the screen visit. Style of reviewing PF varied significantly across therapist and participant. Time spent on PF ranged from 4 to 54 minutes (M = 19.2, SD = 8.2) and depended greatly on the participant’s response (e.g., interest, engagement). In 42% of cases, PF was read verbatim.

Coding Videotapes of PF

All therapy sessions were videotaped. Ninety were complete and codeable for therapist behavior and participant reactions to PF. Participants with codeable tapes were not significantly different on any key variables from the rest of the sample. Because only the therapist was on screen, coded participant behaviors were based on the audio portion only. Videotaped sessions of PF were coded by five raters—all research assistants on the RCT. Three of the raters participated in ongoing therapy supervision meetings with study therapists and were very familiar with feedback, its purpose, and its theoretical underpinnings. All sessions were coded by at least two raters, one familiar with PF and one not. On all items, there was an initial 93% agreement rate among the coders. Where rater codes were discrepant, there was group discussion among raters and the project director; consensus was achieved where possible, and in cases where no consensus could be achieved, the item was coded as missing. Missing data accounted for less than 1% of the observations. Raters observed and recorded therapist behaviors, such as whether the PF was read verbatim. In addition, raters coded the participant’s reaction to PF, described further below. Raters made notes on why they chose specific coded responses, which were used in group discussions about discrepant ratings.

Measures

Demographics

Age, education, income, race and ethnicity, and HIV status were obtained through structured interview procedures.

Substance Use

The Composite International Diagnostic Instrument, Substance Abuse Module (Cottler, Robins, & Helzer, 1989) was used to evaluate substance dependence exclusion criteria and potential diagnoses.

Daily Drinking Data From IVR

The daily IVR questionnaire prompted participants to report the quantity of standard drinks consumed in the last 24 hours. Specific dates of when PF was delivered were used to identify the seven days prior to PF and the seven days post-PF. Mean drinks per drinking day (MDDD) was calculated for both the pre- and post-PF weeks for each participant. While most participants completed the survey five or more days in each week, some participants had four or fewer days. Mean days completed the IVR was 4.5 (SD = 2.4) for pre-PF and 5.24 (SD = 2.6) for post-PF. All participants with three or fewer days of data in either week were excluded from the analysis, resulting in N = 70 for analyses on drinking outcome. MDDD was designated the primary outcome because it was the least biased drinking variable in the context of missing data.

Drinking Data From Timeline Followback Interview (TLFB)

The timeline followback interview (TLFB; Sobell et al., 1980) is an interviewer-assisted, calendar-based method that utilizes specific recall techniques for participants to report daily drinking (in standard drink equivalents). All normative comparisons were made using TLFB data. Additionally, TLFB data from the screen and end of treatment assessments were aggregated into summary level variables of MDDD for each of the two, 90-day time periods: pre-screen drinking and drinking during treatment (referred to as end of treatment drinking).

Percentile Rank of Drinking

Drinking reported by participants during the screen visit was compared to data from the 1998 National Health Interview Survey (Pleis & Coles, 2002) to yield a percentile rank of participants’ drinking relative to adult men in the USA. Due to the inclusion criteria of the RCT, all participants were in the 95th to 99th percentile.

Tolerance

The participant’s peak BAC (based on weight, quantity of standard drinks, and time spent drinking) was calculated from the TLFB at the screen assessment. Participants were then categorized into one of four groups of tolerance: low (peak BAC = 0–.06), medium (peak BAC = .061–.12), high (peak BAC = .121–.18), and very high (peak BAC = .181+).

Other Drug Risk

Other drug risk was an ordinal variable with a response set of 0 “low” (reported no or little other drug use), 1 “medium” (reported other drug use but did not qualify for abuse or dependence), or 2 “high” (qualified for abuse or dependence diagnosis).

Family Risk

Participants reported the number of relatives who struggled with any type of addiction. A cumulative sum score was calculated in which family members in the family of origin (e.g., father) were given twice the weight of distal relatives.

Alcohol Dependence Scale (ADS)

Severity of alcohol dependence was measured using the ADS (Skinner & Allen, 1982). The ADS is a 25-item self-report measure of symptoms and intensity of alcohol dependence, demonstrating strong reliability and validity across studies and populations (Kahler, Strong, Hayaki, Ramsey, & Brown, 2003).

Short Inventory of Problems (SIP)

The SIP (Miller et al., 1995) is a 15-item self-report measure of negative consequences of drinking. The SIP demonstrates strong psychometric properties (Kenna et al., 2005).

Biological Markers for Liver Damage

Standard liver enzyme tests were performed on participant blood samples: gamma-glutamyl transpeptidase (GGT), alanine aminotransferase (ALT), and aspartate aminotransferase (AST), and mean corpuscular volume (MCV). Results of these tests indicated current liver functioning, a range of possible liver abnormalities, and presence or absence of liver disease. Participants were provided results with thresholds for normal ranges.

Severity of Feedback

Three items were chosen a priori as clinical indicators of having severe PF (i.e., drinking >95th percentile of US adult males, tolerance = very high, and having at least one abnormal liver enzyme result). These were considered clinically informed precursors to physiological dependence. Because all participants had high percentile ranks, participants were determined to have severe feedback (S-PF) if they endorsed two of the criteria.

Developed Discrepancy

Four questions were used as proxies for developed discrepancy: expressed surprise, expressed concern, worse than expected, and better than expected.

Expressed surprise.

Raters answered the question “Did the participant express surprise at the results of the feedback?” on a scale from 1 (Not at all) to 5 (Extremely). Participants who responded using phrases such as “this is shocking” were coded as 4 or higher; those who responded more neutrally, such as “this is as I expected” were coded as 2 or lower.

Expressed concern.

Raters answered yes or no to the question “Did the participant express any concern about their PF results” to indicate expressed concern. Raters marked yes if a participant made a statement such as “I had no idea it was this bad” or “this is troublesome.”

Better and/or worse than expected.

As with expressed concern, raters answered yes or no to the following questions: (1) Did the participant state the results were better than expected? (2) Did the participant state the results were worse than expected? These questions were not mutually exclusive. For example, in response to feedback that he was drinking at higher levels than the average adult male, a participant might have stated this was “worse than he thought.” Later, upon receiving a favorable result on the liver enzyme tests, the same participant might have stated the results were better than expected. Raters coded yes if a participant expressed that it was better or worse than he expected at any point during PF.

Perceived Accuracy of PF

To measure the participants’ perceived accuracy of PF, raters were asked “Did the client express belief in the accuracy of the feedback (e.g., they did not question the data, understood the implications of the feedback and their seriousness)?” Raters provided a score ranging from 1 (Not at all) to 5 (Extremely). Participants responding with phrases such as “I don’t believe that; all my friends drink more than me” or “I think your data is wrong” were coded 1.

Analytic Plan

Basic descriptive analyses were performed first to evaluate overall participant reaction to PF. Next, intercorrelations between participant reactions were calculated to explore potential relationships to one another. Due to the structure of the RCT and the fact that PF was imbedded in another intervention, a formal mediation analysis was not possible. We therefore explored the relationships between PF, its components, participant reactions, and outcomes in steps. Baron and Kenny’s mediation model (Baron & Kenny, 1986) was used as a broad guide for our exploratory analyses, namely: (1) to examine changes in outcome resulting from the intervention (PF) as a whole and its component parts (relating to Hypotheses 1–3); (2) to explore whether receipt of PF elicited patient reactions—the theorized mediators of developed discrepancy and perceived accuracy of the feedback—and whether severe feedback differed from average feedback (relating to Hypotheses 2 and 3); and (3) to explore whether there was a connection between the proposed mediators and the outcome (relating to Hypothesis 4). These analyses were exploratory only, as major steps within formal mediation (and/or moderation) analyses were not possible.

Testing Hypothesis 1: The Effect of PF on Drinking Outcome

Timing of when PF was delivered varied across participants. As a result, there was no way to effectively and precisely compare the BBCET only group to those receiving PF to determine if PF was responsible for changes in drinking. Therefore, to detect the impact of PF, changes in MDDD in the weeks pre- and post-PF were tested using a paired samples t-test.

Testing Hypothesis 2: Association of PF Items With Developed Discrepancy, Perceived Accuracy of PF, and Drinking Outcome

PF items were correlated with participants’ reactions: proxies for developed discrepancy (expressed surprise, expressed concern, worse than expected, better than expected) and perceived accuracy of feedback. If more than one item was significantly correlated with a specific participant reaction, they were entered together into a logistic or linear regression model (depending on the dependent variable). Only those significant at the p < .05 level were retained for the final model.

Each PF item’s influence on changes to post-PF MDDD, controlling for pre-PF MDDD, was tested first independently and then in aggregate using linear regression. Items significant at a p < .10 level were entered into an aggregate model. Once together, only those significant at the p < .05 level were retained for the final model.

Testing Hypothesis 3: Association of Severity of PF With Developed Discrepancy, Perceived Accuracy of PF, and Drinking Outcome

To determine group differences, participants with S-PF and average feedback (A-PF) were compared on a number of demographic variables, PF items, and participant reactions utilizing chi-square and t-tests, where appropriate. Next, a repeated measures ANOVA was conducted to investigate the within-subject and between-subject effects of severity of PF on pre-PF to post-PF MDDD.

Testing Hypothesis 4: Association of Developed Discrepancy, Perceived Accuracy of PF, and Drinking Outcome

Participant responses were tested for correlations with pre- and post-PF MDDD. Participant reactions with significant relationships to drinking were input into a linear regression model as a predictor of post-PF MDDD, controlling for pre-PF MDDD, independently and then together. Reactions significant at the p < .05 level were retained in the final model.

Covariates

Age, medication condition, therapist, whether PF was read verbatim, time spent on PF, and HIV status were added independently to all the models generated; however, their effects were non-significant and thus excluded from all final models.

RESULTS

Descriptive Results

Table 1 contains descriptives of PF items and participant reactions. Overall, participants were rated as having been somewhat surprised by PF and had a moderate level of belief that PF was accurate. A majority of participants expressed some concern about their results. Almost half of the participants thought the PF was worse than expected, and just over a fifth thought it was better than expected.

TABLE 1.

Individual personalized feedback items, participant reactions to feedback, and drinking variables by severity of feedback

	Serious PF (N = 30)		Average PF (N = 60)		Overall sample (N = 90)
Variable	M or %	(SD)	M or %	(SD)	M or %	(SD)
		Feedback items
No. of standard drinks/weeka	61.3	(43.0)	36.0	(11.2)	44.4	(28.8)
Percentile rankb	97.4	(1.8)	94.5	(2.9)	95.4	(3.0)
BAC typical weekc	.19	(.10)	.11	(.05)	.14	(.08)
Tolerance leveld
Low	0.0		3.4		2.2
Medium	10.0		20.3		17.0
High	6.7		37.0		27.0
Very high	83.3		39.0		53.9
Other drug risk
Low	66.7		45.0		52.2
Medium	23.3		40.0		34.4
High	10.0		15.0		13.3
Family risk score	4.7	(3.6)	4.3	(3.1)	4.4	(3.2)
Alcohol Dependence Scale	14.6	(4.5)	12.9	(6.4)	13.4	(5.9)
Short Inventory of Problems	17.3	(8.6)	14.8	(6.9)	15.6	(7.5)
≥1 abnormal liver enzymee	63.3		27.1		39.3
		Participant reactions
Developed discrepancy
Expressed surprisef	3.0	(1.2)	3.2	(1.2)	3.16	(1.2)
Expressed concerng	66.7		54.5		58.5
Better tdan expectedg,h	7.4		27.3		20.7
Worse than expectedg	51.9		47.3		49.0
Perceived accuracy of feedbackf	3.5	(0.9)	3.4	(1.0)	3.46	(1.0)
		Drinking variables
Week pre-PF MDDD	6.8	(3.0)	5.9	(3.0)	6.2	(3.0)
Week post-PF MDDD	5.7	(2.7)	4.9	(2.6)	5.2	(2.6)
Pre-screen MDDDi	11.1	(6.2)	6.9	(2.1)	8.3	(4.5)
End of treatment MDDDj	5.6	(2.8)	4.2	(1.4)	4.6	(2.1)

^at(31) = −3.17, p < .01.

^bt(88) = −4.98, p < .001.

^ct(36.3) = −3.68, p < .001.

^dχ² (3) = 16.45, p = .001.

^eχ² (1) = 10.93, p < .001.

^fRated from 1 (Not at all) to 5 (Extremely).

^gResponse was dichotomous yes/no.

^hχ² (1) = 4.35, p < .05.

ⁱt(31) = −3.54, p < .01.

^jt(32.9) = −2.47, p < .05.

Intercorrelations between participant in-session reactions to PF are displayed in Table 2. Expressed surprise at PF was significantly correlated with expressed concern and worse than expected, demonstrating a weak positive relationship and a strong positive relationship respectively. Expressed concern had a strong positive correlation with worse than expected and a medium positive correlation with perceived accuracy of PF. Those who perceived PF to be accurate were also concerned about its results.

TABLE 2.

Correlation coefficients of participant reactions

Variable	2	3	4	5
1. Expressed surprise	.233*	.515**	.134	.058
2. Expressed concern	–	.376**	.064	.294**
3. Worse than expected		–	–.138	.061
4. Better than expected			–	–.027
5. Perceived accuracy				–

^*p <.05.

^**p <.01.

Results for Hypothesis 1: The Effect of PF

MDDD was significantly reduced between the weeks pre-and post-PF, t(68) = 2.46, p < .02, by just under one standard drink from 6.18 (SD = 3.0) to 5.20 (SD = 2.6).

Results for Hypothesis 2: Associations Between PF Items to Developed Discrepancy, Perceived Accuracy of Feedback, and Drinking Outcome

Developed discrepancy.

Table 3 contains correlations of individual PF items and participant reactions. Expressed surprise was significantly negatively correlated with family risk. Expressed concern was significantly negatively correlated with other drug risk. In order to further understand this relationship, other drug risk was entered into a logistic regression model as a predictor of expressed concern. Those individuals in the high-risk group (met criteria for drug abuse or dependence) were 79% less likely to express concern than those with low risk (minimal or no drug use; OR = .207, CI 95% .046–.923, p < .05). No other differences between other drug risk groups were detected.

TABLE 3.

Correlation coefficients of individual personalized feedback items, participant responses to feedback, and drinking

	Developed discrepancy
	Expressed surprise	Expressed concern	Worse than expected	Better than expected	Perceived accuracy of feedback
No. of standard drinks/week	.135	−.076	.002	−.022	−.212
Percentile rank	.057	.091	.110	−.066	−.082
BAC typical week	.115	.013	−.084	.216	−.028
Tolerance level	−.038	.203	−.143	.004	.047
Other drug risk	−.083	−.238*	−.242*	.123	.150
Family risk score	−.417*	−.184	−.090	−.089	−.292
Alcohol Dependence Scale	−.087	−.005	−.130	.248	.058
Short Inventory of Problems	.076	.094	.016	.067	.225*
> 1 abnormal liver enzyme	.060	.124	.232*	−.230*	.028
Prescreen MDDD	.065	−.132	−.110	.001	−.084
Week pre-PF MDDD	−.341**	−.137	−.086	.022	−.026
Week post-PF MDDD	.011	−.058	−.074	−.142	.178
End of treatment MDDD	.060	.150	.114	−.050	.114

^*p < .05.

^**p < .01.

Better than expected was significantly negatively correlated with having an abnormal liver enzyme test result. When entered into a logistic regression model predicting better than expected, those who had an abnormal liver enzyme test result were 74% less likely to have reported the results were better than expected (OR = .259, CI 95% .068–.988, p < .05).

Worse than expected was significantly negatively correlated with other drug risk and positively correlated with having an abnormal liver enzyme test result. When both of these items were entered into a logistic regression model examining the odds of participants reporting feedback was worse than expected, neither item was a significant predictor. When entered independently, the only significant relationship to emerge was those with an abnormal liver enzyme test result were 2.6 times more likely to report feedback was worse than expected (OR = 2.6, CI 95% 1.1–6.9, p < .05) than those with normal liver enzyme test results.

Perceived accuracy of PF.

SIP score was significantly positively correlated with perceived accuracy of PF. Those who reported greater consequences to drinking were associated with a greater belief in PF accuracy.

Drinking outcome.

No PF items were independent predictors of post-PF MDDD at the p < .10 level.

Results for Hypothesis 3: Association of Severity of PF on Developed Discrepancy, Perceived Accuracy of PF, and Drinking Outcome

Table 1 demonstrates group differences between those with S-PF and A-PF. A significantly larger proportion of those A-PF were HIV positive (21%), compared to those with S-PF (3%), (X² (2, N = 86) = 5.99, p = .05), potentially indicating that problem drinking HIV positive men seek treatment at lower levels of problem drinking compared to HIV negative men, perhaps due to a heightened awareness of negative consequences of drinking on HIV. As expected, those with S-PF demonstrated a more intense level of drinking and problem severity at baseline across all the feedback items.

Developed discrepancy.

Among the proxies for developed discrepancy, there was only one significant group difference. Those with A-PF were significantly more likely to report PF was better than expected than those with S-PF (27% vs. 7%, Table 1).

Perceived accuracy of PF.

There were no significant group differences between S-PF and A-PF on perceived accuracy of feedback.

Drinking outcome.

When examining the difference by severity of PF on post-PF MDDD, no significant differences emerged within or between the groups.

Results for Hypothesis 4: Association of Participant Reactions to PF With Drinking Outcome

Table 3 shows the correlation matrix of participant reactions and drinking. One significant negative correlation to drinking emerged: pre-PF MDDD and expressed surprise, indicating heavier drinkers were less surprised by their PF results.

Finally, when each participant response was examined as a predictor of post-PF MDDD, only one response to PF emerged as a significant predictor of immediate PF outcome at trend level: perceived accuracy of PF (B = .529, SE = .295, p = .078), indicating that as perceived accuracy of PF increased, post-PF MDDD also increased, an unexpected finding.

DISCUSSION

This exploratory study was a first step towards isolating potential mediators and moderators of PF among an adult population. This is the first study to code in-session participant reactions to PF to explore their association to drinking outcome in an adult problem drinking sample. Results demonstrated that coding participant reactions was feasible, and codes were related to one another in expected directions. While PF demonstrated a difference in pre- to post-PF MDDD, hypotheses about the relationship between PF, participant in-session responses, and drinking were only partially supported. Below, each of the hypotheses is addressed separately; however, there is a common theme across findings. Overall, it appears participants were not greatly affected by PF even in the context of a majority of the participants perceiving PF to be accurate. Whether PF developed discrepancy overall remains to be determined—scores for concern and surprise were only slightly above the midpoint of each of the scales, and just under half thought feedback was worse than expected. Regardless, the theorized mediators did not demonstrate relationships to alcohol as hypothesized. Furthermore, the one potential moderator of PF explored here—severity of feedback—did not impact results on participant reactions or drinking outcome.

Hypothesis 1.

Hypothesis 1 was supported. A pre to post test suggests that drinking was reduced between the weeks pre- and post-PF among adult PDMSM. While not a definitive main effect of PF on drinking, it suggests there may be an immediate therapeutic impact of PF on drinking for this group.

Hypothesis 2.

There was partial support for Hypothesis 2. Counter to the hypothesis, norm comparisons did not demonstrate any relationship to developed discrepancy, perceived inaccuracy of PF, or drinking outcomes. Consistent with hypotheses, having an abnormal enzyme test result was associated with developed discrepancy (primarily via worse than expected); however, counter to hypothesis, not with perceived accuracy of PF or drinking outcome. While information may be worse than a participant expects, it does not necessarily translate to immediate behavior change. In other words, fear of ongoing liver damage did not directly affect the next week’s drinking in isolation of other information.

Two PF items were related to developed discrepancy proxies: family risk and other drug risk. Individuals reporting larger numbers of relatives with addiction problems were associated with reporting less surprise about the feedback. It could be that familiarity with addiction primed them to expect the results they received. Interestingly, individuals with high other drug risk were much less likely to express concern. This could be due to a general awareness of and emotional detachment from potential consequences of drug use for those with drug abuse or dependence. It should be noted that due to exclusion criteria only 10 people were coded as having drug abuse or dependence, and those were related to nicotine and cannabis only.

Individuals reporting more consequences to drinking also rated higher on perceived accuracy of feedback. It is possible that those reporting consequences to drinking perceive more congruence of the feedback with their experience, making it more believable.

Counter to hypotheses, no one item of PF was responsible for the change in MDDD between the weeks preand post-PF. In conjunction with findings for Hypothesis 1, one explanation may be that it is not possible to detect the signal of a particular item of PF apart from the total impact of PF on drinking outcome without an explicit experimental manipulation of the intervention.

Hypothesis 3.

Counter to hypotheses, severity of feedback had a very limited relationship to participant reaction (weak, negative relationship to better than expected) and no immediate effects on drinking. Receiving serious rather than average feedback did not appear to motivate the participant to immediately change. There are several possible explanations for this. First, intensity of PF may not be at all related to motivating participants to change their drinking. Second, S-PF may not be a factor in motivating changes to drinking because it is neither surprising nor upsetting information—the two reactions theorized SPF would elicit but did not. In this study, S-PF may not be surprising due to assessment reactivity. The information provided in PF may have already been inadvertently communicated to the participant by virtue of his participation in the TLFB assessment—an assessment with known reactivity (Clifford, Maisto, & Davis, 2007). If this was the case, participants may have been surprised at the time of the screen assessment, prompting changes in drinking between the screen and the pre-PF week. In fact, drinking levels substantially decreased between the screen assessment and the pre-PF week (Table 1)—eliminating the significant differences that existed at the time of the screen TLFB between those who had S-PF and those who had A-PF. Also consistent with this explanation, the mean surprise score for those with S-PF was lower than for those with A-PF (though not significantly), potentially indicating an existing awareness of their heavy drinking patterns.

Hypothesis 4.

Hypothesis 4 was not supported. One participant reaction was related to pre-PF drinking. As discussed above, those with high pre-PF MDDD were negatively associated with expressed surprise at the feedback, perhaps explained by TLFB reactivity or an awareness of heavy drinking and its consequences among treatment seekers. None of the proxies for developed discrepancy were associated with reduced drinking. This is consistent with limited existing literature demonstrating that while discrepancy may be developed, it does not necessarily lead to changes in drinking. It may be that treatment seekers are already concerned enough about their drinking that PF does not develop new discrepancy such that it has an additional therapeutic effect. It may be also that developing discrepancy is not a mediator of PF.

Interestingly, perceived accuracy of PF was associated with an increase in post-PF MDDD. This result is inconsistent with DDT, as it does not explain entrenchment in current behavior via equivocality. An alternative explanation is that while participants may believe the PF (and may be concerned by it), they may not know how to change their drinking. The increase in drinking could be an attempt to cope with a potential increase in stress caused by the feedback or an increased salience of alcohol.

Overall, the lack of a relationship between participant reactions and drinking could be explained in several ways. First, it may be that participants’ immediate reactions may be poorly measured. Second, verbal responses uttered in session may not be truly reflective of participants’ internal states or thoughts in reaction to PF. Third, it may also be that although a participant may verbalize a particular reaction, it may not translate to behavior change until the weeks or months following feedback. In this analysis, because the effect of PF could not be isolated from the rest of the treatment, longer-term outcome differences were not tested. Fourth, it may be that a participant’s in-session reaction is simply not related to drinking outcomes.

Clinical Implications

Consistent with the limited existing literature, this study demonstrates that PF can be beneficial to adult problem drinkers. Educating clients about their drinking compared to their peers and the general population has proven and continues to prove to be an important part of reducing drinking—with or without inclusion of expensive biomarkers. Clinicians should be aware that in-session reactions may or may not reflect the potential impact of this intervention. While results of this analysis indicated that those with the most severe drinking problems were already aware that their drinking was potentially beyond the norm and may have initiated drink reduction prior to PF, utilizing PF for adult problem drinkers may reinforce this self-initiated reduction. Finally, clinicians should also note that expressed perception of an inaccuracy of PF in session does not necessarily impact the outcome of PF negatively.

If PF is as successful with adult PDMSM as it is with college students, then it could be utilized as the first phase of intervention in a stepped-care system for alcohol use disorders. This is particularly important because MSM are a unique group of problem drinkers who may actively avoid abstinence-based treatments and have higher consequences at lower levels of problem severity (National Institute on Alcohol Abuse and Alcoholism, 2005).

Limitations

Results of this study should be interpreted with caution. Methodological limitations prevent conclusive results regarding the mediators or moderators of PF. First, without a comparison group that received MBSCT but did not receive PF, it cannot be concluded that PF is solely responsible for the significant changes in drinking. A test of mediation or moderation would first require that the efficacy of the specific treatment (e.g., PF) be demonstrated. This was not done, preventing true identification of mediators and moderators of PF.

Sample size may have prevented having sufficient power to detect significant differences. The number of cases utilized for these analyses were limited due to dependency on video tape quality. In some of the analyses, there were fewer than 10 participants in a given cell.

The results regarding participant reactions to PF rely exclusively on overt, verbal data during session. Variables used for participant reactions to PF were dependent on third party raters, rather than participants’ direct self-report or a validated and quantified coding scheme of sessions. These variables were thus reliant on raters’ subjective interpretation of the content of the session—that may or may not reflect the true experience of the participant. In addition, many of the participant’s reactions may have been lost—particularly if the participant was silent in response to the feedback. While steps to increase the rigor of the coding were taken, such as having more than one rater for each session, without the participant on camera or reporting directly on their own experience of PF, results remain of limited use for interpretation.

Finally, generalizability of results is limited to a treatment-seeking, PDMSM population. Due to the smaller, insular nature of the GLBT community, it may be that adult MSM are more concerned with peer group relations than other groups of adults across the life span (Heath, Lanoye, & Maisto, 2012), making them more similar to college students. This would presumably sensitize MSM to norm comparisons in ways not present for other adult problem drinkers. Additionally, among treatment seekers undergoing reactive assessments, it may be particularly difficult to isolate to what extent or how PF leads to reduced drinking.

Implications for Future Research

This study was innovative in three ways, which helps to guide future research steps. First, it used in-session coding as a way to measure the potential mediators of PF. This is important because in-session response often guides delivery of PF, and it is the only tool available to clinicians to assess as to whether PF is having the desired effect. Future research should include further exploration of the relationship of participant response to PF and outcomes. Second, this is the first study to examine the impact of the severity of feedback on participants’ outcomes. The findings from this study reveal that treatment seekers, particularly those who have participated in a reactive empirical assessment, may have diluted, unrelated, or non-significant responses to PF. Further research needs to explore the impact of PF in the context of rigorous experimental manipulation among both treatment seekers and non-treatment seekers to understand the mechanisms of action of PF. Third, this is the first study to explore the unique impact of using biological markers (i.e., liver enzyme tests) as a part of PF—unique information that may be particularly powerful in developing discrepancy and urgency for change. Rigorous methodological inquiry into whether this may also impact drinking outcomes would provide important information for ongoing intervention development and implementation.

Future research investigating PF among other samples of adult problem drinkers must be implemented repeatedly to begin to understand its effect among a wide range of populations, and it must include a rigorous experimental methodology with a disaggregation of the component parts of PF to identify its active ingredients. It is important that such research includes participant in-session reaction, in addition to validated and reliable self-report measures of potential mediators. For example, client reaction to psychotherapy has been measured previously by the Therapy Session Report (Kolden et al., 2006), a measure that could be adapted for use with PF. Together these methods and measurement tools could more accurately unpack the impact of PF across different populations with varying severities of alcohol use disorders.

GLOSSARY

Mediator

A variable (e.g., increased self-efficacy) that is responsible for the effect of the independent variable (e.g., therapy condition) on the dependent variable (e.g., alcohol use outcomes). Also referred to as an intervening variable

Moderator

A variable (e.g., gender) that affects the strength and/or direction of the relationship between two variables (e.g., therapy condition on alcohol use outcomes). For example, a certain therapy may work differently for men and women, such that the therapy is more effective for men than for women in regards to alcohol use outcomes. In such a case, gender is said to moderate the therapy in regards to alcohol use outcomes

Biography

Alexis Noel Kuerbis, LCSW, Ph.D., is clinical director of Columbia Addiction Services and Psychotherapy Intervention Research (CASPIR) and assistant professor at the College of Physicians and Surgeons at Columbia University. She is a substance abuse specialist with more than 15 years experience in a variety of clinical, public health, and social services settings. Her research focuses on special populations struggling with substance abuse and dependence, including women on welfare, the LGBT community, and older adults. While at CASPIR, Dr. Kuerbis’ research has been focused on mechanisms of action within brief, moderation focused treatments for problem drinking. She completed her doctoral degree in clinical social work at the Silver School of Social Work at New York University in 2009.

Katherine Schaumberg is a doctoral candidate in clinical psychology at the University at Albany, State University of New York. She is currently completing her pre-doctoral internship at the University of Mississippi Medical Center/G.V. (Sonny) Montgomery VA Medical Center in Jackson, Mississippi. Katherine’s research interests include the prevention of eating disorders and obesity, and the relationship between eating pathology and substance use.

Christine M. Davis currently has a part-time faculty position at the Rutgers School of Public Health, Department of Health Education and Behavioral Science, and is a research member of Rutgers Cancer Institute of New Jersey. She is the project manager for the New Jersey Youth Tobacco Survey Data Collection Project funded by the NJ Department of Health. Christine also is employed at Columbia Addiction Services and Psychotherapy Intervention Research (CASPIR) as a biostatistician on a NIAAA-funded study (PI: Andrew C. Chen) investigating the interaction of pharmacotherapy and CBT with genotype, for problem drinkers. She received her Ph.D. in Statistics and Measurement in 2011. Her research interests include: methodological and statistical issues associated with the measurement and interpretation of tobacco, alcohol, and other drug use data; alcohol, tobacco, and other drug use treatment/intervention outcomes; and tobacco surveillance and survey research.

Lisa Hail is pursuing her doctorate in clinical psychology at Fairleigh Dickinson University. She is interested in the mechanisms of behavior change in the treatment of eating and substance use disorders.

Jon Morgenstern, a specialist in the treatment of substance use disorders, is a professor of clinical psychology and director of Addiction Treatment in the Department of Psychiatry at the Columbia University Medical Center. He is also a vice president and director of Treatment Research at the National Center on Addiction and Substance Abuse at Columbia University (CASA). Prior to joining the faculty at Columbia in 2004, Dr. Morgenstern was an associate professor of psychiatry and director of Addiction Treatment at the Mount Sinai Medical Center. He received his B.A. from Columbia University in 1977 and his Ph.D. in clinical psychology from NYU in 1987. His research interests are in the area of cognitive behavioral treatment and the combination of medications and psychotherapy for substance use disorders. Dr. Morgenstern has published over 60 peer-reviewed articles on the treatment of substance use disorders and his treatment research program has been funded continuously by the National Institutes of Health for the last 20 years.