The Assessment of Protective Behavioral Strategies: Comparing the Absolute Frequency and Contingent Frequency Response Scales

Abstract

The purpose of the present studies was to examine the effects of response scale on the observed relationships between protective behavioral strategies (PBS) measures and alcohol-related outcomes. We reasoned that an ‘absolute frequency’ scale (stem: “how many times…”; response scale: 0 times to 11+ times) conflates the frequency of using PBS with the frequency of consuming alcohol; thus, we hypothesized that the use of an absolute frequency response scale would result in positive relationships between types of PBS and alcohol-related outcomes. Alternatively, a ‘contingent frequency’ scale (stem: “When drinking…how often…”; response scale: never to always) does not conflate frequency of alcohol use with use of PBS; therefore, we hypothesized that use of a contingent frequency scale would result in negative relationships between use of PBS and alcohol-related outcomes. Two published measures of PBS were used across studies: the Protective Behavioral Strategies Survey (PBSS) and the Strategy Questionnaire (SQ). Across three studies, we demonstrate that when measured using a contingent frequency response scale, PBS measures relate negatively to alcohol-related outcomes in a theoretically consistent manner; however, when PBS measures were measured on an absolute frequency response scale, they were non-significantly or positively related to alcohol-related outcomes. We discuss the implications of these findings for the assessment of PBS.

Within the past 10 years, the harm reduction approach to college student drinking has led to a marked increase in research on strategies to reduce negative alcohol-related consequences, which are commonly referred to as protective behavioral strategies (PBS) in alcohol research. The operational definition of PBS varies across studies; some consider PBS as behaviors only used while drinking (Martens et al., 2005, Martens et al., 2008), whereas others also include strategies to avoid drinking altogether (Novik & Boekeloo, 2011; Sugarman & Carey, 2007; 2009).

Although research has consistently demonstrated that PBS are negatively associated with alcohol-related problems (Borden et al., 2011; Martens et al., 2009; Martens, Pederson et al., 2007), the relationship between PBS and alcohol consumption is more mixed. Whereas several researchers have found negative relationships between use of PBS and alcohol consumption (Borden et al., 2011; Martens et al., 2005; Martens et al., 2009; Martens et al., 2004; Martens, Pederson et al., 2007), others have found counterintuitive positive relationships (Sugarman & Carey, 2007; 2009). Because measures of PBS were created to assess strategies intended to reduce alcohol consumption and negative alcohol-related consequences, a positive relationship with alcohol use is theoretically inconsistent. In the present series of studies, we examine if the heterogeneity in the assessment of PBS may account for these conflicting results. Specifically, we consider how altering the assessments’ response scales affects the relationship between PBS and alcohol-related outcomes using two published PBS scales: the Protective Behavioral Strategies Survey (PBSS; Martens et al., 2005) and the Strategy Questionnaire (SQ; Sugarman & Carey, 2007).

When assessed with the SQ, PBS are conceptualized as any behavior used to reduce alcohol consumption and/or alcohol-related problems (Sugarman & Carey, 2007), whereas PBSS operationalizes PBS as behaviors used while drinking to reduce alcohol consumption and avoid alcohol-related consequences (Martens et al., 2005). A key difference between the SQ and the PBSS is that the PBSS only considers behaviors directly related to alcohol consumption (i.e., “behaviors used while drinking”), whereas the SQ considers all behaviors that lead to reduced consequences, including avoiding consumption. Therefore, although many of the SQ and PBSS items are quite similar, the SQ includes a unique set of items that address avoiding alcohol use altogether that are not included on the PBSS.

Strategy Questionnaire (SQ)

The SQ was designed to have three subscales: Alternatives to Drinking (4 items), Selective Avoidance (7 items), and Strategies While Drinking (10 items). The SQ has participants report general PBS use during a specific timeframe. Specifically, the stemming question states, “Please indicate how often you have used the following strategies in the past 2 weeks” with responses on a 6-point scale: 0 times, 1 time, 2–3 times, 4–5 times, 6–10 times, 11+ times. We term this type of response scale an ‘absolute frequency’ response scale, because it literally describes how often each type of PBS gets used during the assessment window. The absolute frequency response scale can be advantageous when the goal is to quantify strategy use; however, a potential disadvantage to this type of response scale is that it conflates the frequency of using PBS with the frequency of consuming alcohol, such that one might expect that a higher frequency PBS use among individuals who report more drinking episodes.

When using the SQ, researchers have found that the Strategies While Drinking subscale of the SQ is positively correlated with average number of drinks per week (Sugarman & Carey, 2007; 2009) and average blood alcohol content (Sugarman & Carey, 2007; 2009). Although these positive relationships are inconsistent with theory, we hypothesize that this might be a consequence of using an ‘absolute frequency’ response scale.

Protective Behavioral Strategies Survey

The PBSS has three factors: Stopping/Limiting Drinking (7 items), Manner of Drinking (5 items), and Serious Harm Reduction (3 items; Martens et al., 2005). The original response scale of the PBSS is what we have termed a ‘contingent frequency’ scale, because participants are asked to consider how often they use PBS during episodes of alcohol consumption. Specifically, participants are asked to “please indicate the degree to which you engage in the following behaviors when using alcohol or ‘partying’” and report on a 6-point scale: Never, Rarely, Occasionally, Sometimes, Pretty Often, Always. We consider this type of a frequency scale ‘contingent,’ because the frequency estimate is specifically contingent on the occurrence of another behavior (i.e., drinking or “partying”). The contingent frequency scale does not directly yield information about how many times each strategy is used; however, the advantage to this type of response scale is that it allows the assessment of PBS use to be independent of how often one chooses to consume alcohol (i.e., the number of drinking episodes). In other words, this approach to assessing PBS is likely not indirectly affected by the frequency of drinking episodes.

Researchers have found each PBSS subscale to be negatively associated with various measures of alcohol consumption, including drinks consumed on a typical week of drinking in the past month (Labrie et al., 2011; Martens et al., 2005; Martens et al., 2008), binge drinking episodes in the past month (Martens, Ferrier, & Cimini, 2007), and number of days that alcohol was consumed in the past month (Labrie et al., 2011; Martens et al., 2005). In contrast to the SQ, all PBSS studies have found negative or non-significant relationships between each PBSS subscale and all alcohol use measures.

The purpose of the present research was to examine how the nature of the response scale (i.e., absolute frequency vs. contingent frequency) affects the relationships between PBS factors and alcohol-related outcomes. We believe that clarifying of the effects of the specific wording of alternate response scales is essential for continued scale creation and refinement with regards to studying the prediction and understanding of alcohol-use behaviors.

Study 1

In the first study, we wanted to compare the concurrent validity of the PBSS and SQ using the same type of response scales as described in the respective publications. Because the PBSS uses a ‘contingent frequency’ response scale, we expected that the PBSS would yield theoretically consistent negative relationships with alcohol-related outcomes. Conversely, we expected that the SQ, which uses an ‘absolute frequency’ response scale, would potentially evince theoretically inconsistent, positive relationships with alcohol-related outcomes.

Participants and Procedure

Three hundred eighteen college student drinkers at a large southeastern university¹ were recruited to participate for course credit; 12 participants who did not report drinking during the past 30 days and 15 participants with unrealistic or inconsistent responses were removed, resulting in a final analytic sample of 291 college student drinkers. Most participants self-reported being Caucasian/White (64.9%) or African American/Black (21.6%), and over two-thirds were women (69.4%). All participants read a notification statement prior to participating in the present study, and the study was approved by the Human Subjects Committee at the participating university.

Measures

Protective behavioral strategies

Protective behavioral strategies were assessed using the 15-item Protective Behavioral Strategies Survey (PBSS; Martens et al., 2005) and the 21-item Strategy Questionnaire (SQ; Sugarman & Carey, 2007). For the PBSS, the stem asked, “How often do you use the following drinking behavior?”, and we used a 5-point Likert-type response scale: Never, Rarely, Sometimes, Pretty Often, and Always. Martens et al. (2005) confirmed three factors in the PBSS scale: Limiting/Stopping Drinking, Manner of Drinking, and Serious Harm Reduction.

The stem for the SQ stated, “Please indicate how often you have used the following strategies in the past 3 months,” and responses were measured on a 6-point Likert-type scale: 0 [times], 1 [time], 2–3 [times], 4–5 [times], 6–10 [times], 11+ [times]). Sugarman and Carey (2007) identified three factors: Selective Avoidance of Heavy Drinking Activities and Situations, Strategies Used While Drinking, and Alternatives to Drinking.

Alcohol consumption

Alcohol consumption was measured using a modified version of the Daily Drinking Questionnaire (DDQ; Collins, Parks, & Marlatt, 1985). This questionnaire uses seven-item (Monday through Sunday) grids to assess daily drinking patterns. This questionnaire was used to assess the number of standard drinks consumed for a typical drinking week and number of drinks consumed for the heaviest drinking week within the past 30 days. Instructions for this scale state: “Think about your drinking behaviors during the last month (i.e., past 30 days) for the following questions. With respect to alcohol consumption, 1 standard drink is equivalent to 12 oz. beer OR 4 oz. wine OR 1 oz. shot of liquor straight or in a mixed drink.”

Despite the strong correlations between measures of alcohol use (.53 < rs < .89), frequency vs. quantity and typical vs. heaviest drinking do reflect distinct ways of examining drinking patterns; thus, alcohol use was quantified in four ways: 1) quantity of alcohol use in a typical drinking week (i.e., number of standard drinks), 2) quantity of alcohol use in the heaviest drinking week, 3) frequency of alcohol use in a typical drinking week (i.e., number of drinking days), and 4) frequency of alcohol use in the heaviest drinking week during the previous 30 days.

Alcohol-related problems

Alcohol-related problems were assessed using the 23-item Rutgers Alcohol Problem Index (RAPI; White & Labouvie, 1989) and a 23-item version of the Brief-Young Adult Alcohol Consequences Questionnaire (B-YAACQ; Kahler, Strong, & Read, 2005). Although the RAPI originally used a Likert-type response scale, participants were presented with checklist forms of both scales, where they checked a box for each problem that they experienced in the past 90 days. Each item was scored dichotomously to reflect presence/absence of the alcohol-related problem, and items for each scale were summed separately to create two composite scores: RAPI score (α = .87) and B-YAACQ score (α = .89). Previous research has demonstrated the usefulness of scoring alcohol problems measures dichotomously (Martens, Neighbors, Dams-O’Connor, Lee, & Larimer, 2007). Although the RAPI and the B-YAACQ were strongly correlated in the present study (r = .79), each were examined separately to facilitate comparison across studies where only one of the measures were used. Further, the B-YAACQ was designed to include more common alcohol-related consequences that are not included on the RAPI; thus, to some extent, the types of problems on the two scales differ in severity. The RAPI problems listed include, “Not able to do your homework or study for a test”, “Felt that you had a problem with alcohol”, and “Passed out or fainted suddenly”. B-YAACQ items include, “I have spent too much time drinking”, “While drinking, I have said or done embarrassing things”, and “I have felt badly about myself because of my drinking”.

Results

Bivariate correlations (see Table 1²) revealed that each PBSS subscale was significantly negatively correlated with all alcohol use/problems measures, except for a non-significant correlation between Serious Harm Reduction and Heaviest Alcohol Use Frequency (p < .10). In contrast, the SQ only yielded significant negative relationships with the Alternatives to Drinking subscale; although the remaining correlations between SQ subscales and alcohol outcomes were non-significant, contrary to our hypothesis, they were all in the expected direction.

Table 1.

Correlations and descriptive statistics of all study variables: Study 1

	PBSS			SQ			Alcohol Use				Prob
Variable	1	2	3	4	5	6	7	8	9	10	11	12	M	SD
1. Limiting/Stopping	.83												2.74	.86
2. Manner of Drinking	.53	.72											3.08	.80
3. Serious Harm Reduction	.22	.28	.72										4.17	.83
4. Selective Avoidance	.44	.36	.20	.83									3.03	1.14
5. Strategies Used While Drinking	.23	.15	.31	.67	.86								3.68	1.11
6. Alternatives	.19	.14	.24	.56	.55	.79							3.84	1.25
7. Typical Quantity	−.26	−.37	−.13	−.11	−.01	−.19	----						9.42	8.58
8. Heaviest Quantity	−.23	−.35	−.12	−.11	−.03	−.19	.89	----					14.83	12.93
9. Typical Frequency	−.17	−.17	−.15	−.06	−.00	−.17	.63	.53	----				2.44	1.49
10. Heaviest Frequency	−.15	−.16	−.11	−.05	−.02	−.18	.63	.68	.77	----			2.71	1.64
11. Alcohol Problems-RAPI	−.17	−.27	−.22	−.04	−.01	−.06	.42	.39	.26	.29	.88		2.98	3.96
12. Alcohol Problems-B-YAACQ	−.21	−.35	−.12	−.05	−.02	−.07	.50	.46	.27	.32	.79	.90	4.79	4.88

Note. N = 291. PBSS = Protective Behavioral Strategies Survey; SQ = Strategy Questionnaire; Prob = Alcohol-related Problems; RAPI = Rutgers Alcohol Problem Index; B-YAACQ = Brief Young Adult Alcohol Consequences Questionnaire. Significant effects (p < .05) are bolded for emphasis. The underlined values on the diagonal reflect Cronbach’s alphas for multi-item inventories.

Alcohol Consumption

Separate multiple regressions were conducted to examine the concurrent validity of each PBS measure in predicting the four alcohol consumption outcomes, yielding a total of 8 regressions (see Table 2). Across alcohol use outcomes, the PBSS Manner of Drinking subscale was significantly negatively related to typical and heaviest quantity of alcohol consumption after controlling for the other PBSS subscales. The SQ Alternatives to Drinking subscale was significantly negatively related to all alcohol use outcomes controlling for the other SQ subscales, and the SQ Strategies Used While Drinking subscale was significantly positively related to typical quantity and heaviest frequency of alcohol use.

Table 2.

Multiple regressions predicting alcohol outcomes from protective behavioral strategies: Study 1

	Typical Quantity			Heaviest Quantity			Typ. Frequency			Heav. Frequency
Predictor Variable	β	p	R2	β	p	R2	β	p	R2	β	p	R2
PBSS Limiting/Stopping Drinking	−.08	.201		−.05	.460		−.10	.165		−.08	.256
PBSS Manner of Drinking	−.32	.000		−.32	.000		−.09	.180		−.10	.159
PBSS Serious Harm Reduction	−.02	.733		−.02	.698		−.10	.098		−.07	.252
Total PBSS			.139			.128			.047			.035
SQ Selective Avoidance	−.10	.228		−.08	.330		−.01	.878		−.02	.835
SQ Strategies While Using Alcohol	.18	.023		.16	.054		.14	.084		.18	.027
SQ Alternatives to Drinking	−.34	.001		−.23	.001		−.24	.001		−.27	.000
Total SQ			.053			.049			.042			.052
	RAPI			B-YAACQ
Predictor Variable	β	p	ΔR2	β	p	ΔR2
Typical Quantity	.34	.011		.45	.001
Heaviest Quantity	.00	.981		−.01	.959
Typical Frequency	−.16	.517		−.13	.152
Heaviest Frequency	.09	.390		.11	.252
PBSS Limiting/Stopping Drinking	.02	.795		−.00	.981
PBSS Manner of Drinking	−.11	.107		−.18	.004
PBSS Serious Harm Reduction	−.15	.007		−.02	.765
Total PBSSa			.037			.031
Typical Quantity	.37	.022		.49	.000
Heaviest Quantity	.05	.451		.05	.727
Typical Frequency	−.04	.475		−.12	.202
Heaviest Frequency	.06	.410		.08	.447
SQ Selective Avoidance	.13	.085		−.03	.656
SQ Strategies While Using Alcohol	−.07	.380		.02	.755
SQ Alternatives to Drinking	−.02	.863		.03	.631
Total SQa			.009			.001

Note. PBSS = Protective Behavioral Strategies Survey; SQ = Strategy Questionnaire; RAPI = Rutgers Alcohol Problem Index; B-YAACQ = Brief Young Adult Alcohol Consequences Questionnaire.

^aReflects the change in R² when the PBS subscales were entered in a hierarchical regression in step 2 after controlling for alcohol use measures in step 1.

Alcohol Problems

After controlling for alcohol use and the other PBSS subscales, the PBSS Serious Harm Reduction subscale was significantly negatively related to alcohol problems assessed by the RAPI, and the PBSS Manner of Drinking subscale was significantly negatively related to alcohol problems as assessed by the B-YAACQ. In contrast, none of the SQ scales were uniquely predictive of problems.

Discussion

Consistent with previous research (Borden et al., 2011; Martens et al., 2005; Martens et al., 2009; Martens et al., 2004; Martens, Pederson et al., 2007), we found the PBSS subscales that used a contingent frequency response scale were negatively related to both alcohol use and alcohol problems controlling for use. In contrast, the SQ subscales that used an absolute frequency response scale demonstrated a significant positive relationship with two alcohol use measures (Typical Quantity and Heavy Frequency), and none of the SQ subscales were significantly related to alcohol problems. Thus, the SQ results indicate two theoretical problems: 1) they suggest that using some PBS can increase alcohol use, and 2) they suggest that using PBS is unrelated to alcohol problems. Thus, although the PBSS results are consistent with theory, the SQ results are antithetical to the premise that guided the development of PBS measures (Sugarman & Carey, 2007; 2009), that PBS use protects individuals from alcohol-related problems.

An examination of the item content of these two scales does not suggest face validity differences that could explain these disparate scale patterns. In fact, the main difference between the PBSS and the SQ items is that the SQ includes strategies that involve avoiding alcohol use. Further, it was the SQ Alternatives to Drinking subscale, which contains some of these items, that had theoretically consistent negative relationships to alcohol use.

We believe that the main difference between these two scales is not in the content, but rather that the PBSS uses a contingent frequency response scale, whereas the SQ uses an absolute frequency response scale that conflates the use of protective behavioral strategies with general alcohol use. Given that there are other differences between the PBSS and SQ beyond the stem/response scale (i.e., item content), it is important that the response scale be manipulated to identify that the differences are actually caused by the differences in response scales.

Study 2

To test the hypothesis that the type of response scale affects the estimated relationships between PBS and alcohol outcomes, we swapped the PBSS and SQ response scales. Therefore, because the PBSS that uses the contingent frequency response scale has evinced only negative relationships with alcohol-related outcomes, we examined whether using an absolute frequency response scale for this measure would result in theoretically inconsistent positive relationships with alcohol outcomes. Likewise, because the SQ with the absolute frequency response scale has evinced some counterintuitive positive relationships with alcohol use measures, we examined whether using a contingent frequency response scale for this measure would result in negative relationships with alcohol use measures.

Method

Participants and Procedure

Two hundred and sixty three (53% women) college student drinkers at a large university in Southeast Virginia took part in this study. In order to be eligible for participation, students had to have been at least 18 years of age and report having consumed at least one alcoholic beverage within the past 30 days. Participants enrolled and participated in an anonymous survey through an online research participation system in exchange for course credit. All participants reviewed a notification statement electronically prior to participation. This study was approved by the College of Science Human Subjects Committee and all APA ethical guidelines were followed.

Measures

Protective Behavioral Strategies

As in Study 1, PBS use was assessed with both the Protective Behavioral Strategies Survey (PBSS; Martens et al., 2005) and the Strategy Questionnaire (SQ; Sugarman & Carey, 2007). However, the PBSS was presented with the stem (“Please indicate how often you have used the following strategies in the past 2 weeks”) using the SQ’s absolute frequency response scale: 0 times, 1 time, 2–3 times, 4–5 times, 6–10 times, 11+ times. Conversely, the SQ was administered using the stem (“Please indicate the degree to which you engage in the following behaviors when using alcohol or ‘partying’”) and contingent frequency response scale typically used with the PBSS: Never, Rarely, Occasionally, Sometimes, Pretty Often, Always. All subscales had good internal consistency (see Table 3).

Table 3.

Correlations and descriptive statistics of all study variables: Study 2

	PBSS			SQ			Alcohol Use				Prob
Variable	1	2	3	4	5	6	7	8	9	10	11	12	M	SD
1. Limiting/Stopping	.88												1.99	1.02
2. Manner of Drinking	.72	.77											2.35	1.06
3. Serious Harm Reduction	.51	.60	.83										3.31	1.57
4. Selective Avoidance	.30	.24	−.05	.88									3.15	1.16
5. Strategies Used While Drinking	.26	.27	.20	.64	.83								3.82	0.97
6. Alternatives	.15	.18	.08	.59	.64	.88							3.72	1.30
7. Typical Quantity	.05	.10	.27	−.34	−.17	−.23	----						10.55	9.89
8. Heaviest Quantity	−.01	.10	.20	−.33	−.17	−.18	.86	----					15.57	14.98
9. Typical Frequency	.15	.17	.25	−.15	−.11	−.15	.71	.55	----				2.50	1.57
10. Heaviest Frequency	.08	.16	.20	−.19	−.13	−.12	.67	.70	.82	----			2.77	1.79
11. Alcohol Problems-RAPI	.07	.08	.16	−.21	−.16	−.13	.50	.44	.47	.46	.84		3.20	3.71
12. Alcohol Problems-B-YAACQ	−.03	−.02	.18	−.28	−.19	−.14	.48	.46	.42	.43	.77	.87	4.85	4.47

Note. N = 263. PBSS = Protective Behavioral Strategies Survey; SQ = Strategy Questionnaire; Prob = Alcohol-related Problems; RAPI = Rutgers Alcohol Problem Index; B-YAACQ = Brief Young Adult Alcohol Consequences Questionnaire. Significant effects are bolded (p < .05) for emphasis. The underlined values on the diagonal reflect Cronbach’s alphas for multi-item inventories

Alcohol Outcomes

Alcohol use (DDQ; Collins et al., 1985) and alcohol-related problems (B-YAACQ: Kahler et al., 2005; RAPI: White & Labouvie, 1989) were assessed using the same measures from Study 1.

Results

We hypothesized that by swapping the response scales, we would observe the counterintuitive positive relationships between PBS and alcohol outcomes with the revised PBSS scale that used the absolute frequency scale. Bivariate correlations were used to determine the nature of the relationships between scores on the modified versions of the PBS scales and alcohol-related outcomes (see Table 3). Using the contingent frequency response scale, all three SQ subscales now are significantly negatively correlated with all six alcohol use/problems measures with the exception of one non-significant negative correlation between Strategies Used While Drinking and Typical Drinking Frequency, r = −.11, p = .089; all of the estimated relationships were consistent with theory. However, using the absolute frequency response scale, none of the PBSS subscales were significantly negatively correlated with alcohol-related problems. In fact, of the three PBSS factors, Serious Harm Reduction was significantly positively correlated with all alcohol-related outcomes. The other PBSS subscales had mostly positive, non-significant relationships with alcohol-related outcomes.

Discussion

By switching the type of response scale for two published PBS measures, we sought to clarify inconsistent findings in the PBS literature and improve the assessment of PBS by examining the effect of response scales on the estimated relationships between PBS measures and alcohol-related outcomes. Specifically, we reasoned that the absolute frequency response scale typically used by the Strategy Questionnaire (SQ; Sugarman & Carey, 2007; 2009) conflates the frequency of PBS use with the frequency of alcohol use, leading to counterintuitive positive relationships between PBS and alcohol use measures.

These counterintuitive relationships have been found previously by the scale developers (Sugarman & Carey, 2007; 2009), and were found in our Study 1. As predicted, when we used a contingent frequency response scale with the SQ as opposed to an absolute frequency response scale, all SQ subscales were negatively correlated with all alcohol-related outcomes including measures of frequency/quantity of alcohol use and alcohol-related problems. These findings support using a contingent frequency response scale for the SQ and perhaps for the assessment of PBS generally.

To demonstrate that the limitations of the absolute frequency response scale are not limited to the items assessed by the SQ, we used an absolute frequency response scale for the PBSS instead of its customary contingent frequency response scale. Although studies have consistently found that all PBSS subscales are negatively related to alcohol outcomes (Labrie et al., 2011; Martens et al., 2005; Martens et al., 2008), Study 2 showed that all PBSS subscales were either non-significantly related or positively correlated with all alcohol use measures when assessed on an absolute frequency response scale. Perhaps more importantly, one subscale (Serious Harm Reduction) was even positively associated with alcohol-related problems.

The results of Study 1 and 2 suggest that both the PBSS and the SQ have theoretically consistent concurrent validity with alcohol outcomes when assessed using a contingent frequency response scale. However, both of these measures have theoretically inconsistent relationships with alcohol outcomes when assessed using an absolute frequency response scale. Basically, we have seen that using the response scale from the PBSS for the SQ essentially ‘fixes’ this scale. However, the original stemming question of the SQ does have one important feature lacking in the stem of the PBSS. Specifically, it mentions an assessment window (e.g., “in the past 2 weeks”), which is important if one is assessing PBS in a longitudinal study as it ensures that individuals are all reporting their behavior based on the same time frame (i.e., before intervention, after intervention). Therefore, our recommendation is that researchers use a contingent frequency response scale with a stemming question that defines the assessment window (e.g., “In the past 30 days, how often did you use the following strategies while drinking or partying?”; Author et al., 2012a).

Study 3

The main purpose of Study 3 was to examine whether a contingent frequency response scale with a defined assessment window would further improve the assessment of protective behavioral strategies. In addition, as the first study using this contingent frequency response scale with both the PBSS and SQ, we are able to compare the concurrent validity of the PBSS and SQ when using the same response scale.

Participants

Two hundred thirty-nine students (78.2% women) participated at a large university in Southeast Virginia. Thirteen participants were dropped for not having reported drinking during a ‘typical drinking week’ on the DDQ, making the analytic sample 226. The recruitment instructions asked that “only participants who drink alcohol” sign up for this study. Subjects enrolled in the study online and received research participation credit for their participation in this study. All participants volunteered their participation after reading a notification statement that explained what the study involved and all data were kept anonymous.

The participants were few freshmen (15.5%) and sophomores (24.4%), with mostly juniors (29.4%) and seniors (29.4%). Most of the sample self-identified their racial group as Caucasian or White (52.9%), 28.6% as African-American or Black, 3.8% as Latino or Latina, 7.1% as Asian or Pacific Islander, and 6.7% as a group other than those stated. The vast majority identified themselves as single (80.7%), 9.7% as married, and 2.5% as divorced. The average age of participants was 22.31 (Median = 21, SD = 5.44).

Measures

Protective Behavioral Strategies

As in Study 1 and Study 2, PBS were assessed with both the Protective Behavioral Strategies Survey (PBSS; Martens et al., 2005) and the Strategy Questionnaire (SQ; Sugarman & Carey, 2007). However, both scales were presented with a contingent frequency response scale with a defined assessment window (“For the following set of items, think about your behavior in the past 30 days. How often do you use the following behaviors when using alcohol or ‘partying’?”), and a 6-point Likert-type response scale (Never, Rarely, Occasionally, Sometimes, Usually, Always). All subscales had good internal consistency (see Table 4).

Table 4.

Correlations and descriptive statistics of all study variables: Study 3

	PBSS			SQ			Alcohol Use				Prob
Variable	1	2	3	4	5	6	7	8	9	10	11	12	M	SD
1. Limiting/Stopping	.85												2.88	1.99
2. Manner of Drinking	.44	.65											3.36	1.10
3. Serious Harm Reduction	.30	.35	.76										4.95	1.22
4. Selective Avoidance	.53	.70	.26	.85									3.47	1.09
5. Strategies Used While Drinking	.52	.49	.44	.64	.86								4.01	0.95
6. Alternatives	.41	.37	.36	.61	.69	.83							3.97	1.15
7. Typical Quantity	−.20	−.31	−.22	−.39	−.32	−.34	----						9.36	9.43
8. Heaviest Quantity	−.15	−.24	−.08	−.18	−.24	−.23	.60	----					13.78	14.96
9. Typical Frequency	−.07	−.11	−.10	−.35	−.26	−.27	.69	.48	----				2.50	1.31
10. Heaviest Frequency	−.04	−.07	−.07	−.18	−.22	−.18	.38	.71	.61	----			2.72	1.65
11. Alcohol Problems-RAPI	−.10	−.20	−.19	−.28	−.30	−.27	.43	.37	.39	.38	.89		2.92	4.11
12. Alcohol Problems-B-YAACQ	−.12	−.26	−.15	−.39	−.33	−.36	.40	.31	.48	.40	.79	.90	4.83	4.89

Note. N = 226. PBSS = Protective Behavioral Strategies Survey; SQ = Strategy Questionnaire; Prob = Alcohol-related Problems; RAPI = Rutgers Alcohol Problem Index; B-YAACQ = Brief Young Adult Alcohol Consequences Questionnaire. Significant effects are bolded (p < .05) for emphasis. The underlined values on the diagonal reflect Cronbach’s alphas for multi-item inventories.

Alcohol Outcomes

Alcohol use (DDQ; Collins et al., 1985) and alcohol-related problems (B-YAACQ: Kahler et al., 2005; RAPI: White & Labouvie, 1989) were assessed using the same measures from Studies 1 and 2.

Results

Bivariate correlations were used to determine the nature of the relationships between scores on the modified versions of the PBS scales and alcohol-related outcomes (see Table 4). Using the contingent frequency response scale with a defined assessment window, all PBSS and SQ subscales were negatively related (significantly or non-significantly) to all six alcohol outcomes. The SQ subscales were significantly negatively correlated with all alcohol outcomes. For the PBSS subscales, five of the six correlations with alcohol quantity were significant, none of the six correlations with alcohol frequency were significant, and three of the six correlations with alcohol problems were significant.

Discussion

With two different PBS measures, Study 3 showed that the assessment of PBS use with a contingent frequency response scale on a defined assessment window (i.e., past 30 days) can result in reliable scales with good concurrent validity. Both the PBSS and SQ subscales were related to alcohol-related outcomes in theoretically consistent ways, and although some previous research favors the use of PBSS over the SQ (Author et al., 2012a) based on factor structure and concurrent validity, Study 3 shows that the SQ had concurrent validity when assessed using the contingent frequency response scale. In fact, the SQ had stronger relationships with alcohol outcomes than the PBSS.

General Discussion

The purpose of the present studies was to improve the assessment of PBS by examining the effect of type of response scale on the estimated relationships between PBS factors and alcohol-related outcomes. Specifically, across the three studies, we manipulated whether an absolute frequency response scale or a contingent frequency response scale was used for each measure. Our goal was to offer practical recommendations to PBS researchers based on these findings.

Summary of Findings

Study 1 showed that the PBSS with a contingent frequency response scale evinced consistent negative relationships with alcohol-related outcomes, whereas the SQ with an absolute frequency response scale evinced some theoretically inconsistent positive relationships with alcohol use outcomes. By using an absolute frequency response scale for the PBSS and a contingent frequency response scale for the SQ in Study 2, we found that we essentially reversed the pattern of relationships; the PBSS evinced the theoretically inconsistent positive relationships with alcohol outcomes, and the SQ evinced all theoretically consistent negative relationships with alcohol outcomes. In Study 3, we saw that both the PBSS and SQ had theoretically consistent positive relationships with alcohol-related outcomes when both were administered using a contingent frequency response scale with a defined assessment window.

To make the comparison of correlations across measures more explicit, we conducted Fisher’s r-to-z transformations to examine whether correlations across Studies 1–3 were significantly different from each other. Correlations between PBS measures and alcohol problems seem to be the most definitive test of concurrent validity as these measures are specifically designed to be related to reducing negative consequences from drinking. Thus, we limited our comparisons to the correlations of each subscale with both alcohol problems measures across Studies 1–3 (see Table 5). For the PBSS, the correlations between each PBSS subscale and each alcohol problems measure were not significantly different from each other in Study 1 and Study 3 when the contingent frequency response scale was used. However, the correlations from Study 2 when the absolute frequency response scale was used were significantly different from both Study 1 and Study 3 for Manner of Drinking and Serious Harm Reduction PBS, and significantly different from Study 1 correlations for Limiting/Stopping Drinking PBS. Thus, overall, the correlations when the absolute frequency response scale was used with the PBSS were significantly less negative (in fact, most were positive) than the correlations when the contingent frequency response scale was used.

Table 5.

Comparing correlations between use of protective behavioral strategies and alcohol problems across Studies 1–3

PBSS	Limiting/Stopping Drinking		Manner of Drinking		Serious Harm Reduction
	RAPI	B-YAACQ	RAPI	B-YAACQ	RAPI	B-YAACQ
Study 1	−.17a	−.21a	−.27a	−.35a	−.22a	−.12a
Study 2	.07b	−.03b	.08b	−.02b	.16b	.18b
Study 3	−.10ab	−.12ab	−.20a	−.26a	−.19a	−.15a

SQ	Selective Avoidance		Strategies Used While Drinking		Alternatives
	RAPI	B-YAACQ	RAPI	B-YAACQ	RAPI	B-YAACQ
Study 1	−.04a	−.05a	−.01a	−.02a	−.06a	−.07a
Study 2	−.21b	−.28b	−.16a	−.19b	−.13ab	−.14a
Study 3	−.28b	−.39b	−.30b	−.33b	−.27b	−.36b

Note. Correlations within a column that share a subscript are not significantly different from each other (p < .05).

For the SQ, all six correlations between each SQ subscale and each alcohol problems measure in Study 1 when the absolute frequency response scale was used was significantly less negative than the correlations from Study 3 when the contingent frequency response scale was used. Three of the six correlations from Study 1 were significantly different from the correlations from Study 2 when the contingent frequency response scale was also used. Interestingly, two correlations (Strategies Used While Drinking→RAPI and Alternatives→B-YAACQ) from Study 3 were significantly more negative than correlations from Study 2. The key difference between these two studies was that the assessment window was defined in Study 3. We do not want to overinterpret two out of six significant differences, but it could be the case that defining the assessment window strengthens the concurrent validity of the SQ. Together, we believe our findings give rather convincing evidence that the assessment of PBS with a contingent frequency response scale is preferable to an absolute frequency response scale. It is important to note that there were no significant mean differences between any of the alcohol outcome variables across the three studies, so our findings could not be accounted for by mean differences or differential restriction of range across the studies.

Different Conceptualizations of PBS

An important issue highlighted by the comparison of these two scales is the different conceptualizations of PBS. Although the PBSS is purposely limited to only behaviors that one can engage in immediately prior to or while drinking, the SQ includes items that involve simply avoiding alcohol as part of the Alternatives to Drinking subscale (e.g., “Choose to participate in enjoyable activities that do not include alcohol consumption”). Among certain populations, for example, children and young adolescents, interventionists are likely focused on helping young individuals avoid alcohol altogether (e.g., Conrod, Castellanos-Ryan, & Mackie, 2011; Loveland-Cherry, Ross, & Faufman, 1999), rather than teaching them how to drink moderately. In such cases, this subscale may be very important and essential to testing interventions designed to increase these alcohol avoidance strategies. In other contexts, for example, among college students, low abstinence rates (~20%) suggest that the vast majority of individuals are likely not interested in avoiding alcohol altogether (Johnston, O’Malley, Bachman, & Schulenberg, 2012); therefore, the ‘safe drinking behaviors’ may warrant more emphasis as they are more consistent with the students’ overall goals.

Limitations

One limitation of the present studies is that they were cross-sectional. In terms of validating the scales, we were only able to show concurrent validity, and not able to demonstrate predictive validity. Although we do believe that future research using longitudinal designs are important to show how use of PBS can influence risky drinking, we do not believe that our cross-sectional design limits the importance of how response scale can rather drastically change the observed relationships between PBS use and alcohol-related outcomes. Another limitation of the present studies was the modest sample sizes. Although our sample sizes were sufficient for examining the concurrent validity of the PBS scales as they have rather robust relationships to alcohol outcomes (when assessed using an optimal response scale), we were unable to detect smaller effects which precluded examination of potential moderators (i.e., gender, race, age).

Conclusion

By switching the response scales of two published measures of PBS, the PBSS (Martens et al., 2005) and the SQ (Sugarman & Carey, 2007), we have shown the importance of using a contingent frequency response scale (i.e., frequency of use “when drinking or ‘partying’”) rather than an absolute frequency response scale when assessing PBS. Consistent with previous research (Sugarman & Carey, 2007; 2009), an absolute frequency response scale led to theoretically inconsistent positive relationships between PBS use and alcohol-related outcomes (Studies 1–2), whereas a contingent frequency response scale led to theoretically consistent negative relationships (Studies 1–3). Although future research should examine how other aspects of PBS assessment (i.e., assessment window) relate to concurrent validity, the present study takes the field of PBS research one step further by showing the unintended effects of using an absolute frequency response scale.