itMatters: Optimization of an online intervention to prevent sexually transmitted infections in college students

Abstract

Objective:

To describe an iterative approach to developing an online intervention targeting the intersection of alcohol use and sexual behaviors among first year college students.

Methods and Participants:

Using the multiphase optimization strategy (MOST), we conducted two iterative optimization trials to: (1) identify candidate intervention components (i.e., descriptive norms, injunctive norms, outcome expectancies, perceived benefits of protective behavioral strategies, and self-efficacy to use strategies); (2) revise components; and (3) identify the optimized intervention. Participants were first year college students at six geographically diverse universities (optimization trial 1 N = 5,880; optimization trial 2 N = 3,551)

Results:

For both optimization trials, the results indicated that only descriptive and injunctive norms produced a significant effect (p<.05).

Conclusions:

The iterative process of MOST allowed us to develop an optimized intervention which is an essential tool to maximize intervention effectiveness and efficiency to improve uptake, sustainability, and public health impact.

INTRODUCTION

In the United States (U.S.), the rate of sexually transmitted infections (STIs) among the college student population is alarmingly high. In 2015, 51% of female and 40% of male students had gonorrhea, chlamydia, and/or genital herpes¹. Casual and higher risk sexual activity is common among college students and contributes to STI risk^2,3. About 75% of college students are sexually experienced⁴ with the majority (60% to 80%) of students reporting at least one “hookup” (i.e., casual sexual encounters without explicit expectation of a dating or romantic relationship) during their college tenure^2,5–9. Overall, 75% of sexually active college students report inconsistent or no condom use¹⁰ and only about half report using a condom at their last sexual encounter¹¹. Condom use among individuals between the ages of 18–24 is particularly low with casual partners or during hook-ups (47% for males; 31% for females)^12,13.

Alcohol use is often implicated in the sexual behaviors that increase college students’ risk for STI acquisition. Among college students drinking is common; 74% consumed alcohol in the past year, 56% consumed alcohol in the past month, and nearly 40% report binge drinking (i.e., 4+ or 5+ alcoholic drinks per occasion for women and men, respectively)¹⁴. Alcohol use, particularly heavy alcohol use, has been shown to lower inhibitions, impair cognitive functions, and increase sexual desire¹⁵. Sexual activity while drinking alcohol is common. For example, one studied reported 21% of college students consumed alcohol at last sexual intercourse¹⁷. Students often attribute positive expectancies (e.g., sexual enhancement) to alcohol use; one survey reported that 52% of U.S. college students believed alcohol “facilitates sexual opportunity”¹⁷. Positive alcohol-related expectancies by college students have been associated with increased number of partners among male students¹⁵, reduced condom use^16,18–20, and increased hookup frequency^15,21–24. Thus, sex in the context of alcohol can lead to engagement in higher risk sexual behaviors and increase the likelihood of contracting an STI¹⁶.

Numerous interventions have targeted alcohol use^25,26 or condom use^27–29 independently, but the common co-occurrence of alcohol use and sexual behaviors among college students necessitates the need for a specialized intervention aimed at STI prevention by purposefully emphasizing the alcohol-sex relationship and incorporating strategies aimed at the intersection. Accordingly, the article’s primary objective is to describe our iterative approach to developing and testing intervention components to be included in an optimized intervention package filling this notable gap in the literature. Our online intervention was designed to reduce sexual risk behaviors and alcohol use among first year college students by directly addressing the alcohol-sex relationship. Given the competing demands on college students’ time, it is critical that this type of intervention is both effective and efficient. Thus, a secondary objective is to illustrate the application of the optimization phase of the multiphase optimization strategy (MOST; detailed below) to arrive at an intervention that is not only effective, but also has been optimized for efficiency, i.e., is made up exclusively of components that provides needed foundational information or has empirically demonstrated effectiveness.

METHODS

This section outlines the development and optimization methods; all study protocols were approved by the Institutional Review Board at the host institution.

Overview of MOST

MOST is an engineering-inspired framework for use in intervention science (see details in Collins³⁰). In the classical treatment package approach that has formed the basis of much of intervention science, an investigator proceeds directly from the preparation phase to the evaluation phase (in a randomized control trial (RCT)), with the implicit assumption that all of the components identified in the preparation phase will be included in the intervention. In contrast, MOST comprises three phases: preparation, optimization, and evaluation. The preparation phase involves establishing a detailed conceptual model that provides the basis for the intervention under development; identifying the components that are candidates for inclusion in the intervention; and pilot testing the components. In the optimization phase of MOST, one or more randomized experiments, called optimization trials, are conducted. The purpose of the optimization trials is to assess the effect of individual intervention components, and possibly, depending on the experimental design used, to examine whether the presence, absence, or level of one component has an impact on the performance of others. In MOST, intervention components are often referred to as candidate components, because it is not a foregone conclusion that any component will be a part of the intervention package. Instead, the eligibility of candidate components for inclusion in the intervention is determined by their performance in the optimization trial(s). Once the optimized intervention has been identified, it can then be evaluated by means of a standard RCT in the evaluation phase of MOST.

itMatters Preparation Phase

The preparation phase focused on development of a set of candidate components to constitute the itMatters intervention. The cornerstone of our process of identifying candidate intervention components consisted of building a conceptual model based on empirical scientific literature and theory and substantive expertise working with our Expert Advisory Panel (EAP).

The itMatters conceptual model (described in Kugler et al.³¹) specifies five candidate intervention components which target distinct mediators, each of which is designed to target sexual behaviors in the context of alcohol use (Table 1). The descriptive norms component was designed to produce accurate perceptions of the prevalence of college student use of alcohol before or during sex. The injunctive norms component was designed to produce accurate perceptions of how acceptable college student peers find the use of alcohol before or during sex. Outcome expectancies aimed to challenge positive expectancies (e.g., that using alcohol will increase the likelihood of engaging in sex) held by college students related to alcohol use before or during sex^19,22,32. Thus, the component was designed to demonstrate to participants that alcohol use is not necessary before or during sex. Perceived benefits targeted desirability of using protective behavioral strategies related to alcohol use (e.g. avoiding drinking games), sex (e.g. condom use), and sexual encounters when drinking (e.g. using a buddy system to avoid sexual assault). Self-efficacy was designed to increase self-efficacy to apply protective behavioral strategies (e.g. condom use when drinking alcohol). In addition, a component provided information, such as standard alcoholic drink definitions and instructions on proper condom use. As this information is foundational to the other components, the information component was not considered a candidate component, so it was not examined in an optimization trial; instead, it was included in the intervention package a priori.

Table 1.

Intervention components, intervention strategies, and outcome measures

Component	Objectives	Intervention Strategies	Outcome Measure
Information	Increase knowledge: STIs, STI risk, alcohol impairment, condom use skills, alcohol use behavior tracking skills, testing & treatment services.	Information on STI prevalence, symptoms, & transmission; factors affecting alcohol impairment (e.g., biphasic effect of alcohol), stratified by gender. Breakdown and modeling (graphics and video) of condom use skills & alcohol use monitoring/tracking. How & where to find STI treatment.	N/A
Descriptive Norms	Correct misperceptions regarding prevalence of specific behaviors. Primary focus: alcohol-induced sexual risk behaviors. Secondary focus: prevalence of alcohol use/misuse, sexual risk behaviors.	National college student data on alcohol use & sexual risk behaviors (e.g., National College Health Assessment, Monitoring the Future) presented as a “guess the norm” activity. Normative, brief personal feedback is presented.	Perceptions of the prevalence of engaging in sexual behaviors while drinking alcohol.
Injunctive Norms	Correct misperceptions. Primary focus: acceptability of sexual behaviors with alcohol use. Additional content focused on correcting misperceptions regarding acceptability of alcohol misuse & sexual risk taking.	Acceptability/approval data from national surveys (e.g., National Survey of Family Growth) & published research. Simulations depicting both subtle and overt approval/disapproval were created using these data.	Perceptions of peer approval of engaging in sexual behaviors while drinking alcohol.
Outcome Expectancies	Primary focus: decrease expectations of positive outcomes of alcohol-induced sexual risk behaviors (e.g., increased sexual enjoyment). Secondary focus: decrease positive expectations of alcohol use & misuse (e.g., increased enjoyment of social interactions).	Identify positive expectancies related to sex & condom use with & without alcohol. Data that challenge positive expectancies is presented. Effects of positive expectancies on risk behaviors are explored. Expectancy science explained using Alcohol Expectancy Challenge “bar labs” examples	Positive expectancies related to engaging in sexual behaviors while drinking alcohol.
Perceived Benefits of Protective Behavioral Strategies	Primary focus: increase the perceptions of the benefits of using specific protective behavioral strategies - limiting drinking, sharing boundaries, using the buddy system, and using a condom.	Identify recommended protective behavioral strategies. Rate the perceived benefit of each strategy within specific circumstance. Provide normative feedback on the use of specific strategies.	Perceived benefits of using protective behavioral strategies
Self-Efficacy	Primary focus: to increase participant self-efficacy to use protective behavioral strategies related to alcohol use (e.g., avoiding drinking games), sex (e.g., condom use), and sexual encounters when drinking (e.g., using a buddy system to avoid unwanted sexual encounters).	Model the use of recommended protective behavioral strategies. Animated simulations and analysis of critical actions and conditions of viewed simulations.	Self-efficacy of using protective behavioral strategies

The process of producing the online material for each component was guided by the conceptual model; our prior online intervention development experience; and recommendations gathered from the literature, campus administrators, college students (the target population), and the EAP. Each of the itMatters candidate components were designed to be sex-positive and developed with end-user engagement in mind.

itMatters Optimization Phase

The optimization phase, which is the focus of this paper, of the itMatters study focused on assessing and improving the effectiveness of the five candidate components via two sequential optimization trials. The overall iterative approach was to conduct optimization trial 1; use the results to identify which of the five candidate components and would be designated Satisfactory (i.e. demonstrated a statistically significant (α=.05) effect at the immediate post-test) and which would be designated Needs Improvement (i.e. did not demonstrate a statistically significant effect); revise the components designated Needs Improvement, making minor revisions to the Satisfactory components where necessary to maintain consistency; and then conduct optimization trial 2 to determine which of the final set of components would be designated Satisfactory. The final optimized itMatters intervention was constructed out of the information component plus the Satisfactory components, that is, those shown to be effective by the optimization trials.

Both of the itMatters optimization trials used the same 2⁵ factorial design depicted in Table 2. A factorial design was selected because it enabled examination of five candidate components using about 20 percent of the participants that would have been required if an individual experiment was conducted for each component^33,34. In this design there is a factor corresponding to each of the five candidate components and each factor has two levels: Yes, where the component was provided, and No, where the component was not provided. A participant in a particular experimental condition received the components in that condition, plus the information component, which was provided to everyone. This experimental design enables estimation of the main effect of each component, and all interactions between components.

Table 2.

Design of Optimization Trials I and II

	Design of Optimization Trials I and II
	Intervention components
Experimental condition number	Descriptive norms	Injunctive norms	Outcome expectancies	Perceived benefits	Self-efficacy
1	No	No	No	No	No
2	No	No	No	No	Yes
3	No	No	No	Yes	No
4	No	No	No	Yes	Yes
5	No	Yes	No	No	No
6	No	Yes	No	No	Yes
7	No	Yes	No	Yes	No
8	No	Yes	No	Yes	Yes
9	Yes	No	No	No	No
10	Yes	No	No	No	Yes
11	Yes	No	No	Yes	No
12	Yes	No	No	Yes	Yes
13	Yes	Yes	No	No	No
14	Yes	Yes	No	No	Yes
15	Yes	Yes	No	Yes	No
16	Yes	Yes	No	Yes	Yes
17	No	No	Yes	No	No
18	No	No	Yes	No	Yes
19	No	No	Yes	Yes	No
20	No	No	Yes	Yes	Yes
21	No	Yes	Yes	No	No
22	No	Yes	Yes	No	Yes
23	No	Yes	Yes	Yes	No
24	No	Yes	Yes	Yes	Yes
25	Yes	No	Yes	No	No
26	Yes	No	Yes	No	Yes
27	Yes	No	Yes	Yes	No
28	Yes	No	Yes	Yes	Yes
29	Yes	Yes	Yes	No	No
30	Yes	Yes	Yes	No	Yes
31	Yes	Yes	Yes	Yes	No
32	Yes	Yes	Yes	Yes	Yes

To enable us to conduct two optimization trials in two years, we evaluated the components in terms of mediators rather than longer-range behavioral outcomes. Because the conceptual model specified that each intervention component operates via a distinct mediator, we evaluated each component using a measure of its respective proximal mediator. Collins³⁰ has argued that this approach is justified in the optimization phase because it will be followed by the evaluation phase, in which the effectiveness of the optimized intervention package will be evaluated in a standard RCT using measures of the ultimate outcomes of interest, in this case the intersection of sexual behaviors and alcohol use. We hypothesized that each component would have a statistically significant main effect on its respective mediator at the immediate post-test. We made no a priori hypotheses concerning interactions, although we tested for interactions in the statistical analysis.

Participating schools and random assignment

Participating schools were co-educational, four-year, public U.S. universities. The universities varied in characteristics such as size, geographic location, and racial/ethnic composition. Universities met the following inclusion criteria: (a) were not currently implementing an online alcohol or safe sex intervention for first-year students; (b) willing to assist in recruiting first-year students by supplying names/email addresses; (c) permitted pretest surveys to be conducted early in the academic years in which experiments were conducted; (d) allowed random assignment of students to intervention components; and (e) agreed to encourage all first-year students to complete the assigned intervention components. A total of six schools were involved in the optimization trials; two were in optimization trial 1 only, two were in optimization trial 2 only, and two were in both optimization trials. First year students were randomly assigned to experimental conditions at the individual level. A priori power analysis indicated that an N = 1,398 would provide us with power ≥ .8 for detecting component main effects and interactions between components. Universities and individual students were provided with modest incentives for participation, as described below.

Optimization Trial 1

Procedure

Data for optimization trial 1 were collected between August and December 2016. All first-year students received an email invitation to participate. Once consented, students had access to data collection instruments and their assigned itMatters components. Student participants had access to Survey 1 for a 3-week window. Following the Survey 1 implementation window, they received a second email inviting them to access to their assigned itMatters components during another 3-week window. After completing their assigned itMatters components, student participants received immediate access to Survey 2 for a 3-week window. Finally, student participants were invited to return and complete the third survey 30 days after completion of Survey 2. Surveys 1 and 3 took approximately 15 minutes to complete, whereas Survey 2 which did not include behavioral items took approximately 10 minutes to complete.

To promote survey participation across all three surveys, monetary incentives were used. Students who completed all three surveys were entered into a prize draw to receive 1 of 40 $50 gift cards to their respective university bookstore. Universities were offered up to $10,000 for serving as an implementation site for optimization trial 1. Institutional incentives were based on a prorated scale; that is, the greater the participation, the larger the incentive. Ultimately, each school received $5,000 for their participation.

Measures

Demographics.

Students provided basic demographic information at all three survey time points. Age, collected as a categorical variable, ranged from 18 to 25+, as students were eligible to participate as long as they were first year students. Students were asked to report their current gender identity (male, female, and other which includes transgender students), sexual orientation (heterosexual or non-heterosexual which includes gay, lesbian, bisexual, and other), race, and ethnicity. Students also indicated if they lived on-campus (e.g., dorm/residence hall, fraternity/sorority housing, or other on-campus housing) or off-campus (e.g., apartment/house off-campus or at home with family).

Behaviors.

To provide context of behaviors, students were asked at Surveys 1 and 3 to report how often they used alcohol and if they indicated any alcohol use, the number of drinks they consumed in the past 30 days; sexual intercourse in past 30 days; hookups in the past 30 days (defined for the students as: “non-penetrative [kissing, touching, oral sex] and/or penetrative [vaginal, anal] behaviors with someone with whom you are not in a committed relationship [friends with benefits] or with someone you just met [one night stand]”); if they reported recent sex or hookup, whether they consumed alcohol before or during; and if they had been tested for HIV/other STIs in the past 6-months.

Mediators.

Each of the five candidate components targeted one mediator and the measurement of these mediators were the outcomes for making decisions about whether a particular component needed revision between optimization trials. The mediator variables were measured at all three survey time points. Measures were informed by previous eHealth research aimed at college students^35,36,37.

Descriptive norms measured students’ perceived norms regarding college students engaging in specific behaviors (e.g., “In the past 30 days, approximately what percentage of college students do you think have…?”). Specifically, the scale included norms about alcohol use, heavy alcohol use, sex, sex with alcohol, hookups, and hookups with alcohol. The seven descriptive norms items (α_t1 = .90) used a 10-point scale (1=0–10%, 2=11–20% etc.). Injunctive norms assessed students’ perceived peer acceptance of specific behaviors (e.g., “In your opinion, how do most college students feel about other college students…?”). The injunctive norm scale included norms about alcohol use, heavy alcohol use, sex, sex with alcohol, hookups, and hookups with alcohol. The seven injunctive norm items (α _t1 = .86) used a 5-point Likert scale (1=strongly disapprove to 5=strongly approve). Outcome expectancies focused on negative and positive alcohol-sexual expectancies. Students were instructed to select the number of alcoholic drinks it would take for them to personally: enjoy sex more, feel closer to a sexual partner, be less nervous about sex, be better at having sex, be less likely to use a condom, be less likely to talk to a new partner about STIs, be less likely to ask a partner to use a condom, have sex with someone with whom they would not normally have sex, be more likely to engage in sexual activities they would not normally do, and find it hard to say no to sexual advances. The 10 expectancies items (α _t1 = .89) used a 5-point scale (1=alcohol would not have this effect on me; 2=1–2 drinks; 3=3–4 drinks; 4=5–6 drinks; and 5=7+ drinks). Perceived benefits assessed the students’ perceived benefit of utilizing protective behavioral strategies (i.e., reduce risk of getting an STI). The 11 perceived benefits items (α _t1 = .92) used a 5-point Likert scale (1=extremely unlikely to 5=extremely likely). The self-efficacy mediator assessed students’ confidence in their ability to enact protective behavioral strategies. The 9 self-efficacy items (α _t1 = .89) were rated on a 5-point Likert scale (1=not at all confident to 5=completely confident). For the present analyses, the proximal mediator outcome variables were calculated as averages of all the scale item scores.

Analytic Methods

A total of 5,880 unique eligible participants consented to participate in optimization trial 1 out of 11,184 invited (52.6% of the total available population). Of the available sample (n = 5,880), 94% completed Survey 1 and 37% completed Survey 2. Initially equal numbers of participants were assigned per experimental condition, after consent procedures and establishment of eligibility per-condition sample sizes ranged between 171 and 201. For participants who had duplicate entries (e.g., participants started survey and needed to restart), the most complete entry was retained and used in analyses; if more than one entry was similarly complete, we selected the earliest one. We used multiple imputation (100 iterations) to handle missing data; 39 participants (n=20 and n=19 for optimization trial 1 and 2, respectively) did not have enough available data to successfully impute their missing scores so they were not included in the final analyses.

Data were analyzed using factorial ANOVA. The ANOVAs included all 5 main effects and all possible interactions (a total of 26 interactions: 10 two-way interactions, 9 three-way interactions, 5 four-way interactions, and 1 five-way interaction). The pre-test measures of the outcome variables were included as covariates in the model. The ANOVAs were conducted separately with immediate post-test scores as outcome variables first and the 1-month follow up scores as outcome variables second. All 33 estimates of the factorial ANOVA (the intercept and 32 effects) were examined analytically. PROC GLM and PROC MIANALYZE were used to perform the analyses on the imputed dataset through SAS 9.4 statistical software. While all estimates were examined, presented here are the p-values for the main effects only. The reason for this is twofold: (1) a significant p-value (p <.05) was the a priori specified criterion for inclusion in the optimized intervention and (2) the principles of effect hierarchy and effect sparsity suggest that main effects are the most important effect scientifically and that of many effect estimated, only a few are important scientifically^30,38. Indeed, no higher order interactions had a significant p-value.

Results

Table 3 summarizes optimization trial 1 demographic characteristics and behavioral frequencies. The sample (n=5,880) was predominantly 18 years old (68%), female (58%), heterosexual (87%), and White (47%). The majority of participants reported living on-campus (53%), but notably of the 40% that lived off-campus, 64% lived at home with family. At baseline, 29% of the sample reported never drinking alcohol, 35% never had sex, and 56% never engaged in a hookup. Of those who reported a hookup (1908), 21% reported using alcohol before or during the hookup. Only 16% of the sample reported testing for HIV/other STIs during the past 6-months.

Table 3.

Baseline Characteristics and Behaviors⁺ of Participants in Optimization Trial 1 and 2

	Optimization Trial 1 (N = 5,880)		Optimization Trial 2 (N = 3,551)
Demographics	n	%	n	%
Age
18	3,997	68	2,172	61
19	470	8	388	11
20	180	3	85	2
≥ 21	883	15	301	8
Sex
Female	3,335	58	1,882	53
Male	2,172	37	1,038	29
Other	23	.40	26	.73
Sexual Orientation
Heterosexual	5,114	87	2,655	75
Non-heterosexual	374	6	272	8
Race
White	2,775	47	1,718	48
Black	843	14	342	10
Other	771	13	637	18
Hispanic	1,746	30	1,040	29
Housing
On campus	3,104	53	1,489	42
Off campus	2,371	40	1,272	36
Engaged in campus activities	1,745	30	1,342	38
Behaviors
Alcohol Use
Never	1,723	29	1,105	31
Used, but not in past 30 days	1,321	22	763	21
Any use in past 30 days	2,182	37	979	28
Sex
Never	2,030	35	1,208	34
Not in the past 30 days	1,051	18	535	15
Any sex in past 30 days	2,108	36	1,050	30
Hookup
Never	3,288	56	1,902	54
Not in the past 30 days	1,109	19	482	14
Any hookup in past 30 days	799	14	404	11
STI Test in past 6 months	917	16	476	13

⁺Note: Participants did not provide data for all variables, so sample size varies

Table 4 presents optimization trial 1 results. One component, injunctive norms, had a statistically significant effect on the respective outcome at the immediate post-test. None of the other four components had significant main effects on their respective outcomes. No component had a significant effect on an outcome other than its own, and there were no significant interaction effects.

Table 4.

Optimization Trial 1 Factorial ANOVA results of the effect of candidate components on mediators assessed at Time 2 (p-values)

		Mediators
Component		Descriptive Norms	Injunctive Norms	Expectancies	Perceived Benefits	Self-Efficacy
	Descriptive Norms	0.058	0.24	0.92	0.95	0.83
	Injunctive Norms	0.24	<.0001*	0.63	0.53	0.55
	Expectancies	0.53	0.99	0.91	0.49	0.57
	Perceived Benefits	0.70	0.89	0.44	0.55	0.76
	Self-Efficacy	0.71	0.63	0.83	0.73	0.85

^*indicates component met a priori criteria of significance p <.05

Revision process

Optimization trial 1’s results indicated that four components—descriptive norms, outcome expectancies, perceived benefits, and self-efficacy—should be designated Needs Improvement. An optimization trial can point to which components need revision, but it does not suggest what should be done to revise a component. To assist us in the revision process we gathered input from focus groups with students and campus administrators at each campus as well as an interview with each EAP member. We also continued to rely on the conceptual model. The revision process focused on the four components designated Needs Improvement, but revisions were made to the Satisfactory component as needed to maintain consistency, or where a compelling suggestion emerged. Based on the collective feedback, revisions included: changed narrator (student feedback), shortened component length (administrator feedback), and measures (EAP feedback).

Optimization Trial 2

Procedure

Data for optimization trial 2 were collected between August and December 2017. Procedures for optimization trial 2 varied slightly from optimization trial 1. Student participants had access to Survey 1, their assigned itMatters components, and Survey 2 all in the same 3-week implementation window; each was a prerequisite for the next. This procedure reduced the number of invitations a student would receive throughout the implementation process and thus reduced potential participant burnout. Student participants were invited to return and complete the third survey 30 days after completion of Survey 2. As in optimization trial 1, Surveys 1 and 3 took approximately 15 minutes and Survey 2 took approximately 10 minutes to complete.

Based on student feedback, a different incentive plan was used during optimization trial 2. Rather than a lottery style incentive, students suggested that a guaranteed incentive payment would result in greater interest to participate. Therefore, upon completion of surveys 1 and 2, students received a $5 Amazon gift card. Once they completed Survey 3, the student participants then received an additional $5 Amazon gift card. During optimization trial 2 universities were offered $4000 for participating - $2500 was guaranteed and $1500 could be used for marketing purposes.

Measures

Following optimization trial 1, measures were updated to reflect changes in modules. Demographic questions remained the same but were only asked at Survey 1 and 3 to reduce response burden (and because students completed surveys 1 and 2 at close timepoints). Behaviors were assessed at Survey 1 and 3, as in optimization trial 1. Each of the five candidate components again targeted one mediator, the measurement of which was used to decide if the component would move on to the optimized intervention. The mediators remained the same conceptually, but small revisions were made to the construction of each scale.

Based on student focus group feedback, during the revision process we made it a priority to make the assessments more concise, removing extraneous item where possible. To this end, the descriptive norms scale was revised to include 4 items (α _t1 = .84): prevalence of any alcohol use, heavy alcohol use, sex with alcohol, and hookups. The injunctive norms scale was revised to include the perceived peer norms of the same 4 items as the descriptive norm scale (α _t1 = .84). Due to the significant revision of the expectancies component, the corresponding scale was revised extensively. All items from optimization trial 1 were replaced with the 10-item Brief Comprehensive Effects of Alcohol (B-CEOA) questionnaire (α _t1=.81)³⁹. Students indicated to what degree they agreed that a particular effect would occur to them from drinking (e.g., courageous, better lover, clumsy, dizzy, etc). The perceived benefits and self-efficacy scales were replaced with the Protective Behavior Strategies Scale-20⁴⁰. The perceived benefits scale included 4 items (α _t1 = .58) assessing how beneficial the student believed it was to: limit drinking, use a condom while having sex when using alcohol, use the buddy system, and share boundaries about having sex or sexual behaviors with a partner. The self-efficacy scale included the students’ confidence in using the same 4 protective behaviors as described in the perceived benefits scale (α _t1 = .60). As in optimization trial 1, the mediator outcome variables were calculated as averages of all the scale item scores for the optimization trial 2 analyses.

Analytic Methods

A total of 3,551 unique eligible participants consented to participate in optimization trial 1 out of 20,777 invited (17% of the total available population). Of the available sample (n=3,551), 83% completed Survey 1 and 50% completed Survey 2. Again, we initially assigned equal numbers of participants per condition, consent to participate and attrition over study duration caused cell sizes to range from 94 to 137. The data analysis approach, including handling of missing data, was identical to that used in optimization trial 1.

Results

Table 3 summarizes optimization trial 1 demographic characteristics and behavioral frequencies. The sample (n=3,551) was predominantly 18 years old (61%), female (53%), heterosexual (75%), and White (48%). The majority of participants reported living on-campus (42%), and of the 36% that lived off-campus, 60% lived at home with family. At baseline, 31% of the sample reported never drinking alcohol, 34% never had sex, and 54% never engaged in a hookup. Of those who reported a hookup, 23% (203) reported using alcohol before or during the hookup. Only 13% of the sample reported testing for HIV or other STIs during the past 6-months. Note, demographic characteristics and behaviors were not statistically compared across optimization trials.

Results of optimization trial 2 are presented in Table 5. The results indicated that descriptive norms and injunctive norms, and none of the other three candidate components, demonstrated significant effects. As in optimization trial 1, no component had a significant effect on an outcome other than its own, and there were no significant interaction effects.

Table 5.

Optimization Trial 2 Factorial ANOVA results of the effect of candidate components on mediators assessed at Time 2 (p-values)

		Mediators
Component		Descriptive Norms	Injunctive Norms	Expectancies	Perceived Benefits	Self-Efficacy
	Descriptive Norms	0.0001*	0.07	0.86	0.39	0.60
	Injunctive Norms	0.05*	<.0001*	0.20	0.31	0.35
	Expectancies	0.54	0.66	0.59	0.69	0.99
	Perceived Benefits	0.77	0.79	0.39	0.27	0.26
	Self-Efficacy	0.54	0.79	0.59	0.91	0.14

^*indicates component met a priori criteria of significance p <.05

DISCUSSION

Interventions to support college student sexual health are essential, however, the mechanisms (e.g., descriptive and injunctive norms) that may increase health risk are complex. For instance, college students tend to underestimate the prevalence of protective behaviors (e.g., condom use) and overestimate the prevalence of risk behaviors (e.g., drinking prior to sex)⁴¹. As such, this article describes: (1) our iterative approach to developing candidate intervention components informed by a conceptual model³¹; (2) assessment of the effectiveness of the candidate components in achieving their short-term outcomes; (3) the decision making process for which components to include in the itMatters behavioral intervention; and (4) the application of MOST as a best practice framework for the optimization of behavioral interventions. Aligning with prior work acknowledging the association of norms with alcohol use and sexual behaviors^41,42, the final results of the optimization trials indicate that the two components focused on norms, descriptive norms and injunctive norms, met the specified performance criterion on their respective mediators, and that the other components did not. Two subsample analyses were conducted for optimization trials 1 and 2 among participants who (a) reported alcohol use and (b) having sex at baseline. Results from the subsample analyses replicate the findings from the full sample.

An understanding and appreciation of the overall process and/or order of events is critical to truly benefiting from the insights and lessons shared in this paper. The process was unique in that it did not begin with developing itMatters, an online intervention to prevent STIs in college students, as a package, and then unpacking or separating it into candidate components for testing. Our systematic process of applying the preparation and optimization phases of MOST³⁰ led us to first identify five empirically and theoretically driven candidate components related to the intersection of sexual behaviors and alcohol use as informed by the conceptual model. An iterative process of developing, testing, and revising each candidate component was undertaken to identify which version(s) of the candidate components were active. By default, a candidate component was not considered “active” if neither version produced a meaningful effect on the targeted outcome. Initial development of the five candidate components was guided by our previous research and development experience as well as recommendations from the literature, college students, campus administrators, and the EAP. Similarly, the revision process of the candidate components was guided by our own analysis of the component content and instructional strategies as well as feedback from college students, campus administrators, and the EAP.

Based on the results of the sequential optimization trials, the optimized itMatters intervention is made up of three components: the two norms components along with the a priori required information component. The end result of the optimization phase of MOST ensured that of the five candidate components, only those that demonstrated effectiveness were selected to constitute itMatters. It is important to note that it was not until the end of the optimization phase that we actually had the behavioral intervention (itMatters). Up to this point, we focused on developing and evaluating candidate intervention components. The end result is a behavioral intervention that is optimized on efficiency (i.e., no dead weight/inactive components) and effectiveness (i.e., includes only active components that met an a priori standard of performance). We are currently evaluating the optimized itMatters intervention in an RCT to assess its impact on the more distal outcomes related to the intersection of sexual behaviors and alcohol use among college students.

Our conceptual model³¹ identified key mediators essential for college student sexual health promotion and all of these mediators remain important. Upon initial review of the findings of the optimization trials, there are some aspects of the results that could be viewed as discouraging. Only one of the five candidate components met the specified performance criterion in the first optimization trial and that despite our careful approach to revisions only two of the candidate components met the criterion in the second optimization trial. The ineffectiveness of some of the candidate components (i.e., outcome expectancies, perceived benefits, and self-efficacy) highlights the inherent challenge of developing effective components and therefore, there must be careful interpretation of these results. For example, failure of producing an active component does not mean that selection of the targeted mediating variable is not justified. There are three possible conclusions. First, as outlined in the results sections, we saw relatively low prevalence of sexual behaviors and alcohol use among our targeted sample. It is quite possible that the students we were able to successfully engage through our recruitment efforts held more conservative outcome expectancies, more accurate perceptions on the benefits of harm reduction strategies, and high levels of self-efficacy to use harm reduction strategies compared to other national samples^1,4,11. This would make it more difficult to observe a statistically significant effect size that meets the a priori performance criterion. Second, alternative revisions to content and instructional strategies might have produced better results. The challenge is in developing content and instructional strategies within candidate components that are capable of producing change in the targeted mediating variables since an optimization trial can reveal which components are not working as expected, but it cannot reveal what does not work. A final possibility is that we did have change, but the measures of the mediators used may not have captured the change due to the large amount of error. Due to this issue, we changed measures between the trials (which also introduces a potential measurement issue for differences in the results between trials). As such, there is a need for better measurement of the mediators at the intersection of sexual behaviors and alcohol use.

Other aspects of the results were more positive. First, we demonstrated the effectiveness of the injunctive norms component in optimization trial 1 and replicated this finding based on a new sample in optimization trial 2. Second, through the intervention module revision process, the descriptive norms component moved from inactive (optimization trial 1) to active (optimization trial 2). Lastly, we learned what does not work when targeting outcome expectancies, perceived benefit of harm reduction strategies, and self-efficacy to use harm reduction strategies. This provides a specific starting point for taking the next steps in future research to improve the inactive components, potentially further optimizing itMatters by adding more active components to the intervention. Towards this end, our research team has begun to consider what alternative revisions may produce more desirable effects.

Consistent with a long and varied literature^43–45, the results of the two optimization trials demonstrate the importance of targeting norms in behavioral interventions aimed separately at sexual behaviors and alcohol use. Therefore, we plan to use a new norms approach to develop components aimed at outcome expectancies, perceived benefit of harm reduction strategies, and self-efficacy to use harm reduction strategies. For example, instead of trying to increase negative alcohol-related expectancies and/or decrease positive alcohol-related expectancies directly, the behavioral expectations component will use an approach centered on establishing social norms about the prevalence and acceptability of positive and negative behavioral expectations. We plan to conduct an optimization trial to assess the effectiveness of this innovative approach.

The iterative approach

Our iterative approach involved two sequential optimization trials. The optimization phase of MOST may involve one or more optimization trials. In other words, the iterative approach we used is not a required part of MOST. Previous applications of MOST (e.g. smoking cessation⁴⁶, weight loss⁴⁷) have used a single optimization trial, or several trials that examined different components, rather than iteratively examining a set of components, revising as needed, and re-examining them. The more typical non-iterative approach to MOST enables optimization of the intervention by selecting from a given set of components but does not enable any component-specific improvements.

The iterative approach may be particularly well suited to intervention research in educational settings. One limiting factor for conducting sequential optimization trials is the length of time it takes to recruit subjects. In clinical settings, recruitment of prospective participants can take months or years. In educational settings, a new cohort of subjects presents itself at least once per year. In addition, our intervention components were all delivered online, making training of human facilitators unnecessary (improving fidelity) and greatly simplifying the logistics of implementing a factorial experiment. These two considerations help make it feasible to conduct several optimization trials within the typical grant funding period.

Limitations

Several limitations of this work should be addressed. The percentage of the available student population agreeing to participate across the two optimization trials was relatively low (53% & 17%). Although internet-based interventions are useful given their reach^48–50, they may not be reaching the students at highest need (e.g., heavy drinkers and those who use condoms inconsistently) or those not reading their emails. Thus, future iterations may require a university-imposed mandate to ensure that all first-year students complete the intervention. Additionally, there was a significant drop in the percentage of available students agreeing to participate from optimization trial 1 to optimization trial 2. There are three possible explanations for this drop. First, in optimization trial 2 one school contained over 50% of our target population but only contributed 20% of the engaged participants. Second, during optimization trial 1 the pretest survey (Survey 1) was administered during a three-week window. The intervention (itMatters) and the immediate posttest (Survey 2) were packaged together. The fact that the initial ask was to only complete Survey 1 may have been viewed by students as not burdensome and therefore encouraged their participation. Third, in optimization trial 2, the pretest (Survey 1), intervention (itMatters), and the immediate posttest (Survey 2) were bundled in one package. The significant drop in participation could have been partly due to the fact that the initial ask of three tasks was viewed as overly burdensome resulting in decreased participation. The response rates for Survey 2 suggest some benefit for bundling the tasks together.

As aforementioned, we revised and streamlined our assessments between trials based on student and EAP feedback to reduce participant time burden and ensure alignment with the itMatters components. These changes could have inadvertently produced more positive results. Notably, changes were made across all mediator measures and significant results were only seen in descriptive and injunctive norms.

The effectiveness/lack of effectiveness of the five components examined in the optimization trials is conditional on the inclusion of the information component. Unlike the candidate components, the information component was not experimentally manipulated, because it contained information that was considered foundational for the candidate components. The experimental design used does not permit examination of whether the information component interacted with any of the other components. For example, it is possible that if the information component were not included, one or both of the norms components would not be effective. This is worth noting, but ultimately may be of little consequence given that the information component is an essential part of the intervention and likely would not be a candidate for removal.

CONCLUSIONS

This systematic and rigorous process to the identification and testing of candidate intervention components and ultimate packaging of active components to create a behavioral intervention to address college students’ alcohol use and sexual behaviors was informed and guided by MOST. To our knowledge, this is the first application of the MOST framework to develop an optimized intervention aimed at college student sexual health. This approach has enabled us to develop itMatters, a highly efficient intervention made up of just three components. Given both the complexity of students’ alcohol and sexual behaviors⁵¹ and the busy lives led by many college students, an effective and efficient intervention that does not waste time with inactive components is desirable. Readers who may be made uncomfortable by this minimalist perspective may wish to consider that for the vast majority of evidence-based interventions it is currently unknown whether the observed effects are attributable to all, most, a small subset, or one of the components included in the intervention. This cannot be revealed by an RCT; it takes an optimization trial to establish the effectiveness of individual intervention components.