Open Pilot Trial of an Interactive Digital Application for Campus Sexual Violence Prevention

Abstract

Digital applications, or “serious games” for health address learning goals in a cognitively active, interactive manner, with potential for widespread dissemination. This study used a mixed methods approach to develop and conduct a formative evaluation of a digital application for sexual assault prevention. Make a Change is a digital application which uses the principles of games for health to foster learning, engagement and skill-building around risk and protective factors for sexual victimization, sexual aggression, and bystander intervention. The digital application includes four narrative chapters, six embedded activities, as well as a user-derived change plan in which individuals establish goals for behavior change following program completion. This multisite study at a 2- and a 4-year college utilized student interviews (n = 14), stakeholder interviews (n = 10), and focus groups with students (n = 40) to inform intervention development. A total of 41 college students then participated in an open trial and completed self-report surveys (pre, post, and one month follow-up) to evaluate the feasibility, acceptability, utility, and preliminary outcomes. Most of the sample reported enjoyment, usefulness, and perceived competence after completing the application. Data evidenced a trend to reduce frequency of heavy drinking, and perceptions of social norms evidenced change over time. Findings support the feasibility and effectiveness of this novel format for delivery of sexual assault prevention programming.

Sexual violence, which includes a range of unwanted sexual experiences, from unwanted touching to completed rape, is a serious public health problem at both 2-year and 4-year colleges in the United States (Fedina et al., 2018). Approximately 20% of undergraduate women have experienced some form of sexual victimization during college (Krebs et al., 2009). Sexual assault is also prevalent among students enrolled at 2-year/community colleges (Potter et al., 2020). For example, one study found that 34% of community college women in New England indicated some form of sexual victimization since the age of 14 (Orchowski et al., 2018). Sexual victimization is associated with numerous mental and physical health consequences, including posttraumatic stress, substance use, and chronic health conditions (Zinzow et al., 2012). Accordingly, the prevention of sexual violence across both 2-year and 4-year college campuses is a public health priority.

Despite the importance of preventing sexual violence on college campuses, and the growth of sexual assault prevention approaches in higher education (Kafonek & Richards, 2017), there are a limited number of prevention programs which show efficacy in reducing rates of sexual aggression among men (Gidycz et al., 2011; Salazar et al., 2014), and victimization rates among women (Hollander & Cunningham, 2020; Orchowski et al., 2008; Senn et al., 2017). Because the aforementioned sexual violence prevention programs are gender-specific, there is a high need for programs that can be implemented for college students regardless of gender. Although programs that specifically engage bystanders as agents of change in reducing rates of violence are implemented for all members of a campus community (e.g., Banyard, Moynihan, & Plante, 2007), there are limited data to suggest these programs are effective in reducing perpetration of sexual aggression (Jouriles et al., 2018). Given the prevalence of sexual violence on college campuses, researchers in the field highlight the need to develop and evaluate comprehensive sexual violence prevention programs which integrate content pertinent to victimization risk, perpetration risk, and bystander intervention skills (Orchowski Edwards et al., 2018). In addition, none of the sexual violence prevention programs reviewed in DeGue et al.’s (2014) review of the literature focused on 2-year/community college students. Given that 2-year community college students make up a considerable group of post-secondary students – with nearly 10.3 million students enrolled for credit or non-credit courses in the United States in 2022 (American Association of Community Colleges, 2022) – research which includes this demographic of young adults in the development and evaluation of sexual violence prevention programs is warranted.

The Use of Digital Applications to Promote Attitude and Behavior Change

Digital applications are now recognized as an important component of violence prevention efforts (Lilley & Moras, 2017; Lindsay et al., 2013; Potter et al., 2020; Russon, 2015). Eisenhut et al. (2020) identified over 171 digital applications that addressed violence against women, with content focused on emergency response, avoidance behavior, education, reporting, evidence building, and support. Despite the growth of digital applications for violence prevention and response, few digital applications for sexual violence prevention have been rigorously evaluated (Beaton, 2015).

Digital applications that function as “serious games” and “games for learning” are established and easily accessible products for promoting learning and behavior change (Connolly et al., 2012; DeSmet et al., 2014; 2015; Hainey et al., 2016; Hirsh-Pasek et al., 2015). Serious games combine narrative storyboards with interactive digital experiences to address specific learning goals in a manner that is cognitively active, engaging, and meaningful (Whitton, 2010). Like other digital applications, serious games include a backstory situated within an immersive environment, and facilitate engagement through interactivity, challenge, and competition (Novak, 2015). Serious games are set apart from other digital apps through their utilization of supported theories of behavior change – such as social cognitive theory (Bandura, 2004) and self-determination theory (Ryan & Deci, 2000)– which serve to facilitate specific, measurable, sustained changes in behavioral outcomes (Baranowski et al., 2016). Similar to how in-person health interventions are driven by a logic model, the narrative content and embedded activities of a serious game align with the putative changes in attitudes and skills necessary for facilitating a specific behavioral outcome (Duncan et al., 2014; Shen et al., 2009; Starks, 2014). Although meta-analyses document the efficacy of serious games for health (Clark et al., 2016; DeSmet et al., 2015; Wouters et al., 2013), most evidence-based serious games are not widely available for public use, and most commercial serious games have not been rigorously evaluated (Hirsh-Pasek et al., 2015). To our knowledge, there are currently no commercially available serious games that show efficacy in reducing rates of sexual victimization and sexual aggression among college students.

Development of a Digital Application for College Sexual Assault Prevention

To address this gap in prevention technology, this study sought to develop and conduct a preliminary evaluation of Make a Change, a novel digital application for college sexual violence prevention for both 2-year and 4-year college students. The digital application employs a serious game design to target a range of empirically-based and theoretically-grounded risk and protective factors for victimization and perpetration risk (Edwards & Sessarego, 2018; Tharp et al., 2013), as well as bystander intervention skills (Burn, 2009). More specifically, the grounding theoretical and etiological models utilized to guide content development for the digital application include the Integrated Model of Sexual Aggression (i.e., “Integrated Model”, Berkowitz, 1994; 2003; 2010; Orchowski & Berkowitz, 2022), the Assess Acknowledge Act (AAA) approach to sexual violence risk reduction and resistance education (Orchowski et al., 2008; Rozee & Koss, 2001; Senn et al., 2017), and the situational model of bystander intervention (Burns, 2009). Further, given that sexual violence prevention is likely to be most efficacious when multiple risk and protective factors are addressed across the social ecology (Centers for Disease Control and Prevention, 2004), the intervention addresses individual risk/protective factors, perceived peer influences, as well as community norms.

In addition to providing definitions of sexual violence and consent, the intervention targets individual protective factors for victimization risk. According to an “AAA” approach to risk reduction and resistance education (Rozee & Koss, 2001) recognizing risky social and dating situations is vital to reducing risk for sexual victimization (Brecklin & Ullman, 2005; Vitek et al., 2018; Yeater & O’Donohue, 1999). Accordingly, Make a Change includes content designed to enhance students’ abilities to assess risky social and dating situations, delineate the psychological barriers that make it difficult to acknowledge when situations are risky, and highlight the importance of acting quickly (on behalf of yourself or a friend) when risk is detected (Rozee & Koss, 2001). Programs which utilize an AAA approach to risk reduction show promising results in increasing self-protective behavior (Gidycz et al., 2001; Gidycz et al., 2006; Gidycz et al., 2015) and reducing rates of victimization over short-term (Orchowski et al., 2008) and long-term follow-up periods (Senn et al., 2017).

Aligning with the Integrated Model (Berkowitz, 1994; 2003; 2010; Orchowski & Berkowitz, 2022), the intervention also addresses attitudes commonly associated with perpetration of sexual aggression (i.e., rape myth acceptance, adherence to traditional masculine beliefs), the role of alcohol use in sexual violence, and perceptions of peer norms. Numerous studies document an association between rape myth acceptance (Edwards et al, 2011; Tharp et al., 2013) as well as hostile masculinity (Murnen et al., 2002; Thompson et al., 2011; Widman et al., 2013) and sexual aggression. Consistent with other sexual violence prevention programs (Gidycz et al., 2011; Orchowski Barnett et al., 2018; Salazar et al., 2019), Make a Change targets rape myth acceptance and ascription to hypermasculine beliefs as a core component of the curriculum.

According to the Integrated Model, rape supportive attitudes and beliefs interact with situational factors, such as alcohol use and peer norms, to facilitate sexual aggression (Orchowski & Berkowitz, 2022). Prior research has established an association between men’s personal alcohol use and their perpetration of sexual violence (Abbey et al., 2022; Abbey et al., 2011; Parkhill & Abbey, 2008). Alcohol use can also decrease the likelihood that individuals step in to intervene as proactive bystanders when witnessing risk for sexual violence among their peers (Leone et al., 2018; Orchowski et al., 2016). Based on this research, Make a Change maintains a focus on how alcohol use influences risk for sexual violence.

The Integrated Model also discusses how perceived peer norms serve to facilitate sexual aggressive behavior and limit bystander intervention behavior (Orchowski & Berkowitz, 2022). Numerous studies document an association between perceived peer norms and risk for violence perpetration (Bohnner et al., 2006; Dardis et al., 2016), as well as likelihood of engaging in bystander intervention (Austin et al., 2016; Mulla et al., 2022). For example, students tend to overestimate the extent to which peers are engaging in heavy drinking and approve of coerced sex and underestimate the likelihood that peers will support active bystander intervention (see Berkowitz et al. 2022 for a review). Several studies suggest that provision of real time feedback to correct these misperceptions can serve to foster healthier behavior among individuals (Gidycz et al., 2011; Mulla et al., 2019; Orchowski Barnett et al., 2018). Drawing upon this research, Make a Change includes activities where participants were asked to estimate the beliefs and behaviors of their peers and were provided with feedback designed to correct misperceptions of peer norms.

Finally, the Make a Change digital application is grounded in bystander intervention theory and training models. Based on the social psychological theory of Darley and Latané (1968) describing factors that influence helping behavior, and articulation of a situational model of bystander intervention to address sexual violence (Burn, 2009), Make a Change seeks to increase students’ ability to notice situations that pose risk for sexual violence, label these situations as problematic, and take responsibility for intervening. In a review of the literature, Mujal et al. (2021) described 44 different studies which reported on the outcomes of bystander intervention training programs for sexual violence prevention, with most studies documenting positive outcomes. Bystander intervention skills training is seen as an important component of comprehensive sexual violence prevention efforts which engage all members of college campuses in taking action to create a positive climate which supports victims and interrupts behavior that facilitates harm (Banyard et al., 2004; Orchowski et al., 2018).

Components of the Make a Change Digital Application

The Make a Change digital application consists of four branching narrative chapters delivered in a graphic novel style, five embedded activities, and a user-derived “Change Plan.” The branching narrative focuses on a group of college students, and risks for sexual assault in a party scenario were described. The choices students made in the branching narrative exposed them to varied situations and could result in different outcomes. After completing the digital application, students have the opportunity to play through the application additional times to explore how varied choices in the application resulted in different outcomes. Within the context of the current research study, students were required to proceed through the branching narrative of the digital application at least twice. Although the length of time spent in the application varied as a function of the number of times each activity is played and the user’s reading rate, most users complete one play-through of the application in approximately 30 minutes.

The embedded activities are designed to promote “minds-on learning” (Baranowski et al., 2016). The embedded activities align with the content of each preceding branching narrative chapter, evoking a feeling of progressing through the digital application. Each section of the narrative was “unlocked” after activities were completed, incentivizing engagement, repetitive practice of skills, and deeper learning. Embedded activities are grounded in principles of “serious games” and sought to engage users and promote skill building. Activities included: 1) This or That: Making choices about how to respond to difficult situations, with corrective feedback provided; 2) Card Sorting: Sorting choices in the narrative component of the application according to level of risk, with corrective feedback provided; 3) Social norms questions with feedback: Answer questions and receive real time feedback regarding actual norms; 4) What Would You Say: Choose how to respond to a situation that poses a risk for harm with several potential options, followed by corrective feedback; 5) Incentivizing Heart System: Provides feedback on how interactions within the application impact various characters in the branching narrative. In addition to these five embedded activities, participants also complete a “Change Plan” activity after completing the app, which included the development of a personalized plan including specific behavior changes that a participant commits to implement following the intervention. All activities are completed within the digital application, which was completed online via a tablet computer. Of note, real-time feedback on incorrect perceptions of peer social norms is a well-established strategy for addressing substance use among college students (Larimer et al., 2007; Lee Neighbors et al., 2010), which is now being applied to address risk for sexual violence (Jaffe et al., 2021). Studies also support the utilization of developing a change plan to facilitate change in health behaviors (Lee Baird et al., 2010; Magill et al., 2010).

Purpose of the Current Study

The purpose of this study was to develop and test the feasibility and preliminary efficacy of a prototype of the Make a Change digital application in a sample of 2-year and 4-year college students. The intervention was iteratively developed through a series of stakeholder interviews, student interviews, and focus groups with students at two study sites. The resulting intervention was then tested in an open pilot trial at the two study sites, with outcomes measured through baseline, post-test, and 1-month follow-up assessments. Given the formative scope of the study, assessments focused on the feasibility of the digital application, as well as participant engagement, perceived usability, comprehension, and competence. Outcome assessments aligned with the putative mechanisms of change and targets of the intervention. Specific aims of the study were as follows:

• Aim 1) Evaluate product design and execution viability on three vectors: 1) overall utility, ease of use, and satisfaction with the application content; 2) examination of the utility of specific application components; and 3) describe the ways in which participants utilized an activity that entailed creating a personalized plan for behavior change.

• Aim 2) Document the preliminary efficacy of Make a Change in promoting change in proximal (attitudinal) and distal (behavioral) outcomes relating to sexual violence with valid and reliable self-report surveys, including: (a) Examining rates of sexual victimization and sexual aggression perpetration at 1-month; (b) Reducing risk factors for sexual aggression at 1-month, including rape myth acceptance, perceived peer approval for sexual aggression, peer pressure for sexual activity, knowledge of consent; (c) Increasing self-protective behavior for sexual violence 1-month post-intervention by decreasing overall levels of psychological barriers to resistance, and increasing overall levels of alcohol use self-protective behavior; and (d) Increasing proactive bystander behavior to address risk for violence among peers at 1-month post-intervention, including self-efficacy in intervening and actual engagement in bystander intervention. Exploratory analyses also examined the impact of the intervention on quantity and frequency of drinking.

• Aim 3) Examine the extent to which the digital application facilitated change in participants’ estimates of beliefs and behaviors of their peers, which were addressed via social norms questions within the digital application.

Method

Intervention Development

All developmental activities were IRB approved and implemented with informed consent. The development of Make a Change was first guided by informant interviews with students, stakeholder interviews with campus administrators, and focus groups with students. To ensure that the resulting product was viable for dissemination at a wide range of post-secondary institutions, development activities were iterative and concurrently administered at two study sites, including one large public 4-year university and one large 2-year community college. Research participants were provided with $40 for participating in the interviews and focus groups. Interviews and focus groups were held on campus at each study site. Participants were recruited through email correspondence and flyers distributed by the study team, which advertised the research as an opportunity to provide feedback on a program promoting safe social and dating behavior.

First, with research consent, two waves of interviews with 14 students (eight men, six women) were utilized to inform the development of the digital app. Students were provided with a copy of the “storyboard” and provided feedback on the learning goals, characters and narrative, comprehension of program content, and the utility and clarity of skill-building activities. Next, with research consent, stakeholders were asked to comment on the commercial viability of the intervention (i.e., awareness of the problem, prior strategies utilized to address the problem, receptivity to digital applications), pragmatic implementation concerns, and compatibility (e.g., “Who would participate in the decision-making process to implement the program?”). Enrollment included 10 stakeholders, including one man and nine women. All interviews were audio-recoded and transcribed. Feedback was utilized to iteratively revise and update app components. Once the prototype was developed, the digital application was presented to four gender-stratified focus groups (N = 40; 19 men, 20 women, and one transgender participant) for review, piloting, and feedback. All participants in the focus groups provided informed consent. Focus groups were conducted at each study site. Feedback was audio-recorded and transcribed, and information was utilized to further refine the digital application. These iterative development activities resulted in the final version of the Make a Change intervention, which was evaluated in the open trial. Following the recommendations of Duncan et al. (2014), the research team also prepared a Digital Intervention Manual, which documented how the content of the Make a Change digital application aligned with social cognitive theory (Bandura, 2004) and self-determination theory (Deci & Ryan, 2000).

Procedure for Open Pilot Trial

Make a Change was piloted in a non-randomized open trial. The study was registered at clinicaltrails.gov (NCT03820609). Procedures were approved by the IRB. Participants were recruited from a 2-year and 4-year college. A total of 41 students provided informed consent for the research and enrolled (n = 21 from the 4-year university and n = 20 from the 2-year college). Students were recruited via flyers, email, and online announcements. Emails and advertisements described the study as an opportunity to provide feedback on an intervention addressing social and dating behavior. After providing written informed consent, participants completed a baseline survey. A member of the research team then asked participants to complete the digital application program on a tablet device, and then completed the post-test survey. At the conclusion of the study session, participants were asked to take the tablet with them and play through the app a second time, making different choices in the narrative to see how different choices affected story outcomes. Participants returned to complete a one-month post-intervention survey. The surveys were anonymous and linked only by a self-generated ID. Study sessions were held in classrooms or small group meeting rooms at each respective study site. All participants who consented to participate in the research completed the application, and we retained 100% of participants at post-test and follow-up. Participants received a $30 electronic gift card after completion of the post-test survey, and a $50 electronic gift card after completion of the one-month follow-up survey and could keep the tablet.

The pre-test was conducted between June – August of 2018, and the post-test was conducted between July-September of 2018. No participants declined to participate after receiving the informed consent information. No participants withdrew from the study while completing the intervention or study questionnaires. All participants opened all narrative comments. All users played through all embedded activities. Time spent in each narrative chapter ranged between four to eight minutes. The average time to complete the application was 30 minutes. All participants were retained at the one-month follow-up.

Participants

Participants in the open trial were 39% men (n=16), 56% women (n=23), and 2.4% transgender (n=1). One participant did not report demographic characteristics. Participants were between 18 and 24 years of age (M = 21.02, SD = 1.62), and included 7% freshman (n=3), 32% sophomore (n = 13), 22% junior (n = 9), 27% senior (n = 11) and 12% 5^th year students (n = 5). Participant race was 15% African American (n= 6), 5% Asian (n= 2), 70% Caucasian (n= 29), and 10% unreported (n= 4). The sample was 15% Hispanic/Latino (n= 6). In this sample, 12% reported a history of sexual aggression since the age of 14, and 49% reported a history of sexual victimization since the age of 14, as assessed with the Revised Sexual Experiences Survey (Koss et al., 2007).

Measures

Demographics.

A brief questionnaire assessed gender, race, ethnicity, sexual orientation, and year at the college/university.

Alcohol use.

Current quantity and frequency of alcohol use were assessed with items from the Drinking and Drug Use History Questionnaire (DDHQ; Zucker, Fitzgerald, & Noll, 1990). Quantity of alcohol use was first assessed. Participants also indicated the largest number of drinks that they consumed in a 24-hour period in the past 30 days, ranging from “0 drinks” to “36 or more drinks”. Frequency of heavy drinking was also assessed. Heavy drinking episodes were assessed with a question which asked, “In the last month, how many times have you had five or more drinks (for men) or 4 or more drinks (for women) in a row in a two-hour period?”. Responses ranged from “none” to “10 or more times”. The DDHQ is commonly utilized to assess quantity and frequency of drinking in alcohol use research (e.g., Wong et al., 2006).

Alcohol use protective behaviors:

Utilization of strategies to limit harm relating to alcohol use was assessed with the Alcohol Use Self-Protective Behaviors scale (Martens et al. 2005). The scale includes 15 items. Participants indicate the extent to which they have engaged in each behavior over the past month on a 5-point scale, ranging from “never” to “always”. Higher scores reflect greater use of safe drinking behavior. The measure includes several types of protective drinking behaviors, including: 1) seven items addressing stopping and limiting drinking (e.g., “drink water while drinking alcohol”), 2) five items addressing the manner of drinking/healthier drinking (e.g., “avoid drinking to keep up or out-drink others”); and 3) three items on serious harm reduction (e.g., “use a designated driver”). For the current study, items were summed to reflect overall levels of self-protective drinking behavior. The scale demonstrates good reliability and validity across over 30 studies (Pearson, 2013). Cronbach’s alpha was .86 in the current sample.

Prior experiences of sexual victimization and perpetration of sexual aggression.

The Revised Sexual Experiences Survey (Koss et al., 2007) was administered to assess for history of sexual victimization and history of sexual aggression since the age of 14 to the time of the study. The Sexual Experiences Survey is the most widely used and validated measure of sexual victimization and perpetration of sexual aggression among college students, and demonstrates good reliability and validity (Canan et al., 2020; Johnson et al., 2017). At baseline, the scale assessed experiences since the age of 14, and at follow-up the scale assessed experiences over the 1-month interim. The 35-item scale assesses for unwanted sexual contact, sexual coercion, attempted rape, and completed rape. Two versions of the scale were implemented, one for perpetration of sexual aggression and one for victimization. All participants completed both scales, regardless of gender. Baseline items asked about behavior since age 14 and follow-up survey items asked about behavior over the past month. Number of perpetration behaviors and victimization experiences were assessed at each time point by summing all items endorsed.

Rape supportive beliefs.

The short version of the Illinois Rape Myth Scale (Payne et al., 1999) was used to assess for rape supportive attitudes. The 17 items recommended by Payne et al. (1999) were answered on a 7-point scale, ranging from “not at all agree” to “very much agree”. Scores were summed so that high scores indicated higher levels of rape supportive beliefs. Example items include: “If a woman is raped while she is drunk, she is at least somewhat responsible for letting things get out of control”, and “Rape accusations are often used as a way of getting back at men”. A series of studies support the reliability and validity of the scale (Payne et al., 1999). Cronbach’s alpha was .60.

Knowledge of consent.

The Sexual Consent Scale-Revised utilized within the Administrator-Researcher Campus Climate Collaborative (ARC3) survey was implemented to assess communication of consent during sexual activity. This scale was adapted from the Sexual Consent Attitudes Scale (Humphreys & Brousseau, 2010; Humphreys & Herold, 2007) and its implementation in the full ARC3 survey is described by Swartout et al. (2019). This 7-item scale includes items answered on a 5-point scale from “strongly disagree” to “strongly agree.” An example of an item on the scale is: “Consent must be given at each step in a sexual encounter”. Some items are reverse coded, and responses were summed. A mean score was calculated, with higher scores indicating greater knowledge of effective consent. Cronbach’s alpha was .53.

Perceived peer support for sexual aggression.

Men in the sample completed three items to assess perceived peer approval of sexual aggression, previously utilized by Abbey and McAuslan (2004). Items were derived from Boeringer et al. (1991). An example item is: “Do your friends approve of getting a person drunk or high to have sex with them?” Items were answered on a 5-point scale, ranging from “very disapproving” to “very approving”. Responses were summed, with higher scores indicating stronger perceived peer approval for sexual aggression. Cronbach’s alpha was .67.

Perceived peer pressure for sex.

Men in the sample completed three items to assess perceived peer pressure to engage in sexual behavior. An example of items assessing peer pressure for sex is “How much pressure do you feel from your friends to have sex with many different women”. Items assessing peer pressure to engage in sexual behavior were answered on a 4-point scale, ranging from “Not at all” to “A lot”, with higher total scores indicating higher perceived peer pressure to engage in sexual behavior. Validity of this measure has been supported in prior research (Thompson et al., 2015). Cronbach’s alpha was .86.

Psychological barriers to resistance.

Women in the sample completed the Psychological Barriers to Resisting Questionnaire (Norris et al., 1996; Nurius, 2000). The 21-item scale assesses various psychological barriers to resisting a potential attacker, including concerns about self-consciousness (e.g., ”I don’t want him to think I am uptight or a ‘prude’”), concerns about presenting a relationship (e.g., “I like him and don’t want to ruin things for the future”), and concern for injury exacerbation (e.g., “I am afraid of being physically hurt if I don’t go along with it”). Participants are asked to indicate the extent each item makes it difficult to control the situation or protect yourself, responding on a five-point scale ranging from “not at all significant” to “very much significant”. Responses were summed to indicate greater psychological barriers to resistance. In prior research, psychological barriers to resistance were associated with how women responded to unwanted sexual advances (Norris et al., 1996). Cronbach’s alphas was .94 in the current sample.

Bystander behaviors.

To assess the use of bystander intervention skills, a set of items were adapted from the Bystander Efficacy Scale utilized within the Administrator-Researcher Campus Climate Collaborative (ARC3) (Swartout et al., 2019). Specifically, participants were asked, “Over the past month consider the extent to which you have engaged in each of these behaviors, in a situation where a friend or stranger may have been at risk for experiencing sexual misconduct”. Participants responded to 7 items on a 5-point scale ranging from “never” to “always”. An example item includes: “Walked a friend who has had too much to drink home from a party, bar or other social event”, and “Spoke up against sexist jokes”. Responses were summed to create a total score reflective of a greater extent of actual bystander behavior in the past month. Cronbach’s alpha for the current sample was .84.

Bystander efficacy.

Participants’ self-confidence in engaging in bystander intervention to address the risk for sexual violence and relationship violence was assessed with the Bystander Efficacy Scale (Banyard, 2008). The scale includes 18 items. Participants are asked: “Indicate in the column how confident you are that you could do each of the following items. Rate your degree of confidence by recoding a number from 0 to 100 using the scale below”. In response to each item, participants indicate a value ranging from 0–100%, along a scale of 0 = “I can’t do” and 100% = “very certain I could do”. Sample items include: “Get help and resources for a friend who tells me they have been raped”, and “Talk to a friend who I suspect is in an abusive relationship”. Items are summed so that higher scores reflect greater levels of confidence in bystander intervention. Cronbach’s alpha was .78 in the current sample.

Perceived peer norms.

The digital application contained seven embedded “quiz” items asking about peer norms. Participants were first asked to estimate what they believed to be an accurate response, were given feedback on the accuracy of their responses within the digital application and were then presented with feedback on their responses. Participants were again asked to provide an answer on the same items on the post-test and follow-up surveys. The items were developed by the research team. The feedback provided was based on both national data and data collected from one of the study sites. Questions included the following: 1) What percentage of people would blame the woman for being raped based on her revealing clothing?; 2) What percentage of college men believe that women exaggerate the effects of sexual assault?; 3) What percentage of your peers do you think would respect you for intervening to prevent sexual violence from occurring?; 4) Over the course of a year, what percentage of college students have had less than 2 sexual partners?; 5) What percentage of sexual assault accusations turn out to be false?; 6) If a friend is taking a drunk person back to their room to have sex, what percentage of your peers would speak up to stop it?;7) Amongst your peers, what percentage of casual hook-ups involve alcohol?

Enjoyment, perceived competence, and utility.

At post-test, three subscales of the Intrinsic Motivation Inventory (McAuley et al., 1987) were utilized to assess interest/enjoyment (7 items), perceived competency with the digital application (6 items), and the value/utility of the digital application (7 items). Examples of items include: “I enjoyed doing this app “I think I am pretty good at using this digital application”, and “I believe this activity could be of some value to me”. The wording of the items was adapted so that items described the “digital application” or “app” instead of “the activity” (which is described on the original version of the scale). Items responses were provided along a 7-point scale, ranging from “not at all true” to “very true”. Some items were recoded, and a mean score for each subscale was calculated, with higher values representing greater enjoyment, competency, or utility. Cronbach’s alpha for the subscales were .94, .73, and .93, respectively.

Overall engagement, satisfaction, and acceptability of digital applications.

At post-test, an Intervention Specific Acceptability Survey was developed for use in the current study to garner overall feedback relating to engagement with the digital application, and on specific parts of the intervention. A series of seven questions assessed overall engagement. Example questions include: “I would like to play this app again” and “This story made me think”. A series of three questions assessed satisfaction. Examples include: “I am satisfied with this app”, and “The characters in this app were believable”. A set of 4 questions assessed acceptability of the digital application. Examples include “This app offers a new way to prevent sexual violence among youth”, and “Digital apps can help to educate students about difficult topics”. For items assessing overall engagement, participants responded along a 7-point scale ranging from “Not at all true” to “Very true”. For items assessing satisfaction and acceptability, participants responded along a 5-point scale ranging from “Strongly Disagree” to “Strongly Agree”. Responses were summed, and a mean score was generated, with higher scores reflecting higher levels of engagement, satisfaction, and acceptability of digital applications. Cronbach’s alpha for the engagement, satisfaction, and acceptability scales were .95, .84, and .78 respectively.

Satisfaction with intervention components.

For several of the embedded activities – including “This or That”, “Sorting”, the Social Norms Feedback, and Change Plan – participants completed items assessing utility, satisfaction, and perceived learning. The three items (per app component) assessed: 1) satisfaction; 2) utility; and 3) perceived learning. Example items include: “I was satisfied with the change plan in this app”, “The change plan was a useful part of this app” and “The change plan helped me to learn”. When providing feedback on the change plan at the 1-month follow-up, participants also responded to the following prompt: “I tried to implement the change plan that I chose in this app”. Responses were provided along a 5-point scale, ranging from “Strongly Disagree” to “Strongly Agree”. Responses were summed, and a mean score for each app component was derived, with higher scores reflecting a more positive appraisal of the component. Cronbach’s alpha for feedback pertaining to “This or That”, “Sorting”, Social Norms Feedback, and the Change Plan were .92, .95, .80, and .90, respectively.

Application-specific data.

The intervention recorded user data within its interface. The digital application recorded the number of times accessed, as well as completion of each activity and branching narrative component. Responses to the Change Plan activity were also recorded to examine what specific goals participants set while interacting with the digital application.

Data Analysis Plan

SPSS 25 was used to analyze the data. Descriptive statistics were used to examine frequencies and measures of central tendencies among all study variables. Paired samples t-tests were used to compare pre-test and 1-month follow-up scores on study outcomes. Repeated measures analyses of variance (ANOVAs) examined changes between baseline, post-test, and follow-up on perceptions of peer norms. Given the formative nature of the research, findings were considered to be significant when p <.10.

Results

Study Aim 1: Satisfaction with the Digital Application

Responses to the Intrinsic Motivation Inventory revealed high value/utility (M = 5.88, SD = 1.19), competency (M = 6.32, SD = 0.76), and overall enjoyment (M = 5.26, SD = 1.32) (see Table 1). Participants also reported high levels of overall engagement (M = 6.13, SD = 1.12), satisfaction (M = 4.23, SD = 0.74) and acceptability with digital applications (M = 4.41, SD = 0.58) (see Table 2). Examination of each specific application component revealed that they were rated as useful and helped participants to learn something new (see Table 3). Specifically, participants indicated positive overall feedback regarding “This or That (M = 4.09, SD = 0.90), Sorting (M = 4.05, SD = 1.13), the Social Norms Feedback (M = 4.49, SD = 0.69) and the Change Plan (M = 4.13, SD = 0.67). Analyses of participants’ in-app responses to the Change Plan activity suggested that respondents reported an interest in using several of the potential strategies for change (see Table 4). Most participants indicated they planned to “have the courage to step in for what’s right,” “act responsibly,” “identify potentially dangerous situations,” and “spot danger signs and protect myself and others.”

Table 1.

Value/Utility, Task Competency and Enjoyment

	Men (n = 16)		Women (n = 23)		Transgender (n = 1)
	Mean	SD	Mean	SD	M
Interest/Enjoyment
I enjoyed doing this app	4.94	1.69	5.83	0.83	7.00
This app was fun to do.	4.69	1.99	5.52	1.04	7.00
I thought this app was a boring app.	2.75	2.21	1.64	1.00	1.00
This app did not hold my attention at all.	2.75	2.14	1.26	0.62	1.00
I would describe this app as very interesting.	4.56	1.86	5.22	1.09	7.00
I thought this app was quite enjoyable.	4.56	1.79	5.48	1.22	7.00
While I was doing this app, I was thinking about how much I enjoyed it.	3.88	1.75	4.13	1.71	4.00
Value/Usefulness
I believe this activity could be of some value to me.	5.38	1.86	5.78	1.20	7.00
I think doing this activity is useful for developing healthy relationships.	5.25	1.88	6.22	0.80	7.00
This is important to do because it can help me prevent others from being hurt.	5.69	1.85	6.27	0.75	7.00
I would be willing to do this again because it has some value to me.	5.00	2.13	6.13	0.97	7.00
Doing this activity could help me to avoid being involved in a sexual assault.	5.56	1.71	5.96	1.19	7.00
I believe doing this activity could be beneficial to me.	5.50	1.67	6.30	0.88	7.00
I think this is an important activity.	5.50	1.86	6.39	0.84	7.00
Task Competence
I think I am good at knowing how to use this app.	6.38	0.96	6.52	0.79	6.00
I did pretty well knowing how to use this app, compared to other students.	5.44	1.75	6.00	1.09	1.00
After working on this app for a while, I felt confident that I knew how to use it.	6.60	0.74	6.78	0.42	6.00
I am satisfied with my performance on this app.	6.56	0.63	6.61	0.66	7.00
I was pretty skilled in using this app.	6.19	1.52	6.52	0.66	6.00
This was an app that I couldn’t do very well.	1.75	1.77	1.91	1.95	1.00

Table 2.

Overall Engagement and Satisfaction with the Digital Application

	Men (n = 16)		Women (n = 23)		Transgender (n = 1)
	Mean	SD	Mean	SD	M
Engagement
This is an app that I would want my school to use.	5.69	1.85	6.17	0.89	7.00
I would like to play this app again.	4.63	2.19	5.74	1.29	7.00
This story made me think.	5.50	1.90	6.52	0.67	7.00
I would want other students to play this app.	5.75	1.81	6.65	0.65	7.00
This app could help educate students about sexual violence.	6.19	1.47	6.87	0.34	7.00
This app is good for starting conversations about how to prevent sexual violence.	5.94	1.57	6.78	0.42	7.00
I feel more confident in how to prevent sexual violence after playing this app.	5.56	1.79	6.17	1.03	7.00
The activities in this app made me think.	5.81	1.60	5.52	0.85	7.00
Satisfaction
I was satisfied with this app.	3.81	1.05	4.43	0.59	5.00
The characters in this story were believable.	3.63	0.89	4.65	0.57	4.00
This is a realistic story.	3.69	0.87	4.57	0.66	4.00
This app held my attention.	3.56	1.03	4.22	0.74	5.00
Perception of Digital Application
This app offers a new way to prevent sexual violence among youth	4.25	0.77	4.65	0.65	5.00
I would want to help improve this app so that it could be used in my school.	4.06	0.93	4.65	0.71	5.00
Digital apps can help to educate students about difficult topics.	4.56	0.51	4.87	0.46	5.00

Table 3.

Utility of Specific Components of the Digital Application

	Men (n = 16)		Women (n = 23)		Transgender (n = 1)
	Mean	SD	Mean	SD	Mean
This or That Activity
I was satisfied with the “This or That” activity.	3.56	1.15	4.17	0.98	5.00
“This or That” was a useful part of this app.	3.69	1.01	4.43	0.79	5.00
“This or That” helped me learn something new.	3.75	1.00	4.44	0.73	5.00
Sorting Activity
I was satisfied with the Sorting activity in this app.	3.31	1.30	4.57	0.66	5.00
The Sorting activity was a useful part of this app.	3.31	1.40	4.65	0.57	5.00
The sorting activity helped me to learn something new.	3.30	1.41	4.57	0.59	5.00
Social Norms Feedback
I was satisfied with the polls in this app.	4.00	1.10	4.65	0.57	5.00
The poll were a useful part of this app.	4.25	0.77	4.74	0.45	4.00
The polls helped me to learn something new.	4.25	1.13	4.74	0.75	5.00
Change Plan
I was satisfied with the change plan in this app	4.00	0.73	4.35	0.65	4.00
The change plan was a useful part of this app.	4.00	0.73	4.39	0.66	4.00
The change plan helped me to learn something new.	3.81	0.83	4.26	0.75	4.00
I tried to implement the change plan I chose in this app.	3.63	0.89	4.35	0.83	4.00

Table 4.

Participants’ Responses to the Change Plan

	% Indicating the Goal
As a part of our campus community, I want to change my attitudes. Choose your 3 priorities:
Have the courage to step in for what’s right.	57%
Be honest about how I feel.	43%
Know I can and should speak out.	40%
Own what I say and how I act.	40%
Don’t judge or fear being judged.	38%
Know it’s ok to change my mind.	36%
Be respectful of others.	26%
Don’t excuse harmful actions of others.	21%
As a member of our campus community, I want to change what I say and do. Choose your 3 priorities:
Act responsibly.	55%
Take action when I should.	50%
Get help when I need it.	40%
Don’t assume. Ask for consent.	33%
Don’t give in to peer pressure.	29%
Make my own decisions.	24%
Be selective with the words I choose.	17%
Speak my mind.	2%
To keep myself and others in the campus community safe, I want to stay focused on decreasing risks. Choose your 3 priorities:
Identify potentially dangerous situations.	76%
Spot danger signs and protect myself and others.	69%
Intervene in dangerous situations.	45%
Stay in control of myself.	38%
Create a plan before I enter potentially dangerous situations.	36%
Do not initiate sexual contact without clear, unambiguous consent from my partner.	24%

Study Aim 2: Preliminary Efficacy

A series of paired sample t-tests examined the extent to which participants reported changes from baseline to one-month follow-up on proximal (attitudinal) and distal (behavioral) outcomes relating to sexual assault. No participants perpetrated sexual aggression over the follow-up and three participants experienced sexual victimization over the follow-up. No significant differences were observed on risks for sexual aggression, risks for sexual victimization, or bystander behaviors (Table 5). However, results demonstrated a trend in reductions of the frequency of heavy drinking episodes over time (p <.10). Specifically, the frequency of heavy drinking showed significant reductions from pre-test (M = 1.27, SD = 1.64) post-test (M = 1.02, SD = 1.44) among intervention participants, t (40) = 1.71, p <.10.

Table 5.

Preliminary Intervention Outcomes

Outcomes	Pre-Test Mean (SD)	1 month Follow-Up Mean (SD)	t (df)
Risks for Sexual Aggression
Rape Myth Acceptance	1.20 (0.22)	1.19 (0.22)	0.15(39)
Sexual Consent Knowledge	3.80 (0.28)	3.81 (0.32)	0.54(39)
Perceived Peer Approval for Sexual Aggression	0.37 (0.62)	0.49 (0.68)	−.74 (16)
Peer Pressure for Sex	0.51 (0.70)	0.43 (0.50)	0.85(16)
Risk Factors for Victimization
Psychological Barriers to Resistance	31.54 (19.24)	29.29 (18.60)	0.79 (23)
Bystander Intervention Skills
Bystander Efficacy	13.56 (8.97)	13.67 (11.10)	−.10(40)
Bystander Behavior	2.11 (1.14)	2.12 (1.04)	−.07(40)
Alcohol Use
Alcohol Use: Quantity (Number of Drinks)	2.95 (2.66)	3.02 (2.69)	−.48 (40)
Alcohol Use: Frequency of Heavy Drinking	1.27 (1.64)	1.02 (1.44)	1.71 (40)*
Alcohol Use Self-Protective Behaviors	34.70 (11.21)	33.00 (13.31)	1.00 (28)
Sexual Violence
# perpetration behaviorsa	0.37 (1.09)	0.00 (0.00)	N/A
# victimization experiencesa	2.68 (3.34)	0.17 (0.62)	N/A

Note:

^aPerpetration experiences and victimization history were assessed at baseline from the age of 14 to the time of the study and were assessed over the interim period at the follow-up assessment. Given the varying time frames of assessments, comparisons of pre- and post-test changes in violence involvement were not conducted.

^*p <.10.

Study Aim 3: Change in Perceived Norms

A series of ANOVAs examined the extent to which the digital application facilitated change in perceived social norms, a putative mechanism of change within the intervention. A separate analysis was conducted to examine responses on each of the questions administered as a part of the normative feedback provided. There were significant differences from pre-test to post-test and follow-up on most items assessing perceptions of peer norms, with participants’ perceptions becoming more accurate over time (Table 6). Specifically, participants’ estimation of the percent of people who would blame the woman for being raped based on her revealing clothing varied over time F (2,74) = 24.70, p<.001. Participants’ estimation of the percent of college men who believe that women exaggerate the effects of sexual assault, as well as the percent of their peers they estimated would respect them for intervening to prevent sexual violence also changed over time, F (2,74) = 27.95, p<.001; F (2,74) = 13.73, p<.001. Three other outcomes also showed variation over time including participants’ estimate of the percentage of college students who have had less than two sexual partners, participants’ estimate of what percentage of sexual assault accusations turn out to be false, and participants’ estimate of what percentage of peers would speak up to stop a friend from taking a drunk person back to their room to have sex, F (2,70) = 18.36, p<.001; F (2,72) = 9.45, p<.001; F (2,72) = 23.71, p<.001.

Table 6.

Perceived Peer Norms: Responses at Baseline, Post-Intervention, and 1-Month Follow-up

Item	Actual %	Estimate at First Play	Post-test	1-month Follow-Up	F
What percentage of people would blame the woman for being raped based on her revealing clothing?	29.2%	46.06	19.37	17.08	24.70***
What percentage of college men believe that women exaggerate the effects of sexual assault?	8%	45.71	17.03	14.92	27.95***
What percentage of your peers do you think would respect you for intervening to prevent sexual violence from occurring?	95%	73.29	88.42	93.34	13.73***
Over the course of a year, what percentage of college students have had less than 2 sexual partners?	72.5%	41.42	67.72	65.25	18.36***
What percentage of sexual assault accusations turn out to be false?	5%	14.60	5.62	6.84	9.45***
If a friend is taking a drunk person back to their room to have sex, what percentage of your peers would speak up to stop it?	84.1%	49.58	78.89	74.49	23.71***
Amongst your peers, what percentage of casual hook-ups involve alcohol?	65.85%	65.62	57.54	62.06	1.70

Note:

^***p<.001.

Discussion

The current study advances the science of sexual assault prevention on college campuses by developing and conducting a preliminary evaluation of a digital application addressing sexual victimization risk, perpetration risk, and bystander intervention. The intervention was comprehensive in uniting theories of sexual assault prevention, including the Integrated Model, the AAA model, and the situational model for bystander intervention. Given the intersection of alcohol use and sexual assault (Abbey et al., 2022), intervention content also maintained a specific focus on the role of alcohol in sexual assault. The digital application was also novel in using a serious games approach to encourage interactive learning and opportunities to practice skills in an engaging and responsive digital environment. Given that most students actively use mobile technology, education programs presented in digital formats offer a widely accessible and confidential tool that can be broadly disseminated. Because most sexual assault prevention programs have undergone development and evaluation for 4-year college students, the current study is also notable in its inclusion of both 2-year and 4-year college students.

Several key findings emerged from the evaluation. First, the study sought to evaluate the utility, ease of use, and satisfaction with the program content. For men, women, and students identifying as transgender, the value and utility of the intervention were seen as high. Participants reported overall high levels of engagement and enjoyment. Specific activities were also rated positively, suggesting that the content was believed to be of value to the participants across both study sites. Participants also delineated specific activities in their change plan. Given that change plans are not commonly utilized in sexual assault prevention, these findings suggest that this strategy for change can be a useful way to support participants in implementing changes following sexual assault prevention programs.

Regarding Aim 2 of the study, participants reported a lower frequency of heavy drinking episodes over the 1-month follow-up compared to baseline. Numerous scholars highlight the importance of addressing alcohol use in the context of sexual assault prevention (Abbey et al., 2022; Leone et al., 2018). Several prevention programs have a specific focus on the role of alcohol use as a risk factor for sexual victimization (Gilmore et al., 2015) and sexual aggression (Orchowski Barnett et al., 2018). Given that the current study did not include a control group, it is possible that the reduction on alcohol use over the follow-up was the result of a third variable, such as natural reductions in alcohol use over time. Whereas the intervention did not include formal alcohol intervention techniques such as brief motivational interviewing, it did include discussion of the role of alcohol use as a risk factor for sexual aggression throughout the branching narrative and embedded activities. It is possible that effects may be stronger if the intervention was implemented in a sample of heavy drinking college students, or when utilized in tandem with a formal alcohol use intervention. Normative feedback on drinking could also be incorporated into the intervention to maximize its impact.

Third, given that activities in the intervention centered around correcting common misperceptions relating to sexual assault (i.e., social norms feedback), the evaluation also sought to determine if the intervention was successful in correcting misperceived attitudes and beliefs. The in-app data provided by participants in response to each of the social norm’s questions highlighted vast misperceptions of social norms in all but one of the questions. Specifically, although respondents had inaccurate perceptions of others’ level of victim blame, false accusations of sexual assault, number of sexual partners, and support for bystander intervention, they were accurate in estimating the number of causal hookups that involve alcohol. Specifically, participants estimated that 65% of hookups involve alcohol, which matched with the actual data garnered from prior research (LaBrie et al.., 2014). Yet, when asked to estimate the number of college students who had less than 2 sexual partners in the past year, participants estimated this to be 41%, which was far lower than the rate indicated in the National College Health Assessment (2017) of 72.5%. Similarly, while research by Lisak et al. (2010) estimates the rate of false accusations of sexual assault to be 5%, participants in the sample estimated that 15% of accusations of sexual assault are false. The Integrated Model (Orchowski & Berkowitz, 2022) posits that misperceptions of social norms contribute to sexual aggression by increasing the likelihood that individuals act on rape supportive beliefs.

Supporting the utility of providing normative feedback as a strategy for correcting misperceived attitudes regarding sexual assault, participants also corrected their perceptions when provided with feedback. For example, although research suggests that approximately 30% of individuals would blame a woman for being raped because of wearing revealing clothing (McGee et al, 2011), the current sample estimated that 46% of individuals would do so. After completing the intervention, participants aligned their response more closely with the actual data, indicating that 19% of individuals would do so at post-test. This positive change was maintained at the 1-month follow-up, with participants estimating that 17% of individuals would blame someone for sexual assault due to what they were wearing. Similar changes in misperceptions of norms were found for the other study constructs, suggesting that this application mechanic was effective in promoting learning, which was sustained over the follow-up.

Several limitations should be noted. The current study represents a small open pilot trial. Pilot trials generally lack power to detect significant intervention effects (Leon et al., 2010). Further, given the small sample size, some of the outcome measures in the study demonstrated relatively low levels of reliability. These measures are commonly utilized in the field and are expected to generate more stable internal consistency when utilized in larger sample sizes. Despite these limitations, it is nonetheless notable that the study was successful in demonstrating the capacity to recruit and retain participants at both campuses over a one-month follow-up, evidencing 100% retention over time. In support of the feasibility of the application, all participants completed the full intervention, played the intervention more than once, and completed all sections of the branching narrative and embedded activities.

It is also notable that the current sample also began the study with relatively low levels of risk factors and high levels of protective factors, suggesting that outcomes may have been influenced by floor and ceiling effects. For example, many of the means were in the upper range of the scale (e.g., consent mean was 3.8 on a 0–4 scale; rape myth acceptance was 1.2 on a 1 to 7 scale, with a range from .88 to 1.8). Given that the study was advertised via emails and fliers, further studies should ensure recruitment of a representative sample. It is possible that students enrolled in the research had a specific interest in violence prevention programming and were also exposed to prior prevention programming in their campus communities. Samples with a wider range of attitudes and knowledge can help to further substantiate the utility of the program.

Whereas the current study assessed history of sexual victimization and sexual aggression at baseline, as well as experiences of victimization and perpetration over the follow-up, the study did not include a control group. Thus, it is unclear how the intervention impacted rates of sexual assault over time. Follow-up studies implementing a control group are needed to examine this important study outcome. Future studies should also include a longer follow-up period, given that a one-month interim is not sufficient for examining the extent to which changes in attitudes, or behaviors are maintained over time. The current data also cannot be generalized beyond the inclusion criteria for the current sample of 2- and 4-year college students. Administering the program prior to the entry of college is also warranted to examine its feasibility and acceptability among younger samples.

It is important to note that the intervention was completed in 30 minutes. Whereas short interventions are needed to supplement other forms of sexual assault prevention on college campuses, researchers and advocates recognize that to be effective, sexual assault prevention must include a range of different synergistic interventions, implemented at multiple developmental periods (Baynard & Potter, 2018). It is important that no sexual assault prevention program be seen as a “silver-bullet” (Lonsway et al., 2009). In addition to implementing the current program in concert with other prevention activities, it may also be useful to extend the scope of the program with booster sessions and other supplemental activities which align with the theoretical foundation of the intervention. The intervention could also be extended through several technological advancements, including the integration of audio, avatar personalization, and elaboration of the character stories.

In sum, sexual assault prevention programs are needed for both 2-year and 4-year college campuses. Participants perceived high value to the activities, expressed an interest in using this tool at their schools, and responded favorably to the content, suggesting that this intervention approach has merit for implementation in a larger sample size. The feasibility of recruitment and retention procedures also lays the groundwork for further evaluation of this intervention in a larger randomized trial. A fully powered randomized trial is needed to determine the efficacy of the digital application and examine its impact on rates of sexual assault perpetration and victimization among participants. The present study documents the feasibility, acceptability, and utility of a novel digital application for sexual assault prevention on college campuses which addresses sexual victimization, sexual aggression, and bystander intervention, while also maintaining a focus on the role of alcohol as a risk factor. Whereas future research is needed to substantiate the efficacy of the intervention, the development and testing of digital interventions for sexual assault prevention is warranted. Digital applications offer a customizable approach for addressing multiple risk factors for sexual assault that can be easily disseminated to college campuses.