Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 Oct 28.
Published in final edited form as: Psychol Violence. 2018 Sep;8(5):537–545. doi: 10.1037/vio0000151

Measurement Tools to Assess Relationship Abuse and Sexual Assault Prevention Program Effectiveness Among Youth

Katie M Edwards 1, Victoria L Banyard 2, Stephanie N Sessarego 3, Linda R Stanley 4, Kimberly J Mitchell 5, Robert P Eckstein 6, Kara Anne E Rodenhizer 7, P Caroline Leyva 8
PMCID: PMC6816802  NIHMSID: NIHMS1010137  PMID: 31660253

Abstract

Objective:

This article describes the adaptation, development, and psychometric properties of survey instruments to assess outcomes of bystander-focused violence prevention efforts among high school students, including media literacy, rape myths, bystander readiness, bystander barriers and facilitators of bystander action, bystander intentions, perceptions of school personnel helping, perceptions of peer helping, and victim empathy.

Method:

The study was based on data collected from 3,172 high school students across 25 schools in northern New England.

Results:

Overall, the measures demonstrated acceptable fit indices in multilevel exploratory and confirmatory factor analyses. Whereas most measures and subscales had adequate reliability, several measures had less than ideal internal consistency, likely because of the limited number of items.

Conclusion:

Although additional measurement work is needed, these measures provide researchers and practitioners with foundational tools for basic research and program evaluation.

Keywords: bystander, evaluation, measurement, relationship abuse, sexual assault

Relationship abuse and sexual assault are pervasive problems among high school students in the United States (Centers for Disease Control and Prevention, 2014). Because there is an increasing focus on the critical importance of preventing relationship abuse and sexual assault among high school students, there is also a need for reliable and validated quantitative evaluation tools that are conceptually sound and relevant to prevention evaluation, yet parsimonious enough to be incorporated into school-based research. The purpose of the current study is to describe the adaptation, development, and psychometric properties of a series of survey instruments to assess primary and intermediate outcomes of a bystander-focused relationship abuse and sexual assault prevention program among high school students.

One promising area of prevention is mobilizing active bystanders to step in and challenge norms that condone relationship abuse and sexual assault before they happen, interrupt situations of escalating risk, and support survivors after an assault (Banyard, 2015; Edwards, Rodenhizer-Stämpfli, & Eckstein, 2015). Logic models for these programs draw from research on factors that increase the likelihood of third parties stepping in to help while overcoming barriers to taking action. Correlates related to intervening include lower rape myth acceptance (McMahon & Farmer, 2011), higher victim empathy (Ahrens & Campbell, 2000), intentions to help (Banyard, 2008; Banyard & Moynihan, 2011), and positive peer norms toward helping (Brown, Banyard, & Moynihan, 2014).

Furthermore, research on bullying describes how perceptions of school personnel’s attitudes and behaviors, as well as perceptions of school climate and norms, are relevant to either spurring or inhibiting students’ bystander actions (Espelage, 2014; Rinehart & Espelage, 2016; Yoon & Barton, 2008; Yoon & Bauman, 2014). Moreover, given the media is the largest purveyor of social norms surrounding dating and relationships (Ward, 2003), violence prevention programs have used media depictions of violence against women to facilitate discussions about healthy relationships, signs of abuse, and sexuality (Ashcraft, 2006; Barker, Ricardo, & Nascimento, 2007; Bonomi, Nichols, Carotta, Kiuchi, & Perry, 2016). To date, however, there are no existing measures of media literacy or perceptions of school personnel’s attitudes and behaviors specific to relationship abuse and sexual assault.

Moreover, measurement development efforts to date have been specific to relationship abuse and sexual assault has been on college campuses (Banyard, Moynihan, Cares, & Warner, 2014) or focused on understanding bullying behavior among middle and high school students. In the only identified study to assess high school students’ bystander behavior in situations of sexual assault and relationship abuse, Edwards and colleagues (2015) documented via focus groups, facilitators (e.g., victim at risk for serious harm) and barriers (e.g., getting involved with “drama”) to bystander action, “hot spots” (e.g., school buses, house parties) for relationship abuse and sexual assault where opportunities for helping were most likely, and specific ways in which youth helped (e.g., causing a distraction, directly confronting perpetrators). This work highlighted the need to develop measures that accurately reflect the experiences of high school students in the context of relationship abuse and sexual assault. In addition, the field of implementation science highlights the challenges of moving prevention efforts from research to practice. Measures developed in a research context such as a laboratory setting may often be infeasible when used in community settings, such as schools, where researchers have minimal access to students.

The purpose of the current paper was to describe the adaptation, development, and psychometric properties of survey instruments to assess outcomes of bystander-focused relationship abuse and sexual assault prevention programs among high school students. The measures were designed to capture correlates of students’ prevention behaviors that are frequently part of school curricula and to be brief to minimize impact on classroom instruction and teacher burden. Specific aims are to describe the survey instruments that we adapted and developed through an examination of the factor structure of the survey instruments (Aim 1) and examine the reliability of survey instruments (Aim 2).

Method

Procedure

Data are from a larger cluster randomized control trial to evaluate a bystander-focused violence prevention curriculum (Edwards, Banyard, Sessarego, Mitchell, & Chang, 2017). Twenty-five high schools in northern New England participated in the 1.5-year study. We worked with school administrators to select classes: (a) where the surveys and programming would fit within the learning goals of the courses, (b) totaled approximately 100–120 students who represented all grade levels within each school, and (c) were representative of the entire student body (e.g., health, physical education). There were an average of 9.2 classrooms per school, ranging from 5 to 20 per classes. Human subject participation was reviewed and approved by the University of New Hampshire Institutional Review Board (IRB) and conformed to the rules mandated for research projects funded by the U.S. Department of Health.

Following (IRB) approval, guardians of students under the age of 18 from 241 selected classrooms were sent an opt-out form with information on the project and how to withdraw their child if they did not want them to participate. Students under the age of 18 were eligible to participate if a guardian did not withdraw them via opt-out consent procedures. Students who were 18 or 19 years old (7.9%) provided their own consent to participate, and students under 18 who were eligible to participate read an assent form to decide if they would like to complete the surveys. The vast majority (89.7%) of invited students participated in the current study.

During data collection sessions, the project investigators or highly trained research assistants verbally reminded participants of the voluntary nature of the study, confidentiality practices, and other important information to ensure participant safety and data integrity (e.g., taking the survey on your own, remaining quiet during the survey, having at least one empty chair between students). The data were collected in-person, using a paper-and-pencil survey, during the designated class time for each school. Participants received a debriefing and referral sheet and a fruit snack at the end of the survey session. Data were hand-entered by trained research assistants into SPSS, and all surveys were double-checked for accuracy by a second trained research assistant. Although our initial sample at Time 1 was 3,837, we removed 665 cases (16.34%) because of an inability to match surveys across time points (n = 625, 15.36%; which would mean that a single participant would be in the data set as different participant across time points), mischievous (n = 31, .76%; e.g., wrote their age was 3) and/or extreme (n = 6, .15%; i.e., indicated the highest possible response on two more measures) responders, and/or transferring from a treatment to control school or vice versa (n = 3, .07%; and thus concerns about contamination). Thus, we included 3,172 students in this paper (see Table 1).

Table 1.

Demographics for the Exploratory Sample, Confirmatory Sample, and Total Sample

Characteristic Exploratory sample
(N = 1583)		 Confirmatory sample
(N = 1582)		 Total sample
(N = 3172)
Average age
(SD; range)		 15.71 (1.15; 13–19) 15.70 (1.20; 13–19) 15.71 (1.17; 13–19)
Gender
 Female 51.1% 51.1% 51.4%
 Male 48.5% 48.1% 48.6%
 Other .4% .8% .0%
Race
 White/Caucasian 89.4% 89.9% 89.5%
 Black/African American 2.1% 2.5% 2.3%
 American Indian or
Alaska Native		 1.3% .9% 1.1%
 Asian 4.6% 2.8% 4.2%
 Native Hawaiian or
Pacific Islander		 .1% .3% .2%
 More than one race 2.5% 2.7% 2.6%
Sexual orientation
 Heterosexual 87.8% 86.5% 87.1%
 Not heterosexual 12.2% 13.5% 12.9%
Grade
 9th 30.1% 32.2% 31.2%
 10th 34.2% 31.1% 32.6%
 11th 21.1% 19.5% 20.2%
 12th 14.6% 17.2% 15.9%
Reduced lunch
 Yes 21.2% 19.6% 20.4%
 No 78.8% 80.4% 79.6%
Open in a new tab

Note. There are 7 fewer observations between the exploratory and confirmatory samples and the total sample as a result of missing values.

Participants

We use baseline (Time 1) data. The mean age of participants was 15.71 (Range 13–19, SD = 1.17). About one third (31.2%) of the sample was in 9th grade, 32.6% was in 10th grade, 20.2% was in 11th grade, and 15.9% was in 12th grade. The majority of participants identified as female (51.4%), White (89.5%), and heterosexual (87.1%). Approximately one in five (20.4%) students reported receiving free or reduced lunch, a proxy for assessing poverty. Analyses indicate that students in our sample did not differ on indicators of gender, race, and free/reduced lunch from the overall student body (collected from the respective Departments of Education; Maine Department of Education, 2015; Massachusetts Department of Elementary and Secondary Education, 2017; New Hampshire Department of Education, 2010).

Measures

Instrument adaptation and development.

In the sections that follow, we discuss each of the measures examined in the current study. Whereas some measures were adapted, others had to be created for the purposes of this project. Also, when creating measures, ideally researchers can include a comprehensive number of questions in the item pool. However, the reality of time constraints imposed by schools as well as concerns regarding survey fatigue (Porter, Whitcomb, & Weitzer, 2004), led us to develop brief, as opposed to comprehensive, instruments. For measures that we created (e.g., media literacy), we drew from existing scales in other fields of health behavior when possible (e.g., substance use) and tailored them to be specific to relationship abuse and sexual assault, as well as grounded in previous qualitative research with high school students and school personnel (Edwards, Rodenhizer-Stampfli, & Eckstein, 2016; Edwards et al., 2015). Given the need for measures to be as short as possible, we removed items that were redundant or would have low base rates of endorsement based on previous research on this topic.

Media literacy.

The six-item Relationship Media Literacy Scale (RMLS) was created for this study and modeled after prior research of media literacy for other health behaviors (e.g., smoking; Bickham & Slaby, 2012; Primack et al., 2006). Items were created to specifically tap into perceptions of how the media portrays relationships. Media was defined for participants to be inclusive of varied forms of entertainment (e.g., TV, movies, music, the Internet). The RMLS consists of six items (e.g., “The media is a good way for me to learn about how teens should be in relationships.”). Response options range from 1 (disagree strongly) to 4 (agree strongly). The items we selected were guided by cognitive theories of media literacy that focus on the degree to which people become desensitized to or no longer bothered by violent portrayals in media and faulty information, or the degree to which individuals learn from media, accepting without question what they see in the media as fact (Potter, 2004).

Rape myths.

We used a previously shortened version (Coker, Cook-Craig, Williams, Fisher, Clear, Garcia, et al., 2011; Cook-Craig, 2012) of the Illinois Rape Myth Acceptance Scale (IRMA; Payne, Lonsway, & Fitzgerald, 1999) to assess students’ agreement with rape myths. The IRMA-Short Form (IRMA-SF) consists of seven items (e.g., “When girls are sexually assaulted, it is often because the way they said ‘no’ was unclear.”) used in previous research with high school students (Cook-Craig, 2012). Response options range from 1 (disagree strongly) to 4 (agree strongly). Although we did not modify or adapt the IRMAS-SF, given the short form has only been used in a few previous studies, we examined the factor structure.

Bystander intentions.

The Bystander Intent to Help Questionnaire (BIHQ; Cook-Craig et al., 2014) was used to capture students’ intentions to intervene in situations of relationship abuse and sexual assault. The BIHQ has been used in previous research with high school students (Cook-Craig, 2012). The BIHQ consists of nine items (e.g., “How likely would you be to talk to a friend who was being physically hurt by a boyfriend/girlfriend?”). Response options range from 1 (very unlikely) to 4 (very likely). Although we did not modify or adapt the BIHQ, given the measure has been used in only one previous study, we examined the factor structure.

Bystander readiness-denial.

We used the Denial subscale of the Readiness to Help Scale (D-RHS) to assess the extent to which students rejected the role that they could play in preventing relationship abuse and sexual assault. The D-RHS consists of four items (e.g., “There is not much need for me to think about relationship abuse and/or sexual assault among high school students.”) that have been used in previous research with college students (Banyard et al., 2014); this is the first study to use this measure with high school students. Response options range from 1 (disagree strongly) to 4 (agree strongly). We modified the items by adding the phrase “among high school students” since the measure was originally created for college students.

Barriers and facilitators of bystander actions.

The Pros and Cons of Bystander Action Scale (PCBAS) was created for the purposes of this project to assess students’ perceptions of pros and cons of bystander action in situations of relationship abuse and sexual assault. To create this measure, we used five items from the Decisional Balance Scale (DBS; Banyard, 2013) that were consistent with findings from focus groups with high school students in a previously conducted qualitative study (Edwards et al., 2015). We also created items for all of the barriers to bystander action that youth identified in the Edwards et al. (2015) study. The PCBAS includes 10 items (e.g., “I might get made fun of or picked on if I help.”). Response options range from 1 (disagree strongly) to 4 (agree strongly).

Perceptions of peer helping.

Grounded in perceptions of peer norms (Brown et al., 2014), we created a 3-item scale, the Perceptions of Peer Helping Scale (PPHS), to assess students’ perceptions of their peers’ attitudes toward helping in situations of relationship abuse and sexual assault. Items were based on qualitative focus groups related to high school students’ bystander behavior and attitudes (Edwards et al., 2015). The response options for items (e.g., “It is important to students at this school to try to stop relationship abuse and sexual assault from happening.”) range from 1 (disagree strongly) to 4 (agree strongly).

Perceptions of school personnel helping.

Using items based on previous research with high school students and high school personnel (Edwards et al., 2015, 2016), we created a measure to assess students’ perceptions of how likely school staff are to intervene in situations of teen relationship abuse and sexual assault. The Perceptions of School Personnel Helping Scale (PSPHS) includes nine items (e.g., “About how many staff at your school would verbally tell a dating couple who is in a physical fight to stop fighting?”). Students were told that school staff included “anybody who works at your high school including teachers, coaches, administrators, counselors and social workers, custodial and cafeteria personnel, bus drivers, etc.” Response options range from 1 (no school staff) to 4 (all school staff).

Victim empathy.

A measure of victim empathy was created for the current study, based on existing measures (Ahrens & Campbell, 2000; Smith & Frieze, 2003). The Victim Empathy Scale (VES) consists of three items (e.g., “I feel that I am able to understand what the victim of relationship abuse and/or sexual assault goes through.”). Response options range from 1 (disagree strongly) to 4 (agree strongly). We selected items that, based on previous research and pilot testing, had the most relevance to youth.

Analysis

The current study uses factor analytic techniques for complex samples to investigate the factor structure of measures of students’ attitudes and behaviors specific to relationship abuse, sexual assault, and bystander action. Because students are nested within schools, the data needed to be factor analyzed in a way that accounts for the nonindependence of observations in the items measured. In this study, we focus on the student-level factor structure only, estimating the usual parameters of factor analysis but adjusting the standard errors and the chi-square tests of model fit to account for the nonindependence of observations.

Consistent with other research employing multilevel factor analyses (Huang et al., 2015), we randomly divided the sample into two groups: the exploratory sample (ES; n = 1583) and the confirmatory sample (conditional stimulus (CS); n = 1582). Exploratory factor analysis (EFA) was used to estimate the student-level factor structure of each measurement scale with four items or more using the ES. A succession of one to up to four factors were extracted using geomin rotation, with a maximum of two extracted factors for most scales. Geomin rotation, the default for EFA in MPlus, was used as it is an oblique method and thus allows for correlated factors. Factor loadings, fit indices, and conceptual understandings of the factors were used to determine the optimal factor structure. For measures with three items, we estimated a one-factor EFA. Following recommendations in the literature (Huang & Cornell, 2016), we report a variety of fit indices for each model with continuous outcome variables, including the Root Mean Square Error of Approximation (RMSEA; good fit < .06), Standardized Root Mean Square Residual (SRMR; good fit < .08), the Comparative Fit Index (CFI; good fit ≥ .95), and Tucker-Lewis Index (TLI; good fit ≥ .95). The CFI and TLI indicate how much better a model fits the data compared to a baseline model where all variables are uncorrelated. We also report the Bayesian Information Criterion (BIC), which is valuable for comparing multiple models; a smaller BIC indicates a better fit. Also, in keeping with the general recommendations for EFA, other criteria used in arriving at a final factor structure included removing items that load high on more than one factor and removing items with loadings <.30.

Once the factor structure was determined with EFA, confirmatory factor analysis (CFA) was used to test that factor structure using the CS data, with the fit indices described above estimated. For these models, the variances of factors were set to one and the loadings allowed to vary, and all factors were correlated. Finally, for the three-item scales, the model is saturated, and thus, we report only the loadings for a one-factor model.

For all analyses, the ‘type = complex’ command in Mplus 6.1 (Muthén & Muthén, 1998–2015) was used to account for the nested structure of the data. The default method of full information maximum likelihood was used to derive the factors, thus accounting for missingness. The reliability of the survey instruments was examined using IBM SPSS Statistics, Version 24. Cronbach’s alphas are presented for each factor, where alphas were calculated using the entire dataset. Reliability scores of .70 or higher were considered acceptable in accordance with general recommendations (Nunnally & Bernstein, 1994). Cronbach’s alphas implicitly assume a single-level factor structure and may be biased either positively or negatively depending on the intraclass correlation (ICC) and the reliability at each level (Geldhof, Preacher, & Zyphur, 2014). Given the relatively low ICC for the items used in the CFA estimations, Cronbach’s alpha is likely a reasonable estimate.

The fit indices for all models with four or more items are presented in Table 2. For each scale, the indices for the EFA models are presented; these are followed by the CFA results. Scales with three items are not included in this table as no fit indices are estimated due to just-identification of the model. Table 3 presents the standardized factor loadings from the “best” model for both the EFA and CFA analyses for each measurement scale. Regarding Aim 2, reliabilities are presented in the text and in Table 3. Below we summarize the findings regarding Aims 1 (examination of factor structure of survey instruments and 2 (examination of the reliability of survey instruments) separate for each of the measured we examined.

Table 2.

Fit Indices for Exploratory and Confirmatory Factor Models of Measures of Bystander Behavior

Measure χ2 df RMSEA SRMR CFI TLI BIC
Media literacy - 6 items
 EFA - 1 factor 240.5 9 .128 .076 .72 .54 20093
 EFA - 2 factors 22.6 4 .057 .018 .98 .91 19897
 CFA - 2 factors 37.7 8 .048 .032 .97 .94
Rape myths - 7 items
 EFA - 1 factor 535.3 14 .154 .082 .77 .66 19071
 EFA - 2 factors 57.6 8 .063 .021 .98 .94 18470
 CFA - 2 factors 46.1 8 .055 .029 .98 .96
Bystander intentions - 9 items
 EFA - 1 factor 828.5 27 .139 .070 .81 .75 29228
 EFA - 2 factors 243.7 19 .088 .030 .95 .90 28509
 EFA - 3 factors No convergence
 CFA - 2 factors 305.1 26 .083 .042 .93 .90
Bystander readiness-Denial - 4 items
 EFA - 1 factor 10.8 2 .052 .017 .99 .97 11846
 CFA - 1 factor 8.7 2 .047 .018 .99 .97
Barriers & facilitators of bystander action - 10 items
 EFA - 1 factor 685.6 35 .111 .100 .64 .53 28787
 EFA - 2 factors 155.2 26 .058 .033 .93 .88 27937
 EFA - 3 factors 104.8 18 .057 .023 .95 .88 28128
 CFA - 2 factors 97.6 19 .052 .040 .96 .93
Perceptions of school personnel helping
 EFA - 1 factor 1041.0 27 .160 .071 .78 .71 28640
 EFA - 2 factors 445.9 19 .124 .036 .91 .83 27914
 EFA - 3 factors 56.6 12 .050 .015 .99 .97 27736
 CFA - 3 factors 131.4 17 .068 .030 .97 .96
Open in a new tab

Note. Measures for Victim Empathy and Perceptions of Peer Helping were not included since fit indices could not be determined because of having 3 items. RMSEA = Root Mean Square Error of Approximation; SRMR = Standardized Root Mean Square Residual; CFI = Comparative Fit Index; TLI = Tucker-Lewis Index; BIC = Bayesian Information Criterion; EFA = Exploratory factor analysis; CFA = confirmatory factor analysis.

Table 3.

Factor Loadings for Final Exploratory and Confirmatory Factor Models With Geomin Rotation

EFA factor loadings CFA factor loadings
Measure F1 F2 F3 F1 F2 F3
Media literacy
 1. The media normalizes SA −.09 .40 .38
 2. The media is a good way to learn about SA .73 −.00 .79
 3. The media shows the consequences of SA .35 −.06 .41
 4. I’m bothered by the media’s portrayal of girls and guys −.09 .59 .67
 5. I am bothered by the media’s portrayal of relationship abuse & SA .01 .86 .81
 6. I get information about relationships from the media .40 .13 .37
Cronbach’s alpha .49 .64
Rape myths
 1. Girls should have sex with the guy they are dating when he wants .78 −.00 .76
 2. If a guy spends money on a date, the girl should have sex with him .88 −.00 .88
 3. Guys should respond to challenges by dates/girlfriend to authority by
insulting them or putting them down		 .57 .12 .57
 4. If a girl is sexually assault while drunk, she is to blame .31 .36
 5. SA charges are often used as a way of getting back at guys .02 .71 .71
 6. Many girls lead a guy on, and then claim it was SA −.08 .82 .77
 7. When girls are sexually assault, it is often because the “no” was unclear .13 .50 .59
 Cronbach’s alpha .78 .73
Bystander intentions
 1. Tell someone to stop talking down .66 .01 .66
 2. Speak up when you heard someone victim blaming .77 −.02 .74
 3. Talk to a friend who was physically hurt by a partner .62 .15 .68
 4. Asked someone who looked upset if they needed help .40 .16 .50
 5. Speak up to someone bragging about forcing someone to have sex .79 −.05 .71
 6. Get help for a friend who was forced to have sex or physically hurt .65 .12 .73
 7. Talk with friends about things to do to prevent SA & abuse .16 .68 .81
 8. Use social media to show you don’t agree with SA & abuse .25 .41 .61
 9. Talk with friends about being safe in dating relationships −.01 .92 .82
Cronbach’s alpha .83 .78
Bystander readiness-denial
 1. I don’t think abuse & SA is a problem among high school students .59 .60
 2. I don’t think there is much I can do about abuse & SA .56 .61
 3. There is not much need for me to think about abuse & SA .80 .81
 4. Doing something about abuse & SA is the job of rape crisis center or
domestic violence shelter		 .45 .41
 Cronbach’s alpha .69
Barriers & facilitators of bystander action
 1. If I try to do something, I can keep someone from getting hurt .60 − .03 .32
 2. It is important for student to be part of keeping everyone safe .64 .02 .33
 3. Students will think I’m cool if I help .24 −.08
 4. Helping could make people mad at me .00 .69 .44
 5. I could get physically hurt by helping .07 .62 .42
 6. I might get in trouble if I help −.10 .62 .44
 7. People will think I’m trying to get involved in drama if I help .04 .57 .42
 8. Even if I don’t know the person, I can still help .58 .01 .41
 9. It might not be serious enough for me to help −.30 .25
 10. I might get made fun of for helping −.04 .63 .50
Cronbach’s alpha .63 .76
Perceptions of peer helping
 1. Students at my school know a lot about SA & relationship abuse .39 .45
 2. It’s important to students to stop relationship abuse & SA .32 .38
 3. Students at my school think it’s important to help .88 .78
Cronbach’s alpha .51
Perceptions of school personnel helping
 1. Staff would tell a group of boys calling a girl a “slut” to stop .43 .19 .12 .63
 2. Staff would physically intervene in a physical fight between couples .51 −.00 .22 .67
 3. Staff would verbally tell a couple in a physical fight to stop .85 −.01 .02 .77
 4. Staff would verbally tell a couple in a verbal fight to stop .67 .19 −.02 .75
 5. Staff would talk to teens about getting help .02 .84 .01 .92
 6. Staff would talk to teens about health relationships −.10 .91 .00 .71
 7. Staff would gossip to other teachers about what teens tell them .02 .18 −.20
 8. Staff would comfort a teen who is a victim of abuse .02 .21 .66 .85
 9. Staff would get help for a teen who is a victim of abuse −.02 .00 .93 .85
 Cronbach’s alpha .81 .85 .84
Victim empathy
 1. Could imagine being in the place of the victim of SA or abuse .72 .71
 2. Can empathize with the emotions of the victim of SA or abuse .73 .76
 3. I feel I am able to understand what a victim goes through .78 .77
 Cronbach’s alpha .79
Open in a new tab

Note. EFA Factor loadings ≥.30 are in boldface. EFA = Exploratory factor analysis; CFA = confirmatory factor analysis.

Results

Media Literacy

Aim 1: A comparison of fit indices for a one-factor model vs. a two-factor model from the EFA indicated that the two-factor model was a significantly better fit for the RMLS. The RMSEA (.057) and SRMR (.018) were both below the respective cutoffs that indicate a good fit. In addition, the CFI was .98 while the TLI was .91, indicating good to reasonable fit. As shown in Table 3, all loadings for the 2-factor model were greater than .30 on their respective factor while the cross-loadings were less than .30 (in absolute value). The next step was to fit the two-factor model with the CS dataset to test the structure found with the EFA. All fit indices showed good fit (RMSEA = .048; SRMR = .032; CFI = .97), except for the TLI, which at .94 is just below the cutoff for good fit. Interpretation of the items loading on each factor led to naming factor 1 (items 2, 3 and 6) “Obtaining Information from the Media” and factor 2 (items 1, 4, and 5) as “Bothered by the Media.” Aim 2: Whereas factor 1 had poor reliability (α = .49), factor 2 demonstrated reliability (α = .64) slightly below what is considered acceptable.

Rape Myths

Aim 1: A comparison of fit indices for a one-factor model versus a two-factor model from the EFA indicated that the two-factor model was a significantly better fit for the IRMA-SF. The RMSEA (.063) was just above the accepted cutoff and the SRMR (.021) was well below the accepted cutoff for good fit. Similarly, CFI was .98 and the TLI was .94, indicating reasonable to good fit. All items had factor loadings greater than .30 on their respective factor, but one item “If a girl is sexually assaulted while drunk, she is to blame for the assault,” cross-loaded on both factors at just above .30 and was removed from further analyses. As shown in Table 2, the CFA for the two-factor model with six total items showed good fit (RMSEA = .055; SRMR = .029; CFI = .98; TLI = .96). The standardized factor loadings, found in Table 3, are significantly above .30. Additionally, the factors are conceptually sound. Aim 2: Cronbach’s alpha are acceptable for the factors. Factor 1 with items 1–3 measures “Traditional Gender Expectations” (α = .78) while factor 2 with items 5–7 measures “Rape Denial” (α = .73).

Bystander Intentions

Aim 1: The two-factor model for the BIHQ showed the best fit (with the 3-factor model not reaching convergence). The fit indices are somewhat mixed although they all show at least reasonable fit of the data (RMSEA = .088; SRMR = .030; CFI = .95; TLI = .90). As shown in Table 3, all factor loadings were greater than .30 on their respective factor, and there were no large cross-loadings. The two-factor model was then fit using the CS dataset. As with the EFA, all fit indices showed at least reasonable fit (RMSEA = .083; SRMR = .042; CFI = .93; TLI = .90). The standardized loadings were all well above the .30 level, and the factors were conceptually sound. Aim 2: Both factors demonstrated good reliability as assessed by Cronbach’s alphas. Factor 1 (items 1–6) measures “Reactive Bystander Intentions,” and produced α = .83 and factor 2 (items 7–9) measures “Proactive Bystander Intentions” (α = .78).

Bystander Readiness-Denial

Aim 1: With only four items, fit indices for only a one-factor model for the D-RHS were obtained. The fit indices, shown in Table 1, all indicated good model fit (RMSEA = .052; SRMR = .017; CFI = .99; TLI = .97). All items had factor loadings greater than .30. The one-factor CFA using the CS dataset again showed good model fit (RMSEA = .047; SRMR = .018; CFI = .99; TLI = .97). Aim 2: The Cronbach’s alpha (α = .69) indicated that the reliability for the measure was close to acceptable.

Barriers and Facilitators of Bystander Action

Aim 1: A comparison of fit indices for a one-factor, two-factor, and a three-factor model indicated best fit for the two-factor model for the PCBAS. With the exception of the TLI, fit indices indicated reasonable to good model fit (RMSEA = .058; SRMR = .033; CFI = .93; TLI = .88). As shown in Table 3, item 3, “Students will think I’m cool if I help,” and item 9, “It might not be serious enough for me to help,” did not load well onto either factor. All other items had factor loadings above .30, and with the exception of item 1, “If I try to do something, I can keep someone from getting hurt,” cross-loadings were below .30. As shown in Table 2, the CFA using the two-factor model, with items 3 and 9 excluded, had relatively good model fit (RMSEA = .052; SRMR = .040; CFI = .96; TLI = .93). Item 1 was placed in factor 1 because it fit the conceptual framework of having positive attitudes toward helping. Aim 2: Reliability analyses were also adequate. Factor 1 (items 1, 2, and 8) measures “Positive Attitudes Towards Helping,” (α = .63) while factor 2 (items 4–7 and item 10) measures “Barriers to Helping” (α = .76).

Perceptions of Peer Helping

Aim 1: Because of the measure consisting of three items, only a one-factor model for the PPHS was fit using EFA. All factor loadings were greater than .30, though two of the three items were below .40. The one-factor model was then fit using the CS dataset. Factor loadings on items 1 and 2 were again relatively low, .45 and .38, respectively. Aim 2: The Cronbach’s alpha (α = .51) also indicated relatively poor reliability for this measure. The results suggest that either these items are measuring different constructs or that there may be significant measurement issues with the items (e.g., wording of the items).

Perceptions of School Personnel Helping

Aim 1: A comparison of fit indices for the one-factor, two-factor, and three-factor models indicated best fit for a three-factor model for the PSPHS. All fit indices indicated good model fit (RMSEA = .050; SRMR = .015; CFI = .99; TLI = .97). Item 7, “Staff would gossip to other teachers about what teens tell them,” had low factor loadings on all factors, but all other items had factor loadings greater than .30 on their respective factors and no cross-loadings. The three-factor model, with item 7 excluded, was then fit using the CS dataset. Overall, the CFA showed good model fit (RMSEA = .068; SRMR = .030; CFI = .97; TLI = .96). In addition, factor loadings are relatively high and the factors are conceptually sound. Aim 2: Cronbach’s alphas are acceptable. Factor 1 with items 1–3 measures “Helping During” (α = .81); factor 2 with items 5 and 6 measures “Helping Before” (α = .85); and factor 3 with items 8 and 9 measures “Helping After” (α = .84). However, Helping Before and Helping After have only two items each, which is typically considered less than optimal for measuring an underlying construct (Eisinga, Grotenhuis, & Pelzer, 2013).

Victim Empathy

Aim 1: Because of the measure consisting of three items, only a one-factor model for the VES was fit using EFA to report factor loadings. All factor loadings, shown in Table 3, were greater than .30. The model was then fit using the CS dataset. Again, factor loadings were greater than .30. Aim 2: Cronbach’s alpha also indicated good reliability (α = .79).

Discussion

The current paper describes the adaptation, development, and psychometric properties of survey instruments that assess primary and intermediate outcomes of bystander prevention trainings for high school students with utility in practice settings such as schools. Overall, the measures that were created or adapted for the current research demonstrated acceptable fit indices. Furthermore, whereas most measures and subscales had adequate reliability, several measures had less than ideal internal consistency, which is likely attributable to the fact that these scales comprise only a few items. Ideally, researchers include a comprehensive number of questions in the item pool. However, the reality of time constraints imposed by schools, as well as concerns regarding survey fatigue (Porter et al., 2004), led us to develop brief instruments which could be implemented in a typical classroom time period as opposed to more comprehensive measures which would not be feasible in a school-based setting. The measures described fill a critical gap in research on bystander intervention to date. Specifically, the measures are brief assessments of bystander action, as well as key correlates, that training programs often leverage to promote greater bystander action to reduce relationship abuse and sexual assault.

Some of the longer instruments were multifactorial. For example, it is noteworthy that bystander intentions for both oneself and for school personnel emerged into actions taken before, during, and/or after instances of relationship abuse and/or sexual assault, which is consistent with models described among college students (McMahon & Banyard, 2012). Bystanders are often present when risk for assault is escalating and frequently have the chance to support survivors who disclose to them after an incident of relationship abuse (Edwards et al., 2015). The current study extends our understanding of this model by creating a measure that is developmentally appropriate for a younger audience.

The current study also went beyond the nearly exclusive focus in current research on bystander correlates at the individual level (e.g., personal endorsement of rape myths) to understanding an individuals’ perceptions of peer, teacher, and community norms. Thus, the current study is consistent with recent calls for a more expanded view of bystander action (Banyard, 2015; McMahon, Postmus, Warrener, & Koenick, 2014). For example, although measures of media literacy for other health-related behaviors exist (e.g., smoking; Bickham & Slaby, 2012), the current study is the first to create and evaluate a measure of media literacy that is specific to how the media portrays relationships. Nevertheless, the media literacy and perceptions of peer norms measures had less than ideal internal consistency and the model fits were, at best, adequate. The results suggest that either these items are measuring different constructs or that there may be notable measurement issues with the items (e.g., item wording). Clearly, an important area for future research is to improve the measurement of these constructs.

The current study takes next steps in exploring measures of bystander attitudes and behaviors that can be used both to understand under what conditions high school students will step in to help others and when they will not. Further, these measures can be used to better evaluate the effectiveness of bystander-focused violence prevention programs. Bystander-focused prevention initiatives have proliferated on college campuses, and are now growing in popularity in high schools, as school personnel seek to augment traditional health curricula with specific skills to decrease relationship abuse and sexual assault. Although this idea is gaining in popularity, research, especially in the high school context, is lagging behind. A critical component to taking next steps in basic research on bystanders and development of bystander programs are reliable and valid measures that are brief enough to be used in school contexts.

Limitations and Future Directions

A few shortcomings should be noted, as well as the implications of such shortcomings for future research. First, the sample, although large, lacked racial and ethnic diversity. Future research needs to identify measures and items that have utility across the diverse groups of youth, as well as those that address specific culturally relevant factors unmeasured in the current study. Further, some of the measures demonstrated factor structures and/or internal consistencies that are less than ideal. Yet, research indicates that measures with few items and/or poor internal consistency can be both valid and reliable (Lorber & Smith Slep, 2017; Zimmerman et al., 2006).

Research Implications

Given that some of the internal consistencies and indicators of factors structure were less than ideal, additional measurement development research is warranted. Also, because these are relatively new constructs in research on bystander action and relationship abuse and sexual assault, an important next step may include qualitative work, including focus groups and cognitive interviews with students of diverse cultural backgrounds. Focus groups could help us determine more about the ways in which youth think about peer norms regarding helping and media literacy, in particular. To date, most bystander research has primarily focused on the situational model and internal attitudes like rape myths and efficacy. The current paper takes a logical next step by developing measures about peer norms and other contextual variables that are important for leveraging bystander action.

Clinical and Policy Implications

The context of the current study was a field experiment to evaluate the effectiveness of a bystander-focused relationship and sexual assault prevention program for high school students. Although the primary aims were research in nature, the design of the measures were informed by implementation science and the importance of translating research tools into practice. Whereas the current measures could benefit from additional research, given their brevity and focus on variables that represent constructs that are often central to prevention programs, they have the potential to be used by teachers and practitioners who wish to track the impact of their work in classrooms. The current study also demonstrates the importance of developing valid and reliable measures that can be used by both researchers and practitioners in evaluating bystander-focused relationship abuse and sexual assault prevention efforts and can assist in rigorous evaluations of relationship abuse and sexual assault prevention programming among various populations for use by researchers.

We hope that the data presented in this paper provide a starting point for further measure development specific to relationship abuse and sexual assault among high school students.

Acknowledgments

Funding for this study was provided by the Centers for Disease Control and Prevention (CDC), Grant R01-CEO02524. The findings and implications presented in this paper do not represent the official views of the CDC. We owe a great deal of gratitude to our school and community partners and the 50+ research assistants and program facilitators. Without these agencies and individuals, this project would not have been possible.

Contributor Information

Katie M. Edwards, Departments of Psychology and Prevention Innovations Research Center, University of New Hampshire, Durham, New Hampshire;

Victoria L. Banyard, Departments of Psychology and Prevention Innovations Research Center, University of New Hampshire, Durham, New Hampshire;

Stephanie N. Sessarego, Department of Psychology, University of New Hampshire, Durham, New Hampshire;

Linda R. Stanley, Tri-Ethnic Center for Prevention Research, Colorado State University;

Kimberly J. Mitchell, Departments of Psychology and Crimes Against Children Research Center, University of New Hampshire, Durham, New Hampshire;

Robert P. Eckstein, Departments of Psychology and Prevention Innovations Research Center, University of New Hampshire, Durham, New Hampshire;

Kara Anne E. Rodenhizer, Department of Psychology, University of New Hampshire, Durham, New Hampshire;

P. Caroline Leyva, Departments of Psychology and Prevention Innovations Research Center, University of New Hampshire, Durham, New Hampshire..

References

  1. Ahrens CE, & Campbell R (2000). Assisting rape victims as they recover from rape: The impact on friends. Journal of Interpersonal Violence, 15, 28 10.1177/088626000015009004 [DOI] [Google Scholar]
  2. Ashcraft C (2006). “Girl, you better go get you a condom”: Popular culture and teen sexuality as resources for critical multicultural curriculum. Teachers College Record, 108, 2145–2186. 10.1111/j.1467-9620.2006.00777.x [DOI] [Google Scholar]
  3. Banyard VL (2008). Measurement and correlates of prosocial bystander behavior: The case of interpersonal violence. Violence and Victims, 23, 83–97. 10.1891/0886-6708.23.1.83 [DOI] [PubMed] [Google Scholar]
  4. Banyard VL (2013). Go big or go home: Reaching for a more integrated view of violence prevention. Psychology of Violence, 3, 115–120. 10.1037/a0032289 [DOI] [Google Scholar]
  5. Banyard VL (2015). Toward the next generation of bystander prevention of sexual and relationship violence: Action coils to engage communities (1st ed.). Geneva, Switzerland: Springer International Publishing; 10.1007/978-3-319-23171-6 [DOI] [Google Scholar]
  6. Banyard VL, & Moynihan MM (2011). Variation in bystander behavior related to sexual and intimate partner violence prevention: Correlates in a sample of college students. Psychology of Violence, 13, 287–301. 10.1037/a0023544 [DOI] [Google Scholar]
  7. Banyard VL, Moynihan MM, Cares A, & Warner R (2014). How do we know if it works? Measuring outcomes in bystander-focused abuse prevention on campuses. Psychology of Violence, 4, 101–115. 10.1037/a0033470 [DOI] [Google Scholar]
  8. Barker GT, Ricardo C, & Nascimento M (2007). Engaging men and boys in changing gender-based inequity in health: Evidence from programme interventions. Geneva, Switzerland: World Health Organization. [Google Scholar]
  9. Bickham DS, & Slaby RG (2012). Effects of a media literacy program in the US on children’s critical evaluation of unhealthy media messages about violence, smoking, and food. Journal of Children and Media, 6, 255–271. 10.1080/17482798.2012.662031 [DOI] [Google Scholar]
  10. Bonomi AE, Nichols EM, Carotta CL, Kiuchi Y, & Perry S (2016). Young women’s perceptions of the relationship in Fifty Shades of Grey. Journal of Women’s Health, 25, 139–148. 10.1089/jwh.2015.5318 [DOI] [PubMed] [Google Scholar]
  11. Brown AL, Banyard VL, & Moynihan MM (2014). College students as helpful bystanders against sexual violence: Gender, race, and year in college moderate the impact of perceived peer norms. Psychology of Women Quarterly, 38, 350–362. 10.1177/0361684314526855 [DOI] [Google Scholar]
  12. Centers for Disease Control and Prevention. (2014). Understanding teen dating violence - Fact sheet 2014. Retrieved from http://www.cdc.gov/violenceprevention/pdf/teen-dating-violence-2014-a.pdf
  13. Coker AL, Cook-Craig PG, Williams CM, Fisher BS, Clear ER, Garcia LS, & Hegge LM (2011). Evaluation of Green Dot: An active bystander intervention to reduce sexual violence on college campuses. Violence Against Women, 17, 777–796. 10.1177/1077801211410264 [DOI] [PubMed] [Google Scholar]
  14. Cook-Craig PG (2012). Evaluation of the Green Dot bystanding intervention program in Kentucky high schools. Paper presented at the 6th Biennial National Conference on Health and Domestic Violence Cook-Craig PG, Coker AL, Clear ER, Garcia LS, Bush HM, [Google Scholar]
  15. Cook-Craig PG, Coker AL, Clear ER, Garcia LS, Bush HM, Brancato CJ, … Fisher BS (2014). Challenge and opportunity in evaluating a diffusion-based active bystanding prevention program: Green Dot in high schools. Violence Against Women, 20, 1179–1202. 10.1177/1077801214551288 [DOI] [PubMed] [Google Scholar]
  16. Edwards KM, Banyard VL, Sessarego SN, Mitchell KJ, & Chang H (2017). Evaluating a bystander-focused relationship abuse and sexual assault prevention program with high school youth: A cluster randomized control trial. Manuscript in preparation. [Google Scholar]
  17. Edwards KM, Rodenhizer-Stämpfli KA, & Eckstein RP (2015). Bystander action in situations of dating and sexual aggression: A mixed methodological study of high school youth. Journal of Youth and Adolescence, 44, 2321–2336. 10.1007/s10964-015-0307-z [DOI] [PubMed] [Google Scholar]
  18. Edwards KM, Rodenhizer-Stampfli K, & Eckstein RP (2016). School staffs’ perceptions of bystander intervention in situations of dating and sexual violence among high school youth: A qualitative analysis. Manuscript under review. [Google Scholar]
  19. Eisinga R, Grotenhuis M, & Pelzer B (2013). The reliability of a two-item scale: Pearson, Cronbach, or Spearman-Brown? International Journal of Public Health, 58, 637–642. 10.1007/s00038-012-0416-3 [DOI] [PubMed] [Google Scholar]
  20. Espelage DL (2014). Ecological theory: Preventing youth bullying, aggression, and victimization. Theory into Practice, 53, 257–264. 10.1080/00405841.2014.947216 [DOI] [Google Scholar]
  21. Geldhof GJ, Preacher KJ, & Zyphur MJ (2014). Reliability estimation in a multilevel confirmatory factor analysis framework. Psychological Methods, 19, 72–91. 10.1037/a0032138 [DOI] [PubMed] [Google Scholar]
  22. Huang FL, & Cornell DG (2016). Using multilevel factor analysis with clustered data: Investigating the factor structure of the Positive Values Scale. Journal of Psychoeducational Assessment, 34, 3–14. 10.1177/0734282915570278 [DOI] [Google Scholar]
  23. Huang FL, Cornell DG, Konold T, Meyer JP, Lacey A, Nekvasil EK, … Shukla KD (2015). Multilevel factor structure and concurrent validity of the teacher version of the Authoritative School Climate Survey. The Journal of School Health, 85, 843–851. 10.1111/josh.12340 [DOI] [PubMed] [Google Scholar]
  24. Lorber MF, & Smith Slep AM (2017). The reliability paradox of the Parent-Child Conflict Tactics Corporal Punishment subscale. Journal of Family Psychology. Advance online publication. 10.1037/fam0000307 [DOI] [PubMed] [Google Scholar]
  25. Maine Department of Education. (2015). Student data. Data and Reporting. Retrieved from http://www.maine.gov/doe/data/student/index.html
  26. Massachusetts Department of Elementary and Secondary Education. (2017). 2016–17 enrollment by race/gender report (district). Retrieved from http://profiles.doe.mass.edu/state_report/enrollmentbyracegender.aspx
  27. McMahon S, & Banyard VL (2012). When can I help? A conceptual framework for the prevention of sexual violence through bystander intervention. Trauma, Violence, & Abuse, 13, 3–14. 10.1177/1524838011426015 [DOI] [PubMed] [Google Scholar]
  28. McMahon S, & Farmer GL (2011). An updated measure for assessing subtle rape myths. Social Work Research, 35, 71–81. 10.1093/swr/35.2.71 [DOI] [Google Scholar]
  29. McMahon S, Postmus JL, Warrener C, & Koenick RA (2014). Utilizing peer education theater for the primary prevention of sexual violence on college campuses. Journal of College Student Development, 55, 78–85. 10.1353/csd.2014.0001 [DOI] [Google Scholar]
  30. Muthén LK, & Muthén BO (1998–2015). Mplus user’s guide (Seventh ed.). Los Angeles, CA: Author. [Google Scholar]
  31. New Hampshire Department of Education. (2010). NH school and district profiles. Retrieved from https://my.doe.nh.gov/profiles/
  32. Nunnally JC, & Bernstein I (1994). The assessment of reliability. Psychometric Theory, 3, 248–292. [Google Scholar]
  33. Payne DL, Lonsway KA, & Fitzgerald LF (1999). Rape myth acceptance: Exploration of its structure and its measurement using the Illinois Rape Myth Acceptance Scale. Journal of Research in Personality, 33, 27–68. 10.1006/jrpe.1998.2238 [DOI] [Google Scholar]
  34. Porter SR, Whitcomb ME, & Weitzer WH (2004). Multiple surveys of students and survey fatigue. New Directions for Institutional Research, 2004, 63–73. 10.1002/ir.101 [DOI] [Google Scholar]
  35. Potter WJ (2004). Theory of media literacy: A cognitive approach. Thousand Oaks, CA: Sage Publishers; 10.4135/9781483328881 [DOI] [Google Scholar]
  36. Primack BA, Gold MA, Switzer GE, Hobbs R, Land SR, & Fine MJ (2006). Development and validation of a smoking media literacy scale for adolescents. Archives of Pediatrics & Adolescent Medicine, 160, 369–374. 10.1001/archpedi.160.4.369 [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Rinehart SJ, & Espelage DL (2016). A multilevel analysis of school climate, homophobic name-calling, and sexual harassment victimization/perpetration among middle school youth. Psychology of Violence, 6, 213–222. 10.1037/a0039095 [DOI] [Google Scholar]
  38. Smith CA, & Frieze IH (2003). Examining rape empathy from the perspective of the victim and the assailant. Journal of Applied Social Psychology, 33, 476–498. 10.1111/j.1559-1816.2003.tb01907.x [DOI] [Google Scholar]
  39. Ward LM (2003). Understanding the role of entertainment media in the sexual socialization of American youth: A review of empirical research. Developmental Review, 23, 347–388. 10.1016/S0273-2297(03)00013-3 [DOI] [Google Scholar]
  40. Yoon JS, & Barton E (2008). The role of teachers in school violence and bullying prevention School violence and primary prevention (pp. 249–275). New York, NY: Springer; 10.1007/978-0-387-77119-9_13 [DOI] [Google Scholar]
  41. Yoon JS, & Bauman S (2014). Teachers: A critical but overlooked component of bullying prevention and intervention. Theory into Practice, 53, 308–314. 10.1080/00405841.2014.947226 [DOI] [Google Scholar]
  42. Zimmerman M, Ruggero CJ, Chelminski I, Young D, Posternak MA, Friedman M, … Attiullah N (2006). Developing brief scales for use in clinical practice: The reliability and validity of single-item self-report measures of depression symptom severity, psychosocial impairment due to depression, and quality of life. The Journal of Clinical Psychiatry, 67, 1536–1541. 10.4088/JCP.v67n1007 [DOI] [PubMed] [Google Scholar]

RESOURCES