The Role of School Context in Implementing a Statewide Anti-Bullying Policy and Protecting Students

Abstract

Bullying is a significant problem in U.S. schools. Policies have been developed to reduce bullying, yet policy implementation by educators is an essential yet difficult and complex process. Few studies have investigated factors that act as barriers to or facilitators of bullying policy implementation and teacher protection of students. This study examined the influence of school context on educators’ capacity to implement a statewide bullying law and protect students from bullying following the enactment of the policy. Data were collected from 505 educators in 324 schools. School administrators tended to rate fidelity of policy implementation and teacher protection of students higher than teachers, education support professionals, and student service professionals. Policy implementation fidelity scores were higher in high schools than elementary schools. School size and the prevalence of student suspensions were inversely related to implementation fidelity. Higher levels of teacher protection were reported in elementary schools.

Introduction

Bullying is a pervasive and persistent threat to the well-being and school success of students. Bullying has been conceptualized a number of ways, with definitions varying slightly between countries, stakeholders (e.g., students, teachers, and parents), and scholars (Arora, 1996; Cheng, Chen, Ho, & Cheng, 2011; Cuadrado-Gordillo, 2012; Guerin & Hennessy, 2002; Mishna, 2004; Monks & Smith, 2006; Naylor, Cowie, Cossin, de Bettencourt, & Lemme, 2006; P. K. Smith et al., 2002; Vaillancourt et al., 2008). However, the most widely used definition of school bullying in the literature refers to it as unwanted aggressive behaviors enacted intentionally over time by a student or group of students using some form of power to cause physical and/or psychological harm to another student in a school setting (Gladden, Vivolo-Kantor, Hamburger, & Lumpkin, 2014; Olweus, 2013). A meta-analysis of 21 U.S. studies showed that on average, 18% of youth were involved in bullying perpetration, 21% of youth were involved in bullying victimization, and 8% of youth were involved in both perpetration and victimization (C. R. Cook, Williams, Guerra, & Kim, 2010). In addition, the Youth Risk Behavior Survey has shown a constant prevalence rate of 20% for bullying victimization since tracking began in 2009 (Eaton et al., 2010; Eaton et al., 2012; Kann et al., 2014).

Students who are victimized often suffer the most among those involved in the bullying dynamic. Victims of bullying often feel unsafe and unhappy in school, which can lead to school disengagement, absenteeism, and academic difficulties (Arseneault et al., 2006; Buhs, Ladd, & Herald, 2006; Glew, Fan, Katon, Rivara, & Kernic, 2005; Juvonen, Nishina, & Graham, 2000). Victimized students can also experience problems with concentration and attention regulation, which may contribute to problems in academic performance (Schwartz, McFadyen-Ketchum, Dodge, Pettit, & Bates, 1999). A meta-analysis of 33 studies found that being bullied was associated with lower grades, standardized test scores, and teacher ratings of academic achievement (Nakamoto & Schwartz, 2010).

Being bullied also contributes to mental health problems, including anxiety, depression, suicidal ideation and behavior, psychosomatic problems, and psychotic symptoms (Copeland, Wolke, Angold, & Costello, 2013; Dake, Price, & Telljohann, 2003; Gini & Pozzoli, 2009; Gini, Pozzoli, Lenzi, & Vieno, 2014; Hawker & Boulton, 2000; Kim & Leventhal, 2008; Klomek, Sourander, & Gould, 2010; Reijntjes, Kamphuis, Prinzie, & Telch, 2010; Rigby, 2003; Ttofi, Farrington, Lösel, & Loeber, 2011a). Researchers have demonstrated that psychological distress as a result of being bullied mediates the relationship between victimization and academic problems (S. Graham, Bellmore, & Mize, 2006; Juvonen et al., 2000; Nishina, Juvonen, & Witkow, 2005; Schwartz, Gorman, Nakamoto, & Toblin, 2005). Thus, bullying victimization is directly and indirectly related to poor educational outcomes.

Students who bully others also experience school problems. Teachers are more likely to report that bullies are academically disengaged (S. Graham et al., 2006), and many bullies do not see themselves as academically competent (Ma, Phelps, Lerner, & Lerner, 2009). Students who bully are more likely to skip school, perform poorly, and drop out (Jankauskiene, Kardelis, Sukys, & Kardeliene, 2008; Kokko, Tremblay, Lacourse, Nagin, & Vitaro, 2006; Ma et al., 2009; Nansel et al., 2001; Pereira, MendonçA, Neto, Valente, & Smith, 2004). In addition, bullying others in childhood predicts a number of serious behavioral problems (e.g., hyperactivity, disruptive behavior, assault, stealing, vandalism, and weapon-carrying) into adolescence and early adulthood (Farrington, Ttofi, & Lösel, 2011; Kumpulainen & Räsänen, 2000; Scholte, Engels, Overbeek, De Kemp, & Haselager, 2007; Ttofi, Farrington, & Lösel, 2012; Ttofi, Farrington, Lösel, & Loeber, 2011b). Given the prevalence of bullying and the deleterious outcomes among students involved in bullying, this study aimed to investigate the role of school contextual factors as barriers to or facilitators of two sets of actions intended to address bullying: the implementation of an anti-bullying policy and teacher protection of students from bullying.

Implementing Policy Interventions for Bullying

Currently, the United States does not have a federal law against school bullying. However, there are federal laws (e.g., Title IX of the Education Amendments of 1972) that address types of discriminatory harassment, which is a form of aggression similar to bullying (U.S. Department of Health and Human Services [USDHHS], 2014). Discriminatory harassment could include physical or verbal actions that are severe, persistent, or pervasive enough to adversely affect a student’s education or create a hostile or abusive school environment for a student and are based on a student’s race, color, national origin, sex, disability, or religion (USDHHS, 2014). Nonetheless, there has been a growing public concern and policymaking focus on bullying recently. Spurred by the shootings at Columbine High School in 1999 and the increasing awareness and concern about bullying and school safety since that time, all 50 states have passed anti-bullying laws (Birkland & Lawrence, 2009; USDHHS, 2015). These laws apply to approximately 98,000 K-12 public schools, with the goal of protecting more than 56 million students from involvement in bullying (Snyder & Dillow, 2013; USDHHS, 2015). These laws often define bullying as aggressive, intentional, and harmful acts among students (Stuart-Cassel, Bell, & Springer, 2011); however, some state laws do not specifically define bullying (e.g., Wisconsin) or give examples of bullying behaviors. Bullying can take many forms including physical bullying, verbal bullying, social/relational bullying, cyber-bullying, property bullying, and sexual bullying (Hall, 2015a). In addition, many definitions of bullying in the laws did not include two elements typically present in scholarly definitions of bullying: repetition over time and power imbalance between a bully and victim (Stuart-Cassel et al., 2011). Definitions of bullying in the laws that are broader than typical scholarly definitions of bullying (e.g., Olweus, 2013) may indicate policymakers’ intent to address a broader range of aggressive behaviors among students.

The mere passage of a policy by legislators or a board of education does not mean that a policy will be immediately and efficiently put into operation precisely as intended. Indeed, implementation is a complex, dynamic, and ongoing process involving a vast assortment of people, resources, organizational structures, and actions. State education policies are implemented in school systems primarily by district officials (e.g., superintendents and central office administrators) and school personnel (e.g., principals, assistant principals, teachers, education support professionals [ESPs], counselors, psychologists, and social workers), who work directly with the ultimate beneficiaries of education policy: students. Like many public agencies, schools operate in an environment of local, state, and federal systems and the associated social, cultural, economic, and political factors, which can change over time (Fixsen, Naoom, Blase, & Friedman, 2005). These multiple systems and factors can facilitate or impede policy implementation, which is an already challenging process nested within a complex, multilevel education system.

For a programmatic or policy intervention to accomplish its intended effects, it must be implemented with a high degree of fidelity (Carroll et al., 2007; Durlak & DuPre, 2008; Fraser, Richman, Galinsky, & Day, 2009). For policy interventions, fidelity refers to the extent to which a policy is implemented as intended based on the directives expressed in the policy document. Directives outlined in state anti-bullying laws vary somewhat but often require schools to formulate local bullying policies, train school personnel on the policy and bullying intervention, notify students and parents about the policy, establish procedures for reporting and investigating bullying incidents, establish appropriate consequences for bullies, and provide mental and behavioral health services for victims and bullies (Stuart-Cassel et al., 2011).

Researchers have found considerable variability in the fidelity of implementation of policy interventions for bullying. For example, 51% to 98% of educators reported that their school systems had adopted a local anti-bullying policy in compliance with their state’s policy (Bradshaw, Waasdorp, O’Brennan, & Gulemetova, 2011; Hedwall, 2006; Jordan, 2014; MacLeod, 2007; Robbins, 2011; Smith-Canty, 2010; Terry, 2010). In terms of training and notification regarding bullying policies, 46% to 94% of educators reported receiving training on the policy (Bradshaw et al., 2011; Hedwall, 2006; Holmgreen, 2014; Robbins, 2011; Smith-Canty, 2010; Terry, 2010), and 56% to 84% of educators reported that students were notified about the policy (Holmgreen, 2014; Jordan, 2014; LaRocco, Nestler-Rusack, & Freiberg, 2007; Robbins, 2011; Smith-Canty, 2010). Regarding school procedures, 60% to 94% of educators indicated that their school maintained procedures for reporting bullying (Holmgreen, 2014; LaRocco et al., 2007; Robbins, 2011), 78% to 92% of educators indicated that their school had procedures for investigating reports or complaints about bullying (Holmgreen, 2014; LaRocco et al., 2007; Smith-Canty, 2010), and 52% to 80% of educators indicated that their school provided mental health assistance to students involved in bullying (Hedwall, 2006; Holmgreen, 2014; Smith-Canty, 2010). These findings show that implementation fidelity varies across study locations and policy components. A policy must be implemented with a high degree of fidelity to have an effect and reduce bullying in schools.

There are a number of possible reasons education policies are not implemented with a high level of fidelity: Financial and human resources were insufficient for implementation, parents and community members opposed the policy, the policy was written using vague or confusing language, school personnel do not support or do not know how to implement the policy, and the work environment may be overly demanding and constraining for school personnel (Fowler, 2013). Indeed, many educators are situated in schools with high-need students and limited resources where they are asked to respond to a host of student needs related to their physical, psychological, social, and educational development. In the current climate of high-stakes academic testing, any new mandate may feel overwhelming unless it is accompanied by budgetary and professional resources (Fowler, 2013). Thus, implementing a new policy on top of a multitude of existing responsibilities with insufficient resources may be quite burdensome for educators.

Few studies have investigated the factors that act as barriers to or facilitators of school bullying policy implementation. Barriers to implementation included incomplete understanding of the policy by school members, poor agreement among personnel about what constituted bullying, bombardment of media attention about the policy, inadequate faculty and staff training, limited staff knowledge about bullying intervention strategies, lack of coordination among staff regarding protocols, lack of consistent follow-through by school personnel, lack of support from parents and school leaders, time constraints, and competing needs of students (Isom, 2014; LaRocco et al., 2007; Richman, 2010; Robbins, 2011; Schlenoff, 2014; Smith-Canty, 2010; Terry, 2010). Facilitators or drivers of bullying policy implementation include documents and tools developed by the district to assist school personnel with interpreting and implementing the policy (e.g., a flowchart of steps to take when investigating a bullying incident), prioritization of the policy by school and district leadership, and teacher commitment to stop bullying (Isom, 2014; Richman, 2010).

Teachers Intervening in Student Bullying

Teachers are the key actors involved in bullying intervention and prevention efforts (Newman, Frey, & Jones, 2010). In a national study, 92% of teachers indicated that bullying was problematic to some degree in their schools, and 98% of teachers agreed that it was their responsibility to intervene in bullying incidents (Bradshaw et al., 2011). However, almost half (45%) of the teachers had not received training on school bullying rules and procedures. Indeed, many educators have reported not feeling comfortable intervening or not knowing how to intervene in bullying (Bradshaw, Waasdorp, O’Brennan, & Gulemetova, 2013; Mishna, Pepler, & Wiener, 2006). Several barriers to educators addressing bullying have been identified, including lack of time, resources, and training (Bradshaw et al., 2013; Maunder & Tattersall, 2010; Mishna et al., 2006). In addition, teachers frequently felt both pressured to address student academic needs and exhausted from the demands associated with their many roles, which interfered with their capacity to effectively address bullying (Maunder & Tattersall, 2010; Mishna et al., 2006).

Educators’ intervening behaviors vary by individual and school characteristics. Compared with males, female educators were more likely to report responding to bullying (Bauman, Rigby, & Hoppa, 2008; Duy, 2013; Power-Elliott & Harris, 2012). Responsiveness may also differ by one’s role in the school. Compared with teachers, school counselors were less likely to ignore a bullying situation (Bauman et al., 2008). And in terms of responses, teachers were more likely to discipline the bully and counselors were more likely to try and help the victim. No differences in responding to bullying were found by educators’ age, education level, or years of experience (Bauman et al., 2008; Duy, 2013; Goryl, Neilsen-Hewett, & Sweller, 2013; Power-Elliott & Harris, 2012; Yoon, 2004). Results were mixed on the relationships between responding to bullying and teachers’ empathy for victimized students as well as their self-efficacy in managing student behavior problems (Yoon, 2004; Yoon, Sulkowski, & Bauman, 2016).

Teacher reports of socio-emotional variables related to the school climate have been positively associated with teachers’ propensity to intervene in bullying, including feeling connected with the school, staff, and students; the quality of educators’ relationships with students and parents; teacher trust in students, parents, colleagues, and the principal; feeling that the school environment is respectful and pleasant; perceptions of professional behavior among teachers; collegial principal leadership; and collective efficacy (Anderton, 2012; Hyde, 2014; Maunder & Tattersall, 2010; O’Brennan, Waasdorp, & Bradshaw, 2014; P. A. Smith & Birney, 2005; P. A. Smith & Hoy, 2004; Yoon et al., 2016). In addition, having resources available for bullying intervention in the school and receiving anti-bullying training were positively related to intervening (Bauman et al., 2008; O’Brennan et al., 2014).

The relationship between teacher protection of students from bullying and the socioeconomic status of the school is unclear (Anderton, 2012; Hyde, 2014; P. A. Smith & Birney, 2005). The size of the student body (Hyde, 2014; P. A. Smith & Birney, 2005) and school type (e.g., elementary, middle, or high school) do not appear to affect teacher intervention in bullying (Bauman et al., 2008; P. A. Smith & Hoy, 2004). In sum, these findings illustrate that many individual and organizational factors can aid or impede educators’ efforts to put bullying policies into practice and protect students from bullying; however, training, resources, intra- and inter-organizational coordination, competing needs and constraints, and the school climate may be particularly influential factors. Also, some factors do not appear to influence these processes, and the relationships between some explanatory factors and outcomes are still unclear.

Purpose of the Current Study

In this study, we intended to fill gaps in the literature by examining the relationships between school contextual variables that had not been used in previous studies (e.g., school geographic area, student to teacher ratio, and percent of students below grade level) and the outcomes of interest: fidelity of implementation of a bullying policy and teacher protection of students. This study also attempted to help clarify disagreements in the literature where some studies found significant relationships between contextual factors and outcomes and other studies found non-significant relations (e.g., school socioeconomic status and teacher protection of students). This study also improved on the methods used in other studies in the literature, which often relied on descriptive statistics and bivariate analyses as well as cross-sectional designs, by using multivariate analyses, corrections for clustering, and data from two time points to examine predictive relationships between the independent variables and the outcomes. The current study used data collected from educators at the end of 2010 concerning the School Violence Prevention Act of 2009 (SVPA), which is the anti-bullying law in North Carolina, and school context data from the 2009-2010 school year.

The purpose of this study was to examine the relationships between school contextual factors (i.e., school type, school geographic area, school size, student to teacher ratio, proportion of economically disadvantaged students, prevalence of student behavior problems, proportion of students above grade level in math and reading, student attendance rate, proportion of teachers with advanced degrees, teachers’ years of teaching experience, teacher turnover rate, and the per pupil expenditure) during the outset of the implementation of the SVPA and two outcome variables: fidelity of implementation of the law and protection of students from bullying. The capacity of educators to implement components of an anti-bullying policy and protect students from bullying may indeed hinge on the school context. For example, some educators may be in schools where they are teaching and managing large groups of students who may present challenges in terms of missing school, struggling academically, and misbehaving. Such conditions may likely act as barriers to educators implementing a new policy and ensuring the protection of all students from aggressive behaviors. However, some educators may be in schools with smaller class sizes, higher levels of funding, fewer students from economically disadvantaged backgrounds, and fewer students with academic difficulties. Such conditions may likely facilitate educators’ capacity for implementing new policies and intervening in student bullying.

Based on the review of the literature above, the following hypotheses were developed:

Hypothesis 1: Student to teacher ratio, proportion of economically disadvantaged students, proportion of students below grade level on end of grade tests, prevalence of student suspensions, and teacher turnover rate will be inversely related to teacher protection and implementation fidelity.

Hypothesis 2: Student attendance rate and per pupil expenditure will be positively associated with teacher protection and implementation fidelity.

Hypothesis 3: School type, school geographic area, school size, the proportion of teachers with advanced degrees, and the proportion of teachers with more or less years of experience will not be significantly related to teacher protection and implementation fidelity scores.

Method

Policy Design

The SVPA was signed into law on June 23, 2009. In the law, bullying was defined as verbal, written, electronic, or physical actions that induced fear of harm or created a hostile environment for a student. Such behaviors were prohibited as well as bullying behavior based on actual or perceived race, color, ancestry, national origin, religion, gender, socioeconomic status, academic status, sexual orientation, gender identity, physical appearance, and disability. The law applied to behavior on school property, at school-sponsored functions, and on school buses. According to the law, school personnel who witnessed or possessed information about bullying were required to report incidents to the appropriate school officials. However, students and school volunteers were encouraged but not required to report bullying incidents.

The law also required that school districts adopt their own local anti-bullying policies by December 31, 2009 and train all school employees by March 1, 2010. Each local policy had to include the provisions described above in terms of the definition of bullying, prohibition of bullying behaviors, enumerated statuses protected, scope of the policy, and bullying reporting requirements as well as other components:

1. behavioral expectations for students and school personnel,

2. procedures for reporting bullying incidents,

3. identification of a school employee designated to investigate reports of bullying,

4. procedures for investigating reports of bullying incidents,

5. prohibition of reprisal or retaliation against individuals who reported bullying incidents,

6. consequences and appropriate remedial actions for students who committed acts of bullying,

7. plans to publicize and disseminate the local policy,

8. inclusion of the local policy in student and employee handbooks, and

9. inclusion of the local policy in employee training.

The quality of the content of anti-bullying policies shapes their capacity to effectively reduce bullying. Two formal evaluations were completed on the content of the SVPA. First, the SVPA contains 13 out of 16 or 81% of key policy components identified in a national review of state anti-bullying policies by the U.S Department of Education (Stuart-Cassel et al., 2011). Second, a study found that the SVPA included 67% of protective factors identified in the literature as associated with reduced bullying behaviors, risk of bullying, or consequences from bullying (Weaver, Brown, Weddle, & Aalsma, 2013). The protective factor score for the SVPA was in the top 15% of state policy scores. These findings suggest that the content of the SVPA is good in that a majority of its components can potentially reduce or prevent school bullying.

Study Design and Procedures

This study involved a cross-sectional survey merged with administrative data on school contextual variables present the school year following the passage of the SVPA. Because school-level educators are the primary implementers of education policy, we surveyed members of a statewide professional association of educators and school employees in North Carolina. The survey was announced in an email message sent through the association’s membership listserv. The email invitation contained a brief description of the survey, stating that it was focused on bullying, was optional and anonymous, and could be completed in 15 min. The email also contained a link to the welcome and informed consent page of the online survey. An online survey format was selected because of several advantages: Participants can respond to a Web survey at times and places convenient for them, participants can often complete Web surveys quickly, and participants may be less affected by social desirability bias in their responses because they are not directly disclosing the information to another person (Evans & Mathur, 2005; Granello & Wheaton, 2004; Rhodes, Bowie, & Hergenrather, 2003).

In the survey, participants initially completed three demographic questions and were asked to identify the school and district in which they worked. Identifying their school allowed us to merge the survey data with school-level data. The remainder of the survey items assessed teacher protection of students and the implementation of the SVPA. No material incentives were used to solicit participation. The survey was available from mid-November 2010 to early January 2011. It can take 3 to 5 years from the time a school-wide policy or program is adopted to the time it can be implemented with fidelity and have a measureable effect (Bradshaw, Reinke, Brown, Bevans, & Leaf, 2008; Cooper, Fusarelli, & Randall, 2004). Evaluating fidelity may be more useful within the early stages of implementation to identify implementation problems and problems inherent in the policy design that may need to be addressed. Therefore, we decided to collect data on implementation a year following the date that school districts were required to enact local anti-bullying policies.

Sample

Of the approximately 5,000 educators who were invited to participate, 664 (13.3%) responded to the survey to some extent. However, 159 respondents were excluded because they did not complete the survey beyond the demographic items, worked in private or charter schools, worked in special education or vocational schools, or did not identify their school. These latter three criteria prevented merging the survey data with the school-level administrative data. Thus, a total of 505 participants were included for data analysis in this study, which was 10.1% of the educators invited to participate. Bivariate analyses comparing the 505 included and the 159 excluded respondents showed no significant differences in terms of the proportions of White, non-White, male, and female respondents.

The sample of 505 educators included 78% teachers, 11% ESPs, 4% school administrators, 3% school counselors, 2% school social workers, and 1% school nurses. The racial/ethnic breakdown of the sample was 77% White/Caucasian, 17% Black/African American, 2% Hispanic/Latino/Latina, 1% American Indian or Alaska Native, and 3% multiracial/multiethnic. The sample included 83% females and 17% males. These sample demographics are closely aligned with statewide representative demographic data of NC K-12 public school teachers, which shows that 81% of teachers were White, 16% were Black, 1% were Hispanic, 1% were American Indian or Alaska Native, 1% were Asian, 80% were female, and 20% were male (U.S. Department of Education, 2009).

Respondents were employed in 324 schools in 85 (74%) of the 115 school districts in North Carolina. All of the participants worked in regular education K-12 public schools. The grade levels at the schools where participants worked varied with 40% in elementary schools, 2% in elementary–middle schools, 25% in middle schools, 3% in middle–high schools, and 29% in high schools. In terms of the geographic area of participants’ schools, 62% were in small town or rural areas, 23% were in urban areas, and 15% were in suburban areas. These figures are similar to those among all NC K-12 public schools: 51% were elementary schools, 18% were middle schools, 21% were high schools, 5% were elementary–middle schools, 3% were middle–high schools, and 2% were K-12 schools (North Carolina Department of Public Instruction, 2012). And in terms of geographic area, 63% were in small town or rural areas, 25% were in urban areas, and 12% were in suburban areas.

Dependent Variables

Fidelity of bullying policy implementation

Fidelity of implementation of the SVPA was measured using nine items designed by a group of individuals who were involved in advocating for the passage of the SVPA, which included one educator, one parent, and two researchers. The items were constructed based on the content of the SVPA and assessed implementation fidelity of nine policy components. Fidelity of implementation related to protected social classes (i.e., race, national origin, gender, socioeconomic status, sexual orientation, gender identity, physical appearance, and disability status) was emphasized for two reasons. First, this aspect of the law was highly controversial during the formulation of the policy (Comer, 2009), and thus, might not be executed as intended. Second, youth who are vulnerable or members of minority groups are often targeted for bullying and report high rates of victimization (Elamé, 2013; Peguero, 2012).

To assess implementation fidelity, participants were asked the following:

1. whether or not they had received training on the SVPA, with response options of yes and no;

2. how often they knew whom to report incidents of bullying to at their school, with response options of never, rarely, sometimes, most times, and always;

3. how often students at their school knew whom to report incidents of bullying to, with response options of never, rarely, sometimes, most times, and always;

4. which social statuses were protected from bullying in their school’s local policy, and participants could select I don’t know for the item or select yes or no beside each of eight social statuses;

5. whether they had received training about bullying based on the eight social statuses, and participants could select I don’t know for the item or select yes or no beside each of the eight social statuses;

6. whether students in their schools had been informed that bullying was prohibited based on the eight social statuses, and participants could select I don’t know for the item or select yes or no beside each of the eight social statuses;

7. how often employees at their school reported witnessing bullying incidents based on the eight social characteristics to the designated school official, with response options of never, rarely, sometimes, most times, and always;

8. how often school officials investigated reports of bullying based on the eight social status characteristics, with response options of never, rarely, sometimes, most times, and always; and

9. how often appropriate remedial action was given to students who perpetrated bullying based on the eight social status characteristics, with response options of never, rarely, sometimes, most times, and always.

Fidelity scores are often expressed as percentages where 0% would indicate that an intervention was not at all implemented as intended and 100% would indicate that an intervention was completely implemented as intended (Linnan & Steckler, 2002). Using this logic, participants’ responses were coded or calculated as percentages for the nine implementation variables. For Item 1, responses of yes and no or I don’t know were coded as 100% and 0%, respectively. For Items 2 and 3, responses of never, rarely, sometimes, most times, and always were coded as 0%, 25%, 50%, 75%, and 100%, respectively. For Items 4 through 9, a percentage was calculated based on the number of yes responses selected out of eight. Thus, no yes responses received a 0% score, one yes response received a 12.5% score, two yes responses received a 25% score, and so on. Finally, an overall implementation fidelity score was calculated by averaging the nine percentages.

Teacher protection of students from bullying

The protection of students from bullying was measured using a subscale from the Bully Index (P. A. Smith & Hoy, 2004), which is an educator-report measure assessing (a) perceptions of bullying among students at school and (b) teacher protection of students. Only the four-item subscale pertaining to teacher protection was used for this study. Participants were asked to think about the school in which they worked and rate their agreement with four statements (e.g., “Teachers in my school reach out to help students who are harassed by other students”) using a 5-point Likert-type scale ranging from 1 (disagree) to 5 (agree). One item was reverse coded. A teacher protection score was calculated by averaging responses from the four items. Higher scores indicate higher levels of teacher engagement in protection of students in the school from bullying (P. A. Smith & Hoy, 2004). In other studies, this subscale has demonstrated acceptable to good internal consistency reliability (αs = .73-.96) as well as evidence of convergent and divergent validity (Anderton, 2012; Hyde, 2014; P. A. Smith & Birney, 2005; P. A. Smith & Hoy, 2004). The internal consistency reliability of this subscale in the present study was α = .77, which was acceptable.

Independent Variables

Individual-level variables

Three individual-level demographic variables of educators were measured:

1. type of educator, which included the following options: teacher, ESP, student service professional (i.e., school counselor, social worker, and nurse), and school administrator, which was the reference group;

2. educator race/ethnicity was recoded into a binary variable as person of color and White, which was the reference group; and

3. educator sex/gender included female and male, which was the reference group.

School-level variables

School-level data collected through the North Carolina Department of Public Instruction for the 2009-2010 school year included the following variables:

1. school type (i.e., elementary, elementary–middle, middle, middle– high, and high school), with elementary as the reference group;

2. school geographic area (i.e., small town/rural, urban, or suburban), which was based on the coding scheme developed by the National Center for Education Statistics and the U.S. Census Bureau (U.S. Department of Education, n.d.), with urban as the reference group;

3. size of the student body in terms of average daily membership;

4. student to teacher ratio, which was attained by dividing the average daily membership by the total number of classroom teachers in the school;

5. percent of economically disadvantaged students (i.e., those eligible for free or reduced-price lunch);

6. number of short-term suspensions per 100 students;

7. percent of students scoring below grade level on the end of grade math test;

8. percent of students below grade level on the end of grade reading test;

9. student attendance rate;

10. percent of teachers with advanced degrees (i.e., master’s, educational specialist, or doctoral degrees);

11. percent of teachers with less than 4 years of teaching experience;

12. percent of teachers with 4 to 10 years of teaching experience;

13. percent of teachers with more than 10 years of teaching experience;

14. teacher turnover rate (i.e., the percent of teachers in the school who left their positions in the past year); and

15. the total per pupil expenditure in dollars (i.e., the sum of local, state, and federal expenditures per student).

Data Analysis

Prior to analysis, a number of diagnostics were performed using Stata (Version 13) to examine the linearity between the independent and dependent variables, the distributions of the residuals, the distributions of the independent and dependent variables, influential outliers, heteroscedasticity, multi-collinearity, missing values, and clustering. Plots showed no clear departures from linearity, and the residuals were approximately normally distributed for both dependent variables. All of the non-categorical variables were approximately normally distributed except for the number of suspensions per 100 students. This variable had a positively skewed distribution, and thus, a natural log transformation was used to achieve a normal distribution. No cases of influential outliers (Cook’s distance values > 1) were found, and no significant heteroscedasticity problems were found. However, two multicollinearity problems (variance inflation factor scores > 10) were found for the variables representing the proportion of students below grade level in math and reading. These two variables were highly correlated (r = .91, p < .01), and thus, were averaged together to create one new variable.

Eight percent of values were missing; thus, full information maximum likelihood (FIML) was used to handle missing data. This procedure allows for all cases to be included in analyses, even if they are missing values on some variables. FIML has been shown to perform better than listwise deletion, pairwise deletion, mean imputation, and multiple imputation in terms of generating unbiased estimates (Allison, 2012; Dong & Peng, 2013; Enders, 2001; J. W. Graham, 2009; Widaman, 2006).

A final methodological issue that needed to be addressed in this study was the multilevel nature of the data—educators clustered within schools. Educators at the same school may share common characteristics on an outcome variable compared with those in other schools. Not accounting for clustering can lead to biased estimated standard errors and spurious results (Guo, 2005). Using the intraclass correlation coefficient (ICC) developed by Raudenbush and Bryk (2002), we examined the clustering effects for the two outcome variables. The ICC is the proportion of the total explained variation in an outcome that is attributable to differences between contexts, in this case, schools. Results showed that the clustering effects were low: ICC = .059 for implementation fidelity and ICC = .057 for teacher protection. These results indicate that less than 6% of the variation in the outcome variables is between schools. Therefore, problematic clustering effects were negligible. Generally, multilevel modeling is not necessary when the ICC is less than .10 (Heinrich & Lynn, 2001; Kreft, 1996; Lee, 2000; What Works Clearinghouse, 2008) because the results would most likely not be different than results from regression modeling. Nonetheless, the generalized Huber–White sandwich estimator (Rogers, 1993) was used to account for clustering in the models. Stata (Version 13) was used to analyze the data because it allows for correcting for clustering effects via the Huber–White sandwich estimator and FIML.

Results

Descriptive statistics showed that the extent to which the SVPA was implemented varied across schools and policy components (Table 1). To examine the relationships between school contextual factors and the outcome variables, we used multivariate robust regression modeling. Table 2 shows the means, standard deviations, and intercorrelations for the non-categorical study variables. School size was inversely correlated with teacher protection, and the prevalence of student suspensions was inversely correlated with both teacher protection and implementation fidelity. Fidelity of policy implementation was positively correlated with teacher protection of students.

Table 1.

Descriptive Statistics on the Fidelity of Implementation of the School Violence Prevention Act.

Policy component	M	SD	Range
Training school personnel on the law	39.2	48.9	0-100
Educator knowledge of bullying reporting  procedures	91.6	17.7	0-100
Student knowledge of bullying reporting  procedures	78.4	21.1	25-100
Inclusion of protected social statuses in the  local bullying policy	73.1	40.8	0-100
Training school personnel about protected  social statuses	35.6	44.4	0-100
Student knowledge of social statuses  protected from bullying	63.1	44.2	0-100
School personnel reporting bullying  incidents based on social statuses	78.6	22.5	0-100
Investigating reports of bullying based on  social statuses	82.0	24.1	0-100
Taking appropriate remedial action with  bullying perpetrators based on social  statuses	73.0	27.2	0-100
Overall policy implementation fidelity  composite score	64.9	27.0	0-100

Note. All values are percentages.

Table 2.

Means, Standard Deviations, and Intel-correlations for Continuous Variables.

Variable		M	SD		2	3	4	5	6	7	8	9	10	11	12	13
1.	School size or number of students	722.06	448.18	—
2.	Student to teacher ratio	13.89	2.67	.60*	—
3.	Percent of economically disadvantaged students	52.96	19.63	−.41*	−.42*	—
4.	Number of suspensions per 100 students	1.08	0.56	−.17*	−.18*	.32*	—
5.	Percent of students below grade level	27.56	13.79	−.32*	−.55*	.71*	.59*	—
6.	Attendance rate	94.41	2.90	.34	.46*	−.36*	−.57*	−.67*	—
7.	Percent of teachers with advanced degrees	27.16	9.58	.12*	.11*	−.23*	−.24*	−.22*	.12*	—
8.	Percent of teachers with 0 to 3 years experience	20.10	9.58	−.14*	−.27*	.35*	.27*	.43*	−.27*	−.37*	—
9.	Percent of teachers with 4 to 10 years experience	29.48	9.36	.13*	.23*	−.16*	−.24*	−.11*	.28*	.09*	−.23*	—
10.	Percent of teachers with 1 1 or more years experience	50.57	11.76	.01	.04	−.16*	−.03	−.27*	.00	.24*	−.63*	−.60*	—
11.	Teacher turnover rate	11.78	6.45	−.09	−.16*	.36*	.40*	.46*	−.37*	−.30*	.45*	−.14*	−.27*	—
12.	Per pupil expenditure in dollars	8,802.85	1,013.07	−.31*	−.28*	.23*	−.02	.06	.03	−.01	.04	−.22*	.14*	.09	—
13.	Policy implementation fidelity score	64.89	26.99	−.12	−.00	.03	−.10*	−.06	.01	−.04	.01	−.08	.06	.04	.07	—
14.	Teacher protection of students score	4.24	0.78	−.13*	−.02	−.03	−.23*	−.08	.08	−.00	.02	.05	−.05	−.02	−.01	.51*

^*p < .05.

Two robust regression models were run predicting fidelity of policy implementation and teacher protection of students using the same set of individual demographic and school contextual variables for both models. Educator type, race/ethnicity, sex/gender, school type, and geographic area were dummy coded so that these categorical variables could be used as predictors in the models. Table 3 shows the results of the regression analyses, including unstandardized regression coefficients, standard errors, 95% confidence intervals, and R² values for the two models. The independent variables accounted for significant yet fairly small amounts of the variance in implementation fidelity (R² = .104) and teacher protection (R² = .130).

Table 3.

Robust Regression Analyses Predicting Fidelity of Bullying Policy Implementation and Teacher Protection of Students (N = SOS).

	Policy implementation fidelity model			Teacher protection model
Independent variables	B	SE	95% CI	B	SE	95% CI
Individual demographics
Educator type (school administrator)
Teacher	−22.40*	4.54	[−31.30,−13.50]	−0.46*	0.12	[−0.71,−0.22]
Education support professional	−24.74*	5.3S	[−35.22,−14.26]	−0.61*	0.17	[−0.94, −0.28]
Student service professional	−20.77*	6.53	[−33.56, -7.97]	−0.73*	0.21	[−1.14,−0.32]
Educator race/ethnicity (person of color = 1)	−1.48	3.18	[−7.71,4.76]	0.12	0.1 1	[−0.10,0.33]
Educator gender (female = 1)	3.79	2.97	[−2.03, 9.60]	0.09	0.09	[−0.27,0.10]
School-level characteristics
School type (elementary)
Elementary-middle	−18.14	9.36	[−36.48, 0.20]	−0.49*	0.21	[−0.90, −0.09]
Middle	4.31	4.38	[−4.28, 12.89]	−0.16	0.1 1	[−0.38, 0.06]
Middle–high	3.82	7.47	[−18.45, 10.81]	−0.29	0.18	[−0.64, 0.06]
High	10.55*	4.70	[1.33, 19.76]	−0.21	0.14	[−0.49, 0.07]
School geographic area (urban)
Suburban	3.73	3.98	[−4.08, 1 1.54]	0.05	0.10	[−0.15,0.25]
Rural	−3.41	3.33	[−9.93,3.1 1]	−0.06	0.09	[−0.24,0.13]
School size or number of students	−0.02*	0.00	[−0.03,−0.01]	−0.00	0.00	[−0.00, 0.00]
Student to teacher ratio	1.84	0.81	[0.26, 3.42]	0.01	0.02	[−0.03, 0.05]
Percent of economically disadvantaged  students	0.09	0.09	[−0.09, 0.27]	−0.00	0.00	[−0.01,0.00]
Number of suspensions per 100 students	−9.07*	3.93	[−16.77, −1.36]	−0.18	0.1 1	[−0.41,0.04]
Percent of students below grade level	−0.04	0.19	[−0.40, 0.33]	−0.01	0.01	[−0.02, 0.00]
Attendance rate	−0.86	0.59	[−2.03, 0.30]	−0.03	0.02	[−0.06,0.01]
Percent of teachers with advanced degrees	−0.13	0.14	[−0.39,0.14]	−0.00	0.00	[−0.01,0.01]
Percent of teachers 0-3 years experience	0.74	2.14	[−3.46, 4.94]	0.02	0.07	[−0.1 1,0.15]
Percent of teachers 4-10 years experience	0.7S	2.14	[−3.46, 4.96]	0.02	0.06	[−0.1 1,0.14]
Percent of teachers with 11 or more years  experience	0.91	2.IS	[−3.28,5.1 1]	0.02	0.06	[−0.1 1,0.14]
Teacher turnover rate	0.17	0.2S	[−0.33, 0.66]	0.01	0.01	[−0.01,0.02]
Per pupil expenditure in dollars	0.00	0.00	[−0.00, 0.00]	−0.00	0.00	[−0.00, 0.00]
R2	.104			.130

Note. School administrator is the reference group for educator type. Race/ethnicity was coded 0 = White, 1 = person of color. Gender was coded 0 = male, 1 = female. Elementary is the reference group for school type coded as 0. Urban is the reference group for geographic area coded as 0. CI = confidence interval.

^*p .05.

In both models, school administrators rated policy implementation fidelity and teacher protection scores significantly higher than teachers, ESPs, and student service professionals.

In addition, high school educators reported significantly higher implementation fidelity scores than those in elementary schools. Also, in this model, the number of students and prevalence of suspensions were significantly inversely related to implementation fidelity. Regarding the second model, elementary school educators had the highest teacher protection scores, and this difference was significantly higher when elementary school educators were compared with those in elementary–middle schools, which had the lowest teacher protection scores.

Discussion

Findings from this study indicate that the SVPA has not been implemented with a high level of fidelity in North Carolina schools. In fact, the results show considerable variability in the levels of implementation. For almost every policy component, scores ranged from 0% to 100%, which suggests that in some schools, educators are not implementing the SVPA at all, and in other schools, educators are implementing the SVPA as intended. A small part of this variability in implementation is due to school contextual factors that were present during the first school year following the passage of the SVPA into law.

Two factors that acted as barriers to implementation included school size and student suspensions. Thus, as the number of students in a school increased, the fidelity of implementation of the SVPA decreased. Similarly, as the number of suspensions per 100 students increased, the fidelity of implementation of the SVPA decreased. One explanation for this finding rests on the assumption that suspensions are a consequence of a range of serious behavior problems among students. Historically, suspensions most often resulted from physical fighting, insubordination, class disruption, skipping class, drug use and selling, vandalism, and weapon possession (Cameron, 2006). Thus, educators in schools with high suspension rates may be facing a range of frequent student behavior problems and have less time to faithfully implement the components of a new policy. Another possible explanation for the finding is that suspensions are overused as a punitive response to serious and moderate student behavior problems, which contributes to an exclusionary school culture. In an exclusionary culture, certain students are excluded from fully participating in and benefiting from the education system (Brady, 2005). Research shows that Black and Latino/Latina students are more likely to be suspended and excluded from schools than White students for the same or similar behavior problems (Skiba et al., 2011). Thus, in schools where suspensions are frequently used for even minor behavioral infractions, which result in the exclusion of certain minority student groups, educators may be less inclined to implement elements of an anti-bullying policy that address bias-based bullying because there is a culture of exclusion at the school.

Besides the current study, only one other study investigated the relationship between implementation of a bullying policy and school size. In this study of high school teachers, respondents from larger schools were more likely to report having a bullying policy at their school but were less likely to report understanding the policy, receiving training on the policy, being periodically reminded about the policy, and having school-wide systems in place to help victims and work with bullies (Hedwall, 2006). Being in a large school may hinder educators’ capacity to consistently implement new bullying policy practices because larger organizational systems have more employees, which may involve challenges to communication, collective decision-making, and coordination of actions. Having a large number of school employees may also contribute to a diffusion of responsibility where educators may not take action in terms of following bullying procedures (e.g., reporting a bullying incident witnessed in the cafeteria to an administrator) because many other educators are present in the school and it is assumed that another adult will take action.

Regarding teacher protection, scores differed by school type. Teacher protection scores were higher among elementary school educators compared with those in higher school grade levels; however, this difference was only statistically significant between elementary and elementary–middle schools. Compared with elementary schools, higher grade level schools generally have more students and larger student to teacher ratios, and students spend less time with teachers (Snyder & Dillow, 2013). In addition, after the elementary grades, the academic culture of school often becomes more focused on demonstrating high ability through testing and competing for grades, whereas the elementary grades focused on developing, mastering, and understanding knowledge and skills (Meece, Anderman, & Anderman, 2006). These systemic differences may influence the relationships and interactions between teachers and students as they progress out of the elementary grade levels. Studies show that relationships between teachers and students decline through elementary school and as students transition into middle school in terms of closeness and relationship quality (Jerome, Hamre, & Pianta, 2009; O’Connor, 2010; O’Connor & McCartney, 2007; Pianta & Stuhlman, 2004). Thus, teachers may become less nurturing and protective of students as they mature and transition out of the elementary grade levels.

Results of the present study also showed higher levels of implementation fidelity in high schools as opposed to elementary schools. Higher levels of implementation fidelity in high schools may have been due to the focus on bias-based bullying in the measurement of fidelity. Six of the nine items assessing implementation of the SVPA focused on bias-based bullying (e.g., informing students that bullying based on race, national origin, gender, socio-economic status, sexual orientation, gender identity, physical appearance, and disability status was prohibited). A nationally representative study showed that prevalence rates for bias-based aggression were significantly higher among high school–age students than elementary school–age students (Finkelhor, Turner, Ormrod, & Hamby, 2009). Physical bullying is most prominent among elementary school–age children, and biased-based attacks are quite rare (Finkelhor et al., 2009), which may be due to social-cognitive developmental differences. High school educators may have reported higher levels of engagement in implementing the SVPA because most of the survey items focused on bias-based bullying, which is more prevalent in high schools.

We also found that school administrator scores on implementation fidelity and teacher protection were significantly higher than those among teachers, ESPs, and student service professionals. Two other studies of bullying policy implementation also found reporting differences between respondent groups. Compared with teachers and school counselors, school administrators reported higher levels of bullying policy implementation in terms of having a bullying policy, communicating the policy to members of the school community, reporting bullying incidents to appropriate officials, and disciplining perpetrators of bullying (Barnes, 2010; Jordan, 2014). In addition, compared with teachers and counselors, administrators were more likely to report that the school was effective at deterring and reducing bullying behavior (Barnes, 2010; Jordan, 2014). Findings from these studies as well as the present study suggest that school administrators’ responses regarding actions to address bullying in their schools may be influenced by response bias. Administrators are the primary leaders of their schools and part of their identity as well as their job performance is likely tied to the successful operation of their schools. Thus, they may be influenced by a social or political desirability response bias in which it is advantageous to deny the presence of undesirable characteristics in their schools (e.g., failing to implement a new law) and to affirm the presence of socially desirable characteristics (e.g., teachers protecting students from bullying).

Limitations

This study has several limitations. First, this study did not use probability sampling, and thus, the sample may not be representative of educators across North Carolina. Another possible limitation was the somewhat low response rate; however, a low response rate does not necessarily lead to nonresponse error, and there are examples of surveys with lower response rates that were as accurate as or more accurate than those with higher response rates (Keeter, Kennedy, Dimock, Best, & Craighill, 2006; Krosnick, 1999; Visser, Krosnick, Marquette, & Curtin, 1996). The representativeness of a sample is more important than the response rate (C. Cook, Heath, & Thompson, 2000), and our sample demographics parallel those that are representative of teachers in the state. Second, numerous respondents were excluded from the analyses because they did not identify the school in which they worked, which prevented merging their data with school-level data. This resulted in a drop in sample size. Third, there may have been selection bias because educators who took the survey may have been more interested in bullying, and therefore, may have been overly critical in their assessment of the implementation of the SVPA and teacher efforts to protect students. However, participants’ responses may have been influenced by social desirability response bias because educators were asked to report on legally mandated actions related to the SVPA and actions by teachers in their schools to protect students from bullying. Fourth, relevant explanatory contextual variables, such as the socio-emotional climate of a school, were not collected in this study, which could have provided richer assessments of school settings. Fifth, implementation was assessed at only one point in time, yet it is an ongoing process and additional assessments might have shown improvements in fidelity. A final limitation was that bullying was not explicitly defined in our measures, and therefore, some respondents may have included similar yet legally demarcated instances of other aggressive behaviors (e.g., discriminatory harassment) in their understanding of bullying.

Future Research

In the future, researchers should collect data from multiple respondent groups (e.g., administrators, teachers, and school mental health professionals) regarding bullying policy implementation and teacher protection of students to triangulate findings, and caution should be taken in relying exclusively on administrator responses. In addition, to attain larger, more representative samples, researchers should partner with state departments of education or public instruction and use some form of probability sampling. Studies examining bullying policies were more likely to have used probability sampling when researchers collaborated with educational agencies in their data collection (Hall, 2015b). Educational agencies have a vested interest in the implementation and success of anti-bullying initiatives, especially those codified as law.

An area for future research could focus on differences in policy implementation and teacher intervention across types of bullying. Although there are several different types of bullying (e.g., physical bullying, verbal bullying, social/relational bullying, cyber-bullying, property bullying, and sexual bullying), anti-bullying laws vary in terms of which behaviors are specifically prohibited (Stuart-Cassel et al., 2011). Future research may reveal that addressing certain types of bullying may be more difficult than others (e.g., cyber-bullying vs. physical bullying).

Future studies should also examine individual and school factors that may act as barriers to or facilitators of bullying intervention in terms of policy-related procedures and protective teacher behaviors. Individual factors could include educators’ views of bullying as problematic or normative, support for anti-bullying policies and programs, and level of training and competency to intervene in bullying. School climate factors that may also influence anti-bullying efforts include respect for diversity, school connectedness, parent and community involvement, professional development, school resources and instructional support, and school leadership. Some states survey educators annually about the school climate because these data can be used to understand the quality and character of school life. Important school structural factors to include in future bullying-related studies concern levels of funding marked specifically for bullying initiatives and the presence of personnel in schools and district offices who are specifically responsible for overseeing or coordinating anti-bullying policies and programs. In addition to these variables, standard school characteristics (e.g., school size and teacher to student ratio) should still be collected and included in analytic models because they do account for some of the variance in the outcomes and such variables are uniformly collected from public schools across the United States and are publicly available. These contextual variables may moderate the relationships between individual, cultural, and structural factors and the outcomes. Additional research on the actions of educators as they continue to put bullying policies into practice in schools across the country and protect students from engagement in and the consequences from bullying is imperative to reduce the deleterious phenomenon of bullying.

Author Biographies

William J. Hall is a Postdoctoral Fellow in the Cecil G. Sheps Center for Health Services Research at the University of North Carolina at Chapel Hill. His research focuses on the school experiences and mental health of lesbian, gay, bisexual, transgender, and queer youth.

Mimi V. Chapman is an Associate Professor in the School of Social Work at the University of North Carolina at Chapel Hill. Her research focuses on interventions with helping professionals regarding Latino/Latina immigrant adolescents and their families.