Abstract
Research suggests that involvement in relational aggression is associated with serious adjustment problems, including concurrent and future social maladjustment (e.g., problematic friendships; rejection), internalizing problems (e.g., depressive symptoms), and school avoidance. Despite the burgeoning literature focusing on the harmful and damaging nature of relationally aggressive behavior, this research has only recently begun to be used to inform school-based prevention and intervention programming. This article reviews the developmental research related to relational aggression and presents a systematic examination of nine published school-based prevention and intervention programs to prevent relational aggression. Programs reviewed target preschool through eighth-grade students. Strengths and limitations of each program are discussed. Recommendations are offered for future research to develop and validate school-based programming for relational aggression, and implications for school psychologists are discussed.
Physical aggression, defined as the intent to harm another through physical force or dominance, has been studied extensively for many years (Dodge, Coie, & Lynam, 2006). As a result, research has shown that physical aggressors have many social problem-solving and emotional regulation deficits (Crick & Dodge, 1994; Vasey, Dangleish, & Silverman, 2003), extensive peer relationship difficulties, and a number of adjustment difficulties as they get older (Leff, Power, Manz, Costigan, & Nabors, 2001). In the past two decades, researchers have developed a broadened conceptualization of aggression to include nonphysical behaviors that damage one’s status or social standing within the peer group (Crick, Ostrov, & Kawabata, 2007). Crick and Grotpeter (1995) coined the term relational aggression to describe these behaviors. Relational aggression is defined as nonphysical aggression in which one manipulates or harms another’s social standing or reputation. Relationally aggressive behaviors can be direct, such as when peers indicate that they will no longer be a person’s friend if that person does not do what they say, or indirect, such as when peers spread rumors behind a person’s back so that others will not like that person. Relational aggression is a serious concern for youth, as it has been associated with a range of deficits including social problem-solving and emotion regulation deficits, peer relationship difficulties, internalizing problems such as anxiety, depression, and loneliness, and is predictive of future psychosocial maladjustment (Card, Stucky, Sawalani, & Little, 2008; Crick et al., 2007). Despite this, relatively little attention has been paid to how developmental research on relational aggression can inform the development and validation of evidence-based prevention and intervention programming (Leff, Gullan et al., 2009).
The goals of the current article are to: (a) briefly review the developmental research on relational aggression and associated features that may inform development of prevention/intervention programs, (b) conduct a thorough review of the strengths and limitations of selected published intervention programs that address relational aggression, and (c) discuss recommendations and implications for future research and school psychology practice.
Correlates of Relationally Aggressive Behaviors
Adjustment Difficulties
Studies have demonstrated that relational aggression is moderately stable during both early childhood (Crick, Ostrov, Burr, Cullerton-Sen, Jansen-Yeh, & Ralston, 2006; Ostrov, Woods, Jansen, Casas, & Crick, 2004) and middle childhood (Zimmer-Gembeck, Geiger, & Crick, 2005); further, it significantly predicts peer rejection one year later, even after accounting for youths’ levels of physical aggression (Crick, 1996). Relational aggression is also related to a number of other difficulties including behavioral challenges, internalizing problems, academic deficits, teacher–student conflicts, lack of school engagement (Crick & Grotpeter, 1995; Murray-Close, Ostrov, & Crick, 2007; Prinstein, Boergers, & Vernberg, 2001), and later mental health disorders among girls such as mood and eating disorders (Werner & Crick, 1999). In addition, relational aggression is highly associated with physical aggression for some children (Card et al., 2008; Cillessen & Mayeux, 2004), which may compound the complexity of the problems. In sum, programs addressing relational aggression may need to use strategies that have been effective for the reduction of physical aggression and may benefit by addressing the extensive social and emotional difficulties related to relational aggression.
Social Processing and Emotional Arousal Deficits
A combination of biological vulnerabilities, sociocultural contexts, and life experiences interact with social and emotional processing skills to determine children’s responses to social situations (Dodge & Petit, 2003). Along with emotional processing, youth progress through a series of social information processing (SIP) steps that likely mediate between biological/psychological risk factors and aggressive behaviors. These steps include (1) encoding environmental cues, (2) interpreting cues, (3) selecting social goals, (4) generating possible alternative responses, and (5) making decisions to enact a behavioral response. Many studies have shown that youth who are physically aggressive demonstrate deficits in these SIP steps (Vasey et al., 2003). Other studies suggest that relationally aggressive youth demonstrate some of the same social processing deficits as physical aggressors. For instance, relational aggressors tend to attribute hostile intentions to others in relationally provocative social situations (SIP Step 2; Crick, Grotpeter, & Bigbee, 2002; Leff, Kupersmidt, & Power, 2003), and evaluate aggressive solutions more favorably (SIP Step 4; Crick & Werner, 1998) as compared to non-relationally aggressive children. Although more research is needed to determine whether relational aggressors have additional processing deficits, a number of researchers advocate for the use of social cognitive reframing interventions with relationally aggressive youth, similar to interventions commonly used with physical aggressors (Fraser et al., 2005; Leff, Gullan et al., 2009).
Perceived Popularity
Despite the wide range of social processing, peer relationship, and social adjustment difficulties experienced by relational aggressors, a number of these youth may also have considerable social influence within the peer group (Cillessen & Mayeux, 2004; Puckett, Aikins, & Cillessen, 2008). Although this initially seems counterintuitive, it appears that some relationally aggressive youth reap benefits through being adept at social manipulation. For instance, studies have found that although relationally aggressive youth are often disliked by their peers, some are also viewed as quite influential and popular (Cillessen & Mayeux, 2004; Prinstein & Cillessen, 2003). Further, the association between high social status and relationally aggressive behavior has been found for both boys and girls (Hoff, Reese-Weber, Schneider, & Stagg, 2009; Puckett et al., 2008). To deal with popularity issues, programming for relationally aggressive youth may need to include opportunities for relational aggressors to demonstrate their influence and leadership in a prosocial manner.
Differences in the Expression of Relational Aggression
Developmental Differences
Although relational aggression has largely been depicted in the popular press as a complicated and manipulative process used during adolescence, research demonstrates that relatively simple forms of relational aggression can be detected among children as young as 3 years of age (Ostrov et al., 2004). Relationally aggressive actions for preschoolers tend to be direct, with the victim usually present. For instance, children may put their hands over their ears as a way to indicate that they are ignoring a peer. Thus, relational aggression is more easily observed at this young age than it is among older children. Some research suggests that preschoolers’ relationally aggressive actions may be influenced by their early experiences within the home with older siblings and parents (Stauffacher & De-Hart, 2006). The implication is that prevention/intervention programs for young children may need to address direct forms of relational aggression and include parent programming whenever possible.
As children progress through elementary and early middle school, relationally aggressive actions can be direct or indirect and begin to manifest in a more complex manner. For example, exclusionary behaviors are often seen at this age. Also, the importance of the peer group and specifically of peer reputations takes on increasing salience during these years (Leff et al., 2003), making it more difficult to change relationally aggressive behaviors. Thus, not only is it important to help relationally aggressive perpetrators and/or victims learn new and more adaptive behavioral repertoires, but it may also be important to change the peer group culture that supports and maintains relational aggression (Leff et al., 2003). Similarly, programs for school-age youth are likely to be more effective if they take into account the broader social-ecological contexts in which aggression occurs, such as the unstructured school settings of the playground, lunchroom, hallways, and bus stops (Leff, Costigan, & Power, 2004).
As youngsters approach adolescence, relationally aggressive actions continue to become more complex and subtle. In addition, with the increased use of electronic media (e.g., e-mail, text messaging, Facebook) during adolescence, youth can also use this as a medium for relationally aggressive behaviors (Williams & Guerra, 2007). Researchers have also started to study relational aggression within the context of romantic relationships (Ellis, Crooks, & Wolf, 2009). For instance, Ellis and colleagues found that girls who were relationally aggressive in both their peer and dating relationships were much more likely to display delinquent behaviors than girls who display relational aggression in only the peer context. Thus, prevention and intervention programs designed for adolescents may need to include ways in which to address the effect of electronic media on the expression of aggression and to potentially address the romantic relationship as another important aspect of the social-ecological context.
Gender Differences
Over the past few decades there has been a growing recognition that girls and boys express their anger differently, with boys tending to display aggression in a physical manner related to dominance, in contrast to girls tending to use relationally manipulative behaviors (Björkqvist, Osterman, & Kaukiainen, 1992; Crick et al., 2007; Galen & Underwood, 1997). Given research showing that girls place more importance on close dyadic peer relationships than boys (Crick & Grotpeter, 1995), it is not surprising that girls express much of their aggression in a form that involves damage to peer relationships. As a result, girls are more distressed by relational conflicts than are boys (Crick, 1995; Crick, Grotpeter, & Bigbee, 2002) and experience a substantially higher physiological response (systolic blood pressure) to relational situations as compared to boys (Murray-Close & Crick, 2007).
While recognizing the salience and meaning of relationally aggressive behaviors among girls, it has become clear that relational aggression also affects boys and the broader context in which it occurs. For example, research indicates that relational aggression occurs quite frequently among boys (Card et al., 2008; Swearer, 2008), and boys who display relational aggression (as opposed to physical aggression) experience greater psychosocial maladjustment than boys who display gender-normative expressions of aggression (Crick, 1997). With regard to the effect of relational aggression on the broader context, research has found that students feel less safe in schools in which relational aggression is frequent (Kuppens, Grietens, Onghena, Michiels, & Subramanian, 2008). Given that relational aggression occurs quite often among both boys and girls and has an effect on the school environment, it appears that school-based programming for relational aggression is needed for both girls and boys.
Taken together, the research focusing on relational aggression underscores the importance of targeting this form of aggression when designing programs for aggressive youth (Leff, Angelucci, Goldstein, Cardaciotto, Paskewich, & Grossman, 2007). The scientific literature has been relatively slow to inform relational aggression interventions; however, several researchers have begun to use this burgeoning literature base to update ongoing interventions that did not include relational aggression in prior trials, as well as to develop new programs with a concentrated focus on relational aggression.
Program Reviews
To identify programs to reduce relationally aggressive behaviors and related features, we conducted a systematic keyword search of peer-reviewed articles across four databases: Academic Search Complete, Educational Research Information Clearinghouse (ERIC), MEDLINE, and PsycINFO. The first keyword search used the terms relational aggression, social aggression, and/or indirect aggression crossed with prevention/intervention and/or program.1 The second and third searches differed only by substituting the terms bullying and victimization for the word aggression. These searches identified 138 separate articles. Abstracts and keyword terms from each of these articles were then reviewed by two research team members to determine whether an intervention or prevention program was implemented and/or evaluated within the article. This search resulted in 22 articles describing 21 separate prevention/intervention programs. The two reviewers agreed about the inclusion of relational aggression programming in almost all articles (98.6%); they discussed and resolved any discrepancies.
Following this, two members of the research team screened each of the 21 abstracts and associated articles to exclude those not conducted within a school or after school setting (n = 3) or not available in English (n = 2). The remaining 16 programs were reviewed for inclusion using a slightly adapted version of the new standards to establish efficacy defined by the Society for Prevention Research (2006).2 The following were the adapted criteria used: (a) intervention description and outcomes—a clear and detailed program description with manual, target population, and objective outcome measures; (b) clarity of causal inferences—an experimental design (i.e., random assignment into treatment groups) or a well-controlled quasi-experimental study (e.g., the use of control groups); and (c) generalizability of findings—a definition of study sample and method of participant selection. All decisions for inclusion/exclusion were reviewed by at least two of the authors and if disagreement occurred, a third person from the research team was asked to review the article before making a final decision. In this manner, the authors agreed that 8 of the 16 programs met the above criteria. A ninth program, which was not part of the original literature search (likely because the use of the term social exclusion instead of relational, social, or indirect aggression), was also included for review as it was cited in several of the included programs and met the criteria previously discussed. Thus, nine programs are reviewed and are presented by developmental age of the program’s targeted population. Table 1 provides descriptive features of each program and Table 2 summarizes research findings from the primary research studies related to each program. Cohen’s d (Cohen, 1988) is presented for any program in which this information was provided.3
Table 1.
Descriptive Features of Programs
Title of Program | Mode of Operation | Theorized Population for Program | Implementation | Materials/Instructional Strategies | Targeted Behaviors/Skills |
---|---|---|---|---|---|
Early Childhood Friendship Project (Ostrov et al., 2009) | Classroom based Universal prevention |
3–5 years old SES & ethnically diverse Boys & girls |
6-week curriculum (3 ten-min activities & 3 one-hour reinforcement sessions per week) Postcollege or master’s level clinicians Teachers meet regularly with program facilitators Program facilitators meet for weekly supervision |
Puppet show Participatory & rehearsal activities Manual available |
Physical aggression/victimization Relational aggression/victimization Prosocial behavior |
You Can’t Say You Can’t Play (Harrist & Bradley, 2003) | Classroom based Universal prevention |
Kindergarten-aged children SES & ethnically diverse Boys & girls |
8–10 sessions over 6–8 weeks Graduate-level research assistants read classroom stories Researchers met with teachers prior to implementation & during the intervention |
Classroom rule “you can’t say you can’t play”, role-play & group discussions 10-min story (on social rejection) Paley book publically available Manual available |
Discourage socially exclusive behaviors Focus is on changing the peer group/peer climate |
ICPS (Boyle & Hassett-Walker, 2008) | Classroom based Universal prevention |
K–Primary SES & ethnically diverse Boys & girls |
83 lessons, 20 min. per lesson, 3–5 times per week Teacher manual & training, with boosters |
Formal classroom lessons Lessons for integrating into daily activities, role-playing, & dialoging Manual available |
Increase prosocial behavior & problem-solving skills Reduce aggression |
WITS (Leadbeater et al., 2003) | School & community-wide Universal prevention |
K–3rd grade U.S. & Canadian SES & ethnically diverse Boys & girls |
Read 3 or more WITS books, implement 3 or more WITS classroom activities Teachers, principals, playground assistants, librarians Adults receive 2-hr in-service training |
Literature & activities to reinforce your WITS, police liaison visit class Activity books, siblings pamphlets to take home WITS posters Manual available |
Reduction/prevention of victimization Promote positive conflict resolution strategies Increase social responsibility school & community-wide |
MC (Fraser et al., 2004; Fraser et al., 2005) | School-based curriculum Indicated intervention with at-risk & prosocial youth |
3rd–6th grade SES & ethnically diverse Boys & girls |
Weekly lessons 22 sessions, typically 18–28 classroom hr Program specialists with training in educational counseling, psychology, or social work 6 hr of training on the manual |
Classroom lessons to address each social information processing step Role-play Practicing skills manual available |
Address SIP deficits Decrease peer rejection Increase social competence & prosocial interactions (e.g., emotion regulation & correctly encoding social cues) |
F2F (Leff et al., 2009) | School based Indicated intervention with relationally aggressive girls & prosocial girls Small group |
3rd–5th grade Urban elementary schools African American Girls |
20-session therapy group 8 classroom sessions 40 min, 2 times per week Graduate students in psychology partner with teachers to implement intervention (3–4 times for 30 min) Graduate students trained by a licensed psychologist (4-hr per week for 2 months) |
Culturally relevant cartoons & videos to depict SIP skills Role-play Discussions Participating students then facilitate/lead the intervention in their classrooms Manual & integrity monitoring sheets available |
Identifying physiological signs of arousal & applying SIP skills to a range of social situations Decrease relational & physical aggression Improve problem-solving Increase prosocial behaviors Gain leadership skills |
Second Step (Van Schoiack-Edstrom et al., 2002) | Classroom based Universal program |
6th–8th graders U.S. & Canadian Boys & girls |
30-class lessons 35 min each 16–20 weeks Mostly teachers implement, some schools have psychologists provide lessons Teachers receive a 1-day training & implement |
Scripted lessons with clear objectives Videotaped vignettes Role-play Homework assignments Manual available |
Program emphasizes three core competencies (empathy, social problem solving, impulse control) while teaching new behavioral skills Establish an anti-aggression school climate Reduce aggressive & impulsive behaviors |
SAPP (Cappella & Weinstein, 2006) | Small single-gender groups Universal prevention (for girls) |
5th grade Ethnically diverse Girls |
One 40-min session per week 10 weeks Graduate students in clinical psychology & counselors implemented the program Interventionists had a 3-hr training & supervision |
Reading & discussions of hypothetical scenarios & role-plays Games Practicing & feedback Manual available |
Learn strategies for handling socially aggressive conflicts Reduce girls’ use of social aggression Increase empathy Increase social problem-solving abilities Increase prosocial behaviors |
Sisters of Nia (Belgrave et al., 2004) | Small single-gender group intervention Universal prevention (for girls) |
Early adolescent (middle school aged) African American Girls |
15 sessions (1.5-hr sessions) with 30 1-hr tutoring sessions African American female intervention staff served as role models & ran the program Interventionists received a 1-day training |
Unity circle for team building African rituals celebrating ancestors Reading African proverbs Educational & cultural presentations Group discussions that emphasized positive ways of interacting |
Enhance ethnic identities Promote an androgynous gender role Strengthen mutually positive relationships & decrease relational aggression |
Note. ICPS = I Can Problem Solve; WITS = Walk away, Ignore, Talk, Seek help; MC = Making Choices: Social Problem-Solving Skills for Children; SES = socioeconomic status; SIP = social information processing; F2F = Friend to Friend; SAPP = Social Aggression Prevention Program.
Table 2.
Summary of Research Findings
Title of Program Examined | Sample and Design | Outcome Measures | Primary Results |
---|---|---|---|
Early Childhood Friendship Project (Ostrov et al., 2009) | 18 classrooms (M = 49.64 months) Treatment vs. no-treatment control Random assignment Evaluation of program at the classroom level |
Teacher report of prosocial behavior Adapted version of the Preschool Social Behavior Scale— Teacher Form (Crick, Casas, & Mosher, 1997) Observations of aggression & victimization Early Childhood Observation System (Ostrov & Keating, 2004) revised to shorter intervals and more children per class per day |
Reduced physical aggression/victimization Reduced relational aggression/victimization Increased prosocial behaviors |
You Can’t Say You Can’t Play (Harrist & Bradley, 2003) | 144 boys & girls Full-day kindergarten Schools were assigned into intervention (n = 6) or control (n = 4) Partial randomized group design (takes into account logistical constraints) Control teachers were told they were participating in a study of social exclusion |
Sociometric interviews of liked least, liked most, & how much they like to play with each child Assigned sociometric status according to Coie & Dodge (1983) criteria Teacher ratings of isolation and nonacceptance Preschool Socioaffective Profile (LaFrenière, Dumas, Capuano, & Dubeau, 1992) Teacher’s Checklist of Peer Relationships (Dodge, 1986) Child self-report self-perceived acceptance and social dissatisfaction via face-to-face interviews with a research assistant Self-Perceived Acceptance Profile (Harter & Pike, 1984) Loneliness & Social Dissatisfaction Scale (Asher, Hymel, & Renshaw, 1984) Observations of social behaviors assessed by research team: time alone, entry attempts, & entry accepted. Observers blind to group condition |
No changes in observations of social rejection No changes in teacher report of social exclusion Students in intervention classrooms reported liking each other more (through peer nominations) Students in intervention classrooms reported more dissatisfaction with peer relationships |
ICPS (Boyle & Hassett-Walker, 2008) | 226 K or 1st (6–8 years) 80% Hispanic, urban setting Matched schools based on demographics Random assignment to instruction or control groups Some schools refused to participate as controls & were dropped |
Teacher report of students’ behaviors Preschool Social Behavior Scale—Teacher Form (Crick et al., 1997) Hahnemann Behavior Rating Scale for aggression, impulsivity, passivity, & prosocial behavior (Shure, 2002) |
Increased prosocial behavior Reduction in overt & relational aggression, with effects larger for overt Modest additive effects seen for students receiving 1 year of the program versus 2 years |
WITS (Leadbeater et al., 2003) | 432 children (M = 6 years), followed for 3 years 12 treatment matched with 5 control classrooms No random assignment Schools had a variety of programs being run in addition to WITS |
Self-report of peer relational & physical victimization, prosocial acts Social Experience Questionnaire (Crick & Grotpeter, 1996) Teacher reports of social competence, emotional, & behavioral problems Early School Behavior Rating Scale (Caldwell & Pianta, 1991) |
Moderate reduction in classroom levels of relational & physical victimization (self-report) Increases in classroom levels of social competence (teacher report) Compared program effects across low-& high-poverty schools |
MC (Fraser et al., 2004) | 115 children (M = 8.9 years) 9 sites were randomly assigned to wait list control & intervention (i.e., MC 3 Strong Families) After school sites & nonacademic classes, selected based on interest |
Teacher report of prosocial behavior, emotional regulation, social contact, cognitive concentration, relational aggression, & authority acceptance Carolina Child Checklist—Teacher Form (Macgowan, Nash, & Fraser, 2002) |
Program increased prosocial behaviors, emotional regulation skills, & social interactions Improvements in problem classroom behaviors Decreases in relational aggression |
MC (Fraser et al., 2005) | 548 children (M = 8.93) 3rd-graders followed for 3 years Compared the MC program & the MC Plus with a control group Teachers were also encouraged to use the Good Behavior Game (Evertson, Emmer, Sanford, & Clements, 1983) No random assignment |
Teacher reports of child classroom behaviors, overt aggression, & relational aggression Carolina Child Checklist—Teacher Form (Macgowan et al., 2002) including a modified version of the relational victimization scale of the SEQ (Crick & Grotpeter, 1996) Research staff administered a measure of SIP skills to obtain children’s responses Skill Level Activity, adapted from Dodge’s Home Interview for attributional bias (Dodge, 1980) |
Both intervention groups showed decreases in social aggression & hostile attribution biases Both intervention groups showed increases in prosocial behaviors & in some SIP skills MC Plus had larger effect sizes than MC |
F2F (Leff et al., 2009) | 32, 3rd–5th-grade girls 98% African American, from an urban setting 11 classrooms randomly assigned into intervention & control |
Unlimited peer nominations of relational & physical aggressors used to identify girls for inclusion in study Items based on Crick & Grotpeter (1995) peer nomination survey Teacher report of relational, physical, & peer likeability Children’s Social Behavior Questionnaire (Crick, 1996) Self-report of hostile attribution biases, loneliness, depression Cartoon-based version of an HAB measure (Crick, 1995; Leff et al., 2006) Asher and Wheeler Loneliness Scale (Asher & Wheeler, 1985) Children’s Depression Inventory (Kovacs, 1985) |
Improved peer likeability Decreased relational & physical aggression Reduced hostile attributions & feelings of loneliness |
Second Step (Van Schoiack-Edstrom et al., 2002) | 5 schools from Canada & U.S. 714 middle & junior high students Random assignment into intervention & control among Canadian sample only 2 intervention groups examined (see text for details) Original 1-year program was expanded into a 3-year program |
Self-report of endorsement of aggression & perceived social difficulty Endorsement of aggression scale (Slaby & Guerra, 1988) with 5 items added for relational aggression adapted from Crick & Bigbee, 1998, and Crick & Grotpeter, 1995, written as belief statement Perceived Social Difficulty Scale (Perry, Perry, & Rasmussen, 1986) |
Attitudinal change towards physical aggression, social exclusion, & verbal derogation |
SAPP (Cappella & Weinstein, 2006) | 134 girls 5th grade & 4th/5 th & 5th/6th combination classrooms (M = 10.5 years) Random assignment into intervention & control Ethnically diverse sample Control group receive a comparison intervention (a small-group reading club, RC) |
Peer nominations of relational & overt aggression (renamed for this program as social & physical/verbal aggression, respectively) Children’s Social Behavior Scale—peer report (Crick, 1995) Teacher ratings of empathy, and relational and overt aggression (renamed social & physical/verbal aggression, respectively) Teacher ratings of empathic behavior scale—short form (see Cappella & Weinstein) CSBS (Crick, 1995, 1996) adapted to include 2 items for socially aggressive behaviors Individual interviews of participants social aggression & problem-solving skills Social aggression problem-solving scenarios designed for this study based on Galen & Underwood (1997) & Lochman & Dodge (1994) measures |
No significant differences reported perpetration of social aggression Increased prosocial behaviors & increased abilities to resolve difficult social situations for children with social problems at baseline for peer-or teacher- |
Sisters of Nia (Belgrave et al., 2004) | 59 African American girls Schools randomly assigned to intervention & control Intervention received 30 weeks of tutoring & 15 weeks of Nia; Control received 30 weeks of tutoring |
Self-report of perceptions of gender roles, African American identity, & relational violence Children’s Sex Role Inventory (Boldizar, 1991) Multi-Construct African American Questionnaire (Smith & Brookins, 1997) Frequency of Violent Behaviors scale (Farrell, Meyer, Kung, & Sullivan, 2001) |
Positive changes in perceptions ethnic identity Decrease in relational aggression for intervention group |
Note. ICPS = I Can Problem Solve; K = kindergarten; WITS = Walk away, Ignore, Talk, Seek help; MC = Making Choices: Social Problem-Solving Skills for Children; SEQ = Social Experience Questionnaire; SIP = social information processing; F2F = Friend to Friend; SAPP = Social Aggression Prevention Program; RC = Reading Club; CSBS = Children’s Social Behavior Scale.
Early Childhood Friendship Project
The Early Childhood Friendship Project (Ostrov et al., 2009) is a classroom-based preventive intervention for young children, designed to reduce physical aggression, relational aggression, and peer victimization while increasing prosocial behaviors. A preliminary trial of the program was implemented in 18 classrooms of socioeconomically and ethnically diverse samples across both urban and suburban contexts (Ostrov et al., 2009). Classrooms were randomly assigned, resulting in nine intervention and nine control classrooms. Systematic observations of relational and physical aggression/victimization were conducted at the classroom level by randomly sampling students in the classroom. The observational tool used by the research staff was adapted to assess classroom levels of aggression and victimization, pilot tested, and demonstrated adequate inter-rater reliability in the current study (intraclass correlations >.70 for all variables). Prosocial behavior was assessed by adapting a teacher measure that had previously demonstrated reliability and validity (all alpha values >.79). The adapted teacher measure demonstrated adequate internal consistency and parallel-form reliability for the current sample. (See Table 2 for lists of outcome measures.) Moderate effect sizes were obtained for observations of physical aggression (d = −0.54) and teacher reports of prosocial behavior (d = 0.54); large effects were found for observations of relational aggression (d = −0.88) and physical victimization (d = −0.91); and small effects were found for observations of relational victimization (d = −0.23; Ostrov et al., 2009). Key content and process implementation fidelity were rated as being extremely strong based on observations, and teachers evaluated the program very positively on an acceptability measure.
Preliminary results suggest that the intervention may be promising for reducing physical aggression/victimization and relational aggression. Strengths of the program include the following: the outcome measures have demonstrated adequate reliability and validity across several prior samples and studies; both content and process implementation were monitored and found to be extremely high; and teacher acceptability for the program was rated favorably. A limitation is that although observers were blind to intervention status, observations of program implementation appear to have been conducted by program implementers who were not blind to study hypotheses. Another limitation is that the study was underpowered and was only able to be evaluated at the classroom level. Thus, the program should be replicated with a larger randomized trial in which children nested within classrooms can be taken into account and effects can be studied both at an individual and classroom level. Further, given the developmental research suggesting the importance of early parenting and home experiences in relational aggression at this age (Stauffacher & DeHart, 2006), having a parent/home component would possibly strengthen this program. Given that four of five outcome indices were based on an observational methodology, it would be important to examine the program’s effectiveness across a broader multimethod multi-informant assessment battery in the future.
You Can’t Say You Can’t Play
Based on a children’s book, You Can’t Say You Can’t Play (Paley, 1992), the emphasis of this program is on the peer group and classroom climate in which rules can be made to teach young children to avoid using social exclusion. A preliminary study of the program (Harrist & Bradley, 2003) used a partial randomized group design with classes randomly assigned to intervention or control groups.4 One-hundred and forty-four kindergarteners from 10 classes (6 intervention, 4 control) participated in the study. Fifty-seven percent were European American, 34% Mexican American, 5% Asian American, and 4% African American, representing a range of socioeconomic status classes from the poverty level to upper middle class. Fidelity was examined by the research team through informal meetings with the teachers once a week. Outcomes were assessed using sociometric interviews, teacher ratings, child self-reports, and observations of social behaviors; all indices demonstrated reliability and validity in prior studies and adequate reliability within the current study (teacher report all alpha values >.86; self-report all alpha values >.72; observation Cohen’s kappa ranged from .76 to .90). The study produced mixed results. There were no changes in rates of observed social rejection or teacher reports of social exclusion. Children in the intervention classrooms reported liking each other more (through sociometric interviews) following the program (d = 0.90), yet also reported more dissatisfaction with their peer relationships compared to control classes (d = 1.00).
Given research indicating the importance of the school climate and that witnessing higher levels of relational aggression can affect feelings of safety (Kuppens et al., 2008), this program has a clear strength in that it addresses the classroom social climate. Further, the You Can’t Say You Can’t Play (Paley, 1992) book is inexpensive and easy to purchase, and the program is generally easy to implement. A major limitation is that the study did not demonstrate the program’s effectiveness in decreasing social exclusion. Another limitation is that implementation fidelity was not evaluated in an objective and systematic manner, making it challenging to determine whether the unexpected or lack of findings related to social exclusion indicate that the program needs to be modified or that the program was not implemented as intended (see Leff, Hoffman, & Gullan, 2009). Therefore, this program requires a considerably more systematic trial in the future to ensure that it is evaluated thoroughly. The program also may be strengthened by adding a parent or family component. Finally, it may be helpful to combine this program with other developmentally appropriate activities or programs that address other aspects of relational aggression in addition to social exclusion.
I Can Problem Solve (ICPS)
ICPS (Boyle & Hassett-Walker, 2008) is a curriculum-based, universal prevention program designed for kindergarten and elementary-aged children. It is intended to reduce aggressive behavior and increase prosocial behavior (Shure, 2001). Boyle and Hassett-Walker (2008) examined the effects of ICPS on relational and overt aggression in an independent, randomized control trial of ICPS in one school district among urban predominantly Hispanic kindergarten and first-grade children (n = 226). The study compared three randomly assigned groups: kindergarten classrooms that received 2 years of the program (n = 96), kindergarten and first-grade classrooms that received only 1 year of the program (n = 106), and kindergarten and first-grade classrooms that served as the control group (n = 24). Teachers received initial training, an ICPS manual, and one booster training a year. The program was evaluated for implementation fidelity through observations twice per year and interviews with teachers implementing the program. Outcomes were assessed using teacher-report measures that have demonstrated prior reliability and validity and most illustrated adequate reliability across the current 2-year study (alpha values ranged from .64 to .90). The results showed an increase in prosocial behavior in intervention groups versus the control group, and these effects were stronger among children who received 2 years of the program (d = 0.55). Moreover, children who received 2 years of the program exhibited less relational aggression (d = 0.35) and an improvement in overt aggression (d = 0.40) as compared to the other two groups.
Strengths of the ICPS Program include its long history of use within varied schools and school districts, especially for preschoolers and kindergarteners. In addition, the program is manualized and trainings are available for purchase (http://www.researchpress.com/product/item/4628/). Although the program does not appear to have problem-solving components specific to relational aggression, results suggest that the general problem-solving skill set taught by the program may be reasonably effective in reducing relational aggression. Although the specific study reviewed did not focus on parent implementation of the program, the broader ICPS Program has been conducted with parents/caregivers (Shure, 2001). Limitations of the overall program are that it is extremely long (83 sessions) and intensive, as it is recommended to be implemented by teachers several times per week. Thus, the program requires strong buy-in and committed teachers and school communities, as well as the ability to provide supports to teachers to monitor and guide implementation. In the present evaluation (Boyle & Hassett-Walker, 2008), there were fewer youth in the control condition, as two school principals who previously agreed to participate regardless of random assignment results withdrew their schools from the study after finding out that they were in the control condition. Another limitation is that those evaluating implementation fidelity in the current study were not blind to condition. A further limitation of the study was that it was conducted in only one urban school district with a predominately Hispanic population of kindergarteners and first-graders, and that the two experimental groups were larger than the control group. It is suggested that the program be systematically evaluated using equivalent groups across more school districts and diverse samples.
Walk Away, Ignore, Talk, Seek Help (WITS)
WITS (Leadbeater, Hoglund, & Woods, 2003) is a comprehensive, multisetting program (i.e., schools, families, and communities) designed to reduce victimization, improve social and emotional competence, and increase social responsibility among children in kindergarten to third grade. Seventeen schools from urban, western Canada were used to assess program feasibility and effectiveness through a quasi-experimental matched-control design (Lead-beater et al., 2003; Leadbeater & Hoglund, 2006). Schools that previously demonstrated high fidelity with program implementation5 were matched to control schools with no prior involvement with the intervention. As such, twelve intervention schools were matched with five schools in the same district who agreed to be control schools. The sample included 432 predominantly white first-grade students (M = 6 years old) who were followed for 3 years. The authors used teacher-report measures that have demonstrated reliability and validity in prior studies and samples and had adequate reliability within the current study (all alpha values >.84). Self-report measures also showed adequate reliability in the current study (all alpha values >.72). As compared to control schools, intervention schools showed reductions in relational victimization for both low-poverty (d = 0.46) and high-poverty (d = 0.67) schools. Intervention schools also showed decreased physical victimization for both low-poverty (d = 0.29) and high-poverty (d = 0.81) schools. In addition, teachers in the intervention schools reported greater increases in children’s social competence as compared to control schools across both high- and low-poverty schools (d = 0.70).
A strength of WITS is that it integrates diverse school and community members into the program implementation. Also, it is akin to implementing a school-wide framework and policy that guides students to adopt a new philosophy or outlook on problem solving, thereby being easy to implement and relatively inexpensive. WITS is one of the few programs that focuses primarily on relational victimization as opposed to perpetration. Further, WITS includes a family and sibling component, which is important given the research demonstrating the importance of sibling and parent relationships and relational aggression among young children. The program materials are free and publically available (http://web.uvic.ca/wits/), making this an accessible program, and the inclusion of a literacy component is commendable given increasing focus on literacy promotion and academic achievement within schools (Morrison, Bachman, & Connor, 2005). A limitation of the program includes the significant methodological concerns of the preliminary study that made it difficult to have confidence in findings (e.g., unbalanced number of intervention and control schools; comparing high-fidelity schools to schools with no prior exposure to the program). Thus, despite being extremely innovative and publically available, WITS requires a more systematic trial to better understand its potential effect.
Making Choices: Social Problem Skills for Children (MC)
The MC program (Fraser, Day, Galinsky, Hodges, & Smokowski, 2004; Fraser et al., 2005) is a school-based group intervention addressing SIP deficits often found in aggressive children. Two studies have examined the effectiveness of MC on relationally aggressive behaviors. The first study (Fraser et al., 2004) implemented the MC Program combined with the Strengthening Families Program (Fraser et al., 2004) across nine settings (seven after school and two school settings) with children ranging in age from 6 to 12 years (Mean = 8.8, SD = 1.4). Mixed groups of children with and without conduct problems and their parents were randomized into intervention (n = 62) and control conditions (n = 53); however, outcomes were only assessed on those children with conduct problems versus control children. Teacher-report measures with adequate internal consistency (all alpha values >.82) and test–retest reliability were used. Program specialists (staff members at each site with education, school psychology, and social work backgrounds) completed treatment integrity forms to track program implementation. Medium to large effect sizes were found when comparing intervention to control condition across prosocial behavior (d = 0.74), emotional regulation (d = 0.50), social contact (d = 0.50), cognitive concentration (d = 0.70), and relational aggression (d = 0.70) variables.
The second study (Fraser et al., 2005) examined the effectiveness of MC in reducing relationally and socially aggressive behaviors. A total of 548 third-graders from two schools participated in a cohort design with a comparison group intervention. Youth were in the comparison group (n = 177), the group receiving the MC program (n = 175), or the group receiving the MC Plus (MC+) program (n = 198). The MC+ program included sending home five newsletters, five evening informational sessions for parents, and an additional classroom-based component in which teachers were encouraged to use the Good Behavior Game (Evertson, Emmer, Sanford, & Clements, 1983). Program implementation was monitored by recording the number of classroom sessions completed, the length of each session, and individual student attendance, as well as teacher logs and observations of the classroom (Fraser et al., 2005). Teacher-and research staff-report measures with demonstrated prior reliability and validity were used. Teacher-report measures demonstrated adequate reliability in the current study (all alpha values >.84). The research staff-report measure of children’s SIP responses had adequate reliability in the current study (alpha values >.76), yet one scale had low reliability (alpha = .52). Medium to large effect sizes were found when comparing both intervention groups to the comparison groups across prosocial behaviors (dMC = 0.46; dMC+ = 0.56) and encoding of social cues (dMC = 0.82; dMC+ = 0.77). Participants in the MC+ group also demonstrated moderate effect sizes when compared to the comparison group related to social aggression (dMC+ = −0.48) and several SIP steps including hostile attribution biases (dMC+ = −0.55), goal formation (dMC+ = 0.66), and response decision (dMC+ = 0.54). In general, the program was equally effective across diverse socioeconomic and racial/ethnic groups.
Strengths of the MC Program include its clear basis in strong psychological theory, including the SIP paradigm (Crick & Dodge, 1994), and its history of delivery in both school and after-school settings. Also, that the program includes both high-risk and prosocial peers in the group is important given that previous research has documented the possible deleterious effects of grouping aggressive youth together (Dishion, McCord, & Poulin, 1999). A limitation of the program evaluations is that the intervention conditions for both studies reviewed contain a combination of MC plus additional family or classroom components, making it difficult to understand which components may be more responsible for findings. Another limitation is that one of the measures used to measure hostile intent in the later study had low internal consistency (Fraser et al., 2005).
Friend to Friend (F2F)
The goal of the school-based F2F program (Leff, Gullan et al., 2009) is to decrease relationally and physically aggressive behaviors, improve problem-solving skills, and increase prosocial behaviors among urban third-through fifth-grade relationally aggressive girls. A preliminary study across two large inner-city elementary schools was undertaken using a random assignment procedure with control group (treatment as usual/referral to counselor; Leff, Gullan et al., 2009). Relational aggressors were identified using an unlimited peer nomination process, and girls from classrooms with two or more relational aggressors were randomly assigned to F2F or a treatment-as-usual control group. Reliable and valid teacher- and self-report measures were used and both content and process integrity were measured and found to demonstrate strong inter-rater agreement (98% for content and 96% for process) by an observation team that was blind to study hypotheses (Leff, Gullan et al., 2009). Results suggest that relationally aggressive girls who were randomized to F2F had extremely large improvements in teacher-reported peer likeability (d = 1.73), large decreases in relational aggression (d = 0.74), and moderate reductions in physical aggression (d = 0.43), hostile attributions (d = 0.61), and loneliness (d = 0.45) as compared to relationally aggressive girls in the control condition. In contrast, F2F did not find clinically meaningful reductions in depression (d = 0.22), although girls’ preprogram scores were not particularly elevated. Finally, the program was rated as quite acceptable by participating students and teachers.
One strength of F2F is that the program and materials were developed using a participatory action research framework by adapting best practice social cognitive violence prevention programs for urban physically aggressive youth through feedback from urban girls, parents, teachers, and community members (Leff et al., 2007). The program was judged to be culturally sensitive and highly acceptable by participants and their teachers. Further, F2F is theoretically based and capitalizes on the social influence that relationally aggressive youth may have (Cillessen & Mayeux, 2004; Prinstein & Cillessen, 2003) by including these girls as cofacilitators in the classroom component of the program. Also, F2F can be implemented with high levels of both content and procedural integrity (Leff, Gullan et al., 2009). Finally, groups are comprised of both high-risk relationally aggressive girls and prosocial role models, thereby reducing the potential deleterious effects of grouping aggressive youth (Dishion et al., 1999). A limitation is that this study was underpowered with an unequal number of youth in the treatment versus control conditions. Therefore, replication across more urban schools through a clinical trial is an important next step for the research. An additional limitation is that F2F was designed for urban, predominately African American girls, and thus the program is likely not generalizable beyond this target population.
Second Step
Second Step (Van Schoiack-Edstrom, Frey, & Beland, 2002) is a classroom-based program that has been found to improve children’s social competence and to decrease levels of physical aggression and antisocial behaviors (Frey, Nolen, Van Schoiack-Edstrom, & Hirschstein, 2005; Grossman et al., 1997). Although several specific lessons targeting relational aggression have been added to the curriculum, only one study has evaluated variables related to relational aggression (Van Schoiack-Edstrom et al., 2002). This 3-year study was conducted with 714 sixth- through eighth-grade students from five schools in the United States and Canada. The curriculum developer met regularly with teachers and observed sessions to ensure treatment fidelity. Outcomes were assessed using self-report measures with established reliability and validity; most measures demonstrated adequate reliability in the current study (alpha values ranged from .65 to .87). There were three groups of students compared in the study (two intervention groups and one control group). Youth in intervention Group 1 were in the lowest grade within their middle (sixth grade) or junior high (seventh grade) school and were taught Level 1 curriculum (e.g., 15 lessons focusing upon empathy, anger management, problem solving, and skill application). Youth in intervention Group 2 were in the second year of middle (seventh grade) or junior high (eighth grade) school and received the Level 2 curriculum (8 lessons related to empathy, anger management, problem solving, and skill application, which were intended to supplement the Level 1 curriculum).6 The overall rate of program completion across teachers was 99%, but Group 2 teachers taught significantly more lessons per week than teachers in Group 1 (M = 3.0 as compared to M = 2.33, p < .05). Students in intervention Group 1 remained relatively stable in their attitudes about social exclusion from pre- to postintervention, whereas students in the control group showed an increase in their endorsement of social exclusion (d = 0.37). Students in intervention Group 2 experienced improved attitudes related to social exclusion (d = 0.73), physical aggression (d = 0.50), and verbal derogation (d = 0.60) as compared to control youth whose attitudes remained stable.
Second Step is a well-known and widely used program that has previously demonstrated effects on social competence and physically aggressive behaviors. The program manuals, training sessions, and materials are available from Committee for Children (http://www.cfchildren.org/programs/ssp/overview/), leading Second Step to be accessible and recognized across many school districts. With regard to limitations, the current study measured outcomes related to relationally aggressive attitudes but not behaviors, and had several methodological issues that make it difficult to fully understand reported results. For example, random assignment was not achieved because two of the five teachers agreed to participate in the study only if they could implement the program. Although this highlights that the program may have been perceived very positively, it also creates a methodological concern. Methodological limitations also made it difficult to decipher what led to seemingly positive results: specific aspects of the program (Level 1 or Level 2 curriculum), intensity of the program (Level 2 was implemented more times per week despite being designed as a briefer module), or differences in grade levels between subgroups of participants. In addition, given that the Level 2 curriculum was implemented in an atypical manner, one may wonder if the combination of Level 1 and Level 2 curriculum would lead to an additive effect. Finally, the subscale measuring attitudes of social exclusion was only two items long, which makes findings preliminary. Future research examining the more typical implementation of the program (teaching the same students Level 1 curriculum and the next year Level 2 curriculum) will be important in understanding the full effect of the program on relational aggression and related outcomes. Further, given that the social exclusion scale had only two items and relatively low internal consistency, future research could include a broader and more rigorous assessment of relational aggression.
Social Aggression Prevention Program (SAPP)
SAPP (Cappella & Weinstein, 2006) was designed to reduce girls’ use of social aggression while increasing their empathy skills, social problem-solving abilities, and prosocial behaviors. The program is a 10-session universal prevention program that uses a small-group format with fifth-grade girls. To examine the effectiveness of SAPP, 134 fifth-grade girls from six schools within an ethnically diverse school district were randomly assigned into intervention (SAPP) and matched control group (Reading Club, RC). Program implementation quality was assessed by group leaders filling out a weekly summary and students reporting their perceptions of the program in the final session. Peer nominations and teacher- and self-report measures were used. Peer nomination instruments demonstrated adequate reliability (all alpha values >.91), as did the teacher-report measures (all alpha values >.88). The self-report measure was designed for use in the present study and demonstrated external validity and inter-rater reliability. There were no differences postintervention between girls in SAPP versus RC on peer or teacher reports of social aggression or prosocial behavior, but some moderate size effects related to social problem solving (d values range from 0.40 to 0.62). However, the authors conducted further analyses comparing girls who were high in social aggression at preintervention (0.5 SD above mean) who were in SAPP versus RC. Findings suggested that the program may have been more effective for the high-risk girls in SAPP versus the high-risk girls in RC across social problem solving (d = 0.90), prosocial behaviors (d = 0.62), and empathy (d = 0.60).
A strength of SAPP is that program effects were examined by randomly assigning girls to SAPP or an alternative active treatment group (RC). This controlled for nonspecific aspects of treatment and thereby provided a more stringent design than most of the programs reviewed in this article. Additional strengths are that the program is relatively brief (10 sessions), making it seem relatively feasible to implement, and that program implementers monitored treatment fidelity. Implementers found that students in the two treatment arms reported that both group conditions were appealing and both conditions had similar attendance rates and dosage of program content. Although the program may address mediating variables of nonphysical aggression (e.g., different indices of problem solving), a limitation of SAPP was that it did not decrease socially aggressive behavior. Increasing the intensity or incorporating additional activities may strengthen SAPP results in the future.
Sisters of Nia
Sisters of Nia (Belgrave, Reed, Plybon, Butler, Allison, & Davis, 2004) is a small-group program designed for all African American female adolescents (universal intervention) that addresses ethnic identity, gender roles, and appropriate social behaviors. The latter portion of the intervention focuses on the reduction of relational aggression through strengthening positive relationships and reducing negative interactions among peers. Program topics include healthy relationships, African culture, judging others, personal hygiene, critical consciousness, and leadership (Belgrave et al., 2004). The effectiveness of Sisters of Nia was assessed by randomly assigning schools to an intervention or a comparison group. Girls within the intervention condition received the Sisters of Nia program for 15 weeks and tutoring for 30 weeks, whereas the comparison group received only the tutoring for 30 weeks. Sessions were conducted immediately after school. Study participants included 59 African American girls (mean age of 11.82 years) who had baseline and postintervention data collected. Outcomes were assessed using self-report measures, most of which demonstrated adequate reliability (alpha values ranged from .65 to .82). Results indicated that intervention girls displayed higher ethnic identity scores (d = 0.57) and lower levels of relational aggression as compared to control girls (d = −0.57) when comparing pre- and postintervention scores.
Strengths of the Sisters of Nia program is its uniqueness, as it is the only program to incorporate a strong historical and ancestral approach to intervention. In addition, this program appears to help early adolescent minority youth think through impending challenges that will be important for their development (e.g., gender roles, leadership, education awareness). Although it is commendable that the program is gender and culturally sensitive, its utility may be somewhat limited to girls of African American descent. Further, given that participants are adolescents, it may be important to integrate additional training related to the development of romantic relationships. Finally, given that the current study used a relatively small sample drawn from one school, the program should be evaluated across more students and school settings to increase confidence in findings.
Directions for Future Research and Implications for Practice
Given that relationally aggressive behaviors are quite complex, programming to reduce relational aggression must take into account important developmental, cultural, gender, and contextual considerations. From the review of the nine programs, it is clear that although prevention and intervention programs for relational aggression are still in the early stages of development and evaluation, a number of these programs show quite a bit of promise and help direct the field in terms of future research and practice implications. The recommendations that follow are based on the review of these programs and the relevant literature.
Appreciating the Complexity of Relational Aggression
Programming for different developmental ages
Adjustment challenges and the experience and expression of relational aggression differ depending upon the age of the child. For instance, prevention programs for very young children need to have concrete and visual activities to help address the direct manifestations of relational aggression. Promising programs for preschoolers and kindergarteners addressed the direct nature of relational aggression through developmentally appropriate visual and concrete strategies that taught and reinforced positive peer interactions. For example, the Early Childhood Friendship Project (Ostrov et al., 2009) and You Can’t Say You Can’t Play (Harrist & Bradley, 2003) have very brief teaching sessions using puppets or the reading out loud of fairy tales, respectively. Further, given programs for slightly older children such as WITS and ICPS showed success in incorporating a parent generalization component, it would be especially beneficial for programs for young children to incorporate such components so that parents can reinforce learned strategies in the home and with siblings.
A number of the promising programs discussed have been designed and initially evaluated for late elementary and/or early middle school children. Given research indicating that the way youth approach and evaluate social cues affects their behavior (Crick & Dodge, 1994), most of these programs incorporate a social problem-solving model (e.g., MC; Fraser et al., 2004; Fraser et al., 2005; F2F, Leff et al., 2009; Second Step, Van Schoiack-Edstrom et al., 2002). Many of these same programs also recognize the importance of taking a systemic approach to prevention, incorporating a broader perspective to enhance the school and community context. Given recent research suggesting that adolescents use electronic media as an additional means of aggressing (William & Guerra, 2007) and that relational aggression may occur within the context of a romantic relationship (Ellis et al., 2009), future relational aggression intervention programs need to take these issues into account for adolescents.
Programming sensitive to gender and cultural contexts
Given developmental research indicating that girls and boys experience relational aggression differently (Murray-Close et al., 2007; Werner & Crick, 2004), perhaps some of the nonsignificant program effects are because of the lack of sensitivity to the qualitative gender differences. Therefore, it is recommended that more research be conducted to determine whether treatment approaches for relational aggression should be tailored to meet the specific needs of girls versus boys. Similarly, future research could focus on the whether the experience of relational aggression differs across various racial and cultural contexts and if interventions should subsequently be tailored accordingly. Two programs reviewed in this article were designed with a concerted effort in making the program culturally sensitive to the specific needs of the participants (F2F, Leff et al., 2007; Sisters of Nia, Belgrave et al., 2004). The field could benefit from learning more about the sufficiency and necessity of these efforts.
Programming addressing the complex social-ecological contexts
Relational aggressors often exhibit peer relationship challenges and psychosocial adjustment difficulties (Card et al., 2008), while at the same time being viewed as quite popular and influential within their peer group (Cillessen & Mayeux, 2004). Given this, intervention programs should integrate more opportunities for these youth to demonstrate their social influence and potential leadership in a prosocial manner. However, this was only emphasized in two programs, the F2F program (Leff et al., 2007) and the Sisters of Nia program (Belgrave et al., 2004).
Drawing from research documenting the relatively high correlations between relational and physical aggression and some of the similar maladjustment profiles experienced by these youth, it is recommended that future interventions for relational aggression build on empirically supported programs for physical aggression. Second Step (Grossman et al., 1997) and ICPS (Shure, 2001) are examples of nationally known and well-respected programs that have added curriculum and/or evaluations related to relational aggression. More research is needed on the cost-effectiveness and efficacy of this approach.
Given the extremely complex nature of relational aggression, it is also important that key individuals within the school and community are integrated into the intervention team. This approach is most clearly evident in the WITS program in which teachers, counselors, administrators, and school police officers serve as key implementation leaders (Lead-beater et al., 2003) and in the MC program in which teachers and/or parents are included in the combined treatment group (Fraser et al., 2004; Fraser et al., 2005). How well a program is integrated within a school and wider community may also be a crucial factor for the program’s sustainability and long-term success (Leff, Gullan et al., 2009).
Evaluating Programs Through Systematic Research
None of the nine promising programs reviewed met the stringent criteria for being efficacious according to the new standards set forth by the Society for Prevention Research (2006). Clearly, even the most promising relational aggression programs still require more systematic investigations using random assignment procedures in which alternative programs are used to control for nonspecific treatment factors. In fact, only the SAPP used a randomized trial with an active control group (Capella & Weinstein, 2006). Further, program reviews highlighted several methodological limitations, including some studies having difficulty achieving full randomization because teachers withdrew their students from the study if assigned to the control group. Other investigations were limited by psychometric properties of their outcome measures. For example, many outcome measures had reliability assessed only by internal consistency and not test–retest or inter-rater reliability. Others used extremely brief outcome measures for the primary constructs of relational aggression (e.g., two items). It is essential that outcome measures, including teacher reports and student self-reports, have well-demonstrated reliability and validity for measuring relational aggression. In general, we recommend that these newer relational aggression intervention programs be evaluated using more extensive multimethod, multi-informant measures across multiple time periods (Leff et al., 2001). Notably, several of the programs adequately monitored treatment fidelity and program acceptability (e.g., F2F, Leff et al., 2009; Early Childhood Friendship Project, Ostrov et al., 2009), which are extremely important aspects of effectiveness research (Society for Prevention Research, 2006).
The Role of School Psychologists
This review suggests a number of implications for school psychologists. First, school psychologists can provide school-wide workshops and trainings raising awareness that children’s aggression includes both physical and nonphysical manifestations (e.g., social exclusion, gossip, ignoring). Second, school psychologists can take a central role in the implementation and evaluation of intervention programs. For example, because school psychologists are trained in data-based decision making, they could be part of an administrative team that reviews potential programs for adoption in their schools, help in the selection of an evidence-based program, collect data on outcomes and treatment fidelity for a selected program, and be part of the team that analyzes the outcome data. Specifically, it is suggested that school practitioners consider their school’s needs and resources in conjunction with the mode of operation, target population, and preliminary findings for each of the promising existing programs in order to determine which program would best serve their school. Notably, although many general bullying programs may target relationally aggressive behaviors, this systematic review did not reveal any programs that have fully examined their effectiveness on relational aggression. Until more programs target and measure outcomes specifically related to relational bullying and victimization, it is important to recognize that general bullying programs require systematic evaluations related to relational bullying and victimization before large-scale implementation to address relational aggression.
Finally, school psychologists can be incorporated into implementation of interventions for relational aggression by serving as teacher trainers or co-implementers with teachers in classrooms. School psychologists can help ensure that for whichever program is selected, teachers and other program facilitators are well trained, are continuously supported, and follow appropriate implementation guidelines (see Leff et al., 2009). Although there are many challenges ahead, the findings from developmental research and some innovative intervention studies suggest that the field is moving in the right direction for preventing and reducing relational aggression and related challenges.
Acknowledgments
This research was supported by several grants, including National Institute of Mental Health Grants R01MH075787 and T32MH18834, National Science Foundation Grant 0819148, and Centers for Disease Control and Prevention Cooperative Agreement 5 U49 CE001093.
Biographies
Stephen S. Leff, PhD, is an associate professor of clinical psychology in pediatrics at the University of Pennsylvania School of Medicine and a psychologist at The Children’s Hospital of Philadelphia. He directs two National Institute of Mental Health funded intervention programs: the Friend to Friend Program, a group intervention program for inner-city relationally aggressive girls, and the Preventing Relational Aggression in Schools Everyday Program, a classroom-based relational aggression program for all third-and fourth-grade students. His publications and research interests include aggression prevention in the urban schools and community, gender and social cognitive differences in the expression of aggression, integrity monitoring of interventions, and partnership-based approaches to measurement and intervention development.
Tracy Evian Waasdorp, PhD, MEd, is a clinical research associate at The Children’s Hospital of Philadelphia and Research Faculty in the Johns Hopkins Bloomberg School of Public Health. She holds a master’s degree in counseling from the University of Pennsylvania and a doctorate in human development from the University of Delaware. Her research focuses on bullying prevention and intervention, relational aggression, coping, and parent–child relationships.
Nicki Crick is director and professor at the Institute of Child Development and director of the Center on Relational Aggression, University of Minnesota. She currently holds three endowed chairs, the Distinguished McKnight University Professorship, the Irving B. Harris Professorship in Child Development, and the Birkmaier Professorship of Educational Leadership. She is internationally known for her pioneering research on relational aggression, work that has been funded by the National Science Foundation, the National Institute of Mental Health, and the National Institute of Child Health and Development as well as from private foundations such as the Harry Guggenheim Foundation and the William T. Grant Foundation.
Footnotes
The terms relational aggression, social aggression, and indirect aggression have been used by different researchers to describe somewhat similar forms of nonphysical aggressive behavior (Björqvist et al., 2002; Crick & Grotpeter, 1995; Galen & Underwood, 2007). Although there are some important distinctions between these terms, which have been reviewed elsewhere (Crick et al., 2007), we wished to identify interventions designed to reduce nonphysical forms of aggression and thereby chose to use all three terms in the keyword searches.
None of the 16 programs in this relatively new field of relational aggression interventions met the stringent criteria for being efficacious as defined by the Society for Prevention Research (2006).
In several cases we converted partial eta values to Cohen’s d (WITS, You Can’t Say You Can’t Play, MC, ICPS) and we calculated Cohen’s d for one study in which authors did not report effect sizes (Sisters of Nia) using the formula: d = M1 − M2/σpooled.
The authors use the term partial randomized group design to indicate a trial in which they had to make adjustments to strict randomization because of logistical issues (e.g., different classrooms having recess periods at different times).
According to Leadbeater and colleagues (2003), high ratings for fidelity were established by a school having satisfactory teacher trainings, the presence of a school champion to advocate on behalf of the program, and the implementation of the program in most kindergarten through third-grade classrooms in the previous year. More specific criteria for how each of these subcategories (satisfactory teacher trainings, and so on) were determined was not discussed in the published article.
Although Second Step is designed such that those receiving Level 2 curriculum typically would have completed Level 1 curriculum the year before, in the current study those seventh- and eighth-graders who received Level 2 curriculum had no prior exposure to the program.
Contributor Information
Stephen S. Leff, The Children’s Hospital of Philadelphia and University of Pennsylvania School of Medicine
Tracy Evian Waasdorp, The Children’s Hospital of Philadelphia and Johns Hopkins Bloomberg School of Public Health.
Nicki R. Crick, University of Minnesota
References
- Asher SR, Hymel S, Renshaw PD. Loneliness in children. Child Development. 1984;55:1456. [Google Scholar]
- Asher SR, Wheeler VA. Children’s loneliness: A comparison of rejected and neglected status. Journal of Consulting and Clinical Psychology. 1985;53:500–505. doi: 10.1037//0022-006x.53.4.500. [DOI] [PubMed] [Google Scholar]
- Belgrave FZ, Reed MC, Plybon LE, Butler DS, Allison KW, Davis T. An evaluation of Sisters of Nia: A cultural program for African American girls. Journal of Black Psychology. 2004;30:329–343. [Google Scholar]
- Björkqvist K, Österman K, Kaukiainen A. The development of direct and indirect aggressive strategies in males and females. In: Björkqvist K, Niemelä P, editors. Of mice and women: Aspects of female aggression. San Diego: Academic Press; 1992. pp. 51–64. [Google Scholar]
- Boldizar JP. Assessing sex typing and androgyny in children: The children’s sex role inventory. Developmental Psychology. 1991;27:505. [Google Scholar]
- Boyle D, Hassett-Walker C. Reducing overt and relational aggression among young children: The results from a two-year outcome evaluation. Journal of School Violence. 2008;7:27–42. [Google Scholar]
- Caldwell CB, Pianta RC. A measure of young children’s problem and competence behaviors: The Early School Behavior Scale. Journal of Psycho-educational Assessment. 1991;9:32–44. [Google Scholar]
- Cappella E, Weinstein R. The prevention of social aggression among girls. Social Development. 2006;15:434–462. [Google Scholar]
- Card NA, Stucky BD, Sawalani GM, Little TD. Direct and indirect aggression during childhood and adolescence: A meta-analytic review of gender differences, intercorrelations, and relations to maladjustment. Child Development. 2008;79:1185–1229. doi: 10.1111/j.1467-8624.2008.01184.x. [DOI] [PubMed] [Google Scholar]
- Cillessen AHN, Mayeux L. From censure to reinforcement: Developmental changes in the association between aggression and social status. Child Development. 2004;75:147–163. doi: 10.1111/j.1467-8624.2004.00660.x. [DOI] [PubMed] [Google Scholar]
- Cohen J. Statistical power analysis for the behavioral sciences. 2. Hillsdale, NJ: Erlbaum; 1988. [Google Scholar]
- Coie JD, Dodge KA. Continuities and changes in children’s social status: A five-year longitudinal study. Merrill-Palmer Quarterly. 1983;29:261–282. [Google Scholar]
- Crick NR. Relational aggression: The role of intent attributions, feelings of distress, and provocation type. Development and Psychopathology. 1995;7:313–322. [Google Scholar]
- Crick NR. The role of overt aggression, relational aggression, and prosocial behavior in the prediction of children’s future social adjustment. Child Development. 1996;67:2317–2327. [PubMed] [Google Scholar]
- Crick NR. Engagement in gender normative versus nonnormative forms of aggression: Links to social-psychological adjustment. Developmental Psychopathology. 1997;33:610–617. doi: 10.1037//0012-1649.33.4.610. [DOI] [PubMed] [Google Scholar]
- Crick NR, Bigbee MA. Relational and overt forms of peer victimization: A multiinformant approach. Journal of Consulting & Clinical Psychology. 1998;66:337–347. doi: 10.1037//0022-006x.66.2.337. [DOI] [PubMed] [Google Scholar]
- Crick NR, Casas JF, Mosher M. Relational and overt aggression in preschool. Developmental Psychology. 1997;33:579–588. doi: 10.1037//0012-1649.33.4.579. [DOI] [PubMed] [Google Scholar]
- Crick NR, Dodge KA. A review and reformulation of social information-processing mechanism in children’s social adjustment. Psychological Bulletin. 1994;115:74–101. [Google Scholar]
- Crick NR, Grotpeter JK. Relational aggression, gender, and social-psychological adjustment. Child Development. 1995;66:710–722. doi: 10.1111/j.1467-8624.1995.tb00900.x. [DOI] [PubMed] [Google Scholar]
- Crick NR, Grotpeter JK. Children’s treatment by peers: Victims of relational and overt aggression. Development and Psychopathology. 1996;8:367–380. [Google Scholar]
- Crick NR, Grotpeter JK, Bigbee MA. Relationally and physically aggressive children’s intent attributions and feelings of distress for relational and instrumental peer provocations. Child Development. 2002;73:1134–1142. doi: 10.1111/1467-8624.00462. [DOI] [PubMed] [Google Scholar]
- Crick NR, Ostrov JM, Burr JE, Cullerton-Sen C, Jansen-Yeh E, Ralston P. A longitudinal study of relational and physical aggression in pre-school. Journal of Applied Developmental Psychology. 2006;27:254–268. [Google Scholar]
- Crick NR, Ostrov JM, Kawabata Y. Relational aggression and gender: An overview. In: Flannery DJ, Vazsonyi AT, Waldman ID, editors. The Cambridge handbook of violent behavior and aggression. New York: Cambridge; 2007. pp. 245–259. [Google Scholar]
- Crick NR, Werner NE. Response decision processes in relational and overt aggression. Child Development. 1998;69:1630–1639. doi: 10.1111/j.1467-8624.1998.tb06181.x. [DOI] [PubMed] [Google Scholar]
- Dishion TJ, McCord J, Poulin F. When interventions harm: Peer groups and problem behavior. American Psychologist. 1999;54:755–764. doi: 10.1037//0003-066x.54.9.755. [DOI] [PubMed] [Google Scholar]
- Dodge KA. Social cognition and children’s aggressive behavior. Child Development. 1980;51:162–170. [PubMed] [Google Scholar]
- Dodge KA. A social information processing model of social competence in children. In: Perlmutter M, editor. Minnesota symposia on child psychology. Vol. 18. Hillsdale, NJ: Erlbaum; 1986. pp. 77–125. [Google Scholar]
- Dodge K, Coie J, Lynam D. Aggression and antisocial behavior in youth. In: Eisenberg N, Damon W, Lerner R, editors. Handbook of child psychology: Vol. 3, Social, emotional, and personality development. 6. Hoboken, NJ: John Wiley & Sons; 2006. pp. 719–788. [Google Scholar]
- Dodge KA, Pettit GS. A biopsychosocial model of the development of chronic conduct problems in adolescence. Developmental Psychology. 2003;39:349–371. doi: 10.1037//0012-1649.39.2.349. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ellis WE, Crooks CV, Wolfe DA. Relational aggression in peer and dating relationships: Links to psychological and behavioral adjustment. Social Development. 2009;18:253–269. [Google Scholar]
- Evertson CM, Emmer ET, Sanford JP, Clements BS. Improving classroom management: Good Behavior Game manual. Elementary School Journal. 1983;83:173–188. [Google Scholar]
- Farrell AD, Meyer AL, Kung EM, Sullivan TN. Development and evaluation of school-based violence prevention programs. Journal of Clinical Child Psychology. 2001;30:207–220. doi: 10.1207/S15374424JCCP3002_8. [DOI] [PubMed] [Google Scholar]
- Fraser M, Day S, Galinsky M, Hodges V, Smokowski P. Conduct problems and peer rejection in childhood: A randomized trial of the Making Choices and Strong Families Programs. Research on Social Work Practice. 2004;14:313–324. [Google Scholar]
- Fraser M, Galinsky M, Smokowski P, Day S, Terzian M, Rose R, et al. Social information-processing skills training to promote social competence and prevent aggressive behavior in the third grades. Journal of Consulting and Clinical Psychology. 2005;73:1045–1055. doi: 10.1037/0022-006X.73.6.1045. [DOI] [PubMed] [Google Scholar]
- Frey KS, Nolen SB, Van Schoiack Edstrom L, Hirschstein MK. Effects of a school-based social-emotional competence program: Linking children’s goals, attributions, and behavior. Journal of Applied Developmental Psychology. 2005;26:171–200. [Google Scholar]
- Galen BR, Underwood M. A developmental investigation of social aggression among children. Developmental Psychology. 1997;33:589–599. doi: 10.1037//0012-1649.33.4.589. [DOI] [PubMed] [Google Scholar]
- Grossman DC, Neckerman HJ, Koepsell TD, Liu P, Asher KN, Beland K, et al. Effectiveness of a violence prevention curriculum among children in elementary school: A randomized controlled trial. Journal of the American Medical Association. 1997;277:1605–1611. [PubMed] [Google Scholar]
- Harrist AW, Bradley KD. ‘You can’t say you can’t play’: Intervening in the process of social exclusion in the kindergarten classroom. Early Childhood Research Quarterly. 2003;18:185–205. [Google Scholar]
- Harter S, Pike R. The pictorial scale of perceived competence and social acceptance for young children. Child Development. 1984;55:1969–1982. [PubMed] [Google Scholar]
- Hoff KE, Reese-Weber M, Schneider WJ, Stagg JW. The association between high status positions and aggressive behavior in early adolescence. Journal of School Psychology. 2009;47:395–426. doi: 10.1016/j.jsp.2009.07.003. [DOI] [PubMed] [Google Scholar]
- Kovacs M. Children’s Depression Inventory. Psychopharmacology Bulletin. 1985;21:995–998. [PubMed] [Google Scholar]
- Kuppens S, Grietens H, Onghena P, Michiels D, Subramanian SV. Individual and classroom variables associated with relational aggression in elementary-school aged children: A multilevel analysis. Journal of School Psychology. 2008;46:639–660. doi: 10.1016/j.jsp.2008.06.005. [DOI] [PubMed] [Google Scholar]
- LaFrenière PJ, Dumas JE, Capuano F, Dubeau D. Development and validation of the Pre-school Socioaffective Profile. Psychological Assessment. 1992;4:442–450. [Google Scholar]
- Leadbeater BJ, Hoglund W. Changing the social contexts of peer victimization. Journal of Canadian Academy of Child and Adolescent Psychiatry. 2006;15:21–26. [PMC free article] [PubMed] [Google Scholar]
- Leadbeater B, Hoglund W, Woods T. Changing contexts? The effects of a primary prevention program on classroom levels of peer relational and physical victimization. Journal of Community Psychology. 2003;31:397–418. [Google Scholar]
- Leff SS, Angelucci J, Goldstein AB, Cardaciotto L, Paskewich B, Grossman MB. Using a participatory action research model to create a school-based intervention program for relationally aggressive girls: The Friend to Friend Program. In: Zins JE, Elias MJ, Maher CA, editors. Bullying, victimization, and peer harassment: A handbook of prevention and intervention. New York: Haworth; 2007. pp. 199–218. [Google Scholar]
- Leff SS, Costigan TE, Power TJ. Using participatory-action research to develop a playground-based prevention program. Journal of School Psychology. 2004;42:3–21. [Google Scholar]
- Leff SS, Crick NR, Angelucci J, Haye K, Jawad AF, Grossman M, Power TJ. Social cognition in context: Validating a cartoon-based attributional measure for urban girls. Child Development. 2006;77:1351–1358. doi: 10.1111/j.1467-8624.2006.00939.x. [DOI] [PubMed] [Google Scholar]
- Leff SS, Gullan RL, Paskewich BS, Abdul-Kabir S, Jawad AF, Grossman M, et al. An initial evaluation of a culturally-adapted social problem solving and relational aggression prevention program for urban African American relationally aggressive girls. Journal of Prevention and Intervention in the Community. 2009;37:260–274. doi: 10.1080/10852350903196274. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Leff SS, Hoffman JA, Gullan RL. Intervention integrity: New paradigms and applications. School Mental Health. 2009;1:103–106. doi: 10.1007/s12310-009-9013-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Leff SS, Kupersmidt JB, Power TJ. An initial examination of girls’ cognitions of their relationally aggressive peers as a function of their own social standing. Merrill Palmer Quarterly. 2003;49:28–53. [Google Scholar]
- Leff SS, Power TJ, Manz PH, Costigan TE, Nabors LA. School-based aggression prevention programs for young children: Current status and implications for violence prevention. School Psychology Review. 2001;30:343–360. [Google Scholar]
- Lochman JE, Dodge KA. Social-cognitive processes of severely violent, moderately aggressive, and nonaggressive boys. Journal of Consulting & Clinical Psychology. 1994;62:366–374. doi: 10.1037//0022-006x.62.2.366. [DOI] [PubMed] [Google Scholar]
- Macgowan MJ, Nash JK, Fraser MW. The Carolina Child Checklist of Risk and Protective Factors for Aggression. Research on Social Work Practice. 2002;12:253–276. [Google Scholar]
- Morrison FJ, Bachman HJ, Connor CM. Current Perspectives in Psychology. New Haven, CT: Yale University Press; 2005. Improving literacy in America: Guidelines from research. [Google Scholar]
- Murray-Close D, Crick NR. Gender differences in the association between cardiovascular reactivity and aggressive conduct. International Journal of Psychophysiology. 2007;65:103–113. doi: 10.1016/j.ijpsycho.2007.03.011. [DOI] [PubMed] [Google Scholar]
- Murray-Close D, Ostrov JM, Crick NR. A short-term longitudinal study of growth of relational aggression during middle childhood: Associations with gender, friendship intimacy, and internalizing problems. Development and Psychopathology. 2007;19:187–203. doi: 10.1017/S0954579407070101. [DOI] [PubMed] [Google Scholar]
- Ostrov J, Keating C. Gender differences in preschool aggression during free play and structured interactions: An observational study. Social Development. 2004;13:255–277. [Google Scholar]
- Ostrov JM, Massetti GM, Stauffacher K, Godleski SA, Hart KC, Karch KM, et al. An intervention for relational and physical aggression in early childhood: A preliminary study. Early Childhood Research Quarterly. 2009;24:15–28. [Google Scholar]
- Ostrov JM, Woods KE, Jansen EA, Casas JF, Crick NR. An observational study of delivered and received aggression, gender, and social-psychological adjustment in preschool: ‘This White Crayon Doesn’t Work. Early Childhood Research Quarterly. 2004;19:355–371. [Google Scholar]
- Paley VG. You Can’t Say You Can’t Play. Cambridge: Harvard University Press; 1992. [Google Scholar]
- Perry DG, Perry LC, Rasmussen P. Cognitive social learning mediators of aggression. Child Development. 1986;57:700–711. [PubMed] [Google Scholar]
- Prinstein MJ, Boergers J, Vernberg EM. Overt and relational aggression in adolescents: Social–psychological adjustment of aggressors and victims. Journal of Clinical Child Psychology. 2001;30:479–491. doi: 10.1207/S15374424JCCP3004_05. [DOI] [PubMed] [Google Scholar]
- Prinstein MJ, Cillessen AHN. Forms and functions of adolescent peer aggression associated with high levels of peer status. Merrill-Palmer Quarterly. 2003;49:310–342. [Google Scholar]
- Puckett MB, Aikins J, Cillessen A. Moderators of the association between relational aggression and perceived popularity. Aggressive Behavior. 2008;34:563–576. doi: 10.1002/ab.20280. [DOI] [PubMed] [Google Scholar]
- Shure MB. An interpersonal problem-solving program (kindergarten/primary grades) 2. Champaign, IL: Research Press; 2001. I Can Problem Solve (ICPS) [Google Scholar]
- Shure MB. Hahnemann Behavior Rating Scale. Hahnemann University; 2002. Unpublished. [Google Scholar]
- Slaby RG, Guerra NG. Cognitive mediators of aggression in adolescent offenders: 1. Assessment. Developmental Psychology. 1988;24:580–588. [Google Scholar]
- Smith EP, Brookins CC. Toward the development of an ethnic identity measure for African American youth. Journal of Black Psychology. 1997;23:358–377. [Google Scholar]
- Society for Prevention Research. Standards of evidence: Criteria for efficacy, effectiveness and dissemination. 2006 doi: 10.1007/s11121-005-5553-y. Retrieved from http://www.preventionscience.org/StandardsofEvidencebook.pdf. [DOI] [PubMed]
- Stauffacher K, DeHart GB. Crossing social contexts: Relational aggression between siblings and friends during early and middle childhood. Journal of Applied Developmental Psychology. 2006;27:228–240. [Google Scholar]
- Swearer SM. Relational aggression: Not just a female issue. Journal of School Psychology. 2008;46:611–616. doi: 10.1016/j.jsp.2008.08.001. [DOI] [PubMed] [Google Scholar]
- Van Schoiack-Edstrom L, Frey KS, Beland K. Changing adolescents’ attitudes about relational and physical aggression: An early evaluation of a school-based intervention. School Psychology Review. 2002;31:201–216. [Google Scholar]
- Vasey M, Dangleish T, Silverman W. Research on information-processing factors in child and adolescent psychopathology: A critical commentary. Journal of Clinical Child & Adolescent Psychology. 2003;32:81–93. doi: 10.1207/S15374424JCCP3201_08. [DOI] [PubMed] [Google Scholar]
- Werner NE, Crick NR. Relational aggression and social-psychological adjustment in a college sample. Journal of Abnormal Psychology. 1999;108:615–623. doi: 10.1037//0021-843x.108.4.615. [DOI] [PubMed] [Google Scholar]
- Werner NE, Crick NR. Maladaptive peer relationships and the development of relational and physical aggression during middle childhood. Social Development. 2004;13:495–514. [Google Scholar]
- Williams KR, Guerra NG. Prevalence and predictors of Internet bullying. Journal of Adolescent Health. 2007;41:S14–S21. doi: 10.1016/j.jadohealth.2007.08.018. [DOI] [PubMed] [Google Scholar]
- Zimmer-Gembeck MJ, Geiger TC, Crick NR. Relational and physical aggression, prosocial behavior, and peer relations: Gender moderation and bidirectional associations. The Journal of Early Adolescence. 2005;25:421–452. [Google Scholar]