Assessment of School Anti-Bullying Interventions

This meta-analysis of randomized clinical trials assesses the effectiveness of school anti-bullying interventions, their population impact, and the association of moderators.

Key Points

Question

What is the effectiveness of anti-bullying interventions, their population impact, and is there an association between moderator variables and the effectiveness of these interventions?

Findings

Across 77 samples from 69 randomized clinical trials (111 659 participants), meta-analyses showed that interventions were statistically significantly effective in reducing bullying and improving mental health problems at study end point. Meta-regression analyses showed that duration of intervention was not statistically significantly associated with effectiveness and that the impact of the anti-bullying programs did not diminish over time during follow-up.

Meaning

Findings of this meta-analysis support the concept that school anti-bullying interventions may have a valuable population impact.

Abstract

Importance

Bullying is a prevalent and modifiable risk factor for mental health disorders. Although previous studies have supported the effectiveness of anti-bullying programs; their population impact and the association of specific moderators with outcomes are still unclear.

Objective

To assess the effectiveness of school anti-bullying interventions, their population impact, and the association between moderator variables and outcomes.

Data Sources

A search of Ovid MEDLINE, ERIC, and PsycInfo databases was conducted using 3 sets of search terms to identify randomized clinical trials (RCTs) assessing anti-bullying interventions published from database inception through February 2020. A manual search of reference lists of articles included in previous systematic reviews and meta-analyses was also performed.

Study Selection

The initial literature search yielded 34 798 studies. Included in the study were articles that (1) assessed bullying at school; (2) assessed the effectiveness of an anti-bullying program; (3) had an RCT design; (4) reported results; and (5) were published in English. Of 16 707 studies identified, 371 met the criteria for review of full-text articles; 77 RCTs were identified that reported data allowing calculation of effect sizes (ESs). Of these, 69 independent trials were included in the final meta-analysis database.

Data Extraction and Synthesis

Random-effects and meta-regression models were used to derive Cohen d values with pooled 95% CIs as estimates of ES and to test associations between moderator variables and ES estimates. Population impact number (PIN), defined as the number of children in the total population for whom 1 event may be prevented by an intervention, was used as an estimate of the population impact of universal interventions targeting all students, regardless of individual risk.

Main Outcomes and Measures

The main outcomes are the effectiveness (measured by ES) and the population impact (measured by the PIN) of anti-bullying interventions on the following 8 variable categories: overall bullying, bullying perpetration, bullying exposure, cyberbullying, attitudes that discourage bullying, attitudes that encourage bullying, mental health problems (eg, anxiety and depression), and school climate as well as the assessment of potential assocations between trial or intervention characteristics and outcomes.

Results

This study included 77 samples from 69 RCTs (111 659 participants [56 511 in the intervention group and 55 148 in the control group]). The weighted mean (range) age of participants in the intervention group was 11.1 (4-17) years and 10.8 (4-17) years in the control group. The weighted mean (range) proportion of female participants in the intervention group was 49.9% (0%-100%) and 50.5% (0%-100%) in the control group. Anti-bullying interventions were efficacious in reducing bullying (ES, −0.150; 95% CI, −0.191 to −0.109) and improving mental health problems (ES, −0.205; 95% CI, −0.277 to −0.133) at study end point, with PINs for universal interventions that target the total student population of 147 (95% CI, 113-213) and 107 (95% CI, 73-173), respectively. Duration of intervention was not statistically significantly associated with intervention effectiveness (mean [range] duration of interventions, 29.4 [1 to 144] weeks). The effectiveness of anti-bullying programs did not diminish over time during follow-up (mean [range] follow-up, 30.9 [2-104] weeks).

Conclusions and Relevance

Despite the small ESs and some regional differences in effectiveness, the population impact of school anti-bullying interventions appeared to be substantial. Better designed trials that assess optimal intervention timing and duration are warranted.

Introduction

A preventive approach to mental health and well-being may be essential to reduce the burden associated with mental health disorders, especially in young people.¹ Traditional bullying is defined as deliberate aggressive behaviors by another youth or group of youths who are not siblings or current dating partners that are repeated and involve a power imbalance favoring the perpetrators.^2,3,4 Bullying is a major target for universal prevention given the high prevalence rates, association with increased lifetime prevalence of mental health disorders,⁵ and converging evidence supporting the feasibility and cost-effectiveness of anti-bullying interventions.^6,7 Based on a 2009-2010 survey, the Health Behaviour in School-aged Children study reported a bullying exposure prevalence of more than 10% among a school aged population,⁸ but other studies^9,10 have reported school bullying rates of 20% to 30% or even higher. There is greater uncertainty regarding estimated prevalence rates for cyberbullying, defined as intentional and repeated harm inflicted through electronic devices and social media. Available studies suggest that cyberbullying may affect approximately 15% to 25% of youth and that it usually coexists with traditional bullying.¹¹

Bullying exposure has been consistently associated with worse mental health in childhood and adolescence.^9,12 The negative consequences of bullying are pervasive, and bullying exposure in childhood is also associated with poor mental and physical health, lack of social relationships, economic hardship, and decreased quality of life in early adulthood and midlife.^{5,9,13,14,15,16,17,18,19,20} Bullying perpetrators also experience worse physical and mental health both in childhood and adulthood along with social disadvantage during adulthood.^21,22,23

Taken together, the available evidence indicates that bullying is 1 of the most prevalent potentially modifiable risk factors for mental health disorders, thus rendering it a major public health concern,^4,24 especially considering the high associated lifetime direct and indirect economic costs.^7,25 The growing awareness of bullying has led to the implementation of different school-based anti-bullying programs in the last 20 years.²⁶ Some meta-analyses^{27,28,29,30,31} have reported small to moderate effectiveness of anti-bullying programs, with a mean decrease of approximately 20% in bullying rates. The results of these meta-analyses support the feasibility of implementing anti-bullying programs in schools and suggest their potential effectiveness. However, the population impact (ie, the number of children in the whole population among whom 1 event of bullying on average may be prevented by anti-bullying interventions) remains unclear.³² Previous meta-analyses^{27,28,29,30,31} also leave several questions unanswered. What is the association of bullying prevention interventions with mental health? What is the optimal duration of an intervention program? Does the benefit of the intervention diminish over time after the intervention ends? Do differential factors moderate the effectiveness of anti-bullying and anti-cyberbullying programs? To address these questions, we conducted a meta-analysis of randomized clinical trials (RCTs) that assessed school interventions with the aim of reducing bullying or cyberbullying rates or improving school climate to evaluate their short-term and medium-term population impact. In addition, we conducted meta-regression analyses to assess whether moderator variables impacted the effectiveness of the interventions.

Methods

Search Strategies

This meta-analysis of RCTs used the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) reporting guideline. We conducted a systematic 2-step literature search to identify RCTs assessing anti-bullying interventions (eTable 1 in the Supplement).³³ We first performed a search of Ovid MEDLINE, ERIC (Eric.ed.gov), and PsycInfo databases from inception through February 2020 (eMethods in the Supplement). Three sets of search terms were used: (1) [“bullying” OR “peer abuse” OR “abuse” OR “aggression” OR “harassment” OR “perpetrator” OR “victim” OR “victimization” OR “peer violence” OR “violence” OR “cyberbullying” OR “anti-bullying”], (2) AND [“school” OR “peer”], and (3) AND [“intervention” OR “curriculum” OR “prevention” OR “program” OR “resilience” OR “school climate” OR “school-based” OR “therapy” OR “treatment” OR “trial”]. We then performed a manual search of the reference lists of articles included in previous systematic reviews and meta-analyses for any RCTs not identified by the literature search.

Study Selection and Data Extraction

eFigure 1 in the Supplement shows a flowchart of the systematic literature search strategy. The initial literature search yielded 34 798 studies. The manual search identified 6 additional records. After removing 18 097 duplicates, we evaluated 16 707 potential studies.

Four of us (M.A., M.D-C., R.A-C., and I.E-B.) double-screened all articles in 3 phases, resolving discrepancies through discussion and consensus. The eMethods in the Supplement describes study selection criteria and procedures in detail. Briefly, in phase 1, inclusion criteria included articles that (1) assessed bullying at school; (2) assessed the effectiveness of an anti-bullying program; (3) had an RCT design; (4) reported results; and (5) were published in English. Of 16 707 studies identified, 371 met the criteria to proceed to phase 2. Phase 2 consisted of a review of the full-text articles; 77 RCTs were identified that reported data that would allow calculation of effect sizes. Of these trials, 69 original independent RCTs met the criteria for inclusion in the final meta-analysis database.

Six of us (D.F., C.M.D-C., M.A., M.D-C., R.A-C., and I.E-B.) extracted data from each eligible study independently and double-checked them by pairs, with discrepancies resolved via discussion. Data extracted included the following: year of publication, region (country and city if available) where the study was conducted, name of the intervention program, date of intervention, duration of intervention, duration of follow-up (when applicable), type of randomization (individual or cluster), type of control group, type of school (public or private), primary (age, ≤11 years) vs secondary (age, 12-18 years) education, sample size, number of randomized groups, mean age, age range, and percentage of females (for both intervention and control groups), type of approach (universal or targeted), type of bullying variable (dichotomous or continuous), and statistics to calculate effect sizes for the meta-analyses and meta-regressions (eMethods in the Supplement).

Classification of Outcome Variables and Quality Assessment

The 69 original independent RCTs used more than 500 different instruments to assess outcome variables. Three of us (D.F., R.A-C., and I.E-B.) independently classified these instruments into a manageable number of outcome variables, with discrepancies resolved by discussion. This classification allowed us to consolidate outcome variables into the following 8 categories based on previous meta-analyses^28,29: (1) overall bullying (pooled measure, including data on bullying perpetration, bullying exposure, and cyberbullying), (2) bullying perpetration, (3) bullying exposure, (4) cyberbullying (both perpetration and exposure), (5) attitudes that discourage bullying, (6) attitudes that encourage bullying, (7) mental health problems (eg, anxiety and depression), and (8) school climate.

We performed meta-analyses for each of the categories at end point of intervention and follow-up (interval between end of intervention and further assessment). Details regarding classification of outcome variables are described in the eMethods and eTable 2 in the Supplement.

We assessed the quality of the 69 selected RCTs using an item checklist constructed for this meta-analysis based on the Cochrane Collaboration’s tool for assessing risk of bias.³⁴ Details on the quality assessment are described in eTable 3 in the Supplement.

Statistical Analysis

We conducted random-effects meta-analyses using Comprehensive Meta-Analysis, version 2.0 (Biostat Inc).³⁵ Cohen d values with pooled 95% CIs were used as estimates of the effect size of each anti-bullying intervention compared with control groups. For purposes of this work, a positive Cohen d value indicates that a specific variable increases more in the intervention group than in the control group during the assessed period, whereas a negative Cohen d value indicates that a specific variable increases more in the control group than in the intervention group during the assessed period. Forest plots were generated using DistillerSR Forest Plot Generator (Evidence Partners).³⁶

We assessed statistical heterogeneity through visual inspection of forest plots and using the Q statistic (a magnitude of heterogeneity) and the I² statistic (a measure of the proportion of variance in summary effect sizes attributable to heterogeneity).³⁷ I² values less than 30% were considered an insignificant amount of heterogeneity.³⁸ We assessed publication bias by visually inspecting funnel plots and using the fail-safe N described by Orwin,³⁹ with a criterion for a trivial standardized difference in means of 0.1 and a mean standardized difference in means in missing studies of 0. Furthermore, we used the linear regression method described by Egger et al⁴⁰ to quantify the bias captured by the funnel plot.

We used meta-regressions with a random-effects model with unrestricted maximum likelihood to test associations of potential moderators with effect size estimates for statistically significant meta-analyses. Statistically significant meta-regression values were confirmed by excluding 1 study at a time, and only meta-regressions for which P values remained statistically significant after this process were considered statistically significant. The threshold for statistical significance was set at .05.

Because recent meta-analyses of the effectiveness of anti-bullying interventions have reported a statistically significant association with geographic location,²⁷ we performed a meta-analytic subgroup analysis by region. We conducted additional subgroup meta-analyses of universal interventions (targeting the total student population, regardless of individual risk) and targeted (nonuniversal) interventions.

Cohen d values were converted to number needed to treat (NNT) as recommended in the method by Furukawa and Leucht.⁴¹ The NNT was used to obtain the population impact number (PIN) as an estimated measure of the population impact of the intervention. The PIN is defined as children in the total population for whom 1 event may be prevented by an intervention^32,42 or, the number needed to participate in an anti-bullying program to prevent 1 case of bullying.⁴³ The PIN values were calculated using RCTs that assessed a universal intervention.

We used false discovery rate correction for multiple comparisons.⁴⁴ The percentage of tolerated false-positive results was 5% (Q <.05). The Q value is the adjusted P value calculated using a false discovery rate approach. The threshold for statistical significance was set at .05 (2 sided). Details on statistical analyses are described in the eMethods in the Supplement.

Results

Characteristics of the Selected RCTs and Samples

This meta-analysis included 77 samples from 69 RCTs, constituting an overall sample of 111 659 participants. These participants included 56 511 (in 609 randomized clusters) in the intervention group and 55 148 (in 601 randomized clusters) in the control group. Among all 69 RCTs, 5 tested interventions targeting cyberbullying, and 15 reported results at follow-up, with a mean (range) follow-up of 30.9 (2-104) weeks.

The weighted mean (range) age of participants in the intervention group was 11.1 (4-17) years and 10.8 (4-17) years in the control group. The weighted mean (range) proportion of female participants was 49.9% (0%-100%) in the intervention group and 50.5% (0%-100%) in the control group. The mean (range) duration of interventions was 29.4 (1-144) weeks (95% CI, 21.5-37.3 weeks). Characteristics of the selected RCTs are listed in eTable 4 and eTable 5 in the Supplement.

Effectiveness of Anti-Bullying Interventions at Study End Point

Table 1 summarizes the results. Anti-bullying interventions were effective in reducing overall bullying (as a pooled measure, including bullying perpetration, bullying exposure, and cyberbullying) at study end point (number of samples [k] = 45; effect size, −0.150; 95% CI, −0.191 to −0.109; P < .001) (Figure 1).^{45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86} Anti-bullying interventions showed statistically significant effectiveness compared with control groups on all assessed bullying-related outcomes after the intervention. The effect sizes were mostly statistically significant and small (mean range, 0.070-0.205), with high statistical heterogeneity and risk of publication bias. Anti-bullying interventions also showed statistically significant effectiveness in improving mental health problems (eg, anxiety and depression) at study end point, with small effect size (k = 20; effect size, −0.205; 95% CI, −0.277 to −0.133; P < .001) (Figure 2).^{47,51,52,55,58,59,61,64,66,73,77,86,87,88,89,90,91,92}

Table 1. Meta-analyses of Effectiveness of Randomized Clinical Trials Assessing School Anti-Bullying Interventions.

Outcome Variable	Duration of intervention, mean (95% CI), wk	Length of follow-up, mean (range), wk	k	No. of participants		Meta-analysisa		Heterogeneityb		Publication biasc
				Intervention groups	Control groups	Cohen d, mean (95% CI)	FDR corrected P value	Q statistic P value	I2 statistic, %	Fail-safe N	Regression intercept P value
Overall bullyingd
End of intervention	32.6 (23.7 to 41.6)	NA	45	46 847	45 744	−0.150 (−0.191 to −0.109)	<.001	<.001	85.3	209	.03
Follow-up	31.5 (15.8 to 47.2)	44.0 (3 to 104)	21	11 020	11 977	−0.171 (−0.243 to −0.099)	<.001	<.001	80.0	16	.09
Bullying perpetration
End of intervention	35.9 (25.3 to 46.6)	NA	35	43 199	42 991	−0.111 (−0.146 to −0.077)	<.001	<.001	78.8	558	.006
Follow-up	33.4 (15.4 to 51.4)	39.2 (3 to 104)	17	7889	7993	−0.175 (−0.276 to −0.073)	.002	<.001	85.9	49	.18
Bullying exposure
End of intervention	34.8 (22.9 to 46.6)	NA	32	37 190	37 001	−0.158 (−0.225 to −0.092)	<.001	<.001	94.1	25	.33
Follow-up	23.5 (9.5 to 37.5)	40.9 (4 to 78)	13	6971	7629	−0.122 (−0.173 to −0.071)	<.001	.060	41.3	12	.20
Cyberbullyinge
End of intervention	33.4 (5.5 to 67.3)	NA	5	3271	2472	−0.135 (−0.201 to −0.069)	<.001	.290	19.7	5	.34
Follow-up	78.0 (NA)	52.0 (NA)	1	NA	NA	NA	NA	NA	NA	NA	NA
Attitudes that discourage bullying
End of intervention	27.7 (19.7 to 35.6)	NA	25	20 537	17 778	0.195 (0.145 to 0.245)	<.001	<.001	78.4	4	.007
Follow-up	34.8 (17.4 to 52.2)	50.1 (2 to 104)	14	5517	4596	0.143 (0.083 to 0.202)	<.001	.011	52.5	2	.06
Attitudes that encourage bullying
End of intervention	27.1 (17.4 to 36.8)	NA	15	15 884	14 037	−0.115 (−0.184 to −0.046)	.04	<.001	85.2	14	.58
Follow-up	19.2 (10.8 to 48.2)	48.6 (4 to 78)	7	3329	3299	−0.123 (−0.197 to −0.048)	.002	.070	48.6	69	.69
Mental health problems
End of intervention	25.7 (11.1 to 40.2)	NA	20	14 543	14 649	−0.205 (−0.277 to −0.133)	<.001	<.001	83.7	10	<.001
Follow-up	20.8 (7.8 to 44.4)	27.3 (2 to 52)	6	1605	1621	−0.202 (−0.347 to −0.056)	.01	.012	65.7	4	.001
School climate
End of intervention	36.5 (13.1 to 59.9)	NA	12	11 417	11 995	0.070 (0.044 to 0.096)	<.001	.700	0	NA	.02
Follow-up	18.8 (8.3 to 44.9)	62.4 (52 to 78)	5	2647	2978	0.135 (0.037 to 0.233)	.009	.006	72.0	1	.92

Abbreviations: FDR, false discovery rate; k, number of samples; NA, not applicable.

^aPositive Cohen d values indicate that a specific variable increases more in the intervention group than in the control group during the assessed period, whereas negative Cohen d values indicate the opposite.

^bQ statistic is a magnitude of heterogeneity, and I² statistic is a measure of the proportion of variance in summary effect size attributable to heterogeneity.

^cFail-safe N by Orwin³⁹ refers to the number of unpublished studies required to move estimates to a nonsignificant threshold. Regression intercept P value refers to the linear regression method by Egger et al.⁴⁰

^dOverall bullying is a pooled measure, including data on bullying perpetration, bullying exposure, and cyberbullying.

^eCyberbullying includes pooled cyberbullying perpetration and cyberbullying exposure data.

Figure 1. Effectiveness of Anti-Bullying Interventions on Overall Bullying at Study End Point.

Meta-analysis of effectiveness on overall bullying (as a pooled measure, including bullying perpetration, bullying exposure, and cyberbullying) at study end point. CPPRG indicates Conduct Problems Prevention Research Group; ES, effect size. The size of each square box indicates the relative weighting of the study in the meta-analysis. The short horizontal lines associated with each square box represent the 95% CIs. The vertical dashed line indicates the pooled effectiveness of anti-bullying interventions on overall bullying at study end point. The diamond represents the pooled effect estimate. The outer edges of the diamond represent the 95% CI around the pooled estimate.

Figure 2. Effectiveness of Anti-Bullying Interventions on Mental Health Problems at Study End Point.

Meta-analysis of effectiveness on mental health problems at study end point. CPPRG indicates Conduct Problems Prevention Research Group; ES, effect size. The size of each square box indicates the relative weighting of the study in the meta-analysis. The short horizontal lines associated with each square box represent the 95% CIs. The vertical dashed line indicates the pooled effectiveness of anti-bullying interventions on mental health problems at study end point. The diamond represents the pooled effect estimate. The outer edges of the diamond represent the 95% CI around the pooled estimate. In the forest plot, the arrow at the top left indicates that the 95% CI is wider on that side than the highest value on the scale.

Among all 69 RCTs, 31 (45.0%) were conducted in Europe, 19 (27.5%) in North America (United States and Canada), and 19 (27.5%) elsewhere. Meta-analyses by region showed that anti-bullying interventions had comparable effect sizes in overall bullying and bullying perpetration in Europe and North America. However, interventions were statistically significantly effective in decreasing bullying exposure and in decreasing attitudes that encourage bullying at study end point in Europe (k = 18; effect size, −0.142; 95% CI, −0.193 to −0.091; P < .001 and k = 10; effect size, −0.155; 95% CI, −0.242 to −0.068; P < .001, respectively) but not in North America (k = 8; effect size, −0.209; 95% CI, −0.563 to 0.145; P = .28 and k = 4; effect size, −0.016; 95% CI, −0.122 to 0.090; P = .78; respectively). Greater effect sizes were found for the effectiveness of interventions in increasing attitudes that discourage bullying at study end point in European trials (k = 15; effect size, 0.243; 95% CI, 0.164-0.323) than in North American trials (k = 6; effect size, 0.110; 95% CI, 0.063-0.157). The other meta-analyses found no statistically significant differences between regions. eTable 6 and eTable 7 in the Supplement provide additional details.

Of the 69 RCTs, 58 (84.1%) assessed a universal anti-bullying intervention. Subgroup meta-analyses of trials testing a targeted intervention showed results comparable to those found for universal interventions in terms of the direction of the associations, albeit with smaller effect sizes for most outcome measures (eTable 8 in the Supplement).

Table 2 lists NNT and PIN values for universal school anti-bullying interventions based on different estimated prevalence rates of school bullying. The PIN values of anti-bullying interventions for overall bullying and mental health problems at study end point were 147 (95% CI, 113-213) and 107 (95% CI, 73-173), respectively, assuming a bullying prevalence of 15%. Figure 3 shows PIN values for each outcome variable. For example, PIN values of anti-bullying interventions at study end point were 207 (95% CI, 153-307) for bullying perpetration, 140 (95% CI, 93-260) for bullying exposure, and 167 (95% CI, 100-360) for cyberbullying, assuming a bullying prevalence of 15%.

Table 2. PIN of Universal School Anti-Bullying Interventions.

Outcome variable	k	NNT (95% CI)	PIN (95% CI) for bullying prevalence
			5%	10%	15%	20%
Overall bullyinga
End of intervention	39	22 (17-32)	440 (340-640)	220 (170-320)	147 (113-213)	110 (85-160)
Follow-up	17	19 (13-37)	380 (260-740)	190 (130-370)	127 (87-247)	95 (65-185)
Bullying perpetration
End of intervention	33	31 (23-46)	620 (460-920)	310 (230-460)	207 (153-307)	155 (115-230)
Follow-up	14	19 (11-56)	380 (220-1120)	190 (110-560)	127 (73-373)	95 (55-280)
Bullying exposure
End of intervention	27	21 (14-39)	420 (280-780)	210 (140-390)	140 (93-260)	105 (70-195)
Follow-up	10	29 (19-57)	580 (380-1140)	290 (190-570)	193 (127-380)	145 (95-285)
Cyberbullyingb
End of intervention	4	25 (15-54)	500 (300-1080)	250 (150-540)	167 (100-360)	125 (75-270)
Follow-up	1	NA	NA	NA	NA	NA
Attitudes that discourage bullying
End of intervention	21	17 (14-24)	340 (280-480)	170 (140-240)	113 (93-160)	85 (70-120)
Follow-up	13	26 (17-50)	520 (340-1000)	260 (170-500)	173 (113-333)	130 (85-250)
Attitudes that encourage bullying
End of intervention	14	29 (17-71)	580 (340-1420)	290 (170-710)	193 (113-473)	145 (85-355)
Follow-up	5	28 (19-55)	560 (380-1100)	280 (190-550)	187 (127-367)	140 (95-275)
Mental health problems
End of intervention	15	16 (11-26)	320 (220-520)	160 (110-260)	107 (73-173)	80 (55-10)
Follow-up	5	16 (8-118)	320 (160-2360)	160 (80-1180)	107 (53-787)	80 (40-590)
School climate
End of intervention	9	52 (37-88)	1040 (740-1760)	520 (370-880)	347 (247-587)	260 (185-440)
Follow-up	4	28 (14-1190)	560 (280-23 800)	280 (140-11 900)	187 (93-7933)	140 (70-5950)

Abbreviations: k, number of samples; NA, not applicable; NNT, number needed to treat; PIN, population impact number (defined as children in the total population for whom 1 event may be prevented by an intervention).

^aOverall bullying is a pooled measure, including data on bullying perpetration, bullying exposure, and cyberbullying.

^bCyberbullying includes pooled cyberbullying perpetration and cyberbullying exposure data.

Figure 3. PIN of Universal School Anti-Bullying Interventions for a Bullying Prevalence of 15%.

Population impact number (PIN) is defined as children in the total population for whom 1 event may be prevented by an intervention. Bars represent means, and vertical lines represent upper and lower 95% CIs for each category.

^aOverall bullying is a pooled measure, including data on bullying perpetration, bullying exposure, and cyberbullying.

^bCyberbullying includes pooled cyberbullying perpetration and cyberbullying exposure data.

Effectiveness of Anti-Bullying Interventions Over Time

Anti-bullying interventions were effective in reducing overall bullying during a mean 44.0-week follow-up (k = 21; effect size, −0.171; 95% CI, −0.243 to −0.099; P < .001). Details are shown in eFigure 2 in the Supplement.

At a mean 30.9-week follow-up, the effectiveness of anti-bullying interventions remained statistically significant for all bullying-related outcomes except for cyberbullying, for which only 1 RCT was available. The effect sizes were small (range, 0.122-0.202) (Table 1). The effectiveness of anti-bullying interventions was also statistically significant in decreasing mental health problems at a mean 27.3-week follow-up, with a small effect size (k = 6; effect size, −0.202; 95% CI, −0.347 to −0.056; P = .01) (eFigure 3 in the Supplement).

The population impact of universal anti-bullying interventions for overall bullying (mean, 44.0-week follow-up) and mental health problems (mean, 27.3-week follow-up) was 127 (95% CI, 87-247) and 107 (95% CI, 53-787), respectively, assuming a bullying prevalence of 15%. Table 2 and Figure 3 summarize details on NNT and PIN values for universal anti-bullying interventions at follow-up.

Meta-regression Analyses

Meta-regression analyses found that (1) duration of anti-bullying programs was not statistically significantly associated with effectiveness, (2) there was no statistically significant association between the length of follow-up and the effectiveness of anti-bullying programs at follow-up (mean [range] follow-up, 30.9 [2-104] weeks), and (3) no additional factor moderated the effectiveness of anti-bullying or anti-cyberbullying programs. The results of the meta-regression analyses are summarized in eTable 9 in the Supplement.

Discussion

Bullying is a major public health problem worldwide.^4,24 This meta-analysis shows that school anti-bullying interventions are statistically significantly effective not only in reducing bullying rates but also in improving mental health problems in young people. Despite the small effect sizes and some regional differences in effectiveness, the findings suggest that universal anti-bullying interventions have a substantial population impact.

Universally delivered psychosocial interventions in adolescence have proven to be effective in improving mental health and reducing risk behavior, including bullying.⁹³ The results of the present study add to previous meta-analyses^{27,28,29,30,93} on this topic, which found that anti-bullying interventions had statistically significant effectiveness by specifically assessing the population impact of the interventions. For an estimated bullying prevalence of 15% (a conservative estimate considering prevalence rates reported in previous studies^10,94), an average anti-bullying intervention needs to include 207 (95% CI, 153-307) people to prevent 1 case of bullying perpetration or 140 (95% CI, 93-260) people to prevent 1 case of bullying exposure and 107 (95% CI, 73-173) people to improve mental health problems. We also found a substantial population impact (PIN, 167; 95% CI, 100-360) for interventions targeting cyberbullying (ie, 167 young people on average need to receive the intervention to prevent 1 case of cyberbullying perpetration or exposure). To put these results into context, the PIN is 35 450 for taking aspirin to avoid 1 death during the 6 months after a first nonhemorrhagic stroke,⁹⁵ and the PIN is 324 for human papillomavirus vaccination in girls to prevent cervical cancer.⁹⁶

Therefore, the findings of the present study have implications for public health given the adverse effects of bullying and cyberbullying on the mental and physical health of those involved^{9,14,17,18,19,20,97} and the cost-effectiveness of bullying prevention programs.^7,25 In this regard, a recent cost-effectiveness analysis of KiVA, a Finnish school-based anti-bullying program delivered by teaching staff, estimated net savings of $66 172 for a cohort of 200 pupils through age 50 years.⁷ This finding is further evidence that bullying prevention interventions should become a priority for universal primary prevention in mental health.¹

Universal anti-bullying interventions were found to be at least as effective as targeted interventions, if not more so, thus providing additional support for a universal approach as a first tier in school bullying prevention. Further research should address whether combining universal and targeted anti-bullying interventions is associated with greater effectiveness, especially in specific subgroups of children with greater needs, who may benefit less from universal school interventions.⁹⁸

We found that duration of intervention (mean duration of interventions, 29.4 weeks; range, 1 session to 144 weeks) was not statistically significantly associated with effectiveness of anti-bullying interventions. This result is consistent with a previous meta-analysis²⁸ that found no differences in the effectiveness of anti-bullying programs for prevention of school violence and bullying exposure between programs with a duration longer vs shorter than 1 year. Because most of the interventions included in the present meta-analysis lasted less than 1 year and some were as short as a few weeks, our findings suggest that short school interventions spanning less than 1 school year may be sufficient to substantially reduce bullying rates and improve mental health in young people, thus supporting their applicability in wider clinical contexts.

One of the key issues in the field of intervention in psychiatry and psychology, including preventive interventions like anti-bullying programs, is the durability of treatment effects.^99,100,101 This meta-analysis shows that the effectiveness of the intervention does not diminish over time after the end of the intervention (mean [range] follow-up, 30.9 [2-104] weeks), thus suggesting that there is long-term effectiveness, with a relevant population impact (PIN, 127; 95% CI, 87-247), and that there are no statistically significant differences between effect sizes at the end of the intervention and after follow-up ranging from 2 to 104 weeks. It remains unclear whether treatment effectiveness can be convincingly maintained over longer periods, whether longer-term interventions or booster sessions might provide additional benefits, and the specific factors that might be associated with a sustained response.

Despite the favorable results of anti-bullying interventions and the fact that these programs are traditionally considered to be low-risk interventions, it should not be presumed that school interventions targeting bullying are always safe.¹⁰² Along these lines, the findings offer encouraging data on the safety of anti-bullying programs. None of the selected RCTs reported an increase in either bullying perpetration or bullying exposure at study end point or follow-up. Furthermore, mental health improved in all trials that assessed this outcome, both at study end point and follow-up. Therefore, our results suggest that anti-bullying interventions are not only efficacious but also safe.

This meta-analysis found no differential factors moderating the effectiveness of anti-bullying and anti-cyberbullying programs. There is still ongoing debate about whether cyberbullying is categorically distinct from traditional bullying and about the role of common and differential factors associated with both types of bullying.^103,104,105 Although bullying and cyberbullying may be mediated by some differential factors, such as emotional problems and the personality of bullies,¹⁰⁶ there is no clear cutoff between moderators of bullying and cyberbullying. This observation could be at least partially owing to the fact that a high proportion of cyberbullying recipients also experience traditional bullying.^107,108 However, this finding could also be a consequence of the difficulty of researching the online world and comparing it with the off-line world.^105,109

Consistent with a previous meta-analysis,²⁷ we found some differences in the effectiveness of interventions in trials conducted in Europe vs North America. In both regions, anti-bullying programs showed comparable effectiveness in decreasing overall bullying and bullying perpetration at study end point and were statistically significantly efficacious in decreasing attitudes that encourage bullying and improving mental health problems, albeit with smaller effect size in the subgroup of trials conducted in North America. However, the effectiveness in decreasing bullying exposure and promoting attitudes that discourage bullying at study end point was statistically significant only in the subgroup of trials conducted in Europe. Reduced effectiveness for some outcome measures in trials performed in North America vs Europe may be the result of methodological heterogeneity (eg, differences in specific programs or study design) or statistical heterogeneity among trials, which was high for RCTs in both regions. However, these findings may also reflect differences in social, educational, or cultural context that could be incorporated into the design of anti-bullying programs. Further studies should also try to clarify the potential reasons underlying differing effectiveness for some outcome measures in North America.

Limitations

This study had several limitations. First, there was substantial methodological heterogeneity among selected RCTs in terms of intervention characteristics, measures of bullying exposure and other outcome variables, sample characteristics, and social context. Additional sources of heterogeneity included differences in study quality and statistical heterogeneity among RCTs, which was high for most outcome variables. Second, few RCTs assessed the same specific anti-bullying program. Therefore, we could only assess the effectiveness of anti-bullying programs as a whole. It is known that not all anti-bullying programs are efficacious and that effectiveness may vary in different settings.^29,99 Meta-regression analyses were conducted to investigate the association of study-related factors. However, the results did not allow us to identify which program works in a specific context. Third, most outcome variables were not clearly defined in the original RCTs, and they were highly heterogeneous, with few RCTs assessing the same outcome variable. Therefore, it was necessary to classify and subsume specific outcome measures into outcome groups, and there may be some degree of overlap and heterogeneity in terms of internal validity among the resulting categories. Fourth, the original RCTs did not report the presence (or rates) of mental, intellectual, or physical disabilities in sample populations. This limitation may have altered both rates of bullying and bullying exposure and intervention effectiveness.^110,111 Fifth, despite our comprehensive search in databases covering the main scientific fields relevant to this meta-analysis, we restricted our search to RCTs published in peer-reviewed journals. Although this strategy may have limited the representativeness of our search to some extent, we made this decision to ensure a minimum quality of the included trials in an attempt to increase the validity of our results. Sixth, none of the RCTs reported concomitant individual interventions, which may have altered the effectiveness of the anti-bullying programs.

Conclusions

Despite the small effect sizes and some regional differences in effectiveness, with greater effectiveness for some outcomes in trials conducted in Europe vs North America, this meta-analysis suggests that school anti-bullying interventions have valuable population impact. This meta-analysis also highlights the need for better designed trials that assess the factors associated with the effectiveness of anti-bullying programs, including optimal timing and duration of interventions, their essential components, and the mediating association between bullying prevention and improvement in mental health problems. Trials should also specifically test targeted interventions in vulnerable populations at higher risk for bullying exposure, such as those living with disabilities and LGBTQ (lesbian, gay, bisexual, transgender, queer; lesbian, gay, bisexual, transgender, questioning) youth.^112,113 These studies could inform more efficacious bullying prevention programs that promote a reduction in bullying rates and improve global and mental health.

Supplement.

eTable 1. PRISMA Checklist

eTable 2. Classification of Outcome Variables

eTable 3. Quality Assessment

eTable 4. Characteristics of the Included Studies (Part 1)

eTable 5. Characteristics of the Included Studies (Part 2)

eTable 6. Meta-analyses of Efficacy of Randomized-Controlled Trials Testing Anti-Bullying School Interventions per Region

eTable 7. Population Impact Number of Universal Anti-Bullying School Interventions per Region

eTable 8. Meta-analyses of Efficacy of Randomized-Controlled Trials Testing Universal or Targeted Anti-Bullying School Interventions

eTable 9. Meta-Regression Analyses (Association Between Variables and Effect Size of Anti-Bullying Interventions)

eMethods. Supplemental Methods

eFigure 1. PRISMA Flow Diagram of the Systematic Literature Search Strategy

eFigure 2. Efficacy of Anti-Bullying Interventions on Overall Bullying at Study Follow-up

eFigure 3. Efficacy of Anti-Bullying Interventions on Mental Health Problems at Study Follow-up

eReferences.