Evaluating Litigation as a Structural Strategy for Addressing Bias-Based Bullying Among Youth

Key Points

Question

Does litigation related to sexual orientation–based harassment and discrimination in schools reduce rates of homophobic bullying?

Findings

This quasi-experimental study included 1 448 778 students in 499 California high schools. For cases where the plaintiff (student) secured a remedy (ie, monetary and/or injunctive relief) through settlement or court decision, there was a 23% reduction in homophobic bullying in schools experiencing litigation relative to schools that did not experience litigation; this effect size was comparable with that of resource-intensive school-based bullying interventions.

Meaning

Litigation may reduce homophobic bullying, depending on the outcome of the case.

This quasi-experimental study examines whether litigation is associated with reductions in homophobic bullying among students in California high schools.

Abstract

Importance

Homophobic bullying—which is motivated by actual or perceived sexual orientation—is a common experience among youth and is more strongly associated with adverse outcomes than bullying unrelated to bias. Yet current approaches to reducing homophobic bullying either lack empirical evidence or encounter significant obstacles. Thus, the field requires the identification of strategies that hold promise for reducing homophobic bullying.

Objective

To examine whether litigation is associated with reductions in homophobic bullying.

Design, Setting, and Participants

In this quasi-experimental study, difference-in-difference analysis was used to estimate the association between litigation and homophobic bullying, comparing students in schools that experienced litigation with students in schools that did not experience litigation, controlling for individual and school characteristics, study year, and county. Survey responses came from high school students from 499 schools participating in the California Healthy Kids Survey, the largest statewide survey of youth risk behaviors and protective factors, between 2001 and 2016. Legal data were collected from September 2018 to September 2019, and data were analyzed from February 2020 to April 2021.

Exposures

Outcomes of litigation related to sexual orientation–based harassment and discrimination in California schools occurring after 2000.

Main Outcomes and Measures

Student reports of homophobic bullying.

Results

Of 1 448 778 included participants, 706 258 (48.7%) were male, 563 973 (38.9%) were White, and the mean (SD) age was 14.6 (1.7) years. For cases where the plaintiff (student) secured monetary and/or injunctive relief through settlement or court decision, there was a 23% reduction in the ratio of odds ratios (ROR) of homophobic bullying in schools directly involved in the litigation relative to schools that did not experience litigation (ROR, 0.77; 95% CI, 0.68-0.86). These benefits of litigation spilled over into schools in the same district as the schools experiencing litigation (ROR, 0.76; 95% CI, 0.70-0.81). However, homophobic bullying slightly increased in the school and district where the defendant (school) avoided adverse legal consequences, suggesting potential backlash.

Conclusions and Relevance

Litigation seeking to address alleged violations of the rights of students who are (or are perceived to be) lesbian, gay, bisexual, or transgender under laws prohibiting harassment or discrimination may lead to reductions in rates of homophobic bullying, with effect sizes comparable with that of resource-intensive school-based bullying interventions. These findings set the stage for future studies to evaluate the consequences of different litigation efforts aimed at redressing stigma-based harms among youth.

Introduction

Bias-based bullying, defined as attacks motivated by membership in a marginalized group,¹ is common among youth, with an estimated 40% of high school students reporting at least 1 instance of bias-based bullying in the past 12 months.² Compared with general forms of bullying, homophobic bullying—a subtype of bias-based bullying—has been linked to significantly worse psychosocial and academic outcomes.^3,4,5 Given the prevalence and negative consequences associated with homophobic bullying, reducing it represents an important public health priority. Yet studies of universal, school-based bullying prevention programs rarely focus on bias and seldom include measures of homophobic bullying⁶; thus, it is unknown whether these programs are effective in reducing homophobic bullying, as they are for bullying unrelated to bias.⁷ Further, addressing homophobic bullying requires the backing of teachers and educational support professionals; however, on average, they report feeling the least comfortable intervening with homophobic bullying compared with bullying related to other issues (eg, race).⁸

Because these existing approaches either lack empirical evidence (school-based programs) or encounter significant obstacles (teacher support), we focus on litigation as an alternative strategy for reducing homophobic bullying. Litigation is a widely used tool for addressing stigma-based harms related to sexual orientation (eg, Bostock v Clayton County),⁹ race (eg, Brown v Board of Education),¹⁰ and sex and gender (eg, Price Waterhouse v Hopkins).¹¹ While there is emerging evidence that litigation can change public attitudes toward stigmatized groups,^12,13 the extent to which litigation affects stigmatizing behaviors remains largely untested.

Addressing this question requires a unique data structure that combines information on the outcomes of multiple court cases with individual-level data on stigmatizing behaviors that have been repeatedly assessed in real-world settings. Because such a data structure did not exist, we created one for this study. We did so by coding the outcomes of litigation addressing alleged violations of the rights of students who are (or are perceived to be) lesbian, gay, bisexual, or transgender (LGBT) under laws prohibiting harassment or discrimination in California schools after 2000, which were linked to survey data on homophobic bullying from 1 448 778 California high school students in 499 schools. We examined whether litigation was associated with reductions in homophobic bullying measured within the same school during the case and after it was resolved. Additionally, we evaluated whether these reductions were observed in schools within the same districts where the case arose, permitting the ascertainment of whether the benefits of litigation spill over into surrounding schools, as anticipated by legal theories of impact litigation.^14,15 Finally, given some concerns in the legal literature regarding the potential for backlash following litigation,¹⁶ we examined whether the results differed based on the outcome of the litigation.

Methods

Data Sources

Legal Cases

Our search of litigation used the following inclusion criteria: (1) the case occurred after 2000 in California; (2) one of the parties was a school or local education agency; (3) another party was a student or parent; (4) the student experienced bullying, discrimination, harassment, and/or other adverse treatment; (5) the student was (or was perceived to be) LGBT; and (6) the school failed to adequately respond to and/or contributed to the adverse treatment. This search produced 31 unique cases. We tracked the case name, date the case was filed, date the last legal document dealing with substantive issues was issued, school and district being sued, and outcome of the case (ie, whether the plaintiff [student] secured monetary and/or injunctive relief), which was determined from the court decision (ie, judicial ruling or jury verdict) or settlement agreement. The legal content analysis was completed by 2 independent legal coders (R.K. and S.L.) and demonstrated strong interrater reliability (κ = 0.90). Disagreements were resolved by consensus with the first author (M.L.H.). eAppendix 1 in the Supplement provides further details of the search and coding.

Survey Data

We obtained 15 consecutive waves of data from the California Healthy Kids Survey (CHKS), the largest statewide survey of youth risk behaviors and protective factors, collected between the 2001-2002 and 2015-2016 academic years. Mean student response rates were typically greater than 70%.¹⁷ The CHKS is a serial cross-sectional survey at the school level.¹⁸ Using school-level identifiers, we linked the legal database to the CHKS; cases that could not be linked to the CHKS (n = 8 schools) or that were missing key indicators (n = 11 schools) were excluded (eAppendix 2 and the eFigure in the Supplement).

The CHKS uses passive (opt-out) parental consent and youth assent. The study received approval from the Institutional Review Board at the University of Texas at Austin.

Exposure and Outcome

Students were coded as being enrolled in case schools (ie, the subject of litigation), spillover schools (ie, schools in districts that were subject to litigation but not the schools where the case arose), or control schools (ie, schools that were neither the subject of litigation nor in the same district as case schools) (Figure 1). Control schools were drawn from the same counties as case and spillover schools. eAppendix 2 in the Supplement provides further details about operationalizing exposure status.

Figure 1. Schematic of School Indicators for Designing Case, Control, and Spillover Schools.

The CHKS queried homophobic bullying via the question, “During the past 12 months, how many times on school property have you been harassed or bullied because you are gay or lesbian, or because someone thought you were?” Response options included never, once, 2 to 3 times, and 4 or more times. Responses were highly left-skewed, with most students indicating no homophobic bullying. Thus, a dichotomous measure was created for each respondent (ie, ever vs never).

Covariates

We controlled for student age, sex, race and ethnicity, and survey year. At the school level, we controlled for total enrollment (obtained from the California Department of Education), because enrollment is related to multiple student outcomes, may make schools more or less vulnerable to litigation, and could affect resource allocation postlitigation.^19,20,21,22 We additionally controlled for the presence of Genders and Sexualities Alliances (GSA; obtained from the GSA Network) to account for reductions in homophobic bullying owing to other contemporaneous factors, such as positive climates for LGBT youth already present in schools.^23,24 Both school-level covariates were modeled as time varying. We adjusted for time-invariant sources of bias owing to county-level differences by controlling for county.

Statistical Analysis

We used a difference-in-difference (DID) approach, a quasi-experimental design, to estimate associations between litigation and student reports of homophobic bullying. This approach involves comparing before-and-after changes in the probability of homophobic bullying among students in case schools to before-and-after changes in the same outcome among students in control schools.^25,26,27 All observations were measured at the student level after confirming that no clustering was present at the school, district, or county level (intraclass correlation for each unit less than 0.01).

For DID analyses to be valid, the parallel trends assumption must be met; that is, trends in homophobic bullying must be parallel between case and control schools prior to the litigation. To test this assumption, we estimated a logistic regression model, with homophobic bullying as the outcome and the exposure defined as the interaction term consisting of linear year and whether the school had litigation, controlling for all covariates.

Because both the filing and resolution of a case may plausibly influence homophobic bullying, schools were examined at 3 time periods: the years before the case (reference), the years after the cases were filed (ie, during case), and the years after the cases were resolved (ie, after case resolution). Analyses were restricted to students who had nonmissing responses for homophobic bullying and attended schools that had student observations at all 3 time periods.

The analytic sample of 1 448 778 respondents represented 10 case schools (n = 24 482 respondents) and 446 control schools (n = 1 338 612 respondents) in 8 counties, as well as 43 spillover schools (n = 85 684 respondents) (Table). Case start dates ranged from 2004 to 2012, and case resolution dates ranged from 2007 to 2014. This time-varying pattern of litigation with staggered start and resolution dates allows for a quasi-experimental, multigroup design that reduces many concerns about confounding in observational studies.^28,29

Table. Summary Characteristics of Analytic Sample.

Characteristic	No. (%)			P value
	Students in case schools (n = 24 482)	Students in spillover schools (n = 85 684)	Students in control schools (n = 1 338 612)
Individual covariates
Age, mean (SD), y	14.3 (1.8)	14.2 (1.8)	14.6 (1.7)	<.001
Race
American Indian/Alaska Native	545 (2.2)	1806 (2.1)	33 782 (2.5)	<.001
Asian	2048 (8.4)	20 491 (23.9)	148 480 (11.1)
Black	586 (2.4)	2067 (2.4)	46 964 (3.5)
Latinx	3387 (13.8)	11 018 (12.9)	219 073 (16.4)
Native Hawaiian/Pacific Islander	392 (1.6)	2133 (2.5)	27 118 (2.0)
White	12 079 (49.3)	28 082 (32.8)	523 812 (39.1)
Multiracial	4457 (18.2)	16 921 (19.7)	290 912 (21.7)
Other	988 (4.0)	3166 (3.7)	48 471 (3.6)
Sex
Male	11 909 (48.6)	41 674 (48.6)	652 675 (48.8)	.75
Female	12 573 (51.4)	44 010 (51.4)	685 937 (51.2)
School covariates
Schools, No.	10	43	446	NA
GSA present	9558 (39.0)	12 067 (14.1)	714 105 (53.3)	<.001
Total enrollment, mean (SD)	1868 (903)	1663 (704)	1640 (803)	<.001

Abbreviations: GSA, Genders and Sexualities Alliances; NA, not applicable.

We next examined DID analyses stratified by the outcome of the litigation (eAppendix 2 in the Supplement) and tested whether stratified estimates diverged (ie, whether the eventual outcome moderated the association between litigation and homophobic bullying) through DID analyses, with case schools further categorized by the litigation outcome. To assess the potential for spillover into other schools in the same district as case schools, we ran DID analyses comparing before-and-after changes in the probability of homophobic bullying among students in spillover schools with students in control schools.

We conducted several additional sets of supplementary analyses to further triangulate evidence. First, we examined associations between litigation and student reports of bullying related to other forms of bias (race and ethnicity, religion, or gender) but not to sexual orientation. Second, we conducted a falsification test for an outcome that litigation would not be expected to influence, occasions of fruit juice consumption, which has been used in prior policy studies.³⁰ A null association between litigation and these 2 outcomes would provide additional support that omitted variables were not driving our results (eAppendix 3 in the Supplement). Third, we used dummy indicators for survey year (rather than modeling year as a continuous variable), which allow time effects on outcomes to be flexible from year to year.²⁷ Fourth, to ensure that the unbalanced design (ie, different case dates and lengths of observation) did not spuriously bias results, we modeled on a selection of student observations from the wave immediately prior to case filings, throughout the case, and the wave immediately after case resolution.

Finally, we evaluated the potential for selection bias and residual confounding in 2 ways. First, we examined whether 103 individual-level, school-level, and county-level characteristics were associated with the exposure (case status) or outcome (homophobic bullying). Second, we examined whether the case schools included in our analytic sample systematically differed from the case schools that were not included by examining correlations between inclusion in the analysis and school-level demographic features available for 30 of the candidate schools from the California Department of Education.

eAppendix 4 in the Supplement describes tests of assumptions for all models. We used complete case analysis, restricting the sample to observations with fully observed covariate information (1 448 778 of 1 655 145 eligible students [87.5%]). Statistical code for all models is shown in eAppendix 5 in the Supplement. Statistical significance was set at α = .05 (using a 2-tailed distribution) and assessed from Wald tests of regression coefficients. All analyses were performed using R version 4.0.5 (The R Foundation).

Results

Changes in Rates of Homophobic Bullying in Case Schools

Of 1 448 778 included participants, 706 258 (48.7%) were male, 563 973 (38.9%) were White, and the mean (SD) age was 14.6 (1.7) years. Trends in the model-based predicted probability of homophobic bullying did not differ between students in case and control schools in the years prior to the case (Figure 2), indicating that there were not preexisting differences in trends in the outcome between the students in case and control groups.

Figure 2. Test of Parallel Trends Assumption Comparing Homophobic Bullying in Case Schools With Control Schools in the Years Prior to the Case.

Figure 3 shows the odds ratios comparing case schools with control schools at each time point and the exponentiated DID estimates (eTable 1 in the Supplement). Compared with the years before the case, students in case schools vs control schools evidenced a nonsignificant 5% reduction in the ratio of the odds ratios (ROR) of homophobic bullying in the years during the case (ROR, 0.95; 95% CI, 0.86-1.06) but a 16% reduction after the case resolution (ROR, 0.84; 95% CI, 0.76-0.93).

Figure 3. Results From Multivariable Difference-in-Difference Regression Models.

Unstratified by case outcome corresponds to models examining case schools compared with control schools as well as spillover schools compared with control schools; plaintiff secured a remedy corresponds to subgroups restricted to cases where the plaintiff (student) secured a remedy (ie, monetary or injunctive relief); and defendant avoided adverse legal consequences corresponds to subgroups restricted to the single case where the defendant (school or district) avoided adverse legal consequences. We modeled the outcome using binomial logistic regression, with the following specification: logit (Pr[homophobic bullying = 1] = β₀ + β₁ (case school) + β₂ (during case) + β₃ (after case resolution) + β₄ (case school × during case) + β₅ (case school × after case resolution) + β₆ (age) + β₇ (non-Hispanic White) + β₆ (Genders and Sexualities Alliances presence at school) + β₇ (survey year) + β₈ (total students enrolled at school) + β_9-15 (county dummy). The model-based difference-in-difference estimators presented here correspond to β₃ and β₄, and when exponentiated are interpreted as the ratio (comparing case schools with control schools) of the odds ratios (ORs; comparing during case and after case resolution with before case).³¹ Covariates are measured at each survey wave and include the presence of Genders and Sexualities Alliance, total school enrollment (continuous), student age (continuous), student race (White or non-White), student sex, county (categorical), and survey year. County dummy variables are not included in the defendant avoided adverse legal consequences model, as all schools were in one county. The dashed blue line depicts the OR for homophobic bullying in the before case period for case schools vs control schools (A) and for spillover schools vs control schools (B). ROR indicates ratio of odds ratios.

We next stratified the results by the outcome of the litigation. For students in case schools vs control schools where the plaintiff secured monetary and/or injunctive relief, there was a nonsignificant 4% reduction in the ROR of homophobic bullying in the years during the case (ROR, 0.96; 95% CI, 0.85-1.08) but a 23% reduction in the ROR of homophobic bullying after the case was resolved (ROR, 0.77; 95% CI, 0.68-0.86) (Figure 3). For students in case schools vs control schools where the defendant (school or district) avoided adverse legal consequences, there was a nonsignificant 10% reduction in the ROR of homophobic bullying in the years during the case (ROR, 0.90; 95% CI, 0.61-1.32); however, this association was reversed in the years after the case resolution, where students experienced a nonsignificant 27% increase in the ROR of homophobic bullying (ROR, 1.27; 95% CI, 0.88-1.83) (Figure 3).

Changes in Rates of Homophobic Bullying in Spillover Schools

Analyses conducted in spillover schools produced results that were consistent with those of case schools (Figure 3). For instance, in districts where the plaintiff secured monetary and/or injunctive relief, students in spillover schools vs control schools experienced a 24% reduction in the ROR of homophobic bullying in the years after the case resolution (ROR, 0.76; 95% CI, 0.70-0.81) compared with a 23% reduction in case schools (ROR, 0.77; 95% CI, 0.68-0.86). In contrast, in districts where the defendant avoided adverse legal consequences, students in spillover schools vs control schools experienced a nonsignificant 7% increase in the ROR of homophobic bullying in the years after the case resolution (ROR, 1.07; 95% CI, 0.98-1.18) compared with a 27% increase in case schools (ROR, 1.27; 95% CI, 0.88-1.83). Tests of interaction by litigation outcome showed that estimates significantly diverged after case resolution for both case and spillover schools.

Supplementary Analyses

Litigation was unrelated to (1) bullying owing to forms of bias other than sexual orientation (eTable 2 in the Supplement) and (2) fruit juice consumption (eTable 2 in the Supplement), indicating the associations were specific to homophobic bullying. Results were consistent when time was modeled as a dummy variable (eTable 3 in the Supplement) and when analyses accounted for the unbalanced design (eAppendix 6 and eTable 4 in the Supplement), indicating that neither of these factors introduced bias.

Addressing Potential Selection and Residual Confounding

No individual-level, school-level, or county-level characteristics were correlated by more than 10% with being a case school or more than 14% with homophobic bullying (eAppendix 7 and eTable 5 in the Supplement). Additionally, only 4 school-level characteristics (eg, lower proportions of students from racial and ethnic minority groups) differed between case schools included in the analysis and those that were excluded (eAppendix 7 and eTable 6 in the Supplement), and none of these were associated with homophobic bullying. These analyses suggest minimal evidence for selection into case schools or for residual confounding.

Discussion

Using a quasi-experimental design with a sample of nearly 1.5 million youth in 499 California schools, we provide evidence that litigation may reduce rates of homophobic bullying. Students in schools involved in cases where the plaintiff (student) secured monetary and/or injunctive relief evidenced a 23% reduction in the ROR of homophobic bullying after the case was resolved relative to schools not involved in litigation (ROR, 0.77; 95% CI, 0.68-0.86). To provide some context for this effect size, a large meta-analysis showed that universal school-based bullying prevention programs reduce bullying rates by 20%.⁷ The effect sizes we observed for litigation are comparable with these programs, suggesting that litigation can meaningfully shape behaviors associated with stigma-based inequalities.

Impact litigation seeks to change behaviors of institutional actors beyond those involved in the immediate lawsuit.^14,15 Consistent with this goal, we found that the benefits of litigation spill over into schools within the same district. The effect sizes for students in spillover schools were comparable with those observed in case schools, which likely occurred because any required changes resulting from the litigation typically were implemented at the district level.

In the case where the defendant (school or district) avoided adverse legal consequences, however, rates of homophobic bullying increased among students in both the case school and in schools in the same district (ie, spillover schools) during the years after the case was resolved. Although the effect size (nonsignificant 27% increase [ROR, 1.27; 95% CI, 0.88-1.83) suggests the potential for backlash, these results should be interpreted with caution given the small number of schools.

Separating the case filing year from the resolution year enabled us to determine whether merely bringing a case (before the outcome is known) is sufficient to produce changes in homophobic bullying. We observed similar nonsignificant, modest decreases in homophobic bullying among case schools during litigation regardless of the ultimate outcome, suggesting that being the subject of litigation puts schools on notice. However, substantive changes in homophobic bullying only occurred after the cases resolved, manifesting either as a protective association or a potential backlash.

We examined average associations between litigation and homophobic bullying, pooling across cases with different remedies and across schools with different observation periods following case resolution. Thus, future research should examine sources of variability in this association—including whether the amount of monetary relief and the type and scope of injunctive remedies predict heterogeneity in homophobic bullying, which will reveal specific aspects of litigation that are most strongly associated with changes in this outcome. Additional sources of heterogeneity that warrant future study include whether changes in homophobic bullying persist or desist over time following litigation.

Limitations

This study has several limitations. First, despite strengths of the large, diverse sample, the CHKS is not representative of California high school students. However, in a previous study using a representative subsample of the CHKS, which was unavailable during our study years, the prevalence of being the target of peer aggression was nearly identical between the representative and full CHKS samples.³² Second, we were unable to use the full sample of relevant litigation during our study period, which may influence generalizability. However, we found minimal evidence of selection bias, as none of the school-level features that differed between our analytic sample and the full universe of cases were related to homophobic bullying. Third, a strength of the DID design is that unmeasured confounding is minimized, assuming model assumptions are met.²⁷ While residual confounding by unobserved, time-varying characteristics could still lead to biased estimates, 2 supplementary findings suggest this is unlikely: the specificity and falsification tests showed that litigation was unrelated to changes in other types of bias-based bullying or in fruit juice consumption, respectively. Additionally, no individual-level, school-level, or county-level measure was meaningfully associated with either exposure or outcome status.

Conclusions

Although homophobic bullying represents a significant public health problem, existing approaches have limited empirical support.^6,8 The current study addresses this knowledge gap by demonstrating that litigation addressing alleged violations of the rights of students who are (or are perceived to be) LGBT under laws prohibiting harassment or discrimination may reduce rates of homophobic bullying. Our approach provides a template for future research focused on determining the consequences of antidiscrimination litigation across a range of issues related to social inequalities among stigmatized youth³³ (eg, Title IX, school discipline), which will also help reveal potential boundary conditions of our results. At the same time, our data suggest that litigation may also engender backlash in some circumstances, consistent with concerns raised by legal scholars.^16,34 Further, legal scholarship has shown that most bullying cases are not found in favor of the student plaintiff.³⁵ Identifying strategies for reducing homophobic bullying beyond litigation therefore represents an important area for future inquiry.

Supplement.

eAppendix 1. Legal content analysis.

eAppendix 2. Strategy for identifying exposure status.

eFigure. Flow chart of eligible cases, number of corresponding eligible case schools, and reasons for exclusion.

eAppendix 3. Specificity and falsification outcomes.

eAppendix 4. Testing difference-in-difference assumptions.

eAppendix 5. R code for main and supplemental analyses.

eTable 1. Results from multivariable difference-in-difference regression models.

eTable 2. Results from multivariable difference-in-difference regression models, specificity and falsification outcomes.

eTable 3. Results from multivariable difference-in-difference regression models, using dummy variables for survey year.

eAppendix 6. Sensitivity analysis with balanced data.

eTable 4. Results from balanced sensitivity analysis.

eAppendix 7. Addressing unmeasured confounding, potential selection, and generalizability.

eTable 5. Correlations across individual, school-level, and county-level features with intervention and with outcome.

eTable 6. Correlations among school-level demographic features from the California Department of Education and inclusion in study among schools with legal cases.