Effects on School Outcomes in Low-Income Minority Youth: Preliminary Findings from a Community-Partnered Study of a School Trauma Intervention

Abstract

Objective

To examine academic outcomes of a community-partnered school mental health intervention for students who have been exposed to community violence.

Design

Randomized controlled trial.

Setting and Participants

Sixth-grade students (n=123) from 2 middle schools in Los Angeles during the 2001–2002 academic year who had exposure to violence and posttraumatic stress symptoms in the clinical range.

Intervention

Students were randomized to either receive a 10-session standardized school trauma intervention (Cognitive Behavioral Intervention for Trauma in Schools) soon after screening (Early Intervention) or after a delay following screening (Delayed Intervention), but within the same school year.

Main Outcome Measures

59 students in the Early Intervention group vs. 64 students in the Delayed Intervention group (screened in September or December) were compared on Spring Semester grades in math and language arts, controlling for the students' standardized state test scores from the previous academic year and other covariates.

Results

Students in the Early Intervention group had a significantly higher Spring Semester mean grade in math (2.0 vs.1.6) but not language arts (2.2 vs. 1.9). Students in the Early Intervention group were more likely than students in the Delayed Intervention group to have a passing grade (“C” or higher) in language arts (80% vs. 61%; p<0.033) by Spring Semester but not in math.

Conclusion

Through a collaborative partnership between school staff and researchers, preliminary evidence suggests that receiving a school trauma intervention soon after screening compared to delaying treatment can result in better school grades.

Studies have documented the broad range of negative sequelae of youth violence exposure, including posttraumatic stress disorder (PTSD) and other anxiety problems, depressive symptoms, and dissociation.^1–4 If anxiety becomes chronic, it can disrupt children's ability to regulate emotional states, leading to hypervigilance, emotional numbing, and inattentiveness. Youth exposed to violence have decreased social competence and increased rates of peer rejection,⁵ as well as decreased IQ and reading ability, lower grade-point average (GPA), more days of school absence, and decreased rates of high school graduation.^{6, 7} Cumulative life stressors in childhood, including exposure to violence, can lead to poor employment productivity, social relationships, and health in adulthood.⁸

To mitigate the long-term effects of violence exposure, effective psychosocial interventions for trauma-related symptoms in youth have been developed;^{9, 10} however, few youth, especially low-income minority youth, receive early interventions to prevent negative developmental outcomes from exposure to violence. As the Surgeon General's National Action Agenda for Children's Mental Health reports,¹¹ providing social emotional support in schools can help address issues of access to care for many children. School-based services may be particularly important for underserved ethnic minority youth who traditionally are less likely to receive such services. For example, a randomized study of effective treatments for youth with posttraumatic stress symptoms in post-Katrina New Orleans found that 91% of the youth completed the school-based intervention compared to only 15% who completed the clinic-based intervention.¹² Another study found that in contrast to ethnic disparities found among children accessing clinic based services, there were no differences across ethnic groups in youth seeking school-based counseling services.¹³

To address the disparities in care for low income ethnic minority youth with exposure to violence, a community-research partnership developed, implemented, and evaluated the Cognitive Behavioral Intervention for Trauma in Schools (CBITS) program, a skills-building approach to improve the well-being of traumatized students. This partnership, initiated by the community partners (author M.W.) who had identified the needs of students exposed to violence as a priority for service delivery, involved Latino bilingual bicultural school clinicians, school administrators, and clinician-researchers. The community partners played a leading role in the partnership, allocating district resources to create a program that could be delivered by school clinicians, work within the confines of schools, and ultimately improve care for students exposed to violence.¹⁴ School partners defined the operational parameters of the program. An after-school version was piloted to minimize missed class time, but school partners found that few students in these urban neighborhoods could regularly attend the groups due to family obligations after school and safety issues related to returning home late. School partners then insisted that the program be delivered in one class period and during the school day when counseling usually occurs. With the limited resources typically available to schools, school partners also required a brief intervention that was not resource intensive. Research partners suggested an evidence-based treatment approach based on cognitive behavioral therapy techniques that had been found in previous studies to be effective in treating trauma symptoms. The resulting intervention, CBITS, has been found to “fit” within the framework of the school community and has characteristics that have been found to predict adoption of new innovations¹⁵ such as relative advantage over usual care practice, compatibility with other behaviorally oriented practices in schools, and minimal complexity through clearly presented lessons that were developed in collaboration with school-based clinicians.

Given that the primary mission of schools is to educate, and that educators are under enormous pressure to demonstrate academic improvements on a yearly basis, a key factor in partnering with schools around the adoption of such early intervention services is the program's impact on academic outcomes such as achievement. Universal school prevention programs, especially those targeting externalizing behaviors, have begun examining achievement, academic engagement, and academic competency in addition to behavioral outcomes.^{16, 17} However, studies of academic performance outcomes are rare for programs addressing internalizing symptoms. One randomized study of a brief cognitive behavioral therapy stress management intervention for a general school population of adolescents in the UK found improved mental health and academic competence in the intervention compared to control group.¹⁸ Despite methodological limitations, this study showed that following a 3-month intervention, achievement scores were significantly higher than in the control group. Another study examining a prevention program for elementary school children at risk for developing anxiety disorders found the program improved math scores, but had little effect on anxiety or reading achievement.¹⁹ There remains, however, a paucity of evaluations examining the impact of early intervention programs for trauma-related mental health problems on classroom performance. In this study, using a community-partnered participatory research approach, the educational outcomes of a promising school-based intervention for youth exposed to violence are examined.

In previous studies of CBITS, our community-research partnership has reported on improvements in PTSD and depressive symptoms.^{10, 20} However, understanding how CBITS affects students' educational outcomes remains important to school stakeholders. Thus, improving knowledge about the impact of early interventions such as CBITS on school performance is critical for both improving the adaptive functioning of underserved populations who are affected by violence and being responsive to the interests and mission of community partners. The current study addresses this issue by describing student grades in language arts and math in two groups of sixth grade students, those who received CBITS soon after detection of symptoms and those who were delayed in receiving treatment.

METHODS

Procedures

This study was conducted at two middle schools in East Los Angeles, with primarily low income, Mexican-American students, described in greater detail in Stein et al, 2003.¹⁰ Following consent procedures, 769 English-speaking sixth grade students participated in a self-report screening for violence exposure and posttraumatic stress symptoms either in September or December 2001 (two cohorts of students screened). Students were eligible for CBITS if they endorsed substantial violence exposure, post-traumatic stress symptoms in the clinical range, were willing to discuss their traumatic experiences in a group setting, and did not appear too disruptive to participate in a group therapy intervention as determined by the school-based mental health clinician.

Of the 159 students who were eligible for CBITS, 126 consented/assented to participate in the study. Sixty-one students were randomized to the Early Intervention group and received CBITS immediately after screening and sixty-five students were randomly assigned to the Delayed Intervention group and received CBITS after waiting 4–5 months following screening. Because school partners preferred to provide the program to both intervention and control students in the same academic year, all participating students received CBITS during the same academic year. Ninety percent of students completed treatment (87% in the immediate group, n=53; 92% in the delayed group, n=60).

One hundred twenty-three students, the sample for this study, had grades and standardized state test scores available. Of the 59 students in the Early Intervention group, 26 students were screened in the first cohort in September 2001 and completed treatment in January 2002, and 33 students were in the second screening cohort and completed treatment at the beginning of April. Of the 64 students in the Delayed Intervention group, 31 students were in the first screening group and completed treatment in mid-May, and 33 students were in the second screening group and completed treatment in June.

The study was conducted in compliance with the LAUSD's Research Review Committee and the Institutional Review Boards of RAND and UCLA.

Intervention

Developed in partnership with clinicians and administrators from the local public school system (see Wong 2006 for community perspective¹⁴; Stein et al 2002 for development of partnership²¹), CBITS incorporates standard cognitive behavioral therapy skills in a group format (5–8 students/group) to address post-traumatic stress, anxiety, and depressive symptoms related to violence exposure over the course of 10 group sessions and 1–3 individual sessions.^{10, 22} Designed for a multicultural student body, CBITS balances fidelity to core components of the cognitive behavioral skills with the flexibility of incorporating culturally appropriate examples and activities to teach those skills (further described in Ngo et al, 2008²³). The intervention had previously been pilot tested for feasibility and acceptability with immigrant Latino populations (Mexican and Central American students) in this school district.²⁰ Two full-time and one part-time school psychiatric social workers delivered the CBITS intervention during the 2001–2 academic year, usually during one class period per week. School partners determined when the intervention was delivered, such as during non-academic periods and at different times each week to minimize the number of times a student would miss the same academic class.

The school mental health clinicians received two days of training in CBITS and weekly group supervision from the clinician investigators (BDS, LHJ, SHK). The school clinicians followed a treatment manual to insure that the intervention was standardized across clinicians.²² Fidelity to the manual was measured by an independent clinician who rated randomly selected audiotapes of sessions (17% of the total number of sessions), assessing extent of session material completion and quality of therapeutic approach (i.e. motivation and participation of group members, therapist empathy). The mean rate of content completion was 96% and quality of sessions was moderate to high across sessions.

Measures

To assess students' post-traumatic stress symptoms, we used the Child PTSD Symptom Scale (CPSS),²⁴ a 17-item child self-report measure (range 0–51) that has been shown to have good convergent and discriminant validity, high reliability²⁴and high internal consistency²⁰ in school-aged children. Children rated how often they were bothered by each symptom in the past month on a scale from 0 (not at all) to 3 (almost always). For program eligibility, post-traumatic stress symptoms in the clinical range were defined as a sum score of 14 or more, consistent with moderate clinical levels of post-traumatic stress symptoms.

Students' depressive symptoms in the past two weeks were assessed using a 26-item Child Depression Inventory (CDI).²⁵ The CDI (range 0–52) assesses children's cognitive, affective and behavioral depressive symptoms, and has good test-retest reliability and validity.^26–28 A single item assessing suicidality was removed at the request of school partners.

Outcome measures of academic performance were based on Spring Semester grades from the 2001–2002 school year for math and language arts. Grades were abstracted from school records and coded as A=4, B=3, C=2, D=1, and F=0 for use as an outcome variable. Previous studies have documented the importance of students receiving a grade of “C” or higher, which often is the minimum grade allowed to advance to higher level courses and to participate in extracurricular activities such as sports.^{29, 30} Since this distinction also resonated with our community partners, we also compared the treatment groups by whether or not students had a passing grade of “C” or higher on their Spring Semester math and language arts final grades. Fall semester grades could not be used in this study as either baseline variables or outcome variables, because they reflect school performance during the first half of the school year, the same time period that some students in the Early Intervention group were in the process of receiving, but had not yet finished, treatment.

Annual state testing results from the previous school year were also abstracted; however, state testing results for the school year during which the trial took place were not available (testing is administered in May but results are not available until the following summer, after the study had ended). Prior year state testing scores were used to establish a baseline of academic functioning for each student. Specifically, standardized total state test scores were used as a covariate for examining Spring Semester math and language arts grades. Standardized test scores were reported on a scale of 0 to 100, with 100 being the highest score possible.

Analyses

To examine baseline characteristics, we compared the Early and Delayed Intervention groups on child and parent characteristics, violence exposure, mental health symptoms, and prior year standardized test scores in reading, language arts, and math using t tests for numerical variables and Chi-square tests for categorical variables.

To assess the difference between the Early and Delayed Intervention groups on Spring Semester mean math and language arts grades, we use hierarchical linear models (HLM) to account for hierarchical data structure (students are nested within treatment group, to address the possible influence of shared exposures among students within the same CBITS treatment group. Given that there are only two schools, school is treated as a fixed effect. In such models, our primary interest is intervention effects at the student level, while not ignoring the variability associated with groups. Intervention status as the main independent variable, and the standardized test score from the prior school year, PTSD symptom score, total violence score, gender, school site, parent employment status, and time of screening are covariates. To show effect sizes, we present unadjusted means and proportions by intervention groups, as well as adjusted differences or odds ratios (ORs) that are adjusted for the covariates listed above. We use PROC MIXED for continuous variables (Language arts and math grade) and GLIMMIX for binary variables (passing grade) in SAS System V9.2.³¹

RESULTS

Students had a mean age of 11 years, with 44% (n=54) being female., and 40% (n=49) had family household incomes below $15,000. Overall, participants in this treatment study had significant levels of violence exposure, with 74% (n=91) reporting directly witnessing or being victim to knife or gun violence in the past year. The mean PTSD symptom score was 24, with a score of 14 or more representing at least moderate levels of PTSD. In general, standardized test scores from the prior academic year were remarkably low, with the mean score for reading at 33, math 41, and language arts 44 out of a total possible score of 100. Students in the Early Intervention group did not differ significantly on any of the baseline characteristics compared to those in the Delayed Intervention group (Table 1).

Table 1.

Sample characteristics at baseline by treatment group^a

Characteristics	Early Intervention (n=59)	Delayed Intervention (n=64)
Child Characteristics
Age, mean years (S.D.)	11.0 (0.3)	10.9 (0.4)
Gender, n (% female)	27 (45.8)	27 (42.2)
Parent Characteristics
Education, y	8.3 (3.6)	8.7 (4.2)
Employment, n (% employed)	23 (39))	31 (49.2)
Married, n (%)	46 (78.0)	44 (69.8)
Household income <$ 15,000, n (%)	22 (37.3)	27 (42.9)
Violence Exposure
Witnessed Violence, No. (%)	59 (100)	64 (100)
Victim to violence, No. (%)	58 (98.3)	60 (93.8)
Weapon related violence, No. (%)	42 (71.2)	49 (76.6)
Total violence score, mean (S.D.)	26.1 (13.5)	26.3 (13.8)
Symptoms
PTSD symptom score, mean (S.D.)	24.4 (6.8)	23.6 (7.2)
Depressive symptom score, mean (S.D.)	17.6 (10.8)	16.7 (7.3)
Standardized test scores from prior school year
Reading score, mean (S.D.)	29.3 (19.3)	36.3 (22.8)
Language Arts score, mean (S.D.)	40.3 (24.4)	47.8 (23.2)
Math, score, mean (S.D.)	39.7 (22.4)	41.4 (22.8)

^aNo significant differences between Early and Delayed groups (p>0.05)

Table 2 shows the unadjusted and adjusted analyses for spring math and language arts grades. Students in the Early Intervention group had a 2.0 mean math grade, which was significantly better than those students in the delayed intervention group who had a mean math grade of 1.6, after adjusting for prior year standardized test scores and other covariates (p=0.048). However Language Arts mean grades did not differ significantly across treatment groups.

Table 2.

Random effects model predicting spring semester grades by randomized group, unadjusted estimates and adjusted analyses (n=123^a)

	Unadjusted Estimate		Adjusted Analysesa
	Early Intervention group (n=59)	Delayed intervention group (n=64)	Between-group Difference	p-value
Grade			Mean difference, 95% CI
Math, mean (S.D.)	2.0 (1.1)	1.6 (1.2)	0.40 (0.0, 0.8)	0.048
Language Arts, mean (S.D.)	2.2 (1.2)	1.9 (1.3)	0.32 (−0.1, 0.7)	0.129
Passing Grade, C or higher			OR, 95% CI
Math, passing grade, n (%)	41 (69.5)	35 (54.7)	OR, 2.3 (1.0, 5.3)	0.053
Language Arts, passing grade, n (%)	47 (79.7)	39 (60.9)	OR, 2.9 (1.1, 7.5)	0.033

^aStudents for whom we did not have group assignment (prematurely dropped out), were assigned to a single group number

^bAdjusted for: standardized test score from prior year, gender, employment of parent, PTSD score, total violence score, time of screening, school site

When Spring Semester passing grades were examined (grade of “C” or higher), 80% of those in the Early Intervention group compared to 61% in the Delayed Intervention group had received a passing grade in Language Arts (O.R. 2.9, CI 1.1, 7.5; p=0.033, see figure 1). A non-significant trend was found for passing math, with 70% of students in the Early Intervention group had passed math while only 55% in the Delayed Intervention group had a passing grade in Math (OR 2.3, CI 1.0, 5.3; p=0.053).

Figure 1.

Spring Language Arts and Math grade point average, by early vs late treatment groups (n=123)

DISCUSSION

This community partnered research study provides important preliminary evidence that a school-based intervention for children exposed to violence may positively impact grades, a finding that is directly relevant to schools' educational mission. Our earlier work demonstrated positive findings on mental health outcomes (PTSD and depressive symptoms), but we had not yet examined the impact on academic achievement. These findings suggest that those students who receive CBITS soon after screening, compared to those who are delayed in receiving treatment, appear to perform better academically in terms of their math grades and having passing grades in language arts. This positive impact on academics from a brief, feasible mental health intervention for students with very high levels of violence exposure highlights important information for school administrators concerned with school performance indicators. This study is one of the first to suggest the linkages between academic and mental health outcomes within a mental health intervention context for children with internalizing problems (e.g., anxiety, depression). This study shows a possible impact on both language arts and math grades, building on the work of Keogh and colleagues (2006) and Cooley-Strickland and colleagues (in press) who found improvements in academics among children receiving interventions for anxiety and stress management.^{18, 19}

The fact that this study was able to demonstrate differences in academics among students who all received the program during the same academic year and only differed in the timing of the intervention is noteworthy. We interpret these findings to mean that receipt of the program immediately after mental health need is detected gave students more of an opportunity to focus and concentrate in school, and thereby improve their grades during the Spring Semester. In contrast, those who received the program following a delay in treatment had less time for their mental health improvements to impact their grades. Had the study included a control group that did not receive the intervention, the effects may have been more profound.

Our study findings also have important practical implications for educators who are considering bringing mental health programs to schools. The program was developed and the study conducted using community partnered participatory research (CPPR) in which community partners were critical in shaping the research question, conduct the study, and interpret the findings. Education is the primary mission of schools, and schools often have limited resources to put toward programs, requiring educators to look for evidence that interventions improve students' academic outcomes as well as mental health. The present study provides such information for educators interested in mental health programs. Conducting the study through an integrated partnership has helped to ensure real world relevance of the findings.

A number of studies have found that exposure to violence affects student academic performance. In a longitudinal study of middle school students, Henrech et al (2004) found that witnessing violence was associated with lower academic achievement over time.³² Those students who had not witnessed any violence were twice as likely to meet state academic performance goals. Others have found associations between violence exposure and lower high school graduation rates.³³ In recent years, a substantial amount of academic resources and educationally focused programs have been devoted to improving the academic performance of low performing students, who are at higher risk for dropping out in high school.³⁴ Our study suggests that at least for some of these students suffering from substantial psychological distress, a targeted and effective program addressing their mental health symptoms may also improve their classroom performance.

Future research should examine the potential mechanisms through which interventions like CBITS may impact academic performance, both directly and through mediating pathways. Cognitive theories of PTSD support the notion that following a traumatic event, one may develop maladaptive cognitive schemas in which individuals believe that the world is always a dangerous place.³⁵ Prolonged intense anxiety coupled with such maladaptive cognitions can ultimately disrupt youths' ability to relate adaptively to others and successfully manage emotions such as in the classroom setting, ultimately resulting in poor school outcomes. Children with posttraumatic stress symptoms may experience difficulty concentrating due to preoccupation and intrusive thoughts related to the trauma memory, or to poor sleep and stressors outside of school. Interventions such as CBITS, delivered at the early signs of psychological distress, may help to decrease these effects and improve students' ability to concentrate on their school work and focus in the classroom.

There are several limitations to the present study that are important to consider. First, both the Early and Delayed Intervention groups received CBITS in the same school year, and given that grades were not measured exactly before and after treatment was delivered, it is difficult to interpret the effect that CBITS has on grades immediately post-treatment. In addition, due to school partners working under “real world” constraints while delivering CBITS for this study, we decided to screen in two cohorts, one at the beginning of the school year, and one midway during the year. Had we screened all students at the beginning of the school year, and then randomized them into four groups each getting CBITS at a different time of year, we could have more easily looked at the effect of receiving the intervention early in the school year vs later. Our comparison was limited to comparing students who received treatment immediately after being screened vs. those who had to wait several months to receive treatment. Future research should include treatment and control groups that receive CBITS during different school years, in order to more fully examine what effect CBITS may have on school success and what potential mediators may be involved in this effect. Finally, given that we did not have resources to collect data following the completion of this study and did not have the foresight to obtain permission to collect achievement testing from the current academic year, this study is limited in not being able to assess change in achievement testing. Additional research is needed that uses multiple measures of school performance, including an administered achievement test before and after intervention as there are limitations to using grades as a measure of academic success.

Despite these limitations, the current study provides important preliminary evidence that a brief school-based intervention for students exposed to violence in the community can impact not only their mental health, but also their grades. At a time when both the mental health system and educational system are faced with difficult choices arising from constrained resources, an efficient, time- limited, school-based group intervention that can improve both students mental health and educational outcomes may play an important role for districts educating students in communities with endemic community violence. Such community-partnered approaches to school-based interventions for students exposed to violence can fill a critical public health gap by providing treatments readily accessible for low-income minority students who could benefit from improved mental health and school functioning.