Impact of School-Based and Out-of-School Mental Health Services on Reducing School Absence and School Suspension among Children with Psychiatric Disorders

1. Introduction

School-based mental health services are as effective as traditional outpatient clinic services in improving clinical outcomes (Armbruster & Lichtman, 1999; Hussey & Guo, 2003; Owens & Murphy, 2008), do a better job of engaging and retaining families (Atkins, Frazier, Birman, Adil, Jackson, et al., 2006) and are associated with reduced stigma (Taras, Frankowski, & McGrath, 2004). In addition to improving clinical outcomes, school-based programs have the advantage of being able to explicitly target behaviors and symptoms that affect school functioning, thereby improving academic outcomes (Lyon, Borntrager, Nakamura, & Higa-McMillan, 2013).

It is important to demonstrate whether school-based mental health programs improve academic outcomes. Academic outcomes are the ones in which school personnel are most invested; if mental health services address those goals, it can increase the perceived value and fit of mental health programs (Atkins, Rusch, Mehta, & Lakind, 2015). Absenteeism is an important example of academic outcomes that mental health programs can address. Nationally, 10–15% of students are chronically absent from school (Balfanz & Byrnes, 2012). Absences increase in fifth grade and middle school, particularly among economically disadvantaged children (Balfanz & Byrnes, 2013). Greater absenteeism in sixth grade is associated with higher risk for high school dropout (Balfanz, Herzog, & Maclver, 2007). School suspensions represent another school outcome that mental health programs can address. Suspensions are associated with school dropout, delinquency and drug use (Sheryl, Stephanie, Herrenkohl, Toumbourou, & Catalano, 2014).

Preliminary evidence supports the effectiveness of school-based mental health services in improving school absences and/or suspensions. For example, Positive Action, a school-based social-emotional and character development program implemented in low-income Chicago schools, has significantly reduced students’ school absences (Bavarian, Lewis, Dubois, Accok, & Vuchinich, 2013). The Rochester Resilience Project, which was tested in a randomized controlled trial for urban children with behavioral and emotional problems, significantly decreased mean school suspension (Wyman, Cross, Brown, Yu, & Tu, 2010). Other studies have found that school-based programs reduce suspensions (Flay, Allred, & Ordway, 2001; Kang-Yi, Mandell, & Hadley, 2013).

While these studies provide preliminary evidence that school-based mental health interventions can improve school absence and suspension, the association between these programs and outcomes has not been extensively studied. Even fewer studies have examined school-based mental health programs implemented by community practitioners in real-world settings. None to our knowledge has examined the role of school-based mental health services on children’s use of community mental health services outside the school and more importantly, the impact of using these services on children’s academic outcomes. Previous research has identified that parents’ perception of children’s functional impairment significantly affects children’s use of school-based and out-of-school community mental health services (i.e., Langer, Wood, Wood, Garland, Landsverk, et al., 2016). The finding suggests that children’s use of community mental health services outside their school will decrease if their parents perceive the school-based mental health services as improving their children’s behavioral functioning.

Findings regarding the relative impact of mental health services delivered inside and outside of school on children’s academic outcomes will inform policymakers of the most effective settings for delivering mental health services. For example, Philadelphia relies on teams from publicly-funded community mental health agencies to provide school-based mental health care to youth in selected kindergarten through 8th grade Philadelphia public schools. This program model is not unique to Philadelphia; other counties and states such as Baltimore, New York City and Florida also provide Medicaid-funded mental health support including individual therapy and/or group therapy in schools (Cammack, Brandt, Slade, Lever, & Stephan, 2014).

Starting in 2007, School Therapeutic Services (STS) teams were deployed to better integrate mental health services in Philadelphia schools. STS was developed in response to the increasing need for integrated mental health services in Philadelphia elementary and middle schools (City of Philadelphia Department of Behavioral Health and Intellectual disAbility Services, 2006). The goals of STS include providing youth with therapy and in-class support (when indicated) in school to address emotional and behavioral concerns, promote social and emotional development, and reduce school absences and suspensions, among other outcomes.

The present study examined the impact of STS on children’s academic outcomes and use of out-of-school mental health services. Specifically, we examined the impact of both STS and out-of-school mental health services on children’s school absence and in- and out-of-school suspension. We also investigated whether STS affected children’s supplementary community-based mental health service use. We hypothesized, based on stated STS goals, that school absences and suspensions of the children who receive STS would decrease over time. We also hypothesized that out-of-school services would have a smaller effect than STS on absences and suspensions, and that more use of STS would be associated with less use of out-of-school community mental health services.

2. Methods

2.1 Study Sample

The sample included 755 children who received STS in 1^st through 8^th grades in 198 schools in the School District of Philadelphia between September 2010 and June 2011. Children who were enrolled in STS for multiple school years and children who did not have school records matched with mental health Medicaid claims data were excluded because school records were only available for one year prior to and after the STS provision for the cohort of children who were enrolled in the program during the school year of 2010–2011. The sample represents 26% of all children who received STS in the 2010–2011 school year.

2.2 Data Source

Community Behavioral Health, a nonprofit managed care organization established to manage Medicaid mental healthcare as part of the Philadelphia’s Department of Behavioral Health and Intellectual disAbility Services (DBHIDS) and the School District of Philadelphia provided mental health Medicaid claims and children’s school records including school enrollment, absence and suspension between September 2009 and June 2012. These data were used to identify the study sample and provided the outcome variables of interest. School-level data, including overall school enrollment, absence and suspension were obtained through the School District of Philadelphia and the Pennsylvania Department of Education. The school-level data were publicly available. The University of Pennsylvania Institutional Review Board (IRB), the City of Philadelphia IRB, and the School District of Philadelphia’s Research Review Committee approved this study.

2.3 Service Delivery Model

STS is designed to involve caregivers in treatment whenever possible and to complement services provided in the home and community, such as outpatient psychiatric services. Services occur within the regular school day and include individual and group therapy and behavioral consultation on an as-needed basis. Children who are referred by school teachers, caregivers and other service providers and who meet the medical necessity criteria having a need for higher level of care than outpatient psychiatric treatment get enrolled in STS. Treatment plans are individually developed and may or may not be aligned with a particular evidence-based practice. The staffing model consists of bachelor’s level behavioral health workers that work with multiple students under the support of a master’s-level lead clinician and group therapist. The lead clinician and group therapist provide specialized therapeutic services at least once weekly. The staff-to-children ratio is set as one bachelor’s health worker per three children (1:3) and one Master’s prepared lead clinician per ten children (1:10). Services focus on issues such as problem solving, conflict resolution, social skills, and anger management. Goals of services include reducing mental health symptoms and improving academic outcomes. Students are permitted to receive out of school mental health services and may be referred by school mental health providers when appropriate.

2.4 Outcome Measures

Academic Outcomes included mean monthly absence per child and mean monthly suspension per child. Mean monthly absence per child was calculated by percentage of days absent in a given month, averaged over the year. Mean monthly suspension per child was calculated as the percentage of days suspended in- or out-of-school in a given month, averaged over the year. The monthly academic outcomes were calculated based on monthly number of days enrolled in school by dividing the total monthly number of days with absences/suspension and then, multiplying by 100 to obtain a percentage per child.

Mental health service use was identified using Medicaid mental health claims. Mental health services were categorized into Behavioral Health and Rehabilitation Services (BHRS); inpatient psychiatric hospitalization, partial hospitalization, residential treatment facility care, outpatient treatment, community support services, and other services. BHRS is designed to provide wraparound services for children aged 3–21 years with serious emotional disturbance, social, or behavioral disorders (Kang-Yi, Locke, Marcus, Hadley, & Mandell, 2016). The observation period of mental health service use was aligned with the STS observation period (one-year pre-STS enrollment, STS enrollment year and one-year post STS enrollment); thus, we could observe concurrent mental health service use in and out of school. Community support services included intensive case management, residential service coordination, specialized case management, crisis residence, and family rehabilitation service. All other services including lab tests and services for substance abuse were recoded as other services.

2.5 Child-Level Variables

Child-level variables included sex (male and female), race/ethnicity (African American, Hispanic, White, and Other race), psychiatric disorder (conduct disorder/oppositional defiant disorder, attention deficit hyper activity disorder, and other disorders), grade level (categorized into 1^st–5^th grades and 6^th–8^th grades), and school transfer status (one or more school transfers during the school year vs. no school transfer). The literature reports a high proportion of elementary and middle school children have school transfers, and children with school transfers are at risk for behavior problems as well as truancy and suspension (Herbers, Reynolds, & Chen, 2013). Thus, we controlled for school transfer in the outcomes analysis.

Consistent with our previous studies (Kang-Yi et al., 2016a; Kang-Yi et al., 2016b; Locke, Kang-Yi, Pellecchia, Marcus, Hadley, et al., 2017), we used the International Classification of Diseases (9^th ed., ICD-9) available through the Medicaid mental health claims data and categorized children with the primary diagnosis of 312.xx or 313.xx as having conduct disorder/ODD and children with the primary diagnosis of 314.xx as having ADHD. The most frequent psychiatric diagnosis was recoded as the primary psychiatric disorder to resolve multiple diagnoses found per child in the Medicaid mental health claims data (Kang-Yi et al., 2013). Of the 134 children with other psychiatric disorders, 39.1% had adjustment disorder, 32.3% had mood disorder and 12.8% had autism. In the analysis, we grouped these diagnoses as “other disorders.” Table 1 presents demographic characteristics and psychiatric disorders of the study sample. More than two thirds were boys (78.0%) and African Americans (73.3%). The most frequent psychiatric disorder was ADHD (49.1%), followed by conduct/ODD (33.3%), and other disorders (17.6%). The largest group by school grade was the 8^th grade (18.4%) and the smallest group was the 1^st grade (6.4%). Other grades were evenly distributed.

Table 1.

Study Sample Demographic Characteristics

Children Enrolled in STS in School Year 2010–11 (total N = 755)	N	%
Sex
Female	166	22.0
Male	589	78.0
Race/Ethnicity
African American	553	73.3
Hispanic	130	17.2
White	62	8.2
Other	10	1.3
Psychiatric disorder
Conduct/oppositional defiant disorder (ODD)	252	33.3
Attention deficit hyperactivity (ADHD)	371	49.1
Other	134	17.6
Grade level (STS enrollment year)
1	48	6.4
2	95	12.6
3	93	12.3
4	104	13.8
5	94	12.5
6	102	13.5
7	80	10.6
8	139	18.4

Note. STS: School Therapeutic Services

2.6 School-Level Variables

School context has been found to significantly affect children’s academic outcomes (Kang-Yi et al., 2013). For example, Kang-Yi et al. (2013) found that children are at higher risk of having school suspension if they attend schools with a higher level of suspension compared to children attending schools with a lower level of suspension. We included school-level absence and suspension in the academic outcomes model to distinguish the effect from the effect of STS and supplementary community mental health service use on children’s school absence and suspension from the effect of absence and suspension at the school level.

School-level monthly absence was calculated as the average monthly percentage of absences at each school. Monthly absence was calculated by dividing the total annual number of absences by the annual number of enrolled students, by ten-school months, and by 20-school days per month and then, multiplying by 100 to obtain a percentage. School-level monthly suspension was calculated by averaging monthly percentages of in- or out-of-school suspended days at the school. Monthly suspension was calculated by dividing total annual number of suspension days at the school by total number of enrolled students, by ten-school months and by 20-school days per month and then, multiplying by 100 to present as a percentage.

2.7 Statistical Analysis

Descriptive statistics were used to describe demographic characteristics, primary psychiatric disorder, and mental health service use. School years observed included one-year pre-STS (9/08/2009 – 6/18/2010), STS enrollment year (9/07/2010 – 6/21/2011), and one-year post STS (9/06/2011 – 6/14/2012). Chi-squares and generalized linear model analyses were used to identify if the percentage of children using out-of-school mental health services significantly changed over time as they began receiving STS in school and whether the change in the mean number of days children used each type of mental health service was statistically significant. Additionally, logistic regression analysis for each type of mental health service use was conducted to examine if children’s mental health service use significantly changed over time controlling for sex, race/ethnicity, psychiatric disorder, grade level, and school transfer.

Two sets of generalized linear mixed models were conducted to examine the effect of STS on children’s school absence and school suspension as well as determine the association of individual- and school-level factors with children’s school absence and suspension. The school absence model included children’s monthly absence as the dependent variable. Individual-level predictors included sex (male vs. female), race/ethnicity (African American, Hispanic, White, and Other), psychiatric disorder (ADHD, conduct disorder/ODD, and other disorders), school grade-level (1^st – 5^th grades vs. 6^th – 8th grades), and school transfer during the school year (Yes=1 vs. No=0). The school-level predictor included school-level absence recoded into quartile categories based on overall monthly absence of all schools. The four categories included top 25% of monthly absence, upper second quarter of school absence, lower second quarter of school absence, and bottom 25% of school absence. The school suspension model examined the probability of having annual in- or out-of-school suspension (recoded as a binary dependent variable suspended =1 vs. not suspended=0). As 67% of the study sample had mean monthly suspension less than 1%, school suspension at the individual level was recoded into a binary variable to better fit the data in running the model. The same individual-level predictors as in the child absence model were included. To identify if children’s supplementary community mental health service use also impacted academic outcomes, psychiatric outpatient visits (recoded as a binary variable of having 4 or more visits =1 vs. having less than 4 visits =0) and BHRS use (recoded as a binary variable of having 15 claims for BHRS services =1 vs. having less than 15 claims for BHRS services =0) were included in the analysis models. As the claims for psychiatric outpatient visits and BHRS had large ranges with zero medians, the criterion for binary categories of the mental health service use was based on the overall mean of the claims throughout the study observation period. The school-level predictor included school-level suspension recoded into quartile categories of top 25% of overall monthly in- or out-of-school suspension of all schools, upper second quarter of school suspension, lower second quarter of school suspension, and bottom 25% of school suspension.

Analyses were performed using PROC Glimmix in SAS 9.3 (SAS Institute Inc., 201). The first Glimmix analysis with a log link function and exponential distribution was performed to examine school absence and the second Glimmix with a logit link function and binomial distribution was performed to examine school suspension. Due to the high percentage of children with school transfers (ranging from 18% to 22% during the three school years observed), we averaged school-level absence and suspension across schools if one or more of school transfers happened during the school year and treated the school-level factors as averaged school environment during the school year. A random intercept effect was used for the annually repeated absence and suspension measures at the individual level and school-level factors were tested as fixed effects.

3. Results

3.1 Academic Outcomes

Table 2 shows changes in children’s mean monthly absence and mean monthly suspension and school-level mean monthly absence and mean monthly suspension over time. Children’s mean monthly absence was the highest (12.5% per month) during the STS enrollment year and slightly decreased one-year post STS enrollment (11.6% per month). School-level mean monthly absence increased between the STS enrollment year (7.2% absence per month) and one-year post STS enrollment (8.4% absence per month). Individual-level suspension also was the highest (1.5% per month) during the STS enrollment year and decreased to 1.0% per month one year after STS enrollment. School-level mean monthly suspension steadily increased from 0.2% to 0.3% to 0.4% for the three study years.

Table 2.

Descriptive Statistics of Educational Outcomes by School Year

School Year	Variable	Overall (n=755)
		Mean %	SD
One-Year Pre (2009–2010)	Individual-level monthly absence	11.6	10.1
	Individual-level monthly suspension	1.0	1.8
	School-level monthly absence	7.4	1.5
	School-level monthly suspension	0.2	0.2
STS Enrollment Year (2010–2011)	Individual-level monthly absence	12.5	10.6
	Individual-level monthly suspension	1.5	2.2
	School-level monthly absence	7.2	1.6
	School-level monthly suspension	0.3	0.2
One-Year Post (2011–2012)	Individual-level monthly absence	11.6	14.7
	Individual-level monthly suspension	1.0	2.0
	School-level monthly absence	8.4	4.2
	School-level monthly suspension	0.4	0.4

3.2 Children’s Use of Out-of-School Mental Health Services

Tables 3 and 4 show unadjusted and adjusted mental health service use among children, respectively. Children’s mental health service use was the highest during the STS enrollment school year. BHRS, outpatient and partial psychiatric treatment use significantly increased one year post-STS enrollment after controlling for sex, race/ethnicity, psychiatric disorder, grade level, and school transfer (all at p < .001). The mean number of BHRS use days significantly decreased from 97 days to 40 days as children started to receive STS (p < .05). Children’s outpatient treatment use increased from 17.6% to 72.1% from one-year pre-STS enrollment year to the STS enrollment year and decreased to 49.7% one-year post STS enrollment (p < .001). Inpatient psychiatric hospitalization use increased between one-year pre-STS enrollment year and STS enrollment year, and decreased from 9.7% to 6.9% after one year of STS enrollment, but the change was not significant in the adjusted model. Use of residential treatment increased after one year of STS enrollment (0.8% to 3.2%), but the change was not significant in the adjusted model. The partial psychiatric treatment use was significantly higher during one year post-STS enrollment (p < 0.01).

Table 3.

Unadjusted Children’s Behavioral Health Service Use and School Transfer by School Year

Type of behavioral health service	Annual percentage of service users				Annual mean number of service use days among users (based on school year)*
	One year pre	STS enrollment year	One year post	P-level	One year pre	STS enrollment year	One year post	P-level
STS	0.0%	100.0%	0.0%	NA	0.0	62.5	0.0	NA
BHRS	23.3%	54.6%	27.5%	N.S.	96.9	39.6	39.6	p < 0.05
Inpatient psychiatric hospitalization	1.6%	9.7%	6.9%	p <0.001	5.0	17.2	14.9	p < 0.01
Residential treatment facility	0.0%	0.8%	3.2%	p <0.001	0.0	66.7	72.0	p <0.001
Community support services	2.8%	12.1%	10.7%	p <0.001	20.5	30.7	25.7	p < 0.05
Outpatient psychiatric treatment	17.6%	72.1%	49.7%	p <0.001	7.6	7.6	10.6	p <0.001
Partial treatment	2.5%	9.0%	4.8%	N.S.	35.3	22.6	18.8	NS
Other services	0.4%	3.8%	5.0%	p <0.001	61.7	53.0	21.4	NS
One or more school transfers during the school year (Yes=1)	17.9%	22.1%	19.5%	N.S.	NA			NA

Note.

^*Mean number of behavioral health service use days among users are presented. STS: School Therapeutic Services. NS: Not significant. NA: Not applicable.

Table 4.

Logistic Regression Analysis of Children’s Behavioral Health Service Use

STS enrollment and Child-Level Factors	BHRS		Inpatient hospitalization		RTF		Community support		Outpatient		Partial treatment		Other service
	Odds Ratio	P-level	Odds Ratio	P-level	Odds Ratio	P-level	Odds Ratio	P-level	Odds Ratio	P-level	Odds Ratio	P-level	Odds Ratio	P-level
Pre- and Post- STS enrollment (ref.=STS enrollment year)
One year pre-enrollment	0.80	0.060	0.22	<.001	NA	NA	0.23	<.001	0.21	<.001	0.52	0.022	0.08	<.001
One year Post-enrollment	3.18	<.001	1.44	0.059	0.59	0.002	1.15	0.405	2.65	<.001	1.98	0.001	0.75	0.251
Female (ref. = Male)	0.84	0.139	0.94	0.766	1.82	0.520	1.06	0.774	1.11	0.382	1.12	0.625	0.97	0.910
Race/Ethnicity (ref. = White)
African American	1.10	0.569	0.99	0.967	8.32	0.417	0.28	<.001	0.71	0.049	0.76	0.396	0.97	0.954
Hispanic	0.83	0.334	1.12	0.761	5.20	0.849	0.37	<.002	1.22	0.323	0.52	0.098	1.83	0.244
Other	1.22	0.638	0.52	0.537	NA	NA	0.00	0.981	0.93	0.866	0.99	0.991	0.00	0.983
Psychiatric disorder (ref. = Other disorders)
Conduct/ODD	0.78	0.077	0.50	0.005	0.91	0.030	0.54	0.003	0.80	0.112	1.01	0.957	1.17	0.653
ADHD	1.04	0.737	0.55	0.008	0.50	0.001	0.44	<.0001	1.04	0.753	0.79	0.358	0.80	0.526
Grade level 6th– 8th (ref. = Kindergarten-5th grades)	1.16	0.110	1.52	0.023	3.58	0.174	1.67	0.001	1.21	0.049	1.69	0.006	2.52	<.001
One or more school transfers per school year	1.09	0.439	2.12	<.001	3.32	0.391	0.99	0.942	1.10	0.416	1.15	0.542	1.51	0.143

Note. STS: School Therapeutic Services, BHRS: Behavioral Health and Rehabilitation Services, RTF: Residential treatment facility care

3.3 Impact of the Receipt of STS, School Transfer, Supplementary Community Mental Health Service Use, and School Context on Child’s Academic Outcomes

Table 5 reports generalized linear mixed model analyses on children’s school absence and suspension. Children’s mean monthly school absence did not significantly improve over time. Children’s in- or out-of-school suspension significantly improved after receiving STS. Children were 40% less likely to be suspended in- or out-of-school. Children with one or more school transfers during a school year had an average of 56% more absences per month (p < .001). Children with four or more psychiatric outpatient visits during a school year had 10% lower monthly absences (p < .05) compared to those with less than four outpatient visits. Children enrolled in schools with the top 25% of school-level absence had 62% higher monthly absence compared to those enrolled in schools with the lowest 25% of school absence (p < .001). Children enrolled in the schools with school absences in the lower 2^nd quarter had 18% higher monthly school absences compared to those in schools with the lowest 25% of absence level (p <.01).

Table 5.

Multilevel Analysis of Children’s School Absence and School Suspension

Child- and school-level factors	Mean Monthly School Absence		In-School or Out-of School Suspension (Yes=1)
	Exponentiated estimate	P-level	Odds Ratio	95% Limits	Confidence	P-level
School Year (Ref. group = STS enrollment year)
One year pre-enrollment	0.92	0.092	0.67	0.53	0.85	0.001
One year Post-enrollment	1.01	0.871	0.60	0.47	0.76	<.001
Gender (Ref. group = Female)
Male	1.07	0.195	1.41	1.08	1.85	0.011
Race/Ethnicity (Ref. group = White)
African American	0.91	0.192	1.47	0.98	2.21	0.060
Hispanic	0.99	0.932	1.25	0.79	1.99	0.339
Other	0.95	0.799	1.33	0.49	3.62	0.573
Behavioral health disorder (Ref. group = Other disorders)
ADHD	0.89	0.048	0.99	0.73	1.34	0.954
Conduct disorder/ODD	0.94	0.347	1.27	0.92	1.75	0.147
Grade level: 6th–8th grades (ref. group = 1st – 6th grades)	1.23	<.001	1.95	1.55	2.47	<.001
One or more annual school transfers (Ref. group = no school transfer)	1.56	<.001	1.57	1.22	2.01	0.000
School-level absence in quartile categories (Ref. group = Lowest 25%)
Monthly school-absence level: Top 25%	1.62	<.001	NA	NA	NA	NA
Monthly school-absence level: Upper 2nd quarter	1.30	<.001	NA	NA	NA	NA
Monthly school-absence level Lower 2nd quarter	1.18	0.006	NA	NA	NA	NA
School-level suspension in quartile categories (Ref. group = Lowest 25%)
Monthly school-suspension level: Top 25%	NA	NA	3.12	2.26	4.30	<.001
Monthly school-suspension level: Upper 2nd quarter	NA	NA	2.41	1.78	3.27	<.001
Monthly school-suspension level: Lower 2nd quarter	NA	NA	1.79	1.33	2.42	0.000
More than 4 psychiatric outpatient visits during a school year (Yes=1)	0.90	0.043	1.00	0.77	1.31	0.983
More than 15 times of BHRS service use during a school year (Yes=1)	0.92	0.115	1.02	0.79	1.31	0.900
Random effect: Within-person variance over time	0.85(0.03)		0.42(0.11)
Goodness-of-fit
−2 Res Log Pseudo-Likelihood	5576.99		8545.92

Note. STS: School Therapeutic Services, The school absence model included children’s monthly absence as the dependent variable. The school suspension model examined the probability of having annual in- or out-of-school suspension (recoded as a binary dependent variable suspended =1 vs. not suspended=0). NA: Not applicable.

Children with one or more school transfers during a school year were 57% more likely to have in- or out-of-school suspension compared to children with no school transfers (p < .001). Children enrolled in schools with higher mean monthly school suspension were significantly more likely to be suspended in- or out-of-school. Children enrolled in schools with the 2^nd lower quarter to the top 25% of mean monthly suspension levels were 2–3 times more likely to be suspended in- or out-of-school compared to those enrolled in schools with the lowest 25% of school suspension level (p < .001).

4. Discussion

Many researchers have suggested the potential positive effects of mental health services in schools (Armbruster & Lichtman, 1999; Atkins et al., 2006), but few studies have investigated the effect of these services on academic outcomes, particularly in community-based programs (Lyon et al., 2013). The results of the present study suggest that community-implemented school-based mental health services reduce school suspensions, an important academic outcome (Flay, Allred, & Ordway, 2001; Kang-Yi et al., 2013; Wyman et al., 2010). Mental health services delivered outside of school – but not those delivered in school – were associated with reduced school absences. These findings suggest that both school-based and outside-school community mental health services contribute to improving different academic outcomes.

Perhaps more than suspensions, school absences are affected by factors outside the school setting such as illness, socioeconomic status, family and neighborhood context, and housing instability (Balfanz & Byrnes, 2012; Dube & Orpinas, 2009; Teasley, 2004). Community mental health treatment can include family therapy and also may involve case management and social work; these components may help children lower their school absences by addressing some of these out-of-school factors. Suspension, however, occurs in response to in-school behavior. The majority of the study sample had diagnoses of ADHD and conduct disorders/ODD. One possibility is that children receiving STS services for these diagnoses improved clinically and exhibited fewer disruptive behaviors resulting in fewer suspensions. Another possibility is that students enrolled in STS received more accommodations overall or were less likely to be subject to disciplinary action at school by virtue of being engaged in treatment. For example, principals may liaise with STS to try to arrange more intensive mental health services in response to disciplinary infractions in lieu of suspending a student.

Children’s use of out-of-school services increased during the STS enrollment year. The children’s need for higher level of mental health care may have led the children to receive STS services. Thus, the increase in the out-of-school service use around the time the children were enrolled in STS is expected. The service use decreased one-year post the receipt of STS services (one-year post STS enrollment). The finding seems to suggest that children’s behavioral functioning improved as the result of receiving STS services or at least the parents perceived children’s behavioral functioning as improving. The findings warrant further research to shed light on the effect of school-based services on children’s behavioral functioning and its relationship to use of supplementary community mental health care services.

Approximately one fifth of children transferred schools during the study year; transfer predicted both school absence and suspension. Transfers may suggest residential instability and the challenges that accompany it. They also may include disciplinary school transfers. The high rate of disciplinary school transfers in the School District of Philadelphia has been raised many times as a cause for concern (American Civil Liberties Union of Pennsylvania, 2015; Public Citizens for Children and Youth, 2015; Youth United for Change, Advancement Project, & Educational Law Center – PA, 2015). Given these high transfer rates, school-based mental health programs should include psychosocial support around the school transition process, as well as continued support when students transfer schools. Additionally, as school transfer also may reflect the school district policy toward school disciplinary actions, further study on how school disciplinary policy influences children’s school transfer is warranted.

Consistent with our hypotheses and previous work (Sheryl et al., 2014), school factors emerged as important predictors of both reduced absences and suspensions. Specifically, children in schools with higher overall absence rates were significantly more likely to be absent themselves. Similarly, children in schools with higher overall suspension rates were significantly more likely to be suspended. These findings point to the critical need for programming to target school context coupled with individual interventions for particularly vulnerable children (Sheryl et al., 2014; Youth United for Change, Advancement Project, & Educational Law Center – PA, 2015). Recent efforts to change the school environment have included interventions such as Positive Behavior Interventions and Supports (Bradshaw, Mitchell, & Leaf, 2010; Walter, Gouze, Cicchetti, Arend, Mehta, et al., 2011) that have taken that approach; it is likely that for the most robust impact on academic outcomes, this combined environment plus individual approach is necessary. Specific components of school-based mental health program such as training and coaching mental health workers on effectively developing collaborative relationship with school staff and incentives for school staff to partner with school-based mental health support staff should be considered to maximize the effect of school-intervention on children’s academic outcomes.

5. Lessons Learned for Future Program Planning and Evaluation

The evaluation of mental health service delivery in a large urban school district taught us a few key lessons. First, and most importantly, we did not have information on the specific clinical interventions delivered through the services, including their quality. Given the system wide emphasis on implementation of evidence-based practices in the City of Philadelphia (Beidas, Aarons, Barg, Evans, Hadley, et al., 2013; Powell, McMillen, Proctor, Carpenter, Griffey, et al., 2012), there may be elements of evidence-based practice in these services, but research is needed to identify what is delivered.

Second, there are other academic outcomes, such as grades and standardized-test scores, that we were unable to examine because the outcomes are inconsistently collected by the school district. Examining these academic outcomes can lead to identifying additional impact of school-based mental health services. At the same time, identifying the feasibility of measuring academic outcomes consistently across different schools and grade levels is important to accurately examine the effect.

Third, the nature of the data required that we exclude children enrolled in STS for multiple years. Our empirical finding that the majority of children received STS services for longer than one year indicates that children may have more chronic mental health needs and addresses the importance of data-driven program planning, monitoring, and policymaking.

Fourth, while the model of utilizing providers who are externally employed by community mental health agencies and contracted to provide services in schools is common (Markle et al., 2014), the results may be idiosyncratic to the Philadelphia system. Evaluations of other states and counties will add further empirical evidence.

Lastly, we did not have family engagement data for this evaluation although the Philadelphia DBHIDS collects the information. Additionally, we did not have information on the implementation fidelity and other mental health promotion programs in school setting and in the community. Future evaluation that take these factors into account will lead policymakers and service providers to learn which component of school-based mental health programs and which mental health promotion programs directly affect the children’s outcome improvement. This examination will guide policymakers and providers on how to tailor their intervention in improving youth behavioral and academic outcomes.

6. Conclusion

This study took advantage of a natural experiment where a large urban system implemented mental health services in public schools through a partnership with community mental health organizations. This model has been replicated in many communities, and understanding the impact of delivering services in schools in this manner is critical, particularly with regard to academic outcomes, which may help facilitate service alignment with the school mission (Lyon et al., 2013). The results suggest that this mental health service delivery system may be effective at moving the needle with regard to important academic outcomes (e.g., suspension) but may be less effective at impacting more nuanced indicators of functioning such as monthly absence. The findings also suggest the importance of focusing on the school context and school wide-interventions to impact individual youth outcomes, particularly the most vulnerable children with mental health difficulties. At present, school-wide interventions, such as Positive Behavioral Intervention and Supports (PBIS), are being implemented in Philadelphia but not necessarily in concert with other mental health interventions like STS. Better integration of school mental health teams within the fabric of the school and alignment of their services with other school context and culture interventions may be helpful in optimizing the impact of school services (Barrett, Eber, & Weist, 2017).

Highlights.

• Objective: Examined impact of school-mental health service use on academic outcomes

• Study Sample: 755 children who were enrolled in 1^st through 8^th grades

• Methods: Longitudinal multi-level analysis of children’s academic outcomes

• Results: Positive impact of school-mental health service use on school suspension

• Conclusion: Mental health program tailored to school setting is important.

Biographies

Christina D. Kang-Yi, PhD is Research Assistant Professor at the Perelman School of Medicine, Department of Psychiatry, University of Pennsylvania. Dr. Kang-Yi has extensive experience of public behavioral health program evaluation. Her research focuses on improving health care delivery though public-academic partnership and promoting wellbeing of diverse populations, including youth, pregnant women, and people entering jail. Dr. Kang-Yi earned her PhD in social work from Columbia University.

Courtney Benjamin Wolk, PhD is a postdoctoral researcher at the Center for Mental Health Policy and Services Research in the Perelman School of Medicine at the University of Pennsylvania. Her research focuses on the dissemination and implementation of evidence-based practices (EBP) for children’s mental health. Dr. Wolk has a particular interest in identifying barriers and facilitators to the implementation of mental health EBPs in non-specialty settings, such as schools, and in improving team functioning. She received her PhD in Clinical Psychology from Temple University.

Jill Locke, PhD, Research Assistant Professor in the Department of Speech and Hearing Sciences, University of Washington, researches the implementation and sustainment of evidence-based practices for individuals with autism spectrum disorder in community-based settings. Her experiences have emphasized the importance of collaborating with community stakeholders and the reality of working within the constraints of publicly funded systems, their timeline (e.g. school calendar year), and with their personnel. Dr. Locke earned her PhD in Education at the University of California, Los Angeles.

Rinad Beidas, PhD, Assistant Professor at the Perelman School of Medicine, Department of Psychiatry, University of Pennsylvania, researches the dissemination and implementation of evidence-based practices (EBPs) for youth in community settings. In 2015, Dr. Beidas was honored as the recipient of the President’s New Researcher Award from the Association for Behavioral and Cognitive Therapies for her work in understanding how to most effectively support therapists, organizations, and systems in the implementation of EBPs. Dr. Beidas earned her PhD in clinical psychology at Temple University.

Ishara Lareef received her Bachelor of Arts degree from the University of Pennsylvania in 2017. She attends the Dartmouth College Geisel School of Medicine and will receive her MD degree in 2021.

Aelesia E. Pisciella, Ph.D., Assistant Director of Research and Evaluation at Community Behavioral Health, researches the relationship between homelessness, housing and behavioral health. She also evaluates behavioral health services targeting children and families, including school-based services. Dr. Pisciella has served as an invited reviewer for several national conferences, including the Administration for Children and Families and the Society for Research on Child Development. Dr. Pisciella earned her Ph.D. in Applied Developmental Psychology from Fordham University.

Arthur C. Evans Jr., Ph.D, is the Commissioner of Philadelphia’s Department of Behavioral Health and Intellectual disAbility Service (DBHIDS). In 2015 the White House recognized him as an “Advocate for Action” by the Office of National Drug Control Policy. In 2013, he received the American Medical Association’s top government service award for his leadership in transforming the Philadelphia behavioral health system. Dr. Evans holds faculty appointments at the University of Pennsylvania School Of Medicine and the Philadelphia College of Osteopathic Medicine and has held faculty appointments at the Yale University School of Medicine and Quinnipiac University.

David S. Mandell, ScD, is Professor of Psychiatry and Pediatrics at the University of Pennsylvania’s Perelman School of Medicine, where he directs the Center for Mental Health Policy and Services Research. He also is Associate Director of the Center for Autism Research at The Children’s Hospital of Philadelphia. The goal of his research is to improve the quality of care individuals with individuals with psychiatric and developmental disabilities receive in their communities.