Management of Return to School Following Brain Injury: An Evaluation Model

Abstract

Traumatic brain injury (TBI) affects children’s ability to succeed at school. Few educators have the necessary training and knowledge needed to adequately monitor and treat students with a TBI, despite schools regularly serving as the long-term service provider. In this article, we describe a return to school model used in Oregon that implements best practices indicated by the extant literature, as well as our research protocol for evaluating this model. We discuss project aims and our planned procedures, including the measures used, our quasi-experimental design using matched controls, statistical power, and impact analyses. This project will provide the evidential base for implementation of a return to school model at scale.

Traumatic brain injury (TBI) of any severity can pose a threat to a child’s future ability to learn and perform in school (Centers for Disease Control and Prevention, 2018; Ewing-Cobbs et al., 2006; Lambregts et al., 2018; Mealings et al., 2012; Phillips et al., 2017; Ransom et al., 2015). Children with moderate and severe injuries are likely to have cognitive, behavioral, and social difficulties that affect their long-term quality of life (Rivara, Vavilala, et al., 2012). Even mild injuries (i.e., concussion) to a developing brain can result in persistent neural alterations that significantly affect educational and social functioning (Eisenberg et al., 2013; Rivara, Vavilala, et al., 2012), becoming more pronounced and debilitating with age (Babikian et al., 2015; Keenan et al., 2018; Kingery et al., 2017; Marlowe, 1992; Ryan et al., 2016). Mild TBI sequelae can lead to increased school absences, decreases in school performance, and reduced social interaction, each of which have a significant effect on health-related quality of life (Eaton et al., 2008; Fiscella & Kitzman, 2009; Moser et al., 2005; Umberson & Montez, 2010).

Although hospitals, emergency departments and clinics treat children and adolescents with TBI in their initial course of recovery, school systems often provide long-term services to these students, following their discharge from the hospital (Centers for Disease Control and Prevention, 2018). While there are existing modalities in schools for providing services to students with TBI, many students continue to experience significant challenges following their return to school (Fuentes et al., 2018; Haarbauer-Krupa, 2012; Rivara, Koepsell, et al., 2012; Rivara, Vavilala, et al., 2012) and, on average, experience poor post-high school outcomes (Sarsfield et al., 2013).

Several barriers limit the provision of effective school services. First, the responsibility for informing the school about the injury generally falls to parents, who often receive inadequate information about their child’s injury in the healthcare setting (Sarsfield et al., 2013) and therefore may not fully understand the potential effects of TBI on school performance. Second, parents may be unfamiliar with the benefits of their child receiving formal or informal supports at school and might not request help (Gfroerer et al., 2008; Roscigno et al., 2015). In addition, many parents do not seek medical care for their children following a TBI (Centers for Disease Control and Prevention, 2018). When children return to school following an injury, the family often misses their primary opportunity to establish a system of supports (Glang, Todis, et al., 2008; Taylor et al., 2003). Finally, the poor linkage between the healthcare and educational systems also contributes to poor tracking of child academic, behavioral, and social needs, which often change over time (Haarbauer-Krupa et al., 2017; Prasad et al., 2017; Taylor et al., 2003).

For more than three decades, researchers, educators, and policymakers have voiced their concern that students with TBI are served by educators who are unprepared to work with them effectively (Farmer & Johnson-Gerard, 1997; Glang, McCart, et al., 2018; Inge et al., 2016; Mohr & Bullock, 2005). Surveys of speech/language pathologists (Evans K et al., 2009), school psychologists (Hooper, 2006), and special and general educators (Davies et al., 2013; Ettel et al., 2016) consistently reveal a lack of knowledge about childhood TBI, suggesting inadequate preparation across professions. The lack of educator awareness contributes to ongoing under-identification of children with TBI for support services in schools (Davies, 2016; Glang et al., 2015). Evidence-based individualized assessments and instructional methods can help mitigate the academic, social, and behavioral challenges associated with TBI (Glang, Ylvisaker, et al., 2008; Ylvisaker et al., 2001). However, many teachers receive little or no pre-service training on TBI (Chapman, 2002) and are thus unaware of and poorly prepared with strategies for addressing its school-related implications.

The goal of this project is to evaluate the effectiveness of the Central Oregon TBI (COR-TBI) model, a comprehensive Return to School model for students with TBI. Coordinated through the regional High Desert Education Service District (HDESD), the model consists of a team of trained educators (e.g., school counselors, school nurses, school psychologists) who identify, assess, and serve students with concussion and more severe TBI from birth to 21 years of age. The study has 3 key research questions:

1. Compared to treatment as usual (TAU), do services provided by the Central Oregon TBI Support Model result in improved health, educational, and social outcomes for children returning to school after a TBI.

2. What are the rates at which educators implement the critical features of the COR-TBI model? Do higher rates of implementation correlate with improved health, educational, and social student outcomes?

3. How has the COR-TBI been adapted and sustained across time? What components and procedures lead to the sustained operation of the COR-TBI model?

We will also examine the influence of child, school, and other environmental factors as moderators of improvements in health, educational, and social student outcomes.

In this article, we provide an overview of the research protocol used to address these aims. We first discuss the Oregon COR-TBI Team model and the key features that distinguish it from typical practice. We then discuss the specifics of the research methods, including our research sample, measures, and measurement schedules. In terms of evaluating the intervention effect, random assignment is not feasible due to practical and ethical concerns. For example, a randomized-control trial (RCT) would require randomization of groups to receive COR-TBI services or TAU. The COR-TBI support model and its system-wide procedures for post-TBI Return to School are the standard of care in Central Oregon, and thus a TAU alternative is not available in Central Oregon for randomization to conditions. Alternative designs, including interrupted time series (ITS) and regression discontinuity (RD) designs were also deemed infeasible, given that the former would require a delay in receipt of treatment, which would likely lead to poorer outcomes, and the latter would require assignment of participants to conditions based on a cut-score from an assignment variable (which does not exist). The matched comparison group design using propensity scores was therefore the strongest possible design, given the ethical and practical concerns involved in this project. We discuss this design and our analyses for evaluating the intervention effect. We estimate both intent-to-treat (ITT) and treatment-on-the-treated (TOT) analyses. The ITT estimate is generally more important for policy decisions (providing an overall estimate of the intervention effect, given that some are unlikely to comply in applied settings), but the latter can provide information on the average treatment effect for compliers, which may have more bearing for parents and educators working with students with TBI.

Central Oregon TBI Team Model

The COR-TBI model is grounded in a strong theory of change, as illustrated in our logic model (see Figure 1). The conceptual framework for the study focuses on the social model of school adaptation, which suggests that it is within the school context where change occurs. The inputs--committed partners from the schools, medical community, ODE, and families--work together to produce the following outputs: a) a clear and efficient communication system, b) school-based professional development, and c) the implementation of evidence-informed practices for students who experience TBI. These activities, grounded in the literature on best practices for an effective return to school program (Dettmer et al., 2014; Gioia et al., 2016; Purcell et al., 2019), ensure the achievement of the short-term outcomes of increasing the knowledge and awareness of key stakeholders (e.g., teachers, medical providers, and families). The model has already produced a collaborative referral and tracking system between schools and the medical community and the implementation of evidence-informed practices, which are intended to achieve long-term positive academic, health, and social outcomes for students who experience TBI.

Figure 1.

COR-TBI Logic Model.

The COR-TBI model, funded by the Oregon Department of Education (ODE), has been in operation since 1994. The COR-TBI model is supported by ODE’s Traumatic Brain Injury Team¹, which offers statewide TBI training and resources and links medical and educational professionals via monthly phone conference (see Figure 2). Components of the model include:

1. Systematic communication among the medical and educational systems and families: Children treated for brain injury in Central Oregon healthcare settings (i.e., emergency department, concussion clinic, pediatrician office, hospital) are referred to the COR-TBI team. The COR-TBI team facilitates the needed school supports as well as in-class consultation with concussion coaches employed by the local educational service district (HDESD).

2. Tracking of child’s progress: Once the child returns to school, the COR-TBI team helps the school team establish a tracking system to monitor the child’s progress. Additionally, the COR-TBI team surveys school staff to access information about attendance, grades, peer interaction, and behavioral referrals.

3. Professional development for school personnel: Every educator in Central Oregon has access to training in managing TBI-related challenges in school. Training opportunities include both online and in-person options.

4. Evidence-informed practices: The COR-TBI team provides additional face-to-face training and offers resources and tools in a variety of formats (http://www.hdesd.org/services/traumatic-brain-injury/). When a student returns to school, school staff are well prepared to create a system of accommodations and supports tailored to the student’s individual needs.

Figure 2.

COR-TBI team model. ODE = Oregon Department of Education.

Methods

Sample

The sample for this quasi-experimental research study will be 600 children in Central Oregon school districts served by the COR-TBI model and 600 children in the comparison school districts. Participants will be children ages 5–18 enrolled in elementary, middle, or high school who experience a TBI of any severity as a result of any mechanism of injury (e.g., motor vehicle crash, falls, sports). The Central Oregon sample includes four school districts serving approximately 28,000 total students, while the comparison site, located in Southwestern Ohio, includes 14 school districts serving over 52,000 total students. Note that, because we use a matching design (as described later), we over-sample from the comparison school districts to ensure a matched comparison sample of 600 children (hence a total sample size nearly twice as large). We will use minutes of loss of consciousness to classify injuries into two categories, mild or moderate–severe (< 30 = mild, > 30 = moderate-severe).

Inclusion/exclusion criteria.

To be included in the study, children must have an identified TBI (see Screening). Exclusionary criteria include: moderate or severe developmental disability prior to the injury and history of severe psychiatric disorder.

Recruitment.

We will identify children with TBI prospectively at both the comparison and model sites. At both sites, a project research assistant will be responsible for recruitment. After being identified as having sustained a TBI (see Screening below), the research assistant will invite the child and their family to participate in a study on return to school following TBI. The research assistant will then describe the research study goals, activities, compensation, and participant rights, and the child/family will choose whether to provide verbal or online consent. Exact procedures for recruitment were established in consultation with stakeholders on our project implementation teams and as approved by the University of Oregon Institutional Review Board (IRB).

Based on national incidence (Centers for Disease Control and Prevention, 2018), and COR-TBI model data, we estimate that we will identify 600 children with TBI annually at the model site, and roughly twice this number at the comparison site. We will then obtain our matched comparison sample using a one-to-one matching procedure, as described in more detail below. We anticipate TBI incidences across all severity levels. Based on our previous experience recruiting children/families to participate in research, we expect to recruit at least 50% of the children identified with TBI (Glang, Todis, et al., 2018). That estimate takes into account children who do not meet the recruitment criteria and those who choose not to participate.

Screening.

We will identify most children in the COR-TBI model site via existing pathways of care. Children in Central Oregon who seek medical care following TBI are seen at St. Charles Hospital, emergency departments in local communities, or pediatrician clinics in the region. At discharge, with parent permission, children are referred to the COR-TBI team, which provides needed supports to students at school. We anticipate identifying all children with moderate–severe TBI at the COR-TBI model site via this established pathway. Because we know that some children in Central Oregon will not seek medical care following an injury, we will also use an active screening process. At the comparison sites for TAU participants, we will use the same active screening process.

The screening process will include two steps, following a general model used to identify injuries in medical settings (Centers for Disease Control and Prevention, 2018; Gioia, 2017). First, following a child’s absence from school, the school’s attendance official will ask for additional information about the absence to determine whether 1) an injury occurred, and 2) if yes, whether the injury involved a blow to the head with changes in thinking, emotional, or physical functioning. Second, the project research assistant will follow up in person or by phone with any student (and their parents) who screens positive for an injury to complete the screening questions (Sady et al., 2014). We will work with participating schools and the project implementation teams at both the model and comparison sites to refine this screening process.

Through the screening process and the collection of baseline data, we will obtain demographic and health information from all participants in both conditions (e.g., prior concussion history, history of chronic headaches). To reduce selection bias and increase internal validity, we will include this information in the propensity score estimation used to match students.

Subject retention procedures.

This project will use data collection and participant tracking procedures that have been effective in retaining participants and limiting attrition in our prior research projects (Glang, Todis, et al., 2018; Todis et al., 2011). These procedures include: (a) in-person or phone recruitment of children and families that allows parents to receive a full explanation of the project and ask questions at the time of enrollment; (b) compensating participants for completing assessments; and (c) using several strategies to track participants who change addresses and phone numbers, including obtaining participant email addresses and contact information for relatives of participants.

Measures

Table 1 lists the measures that we will use to assess child outcomes and the assessment schedule for all participants. Each of these measures will be used to address Research Question 1. The measures all have documented validity evidence and are meaningful and significant indicators of school, social, and health outcomes. Parents, teachers, and children in both conditions will complete baseline measures at enrollment and then at regular intervals until the child has recovered (as determined by when the student is asymptomatic and returns to their baseline functional status). As shown in Table 1, the measures and assessment schedules will vary by injury severity. To capture concussion-relevant manifestations and outcomes, students with concussions will be assessed on a more frequent basis, following the protocol used in the Predicting and Preventing Postconcussive Problems in Pediatrics (5P) study (Zemek et al., 2016), which tracked over 3000 children post-concussion. The assessment schedule for students with moderate-severe TBI is more protracted, reflecting the more extended timing of recovery and functional outcomes. Measures will be completed online using REDcap, a secure website accessed via password.

Table 1.

Study measures and frequency of administration

Measure	Domain	Severity	Schedule
Demographics (parent, student)	Child and family characteristics	All	BL
Parent satisfaction survey (parent)	Satisfaction with RTS model	All	End of study
Health outcomes
Post-Concussion Symptom Inventory (PCSI) (parent, student)	MTBI symptoms: types/domains, duration/recovery trajectory	Mild	BL, 2, 4, 8, 12, every 3 months*
Post-Concussion Executive Inventory (PCEI) (parent, student)	Executive function symptoms	Mild	BL, 2, 4, 8, 12, every 3 months *
Post-Concussion Follow-up Recovery Discovery – Adapted (parent)	Recovery	Mild	2, 4, 8, 12, every 3 months*
		Moderate to severe	4, 12, 24 52, 104
Behavior Rating Inventory of Executive Functioning 2 (parent, student)	Functional memory Functional attention Executive function symptoms	Moderate to severe	BL, 4, 12, 24 52, 104
Academic outcomes
School records review (project staff)	Attendance, GPA, disciplinary referrals	All	Quarterly
Gradebook review (project staff)	Test & work performance	All	Quarterly
Concussion Learning Assessment and School Survey (CLASS) (parent, student, teacher)	Academic performance problems, stresses, supports needed/provided	Mild	BL, 2, 4, 8, 12, every 3 months*
		Moderate to severe	BL, 4, 12, 24, 52, 104
Social outcomes
PROMIS peer relationships (parent)		Mild	BL, 2, 4, 8, 12, every 3 months*
		Moderate to severe	BL, 4, 12, 24, 52, 104
PACE Self-efficacy (student)		Mild	BL, 2, 4, 8, 12, every 3 months*
		Moderate to severe	BL, 4, 12, 24, 52, 104
NIH Toolbox (student)	Loneliness	Mild	BL, 2, 4, 8, 12, every 3 months*
		Moderate to severe	BL, 4, 12, 24, 52, 104
Neuro-QOL Pediatric Social Relations (student)	Interaction with peers	Mild	BL, 2, 4, 8, 12, every 3 months*
		Moderate to severe	BL, 4, 12, 24, 52, 104
Student Supports
Service Utilization (school-based) (CLASS, Question 8) (parent)	Utilization of school supports; communication between professionals and parents	Mild	BL, 2, 4, 8, 12, every 3 months*
		Moderate to severe	BL, 4, 12, 52, 104
Service Utilization (community-based) (Back to School survey) (parent)	Child and family services received in the community	Mild	2, 4
		Moderate to severe	4, 12, 52, 104

Note. Schedule reported in weeks. See reference section for references to technical reports for all measures.

BL = Baseline

^*Or until recovered

The Post-Concussion Symptom Inventory (PCSI), Post-Concussion Executive Inventory (PCEI), Behavior Rating Inventory of Executive Function – 2 (BRIEF2), and Concussion Learning Assessment and School Survey (CLASS) measures will be used to measure and monitor concussion recovery. Briefly, the PCSI assesses symptoms in the physical, cognitive, emotional, and sleep domains, incorporating self-report ratings. The difference in individuals’ symptom levels at the current time point relative to before the injury provides the Retrospectively-Adjusted Post-Injury Difference (RAPID) score. The PCEI assesses and monitors specific aspects of the student’s executive functioning following concussion in students ages 5–18 years. Using the RAPID score identifies whether the student’s post injury ratings of everyday function differ clinically from retrospective preinjury ratings via change metrics. The BRIEF2 is a youth and caregiver rating of a child’s executive functioning abilities. The general executive composite score serves as a broad measure of executive ability/self-regulation with the cognitive, emotional and behavior regulation composites providing a more specific indication of areas of executive dysfunction. The CLASS measures the effects of a concussion on students’ learning, identifying areas of new academic problems and stresses, as well as the need for academic supports in students who have not yet recovered from concussion.

Each of these measures have documented reliability and validity evidence, with internal consistency estimates generally being over 0.80, and measures of test-retest and inter-rater reliability being similarly strong (Bedell, 2009; Bedell & Dumas, 2004; G. J. DuPaul et al., 1998; DuPaul et al., 2016; George J. DuPaul et al., 1998; Gioia, 2017; Gioia & Isquith, 2019; Gioia et al., 2015; Sady et al., 2014). Similarly, each of these measures have documented validity evidence, including moderate to high convergent criterion-related validity evidence (i.e., correlations with similar measures). The BRIEF2, in particular, has been used in over 1,000 peer-reviewed studies with many populations of children with neurological disorders. For more detailed information, please see the corresponding technical reports.

Parents will also complete the Back to School survey, a questionnaire about community educational, mental health and health services received, including mental health, educational supports, specialized care, and follow-up (e.g. follow-up medical appointments) (Glang, Todis, et al., 2018). Further, we will conduct a gradebook review on all participants quarterly. This will allow us to collect data on changes in academic performance (test and homework completion and scores) that might be more sensitive to the effects of the return to school model than the more global grade point average (GPA). We will examine school records concerning attendance, office discipline referrals, and course completion for participating students. Although changes in GPA are a more distal outcome, we will also collect GPA data from student records. In addition, parents in both conditions will be asked to rate their satisfaction with the services and supports students with TBI receive (Glang, Todis, et al., 2008).

Monitoring program implementation.

We will collect both fidelity of implementation data for the COR-TBI program and fidelity measures for our data collection procedures over time. First, the implementation team will develop a COR-TBI fidelity checklist aligned with key program features (e.g., referral process still in place, communication between school and medical facility still operational). In addition, a data collection procedures fidelity checklist will be developed. Each instrument will be completed quarterly to ensure that all procedures are being implemented as designed and that no drift has occurred. The data collected through these fidelity checklist measures will be used to address Research Question 2.

Measuring TAU implementation.

We will use the same fidelity measures developed for our data collection procedures to measure COR-TBI fidelity and implementation of the TAU sites. Identical data collection timelines will be used. This process ensures that the comparison site has not begun to implement any core components of the model site to prevent program contamination. We will use Community Engaged Research (Isler & Corbie‐Smith, 2012) procedures to garner feedback from key stakeholders through our advisory board.

Propensity Score Matching

As mentioned previously, we use a quasi-experimental matched comparison group design using propensity scores to compare the health, educational, and social outcomes of students with TBI who are served by the COR-TBI model with those of students with TBI in comparison schools in Ohio (TAU). At the COR-TBI site, children who sustain brain injuries will be provided with the support services that are currently in place: (a) systematic communication between the medical and educational systems; (b) tracking of the child’s progress over time; (c) professional development for school personnel; and (d) evidence-based identification, screening, and assessment practices. In the comparison schools, children will be served by districts that currently do not have a formalized program to facilitate return to school following a TBI.

Propensity scores will be estimated using logistic regression with treatment condition as the dependent measure and baseline covariates and their interaction terms as the independent measures. The model takes the general form

$$\text{log}\left[\frac{P(\text{Treatment }=1)}{1-P(\text{Treatment }=1)}\right]=\alpha +{\beta }_1\left({\text{X}}_1\right)+{\beta }_2\left({\text{X}}_2\right)\dots {\beta }_k\left({\text{X}}_{\text{k}}\right)+{\beta }_{k+1}\left({\text{X}}_{\text{m}}{\text{X}}_{\text{n}}\right)$$ (1)

where Treatment = 1 = COR-TBI and Treatment = 0 = TAU, X₁ to X_k are baseline covariates, and X_mX_n = interaction terms. The covariates comprise baseline variables associated with the treatment and outcome (i.e., confounders) including a school identifier, prior concussion history (number, number with recovery > 1 week), history of chronic headaches (e.g., migraine), and initial PCSI symptom score. To ensure we can match students’ by initial status, we will also include a pretest measure of all the outcomes. Following suggestions by Brookhart et al. (2006), we will also include the baseline measure of socioeconomic status, sex, and age, regardless of their association with treatment.

Outside of main effects and interactions, we will also explore the use of nonlinear terms via splines (see Austin, 2011a, 2011b). Following the development of the propensity score model, we will match participants from the TAU group one-to-one with participants from the COR-TBI group that have similar scores. We anticipate that the resulting matched group of COR-TBI and TAU participants will have similar distributions of covariates included in the propensity score. The treatment effects can then be estimated directly from the propensity score matched cohorts. We match using propensity scores because comparable methods, such as stratification and covariate adjustment on the propensity score, do not perform as well in eliminating systematic differences between treated and untreated participants (Austin, 2009). However, we will also explore alternative methods if the matched distributions are not essentially equivalent, including using a boosted tree model for the propensity score matching (McCaffrey et al., 2004). We match TAU and COR-TBI participants in pairs, and after a match, the TAU participant will no longer be eligible for additional matches (i.e., matching without replacement). Pairs will be assigned using nearest neighbor matching within a specified caliper distance. Based on recommendations from Austin (2011b), we use a caliper distance width equal to 0.2 of the standard deviation of the propensity score.

Analysis Plan

Primary analysis.

We will use mixed effects/multilevel modeling for all analyses, estimating both an overall treatment effect (primary analysis) and moderators of that effect (secondary analysis). The multilevel modeling framework will allow us to partition variance into within- and between-school components for comparisons between conditions, accounting for the clustering of students within schools (and therefore estimating standard errors correctly; see (Gelman & Hill, 2007). In the event a school has only one student with a TBI, we will incorporate the partially clustered design (Bauer, Sterba, & Hallfors, 2008). Using the notation of Gelman and Hill (2007), the basic model is specified as

$$\begin{array}{l}{\text{Y}}_i~N\left({\alpha }_{j[i]},{\sigma }^2\right)\hfill \\ {\alpha }_j~N\left({\gamma }_0^{\alpha }+{\gamma }_1^{\alpha }\left({\text{Treatment }}_{\text{COR-TBI}}\right)+{\gamma }_2^{\alpha }(\widehat{\text{Pretest}}),{\sigma }_{{\alpha }_j}^2\right),\text{ for School j}=1,\dots ,\text{J}\hfill \end{array}$$ (2)

where the first line represents individual student-level variation on the outcome and the second represents school -level variation. The given outcome is assumed generated from a normal distribution, with separate intercepts estimated for each school. These varying intercepts are themselves assumed generated by a normal distribution, with the mean structure conditional on the treatment status (given that treatment status varies at the school level and not the individual level) and the mean of pretest scores for the given school. The inclusion of mean pretest scores will reduce between-school variation, increasing statistical power to detect a significant effect, which is estimated by ${\gamma }_1^{\alpha }$ (the average difference on the outcome between COR-TBI schools and TAU schools).

For measures with more than two data points (pre-test, used in the propensity score matching, and post-test, used as the outcome), we extend the model above to account for assessment wave, modeling changes in scores over time (linear growth model) and including the treatment effect as a predictor of both students’ initial status and change over time (slope). The model is defined as

$$\begin{array}{l}{\text{Y}}_i~N\left({\alpha }_{j[i],k[i]}+{\beta }_{1j[i],k[i]}(\text{wave }),{\sigma }^2\right)\\ \left(\begin{array}{c}{\alpha }_j\\ {\beta }_{1j}\end{array}\right)~MVN\left(\left(\begin{array}{c}{\mu }_{{\alpha }_j}\\ {\mu }_{{\beta }_{1j}}\end{array}\right),\left(\begin{array}{cc}{\sigma }_{{\alpha }_j}^2& {\rho }_{{\alpha }_j{\beta }_{1j}}\\ {\rho }_{{\beta }_{1j}{\alpha }_j}& {\sigma }_{{\beta }_{1j}}^2\end{array}\right)\right),\text{ for Student j}=1,\dots ,\text{J}\\ \left(\begin{array}{l}{\alpha }_k\hfill \\ {\beta }_{1k}\hfill \end{array}\right)~MVN\left(\left(\begin{array}{l}{\gamma }_0^{\alpha }+{\gamma }_1^{\alpha }\left({\text{Treatment }}_{\text{COR-TBI}}\right)+{\gamma }_2^{\alpha }(\widehat{\text{Pretest}})\hfill \\ {\gamma }_0^{{\beta }_1}+{\gamma }_1^{{\beta }_1}\left({\text{Treatment }}_{\text{COR-TBI}}\right)\hfill \end{array}\right),\left(\begin{array}{cc}{\sigma }_{{\alpha }_k}^2& {\rho }_{{\alpha }_k{\beta }_{1k}}\\ {\rho }_{{\beta }_{1k}{\alpha }_k}& {\sigma }_{{\beta }_{1k}}^2\end{array}\right)\right),\text{ for School k}=1,\dots ,\text{K}\end{array}$$ (3)

where wave is coded 0, 1, 2, …, n. The β₁ term estimates the average change in the outcome across waves and, along with the intercept, is estimated as randomly varying across students and schools. The intercept and slope variation are assumed generated from a multivariate normal distribution at each level with an unstructured variance covariance matrix (i.e., the covariance between the intercept and slope is estimated, along with each variance). At the school level, treatment status is included as a predictor of the intercept and the slope, with the latter representing a cross-level interaction (school-level treatment status predicting individual change over time). The ${\gamma }_1^{\alpha }$ term estimates baseline equivalency (differences between COR-TBI and TAU schools at the first measure) and allows us to evaluate the effectiveness of the propensity score matching in producing initially equivalent groups. The ${\gamma }_1^{{\beta }_1}$ term estimates the differential change in outcomes from the first through the last assessment between treatment conditions.

Trajectories will be constructed following Singer and Willett (2003) to (a) examine empirical growth plots, (b) fit an unconditional model, (c) fit an unconditional linear growth model, (d) fit an unconditional nonlinear model, (e) compare unconditional linear and non-linear models, and (f) add additional predictors. For non-continuous measures (e.g., disciplinary referrals, use of school supports), multilevel generalized linear model equivalents with the appropriate link function (e.g., complementary log-log link for binary outcomes, Poisson for count outcomes) will be specified.

Secondary analysis.

Moderating effects (e.g., injury outside of school) will be explored by adding the moderator to the models displayed above (i.e., interacting with the treatment effect). In many cases, these interaction terms represent cross-level interactions between the school-level treatment variable and the student- or time-level moderator. However, we will also explore school-level moderators, including Title 1 status and enrollment demographics. These analyses will investigate whether (and the degree to which) the magnitude of the intervention effect depends upon the moderating levels. All significant interactions will be probed by computing sample-estimated intercepts and slopes of the trajectories of the outcomes at conditional levels of the moderator within the COR-TBI and TAU groups separately (i.e., simple slope trajectories) using methods described by Curran (2006).

Process analysis.

A critical element in the evaluation is the acceptability of the COR-TBI intervention to the target population, which will be assessed via the rate of attrition and degree of program satisfaction. We will also test the hypothesis that those participants who report greater satisfaction with the COR-TBI intervention will have greater improvement in outcomes, as shown by the growth models. Treating participant satisfaction as a process variable, we will examine the correlation between satisfaction ratings and the amount of treatment engagement/retention.

Attrition and missing data.

Missing data in outcome measures might result from dropouts or item non-response. The mixed-models described above allow the use of maximum likelihood estimates and for the use of all available outcome data from all assessments, reducing bias and increasing power (Schafer & Graham, 2002). In general, maximum likelihood procedures, as well as imputation methods, will be used because they can provide unbiased estimates even in the presence of substantial attrition (Allison, 2001). Multiple imputation procedures will follow best-practice recommendations (Graham et al., 2007), and the observed and imputed data will be compared to ensure that they show similar distributions (Abayomi et al., 2008). These models assume that missing data are missing at random, conditional on the model covariates, and therefore ignorable; thus, we will conduct sensitivity analyses with pattern-mixture models (Hedeker & Gibbons, 1997) to examine that assumption.

Statistical power.

Based on students identified with TBI in Central Oregon during 2015–2018 we expect, on average, we can recruit approximately 11 students identified with TBI for each of 56 schools, for each of three years, pooling students across elementary, middle, and high schools. Of the approximate 600 students we expect to be identified with TBI, approximately 90%, or 540, will have mild TBI. As mentioned previously, we purposely oversample TAU comparison schools (j = 87) to provide a high likelihood of successful matching and a 1:1 ratio of COR-TBI students identified with TBI to TAU students identified with TBI. For the proposed study we expect 540 students identified with mild TBI nested within each of 56 COR-TBI schools and 87 TAU comparison schools (143 total schools and 1,080 total students identified with mild TBI).

After potential losses due to the expected attrition rate of 20%, we expect complete data from approximately 443 students in the 56 COR-TBI schools and 443 matches students from the TAU schools, or approximately 8 students per school. We conservatively estimate power by excluding loss to attrition, but the missing data procedures described above will allow us to analyze all students who participate, increasing the power. All power estimates are based on a Type I error rate of 5% with a Bonferroni adjusted alpha p-value = .002, a Type II error rate of 20% (i.e., power of 80% or greater), and covariate adjustment of R² = .25. Given formulas in Murray and Hannan (2017; see equations 4, 7–8 and Table 1), Janega and colleagues (2004; see equations 6 and 7), and Spybrook, Hedges, & Borenstein (2014; see Table 3) we estimated this design to produce a minimally detectable effect size (MDES) of d = 0.36. We assumed an ICC (ratio of the between-school variation in the outcome to the total variation) of .15, which represents the average ICC we obtained evaluating post-concussion management in high schools (Glang et al., 2015), and is inline with ICC values commonly found in educational interventions (Hedges & Hedberg, 2007).

The number of students, the strength of covariates, and ICCs could differ from those planned, so we conducted a sensitivity analysis. With 5 to 10 students per school, covariates of .40 to .60 (R² = .16 to .36), and ICCs of .10 to .20, MDES values could range between 0.30 and 0.44. For example, with 10 students per school, a covariate R² = .40, and an ICC of .10, the MDES = 0.30. Studies comparing youth who recovered and had not recovered from TBI that included outcomes we propose, reported effect sizes that exceed the MDES for the current study. Gioia and colleagues (2019) showed differences in the PCSI-2 total score between recovered and not recovered youth associated with d =.84 for parent report, d =.93 for child self-report, and d =.87 for adolescent report. Similarly, Gioia and colleagues (2016) also showed differences in CLASS-3 general concern with school score associated with d =.96 for parent report, d =.78 for child self-report, and d =1.26 for adolescent report. Thus, under several conservative assumptions, the study should have the power to detect meaningful effects for students with mild TBI.

Qualitative Methods

The purpose of the concurrent qualitative component of the study is to describe how children, parents, school and medical personnel in both the model and comparison sites experience the return to school process for students with TBI. We will conduct both focus groups and interviews of key stakeholders across four primary groups: (a) students who experience TBI, (b) parents of students with TBI, (c) educators (e.g., counselors, classroom teachers, speech/language pathologists, school psychologists, school administrators), and (d) health care professionals (e.g., athletic trainers, school nurses, physicians, emergency room personnel). A semi-structured focus group protocol will be developed with questions aimed at understanding the barriers and facilitators for the return to school process. We will explore the components of COR-TBI model implementation that support various contexts through focus groups and interviews to provide a rich description for how this model has been sustained across time and to learn about potential replication procedures. These interview data will be used to address Research Question 3.

Data from focus groups and interviews will be analyzed using a thematic analysis approach. The transcribed data will be entered into the qualitative data analysis application Dedoose (2019). Consistent with Braun and Clarke (2006), analysis will provide a rich description of the data sets, use an inductive (bottom-up) coding process, and describe semantic themes from a realist epistemology. The analysis will begin with an intensive review of transcripts by a minimum of two experienced qualitative coders to identify emergent themes across stakeholder groups and settings. This analysis will be iterative and though codes will be generated initially, overarching themes and codes will be derived from the data throughout the analysis process until a final codebook is created. Research staff will independently conduct first-level coding to establish broad themes within the responses to the focus questions. The same research staff will then independently conduct the second level of coding, meeting several times throughout to discuss areas of disagreement until consensus is reached (Braun & Clarke, 2006).

Discussion

The results of this study will inform service provision for students who experience TBI and provide a blueprint for broad dissemination of the COR-TBI model, a longstanding return to school model incorporating all of the components recommended for an effective return to school program. We believe the COR-TBI model will be an efficient and sustainable strategy for improving TBI management that can be easily disseminated and implemented in a variety of educational contexts. The rich data provided through qualitative interviews will provide key information about how children, parents, school and medical personnel in both the model and comparison sites experience the return to school process for students with TBI. In addition, qualitative findings will allow us to define the key levers of implementation to provide information about how to replicate and scale up the COR-TBI model. At the same time, results from the quantitative analyses will provide effect size estimates on an intent-to-treat basis (and, secondarily, on a treatment-on-the-treated basis) documenting the impact of the COR-TBI model on students’ health, social, and academic outcomes relative to typical treatment models.