Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 Nov 1.
Published in final edited form as: Child Youth Serv Rev. 2018 Sep 29;94:306–314. doi: 10.1016/j.childyouth.2018.09.038

Community-Based Learning Collaboratives and Participant Reports of Interprofessional Collaboration, Barriers to, and Utilization of Child Trauma Services

Rochelle F Hanson 1,*, Benjamin E Saunders 1, Samuel O Peer 2, Elizabeth Ralston 3, Angela D Moreland 1, Sonja Schoenwald 4, Jason Chapman 4
PMCID: PMC6516766  NIHMSID: NIHMS1510221  PMID: 31105370

Abstract

Given the high prevalence and severe consequences of child trauma, effective implementation strategies are needed to increase the availability and utilization of evidence-based child trauma services. One promising strategy, the Community-Based Learning Collaborative (CBLC), augments traditional Learning Collaborative activities with a novel set of community-focused strategies. This prospective, observational study examined pre-to post-changes in CBLC participant reports of interprofessional collaboration (IPC), barriers to, and utilization of evidence-based child trauma treatment in their communities. Participants of five CBLCs from a statewide dissemination initiative, comprising 572 child abuse professionals (296 clinicians, 168 brokers, and 108 senior leaders), were surveyed pre-and post-CBLC participation. Results suggested that CBLCs significantly decreased barriers to child trauma treatment and significantly increased IPC and perceived utilization of evidence-based child trauma treatment. Further, changes in barriers partially mediated this relationship. Finally, small to medium differences in participants’ reports were detected, such that senior leaders perceived significantly greater IPC than clinicians and brokers did, while brokers perceived significantly greater barriers to child trauma treatment than clinicians and senior leaders did. Collectively, these preliminary findings suggest the CBLC implementation model–which augments traditional Learning Collaborative models with a focus on fostering IPC–can reduce barriers and increase the utilization of evidence-based mental health treatment services.

Keywords: Community-Based Learning Collaborative, implementation, interprofessional collaboration, child mental health service utilization, Trauma-Focused Cognitive Behavior Therapy

1. Introduction

1.1. Background

One in five children in the United States experiences a mental health disorder with severe impairment and/or distress (U.S. Public Health Service, 2000). Further, data from national surveys indicate a substantial number of children experience some type of traumatic event over their lifetimes (Finkelhor, Turner, Shattuck, & Hamby, 2015; Finkelhor et al., 2013; Saunders & Adams, 2014; U.S. Department of Health and Human Services, 2016), with most exposed to multiple events (Finkelhor, Ormrod, & Turner, 2007; Finkelhor et al., 2015). Trauma exposure during childhood and/or adolescence accounts for 28% of all psychiatric disorders (McLaughlin et al., 2012), with increased risk for myriad physical and behavioral health problems, such as posttraumatic stress disorder, depression, suicidality/self-harm, substance use, delinquent and other risk behaviors, neurological deficits, and physical health concerns (Anda et al., 2006; Begle et al., 2011; Cisler et al., 2011; Danielson et al., 2006; Danielson et al., 2010; Finkelhor, Turner, Ormrod, Hamby, & Kracke, 2009; Hanson et al., 2008; Kendall-Tackett, 2002; Lalor & McElvaney, 2010; Thompson, Arias, Basile, & Desai, 2002; Vermeiren et al., 2002; Waldrop et al., 2007; Zinzow et al., 2009), which often persist into adulthood (De Bellis & Zisk, 2014; Felitti et al., 1998; Hanson et al., 2001; Shonkoff et al., 2012).

Although trauma-focused evidence-based treatments (EBTs) for children and youth have been developed and tested (Chaffin & Friedrich, 2004; Dorsey, Briggs, & Woods, 2011; Silverman, et al., 2008; Wetherington et al., 2008) and increasingly adopted, at least initially, (Goldman Fraser et al., 2014), child mental health and welfare systems face challenges in sustained implementation of these EBTs so that they are readily available and accessible. The National Child Traumatic Stress Network, funded by the Substance Abuse Mental Health Services Administration, has widely disseminated the Learning Collaborative (Ebert, Amaya-Jackson, Markiewicz, Kisiel, & Fairbank, 2012) model, an adaptation of the Institute of Healthcare Improvement’s Breakthrough Series, (IHI, 2003; Kilo, 1999), as a multi-component implementation strategy for training community-based therapists to deliver trauma-focused treatments. In brief, Learning Collaboratives involve agency teams, comprised of clinical providers, supervisors, and senior leaders (e.g., agency directors or program managers), who work together to learn an EBT and sustain its use over time. Learning Collaborative implementation strategies include preliminary or ‘pre-work’ activities (e.g., independent readings prior to in-person training) to establish baseline knowledge about the targeted EBT, in-person trainings to provide direct training and skill acquisition, and action periods between trainings that involve ongoing expert consultation and quality improvement strategies, such as Plan, Do, Study Cycles which outline specific steps to develop, test, and modify a new practice (IHI, 2017); and ongoing data collection with feedback to participants.

Learning Collaboratives typically involve single agencies from a sole service sector, such as mental health. Other service sectors (e.g., schools, juvenile justice, child welfare) are not involved usually, and may not be aware of the availability and/or utility of the new service, nor of how to help their clients access and engage in it. Ergo, simply implementing a new EBT in a single agency may have little effect on its community’s overarching service system.

In response to this limitation, the standard Learning Collaborative model was augmented to include a broad range of community agencies as part of a statewide initiative whose goal was to increase availability and use of child trauma-focused practices (Hanson, et al., 2016; Saunders & Hanson, 2014). This adaptation, the Community-Based Learning Collaborative (CBLC) model, includes many of the same implementation strategies as the original Learning Collaborative model (e.g., in-person learning sessions, training in quality improvement methods, ongoing data collection with feedback to participants, consultation and coaching to address implementation barriers) and inclusion of senior leaders because of their demonstrated importance in creating a supportive infrastructure for sustained EBT delivery (Aarons et al., 2011). However, the CBLC expands the focus of EBT implementation, from a single agency or service system, to agencies across multiple service systems within a community. The CBLC also involves jointly training, not only clinicians and senior leaders, but also “brokers” (e.g., child welfare workers, juvenile justice workers, school counselors, guardians ad litem, victim advocates): individuals often engaged in identification of children who may have experienced abuse or another potentially traumatic event to determine whether they need mental health services, refer them for mental health services if warranted, and monitor their progress (see Hanson et al., 2016; Saunders & Hanson, 2014 for further description of the CBLC model).

CBLC phases map onto the Exploration, Preparation, Implementation, and Sustainment (EPIS; Aarons, Hurlburt, & Horwitz, 2011) framework and include empirically-identified strategies to target inner and outer contextual factors (e.g., strong leadership, information sharing, goals, organizational culture and climate; Aarons, et al., 2011; Chaudoir, Dugan, & Barr, 2013; Durlak & DuPre, 2008; Fixsen, Naoom, Blasé, Friedman, & Wallace, 2005; Greenhalgh, Robert, McFarlane, Bate, & Kyriakidou, 2004; Palinkas et al., 2012) to promote sustained EBT implementation. The CBLC aims to strengthen interprofessional collaboration (IPC), defined as “the process in which different professional groups work together to positively impact health care” (p. 2; Zwarenstein, Goldman, & Reeves, 2009), as prior studies indicate that greater IPC between child welfare and mental health service organizations can help to increase service access, mental health treatment utilization, and positive treatment outcome (Bai et al., 2009; Chuang & Luci, 2011; Glisson, 1994; Hurlburt et al., 2004; Palinkas et al., 2012).

An important question that remains, however, relates to the best methods for building strong IPC. A Cochrane review by Zwarenstein et al. (2009) indicated that multidisciplinary meetings and staffings, especially those with an external facilitator who encouraged collaborative working, were associated with positive changes in health care. Studies have also indicated that IPC can be strengthened by fostering a shared vision and goals, such as ensuring that children in need of treatment services are identified and referred to appropriate providers (e.g., Drabble, 2007; Chong, Aslani, & Chen, 2013; Supper, Catala, Lustman, Chemla, Bourgueil, & Letrilliart, 2015). Thus, training and implementation efforts that provide multiple opportunities for cross-discipline interactions and shared learning and promote IPC as a key component of service provision may work to reduce service barriers and improve outcomes for children.

1.2. Current Study

As described above, CBLC Learning Sessions provide opportunities for clinical, broker and senior leader professionals to interact and learn together, and the training curricula emphasize the importance of professional accountability and shared responsibilities across service systems to promote IPC. Thus, the purpose of this study was to examine whether there were pre-to post-CBLC changes in participants’ reports of IPC, barriers to service access, and treatment utilization (defined as the estimated percentage of children who engaged in and completed evidence-based trauma treatment). Based on prior research, we hypothesized that participants would report (1) increased IPC, (2) decreased barriers to accessing services, and (3) increased treatment utilization. We also hypothesized that reductions in barriers would mediate the relation between changes in collaboration and treatment utilization.

Finally, research studies examining organizational culture and climate suggest that perceptions may vary depending on an individual’s role within a given agency (Martin et al., 2006; Zyphur et al., 2016) and that this lack of concordance can adversely impact organizational outcomes (Aarons et al., 2015; Aarons et al., 2016; Wolf, Dulmus, Maguin, Keesler & Powell, 2014). For example, in Beidas and colleagues’ (2016) study examining perceptions of organizational culture and climate within behavioral health settings, administrators reported more positive cultures and climates than clinicians did, and the authors noted these findings were consistent with prior research (e.g., Aarons et al, 2015; Carljord et al., 2010; Hansen et al., 2011; 2012). The authors posited that individuals have different qualitative experiences based upon their professional roles and responsibilities, and because role differentiation characterizes participants of CBLCs, similar findings may be anticipated with respect to individual perspectives on IPC both within and across agencies and service systems. We thus examined whether perceived IPC and barriers to service access varied across participant roles (i.e., clinicians, brokers, and agency leaders). Given prior research, we hypothesized that agency leaders would perceive greater levels of IPC and fewer barriers as compared to front-line clinician and/or broker professionals.

2. Methods

2.1. Participants

The current study focused on Phase 3 of Project BEST (Bringing Evidence-Supported Treatments to South Carolina Children and Their Families; www.musc.edu/projectbest), a four-phase statewide initiative to disseminate trauma-focused practices across South Carolina. Data from five of the six CBLCs conducted during Phase 3 were analyzed, as the measure of IPC (see below) was added after completion of the first CBLC. The number of participants in each of the five CBLCs varied from 101 to 129 (M = 114.40; SD = 10.24). A total of 52 agencies participated in the 5 CBLCs, with a range of 1 to 25 participants per agency (M = 5.25; SD = 6.65). The majority of participants (80%) worked in community clinics operating under the auspices of the South Carolina State Department of Mental Health; 7% of participants were from Child Advocacy Centers; 2% were in private practice; less than 1% were employed by the state juvenile justice department, hospitals, or pediatrician offices; and the remaining 11% were from “Other” agencies. Across the 52 agencies, 572 professionals (296 clinicians, 168 brokers, 108 senior leaders) participated in the five CBLCs from which data were analyzed. Of the participating clinicians, 224 (84%) were therapists, and 41 (16%) were clinical supervisors who also delivered psychotherapy services.

2.2. Procedures

All procedures and measures were approved by the Institutional Review Board of the Medical University of South Carolina. Since Project BEST was primarily a training and implementation project, data were collected for the purposes of program evaluation and quality improvement. For each CBLC, participants (i.e., clinicians, brokers, and senior leaders) completed an online survey with a battery of questionnaires at two time points: (1) prior to attending the first Learning Session and (2) after all CBLC activities were completed. On average, 16.11 months (SD = 5.42) passed between these pre-and post-CBLC surveys. Questionnaires examined factors related to EBT adoption and implementation (e.g., readiness for treatment adoption, attitudes about EBTs, therapeutic practices and skills), with the three measures analyzed in the present study (see below) administered to all three participants roles (i.e., clinicians, brokers, senior leaders) at both time points.

2.3. Measures

Data analyzed involved three measures rationally designed specifically for the training project to assess IPC, barriers to, and utilization of child trauma services.

2.3.1. Interprofessional collaboration with respect to child trauma services

Participants were asked to indicate how well agencies in their community engaged in the following activities on a 5-point Likert scale, ranging from 0 = poor to 4 = excellent: Work together to overcome barriers; Work to ensure clients complete treatment; Make sure clients don’t fall through cracks; Coordinate services, not overwhelmed; and Overall quality of collaboration. They were also asked how often they engaged in the following on a 5-point Likert scale, ranging from 0 = rarely to 4 = nearly always: Frequency of sharing assessment information; Frequency of sharing treatment progress information; and Frequency that agencies meet. Total scores could range from 0 to 32 with higher scores indicating more collaboration among professionals in the community. For the pre-and post-CBLC samples, the overall internal consistency was excellent for all three participant roles (αs = .91–.95).

2.3.2. Barriers to child trauma treatment

These 13 items measured participant perceptions of commonly occurring barriers that may prevent children and youth in their community who have experienced abuse or another traumatic event from receiving trauma-focused EBTs. Respondents rated how commonly they perceived each barrier to occur (e.g., Not enough trained therapists; Insurance will not pay for EBTs; Brokers unaware of appropriate EBTs; Long wait lists for EBTs; Families unable to attend office-based treatment) in their community on a 5-point Likert scale, ranging from 0 (Rarely a barrier) to 4 (Nearly always a barrier.) Total scores could range from 0 to 52, with higher scores indicating greater perceived problems with trauma treatment barriers. For the pre-and post-CBLC samples, the overall internal consistency was good to excellent for all three types of participant (αs = .88–.91).

2.3.3. Utilization of evidence-based child trauma treatment in the community

Due to budget constraints and the fact that these data were drawn from a training initiative, not a research study, we were unable to track cases through the youth service systems (i.e., child welfare, mental health). Thus, a single item was used to obtain participants’ estimates about trauma treatment utilization across their community. Specifically, participants were asked to estimate the percentage of children and families who engaged in and completed evidence-based trauma treatment before and after the CBLC (Thinking about all of the abused and traumatized children in your community, what is your best estimate of the percentage of those children that engage in, receive, and successfully complete evidence-based trauma treatment?) Participants answered with a percentage estimate ranging from 0 to 100.

2.4. Data Analytic Strategy.

Data were analyzed using SPSS 24 software (IBM Corp., 2016).

2.4.1. Nesting

The potential effects of nesting as a function of CBLC or agency type (e.g., Department of Mental Health, Juvenile Justice, Social Services, Child Advocacy Center, private practice) on variables of interest were modeled as fixed effects, using linear mixed-effects models, and both were found to be non-significant for all variables (i.e., CBLC ps = .08–. 97; agency type ps= .07–. 94), and thereafter dropped from subsequent analyses. Modeling specific agency effects was unviable given considerable data singularity (i.e., the mode and median number of participants per agency was one and two, respectively). Ergo, all analyses were conducted and reported here at the participant level (Hox, 2010). Additionally, time between pre-and post-CBLC surveys was not significantly related to any of the measured variables.

2.4.2. Missing data

Of the 572 initial CBLC participants, 564 (99%) completed at least part of the pre-CBLC survey. These included 293 clinicians, 167 brokers, and 104 senior leaders. In contrast, post-CBLC surveys were completed, at least partially, by 165 clinicians, 65 brokers, and 76 senior leaders, totaling 306 participants (54%). The results of independent t-tests confirmed that post-CBLC survey completers and non-completers did not significantly differ on any pre-CBLC measured variables (i.e., community barriers, community collaboration, self-reported trauma practices, skills and knowledge, organization support/barriers, and supervision).

Among those who submitted survey data, not all participants completed each survey item. For pre-CBLC surveys, all items, 100 cases (17%), and 333 values (3%) had missing data. Little’s test indicated the aforementioned pre-CBLC data were missing completely at random for all three types of participants (i.e., clinicians, χ2[517] = 572.16, p = .10; brokers, χ2[411] = 415.49, p = .43; and senior leaders, χ2[309] = 340.15, p = .11). Similarly for submitted post-CBLC surveys, all items, 35 cases (21%), and 156 values (4%) had missing data. Little’s test indicated these post-CBLC data were missing completely at random for all three types of participant (i.e., clinicians, χ2[353] = 363.65, p = .34; brokers, χ2[121) = 144.11, p = .08; and senior leaders, χ2[225] = 194.26, p = .93).

Given evidence that missing data from submitted surveys were MCAR and constituted less than 5% of values (Graham, 2009), the expectation-maximization algorithm was used to replace missing values for any CBLC participant who partially completed the pre-or post-CBLC survey, as expectation-maximization in these circumstances produces values that closely approximate observed data (Lin, 2010; Twala, 2009). All analyses were conducted with and without expectation-maximization imputation. Since the significance of these results remained invariant, only results with expectation-maximization imputation are reported.

2.4.3. Pre-to post-CBLC changes

Repeated-measures ANOVAs were conducted to evaluate the effect of the CBLC (i.e., from pre-to post-CBLC surveys) on participant perceptions of the three constructs of interest: IPC with respect to child trauma treatment, barriers to accessing such treatment, and utilization of such treatment. For significant main effects of participant role (i.e., clinician, broker, senior leader), follow-up contrasts were conducted, with effect sizes reported as Hedges’ g values due to different subsample sizes. Given the number of significance tests conducted, a Bonferroni correction was applied to correct the inflated risk of Type I errors.

2.4.4. Mediation

To assess the effect of pre-to post-CBLC changes in IPC on treatment utilization, both directly and indirectly through changes in service barriers, a simple (i.e., unmoderated) mediation model was analyzed using Preacher and Hayes’ (2004) bootstrap tests. Specifically, pre-to post-CBLC change scores were calculated by subtracting pre-CBLC values from post-CBLC values for each of the three variables in the mediation model (i.e., IPC, barriers, and treatmemt utilization). Thereafter, the PROCESS macro for SPSS (Version 3.0; Hayes, 2017b) was used to compute bias-corrected bootstrap confidence intervals (Efron, 1987; Efron & Tibshirani, 1993) for total, direct, and indirect effects with 5,000 bootstrap samples.

3. Results

3.1. Pre-to Post-CBLC Changes in IPC, Barriers, and Treatment Utilization

Particularly salient pre-to post-CBLC findings are reported below; see Table 1 for full results.

Table 1.

Pre-and Post-Community-Based Learning Collaborative Differences in Child Trauma Service Variables as Reported by Participant Role

Variable Participant Role Pre-
CBLC		 Post-
CBLC		 Fe p ηp2
M M
(SD) (SD)
Collaboration Totala 22.36
(5.95)		 24.94
(6.27)		 60.04 <.001 .17
Clinicianb 21.88
(6.38)		 23.73
(6.08)
Brokerc 21.62
(5.72)		 24.97
(6.14)
Senior Leaderd 24.03
(4.82)		 27.55
(6.05)
Barriers Totala 35.14
(9.14)		 29.49
(8.26)		 98.48 <.001 .25
Clinicianb 33.55
(8.92)		 28.09
(7.37)
Brokerc 38.32
(8.83)		 33.21
(9.21)
Senior Leaderd 35.90
(9.17)		 29.12
(8.28)
Utilization Totala 31.97
(21.02)		 45.62
(22.88)		 63.04 <.001 .17
Clinicianb 30.75
(21.65)		 45.86
(23.52)
Brokerc 32.39
(17.79)		 40.77
(22.92)
Senior Leaderd 34.27
(22.19)		 49.26
(20.90)
Open in a new tab

Note.

a

N = 306.

b

n = 165.

c

n = 65.

d

n = 76.

e

df = 1, 303.

ηp2 = partial eta squared.

3.1.1. IPC with respect to child trauma treatment

As hypothesized, participants on average reported a significant increase in community collaboration related to child trauma treatment from pre-CBLC (M = 22.36, SD = 5.95) to post-CBLC (M = 24.94, SD = 6.27), F(1, 303) = 60.04, p < .001, ηp2 = .17. Descriptively, IPC items that increased the most during the CBLC were: working together so clients don’t fall through cracks in the system, working to insure clients complete treatment, and working together to overcome barriers to treatment. There was a significant, small-to medium-sized main effect for participant role [F(2, 303) = 8.89, p < .001, ηp2 = .06], such that senior leaders generally perceived significantly greater community collaboration related to child trauma treatment than did clinicians; p < .001, g = .36 (95% CI [.14, .59]); or brokers; p = .005, g = .46; whereas, the difference between clinician-and broker-perceived community collaboration of child trauma services was both trivial and non-significant, p = .52, g = .04. There was no significant interaction between time and participant role, F(2, 303) = 2.64, p = .07, ηp2 = .02 (see Figure 1).

Figure 1.

Figure 1.

Open in a new tab

Pre– to post-Community-Based Learning Collaborative (CBLC) changes in inter-professional collaboration of child trauma services as reported by CBLC participant role.

3.1.2. Barriers to child trauma services

Also as hypothesized, participants typically reported a significant, large decrease in community barriers to child trauma services from pre-CBLC (M = 35.14, SD = 9.14) to post-CBLC (M = 29.49, SD = 8.26), F(1, 303) = 98.48, p < .001, ηp2 = .25. Descriptively, items on the community barriers measure that decreased the most during the CBLC were: brokers being unaware of appropriate EBTs for their clients, EBTs not being included in service plans, not having enough trained therapists, EBTs not being offered in locations near clients, and long wait lists for EBTs and few agencies delivering EBTs with fidelity. In contrast, the most commonly reported barrier, both pre-and post-CBLC, was: families unable to attend office-based treatment.

As with IPC, there was no significant interaction between time and participant role, F(2, = 0.82, p = .44, ηp2 < .01, but there was a significant, small-to medium-sized main effect for participant role; F(2, 303) = 11.79, p < .001, ηp2 = .07. That is, brokers reported significantly greater barriers to child trauma services than did clinicians; p < .001, g = .54; or senior leaders; p .005, g = .27. The difference between clinician-and senior leader-reported community barriers to child trauma services was small and non-significant, p = .09, g = .26 (see Figure 2). 3.1.3. Utilization of evidence-based child trauma treatment

Figure 2.

Figure 2.

Open in a new tab

Pre– to post-CBLC changes in barriers to child trauma services in communities as reported by CBLC participant role.

As hypothesized, participants on average reported a significant, large increase in utilization of evidence-based child trauma treatment from pre-CBLC (M = 31.97, SD = 21.02) to post-CBLC (M = 45.62, SD = 22.88), F(1, 303) = 63.04, p < .001, ηp2 = .17. Reported pre-CBLC utilization as well as pre-to post-CBLC changes in utilization did not significantly differ across participant role; F(2, 303) = 1.65, p = .19, ηp2 = .01; and; F(2, 303) = 1.69, p = .19, ηp2 = .01; respectively (see Figure 3).

Figure 3.

Figure 3.

Open in a new tab

Pre– to post- CBLC changes in community utilization of child trauma services as reported by CBLC participant role.

3.2. Mediation

Although demonstrating a significant total effect prior to estimating direct and indirect effects is presently considered unnecessary for mediation (e.g., Cerin & MacKinnon, 2009; Hayes, 2017a; Rucker, Preacher, Tormala, & Petty, 2011; Zhao, Lynch, & Chen, 2010), the total effect was nonetheless statistically significant (c1 = 1.25, p < .001, 95% CI [0.79, 1.71]). In other words, participants on average estimated a 5% increase in child trauma treatment utilization in a community for each point/tier increase (e.g., poor to fair) in reported IPC. Additionally, and as expected, participant-reported increases in IPC with respect to child trauma treatment was significantly related to decreases in reported barriers to child trauma treatment (a1 = −.25, p =.005, 95% CI [−0.42, −0.08]). Further, decreases in perceived barriers to child trauma treatment significantly predicted increases in reported community utilization of evidence-based child trauma treatment, (b1 = −.56, p < .001, 95% CI [−0.86, −0.25]). Finally, pre-to post-CBLC changes in IPC had a positive, significant direct effect (c′1 = 1.11, p < .001, 95% CI [0.65, 1.57]) and a positive, significant indirect effect through changes in barriers (a1b1 = 0.14, 95% CI [0.03, 0.27]) on changes in child trauma treatment utilization in the community. That is, pre-to post-CBLC decreases in reported community barriers to child trauma treatment partially, but not fully, mediated the pre-to post-CBLC increases in IPC and utilization of evidence-based child trauma treatment in the community (Figure 4). Moreover, the direct and indirect effects were complimentary (Zhao et al., 2010).

Figure 4.

Figure 4.

Open in a new tab

Mediational model with pre-to post-CBLC changes in barriers partially mediating the relation between changes in interprofessional collaboration and utilization of child trauma services in communities. c′1 = direct path, a1b1 = indirect path. **p < .01, ***p < .001.

4. Discussion

The aims of this study were to examine pre-to post-changes in CBLC participant reports of IPC, barriers to, and utilization of evidence-based child trauma treatment in the community. Consistent with hypotheses and in line with the CBLC’s underlying EPIS theoretical framework (Aarons et al., 2011), participants reported a significant increase in IPC, decrease in barriers, and increase in utilization of trauma treatment services from pre-to post-CBLC.

4.1. IPC

Overall, reported IPC increased approximately 12% following CBLC participation. More specifically, typically increased IPC items such as working together so clients don’t fall through cracks in the system, sharing of assessment and treatment information, and working to ensure clients complete treatment, reflect a shared vision and goals among participants. These are factors identified in the empirical literature as important facilitators for collaboration across multidisciplinary professionals (e.g., Drabble, 2007; Chong, Aslani, & Chen, 2013; Supper, Catala, Lustman, Chemla, Bourgueil, & Letrilliart, 2015) and targets of the CBLC training.

Interestingly, senior leaders perceived significantly greater positive changes in IPC than clinicians, with no other differences noted across participant roles. In line with prior research on perceptions of organizational culture and climate (Beidas et al., 2016), it is possible that these differing perspectives stem from role-related job responsibilities and the composition of participants in the CBLC training tracks. Specifically, the CBLC senior leader track includes participants from both clinical and broker agencies, with an emphasis on strategies to build inter-and intra-agency infrastructure and support for sustained implementation of trauma-focused services (Hanson et al., 2016; Saunders & Hanson, 2014), as these are important functions of an agency leader. Thus, the senior leader role provides more opportunities for cross-discipline collaboration than the other roles, which alternatively focus on training in a targeted EBT for clinicians and case management and monitoring skills for the brokers. It is also possible that, given their roles as agency directors or program managers, senior leaders play a greater leadership role in their communities, which requires more collaborative activities and thereby offers them a different perspective than front-line professionals. These findings are consistent with extant literature highlighting the fact that perspectives about collaboration are heavily influenced by individual (e.g., how does collaboration impact my productivity requirements?) and organizational (e.g., how does collaboration influence agency referrals?) level priorities and agendas (Aarons & Sawitzky, 2006; Aarons et al., 2014; Aarons, Wells, Zagursky, Fettes, & Palinkas, 2009; Green & Aarons, 2011). Efforts to increase collaboration must take into account these factors and incorporate strategies that foster a shared vision and goals for collaboration across all professional roles and across all phases of implementation.

An additional explanation for the overall increase in collaboration over time, as well as the differential findings between senior leaders and front-line therapists, could stem from recent mandates to create trauma-informed organizations and service systems. Such initiatives emphasize the importance of cross-training and cross-system collaboration to improve quality of care (Hanson et al., 2018; Hanson & Lang, 2016). As senior leaders play an integral role in spearheading these initiatives, they are likely to be especially aware of the need for collaboration to increase their success.

4.2. Community Barriers

Overall, perceived barriers to accessing trauma treatment decreased by approximately 16% following CBLC participation. Moreover, significant decreases in these barriers were noted across all participant groups. This provides preliminary support for the underlying supply and demand premise of the CBLC. Simply stated, the CBLC’s goal to enhance brokers’ knowledge and skill in screening and identifying children potentially in need of treatment interventions built demand, while the increased availability of trained therapists able to deliver EBTs with fidelity may have helped to reduce barriers to accessing trauma treatment in the participating communities. These findings are in line with the Gateway Provider model (Stiffman, Pescosolido Cabassa, 2004), which emphasizes the importance of increasing brokers’ (i.e., the ‘gateway providers’) knowledge, awareness, and skill in the initial screening and referral of youth, which can facilitate access to needed treatment services.

It is, however, important to acknowledge that brokers reported significantly more barriers than either clinicians or senior leaders did at both pre-and post-CBLC. While the statewide initiative, Project BEST, aimed to increase the capacity of trained therapists and thereby help to decrease service barriers, brokers possess a unique perspective on service access. For example, brokers frequently are the first points of contact with youth and are responsible for determining whether and what services are needed. They often have significantly larger case loads than clinical providers, and may be more knowledgeable about the scope of barriers across a much larger set of cases, distributed across more communities, than therapists and their agency leaders. Also, as discussed by Stiffman et al. (2004), despite an increased awareness of how to identify youth in need of services, the perception that skilled providers remain unavailable can present an ongoing barrier to treatment access and successful completion.

It is noteworthy that clients’ inability to attend office-based treatment was the barrier (i.e., item on the Barriers to Child Trauma Treatment measure) most commonly reported both before and after the CBLCs. This finding suggests that innovative methods of treatment service delivery–such as community-based services (e.g., de Arellano, Waldrop, Deblinger, Cohen, Danielson, & Mannarino, 2005), integrated behavioral health care (Meadows Mental Health Policy Institute, 2016), use of tablets or iPads (Ruggiero, Saunders, Davidson, Cook & Hanson, in press) and telehealth services (Comer et al., 2015; Gloff, LeNoue, Novins, & Myers, 2015; Goldstein & Glueck, 2016)–may offer viable ways to increase access to treatment for abused and traumatized children. As one particularly relevant example, a recent study by Stewart and colleagues (2017) indicated that Trauma-Focused Cognitive Behavioral Therapy (Cohen et al., 2017), a child-focused EBT for trauma-related mental health problems, delivered via telehealth achieved treatment effects that were comparable to treatment delivered in an in-person, office-based setting, providing preliminary evidence for a promising approach to overcome service access barriers.

4.3. Mediational Analyses

Findings of these analyses showed that the relationship between changes in perceived IPC and greater utilization of child trauma treatment were only partially mediated by a reduction in barriers to such treatment. This highlights the direct impact of increased IPC on perceived treatment utilization. These findings were consistent with prior research (e.g., Bai et al., 2009) and also support the basic premise of the CBLC approach, namely that training and implementation strategies designed to increase IPC can help to increase the reach of evidence-based interventions. Furthermore, the direct effect of IPC upon estimated treatment utilization was nearly ten times the magnitude of the analyzed indirect effect, which suggests that other complimentary factors made significant contributions to the associations between increased collaboration and utilization (Zhao et al., 2010). For example, as discussed above, it is possible that the opportunities for collaboration among participants increased awareness of available mental health service providers and thereby resulted in the perception that more children were engaging in and completing treatment. It is also possible that the noted increases in sharing of assessment information, one of the items on the IPC scale, may have facilitated more appropriate trauma-focused treatment referrals and/or tailored treatment, which in turn could have increased the likelihood of engagement and successful treatment completion (Cohen, Mannarino, & Deblinger, 2017; Saunders, Berliner, & Hanson, 2004). Future research to examine these relations is warranted.

4.4. Limitations

Despite these promising findings, several limitations must be noted. First, Project BEST was a training and services initiative, not a research study, and as a result, there was no comparison condition or random assignment to conditions. The initiative enabled us to begin to examine the impact of the CBLC on collaboration, and whether this results in increased service utilization, one indicator of penetration (i.e., defined as how well practices are integrated within service settings), which has been identified as an important implementation outcome (Proctor et al., 2011).

A second limitation stems from sole reliance on self-report, with measures designed specifically for this project. Data analyses provided preliminary psychometric support for these measures, but further validation is clearly needed, especially as normative levels of IPC and the related clinical significance of a reported 12% increase are unknown. Additionally, results would be strengthened by collateral sources of data, such as organizational and administrative-level measures, which could provide objective indicators of collaboration and the numbers of children who are screened, referred, and complete mental health treatment. Unfortunately, as noted above, budget constraints and the fact that this was a training initiative, rather than a research study, precluded our abilities to collect these objective data.

A third limitation concerns the measure of treatment utilization. First, this was a single item, triple-barreled question, which asked about access to, engagement in, and completion of treatment services. This precluded our ability to separately examine the impact of collaboration on service access versus completion of treatment services. This is an important distinction, warranting future research. Second, resources were not available to track child abuse cases through community service systems to objectively determine what percentage of children and families engaged in and completed evidence-based trauma treatment before and after the CBLC. Rather, this item was an estimate by participants, based on their knowledge about children and families within their community. Third, we did not provide any instructions to participants with regard to the type of data they could utilize to answer this question. Thus, we recognize that some of the participants, especially senior leaders, may not have had the requisite first-hand experience (i.e., direct contact with clients) to answer this question.

A fourth limitation is related to missing data and the concern of biased findings since those who completed pre-and post-CBLC measures were likely more invested in the training and more positive about the impact of the CBLC on community practices. However, analyses indicated that data were missing completely at random, and there were no significant differences between completers and non-completers on the variables of interest.

Fifth, we were not able to model nesting of participants within specific agencies, due to the aforementioned singularity of data. We recognize that unmodeled nesting can be a serious problem. Specifically, unmodeled dependency can have a dramatic impact on de facto sample size and related statistical power and hypothesis testing. Moreover, our data’s level of singularity makes it difficult to differentiate participant and agency effects (even if results suggested that the effect of agency type was insignificant).

Finally, we recognize a limitation in our mediation testing. Namely, there is a temporal confound as both the mediator and outcome processes were measured on the same schedule and only at two timepoints. This means that we could not establish the temporal precedence of the mediator process (i.e., as a predictor of later change in the outcome process). Future studies using longitudinal designs with more objective and/or validated measures and greater experimental control will be necessary to bolster and clarify our present findings.

5. Conclusion

Despite these limitations, findings related to the role of collaboration in increasing utilization of evidence-based treatment for children and families converged with prior research (e.g., Bai et al., 2009; Chuang & Lucio, 2011) and provide initial support for the CBLC training and implementation approach. Notably, the CBLC’s emphasis on cross-training of brokers, clinicians, and their senior leaders provide increased opportunities for collaboration, with the goal of improving care coordination to increase the likelihood that children engage in and complete treatment successfully. Study findings also indicated that IPC has more impact than simply reducing specific treatment barriers and point to the extant data highlighting aspects of collaboration that may be important for expanding the reach of EBTs, such as increased awareness of other community professionals and shared vision and goals for youth and families. Further, brokers cited more barriers both pre-and post-training than other CBLC participants, a result that highlights an area of improvement for future CBLCs and similar implementation models. Collectively, these findings provide supportive, if preliminary, evidence for the CBLC model and suggest it facilitates sharing of differing perspectives and mutual goals, cross-discipline discussions to develop strategies to address identified barriers, and ultimately improving outcomes for youth and their families.

Highlights.

  • Learning collaborative participation reduces barriers to child trauma services

  • Cross-discipline training fosters increased collaboration for child trauma services

  • Learning collaborative participation increases perceived use of trauma treatment

  • Perceptions of service barriers and collaboration varies by professional role

Acknowledgements

Preparation of this manuscript was supported by the Duke Endowment (PI: Benjamin Saunders and Elizabeth Ralston) and NIMH Grant No. R34MH104470 (PI: Rochelle Hanson). We also thank participants in this study as well as the help and support of our research assistants, Faraday Davies and Emily Fanguy.

Abbreviations

CBLC

Community-Based Learning Collaborative

IPC

Interprofessional Collaboration

EBT

Evidence-Based Treatments

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Conflict of Interest

We wish to confirm that there are no known conflicts of interest associated with this publication and there has been no significant financial support for this work that could have influenced its outcome. We confirm that the manuscript has been read and approved by all named authors and that there are no other persons who satisfied the criteria for authorship but are not listed. We further confirm that the order of authors listed in the manuscript has been approved by all of us. We confirm that we have given due consideration to the protection of intellectual property associated with this work and that there are no impediments to publication, including the timing of publication, with respect to intellectual property. In so doing we confirm that we have followed the regulations of our institutions concerning intellectual property. We further confirm that this research has been approved by the university institutional review board. We understand that the Corresponding Author is the sole contact for the Editorial process. He/she is responsible for communicating with the other authors about progress, submissions of revisions and final approval of proofs. We confirm that we have provided a current, correct email address, which is accessible by the Corresponding Author.

References

  1. Aarons GA, Ehrhart MG, Farahnak LR, Sklar M, & Horowitz J (2015). Discrepancies in leader and follower ratings of transformational leadership: Relationship with organizational culture in mental health. Administration and Policy in Mental Health 44(4), 480–491. doi: 10.1007/s10488-015-0672-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Aarons GA, Ehrhart MG, Torres EL, Finn NK, & Beidas RS (2016). The humble leader: Association of discrepancies in leader and follower ratings of implementation leadership with organizational climate in mental health organizations. Psychiatric Services, 68(2), 115–122. doi: 10.1176/appi.ps.201600062. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Aarons GA, Fettes DL, Hurlburt MS, Palinkas LA, Gunderson L, Willging CE, & Chaffin MJ (2014). Collaboration, negotiation and coalescence for interagency-collaborative teams to scale-up evidence based practice. Journal of Clinical Child & Adolescent Psychology, 43(6), 915–928. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Aarons GA, Hurlburt M, & Horwitz SM (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38, 4–23. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Aarons GA, & Sawitzky AC (2006). Organizational culture and climate and mental health provider attitudes toward evidence-based practice. Psychological Services, 3(1), 61–72. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Aarons GA, Wells RS, Zagursky K, Fettes DL, & Palinkas LA (2009). Implementing evidence-based practice in community mental health agencies: A multiple stakeholder analysis. American Journal of Public Health, 99(11), 2087–2095. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Bai Y, Wells R, & Hillemeier MM (2009). Coordination between child welfare agencies and mental health service providers, children’s service use, and outcomes. Child Abuse & Neglect, 33, 372–381. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Bickman L, Lambert EW, Andrade AR, & Penaloza RV (2000). The Fort Bragg continuum of care for children and adolescents: Mental health outcomes over 5 years. Journal Of Consulting And Clinical Psychology, 68(4), 710–716. [PubMed] [Google Scholar]
  9. Bickman L, Noser K, & Summerfelt WT (1999). Long-term effects of a system of care on children and adolescents. Journal of Behavioral Health Services & Research, 26, 185–202. [DOI] [PubMed] [Google Scholar]
  10. Carljord S, Andersson A, Nilsen P, Bendtsen P, & Lindberg M (2010). The importance of organizational climate and implementation strategy at the introduction of a new working tool in primary health care. Journal of Evaluation in Clinical Practice, 16(6), 1326–1332. [DOI] [PubMed] [Google Scholar]
  11. Cerin E, & MacKinnon DP (2009). A commentary on current practice in mediating variable analysis in behavioral nutrition and physical activity. Public Health Nutrition, 12, 1182–1188. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Chaudoir SR, Dugan AG, & Barr CH (2013). Measuring factors affecting implementation of health innovations: A systematic review of structural, organizational, provider, patient, and innovation level measures. Implementation Science, 8(22), 1–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Chuang E & Lucio R (2011). Interagency collaboration between child welfare agencies, schools, and mental health providers and children’s mental health service receipt. Advances in School Mental Health Promotion, 4(2), 4–15. [Google Scholar]
  14. Chong WW, Aslani P, & Chen TF, (2013) Shared decision-making and interprofessional collaboration in mental healthcare: A qualitative study exploring perceptions of barriers and facilitators. Journal of Interprofessional Care, 27(5), 373–379. [DOI] [PubMed] [Google Scholar]
  15. Cohen JA, Mannarino AP, & Deblinger E (2017). Treating trauma and traumatic grief in children and adolescents (2nd ed). New York, NY: Guilford Press. [Google Scholar]
  16. Comer JS, Furr JM, Cooper–Vince C, Madigan RJ, Chow C, Chan PT, . . . Eyberg SM (2015). Rationale and considerations for the Internet-based delivery of Parent Child Interaction Therapy. Cognitive and Behavioral Practice, 22, 302–316. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. de Arellano MA, Waldrop AE, Deblinger E, Cohen JA, Danielson CK, & Mannarino AR (2005). Community outreach program for child victims of traumatic events: A community-based project for underserved populations. Behavior Modification, 29, 130– 155. [DOI] [PubMed] [Google Scholar]
  18. DeLeon PH, Wakefield M, & Hagglund KJ (2003). The behavioral health care needs of rural communities. In Stamm B & Stamm B (Eds.), Rural behavioral health care: An interdisciplinary guide (pp. 23–31). Washington, DC: American Psychological Association. [Google Scholar]
  19. Drabble L (2007). Pathways to collaboration: Exploring values and collaborative practice between child welfare and substance abuse treatment fields. Child Maltreatment, 12, 31–42. [DOI] [PubMed] [Google Scholar]
  20. Durlak JA, & DuPre EP (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327–350. [DOI] [PubMed] [Google Scholar]
  21. Efron B (1987). Better bootstrap confidence intervals. Journal of the American Statistical Association, 82, 171–185. [Google Scholar]
  22. Efron B, & Tibshirani RJ (1993). An introduction to the bootstrap Boca Raton, FL: CRC Press. [Google Scholar]
  23. Finkelhor D (2017). Screening for adverse childhood experiences (ACEs): Cautions and suggestions. Child Abuse & Neglect, 10.1016/j.chiabu.2017.07.016 [DOI] [PubMed]
  24. Fixsen DL, Naoom SF, Blase KA, Friedman RM, & Wallace F (2005). Implementation research: A synthesis of the literature Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute; Retrieved November 13th, 2017 from http://nirn.fpg.unc.edu/sites/nirn.fpg.unc.edu/files/resources/NIRN-MonographFull-01-2005.pdf [Google Scholar]
  25. Glisson C, & Hemmelgarn A (1998). The effects of organizational climate and interorganizational coordination on the quality and outcomes of children’s service systems. Child Abuse & Neglect, 22(5), 401–421. [DOI] [PubMed] [Google Scholar]
  26. Gloff NE, LeNoue SR, Novins DK, & Myers K (2015). Telemental health for children and adolescents. International Review of Psychiatry, 27, 513–524. [DOI] [PubMed] [Google Scholar]
  27. Goldman Fraser J, Griffin JL, Barto BL, Lo C, Wenz-Gross M, Spinazzola J, … Dym Bartlett J (2014). Implementation of a workforce initiative to build trauma-informed child welfare practice and services: Findings from the Massachusetts Child Trauma Project. Children and Youth Services Review, 44, 233–242. [Google Scholar]
  28. Goldstein F, & Glueck D (2016). Developing rapport and therapeutic alliance during telemental health sessions with children and adolescents. Journal of Child and Adolescent Psychopharmacology, 26, 204–211. [DOI] [PubMed] [Google Scholar]
  29. Graham JW (2009). Missing data analysis: Making it work in the real world. Annual Review of Psychology, 60, 549–576. [DOI] [PubMed] [Google Scholar]
  30. Green AG, & Aarons GA (2011). A comparison of policy and direct practice stakeholder perceptions of factors affecting evidence-based practice implementation using concept mapping. Implementation Science, 6, 104. doi: 10.1186/1748-5908-6-104. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Greenhalgh T, Robert G, Mcfarlane F, Bate P, & Kyriakidou O (2004). Diffusion of innovations in service organizations: Systematic review and recommendations. Milbank Quarterly, 82(4), 581–629. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Hansen LO, Williams MV, & Singer SJ (2011). Perceptions of hospital safety climate and incidence of readmission. Health Services Research, 46(2), 596–616. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Hanson RF, Lang JM, Goldman Fraser J, Agosti JR, Ake GS, Donisch KM, & Gewirtz AH (2018). Trauma-informed care: Definitions and statewide initiatives. In Kilka JB & Conte JR (Eds.), The APSAC handbook on child maltreatment (4th ed., pp. 272–291). Thousand Oaks, CA: Sage Publications. [Google Scholar]
  34. Hanson RF, & Lang J (2016). A critical look at trauma-informed care among agencies and systems serving maltreated youth and their families. Child Maltreatment, 21(2), 95–100. [DOI] [PubMed] [Google Scholar]
  35. Hanson RF, Schoenwald S, Saunders BE, Chapman J, Palinkas LA, Moreland AD, Dopp A (2016). Testing the
Community-Based Learning Collaborative (CBLC) implementation model: A
study protocol. International Journal of Mental Health Systems, 10(52), 1–13. doi; 10.1186/s13033-016-008404. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Hasson H, Gilbert-Ouimet M, Baril-Gingras G, Brisson C, Vezina M, Bourbonnais R, & Montreuil S (2012). Implementation of an organizational-level intervention on the psychosocial environment of work comparison of managers’ and employees’ views. Journal of Occupational and Environmental Medicine, 54(1), 85–91. [DOI] [PubMed] [Google Scholar]
  37. Hayes AF (2012). PROCESS: A versatile computational tool for observed variable mediation, moderation, and conditional process modeling [White paper] Retrieved from http://www.afhayes.com/public/process2012.pdf
  38. Hayes AF (2017a). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach (2nd ed.). New York, NY: Guilford Press. [Google Scholar]
  39. Hayes AF (2017b). PROCESS macro for SPSS (Version 3.0) [Computer software] Available from http://www.processmacro.org/download.html
  40. Hox JJ (2010). Multilevel analysis: Techniques and applications (2nd ed.). New York, NY: Routledge. [Google Scholar]
  41. Hurlburt MS, Leslie LK, Landsverk J, Barth RP, Burns BJ, Gibbons RD, … Zhang JJ (2004). Contextual predictors of mental health service use among children open to child welfare. Archives of General Psychiatry, 61(12), 1217–1224. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Institute for Healthcare Improvement (2017). Retrieved September 13, 2017 from http://www.ihi.org/resources/Pages/Tools/PlanDoStudyActWorksheet.aspx
  43. Kim J (2016). Emergence: Core ideas and issues. Synthese, 151, 547–559. [Google Scholar]
  44. Lin TH (2010). A comparison of multiple imputation with EM algorithm and MCMC method for quality of life missing data. Quality & Quantity: International Journal of Methodology, 44, 277–287. [Google Scholar]
  45. Meadows Mental Health Policy Institute (2016). Best practices in integrated behavioral health: Identifying and implementing core components Retrieved October 30th, 2017 from http://texasstateofmind.org/wp-content/uploads/2016/09/Meadows_IBHreport_FINAL_9.8.16.pdf
  46. Morrissey JP, Johnsen MC, & Calloway MO (1997). Evaluating performance and change in mental health systems serving children and youth: An interorganizational network approach. Journal of Mental Health Administration, 24(1), 4–22. [DOI] [PubMed] [Google Scholar]
  47. Palinkas LA, Fuentes D, Finno M, Garcia AR, Holloway IW, & Chamberlain P (2014). Inter-organizational collaboration in the implementation of evidence-based practices among public agencies serving abused and neglected youth. Administration and Policy in Mental Health and Mental Health Services Research, 41, 74–85. [DOI] [PubMed] [Google Scholar]
  48. Preacher KJ, & Hayes AF (2004). SPSS and SAS procedures for estimating indirect effects in simple mediation models. Behavior Research Methods, Instruments, and Computers, 36(4), 717–31. [DOI] [PubMed] [Google Scholar]
  49. Proctor EK, Landsverk J, Aarons GA, Chambers DA, Glisson C, & Mittman BS (2009). Implementation research in mental health services: An emerging science with conceptual, methodological, and training challenges. Administration & Policy in Mental Health, 36, 24–34. doi: 10.1007/s10488-008-0197-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Proctor EK, Silmre H, Raghavan R, Howmand P, Aarons G, Bunger A, Griffeym R, & Hensley M (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration & Policy in Mental Health, 38, 65–76. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Ruggiero KJ, Saunders BE, Davidson TM, Cook DL, & Hanson RF (in press). Leveraging technology to address the quality chasm in children’s evidence-based psychotherapy. Psychiatric Services [DOI] [PubMed]
  52. Rucker DD, Preacher KJ, Tormala ZL, & Petty RE (2011). Mediation analysis in social psychology: Current practices and new recommendations. Social and Personality Psychology Compass, 5/6, 359–371. [Google Scholar]
  53. Saunders BE, Berliner L, & Hanson RF (Eds.). (2004). Child physical and sexual abuse: Guidelines for treatment (Revised Report: April 26, 2004) Charleston, SC: National Crime Victims Research and Treatment Center. [Google Scholar]
  54. Saunders BE, & Hanson RF (2014). Innovative methods for implementing evidence supported interventions for mental health treatment of child and adolescent victims of violence. In Reece RM, Hanson RF, and Sargent J (Eds). Treatment of child abuse: Common ground for mental health, medical, and legal practitioners (2nd ed., pp 235–245). Baltimore, MD: The Johns Hopkins University Press. [Google Scholar]
  55. Sawyer RK (2002). Emergence in psychology: Lessons from the history of non-reductionist science. Human Development, 45, 2–28. [Google Scholar]
  56. Stewart RW, Orengo-Aguayo RE, Cohen JA, Mannarino AP & de Arellano MA (2017). A pilot study of Trauma-Focused Cognitive-Behavioral Therapy delivered via telehealth technology. Child Maltreatment doi: 10.1177/1077559517725403. [DOI] [PubMed] [Google Scholar]
  57. Stiffman AR, Pescosolido B, & Cabassa LJ (2004). Building a model to understand youth service access: The Gateway Provider Model. Mental Health Services Research, 6(4), 189–198. [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Supper O, Catala M, Lustman C, Chemla Y, Bourgueil L, & Letrilliart L (2015). Interprofessional collaboration in primary health care: A review of facilitators and barriers perceived by involved actors. Journal of Public Health, 37(4), 716–727. [DOI] [PubMed] [Google Scholar]
  59. Twala B (2009). An empirical comparison of techniques for handling incomplete data using decision trees. Applied Artificial Intelligence, 23, 373–405. [Google Scholar]
  60. United States Public Health Service. (2000). Report of the surgeon general’s conference on children’s mental health: A national action agenda Washington, DC: Department of Health and Human Services. [PubMed] [Google Scholar]
  61. Zhao X, Lynch JG, & Chen Q (2010). Reconsidering Baron and Kenny: Myths and truths about mediation analysis. Journal of Consumer Research, 37, 197–206. [Google Scholar]
  62. Zwarenstein M, Goldman J, & Reeves S (2009). Interprofessional collaboration: Effects of practice-based interventions on professional practice and healthcare outcomes., The Cochrane Database of Systematic Reviews, 8(3), CD000072. [DOI] [PubMed] [Google Scholar]
  63. Zyphur MJ, Zammuto RF, & Zhang Z (2016). Multilevel latent polynomial regression for modeling (in)congruence across organizational groups: The case of organizational culture research. Organizational Research Methods, 19(1), 53–79. [Google Scholar]

RESOURCES