Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 May 1.
Published in final edited form as: Adm Policy Ment Health. 2015 May;42(3):309–322. doi: 10.1007/s10488-014-0571-3

Effectiveness of wraparound vs. case management for children and adolescents: Results of a randomized study

Eric J Bruns 1, Michael D Pullmann 2, April Sather 3, Ramona Denby Brinson 4, Michelle Ramey 5
PMCID: PMC4278946  NIHMSID: NIHMS609655  PMID: 24973891

Abstract

In this study, we compared service experiences and outcomes for youths with serious emotional disorder (SED) randomly assigned to care coordination via a defined wraparound process (n = 47) versus more traditional intensive case management (ICM; n = 46) The wraparound group received more mean hours of care management and services; however, there ultimately were no group differences in restrictiveness of residential placement, emotional and behavioral symptoms, or functioning. Wraparound implementation fidelity was found to be poor. Organizational culture and climate, and worker morale, were poorer for the wraparound providers than the ICM group. Results suggest that, for less-impaired youths with SED, less intensive options such as ICM may be equally effective to poor-quality wraparound delivered in the absence of wraparound implementation supports and favorable system conditions.

Keywords: Children and adolescents, serious emotional disorder, wraparound, case management, treatment, effectiveness

Fifteen million children and adolescents in the United States experience a diagnosable mental health disorder, with approximately half (six to eight million) considered to have serious emotional disturbance (SED), meaning that they have one or more diagnosed mental health disorders that cause impaired functioning in home, school, and/or community (Kataoka, Zhang, & Wells, 2002; Kazak et al., 2010). For many years, advocates (Cooper et al., 2008; Knitzer, 1982; Pires, 2002), researchers (Bruns, Walker, et al., 2010; Burns, Hoagwood, & Maultsby, 1998; Farmer & Farmer, 2001; Tolan & Dodge, 2005; Weisz, Sandler, Durlak, & Anton, 2005), and federal reports (New Freedom Commission on Mental Health, 2003; U.S. Public Health Service, 2000) have asserted the need to provide care management to these children and adolescents (hereinafter called youth), particularly those with the most serious and persistent behavioral health needs.

Care management is recommended for youths with SED because youth with SED typically present with complex and multiple mental health diagnoses, academic challenges, and family stressors and risk factors that typically necessitate multiple interventions, which must then be provided in an organized way (Bruns, Walker, et al., 2013; Kazak et al., 2010; Pullmann, VanHooser, Hoffman, & Heflinger, 2010). In addition, recent research suggests that “usual care” therapists spend a large proportion of their time “addressing external care” – that is, providing case management – which “can interfere with delivery of evidence-based psychotherapeutic approaches” (Garland et al., 2010; p.793). Thus, there is an assumption that care management can help other professionals conduct their duties more effectively.

Despite advocacy for the use of care management, there has been uncertainty over exactly what it should entail (Burns, Farmer, Angold, & Costello, 1996; Burns, Hoagwood, & Maultsby, 1998; Rosenblatt, 1996; Ziguras & Stuart, 2000). Two primary methods are case management and the wraparound process. Wraparound is conceived as an individualized, team-based care planning and coordination process that integrates the efforts of the many helpers who are involved; develops a holistic treatment plan that includes supports for parents/caregivers and siblings; and oversees a process of goal setting and progress monitoring (Bruns, Walker, et al., 2010; Walker, Bruns, & Penn, 2008). Case management is less intensive, often employing a brokerage model, where professionals arrange for the provision of services without direct contact with families (Ziguras & Stuart, 2000). More intensive case management models may emphasize development of a trusting and consistent relationship and a single point of contact for finding needed help (Bender, Kapp, & Hahn, 2011) but not other features specific to wraparound, such as convening of a unique team, explicit attention to family strengths, or care plans based on family preferences. Wraparound also typically features lower caseloads (e.g., 6 to 12 per facilitator), active involvement of natural helpers, an emphasis on increasing social support, attention to youth and family strengths, and prioritization of youth and family preferences.

In recent years, concerns about the “black box” of wraparound have led to efforts to better specify the wraparound practice model (Burns & Goldman, 1999; Walker & Bruns, 2006), provide more consistent training and implementation supports (Walker & Matarese, 2011), and develop and deploy fidelity measures (Bruns, Burchard, Suter, Leverentz-Brady, & Force, 2004; Bruns, Leverentz-Brady, & Suter, 2008; Bruns, Suter, Force, & Burchard, 2005; Walker & Sanders, 2011). The evidence base for wraparound has also grown, with 10 controlled studies now published in peer reviewed journals (Bickman, Smith, Lambert, & Andrade, 2003; Bruns, Rast, Peterson, Walker, & Bosworth, 2006; Carney & Buttell, 2003; Clark, Lee, Prange, & McDonald, 1996; Evans, Armstrong, Kuppinger, Huz, & McNulty, 1998; Grimes et al., 2011; Hyde, Burchard, & Woodworth, 1996; Mears, Yaffe, & Harris, 2009; Pullmann et al., 2006; Rauso, Ly, Lee, & Jarosz, 2009). In 2009, a meta-analysis of seven of these studies (published at the time of the review) found significant effects of wraparound across all five domains examined, including maintenance of youth in community residential placements (Cohen’s d = .44), mental health outcomes (0.31), overall youth functioning (0.25), school functioning (0.27), and juvenile justice outcomes (0.21) (Suter & Bruns, 2009).

Although the accumulation of controlled studies showing positive results has promoted use of wraparound, only four of the above studies employed random assignment, while the rest employed quasi-experimental individual or group matched designs. Two of the randomized studies (Clark et al., 1998; Evans at al., 1998) were early studies that examined programs that lacked characteristics of wraparound as defined today, and employed no fidelity measures. Finally, a more recent study that compared wraparound (n = 213) to Multisystemic Therapy (MST; Henggeler, Pickrel, & Brondino, 1998; n = 54) found greater clinical (but not functional) improvement over 18 months than youths who received wraparound in the same system (Stambaugh et al., 2007). This study, however, featured many of the same methodological shortcomings as the studies reviewed above, including a non-experimental design and lack of statistical correction for group selection bias.

Thus, given the limitations of extant research, and the multiple options for how a local system might provide care management to youths with SED and other complex needs, a rigorous test of wraparound against a relevant alternative option – such as case management – would provide useful information regarding investment of limited resources. With several states and large jurisdictions now overseeing wraparound initiatives that serve many hundreds of youths (Bruns, Walker, et al., 2010), the field would also benefit from such a study that also examines fidelity and other implementation processes of a wraparound effort gone to scale in a “real-world” system.

The context for the current study was a statewide, cross-agency wraparound initiative for youth with SED, over half of whom were in custody of the child welfare system. Results of two quasi-experimental research studies of a small-scale pilot of this program (Bruns et al., 2006; Mears et al., 2009) previously found robust implementation fidelity and significant positive effects compared to services as usual. The current study aimed to more rigorously examine fidelity and outcomes after expansion to serving several hundred youth. The current study also included theory-based process measures not included in previous effectiveness studies of wraparound. The study had two research aims: (1) To evaluate service processes and clinical and functioning outcomes of wraparound versus a relevant alternative (intensive case management) using an experimental research design; and (2) To examine fidelity and implementation of wraparound in a “real world” system that intended to take wraparound to large-scale implementation. Our intent was to add a rigorous test of wraparound to the research base, while also exploring treatment, organizational, and system factors that may influence implementation and outcomes of wraparound and case management for this population in community settings.

Method

Design

The study employed a randomized controlled design. The study compared service processes and outcomes for the wraparound process vs. intensive case management (ICM) among children and youth 6–17 years old with SED. The study took place in a large county in a western state between 2007 and 2009. Randomization was conducted at the individual level: Youths with SED and referred for intensive mental health services to the Division of Child and Family Services (DCFS), a state entity responsible for providing children’s mental health services. Youths were randomly assigned at referral to wraparound (delivered by staff of DCFS) or to ICM (delivered by a private mental health provider organization). Overall, 126 youths were referred to the study; 93 were found to be eligible and consented to participate. Youth outcomes (residential, symptom, and functioning) were collected at baseline, 6 months, and 12 months. To conserve study resources and reduce respondent burden while maintaining temporal alignment with outcomes data, services, fidelity, and organizational context data were collected only once – at 6 months from provider staff delivering wraparound or care management. Results presented here reflect an intent-to-treat approach, with all 93 participating youths included in analyses.

Participants

Youths

Youths were Medicaid-eligible (the ICM provider only served Medicaid clients), between 6 and 17 years of age at intake, and “diagnosed within the preceding 12 months as having a mental, behavioral or emotional disorder as defined in the Diagnostic and Statistical Manual of Mental Disorders (American Psychological Association, 2000) and functional impairment that substantially interferes with or limits the child from developing social, behavioral, cognitive, communicative or adaptive skills or his activities relating to family, school or community.” Youths also had to have a total score on the Child and Adolescent Functional Assessment Scale (CAFAS) (Hodges, 1997) of 50 or higher, indicating “impairment in functioning requiring clinical care.”

Of the 126 youths who were referred to the study by DCFS intake coordinators, eight did not meet criteria for SED and four were not Medicaid-eligible. Consent was declined for an additional 14 youths (nine by parents/caregivers, two by youth, and three by case workers). Three youth had moved or were on the run, three youth were already receiving Wraparound or ICM, and one youth had information missing, leaving a total sample size of 93. Forty-seven youths were assigned to the wraparound condition and 46 to ICM. Little information was available for the 14 eligible youths for whom consent was declined; however, it is noteworthy that 87.9% of those for whom consent was declined were in custody (n = 29), whereas only 66.7% of study participants (n = 62) were in custody.

The study group was majority male (57%), with a mean age of 11.9 (SD = 3.38). Sixty-one percent were from a racial or ethnic minority group (41% African American, 12% mixed race, 1% Native American, and 8% of some other race; 16% were of Hispanic ethnicity). Two-thirds (67%) were in custody of the county child welfare service system at the time of study enrollment. The most common Axis I diagnoses across youths were attention disorders (25%), adjustment disorders (23%), and mood disorders (23%). Other diagnoses included anxiety disorders (including post-traumatic stress disorder; 15%), oppositional and conduct disorders (13%), learning and developmental disorders (12%), and substance use disorders (7%). Eight percent of youths had Axis I diagnoses that did not fall in these major categories (e.g., thought disorders and eating disorders). Twenty-seven percent of youths had more than one Axis I disorder.

Caregivers

Of the 93 study youth, 62 were in the custody of DFS, while 32 were in parental custody. There were no significant differences between groups, with 32 youth in the wraparound group and 30 in the ICM group in DFS custody. Across both groups, 34% of youths were cared for by biological parents, 31% by foster parents, 30% by another relative, 2% by adoptive parents, and 1% by a family friend. Groups differed significantly on residential placement: 38% of youths in the wraparound groups lived with foster parents vs. 24% for ICM; 19% of youths in the wraparound group were cared for by relatives vs. 35% for ICM. A majority of caregivers had attained at least a high school diploma (86%).

Service providers

Youth in the wraparound group were served by n = 23 wraparound facilitators employed by DCFS. ICM youth were served by n = 17 case managers employed by the private mental health agency. Wraparound facilitators served a mean of 1.6 youths in this study, and clinicians served a mean of 2.4. Wraparound facilitators were located in five separate DCFS service sites across the county; ICM staff were located in two offices.

Of the 23 Wraparound facilitators enrolled in the study, demographic information was collected on 18. Overall, Wraparound facilitators had an average of seven years of experience in human services, including an average of 2.78 years at their current agency. Eleven (64.7%) were female and the mean age was 33.7 years of age. Five (27.8%) had a Master’s degree; 13 (72.2%) were Bachelor’s level. Fifty-six percent of wraparound staff were Caucasian, 22% were African American, 6% were Asian, 6% were Native Hawaiian or other Pacific Islander, and 18% identified as “Other.” Eighteen percent reported Hispanic ethnicity. Demographic data was collected for 14 of the 17 ICM case managers. Overall, case managers had an average of 13.5 years of experience in human services, and 6.8 years in their current agency. Ten were female (71.4%) and the mean age was 39. Nearly all (93%) of the case managers had a master’s degree, the majority in Social Work. Seventy one percent of ICM staff were Caucasian, 14% were African American, 7% were Asian, and 7% identified as “Other.” Seven percent reported a Hispanic ethnicity. ICM case managers were significantly more likely to have Masters degrees (p < .001) and had significantly more experience overall and with their agency (p < .01)

Intervention Conditions

Wraparound

Wraparound staff received a three-day training that focused on core skills necessary to facilitate a wraparound process that takes place over four phases. During the engagement phase (several meetings over two weeks), wraparound facilitators use active listening skills to identify family strengths and needs, conduct an initial functional assessment, and develop a crisis and safety plan. During the planning phase (1–2 meetings over two weeks), facilitators identify relevant team members, use a team process to elicit and prioritize family needs, develop creative strategies to meet needs, and relevant indicators of progress. During the implementation phase, facilitators met regularly (at least once per month) to check in on completion of strategies and services, review and celebrate accomplishments, track progress per identified indicators, make collateral contacts on behalf of the youth and family, and ensure connection to and engagement of natural supports. During the transition phase, facilitators were intended to develop an effective transition plan, rehearse responses to potential future crises, and identify future sources of natural support for the family.

Ultimately, 80.8% of youths randomized to wraparound received the service. Among those who did, the mean length of service episode for wraparound was 9.68 months (SD = 3.51) and the mean number of reported hours of direct and collateral service provided by facilitators per month over this time was 12.76 (SD = 8.94; Range 1.16 – 44).

Case management

ICM was provided by Master’s level clinicians in a mental health organization that served the same catchment area as the DCFS-employed wraparound facilitators. Consistent with several studies and descriptions of case management (Austin, 1993; Burns et al., 1996), in the current study, ICM functioned as a mechanism for coordinating segments of the service delivery system to develop a more comprehensive plan of care for the enrolled youth. As for wraparound facilitators, ICM case managers met with caregivers and family members, connected youth and family to community services as well as those available within their own provider organization, and in some cases worked with the youth and family’s support system. ICM fundamentally differed from wraparound in that case managers had higher caseloads (25 vs. 12–15 for wraparound) and also provided direct clinical services to ICM-enrolled youths and families as well as others. ICM did not emphasize core wraparound elements such as family determination; developing plans based on strengths and needs; convening a team individualized to the youth and family; and plan development, tracking, and adapting by the team. Per program guidelines, case managers met with families in person at a minimum of every 90 days and had phone contact with families and all service providers at a minimum of every 30 days. ICM fidelity monitoring was not formal; instead, quality was monitored via individual supervision meetings (minimum every 30 days), email and phone contact with supervisor as needed, and review of progress notes.

Ultimately, 78.3% of youths randomized to ICM received the service. Among those who did, the mean length of service episode for ICM was 7.64 months (SD = 4.21); mean hours of direct and collateral service provided by case managers per month was 4.89 (SD = 2.16; Range 1.5– 11).

Service array

Youth in both the wraparound and ICM conditions had access to a similar array of formal and community supports, including individual and family therapy, treatment foster care, residential treatment, and inpatient hospitalization. A family advocacy organization in the county also employed family support workers available to families of youth enrolled in both groups.

Data Collection

Outcomes data were collected by two Research Assistants who were trained to criteria on measures and directly supervised by the first and fourth authors via weekly and as-needed meetings. Due to the nature of the study protocol and interview topics, research assistants were not blinded to study condition. Data were collected via in-person interviews with parents or caregivers at baseline, 6 months, and 12 months, as well as via administrative data review to fill gaps in residential placement histories. Fidelity, service, and organizational climate and culture data were collected via (1) interviews and surveying of parents/caregivers, youths, facilitators and case managers at six months (both groups); (2) observation of a random sample of wraparound team meetings four to six months after intake (wraparound group only); and (3) web-based survey of key stakeholders assessing adequacy of system supports for wraparound. Diagnoses were established by qualified mental health providers contracted by DCFS; this information was obtained via record review.

Outcomes measures

Outcomes measured included symptoms, functioning, and residential placement. The Strengths and Difficulties Questionnaire (SDQ; Goodman & Scott, 1999) measures children’s emotional symptoms, conduct problems, peer problems, hyperactivity, and prosocial behavior. Each construct is measured with five items on 3-point scales ranging from 0 (not true) to 2 (certainly true). Analyses used the total difficulties score, which gives a total score for each construct and has been found to have good internal consistency (α = .62) and to correlate with other measures of child psychopathology (Bourdon, Goodman, Rae, Simpson, & Koretz, 2005). We also used a single item from the SDQ assessing the burden on the family posed by the youth’s problems.

The CAFAS (Hodges, 1997) uses a structured interview with the child’s caretaker to measure youth functioning in eight areas (home, school/work, community, behavior towards others, moods, thinking, self-harm, substance abuse). Using a set of criteria, the interviewer assigns scores for the eight subscales, from 0 (no impairment) to 30 (severe impairment). Subscale scores are summed to produce a total CAFAS scale score ranging from 0–240. Interrater reliability has been found to range from .92 to .96 for the total, and it has shown predictive validity of youth incarceration and hospitalization (Hodges & Kim, 2000). Raters completed self-training reliability exercises and used the full Parent Interview (Hodges, Xue, & Wotring, 2004) to improve reliability. Analyses presented here used the total CAFAS score.

Residential placement and residential restrictiveness over the last six months were assessed via interview with the parent or caregiver, and missing information was supplemented by administrative placement data from DCFS. Administrative data review was necessary for nine of ninety-three study youths (9.6%) comprising 1,593 (6.1%) of 25,961 total residential placement datapoints. Restrictiveness was categorized (Rautkis et al., 2009) into: 1 = Low (few limitations, e.g., independent living or with birth/adoptive parents), 2 = moderate (more structure and some limitations on personal choices, e.g., foster care), 3 = elevated (greater limitations and restrictions in access to friends and communication, e.g., community treatment homes and treatment foster care), and 4 = high restrictiveness (e.g., inpatient hospitalization, residential treatment, and juvenile detention). Analyses present mean restrictiveness scores.

Process measures

Fidelity, service, and process measures were administered for both study groups. To assess fidelity, caregivers and providers (facilitators and case managers) in both groups were interviewed using the Wraparound Fidelity Index, version 4 (WFI-4) (Bruns et al., 2004, 2005, 2008; Pullmann, Bruns, & Sather, 2013). The WFI is a measure of adherence to the principles of the wraparound process. It is administered via 40-item interviews organized by the four phases of wraparound. Each item is related to one of the 10 principles of wraparound; e.g., “Does the team evaluate progress toward the goals of the plan at every team meeting?” (Outcome-based). Total scale alpha coefficients of .88 and .92 and test-retest reliabilities of .84 and .88 have been found for the caregiver and facilitator forms, respectively (Bruns et al., 2004, 2008; Pullmann, Bruns, & Sather, 2013). Support for validity is found in correlations with alternative fidelity measures (see below); and ability to discriminate between wraparound and other interventions, (Bruns et al., 2006; 2008; Pullmann et al., 2013). Analyses present total fidelity scores by respondent.

The Service Assessment for Children and Adolescents – Parent Form (SACA) is a structured interview designed to assess the use of services by youths across eight inpatient, thirteen outpatient, and four school settings. Overall test-retest kappa for any service use has been found to be .91, with kappas ranging from .41 to .99, depending upon the service, and convergent validity has been demonstrated with medical records (kappa =.76; Horwitz et al., 2001). Analyses categorized services into outpatient, community-based, and home-based (measured in hours received in the past three months) and inpatient (measured in days). Providers (facilitators and case managers) also were asked to use records to complete the Case Management Function Form (CMFF) (Burns et al., 1996) retrospectively at six months. This measure assesses the number of hours and percent of total case management time spent for an individual youth in each of seven activity types: Outreach, assessment, planning and monitoring, linking/advocating, crisis intervention, clinical treatment, and documentation. The CMFF correlates with electronic billing records and discriminates different types of case management programs (Burns et al., 1996). Analyses present group differences for total care management service received.

Caregivers and youths responded to the four “Global satisfaction” questions from the Parent Satisfaction Questionnaire (PSQ) and Youth Satisfaction Questionnaire (YSQ) (Brannan, Sonnichsen, & Heflinger, 1996). These four questions use a five-point Likert scale to assess (1) overall satisfaction, (2) satisfaction with progress made, (3) satisfaction with level of involvement, and (4) overall helpfulness. These items have previously been found to have good internal consistency (α = .94 and α = .88 for parent and youth, respectively) and validity established through correlation with factors such as therapist-parent agreement on appropriateness of termination of therapy (Brannan et al., 1996). Analyses present each item and a total score.

Parents and providers (wraparound facilitators and case managers) were administered separate versions of the Working Alliance Inventory (WAI), Short Form (Hatcher & Gillaspy, 2006; Horvath & Greenberg, 1989). Both versions include 12 seven-point Likert Scale items covering three domains: bond, goals, and tasks. Total score alphas have been found to be .91 and .92 respectively, and to correlate with the original WAI as well as several other alliance scales (Hatcher & Gillaspy, 2006). Analyses here present total alliance scores. Caregivers were administered the Family Empowerment Scale (FES) (Koren, DeChillo, & Friesen, 1992). The FES assesses a parent or caregiver’s empowerment via thirty-four 5-point Likert scale items across three domains: family, service system, and political/community. Alphas range from .87 to .88 for these domains; one-week test-retest correlations range from .77 to .85. The FES does not produce a total score; therefore, analyses present results from each domain separately.

Additional fidelity measures administered to the Wraparound group

The Team Observation Measure (TOM) (Bruns & Sather, 2007) includes 20 items, each scored via three to five indicators of adherence to wraparound practice model during a team meeting. For example, the item “Effective Team Process” includes indicators such as “Tasks and strategies are explicitly linked to goals,” and “Potential barriers to the nominated strategy or option are discussed and problem-solved.” Reliability is good (total scale α = .82; inter-rater reliability = .84 (Bruns, Sather, & Pullmann, 2010). Support for validity is found in significant correlations between mean TOM and WFI-4 scores (r = .486; p<.01; Bruns, Sather, et al, 2010); and correlations with external criteria (Snyder, Lawrence, & Dodge, 2012).

The Community Supports for Wraparound Inventory (CSWI) (Walker & Sanders, 2011) is a 42-item web-based key informant survey that evaluates the presence or absence of system support for wraparound in six areas (community partnership, collaborative action, fiscal supports, service array, human resource support, and accountability). For each item (e.g., “There is centralized monitoring of relevant outcomes for children, youth, and families in wraparound”), respondents rate system development on a 0 (“system is not at all developed”) to 4 (“fully developed”) scale, with item-specific anchor descriptions provided for each end of the scale. The CSWI produces scores in each domain and a total score. Internal consistency ranges from α = .94 to α = .96 across the six domains and inter-rater reliability ICC = .78 (Walker & Sanders, 2011). Evidence for CSWI validity is found in an exploratory factor analysis that corresponded to the six domains and significant correlation with other established system assessments (Walker & Sanders, 2011).

Organizational context

Because organizational factors have been found to predict uptake of innovation as well as directly influence client outcomes organizational culture and staff perceptions of organizational climate of the two primary provider organizations were assessed using the Organizational Social Context (OSC) questionnaire (Glisson & Hemmelgarn, 1998; Glisson & James, 2002; Glisson et al., 2008). The OSC consists of 105 items and measures seven scales in three domains: (1) culture (rigidity, proficiency, and resistance); (2) climate (engagement, functionality, and stress), and (3) worker morale. The OSC was administered to DCFS staff providing wraparound (n = 18) and staff in the private mental health agency (n = 14) who administered ICM. Although wraparound facilitators were located in five separate DCFS service sites across the county, and ICM staff were located in two settings, due to the small number of staff at each site, and for ease of interpretation, analyses here present organizational context results for all wraparound vs. all ICM staff across the seven OSC scales.

Procedures

The study protocol was approved by the institutional review boards of the University of Nevada, Las Vegas; the University of Nevada, Reno; and the University of Washington. Intake coordinators based in the five wraparound/DCFS service sites identified potentially eligible youth who were referred for intensive services due to SED. Consent was sought from the youth’s parent or guardian. Youths in the custody of child protective services were consented by their case worker; consent was also sought from biological parents of these youths via phone and letter to last known address. After consent was obtained, the youth was randomly assigned to either wraparound or ICM. Randomization was managed centrally by the research team using a single, computer-generated randomization stream that was applied to all study-referred youths regardless of which DCFS site sent the referral. Parents/caregivers received $20 for each interview completed (at baseline, six, and 12 months), while youths received $10 for an interview completed at six months. Provider staff were consented by site supervisors and the research coordinator during staff meetings. Staff hired after the start of the study were consented individually by the research coordinator. Staff in both conditions completed a brief interview at six months and completed the OSC, but did not receive honoraria.

Statistical Analyses

Equivalence of groups at baseline was assessed using t-tests and chi-square tests. Differences between the wraparound and ICM groups in fidelity scores were examined using t-tests comparing the population percentile TOM and CSWI scores, and two-level hierarchical linear models (HLM; Raudenbush & Bryk, 2002; Singer & Willet, 2003) for caregiver and clinician WFI scores, with individual team scores nested by clinician/facilitator. Differences in services received were assessed through chi-square and t-tests comparing the percentage of families receiving services and/or hours of services received from baseline to six months (for all youth in the study) by different types of service (e.g., wraparound/ICM; outpatient, home-based, and community-based services; case management activities received by type).

Longitudinal outcomes were tested through three-level longitudinal growth models using HLM with observations (level 1) nested within individuals (level 2) nested within providers (level 3). These modeled the levels of estimated scores and rates of change over time of outcome variables including the CAFAS, SDQ overall problems, SDQ impact on the family, and residential restrictiveness. Variables were fixed or allowed to randomly vary based on standard procedures for parsimonious exploratory model-building (Singer & Willett, 2003), including observation of variance significance at p < .05 and model deviance statistics with chi-square tests. Models for SDQ and CAFAS scores included data from baseline, 6, and 12 months, whereas residential restrictiveness included residence type for every day of the 365 days of the study. Data were modeled using full maximum likelihood estimation and robust standard errors. Standardized effect sizes for the mean difference in the slopes were calculated in order to provide a metric to compare treatment effects among the models (Raudenbush & Xiao-Feng, 2001). Caregiver and youth satisfaction and FES scores were also modeled using three-level HLMs, but treated data from 6 and 12 months as multiple (non-growth) observations.

Analyses were conducted using SPSS 15.0 for Windows for t- and chi-square tests and HLM 6 for HLM model building. We chose not to use alpha corrections for the large number of tests as this would result in extremely small critical alphas; instead, with the understanding that all tests were planned a priori, we looked for consistency among findings and adopted a “weight of evidence approach” in examining the pattern of multiple outcome measures examined.

Results

During the one-year follow-up period, caregiver interviews were completed for 81 of 93 participants (87%) at 6 months and 68 of 93 at 12 months (73%). Among the 25 not completed, 13 were lost to follow-up, seven caregivers declined, and five moved out of state. Study retention was approximately equal for the wraparound and ICM (74% and 72%, respectively). We found no significant differences on demographic (youth/caregiver age, race, and sex; custody status; number of children in the home; caregiver educational level), service (enrollment in wraparound or ICM at baseline), or baseline outcome variables (SDQ, CAFAS) between those retained through 12 months and those lost to follow up.

Group Comparability

As shown in Table 1, we compared the wraparound and ICM groups at baseline on several variables: demographic characteristics; custody status, youth and family risk factors, baseline scores on symptom and functioning scales, and residential restrictiveness. The wraparound group demonstrated a higher mean score on baseline total CAFAS, 121.1 vs. 102.6 [t(92) = 2.14; p = .037] and a higher mean baseline residential restrictiveness rating of 1.63 vs. 1.40 for the ICM group, a difference that approached significance [t(93) = 1.62; p = 0.108]. Closer inspection showed that 12 youths in the wraparound group were in elevated or high levels of restrictiveness at baseline versus only six ICM youths. Both of these differences at baseline were attributed at least in part to the withdrawal of three youths (who were in relatively high levels of restrictiveness and thus may have had more severe functional impairment) from the study by case workers after randomization to ICM.

Table 1.

Baseline Characteristics of Youths by Treatment Group

Total (N = 93) Wrap (n = 47) ICM (n = 46)
Male 53 (57%) 26 (55%) 27 (59%)
Age (SD) 11.9 (3.4) 12.3 (3.1) 11.5 (3.6)
Race
 White 36 (39%) 16 (34%) 20 (44%)
 African American 38 (41%) 19(40%) 19 (41%)
 Native American 1 (1%) 1 (2%) 0
 Mixed Race 11 (12%) 10 (21%) 1 (2%)
 Other/Not specified 7 (8%) 1 (2%) 6 (13%)
Ethnicity
 Hispanic 15 (16%) 8 (17%) 7 (15%)
In custody 62 (67%) 32 (68%) 30 (65%)
 Risk factors
 Prior psychiatric hospitalization 23 (25%) 12 (26%) 11 (24%)
 Prior physical abuse 40 (43%) 21 (45%) 19 (41%)
 Family history of domestic violence 41 (44%) 24 (51%) 17(37%)
 Family history of mental illness 37 (40%) 20 (43%) 17 (37%)
 Biological parent convicted of a crime 54 (58%) 26 (55%) 28 (61%)
Caregiver Relation to Youth
 Biological Parent 32 (34%) 18 (38%) 14 (30%)
 Foster Parent 29 (31%) 18 (38%) 11 (24%)
 Other Relative 25 (30%) 9 (19%) 16 (35%)
 Caseworker/Other staff 3 (3%) 2 (4%) 1 (1%)
 Adoptive Parent 2 (2%) 0 2 (4%)
 Family Friend 1 (1%) 0 1 (1%)
Caregiver Education Level
 K-11th Grade 11 (14%) 3 (7%) 8 (21%)
 High School Diploma or GED 30 (38%) 16 (39%) 14 (36%)
 Associates Degree 6 (8%) 3 (7%) 3 (8%)
 Some College, no degree 22 (28%) 13 (32%) 9 (23%)
 Bachelor’s Degree 6 (8%) 2 (5%) 4 (10%)
 Master’s Degree 5 (6%) 4 (10%) 1 (3%)
Baseline scores on measures
 Mean CAFAS (SD)* 110.43 (42.3) 119.1 (41.9) 101.7 (41.4)
 Mean SDQ (SD) 35.2 (6.3) 35.3 (6.2) 35.1 (6.4)
 Mean residential restrictiveness (SD)* 1.53 (0.69) 1.63 (0.80) 1.40 (0.65)
Open in a new tab

Note: Wrap, wraparound; ICM, Intensive case management.

*

p<.1

Fidelity to the Wraparound Model

Table 2 summarizes fidelity scores by group for the TOM and CSWI, and Table 3 summarizes scores for the WFI. As shown, fidelity as assessed by WFI interviews was significantly higher for the wraparound group than the ICM group per both facilitator and caregiver report, but below average when compared to mean scores from a national database of over 30 WFI user sites (44th and 29th percentile, respectively). Similarly, fidelity as assessed by the TOM was extremely low compared to a national sample of TOM user sites (14th percentile), and community and system supports for wraparound was also found to be far below the national mean (21st percentile).

Table 2.

Summary of Fidelity, Service, and Organizational Context Data by Treatment Group.

Variable Wraparound (n = 51) ICM (n = 43) p
Mean/n %/SD Mean/n %/SD
Wraparound Fidelity
 Team Observation Measure (pop percentile) 1 13.90 4.0
 CSWI (pop percentile) 2 20.60 --
Wrap/ICM services received
 Received any Wrap/ICM services 0–6 mos 41 80% 29 67% .116
 Received any Wrap/ICM 6–12 mos** 35 69% 18 42% .019
 Hours of Wrap/ICM service/mo (0–6 mos)*** 11.74 8.74 4.78 2.23 .001
Other services received (0–6 months)
 Received any inpatient service 7 14% 5 11% .481
 Nights inpatient service received 3 22.29 29.91 45.75 51.10 .348
 Received any outpatient service 26 51% 18 41% .326
 Hours outpatient service received 3 24.91 40.75 33.64 49.2 .532
 Received any home-based service* 15 29% 6 14% .065
 Hours home-based service received 3 ** 36.07 33.41 13.17 13.86 .039
 Received any community-based service 17 33% 10 23% .253
 Hours community-based service received 3 31.94 40.96 17.00 12.91 .125
Organizational Culture 4
 Proficiency (T-score) 49.09 54.57
 Rigidity (T-score) 48.24 39.76
 Resistance (T-score) 57.37 45.75
Organizational Climate
 Engagement (T-score) 43.49 8.40 63.57 6.81 .001
 Functionality (T-score) 65.08 4.50 77.35 3.71 .002
 Stress (T-score) 54.68 2.35 37.84 2.90 .001
Worker morale (mean T-score)*** 56.11 8.72 64.90 3.44 .001
Open in a new tab

Note: Wrap, wraparound; ICM, Intensive case management; WFI, Wraparound Fidelity Index; CSWI, Community Supports for Wraparound Inventory.

1

TOM data were not collected for the ICM group.

2

Community support for wraparound implementation yields a profile at the community or system level, thus, testing of between-group differences is not relevant.

3

Nights and hours of service received only for youths who received the service.

4

Organizational Culture yields profiles at the organizational level, prohibiting direct testing of between-group differences.

***

p<.001

**

p<.05

*

p<.1

Table 3.

Summary of Process Outcomes by Group

Variable Wrap ICM p
Wraparound fidelity
 Caregiver WFI (pop percentile)** 29.85 17.10 .040
 Facilitator WFI (pop percentile)** 44.18 21.6 <.001
Caregiver Satisfaction1
 Overall satisfaction 3.95 4.17 .442
 Satisfaction with involvement 4.01 3.94 .776
 Satisfaction with progress 3.69 3.91 .293
 Helpfulness 3.75 3.53 .337
 Mean total satisfaction score 3.86 3.88 .923
Youth Satisfaction1
 Overall satisfaction 4.09 4.19 .626
 Satisfaction with involvement 3.63 3.88 .346
 Satisfaction with progress 3.99 3.96 .926
 Helpfulness 3.64 3.70 .854
 Mean total satisfaction score 3.83 3.93 .610
Family Empowerment
 Family Empowerment – Family1 4.21 4.26 .695
 Family Empowerment – Services1 4.32 4.27 .777
 Family Empowerment – Community1 3.19 3.01 .337
Caregiver reported working alliance 5.67 5.92 .365
Facilitator/ICM reported working alliance 5.21 5.12 .796
Open in a new tab

Note: Wrap, wraparound; ICM, Intensive case management

1

Scores estimated using 6 and 12 month ratings using HLM

**

p<.05

CSWI data indicated poor community and system support for wraparound across all six domains assessed. For example, scores were far below the national mean in terms of child-serving systems collaborating effectively; stakeholders and staff across systems being educated on their role in the wraparound process; the extensiveness of the service array available to wraparound teams; availability of flexible funding for teams to promote individualized care planning; and adequacy of training, coaching, and supervision for staff in key roles.

Services Received

Table 2 presents profiles of services received by the two groups. As shown, a higher percentage of youth in the wraparound condition received intensive care management during the first six months (80% vs. 69%), though this was not statistically significant, and during the second six-month follow-up period (69% vs. 42%; chi square = 6.80; p = 0.011) than youth in ICM. Youth in the wraparound group also received significantly more care management activity overall during the first six months, as assessed by the CMFF (11.7 vs. 4.8 hr per month; t = 3.84; p<.001).

Youth randomized to wraparound were more likely to receive home-based services with borderline significance (29% vs. 14%; chi square = 3.414; p =.065). Among youths who received them, youths in the wraparound group received more in-home services (36 vs. 13 hrs; t = 2.22; p = .039). Youths in wraparound also received approximately twice as much community-based services on average (32 vs. 17 hrs); however, this difference was not significant.

Organizational Context

Intra-group agreement (rwg) for the three OSC Culture subscales was found to be >.70, meaning that aggregation to organizational-levels of measurement is appropriate (Glisson & James, 2002). As shown in Table 2, wraparound facilitators reported working in units with less advantageous organizational culture across all three subscales, with the ICM group greater than 1 SD higher for the Resistance subscale. Worker perceptions of organizational climate was significantly poorer for the wraparound group for all three subscales (Engagement, Functionality, and Stress; p < .001 for all subscales). Worker morale was found to be significantly lower for WSM facilitators than ICM case managers (56.1 vs. 64.9, p < .001).

Process Outcomes

Table 3 presents results from service process variables. As shown, there were no differences across satisfaction, working alliance, and family empowerment variables.

Youth Outcomes

Figure 1 presents estimated average scores by group across the three study time points for the SDQ, CAFAS, and across 365 time point days for residential restrictiveness. Table 4 displays standardized effect sizes of the differences in the mean slopes for the two groups. Analysis via HLM revealed no significant between-group differences in outcome trajectories for any of these variables. With borderline significance (t ratio = 1.79, p = .078), youth in the ICM group were found to improve at a greater rate on SDQ total problems.

Figure 1.

Figure 1

Open in a new tab

Estimated scores over the 12-month study period for youths in the wraparound (Wrap) and intensive case management (ICM) groups for four outcomes variables.

Table 4.

Hierarchical Linear Modeling Estimates of Standardized Mean Group Differences in Time Slopes

Outcome Effect size p
CAFAS total .274 .710
SDQ total problems* .709 .078
SDQ impact on family .166 .733
Residential restrictiveness linear change1 .221 .382
Open in a new tab
1

Adjusted for custody status at baseline

**

p < .05 based on t-ratio of time slope differences

*

p < .1

Discussion

The current study aimed to expand the research base on the wraparound process by employing randomization; extensive fidelity, process, organizational climate, and outcome measurement; and by comparing wraparound to a relevant alternative treatment approach (ICM) rather than treatment as usual. The study also evaluated fidelity and outcomes in a system that took wraparound implementation to scale after finding positive results from an initial pilot. Study participants were diverse with respect to sex, age, and race, and reflective of youths involved in state and county public mental health and child welfare service systems.

The study found less positive outcomes of wraparound than two quasi-experimental studies that evaluated effects of a smaller-scale project in the same system with only 3–4 WSM facilitators and more extensive fidelity controls (Bruns et al., 2006; Mears et al., 2009). Wraparound-assigned youths showed no better functioning than the ICM group and (with borderline significance) poorer behavioral and emotional outcomes over time. Moreover, there was neither impact on residential placement outcomes – where the largest effect sizes for wraparound have previously been found (Suter & Bruns, 2009) – nor outcomes that are in line with the wraparound theory of change, such as alliance, empowerment, and caregiver satisfaction.

There are a number of possible explanations for these findings. First, fidelity of wraparound implementation was found to be poor by all measures employed. While WFI scores were far above national means during the pilot project (Bruns et al., 2006), fidelity during the current study was at the 29th percentile for parent reports and at the 13th percentile per a team observation measure. Examination of caregiver responses on the WFI suggested that the study site did not consistently engage youths and family members in the development of the wraparound team, involve family natural supports, develop crisis plans based on functional assessments, link caregivers to social supports, and involve youths in community activities. Observation of team meetings also indicated the team process was implemented inconsistently: Statements of mission, goals, or priority needs were not developed; teams were not brainstorming individualized strategies to meet youth and family needs; team members were not following through on tasks; and effective transition plans were not being developed.

Reductions in implementation quality was likely due at least in part to elimination of a comprehensive statewide training and coaching approach (caused by budget cuts that accompanied the onset of the recession in 2008). As has been found in other studies of wraparound (Bickman et al., 2003; Bruns et al., 2008) as well as other community-based services (e.g., Drake, et al., 2001; McHugo, Drake, Teague, & Xie, 1999; Schoenwald, Henggeler, Brondino, & Rowland, 2000), lack of fidelity to the program model may have influenced outcomes. The magnitude of the effect is difficult to estimate; however, it is worth noting that wraparound youths showed mean 6-month improvements of 30 points on the CAFAS in pilot studies (Bruns et al., 2006; Mears et al., 2009), compared to 13 points in this study.

Second, organizational culture and climate was found to be far better for case managers of the university-affiliated mental health organization than those employees of the state mental health system who facilitated wraparound teams. Successful installation of innovative service delivery approaches is contingent upon how well organizations support their implementation (Glisson & Schoenwald, 2005; Glisson, Schoenwald, Hemmelgarn, et al., 2010) and OSC scores have been found in previous research to be associated with better outcomes in human services contexts (Glisson & Hemmelgarn, 1998; Glisson & James, 2002; Glisson, Schoenwald, Kelleher, et al., 2008). Though our research design does not allow us to distinguish possible treatment effects from the effects of organizational context, differences were striking across all OSC scales.

System conditions for wraparound (e.g., role clarity, caseload sizes, availability of flex funds, availability of training and coaching) also were not rated favorably. Among the key “necessary conditions” for successful wraparound implementation, availability of services and supports is viewed as particularly critical (Bruns, Walker, et al., 2010; Walker et al., 2008; Walker & Sanders, 2011). The ICM group was served by case managers, some of whom could also function as therapists, who functioned within a private mental health provider organization. Exit interviews and exploration of SACA data suggested the ICM group had more ready access to clinical care, although the degree to which the care was based on evidence for effectiveness is not clear. Regardless, attention to how the wraparound practice model can facilitate connection to appropriate, evidence-based clinical care is an issue of increasing importance (Bruns, Walker, et al., 2010; Bruns et al., 2013).

Other supports that may have been important to families and wraparound teams may have been lacking as well. SACA data indicated that less than one-third received community-based supports such as mentoring or respite, or in-home services such as behavioral aides. Despite the presence of a family support organization that provided direct peer-to-peer support, and the expanding literature on the importance of family support services (Hoagwood, 2005; Penn & Osher, 2008), only two of 51 families in the wraparound condition were found to have received family support services.

Results could also indicate that wraparound care coordination may not always be a good option for youths involved in the child welfare system. Though previous studies have found positive effects of wraparound for youths involved in child welfare (Clark et al., 1996; Rauso et al., 2009), it is more traditionally deployed to prevent out of home placements. In the current study, over half of youths were in foster care. It may be that the intensive wraparound process, with its focus on supporting parents and relative caregivers, is a less ideal fit for foster parents than birth parents.

Finally, the above explanations are viewed through the lens of interpreting poor fidelity, lack of services, and poor organizational context as having weakened the potential for wraparound to achieve positive effects found in previous studies. However, without starting from the a priori hypothesis that investment in the wraparound team process would enhance outcomes for these children and adolescents, an alternative interpretation is that ICM is an equally effective (and more cost-effective) service option for youths with complex needs in a service system such as this one. Moreover, rather than being constrained by organizational stress, wraparound may have created some portion of it. In addition to insufficient coaching and supervision, added complexity in the form of an additional layer of service coordination and confusion around roles of wraparound facilitators versus child welfare case workers may have caused the worker stress found in poor OSC scores.

Limitations

As mentioned above, a limitation of this study is that we cannot separate treatment effects from the effects of organizational and system context, because organization and treatment group are perfectly correlated, and both groups functioned in the same child-serving system. This threatens internal validity, because the organizations differed on characteristics predictive of success, and external validity, because the hope is that wraparound initiatives elsewhere will attend to the system-level limitations found in this study. This is a common but rarely acknowledged problem, especially prevalent in studies involving complex, ecologically-based programs such as wraparound. Because reducing such organizational and system variation in future studies would be both unfeasible and also ignore its importance, future studies should attempt to isolate contextual and intervention effects by studying multiple sites which have heterogeneity in organizational and community context.

Other common threats to internal validity may also have affected this study. Notably, the ICM group may actually have been exposed to care management that included some features of wraparound, either via contagion effects of being in the same service system, or more overtly as ICM case managers, aware that an experiment was being done, adapted their work to imitate wraparound, provided more intensive services, or in some way “competed” with the treatment group. There is some evidence this may have occurred based on anecdotal evidence. The fact that three child welfare case workers withdrew their consent for a child to participate after discovering their child had been assigned to ICM, potentially biasing comparability at baseline, also illustrates threats to internal validity that occur when conducting an experiment in a complex, “real-world” service system.

Implications for Research, Policy, and Practice

Several studies have shown that wraparound care coordination can produce better outcomes than services as usual. Results of this study, however, indicate that wraparound also can be a more costly approach than an intensive community-based alternative that yields no better outcomes. Results suggest that local and state policymakers and providers must attend carefully to concerns around wraparound-specific issues such as training, supervision, and fidelity controls, as well as system issues that may be independent of the care management approach used, such as collaboration and innovation among systems, organizational context, and availability of high-quality clinical treatments. When the wraparound process is considered as a potential option, especially within a system with existing mandates such as child welfare, policy makers and practitioners should carefully consider the level of additional complexity being introduced. Local systems should also consider how best to promote features of organizations and the system that are necessary to implement a model such as wraparound successfully, and that might also be directly related to positive child and family outcomes (Glisson et al., 2010). Research on different ways to implement wraparound that can best promote its fit in different service settings would support such local decision-making.

Future research in diverse service settings is also needed that tests support technologies that promote fidelity to the practice model. The wraparound research base would also benefit from research that evaluates the fit between different types of children and youth (e.g., children in the care of foster parents versus birth parents), and research that assessing the relative contributions of fidelity, organizational and system conditions. Finally, as discussed above, developing and evaluating methods for more consistently connecting wraparound-enrolled youths to effective clinical and other services is a major priority. All the above research directions hold promise for refinement of the wraparound practice model, and the development of technologies that may be needed to promote positive outcomes when large scale service reforms are attempted in public child-serving systems.

Acknowledgments

This work was supported by the National Institute for Mental Health (R34 MH072759). We are grateful to Christa Peterson and an anonymous reviewer for feedback on an earlier draft and to Ericka Weathers, Ricki Mudd, Spencer Hensley, Kaylyn Watterson, Andrew Rivers, and Heather Bailey for their work on the study and help with manuscript preparation.

Contributor Information

Eric J. Bruns, University of Washington School of Medicine.

Michael D. Pullmann, University of Washington School of Medicine.

April Sather, University of Washington School of Medicine.

Ramona Denby Brinson, University of Nevada, Las Vegas, School of Social Work.

Michelle Ramey, University of Nevada, Las Vegas, School of Social Work.

References

  1. American Psychological Association. Diagnostic and Statistical Manual of Mental Disorders-Fourth Edition-Text Revised. 4. Washington D.C: Author; 2000. [Google Scholar]
  2. Austin CD. Case management: A systems perspective. Families in Society. 1993;74:451–459. [Google Scholar]
  3. Bender K, Kapp S, Hahn SA. Are case management services associated with increased utilization of adolescent mental health treatment? Children and Youth Services Review. 2011;33:134–138. [Google Scholar]
  4. Bickman L, Smith CM, Lambert EW, Andrade AR. Evaluation of a congressionally mandated wraparound demonstration. Journal of Child & Family Studies. 2003;12:135–156. [Google Scholar]
  5. Bourdon KH, Goodman R, Rae DS, Simpson G, Koretz DS. The Strengths and Difficulties Questionnaire: U.S. Normative Data and Psychometric Properties. Journal of the American Academy of Child & Adolescent Psychiatry. 2005;44(6):557–564. doi: 10.1097/01.chi.0000159157.57075.c8. [DOI] [PubMed] [Google Scholar]
  6. Brannan AM, Sonnichsen S, Heflinger CA. Measuring satisfaction with children’s mental health services: Validity and reliability of the satisfaction scales. Evaluation & Program Planning. 1996;19:131–141. [Google Scholar]
  7. Bruns EJ, Burchard JD, Suter JC, Leverentz-Brady K, Force MM. Assessing fidelity to a community-based treatment for youth: The wraparound fidelity index. Journal of Emotional & Behavioral Disorders. 2004;12(2):79. [Google Scholar]
  8. Bruns EJ, Leverentz-Brady KM, Suter JC. Is it wraparound yet? Setting quality standards for implementation of the wraparound process. Journal of Behavioral Health Services & Research. 2008;35(3):240–252. doi: 10.1007/s11414-008-9109-3. [DOI] [PubMed] [Google Scholar]
  9. Bruns EJ, Rast J, Peterson C, Walker JM, Bosworth J. Spreadsheets, service providers, and the statehouse: Using data and the Wraparound process to reform systems for children and families. American Journal of Community Psychology. 2006;38(3–4):201–212. doi: 10.1007/s10464-006-9074-z. [DOI] [PubMed] [Google Scholar]
  10. Bruns EJ, Sather A. User’s Manual to the Wraparound Team Observation Measure. Seattle, WA: University of Washington, Wraparound Evaluation and Research Team, Division of Public Behavioral Health and Justice Policy; 2007. [Google Scholar]
  11. Bruns EJ, Sather A, Pullmann MD. The Wraparound Fidelity Assessment System: Psychometric analyses to support refinement of the Wraparound Fidelity Index and Team Observation Measure. Paper presented at 24th Annual Children’s Mental Health Research Conference; Tampa, FL. 2010. Mar, [Google Scholar]
  12. Bruns EJ, Suter JC, Force MM, Burchard JD. Adherence to wraparound principles and association with outcomes. Journal of Child & Family Studies. 2005;14(4):521–534. [Google Scholar]
  13. Bruns EJ, Walker JS, Chorpita BF, Daleiden E. Family Voice with Informed Choice: Coordinating Wraparound with Research-Based Treatment for Children and Adolescents. Journal of Clinical Child and Adolescent Psychology. 2013 doi: 10.1080/15374416.2013.859081. Online first. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Bruns EJ, Walker JS, Zabel M, Matarese M, Estep K, Harburger D, Pires SA. Intervening in the lives of youth with complex behavioral health challenges and their families: the role of the wraparound process. Am J Community Psychol. 2010;46(3–4):314–331. doi: 10.1007/s10464-010-9346-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Burns B, Goldman K, editors. Systems of care: Promising practices in children’s mental health. IV. Washington D.C: Center for Effective Collaboration and Practice; 1999. [Google Scholar]
  16. Burns BJ, Farmer EMZ, Angold A, Costello EJ, Behar L. A randomized trial of case management for youths with serious emotional disturbance. Journal of Clinical Child Psychology. 1996;25(4):476–486. [Google Scholar]
  17. Burns BJ, Hoagwood K, Maultsby LT. Improving outcomes for children and adolescents with serious emotional and behavior disorders: Current and future directions. In: Epstein MH, Kutash K, Duchnowski AJ, editors. Outcomes for children and youth with emotional and behavioral disorders and their families: programs and evaluations best practices. Austin, TX: Pro-Ed; 1998. [Google Scholar]
  18. Carney MM, Buttell F. Reducing juvenile recidivism: Evaluating the wraparound services model. Research on Social Work Practice. 2003;13(5):551–568. [Google Scholar]
  19. Clark HB, Lee B, Prange ME, McDonald BA. Children lost within the foster care system: Can wraparound service strategies improve placement outcomes? Journal of Child and Family Studies. 1996;5:39–54. [Google Scholar]
  20. Clark HB, Prange M, Lee B, Stewart ES, McDonald BA, Boyd LA. An individualized wraparound process for children in foster care with emotional/behavioral disturbances: Follow-up findings and implications from a controlled study. In: Epstein M, Kutash K, Duchnowski A, editors. Outcomes for children and youth with emotional and behavioral disorders and their families. Austin, TX: PRO-ED; 1998. pp. 513–542. [Google Scholar]
  21. Cooper JL, Aratani Y, Knitzer J, Douglas-Hall A, Masi R, Banghart P, Dababnah S. Unclaimed children revisited: The status of children’s mental health policy in the United States. New York: National Center for Children in Poverty; 2008. [Google Scholar]
  22. Drake R, Goldman H, Leff H, Lehman A, Dixon L, Mueser K, Torrey WC. Implementing evidence-based practices in routine mental health service settings. Psychiatric Services. 2001;52:179–182. doi: 10.1176/appi.ps.52.2.179. [DOI] [PubMed] [Google Scholar]
  23. Evans ME, Armstrong MI, Kuppinger AD, Huz S, McNulty TL. Preliminary outcomes of an experimental study comparing treatment foster care and family-centered intensive case management. In: Epstein MHEK, editor. Outcomes for children and youth with emotional and behavioral disorders and their families: Programs and evaluation best practices. 1998. pp. 543–580. [Google Scholar]
  24. Farmer TW, Farmer EMZ. Developmental science, systems of care, and prevention of emotional and behavioral problems in youth. American Journal of Orthopsychiatry. 2001;7(12):10. doi: 10.1037/0002-9432.71.2.171. [DOI] [PubMed] [Google Scholar]
  25. Garland AF, Brookman-Frazee L, Hurlburt MS, Accurso EC, Zoffness RJ, Haine-Schlagel R, Ganger W. Mental health care for children with disruptive behavior problems: A view inside therapists’ offices. Psychiatric Services. 2010;61(8):788–795. doi: 10.1176/appi.ps.61.8.788. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Glisson C, Hemmelgarn A. The effects of organizational climate and interorganizational coordination on the quality and outcomes of children’s service systems. Child Abuse and Neglect. 1998;22(5):401–421. doi: 10.1016/s0145-2134(98)00005-2. [DOI] [PubMed] [Google Scholar]
  27. Glisson C, James LR. The cross-level effects of culture and climate in human service teams. Journal of Organizational Behavior. 2002;23(6):767–794. [Google Scholar]
  28. Glisson C, Landsverk J, Schoenwald S, Kelleher K, Hoagwood KE, Mayberg S Research Network on Youth Mental Health. Assessing the Organizational Social Context (OSC) of mental health services: Implications for research and practice. Administration and Policy in Mental Health. 2008;35:98–113. doi: 10.1007/s10488-007-0148-5. [DOI] [PubMed] [Google Scholar]
  29. Glisson C, Schoenwald S. The ARC organizational and community intervention strategy for implementing evidence-based children’s mental health treatments. Ment Health Services Research. 2005;7:243–259. doi: 10.1007/s11020-005-7456-1. [DOI] [PubMed] [Google Scholar]
  30. Glisson C, Schoenwald SK, Hemmelgarn A, Green P, Dukes D, Armstrong KS, Chapman JE. Randomized trial of MST and ARC in a two-level evidence-based treatment implementation strategy. Journal Of Consulting And Clinical Psychology. 2010;78(4):537–550. doi: 10.1037/a0019160. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Goodman R, Scott S. Comparing the Strengths and Difficulties Questionnaire and the Child Behavior Checklist: Is small beautiful? Journal of Abnormal Child Psychology. 1999;27(1):17–24. doi: 10.1023/a:1022658222914. [DOI] [PubMed] [Google Scholar]
  32. Grimes K, Schulz M, Cohen S, Mullin B, Lehar S, Tien S. Pursuing Cost-Effectiveness in Mental Health Service Delivery for Youth with Complex Needs. Journal of Mental Health Policy and Economics. 2011;14:73–86. [PubMed] [Google Scholar]
  33. Hatcher RL, Gillaspy J. Development and validation of a revised short version of the Working Alliance Inventory. Psychotherapy Research. 2006;16(1):12–25. [Google Scholar]
  34. Henggeler SW, Pickrel SG, Brondino MJ. Multisystemic treatment of substance abusing and dependent delinquents: Outcomes, treatment fidelity, and transportability. New York: Guilford Press; 1998. [DOI] [PubMed] [Google Scholar]
  35. Hoagwood KE. Family-based services in children’s mental health: A research review and synthesis. Journal of Child Psychology and Psychiatry. 2005;46(7):23. doi: 10.1111/j.1469-7610.2005.01451.x. [DOI] [PubMed] [Google Scholar]
  36. Hoagwood KE. Family-based services in children’s mental health: a research review and synthesis. Journal of Child Psychology and Psychiatry. 2005;46:690–713. doi: 10.1111/j.1469-7610.2005.01451.x. [DOI] [PubMed] [Google Scholar]
  37. Hodges K. Child and Adolescent Functional Asessment Scale. Ypsilanti, MI: Eastern Michigan University, Departmetn of Psychology; 1997. [Google Scholar]
  38. Hodges K, Kim CS. Psychometric study of the Child and Adolescent Functional Assessment Scale: Prediction of contact with the law and poor school attendance. Journal of Abnormal Child Psychology. 2000;28(3):287–297. doi: 10.1023/a:1005100521818. [DOI] [PubMed] [Google Scholar]
  39. Hodges K, Xue Y, Wotring J. Outcomes for children with problematic behavior in school and at home served by public mental health. Journal of Emotional and Behavioral Disorders. 2004;12(2):109–119. [Google Scholar]
  40. Horvath AO, Greenberg LS. Development and validation of the Working Alliance Inventory. Journal of Counseling Psychology. 1989;36:223–233. [Google Scholar]
  41. Horwitz SM, Hoagwood K, Stiffman AR, Summerfeld T, Weisz JR, Costello EJ, Rost K, Bean DL, Cottler L, Leaf PJ, Roper M, Norquist G. Reliability of the Services Assessment for Children and Adolescents. Psychiatric Services. 2001;52:1088–1094. doi: 10.1176/appi.ps.52.8.1088. [DOI] [PubMed] [Google Scholar]
  42. Hyde KL, Burchard JD, Woodworth K. Wrapping services in an urban setting. Journal of Child and Family Studies. 1996;5:67–82. [Google Scholar]
  43. Kataoka S, Zhang L, Wells K. Unmet need for mental health care among U.S. children: Variation by ethnicity and insurance status. American Journal of Psychiatry. 2002;159(9):1548–1555. doi: 10.1176/appi.ajp.159.9.1548. [DOI] [PubMed] [Google Scholar]
  44. Kazak AE, Hoagwood K, Weisz JR, Hood K, Kratochwill TR, Vargas LA, Banez GA. A meta-systems approach to evidence-based practice for children and adolescents. American Psychologist. 2010;65(2):85–97. doi: 10.1037/a0017784. [DOI] [PubMed] [Google Scholar]
  45. Knitzer J. Unclaimed Children: The Failure of Public Responsibility to Children and Adolescents in Need of Mental Health Care. New York: Children’s Defense Fund; 1982. [Google Scholar]
  46. Koren PE, DeChillo N, Friesen BJ. Measuring empowerment in families whose children have emotional disabilities: A brief questionnaire. Rehabilitation Psychology. 1992;37(4):305–321. [Google Scholar]
  47. McHugo GJ, Drake RE, Teague GB, Xie H. The relationship between model fidelity and client outcomes in the New Hampshire Dual Disorders Study. Psychiatric Services. 1999;50:818–824. doi: 10.1176/ps.50.6.818. [DOI] [PubMed] [Google Scholar]
  48. Mears SL, Yaffe J, Harris NJ. Evaluation of wraparound services for severely emotionally disturbed youths. Research on Social Work Practice. 2009;19(6):678–685. [Google Scholar]
  49. New Freedom Commission on Mental Health. Achieving the promise: Transforming mental health care in America. Rockville, MD: DHHS; 2003. (No. DHHS Pub. No. SMA-03-3832) [Google Scholar]
  50. Penn M, Osher T. The application of the ten principles of the wraparound process to the role of family partners on wraparound teams. In: Bruns EJ, Walker JS, editors. The resource guide to wraparound. Portland, OR: National Wraparound Initiative, Research and Training Center for Family Support and Children’s Mental Health; 2008. [Google Scholar]
  51. Pires SA. Building systems of care: A primer. Washington, DC: National Technical Assistance Center for Children’s Mental Health, Georgetown University Child Development Center; 2002. [Google Scholar]
  52. Pullmann MD, Bruns EJ, Sather A. Evaluating fidelity to the wraparound service model for youth: Application of Item Response Theory to the Wraparound Fidelity Index. Psychological Assessment. 2013;25(2):583–98. doi: 10.1037/a0031864. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Pullmann MD, Kerbs J, Koroloff N, Veach-White E, Gaylor R, Sieler DD. Juvenile offenders with mental health needs: Reducing recidivism using wraparound. Crime & Delinquency. 2006;52(3):375–397. [Google Scholar]
  54. Pullmann MD, VanHooser S, Hoffman C, Heflinger CA. Barriers to and supports of family participation in a rural system of care for children with serious emotional problems. Community Mental Health Journal. 2010;46(3):211–220. doi: 10.1007/s10597-009-9208-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Raudenbush SW, Bryk AS. Hierarchical linear models: Applications and data analysis methods. 2. Newbury Park, CA: Sage; 2002. [Google Scholar]
  56. Raudenbush SW, Xiao-Feng L. Effects of study duration, frequency of observation, and sample size on power in studies of group differences in polynomial change. Psychological Methods. 2001;6(4):387–401. [PubMed] [Google Scholar]
  57. Rauso M, Ly TM, Lee MH, Jarosz CJ. Improving outcomes for foster care youth with complex emotional and behavioral needs: A comparison of outcomes for wraparound vs. residential care in Los Angeles county. Emotional & Behavioral Disorders in Youth. 2009;9(3):63–68. [Google Scholar]
  58. Rauktis ME, Huefner J, O’Brien K, Pecora P, Doucette A, Thompson R. Measuring Restrictiveness of Living Environments for Children and Youth: Reconceptualizing Restriction. Journal of Emotional and Behavioral Disorders 2009 [Google Scholar]
  59. Rosenblatt A. Bows and ribbons, tape and twine: Wrapping the wraparound process for children with multi-system needs. Journal of Child & Family Studies. 1996;5:101–116. [Google Scholar]
  60. Schoenwald SK, Henggeler SW, Brondino MJ, Rowland MD. Multisystemic therapy: Monitoring treatment fidelity. Family Process. 2000;39(1):83–103. doi: 10.1111/j.1545-5300.2000.39109.x. [DOI] [PubMed] [Google Scholar]
  61. Stambaugh L, Mustillo SA, Burns BJ, Stephens RL, Baxter B, Edwards D, DeKraai M. Outcomes from wraparound and multisystemic therapy in a center for mental health services system-of-care demonstration site. Journal Of Emotional And Behavioral Disorders. 2007;15(3):143–155. [Google Scholar]
  62. Singer JD, Willett JB. Applied longitudinal data analysis. New York: Oxford University Press; 2003. [Google Scholar]
  63. Snyder EH, Lawrence C, Dodge KA. The impact of system of care support in adherence to wraparound principles in child and family teams in child welfare in North Carolina. Children And Youth Services Review. 2012;34(4):639–647. doi: 10.1016/j.childyouth.2011.12.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  64. Suter JC, Bruns EJ. Effectiveness of the wraparound process for children with emotional and behavioral disorders: A meta-analysis. Clincial Child and Family Psychology Review. 2009;12(4):336–351. doi: 10.1007/s10567-009-0059-y. [DOI] [PubMed] [Google Scholar]
  65. Tolan PH, Dodge KA. Children’s mental health as a primary care and concern: A system for comprehensive support and service. American Psychologist. 2005;60(6):601–614. doi: 10.1037/0003-066X.60.6.601. [DOI] [PMC free article] [PubMed] [Google Scholar]
  66. U.S. Public Health Service. Report of the Surgeon General’s Conference on Children’s Mental Health: A national action agenda. Washington, DC: Department of Health and Human Services; 2000. [PubMed] [Google Scholar]
  67. Walker JS, Bruns EJ. Building on practice-based evidence: Using expert perspectives to define the Wraparound process. Psychiatric Services. 2006;57(11):1579–1585. doi: 10.1176/ps.2006.57.11.1579. [DOI] [PubMed] [Google Scholar]
  68. Walker JS, Bruns EJ, Penn M. Individualized services in systems of care: The wraparound process. In: Stroul BA, Blau G, editors. The System of Care Handbook: Transforming Mental Health Services for Children and Families. Baltimore: Brookes; 2008. [Google Scholar]
  69. Walker JS, Matarese M. Using a theory of change to drive human resource development for Wraparound. Journal of Child and Family Studies. 2011;20(6):791–803. [Google Scholar]
  70. Walker JS, Sanders B. The Community Supports for Wraparound Inventory: An assessment of the implementation context for wraparound. Journal of Child and Family Studies. 2011;20(6):747–757. [Google Scholar]
  71. Weisz JR, Sandler IN, Durlak JA, Anton BS. Promoting and protecting youth mental health through evidence-based prevention and treatment. American Psychologist. 2005;60:628–648. doi: 10.1037/0003-066X.60.6.628. [DOI] [PubMed] [Google Scholar]
  72. Ziguras S, Stuart GW. A meta-analysis of the effectiveness of mental health case management over 20 years. Psychiatric Services. 2000;51:1410–1421. doi: 10.1176/appi.ps.51.11.1410. [DOI] [PubMed] [Google Scholar]

RESOURCES