Effectiveness and Implementation of Evidence-Based Practices in Residential Care Settings

Abstract

Purpose

Prompted by calls to implement evidence-based practices (EBPs) into residential care settings (RCS), this review addresses three questions: (1) Which EBPs have been tested with children and youth within the context of RCS? (2) What is the evidence for their effectiveness within such settings? (3) What implementation issues arise when transporting EBPs into RCS?

Methods

Evidence-based psychosocial interventions and respective outcome studies, published from 1990–2012, were identified through a multi-phase search process, involving the review of four major clearinghouse websites and relevant electronic databases. To be included, effectiveness had to have been previously established through a comparison group design regardless of the setting, and interventions tested subsequently with youth in RCS. All outcome studies were evaluated for quality and bias using a structured appraisal tool.

Results

Ten interventions matching a priori criteria were identified: Adolescent Community Reinforcement Approach, Aggression Replacement Training, Dialectical Behavioral Therapy, Ecologically-Based Family Therapy, Eye Movement and Desensitization Therapy, Functional Family Therapy, Multimodal Substance Abuse Prevention, Residential Student Assistance Program, Solution-Focused Brief Therapy, and Trauma Intervention Program for Adjudicated and At-Risk Youth. Interventions were tested in 13 studies, which were conducted in different types of RCS, using a variety of study methods. Outcomes were generally positive, establishing the relative effectiveness of the interventions with youth in RCS across a range of psychosocial outcomes. However, concerns about methodological bias and confounding factors remain. Most studies addressed implementation issues, reporting on treatment adaptations, training and supervision, treatment fidelity and implementation barriers.

Conclusion

The review unearthed a small but important body of knowledge that demonstrates that EBPs can be implemented in RCS with encouraging results.

1. Introduction

Evidence-based practices or treatments (EBPs)¹ and residential care are usually not mentioned in the same sentence. In fact, EBPs are a direct response or alternative to what are considered to be costly and ineffective treatment modalities such as residential care. Yet residential care settings (RCS) are wrestling with the increasing demand for EBPs by child-serving systems, prompting some experts to argue that the focus needs to shift to transporting such treatments into RCS (American Association of Children’s Residential Centers [AACRC], 2009; McCurdy & McIntyre, 2004). The American Association of Children’s Residential Centers stated in 2009 that “residential agencies across the country are making efforts to implement EBP…adding client-specific models into their programming; introducing milieu-wide interactive approaches; and working with community partners to send youth to evidence-based treatments offered in community settings” (p.249). As the success of such an endeavor could define RCS’ continued role in the continuum of services for children and youth with special needs, the question arises what is known about the implementation of EBPs in RCS. Three questions guided the current review: (1) Which EBPs have been tested with children and youth within the context of RCS? (2) What is the evidence for their effectiveness within such settings? (3) What implementation issues arise when transporting EBPs into RCS?

1.1. Defining residential care

Residential care has a long history in the provision of services to children who have been maltreated or have significant emotional and behavioral problems. It has been described as a higher level placement, a multi-component intervention, and as a setting that contains and sustains “unique cultures” (AACRC, 2009, p.249). It is an umbrella term, capturing various forms of residentially-based living arrangements, from small group homes to large institutions, across three service systems – child welfare, mental health, and juvenile justice. RCS provide therapeutically planned behavioral health interventions to unrelated youth with a wide range of problems in a 24-hour structured and multidisciplinary care environment (Abt Associates, 2008). Programs vary considerably in size, structure, organization, treatment approach and population served. It is difficult to develop a sensible typology of different RCS and the conceptual imprecision about RCS has hindered knowledge advancement in this field (Butler & McPherson, 2007).

For purposes of this review, RCS was defined as short- or long-term group homes or residential treatment centers. Inpatient psychiatric care settings, which provide acute care in hospital settings, were excluded; so were secure juvenile detention or correctional facilities, which differ in significant ways from traditional RCS aimed primarily at providing psychosocial interventions for children and youth with emotional and behavioral problems.

1.2. The case against residential care

Multiple factors have contributed to RCS’ falling out of favor with child-serving systems, policy makers and consumers (Dodge, 2006). The system of care movement provided ideological justification for a shift away from residentially-based services and toward community-based care in the least restrictive setting. Studies have raised concerns about the potential for abuse within institutions (Colton, Vanstone & Walby, 2002), iatrogenic effects such as negative peer processes (Dishion, McCord & Poulin, 1999), reliance on shift staff with often inadequate training and high turnover rates (Colton & Roberts, 2007), and the failure of RCS to adequately involve the biological family or provide post-discharge services (Barth, 2005). The growing number of community-based alternatives to RCS, such as Multisystemic Therapy (Henggeler, Schoenwald, Borduin, Rowland & Cunningham, 2009), Multidimensional Treatment Foster Care (Chamberlain & Smith, 2005), and Wraparound Services (Walker & Bruns, 2006) have further raised questions about the enormous cost of RCS in light of its perceived limited benefits (Whittaker, Greene & Savas, 2006). Yet some experts argue that RCS continue to have a place in the continuum of services for youth with emotional and behavioral disorders (Leichtman, 2006; McCurdy & McIntyre, 2004).

1.3. Evidence-based treatments in residential care

Youth in RCS have been described as some of the most disordered children, presenting with the whole range of developmental psychopathology (Abt Associates, 2008). Research indicates that RCS can be effective in improving outcome in multiple domains of functioning for some youth (Bean, White & Lake, 2005; Bettmann & Jasperson, 2009; Hair, 2003; James, 2011; Lee, Bright, Svoboda, Fakunmoju & Barth, 2011). However, findings are generally confounded by weak study designs, the effect of mediating or moderating variables, and a lack of specificity about the elements of ‘usual care’ RCS. A 2011 study summarized findings from a structured review of RCS milieu-wide treatment models (James, 2011). It identified five models: Positive Peer Culture, Teaching Family Model, The Sanctuary Model, the Stop-Gap Model and Re-ED. Although four of the five models were rated as effective or promising, it was noted that the combined body of knowledge on these models remains small and is to a large degree dated. While comprehensive milieu-wide treatment approaches may be most fitting to address the complexity of problems experienced by youth in RCS (Burns, Hoagwood & Mrazek, 1999), changing an entire system approach may be difficult to accomplish for many RCS. The growing number of evidence-based client- and diagnostic specific interventions for children and adolescents therefore raises the question whether RCS could not improve their services by importing EBPs into their treatment context (AACRC, 2009). This review captures the current state of knowledge in this area.

2. Methods

2.1. Introduction

The current review differs in several significant ways from a standard systematic review. It started with a selection of interventions rather than studies and as such, used a different search approach than is common. Given the preponderance of nonrandomized evaluation designs in the field of residential care, it opted to “cast the net wide” and include pre-experimental alongside randomized studies. For this reason, it was also less focused about the size of effects and presented findings in the form of a narrative synthesis rather than a meta-analysis. Finally, this review went beyond the scope of a traditional systematic review in that it specifically addressed implementation issues. However, this review shares important features with systematic reviews, including the explication of study questions, clear a priori inclusion/exclusion criteria, a systematic selection process, and a standardized appraisal phase.

2.2. A priori inclusion/exclusion criteria

A priori inclusion/exclusion criteria were as follows: (1) The review aimed to identify psychosocial interventions with some level of effectiveness, that is, efficacy or effectiveness had to have been previously established through a comparison group design regardless of the setting in which the study was conducted. Psychosocial interventions were broadly defined as client-level interventions targeting mental health, substance abuse and/or social skills problems and were differentiated from system-wide or milieu-based interventions, e.g., Teaching Family Model, which have been previously reviewed (James, 2011). (2) The EBP had to have been subsequently tested with children and/or adolescents in RCS, which, as already stated earlier, was defined as short- or long-term group homes or residential treatment centers, excluding inpatient psychiatric care settings and secure juvenile detention or correctional facilities. (3) Studies had to have been published in the peer-reviewed literature between the years 1990–2012 since EBPs had limited penetration into social work and other human services professions until the 1990s. (4) Single-subject/case studies and grey literature sources were excluded.

2.3. Search strategy

The multi-phase search strategy was shaped by several considerations: Interventions already rated to have some evidence for effectiveness were targeted for inclusion. However, given the lack of a comprehensive list of psychosocial evidence-based treatments for children and adolescents, such interventions cannot be easily identified. In consultation with several child welfare/mental health experts, clearinghouses providing evidence ratings for a long and growing list of interventions were therefore used as the starting point for the search. These included the California Evidence-based Clearinghouse for Child Welfare (CEBC) (www.cebc4cw.org), SAMHSA’s National Registry for Evidence-Based Programs and Practices (NREPP) (www.nrepp.gov), the Office of Juvenile Justice and Delinquency Prevention (OJJDP) (www.ojjdp.gov), and the Center for the Study and Prevention of Violence (CSPV) (www.colorado.edu/cspv/blueprints). All four clearinghouses aim to assist practice communities in identifying and/or implementing EBPs and have standardized review and rating procedures. Each organization has a slightly different focus. The CEBC focuses on interventions and treatments that are relevant to improve the safety, permanency and well-being of children and families in the care of the child welfare system. The NREPP reviews treatments related to mental and/or substance use disorders for both children and adults. The OJJDP focuses primarily on interventions for youth and families struggling with delinquency. The CPSV provides information about model programs and promising programs relevant for violence and drug prevention. It also publishes a helpful program matrix that includes a long list of evidence-based treatments with information about ratings by different clearinghouses (see websites for more information about each organization’s aims).

The search was conducted between February and April 2012 and involved the following steps:

1. Each clearinghouse website was accessed and broad parameters entered to narrow the search to the target age group, outcomes of interest and comparison group study design. The search was not restricted to interventions specifically developed for children and youth in RCS since interventions are sometimes delivered in settings beyond their originally intended use. Parameters were as follows and yielded the following number of interventions for each website:

   • CEBC: age: 9–18; scientific rating: well supported, supported, promising; topic areas: anxiety treatment, behavioral management, bipolar disorder treatment, casework practice, depression treatment, disruptive behavior treatment, mentoring programs, motivation and engagement, sexual behavior problems in adolescents and children, substance abuse treatment, trauma treatment, youth transitioning in adulthood (n=48 interventions).

   • NREPP: age: 6–12 (Childhood), 13–17 (Adolescent); setting: inpatient, residential, outpatient, correctional, home; outcome categories: alcohol, crime/delinquency, drugs family/relationships, mental health, quality of life, social functioning, suicide, trauma/injuries, treatment/recovery, violence; study designs: experimental, quasi-experimental (n=49 interventions).

   • OJJDP: age: 11–13, 14–21; problem behaviors: aggression/violence, delinquency, family functioning, gang activity/involvement, trauma exposure; program types: aftercare, alcohol and drug therapy/education, CBT, conflict resolution/interpersonal skills, correctional facilities, family therapy, group home, mentoring, residential treatment center, shelter care, wilderness camps, wraparound/case management (n=25 interventions).

   • CPSV: all model and promising programs (n=33 interventions).

     After removal of duplicates, 126 interventions were identified. Each intervention was subsequently reviewed by reading intervention descriptions and synopses of respective outcome studies. If the description of the intervention indicated that it was aimed at high-risk youth or could be delivered in RCS, additional screening occurred to determine whether a priori search criteria were met (e.g., evaluation of intervention through a comparison group design study, implementation of intervention with youth in RCS). This involved reviewing studies on the intervention, which were listed on the clearinghouse website. It also involved accessing relevant electronic databases (e.g., EBSCO, Web of Science, Googlescholar, PubMed) using key terms, such as the name of the intervention and/or author of published studies on the intervention, to triangulate the information provided on the website. Most interventions could be ruled out easily; a few required an in-depth search.

2. A second rater independently reviewed half of randomly chosen interventions on the list to ensure that no intervention was prematurely excluded. Questions about the eligibility of an intervention for inclusion were resolved through consensus.

3. Once an intervention was identified as meeting criteria, electronic databases (see 2) were again accessed to ensure that all outcome studies related to the index intervention, including recently published studies, which may not have been listed yet on the clearinghouse website, had been found.

4. In a parallel process, electronic databases were accessed using key terms and their correlates such as evidence-based treatment/practice and residential care to prevent inadvertent exclusion of interventions that may meet criteria but may not yet have been reviewed by one of the four clearinghouses or were excluded because of the specification of the initial search criteria. Several interventions were found through this method.

5. Lastly, reference lists of outcome studies of eligible interventions were reviewed to ensure that no outcome studies were overlooked.

3. Results

3.1. What EBPs have been tested with children and youth in RCS?

Ten interventions were identified in this search (see Table 1). They covered a range of treatment approaches and target outcomes, reflecting the breadth and depth of problems prevalent in RCS populations. Table 1 indicates through which sources the respective interventions were identified, and what effectiveness rating they had received by the various clearinghouses.

Table 1.

Search Results

Intervention	CEBC www.cebc4cw.org	NREPP www.nrepp.samhsa.gov	OJJDP http://www.ojjdp.gov/mpg/	Blueprints http://www.colorado.edu/cspv/blueprints/	Electronic Databases
Adolescent Community Reinforcement Approach (A-CRA)	Godley et al. (2002) Godley et al. (2006) Supported by research evidence	Godley et al. (2006) 3.3–3.6 (Scale 0.0–4.0)	Not reviewed	Not reviewed Listed on Program Matrix	Godley et al. (2002) Godley et al. (2006)
Aggression Replacement Training (ART®)	To be reviewed next year (per personal communication, Rolls, 02/14/12)	(lists Mendota Juvenile Treatment Center [MJTC] Program under ART – MJTC for youth in secure correctional facilities – does not meet review criteria)	(lists Coleman et al., 1991 unpublished manuscript) Effective	Not reviewed Listed on Program Matrix	Coleman et al. (1992) Nugent et al. (1999)
Dialectical Behavior Therapy (DBT)	To be reviewed next year (per personal communication, Rolls, 02/14/12)	Reviewed only for adults 3.2–3.7 (Scale 0.0–4.0)	Not reviewed	Not reviewed Listed on Program Matrix	Sunseri (2004) The Grove…(2004) Wasser et al. (2008)
Ecologically-Based Family Therapy (EBFT)	Slesnick & Prestopnik (2009) Supported by research evidence	Not reviewed	Not reviewed	Not reviewed Listed on Program Matrix	Slesnick & Prestopnik (2009)
Eye Movement Desensitizati on and Reprocessing (EMDR)	Soberman et al. (2002) Well supported by research evidence	Reviewed for adults only	Not reviewed	Not reviewed Listed on Program Matrix	Soberman et al. (2002)
Functional Family Therapy (FFT)	Slesnick & Prestopnik (2009) Supported by research evidence	Not reviewed	Reviews FFT studies not in RCS Exemplary	Model Program	Slesnick & Prestopnik (2009)
Multimodal Substance Abuse Prevention (MSAP)	Not reviewed	Not reviewed	Friedman et al. (2002) Promising	Not reviewed Listed on Program Matrix	Friedman et al. (2002)
Residential Student Assistance Program (RSAP)	Morehouse & Tobler (2000) Promising	Morehouse & Tobler (2000) 3.0 (Scale 0.0–4.0	Morehouse & Tobler (2000) Effective	Not reviewed Listed on Program Matrix	Morehouse & Tobler (2000)
Solution-Focused Brief Therapy (SFBT)	Not reviewed	Not reviewed	Reviewed, but does not list study conducted in RTC Promising	Not reviewed Listed on Program Matrix	Koob & Love (2010)
Structured Sensory Therapy for Traumatized Adjudicated Adolescents in Residential Treatment (SITCAP-ART)	Raider et al (2008) Promising	Raider et al. (2008) 2.3–2.5 (Scale 0.0–4.0)	Not reviewed	Not reviewed Listed on Program Matrix	Raider et al. (2008)

The Adolescent Community Reinforcement Approach (A-CRA) “is a behavioral intervention that seeks to replace environmental contingencies that have supported alcohol or drug use with prosocial activities and behaviors that support recovery” (www.nrepp.gov). The approach has been adapted to provide aftercare treatment to youth following RCS for alcohol and/or substance abuse problems. Aggression Replacement Training (ART^®) uses structured learning, anger control training and moral education to address the behavior of adolescents with severe behavior problems. It has been used within different settings, including schools and correctional facilities and has been evaluated in numerous studies. Dialectical Behavioral Therapy (DBT) is a cognitive-behavioral intervention with several treatment components aimed at addressing severe emotional and behavioral issues that result in self-injurious or suicidal behaviors. Ecologically-Based Family Therapy (EBFT) integrates concepts from crisis intervention theory and multi-systemic models to offer services to youth and families in crisis in a home-based setting. Eye Movement and Desensitization Therapy (EMDR) is a “one-on-one form of psychotherapy that is designed to reduce trauma-related stress, anxiety, and depression symptoms associated with posttraumatic stress disorder and to improve overall mental health functioning” (www.nrepp.gov). Functional Family Therapy (FFT) integrates and conceptually links behavioral and cognitive intervention methods within an overall systemic approach and has been used successfully for the treatment of youth substance abuse. Multimodal Substance Abuse Prevention (MSAP) involves skills training in several areas (life skills, aggression management, values clarification) to reduce substance abuse and delinquent behaviors. The Residential Student Assistance Program (RSAP) aims to prevent or reduce alcohol and drug use among high-risk youth placed in RCS. Solution-Focused Brief Therapy (SFBT) involves a paradigmatic shift to treatment, which is focused on solutions that are derived from clients’ wants and successes, instead of problems. It can be applied in individual and group settings and has been tested with a range of target outcomes, e.g., depression, parent-child conflict, recidivism (Gingerich & Eisengart, 2000). Lastly, the Trauma Intervention Program for Adjudicated and At-Risk Youth (SITCAP-ART) was designed for youth who have experienced trauma but are also on probation for delinquent acts. Five of the interventions were not specifically designed for delivery in RCS (DBT, EBFT, EDMR, FFT, SFBT). ART^® and SITCAP-ART include RCS in their delivery settings. A-CRA, MSAP and RSAP specifically target youth in RCS.

For the ten interventions, 13 articles presenting outcomes for youth in RCS and/or youth transitioning out of RCS were identified. One study represented two interventions – EBFT and FFT (Slesnick & Prostopnik, 2009), and two papers published by Godley and colleagues (2002, 2006) about A-CRA present preliminary and final findings for the same study.

3.1.1. Exclusions

While most reviewed interventions could be ruled out easily based on a priori criteria, some required an in-depth review, which sharpened exclusion criteria and deserve noting: (1) A long list of interventions (e.g., Multidimensional Family Therapy, Mendota Juvenile Treatment Center, Moral Reconation Therapy, Vicarious Sensitization, Family Integrated Transitions, Operation New Hope, etc.) have been tested within the context of secure correctional facilities, which were not part of this review but are sometimes listed under RCS. The difference between a secure correctional facility and a RCS for juvenile offenders was not always apparent. (2) Given the exclusion of secure correctional facilities and inpatient psychiatric settings a few outcome studies of interventions that otherwise met criteria were also excluded. For instance, DBT and ART^® have been implemented in such facilities (Holmqvist, Hill & Lang, 2009; McDonnell et al., 2010; Trupin, Stewart, Beach & Boesky, 2002). (3) Shelters are sometimes described as short-term RCS. Two studies testing three interventions (ART^®, EBFT, FFT) were identified which were conducted while youth resided in such settings (Nugent, Bruley & Allen, 1998; Slesnick & Prestopnik, 2009). Studies with youth in shelters were excluded if the shelter only served as a place of recruitment and the intervention was delivered after youth had been discharged from the shelter (Slesnick & Prestopnik, 2004). (4) A number of interventions have been designed as alternatives to RCS, and evaluative studies of these interventions may include RCS as a comparison group (e.g., Family Behavior Therapy, Multidimensional Treatment Foster Care, Multisystemic Therapy). Such interventions were not included. (5) Evaluative studies of EBPs whose samples included a subset of youth from RCS were also excluded since the effectiveness of the intervention for youth in RCS could not be determined separately (Nelson-Gray, Keane & Cobb, 2006; Worling & Curwen, 2000). (6) Questions remained about the inclusion of Mode Deactivation Therapy (MDT), an intervention developed by Apsche and tested in RCS (Apsche, Bass, Zeiter & Houston, 2008; Murphy & Siv, 2011; Thoder & Cautilli, 2011). Due to inconsistent information about important aspects of the methods, which could not be resolved, and the fact that MDT had not been reviewed or listed by any of the clearinghouses, this intervention was excluded from this review.

3.2. What is the evidence for the treatments’ effectiveness within RCS?

The identified outcome studies were published between 1992 and 2010. They were conducted in different types of RCS and included short-term RCS or shelters (Nugent et al., 1998; Slesnick & Prestopnik, 2009) and programs specific to delinquent youth (Friedman, Terras & Glassman, 2002; Godley, Godley, Funk & Passetti, 2002, 2006; Raider, Steele, Delillo-Storey, Jacobs & Kuban, 2008), children in the child welfare system (Koob & Love, 2010), and youth with substance abuse problems (Friedman et al., 2002; Godley et al., 2002, 2006; Morehouse & Tobler, 2000; Slesnick & Prestopnik, 2009). One study involved six RCS from different service systems, including one that was a secure correctional facility (Morehouse & Tobler, 2000). (Since the six other RCS met a priori inclusion criteria, the study was included in this review). Not every study provided detailed information about the setting, but the remaining studies appeared to have been conducted in RCS serving youth with a range of emotional and behavioral problems from different service systems. All study participants included youth, ages 10–18. A majority of participants were white. Most RCS were mixed gender programs. Table 2 provides an overview of key features of all reviewed studies.

Table 2.

Overview of Outcome Studies

Study	RCS Setting	Sample	Design	Intervention/ Control	Target Outcome/ Measures	Key Findings
Adolescent Community Reinforcement Approach (A-CRA)
Godley et al. (2002) Preliminary findings	RCS in IL LOS based on individual needs Curriculum based on rational emotive therapy, social learning theory, cultural work on pathways to addiction and recovery Rotating group sessions, e.g., 12-step, counseling, spirituality, family groups, relapse prevention, etc.	N = 114 Dx of current alcohol and/or marijuana dependence Age range: 12–17 Dependents of CW were excluded 75% Caucasian 44% involved with probation M LOS: 49 days	Randomized block design ACC: n=63 UCC: n=51	ACC vs. UCC ACC: UCC services (e.g. CM, CRA), being assigned a case manager for 90-day period post-d/c; CM provided CM services that included CRA procedures; home visits, transportation for job finding and pro-social activities UCC: referrals for continuing care to local outpt providers; great variability in intensity and types of services.	Reduction in time until, amount of, and problems related to relapse (Global Appraisal of Individual Need [GAIN], Form 90 version of the Time Line Follow Back) Increase the likelihood, amount and content of continuing care (Service Contact Logs)	ACC vs UCC more likely to receive: tx as described in the ACRA manual (79% vs. 61%; x2=4.73, p<.05, d=.42) family-related services (71% vs 41%; x2=10.58, p<.001, d.=.64) CM services (79% vs 59%; x2=5.68, p<.05, d=.46) continuing care services (92% vs 59%; x2=17.69, p<.05, d=.86) more overall continuing care services (M=14.4 vs. M=7.6; t=2.78, p<.05, d=.48) Median days to marijuana use was longer for ACC group w/in 90 days after d/c (90 days vs. 31 days; x2=4.07, p<.05, d=.39) More likely to be abstaining from marijuana at 90 mths post d/c (52% vs. 31%, x2=5.08, p<.05, d=.43) ↓ % of days using alcohol (−64% vs −18%; F=5.62, p<.05, f=.24)
Godley et al. (2006) Update of 2002 study	see Godley et al. (2002)	N = 183 Gender:71 % m 73% Caucasian, 18% AfAm M age: 16.2 82% prior involvement with JJ system	see Godley et al. (2002) ACC: n=102 UCC: n= 81 f/up interview data at 3, 6, 9 mos	see Godley et al. (2002)	see Godley et al. (2002)	ACC vs UCC: 90 days post d/c, 94% vs. 54% were linked to continuing care (x2=40.1, p<.001, d=1.07) More days of continuing care session (M=18.1 vs. M=6.3, t=4.66, p<.001, d=.64) Median number of CC sessions was 15 vs. 2 (x2=29.6, p<.001, d=.90) ↑ odds of high adherence to continuing care criteria (OR=3.35, p<.05) Abstinence rates for marijuana at 9 mths f/up 41% vs. 26% (x2=4.45, p<.05) Abstinence during 4–9 mths post d/c predicted by ACC, high continuing care adherence, being abstinent during 1st 3 mths High adherence predicted alcohol & other drug abstinence at 3 mths f/up (OR=2.16, p<.05; early abstinence predicted abstinence at 9 mths f/up (OR=11.16, p<.05); similar pattern for alcohol and marijuana
Aggression Replacement/Training (ART®)
Coleman et al. (1992)	Devereux Foundation-Texas Center for youth with behavioral d/o	Staff selected youth for participation based on specified eligibility criteria N=39 completed the study Gender:10f, 29m Age: range 13–18; M=15.9 67% White, 25% Black; 8% Hisp. 64% CD, 16% depression, 10% ODD, 10% other dx 46% in legal custody of parents almost all had prior RCS placements	Experimental Baseline-E: n=24 and C: n=16 3 mths f/up was planned but could not be implemented Analysis controlled for baseline differences; however, unclear which differences existed	Experimental:ART® -10 wk module w/ structured learning, anger control, moral education Daily sessions during 7th period of school= 50 hrs Taught in groups of 6 ct/s by 2 group leaders Control group: regular 7th period academic/vocational classes	Social skills (Direct Situation Test, ART Checklist) Moral reasoning (Sociomoral Reflections Measure) Self-control (Self-Control Rating Scale) Aggression/anger management (Behavior Incident Report) Aptitude by treatment interaction (Scale 4 of MMPI, Index of Personality Characteristics)	Social skills knowledge improved (F=10.77; p<.01) 3 skills accounted for improvement in social skills knowledge: ‘keeping out of fights,’ ‘dealing w/ group pressure,’ ‘expressing a complaint’ Little impact on empathy Selected personality variables predicted Self-Control Rating (F=12.65; p<.01)
Nugent et al. (1998)	Short-term RCS (shelter) M LOS 3 wks	N = 522 youth M age: 14.9 yrs 55% in custody of parents, 38% in state custody, remaining in custody of relatives Race: 77% White, 18.4% African American, 2.5% Latino, 0.4% Asian American, remainder ‘other’	Interrupted time series design (519-day period) Pre-tx daily Antisocial Behavior (ASB) rate obtained for 310-day period prior to ART® Posttx rate obtained for a 209-day period after ART®	ART® condensed into 15-day program delivered over a 21-day time period Youth participated in 1 skills-training group/day lasting 60–90 min Group size: 7–10 Curriculum did not include moral reasoning component	Daily rate of antisocial behavior (ASB) Daily number of ASB-related incidents (Review of ct case files documenting ASB-incidents, i.e., violation of rules and/or behavior guidelines, legal/social norms, personal property violations, aggression)	Following ART® 20% ↓ in the daily rate of ASB (p<.05) 17.2% ↓ of number of ASB incidents (p<.03)
Dialectical Behavior Therapy (DBT)
Sunseri (2004)	Summitview Child Treatment Center; RCS for 18 girls, ages 12–18, CA	N=68 Ages 12–18 Pre mean age: 14.0; post mean age: 15.2 82% White Full range of psychiatric dx; primarily disruptive behavior	Pre-post Pre 1997–1999 (29 mos) n=42 Post: 2000–2002 (29 mos) n=26 Differences between groups in age, mood d/o, BPD, and possibly level of motivation	DBT: detailed discussion of tx elements; included individual tx, group skills tx, phone coaching, team consultation	premature discharges due to suicidality (case records) number of days in psych hospital (Educational Attendance Form) restraints and seclusions (Unusual Incidence Report Form)	Significant ↓ in premature terminations (16.7% vs. 0%; x2=4.9, p<.04) inpatient days, after controlling for covariates (71 vs. 42 days; x2=10.3, p<.001) duration of restraints and seclusion (20min vs. 11min; z=12.1, p<.001)
The Grove (2004)	RCS in MS for youths, age 13–21; 9-bed facility Uses principles of DBT Target population: youth w/ severe MH problems unable to reside at home	N=20 Gender: 14f; 6m M age: 16.6 yrs M LOS-10.3 mths	Pre-post	DBT: individual tx, group skills, family tx and skills; consultation, coaching - 6 mths tx cycles	Program completion Living arrangement post-discharge Days hospitalized Self-injury (Program records)	12 youth completed program 6mths post d/c, 8 were living in the community, 3 in a RCS, 1 was hospitalized Reduction in self-injury (no statistical parameters or p-values provided)
Wasser et al. (2008)	Youth from RCS in 2 states No details	Primary match sample N=14 (7 per group) Gender: 1f, 6m M age: 14.3 Over 1 year stay in RTC Secondary match sample N=24 (12 per group) Gender: 3f, 9m M age: 14.6	Matched comparison group design Matched on Axis I dx, gender, age	DBT vs Standard Therapeutic Milieu (STM) DBT: weekly skills group for 17 wks; included parents for some groups STM: behavioral, group, family, individual, psycho-pharmacology Limited DBT training; skills training groups only	Subscales of Brief Psychiatric Rating Scale - Behavior problems Depression Thought disturbance Psychomotor excitation Withdrawal Anxiety Organicity Data abstracted from hospital system)	Significant overall effects for depression (p<.001), psychomotor excitation (p<.05), anxiety (p<.05) DBT more effective than STM in ↓ depression (t=.04, p<.01) STM more effective on Psychomotor Excitation subscale (t=.02, p<.05) DBT cts had 100-day longer stays
Ecologically-based Family Therapy (EBFT)
Slesnick & Prestopnik (2009)	2 shelters in NM	N=119 Gender: 45%m, 55%f Age range: 12–17; M age: 15.1 Race: 44% Hisp, 29% Anglo, 11% Native Am., 5% AfAm, 11% other Primary problem alcohol use Wards of the state and those without family to return to were excluded	Experimental (urn randomization), baseline, 3,9,15 mths f/ups E-EBFT: n=37 E-FFT: n=40 C-SAU: n=42 Average shelter stay 9 days at baseline; 17 days at posttx assessment	Home-based Ecologically-Based Family Therapy (EBFT), office-based Functional Family Therapy (FFT), Service as usual (SAU) EFBT & FFT-16× 50 min sessions SAU : primarily case management, crisis intervention and individual therapy as requested	Substance abuse (Form 90, urine toxicology) Psychological functioning (Youth Self Report (CBCL), BDI, NYSDS, CDISC) Family functioning (FES, CTS, Parental bonding instrument)	Time Effects: % days of drug use (F=3.09, p<.05), % days of alcohol use (F=10.45, p<.001), problem consequences (F=5.39, p<.01), Adolescent Drinking Index Score (F=3.34, p<.05), verbal aggression (F=8.53, p<.001), conflict and cohesion (F=3.30, p<.05), # of psych dx (F=4.48, p<.01), YSR externalizing problems (F=2.97, p<.05), NYSDS delinquency (F=11.45, p<.001) Time x Tx Effects: EBFT and FFT signif. ↓ substance use at 15 mths (EBFT: 97% ↓ days of alcohol vs 83% ↓ for FFT; EBFT: F=5.60, p<.01, η2=.20; FFT: F=7.70, p<.001, η2=.25) FFT fewer substance abuse dx than SAU at 3 mths f/up (F=4.36, p</05, η2=.13) Treatment participation Highest tx attendance for EBFT vs. FFT (10.3 sessions vs. 6.51, F=5.31, p<.05) Lowest tx refusal for EBFT vs FFT (3 participants vs. 10, x2=3.91, p<.05)
Eye Movement and Desensitization Therapy (EMDR)
Soberman et al. (2002)	One facility w/ residential and day tx services for boys w/ CD Type of RCS not specified	N=29 boys w/ CD Age 10–16 59% CD, 31% PTSD, and other d/o	Experimental E: n=14, EMDR C: n=15, TAU Subjects matched on: age, type of tx program, time in program, use of meds, and SW rating of level of family involvement No baseline differences Data collection intervals: 1 wk pre, 1 wk post, 2 mths post tx (f/up n=23)	EMDR vs TAU EMDR: Wkly 1-hr EMDR plus TAU, 3 sessions TAU Individual and group tx, special ed, behavior modification, meds and/or psycho ed parent/family counseling	Trauma (Subjective Units of Distress Scale [SUDS], Impact of Events Scale-8 Items, Child Report of PTS, Parent Report of PTS Behavior Problems (Problem Rating Scale [PRS], Behavioral Reward Scale [BRS])	Significant ↓ for EMDR vs. TAU at 2 mths f/up in 2 areas: SUDS score (memory-related distress) (6.1 pts vs. 0.38 pts, F=44.30, p<.0001) Mean PRS (8.7 vs. 2.6 pts, t=−.558, p<.05)
Functional Family Therapy (FFT)
Slesnick & Prestopnik (2009)	see EBFT	see EBFT	see EBFT	see EBFT	see EBFT	see EBFT
Multimodal Substance Abuse Prevention (MSAP)
Friedman et al. (2002)	St. Gabriel’s Hall - RCS for court-adjudicated adolescent males in Philadelphia cottage program, 20 youth per unit standard RCS services, but no “specific systematic programming or staffing for drug prevention”	N=201 adolescent males Age: 13–18 Convicted of at least one illegal offense leading to court proceedings Many risk factors	Experimental w/ 2 post-tests – 9 mths (at d/c) and 6 mths after d/c E: n=110 Triple-modality classroom C: n=91 TAU No baseline differences on age, yrs of ed, occupational level of caregiver, welfare recipient Differences in race (↑ Af Am in E; ↑ intact families in C)	MSAP vs TAU MSAP: “basic residential program” + special classroom program with 3 different modalities: (1) CB social learning model focused on substance abuse (Botvin Life Skills Training); (2) social learning model for violence prevention (Prothrow-Stith Anti-Violence Program); (3) social learning model for value clarification (Values Clarification) TAU: “basic residential program”	Degree of substance abuse Degree and seriousness of illegal offenses Degree of violent illegal offenses committed Frequency of selling of drugs Degree of school problems (Adolescent Drug Abuse Diagnosis Instrument) Behavior problems Youth Self Report (YSR) Social Desirability (Crowne-Marlowe Scale Short Form) Process Evaluation	MSAP vs. TAU ↓ in drug use (t=−2.58, p<.05), frequency of selling of drugs (t=−.199, p<.05) No significant ↓ in total degree of illegal behavior, degree of violent illegal behavior, and school problems
Residential Student Assistance Program (RSAP)
Morehouse & Tobler (2000)	Implemented in 6 NY RCS “extremely high-risk adolescents”	Youth from 7 facilities who had been in RCS for 30 days or more 3 CW RCS w/ 50–100 youth each; age: 13–18 1 nonsecure facility for adjudicated juvenile offenders – n=200, age 13–16 1 tx center for 65 adolescents, age 13–18 w/ severe psychiatric problems 1 locked county correctional facility for 200 youth, age 16–19 1 CW facility w/out RSAP counselor	Pre-post non-equivalent comparison group design Eligibility criteria unclear Intervention group: n=132 2 cross-sectional comparison groups 1991–92 In-house comparison group (no RSAP) n=179 Out-of-house comparison group n=76 1992–93 In-house comparison n=168 Out-of-house comparison n=33 No significant differences between E and C for gender, age, race or substance use in the last 30d	RSAP modeled after EAP for alcohol/substance abuse problems Highly trained counselors work w/ youth individually and in small groups, conduct training for staff, coordinate programs, provide f/up tx and referrals Prevention ed groups of 8–10 6–8 sessions Outreach activities 45-min group counseling for youth w/ substance abusing parents Facilitates involvement in 12-step programs Adolescent Resident Task Force	Alcohol/drug use in last 30d quantity frequency number of drugs program success/failure type of user (nonuser, experimental, user/abuser) intensity/dosage (revised Monitoring the Future questionnaire - Institutional climate (Community Oriented Program Environment Scale)	RASP: ↓ in AOD use from pre to post in quantity/frequency (t=4.25, p<.001), number of drugs (t=4.99, p<.001) Comparison no significant ↓ RASP youth with high dosage intervention showed greater ↓ in AOD quantity/frequency (t=3.32, p<.001) and number of drugs (t=2.14, p<.05). Effect size of .68 for high-dosage group Post-scores (quantity/frequency) in RSAP group were predicted by pretest use (t=10.76, p<.001), Dosage (5–11 hrs) (t=2.82, p<.01) and site rank (t=3.37, p<.01) Considerable variability of AOD across sites Significant differences in program implementation
Solution Focused Brief Therapy (SFBT)
Koob & Love (2010)	RCS for ‘unplaceable’ youth due to hx of failed placements and dx of emotional/behavior d/o Primarily referrals from CW RCS used CBT; no focus on family	N=31 Age: 12–17 Gender: 14f, 17m 58% White; 42% Black Hx of multiple failed placements Eligibility criteria: tx with CBT 1 yr prior to implementation of SFBT; could not be eligible to age out	Pre/post; 1 yr prior to implementation of SFBT; 1 yr f/up period	SFBT: basic tenets, e.g., ct as expert, problems not connected to solutions, solution-/future-focused Techniques: coping, scaling, miracle questions CBT: basic tenets, e.g., positive/negative reinforcement; problem-/present-focused Techniques: education, role play, modeling, anger management, problem-solving	Placement stability (Archival Records)	Significant ↓ in placement disruptions from 6.29 at pre-test to 1.45 at post-test (t=7.51, p<.001)
Structured Sensory Therapy for Traumatized Adjudicated Adolescents in Residential Treatment (SITCAP-ART)
Raider et al. (2008)	Multi-County Juvenile Attention Center, OH Collaboration between JJ and MH	N=20 adjudicated youth Gender:11m, 9f 85% white 2/3 were 16–17 y/o 3/4 had multiple trauma	Randomized waitlist control E: n=10 SITCAP-ART C: n=10 waitlist; started after 10 wks once E group had completed tx 5 study dropouts Inconsistent reporting of sample size and number of dropouts	SITCAP-ART: comprehensive tx approach to decrease terror and facilitate feelings of safety Integrates cognitive strategies with sensory strategies 10–11 sessions (1/wk): 7 in group w/6 members, 1 debriefing session, 1 processing session, 1 parent/youth session Parent component	Trauma (Trauma Symptom Checklist for Children; Child and Adolescent Questionnaire) Mental health sx (Youth Self Report) Case record review	Findings seem to be reported on a different ‘n’ than originally reported ↓ in avoidance of stimuli sx (M=39.22 vs. 22.05; p<.05) ↓ in anxiety/depression sx (M=12.33 vs M=7.26; p<.05) ↓ in internalizing behavior (M=28.11 vs. M=17.63; p<.05)

Notes.- Glossary: ACC – Assertive Continuing Care; C – control/comparison group; CD – Conduct Disorder; CM – Case Management; CRA – Community Reinforcement Approach; ct/s – client(s); CW- child welfare; d/o – disorder(s); dx – diagnosis; E – experimental group; f/up – Follow-up; JJ – Juvenile Justice; LOS – Length of stay; M – mean; mths – months; ODD – Oppositional Defiant Disorder; wk – week;

3.2.1. Quality Appraisal

Studies were appraised using Fowkes and Fulton’s appraisal tool (1991).² The tool aids in the appraisal of six areas of research - study design, study sample, adequacy of control/comparison group, quality of measurements and outcomes, completeness, and distorting influences – by indicating the presence of major, minor, and no problems in each area. A not applicable option is also provided. While the appraisal tool does not offer cutoffs to judge the quality of a study, it attempts to aid reviewers in judging a study’s risk of bias, presence of confounding factors, and whether results may have been derived by chance. All studies were reviewed by two raters independently. Discrepancies about areas of weakness were resolved by consensus. Appraisal results are presented for each area except distorting influences as these are in large part addressed in the implementation section of this paper.

3.2.1.1. Study design

Six studies used randomization.³ They include studies on the A-CRA (Godley et al., 2002, 2006), Coleman et al.’s study on ART^® (1992), Soberman et al.’s study on EMDR (2002) Slesnick and Prestopnik’s study examining EBFT and FFT (2009), Friedman et al.’s MSAP study (2002), and Raider et al.’s study on SITCAP-ART (2008). Three studies used quasi-experimental designs: Nugent et al. used an interrupted time series design to test ART^® in a short-term RCS (1998). Wasser et al. (2008) used a matched comparison group design to test DBT in RCS from two states. Morehouse and Tobler (2000) used a pre-post non-equivalent comparison group design to examine the effectiveness of RSAP. The remaining studies used pre-post designs without comparison group: Sunseri examined the effect of DBT in a RCS for teen girls (2004). A RCS in Massachusetts reported on pre-post findings for a newly implemented DBT program (The Grove, 2004), and Koob and Love (2010) examined the pre-post effect of SFBT on placement stability.

3.2.1.2 Representativeness of study sample

Four studies relied on archival data and their sample included all youth who were in the RCS facility during a specified time period (Koob & Love, 2010; Nugent et al., 1998; Sunseri, 2004; The Grove, 2004), or in the case of Wasser et al.’s study (2008) had received DBT in one RCS organization. The remaining studies used convenience samples, and in most cases participants were recruited or selected from one RCS organization/facility (Coleman et al., 1992; Friedman et al., 2002; Godley et al., 2002, 2006; Koob & Love, 2010; Nugent et al., 1998; Raider et al., 2008; Soberman et al., 2002; Sunseri, 2004; The Grove, 2004). While limiting generalizability, convenience sampling is common in many clinical experimental studies. Also, given the heterogeneity of various RCS settings, conducting outcome studies in one facility may in fact be appropriate. Exceptions included Wasser et al.’s DBT study (2008), which was conducted in RCS settings in two states, Slesnick and Prestopnik’s study of EBFT and FFT (2009), which was conducted in two short-term RCS in New Mexico, and Morehouse and Tobler’s study of RSAP (2000), which was implemented in six New York RCS.

Generalizability was further limited by exclusion criteria. For instance, Godley et al. (2002, 2006) and Slesnick and Prestopnik (2009) excluded youth who were wards of the state, and in the case of the latter study, youth who had no families to return to. As such, these two studies excluded youth with characteristics common to a significant portion of the general RCS population. In several studies exclusion criteria were aimed at sorting out youth with conditions that would prevent them from participating in the treatment or providing consent (Coleman et al., 2002; Friedman et al., 2002; Godley et al., 2002, 2006; Slesnick & Prestopnik, 2009; Soberman et al., 2002). Other studies excluded youth with shorter stays in RCS (Godley et al., 2002, 2006; Morehouse & Tobler, 2000).

The percentage of non-respondents ranged from 2% to 40% (2% in Slesnick & Prestopnik, 2009; 18% in Godley et al., 2002; 19% in Godley et al., 2006; 40% in Soberman et al. 2002). However, the issue of non-respondents was not addressed in four studies (Coleman et al., 1992; Friedman et al., 2002; Morehouse & Tobler, 2000; Raider et al., 2002).

Bias was also introduced in several studies by using subjective means of recruiting youth into the study. For instance, Raider et al. (2008) reported that youth “with documented multiple trauma exposure” were identified by staff for recruitment into the study. Identification appeared to rely on clinical judgment, but the process was not further explicated. Similarly, in Coleman et al.’s study of ART^® (1992) staff were asked to select clients for participation based on age criteria, displaying of aggressive behavior and judgment about potential benefits of the program for the participating youth. In several studies, recruitment, consent and/or exclusion procedures remained vague or were not addressed (Friedman et al., 2002, Morehouse & Tobler, 2000; Raider et al., 2002).

Sample sizes varied widely from 14 (Wasser et al., 2008) to several hundred (Nugent et al., 1998). In the case of Raider et al.’s study, sample size was inconsistently reported and there was lack of clarity on the actual versus analytic sample. Several studies addressed the statistical power of the sample (Friedman et al., 2002; Godley et al., 2002, 2006; Slesnick and Prestopnik (2009); Soberman et al., 2002; Wasser et al. 2008.

3.2.1.3. Adequacy of control/comparison group

Nine studies used a control or comparison group (Coleman et al., 1992; Friedman et al., 2002; Godley et al., 2002, 2006; Morehouse & Tobler, 2000; Nugent et al., 1998; Raider et al., 2002; Slesnick & Prestopnik, 2009; Soberman et al., 2002; Wasser et al., 2008). Two studies utilized sophisticated randomization procedures (randomized block design – Godley et al., 2002, 2006; urn randomization – Slesnick & Prestopnik, 2009) in order to decrease the risk of baseline variability and thereby selection bias. Raider et al. (2008) used a wait-list control. The remaining experimental studies reported randomizing youth to experimental and control group but did not provide specific information about the randomization process. In five studies, the control group involved treatment as usual (Coleman et al., 1992; Friedman et al., 2002; Godley et al., 2002, 2006; Slesnick & Prestopnik, 2009; Soberman et al., 2002). The three quasi-experimental studies included a matched comparison group (Wasser et al., 2008), a comparison group derived through a time-series design (Nugent et al., 1998), and lastly, a non-equivalent comparison group (Morehouse & Tobler, 2000), which was of limited usefulness in increasing the internal validity of the study. Six studies reported that experimental and control/comparison group were comparable at baseline, and if differences were found they were addressed in subsequent analyses (e.g., Friedman et al., 2002). Raider et al. (2008) failed to address similarities and/or differences, and Coleman et al. (1992) stated that they controlled for baseline differences in the analysis, but the analysis provided no clarity about this issue.

3.2.1.4. Quality of measurements and outcomes

Outcomes of interest in the reviewed studies included rates of substance abuse, aggression, trauma, and/or general mental health problems, school problems, service use and permanency outcomes, such as continuing care, premature discharges, days in inpatient psychiatric care, placement stability, and post-discharge living arrangements. All outcomes are relevant to both RCS population and settings. All but one study relied on standardized measures with adequate psychometric properties or relatively objective indicators, such as number of placements (Koob & Love, 2010), number of inpatient days (Sunseri, 2004) or post-discharge living arrangements (The Grove, 2004). Less objective were Nugent’s target outcome of daily rate and number of antisocial behavior and Sunseri’s target outcome of premature discharges due to suicidality. Two studies in particular had strong protocols, paying attention to cross-validation of measures and/or interrater reliability (Godley et al., 2002, 2006; Slesnick and Prestopnik’s, 2009). Blindness to condition occurred in Coleman et al.’s study of ART^® (1992) and Soberman et al.’s study of EMDR (2002). Godley et al. (2006) reported that blinding to condition was not possible to implement, but that data were collected by research staff with no connection to the intervention.

3.2.1.5. Completeness

In this category, studies are appraised in the areas of attrition, compliance with protocol/treatment, i.e. intention to treat analyses, and missing data. While missing data was not explicitly addressed in any of the studies, attrition rates were reported in most studies and were as follows: 8% across all three follow-up interview and all outcomes (Godley et al., 2006); 16% in Friedman et al.’s study (2002) between post-assessment at the time of discharge and the follow-up assessment in the community; 17% in Morehouse and Tobler’s study (2000) between pre- and post-test; 21% in Soberman et al.’s study (2002); 25% in Coleman et al.’s study (1992); and 27% by 15 months in Slesnick and Prestopnik’s study (2009) with 63% of youth completing all assessments. Raider et al.’s paper contained inconsistencies about the number of dropouts (2008). Intent-to-treat and treat-only analyses were addressed in Godley et al. (2002, 2006) and Slesnick and Prestopnik’s studies (2009).

3.2.2. Overall study findings

Within the context of the methodological limitations addressed above, studies determined positive effects for the respective interventions with regard to most target outcomes (see Table 2 for specific findings, statistical tests, p-values, and effect sizes when provided). Some of the effects were considerable (effect sizes in the medium range), in particular in Godley et al.’s study of the A-CRA intervention (2002, 2006). In other cases, there were only partial gains. For instance, Coleman et al. (1992) reported that ART^® improved social skills, but not behavior. Similarly, Friedman et al. (2002) found decreases in drug use and selling of drugs following MSAP, but not in the total degree of illegal activities, violent behavior or school problems. Wasser et al. (2008) reported improvement in youth receiving DBT versus Standard Therapeutic Milieu treatment, but longer stays in RCS for DBT clients. Morehouse and Tobler (2000) reported variability in improvement for the treatment group depending on dosage received. Slesnick and Prestopnik (2009) found EBFT to be more effective than FFT in reducing substance abuse problems, but both treatments were more effective than services as usual provided through the shelter.

However, given weaknesses in many of the designs, biases introduced through the sampling procedures, and, in a few cases, simple lack of clarity about aspects of the study, findings about some of the interventions have to be regarded with great caution.

3.3. What implementation issues arise when transporting EBPs into RCS?

The final question examined whether studies addressed aspects of treatment implementation given that not all of the interventions were specifically designed for delivery in RCS. Implementation issues can significantly confound study findings. Attention was paid to intervention adaptations, training and supervision, fidelity ratings, and barriers to implementation (see Table 3).

Table 3.

Implementation Considerations of Evidence-Based Treatments with Residential Care Populations

Intervention/Study	Manual/ Intervention Adaptations	Training/Supervision	Treatment Fidelity Ratings	Implementation Barriers and Strategies
Adolescent Community Reinforcement Approach (A-CRA)
Godley et al. (2002, 2006)	CMs provided ACC following 2 manuals: (1) CM and home-based approach; (2) community reinforcement adapted for adolescents	Case managers were trained in procedures documented in manuals Supervised by one of the authors of the tx manual Case management sessions with the adolescents were monitored via audiotape review or direct observation and given corrective feedback by the supervisors	At 3-months post-discharge, youth were asked if they received services specified in ACC protocol Presents fidelity results
Aggression Replacement Training (ART®)
Coleman et al. (1992)	No significant departure from Art curriculum, but attempt to implement ART in 20 wks with a 2nd tx group to ensure “over-learning” (p.56). Authors thought 10 wks was too short for meaningful change in behavior and training of empathy	Cross-section of Devereux staff was selected by RTC administration to implement the program 40 staff members (teachers, social workers, child care workers, supervisors and administrators) received 3-day training workshop. Each member learned all 3 tx components. Workshop was taught by university professor with extensive experience with the ART curriculum and in working with cts and staff in RTCs. In general staff meetings ART was explained to all staff. Staff were expected to reinforce learned skills. Target “skill of the week” was posted in various locations throughout Center for staff and cts; Special events to help maintain interest and morale	Observed by 1st author on 2 occasions with corrective feedback given Daily logs by group leaders to record/assess absences, group management, participation, etc. Provides data (percentages) on fidelity	20-wk adaptation could not be implemented due to natural ct attrition, boredom, personal factors, and programmatic disruptions (Authors surmised that greater flexibility in adapting curriculum may have prevented attrition) Lack of continuity of group leaders due to organizational variables (e.g., overtime, shift changes) Difficulties of homework completion among some cts Inconsistency in ct practice and reinforcement of skills outside group Group leaders saw ART as an “add-on,” not an integral part of tx.
Nugent et al. (1999)	“The traditional ART curriculum was condensed into a 15-day program that was delivered…over a 21-day time period” (p.640). Curriculum included full anger-control and partial social skills training but not moral reasoning since “participating adolescents were unlikely to have the requisite mix of moral reasoning levels” (p.641). Ch. 3 and Ch.5 of Goldstein & Glick (1987) used as tx manual.	All shelter staff were taught to conduct ART groups shortly before the program was implemented Staff was taught by senior author in conducting ART groups.		Concluded that condensed version was effective and therefore appropriate for implementation in short-term residential setting
Dialectical Behavior Therapy (DBT)
Sunseri (2004)	Used DBT skills manual Full DBT program, no apparent adaptations Display of DBT skills on post boards in RTC milieu to aid in reinforcing of learned skills by direct care staff	2-day training followed by formation of a study group, which reviewed tx manual and read primary texts related about DBT Formation of a tx team, which attended 2-week intensive training Direct care staff need to have a reasonable working knowledge of DBT but no specialized training Coaching through consultation team and periodic review of videotapes	“To ensure tx fidelity, all individual therapy sessions are videotaped and then periodically reviewed by a weekly consultation team of agency therapists. The mission of the team members is to keep the primary therapist on track with DBT, and to offer advice, suggestions and support” (p.65).	Mentions symptom contagion as a possible confounding factor when implementing DBT in RTC Shift in admission policy between pre/post period Increased focus on client commitment to tx “…we have found DBT to be a much warmer, supportive, and therapeutic approach.. We have become less dependent on consequences to change behavior, and as staff members we find ourselves working much more collaboratively with the client sand their families
The Grove… (2004)	Each DBT round lasted 6 months - Youth may go through 2 or 3 rounds of DBT Youth are expected to lead aspects of skills groups as they progress through the program; linked to increased privileges and peer mentorship Point and level system is based on DBT principles Modification of Linehan’s behavior chain to a shorter mini-chain analysis 2 hr monthly group for parents
Wasser et al. (2008)	Not all DBT tx components were offered; skills training group sessions only Parents were involved in group sessions, but for a limited time period Not all tx sessions were conducted by a DBT-trained clinician DBT terminology was simplified Pace of training was slowed Order of the modules was rearranged to meet needs of target population	Two of six clinicians were trained in DBT 2 full days of on-site training with additional external training Training of child care staff to reinforce learned skills	“The exact exposure of DBT techniques could not be objectively or accurately quantified nor fidelity to DBT assessed for this cohort” (p.117)
Ecologically-Based Family Therapy (EBFT)
Slesnick & Prestopnik (2009)	Followed EBFT and FFT manuals; FFT only office-based in this study	PI conducted 2-day didactic training on EFBT and FFT with experienced family therapists (2 therapists conducted 72% of sessions) Ongoing supervision in university setting Weekly meetings between supervisor and therapists	Audiotape recordings Review of portions of audiotape during weekly meetings
Eye Movement Desensitization and Reprocessing (EMDR)
Soberman et al. (2002)	Followed recommended procedures, which includes adaptations in tx delivery to accommodate children and adolescents No indication of tx adaptations specifically related to RTC	One experienced therapist with extensive EMDR training; completed EMDR training (level 1 and 2) and had administered about 100 EMDR sessions	“No independent assessment of fidelity was available” (p.227) Experience of therapist argues for fidelity, according to authors.	Authors state that EMDR may be difficult to perform even after “standard training”
Functional Family Therapy (FFT)
Slesnick & Prestopnik (2009)	see EBFT	see EBFT	see EBFT	see EBFT
Multimodal Substance Abuse Prevention (MSAP)
Friedman et al. (2002)	Authors mention manuals for each program component LST program (which was 1 component of tx) was adapted to be conducted in 20 sessions during a 4 wk period AV program “was reorganized and intensified…to be conducted in 20 sessions of 55 min each in order to fit into the …regular school schedule.” Developed manual for 3rd component (Value Clarification)			Discusses variability of receptivity and effectiveness of various program components, e.g., “The Values Clarification modality did not appear to engage the participants sufficiently, or to hold their interest. It may be that the ideas presented were too abstract. Also, the lack of relevant exercises to practice and learn may explain the ineffectiveness of this modality as well.’ (p.61)
Residential Student Assistance Program (RSAP)
Morehouse & Tobler (2000)	RSAP, like original program Westchester County Public School’s Student Assistance Program modeled after Employee Assistance Program No mentioning of a manual or adaptations	Community-based prevention agency trains and supervises student assistance counselors, which are then placed at each site. Program provides training and consultation to each site, develops and leads a Staff Task Force to empower RTC staff to promote the program and refer residents to it. Administers and supervises the program in partnership with each RTC. Provides programmatic and clinical supervision and training.	RTC liaison meets weekly with the student assistance counselors to discuss program implementation and address potential barriers.	Authors address “uneven implementation” “Some youth failed to received substantial portions of the intervention” Importance of access to youth by counselors, attractive and suitable meeting place for sessions, dependable transportation for off-site support meetings, scheduling – these factors varied across settings Budgetary constraints disrupted program implementation at one site, led to d/c of 50% of youth Instability and uncertainty related to placement issues affected ‘ability to trust’ Addresses staff turnover and institutional climate as factors impacting implementation and subsequently outcome Staff continuity and leadership were crucial at two most successful sites Implementation without parent involvement possible
Solution Focused Brief Therapy(SFBT)
Koob & Love (2010)		1 wk training of staff by Insoo Kim Berg, co-developer of SFBT Continued consultation by phone and email F/up 3-day training after 3 mths and after 6 mths Training and consultation period lasted 1 yr	F/up visits were in part intended to “assess treatment drift”
Structured Sensory Therapy for Traumatized Adjudicated Adolescents in Residential Treatment (SITCAP-ART)
Raider et al. (2008)	SITCAP-ART is a modification of Structured Sensory Intervention for Traumatized Children, Adolescents and Parents (SITCAP; Jacobs & Steele, 2003) Structured protocol with session-by-session, situation-specific guide to intervention	Therapist from behavioral health department trained and certified in SITCAP-ART by the institute where the model was developed	Fidelity of Treatment Checklist Analysis indicated 98.5% fidelity

3.3.1. Intervention adaptations

Studies addressed implementation issues with considerable variability. Several studies reported adaptations to accommodate delivery of the intervention in the RCS. Adaptations ranged from condensing the time in which the intervention was delivered (ART^® - Nugent et al., 1998; MSAP– Friedman et al., 2002) to extending it (ART^® - Coleman et al., 1992; DBT – The Grove, 2004) or offering only a portion of the treatment (ART^® - Nugent et al., 1998; DBT - Wasser et al., 2008). Nugent et al’s study of ART^® in a short-term RCS, for instance, noted that the moral reasoning module was not included as youth lacked the “requisite mix of moral reasoning levels” (p.641). Thus, adaptations were made to accommodate the short-term nature of the setting as well as the perceived characteristics or ability of the treatment population. Wasser et al. discussed simplifying DBT terminology, rearranging the sequence of the module and slowing the pace of the training to accommodate the needs of the target population. An effort to test an extended version of ART^® had to be abandoned due to multiple barriers encountered in the RCS (Coleman et al., 1992). Authors state that the 20-week version of ART^® (twice the length of the normal course of treatment) was unsuccessful due to youth leaving the program, boredom with the program and programmatic disruptions. Two studies reported that the intervention was implemented as intended by the developer (EMDR –Soberman et al., 2002; DBT - The Grove, 2004). Godley et al. (2002, 2006; A-CRA), Slesnick and Prestopnick (2009; EBFT and FFT) and Raider et al. (2008; SITCAP-ART) did not address adaptations but named the manuals used in the study. Two studies named neither a manual nor addressed adaptations (Morehouse & Tobler, 2000 - RSAP; Koob & Love, 2010 - SFBT).

3.3.2. Training and supervision

All but two studies addressed training in the intervention and/or subsequent supervision and monitoring (The Grove, 2004 - DBT; Friedman et al., 2002 - MSAP). Training and supervision are particularly relevant to RCS as these facilities are affected by shift changes, and a workforce, which is multidisciplinary. Furthermore, child care workers who interact with youth on a daily basis often have limited training and high turnover rates (Colton & Roberts, 2007). Several training models emerged from the 12 studies. (1) Four studies described using expert trainers to provide initial training to either all staff or some staff with subsequent delivery of the intervention by all or partial RCS staff (Godley et al., 2002, 2006 - A-CRA; Coleman et al., 1992; Nugent et al., 1998 - ART^®; Koob & Love, 2010 - SFBT). In the case of SFBT, the co-developer of the intervention was contracted to train and supervise staff over a one-year period. (2) In two studies, select staff attended external workshops and trainings. For instance, in Sunseri’s DBT study (2004), an initial phase of learning about the intervention by a small study group was followed by the formation of a treatment team, which subsequently attended the 2-week intensive DBT training. In Wasser et al’s study (2008), the facility opted to train two of six clinicians through a 2-day onsite DBT training augmented by external workshops. In both studies, direct care staff were oriented to the intervention but did not receive specialized training. (3) A third model involved the delivery of the intervention by trained and certified experts or therapists (Soberman et al., 2002 - EMDR; Slesnick & Prestopnik, 2009 - EBFT and FFT; Raider et al., 2008 - SITCAP-ART). (4) Finally, training, administration and supervision of an intervention can occur through an external liaison organization, as was the case in the delivery of the RSAP intervention (Morehouse & Tobler, 2000).

3.3.3. Treatment fidelity ratings

Seven studies addressed treatment fidelity, but there was a great range in the methods used and the rigor to fidelity checks. Methods included youth feedback about services received (Godley et al., 2002, 2006 - A-CRA), expert observation and feedback (ART^®42), daily group leader logs to record treatment components and processes (Coleman et al., 1992 - ART^®), videotaping and periodic review of sessions (Slesnick & Prestopnik, 2009 - EBFT and FFT; Sunseri, 2004 - DBT), weekly meetings between experts and counselors (Morehouse & Tobler, 2000 - RSAP), follow-up visits by co-developer of intervention over the course of a year (Koob & Love, 2010 - SFBT), and use of a treatment fidelity checklist (Raider et al., 2008 - SITCAP-ART). Two additional studies noted lack of objective measures of fidelity (Soberman et al., 2002 – EMDR; Wasser et al., 2008 - DBT).

3.3.4. Implementation barriers

Except for Slesnick and Prestopnik’s study (2009), which implemented two evidence-based interventions outside the short-term RCS setting while youth were still residing in the shelter, EBPs were implemented in the RCS. Several studies addressed implementation barriers that were encountered in the delivery of the intervention. Four themes were identified. (1) Client receptivity: Both Coleman et al. (1992 - ART) and Friedman et al. (2002 - MSAP) note diminished client interest as one barrier to successful implementation. Coleman described boredom as a reaction to an extended version of ART^®. Friedman et al. reported that the Values Clarification Module did not seem to interest youth. Authors surmised that presented ideas and examples may have lacked relevance and may have been too abstract to engage youth. (2) Staff receptivity: Coleman et al. noted difficulties in the reinforcement of learned skills outside of group sessions, which they attributed in part to a perception of ART^® as being simply an ‘add-on’ treatment rather than being an integral part of the entire treatment approach. In contrast, Sunseri (2004) found the implementation of DBT to positively affect the overall treatment climate, describing it as a paradigmatic shift toward a warmer and more supportive therapeutic approach. Morehouse and Tobler (2000 - RSAP) discussed institutional climate as a factor that either hindered or facilitated successful treatment implementation. (3) Treatment factors: Several factors related to treatment within the context of RCS were discussed in two studies. Morehouse and Tobler (2000 - RSAP) note that instability and uncertainty related to placement issues affected youths’ ability to trust. The same study also emphasized the suitability of RSAP for youth in RCS as substance abuse issues were successfully addressed even without parent involvement and support. Finally, Sunseri (2004) addressed group contagion as a factor that may confound implementation of DBT in RCS. (4) Structural/organizational barriers: Coleman et al. (1992 - ART) address lack of continuity in group leadership due to shift work and overtime regulations as a factor that negatively affected implementation. Morehouse and Tobler’s study of RSAP cite “uneven implementation” as a factor explaining variability in outcome across different RCS. Youth in different settings did not receive the same “dosage,” due to lack of support in providing dependable transportation for off-site meetings, scheduling factors, and lack of suitable meeting rooms. Disruption of treatment due to budgetary constraints and staff turnover and leadership are also noted as barriers to implementation of an EBP in RCS.

Across all studies, the discussion of barriers to implementation remained at a conceptual level. While Morehouse and Tobler (2000 - RSAP) measured the effect of intervention “dosage” on outcome, their conclusions about the causes of this variability (e.g., lack of institutional support, staff turnover) were not based on data.

4. Discussion

This review, covering a 20-year period, demonstrates that EBPs can be implemented and tested, even with experimental designs, within the context of RCS. Taken alongside knowledge about the evidence of system-wide RCS models, such as the Teaching Family Model (James, 2011) or Phoenix Academy (Morral, McCaffrey & Ridgeway, 2004), etc., identified studies constitute a small but important body of knowledge that may encourage RCS facilities to adopt and implement EBPs to meet the needs of their target population. At the same time, the considerable range in quality of the reviewed studies demonstrates the scientific neglect of an area of practice that among intensive treatment options is still the most utilized alternative for youth with severe emotional and behavioral disorders (James et al., 2006).

An analysis of the types of treatments identified in this review indicates that several treatments were behaviorally oriented (ART, DBT, FFT, MSAP, SITCAP-ART) and/or had a trauma focus (EMDR, SITCAP-ART), but in general there was a diversity of theoretical approaches and treatment methods. The combined focus of these interventions can be described as addressing severe emotional and behavioral disorders, substance abuse problems and facilitating service continuity beyond the RCS stay.

Three interventions (DBT, EBFT, FFT) had an explicit parent component. In Slesnick and Prestopnik’s study of EBFT and FFT (2009), wards of the state were excluded as active family involvement was a requirement for participation in the study. In the reviewed DBT studies the parent component was implemented with considerable variability. All other interventions did not require parent involvement, which in RCS may be an advantage given that engagement of parents of this population has been identified as a challenge (Barth, 2005) and is often an exclusion criterion for participation in EBPs. Youth in RCS frequently have lengthy placement histories, which imply long absences from their families of origin. These absences along with histories of abuse and parental dysfunction present significant barriers in the engagement of parents. While it is recognized that RCS need to do more to foster parental engagement, some barriers to parental engagement may be difficult to overcome, and treatments need to be made available that either do not require parental engagement or are adapted to be delivered without a parent component to ensure that all youth can receive the best treatments available regardless of their parents’ ability or willingness to get involved in the treatment process. Morehouse and Tobler (2000) noted the success of RSAP without parent involvement as a strength of their intervention.

Most outcome studies in this review were conducted within one RCS, limiting the generalizability of the effects of a particular intervention. However, given the heterogeneity of RCS along multiple dimensions, conducting outcome studies in one setting or a few very similar settings is appropriate unless the variability of organizational characteristics across RCS can be measured and their effects on implementation and outcome determined (Helgerson, Martiovich, Durkin, & Lyons, 2005).

Currently, few models exist for the implementation of EBPs into RCS (Bright, Raghavan, Kliethermes, Juedemann, & Dunn, 2010). The review pointed to many complexities in the implementation process that have to be considered when transporting EBPs in RCS, ranging from general receptivity among staff and clients, treatment factors and organizational/structural barriers. Some of the key challenges in RCS, such as lack of professional training among child care staff, shift work, high turnover, and organizational climate were identified as barriers to successful implementation of the EBPs reviewed here. In addition, several conceptual articles written about the implementation of interventions in RCS point to the need for organizational ‘buy-in,’ collaborative implementation teams as well as the crucial role of opinion leaders to introduce an EBP into the treatment milieu (Bright et al, 2010; Little, Butler, & Fowler, 2010; Lovelle, 2005; Stewart & Bramson, 2000). Interventions that are carefully planned and are gradually phased in, as described by Sunseri (2004) may have more success in gaining acceptance and buy-in among staff. Reimbursement policies, budgetary constraints and funding streams may also undermine the implementation of EBPs (Bright et al., 2010; Morehouse & Tobler, 2000). Despite formidable barriers, RCS with their treatment milieu and embedded child care staff are seen by some as an ideal resource and opportunity to teach and reinforce new interventions (Lovelle, 2005; Wasser et al., 2008). However, if an EBP is implemented by external staff, orientation of child care staff and internal opinion leaders to the treatment model becomes particularly essential to support successful implementation of the intervention (Little et al., 2010; Lovelle, 2005).

Overall outcomes were encouraging and indicated improvement in multiple domains of functioning. However, many questions remain about methodological bias and factors impacting the internal validity of the studies. Nonetheless, findings show that randomization is possible even within complex treatment settings, which should encourage more rigorous designs in the future.

The aims of this review necessitated a search process that was not linear, but emergent. Given the lack of a complete sampling frame of psychosocial interventions for children and adolescents, it is possible that an intervention and its respective studies may have been overlooked despite efforts to be thorough and systematic. This review discovered only a few interventions that met a priori criteria. However, during the review process a number of interventions were identified that, at least from a theoretical standpoint, would seem ideal for the treatment needs of youth in RCS (e.g., Trauma-Focused CBT, Girls Circle). Also notable was the preponderance of interventions that have been developed and tested with juvenile delinquents. While reflective of the serious needs of this population, the sheer number of interventions that have been tested in correctional facilities raises ethical questions about studies being conducted on youth while they are a “captive audience” (Morehouse & Tobler, 2000).

Finally, many RCS may already be implementing EBPs, but without carefully conducted outcome studies and dissemination of findings knowledge about barriers to the adoption and implementation of EBPs in RCS will remain limited. Collaborative alliances between RCS providers and researchers are encouraged to build this area of knowledge.

Highlights.

• This review examined effectiveness and implementation of EBPs into RCS

• 10 EBPs and respective outcome studies were identified in a multi-phase search

• Outcomes were generally positive across a range of psychosocial outcomes

• Concerns about methodological bias and confounding factors remain

• Studies pointed to multiple barriers in the implementation process