A Positive Youth Development Approach to Improving Mental Health Outcomes for Maltreated Children in Foster Care: Replication and Extension of an RCT of the Fostering Healthy Futures Program

Abstract

Preventing the negative impact of maltreatment on children’s mental health requires interventions to be contextually-sensitive, grounded in theory and research, and effective in reaching and retaining children and families. This study replicates and extends previous findings of the Fostering Healthy Futures (FHF) program, a 30-week mentoring and skills group intervention for preadolescent maltreated children in foster care. Participants included 426 children recently placed in out-of-home care who were randomized to intervention or control conditions. Outcomes measured 6–10 months post-intervention included a multi-informant (child, caregiver, teacher) index of mental health problems as well as measures of posttraumatic stress symptoms, dissociative symptoms, quality of life, and use of mental health services and psychotropic medications. There were high rates of program initiation, retention, and engagement; 95% of those randomized to FHF started the program, 92% completed it, and over 85% of the mentoring visits and skills groups were attended. The FHF program demonstrated significant impact in reducing mental health symptomatology, especially trauma symptoms, and mental health service utilization. These program effects were consistent across almost all subgroups, suggesting that FHF confers benefit for diverse children. Results indicate that positive youth development programming is highly acceptable to children and families and that it can positively impact trauma and its sequelae.

Introduction

In 2017, the percent of children in foster care in the U.S. increased in 39 states and the number of children in foster care increased for the fifth consecutive year to 443,000. Almost six of every 1,000 U.S. children were in foster care in 2017 (Williams & Sepulveda, 2019). Nearly half of these children live in non-relative foster homes, 30% in relative foster homes (referred to as “kinship care”), 8% in institutions, 6% in group homes, and about 5% in other placements (e.g., pre-adoptive homes) or have run away (Child Welfare Information Gateway, 2017). Many youth experience multiple placements while in care, and some move in and out of the system throughout their childhood. Youth in foster care have typically experienced substantial traumatic adversity (e.g., abuse and neglect, exposure to substance use and violence, chronic disruptions in school and living situations, abandonment). These adverse childhood experiences (ACEs) can cascade into other negative outcomes including poor physical and mental health, academic underachievement and school dropout, problematic substance use, poverty and homelessness, and incarceration (Felitti et al., 1998; Weiler, Garrido, & Taussig, 2016).

The Family First Prevention Services Act was signed into law as part of the Bipartisan Budget Act on February 9, 2018. In part, this Act calls for the use of evidence-based, trauma-informed services with youth involved in the child welfare system. This may be in response to prior research which suggests that the mental health needs of these children may not be adequately met. The National Survey of Child and Adolescent Well-Being (NSCAW) is a nationally representative, longitudinal survey of children and families who have been the subject of an investigation by Child Protective Services. The NSCAW study, which examines child mental health functioning as well as service utilization, found that over half the youth in foster care needed mental health services yet only 26% were receiving services (Bellamy, Gopalan, & Traube, 2010). Children in foster care who received mental health services did not evidence better outcomes; in fact, they scored worse on mental health problems than children who did not receive such services (McCrae, Barth, & Guo, 2010). The NSCAW data suggested that mental health treatment for children in long-term foster care conferred no benefit and concluded that youth were receiving “untested treatments with questionable effectiveness” (Bellamy et al., 2010, p. 474).

While referrals to evidence-based programs (EBPs) may be increasing for families involved in the child welfare system, many of these programs were not developed or tested in real-world contexts, thereby reducing their ability to engage and retain children and families who are facing challenges across multiple systems (e.g., child welfare, mental health, criminal justice, housing; Hambrick, Oppenheim-Weller, N’zi, & Taussig, 2016). A systematic review of interventions for children in foster care delineated the multiple challenges in providing effective mental health services for children in care. These include the high number of placement changes, the diverse living settings in which children in out-of-home care reside (e.g., non-relative foster care, kinship care, residential treatment), transportation and time constraints on the part of caregivers, and acute and severe behavioral health needs, all of which can lead to poor engagement and a lack of continuity of mental health services (Hambrick et al., 2016; Taussig & Raviv, 2014). For these reasons, many well-known EBPs aimed at reducing trauma have not been rigorously tested in foster care populations, and their ability to engage and retain youth and their caregivers in is unknown. Stigma and logistical constraints may be other reasons for poor engagement in mental health treatment for youth in foster care (Ganser et al., 2017). After experiencing significant trauma and being removed from their homes, many youth are subsequently identified as “emotionally disturbed,” “delayed,” and/or “beyond control of their parents.” They are restricted from engaging in developmentally-appropriate activities (such as sleepovers and learning to drive) and spend much of their out-of-school time in appointments and visitation.

Not only do we need more contextually-sensitive and acceptable programs that engage and retain children and families, but we also need to develop programs that foster positive youth development and well-being. Positive youth development (PYD) programming is grounded in the assumption that children have strengths and resources that can be fostered and that the deficit-based perspective (that youth have problems which need to be fixed) should be dismissed (Bonell et al., 2016; Catalano, Berglund, Ryan, Lonczak, & Hawkins, 2002; Lerner et al., 2005). For children in foster care, this PYD approach may be more accessible, acceptable and desirable, as the traditional focus tends to be on ameliorating immediate problems as opposed to fostering long-term health and well-being (Bonell et al., 2016). To promote PYD, Lerner (2004) suggests that programs include three critical components: (1) a positive, sustained relationship with a mentor, (2) activities for building life skills, and (3) opportunities to use life skills in meaningful community activities. Mentoring is a desirable PYD-promoting experience for youth in foster care. In qualitative studies, youth in foster care report wanting and experiencing life-changing informational, instrumental, and emotional support from mentors, and, when combined with other components (e.g., youth skills groups, parenting groups), mentoring-based interventions for children in foster care have demonstrated efficacy (Taussig & Weiler, 2017). Yet, limited empirical research signals a critical need to rigorously subject such interventions to randomized controlled trials (RCTs).

Fostering Healthy Futures (FHF) is a preventive PYD intervention for preadolescent children who have experienced maltreatment and placement in out-of-home care. It consists of individualized mentoring and weekly skills groups for 30 weeks. The conceptual and theoretical basis for FHF and anticipated proximal and distal effects are delineated elsewhere (cf. Taussig, Culhane & Hettleman, 2007). A pilot RCT with 156 youth found positive impacts on multi-informant (child, caregiver, teacher) reports of mental health functioning (including trauma symptoms), mental health services, and quality of life (Taussig & Culhane, 2010). The trial’s efficacy was found to be comparable to other widely known EBP trauma treatments (Goldman Fraser et al., 2013) and FHF was subsequently listed on several well-known registries of promising or evidence-based practices (e.g., Washington State Institute for Public Policy, California Evidence-Based Clearinghouse for Child Welfare).

As dissemination and implementation of FHF began, there were questions about for whom the program might be most efficacious and whether there were contraindications to consider. Moderator and subgroups analyses can help determine who is likely to benefit (Supplee, Kelly, MacKinnon, & Barofsky, 2013) and such research can also reveal compensatory (i.e., higher-risk groups tend to benefit more) or leveraging (i.e., lower-risk groups tend to benefit more) program effects (Spoth, Shin, Guyll, Redmond, & Azevedo, 2006). Although children in foster care are, collectively, a high-risk group, children’s demographic characteristics, placement settings, maltreatment histories, degree of exposure to ACEs, and mental and physical health needs vary considerably. For instance, some children in out-of-home care live in congregate care settings, while others are placed in kinship care. Some children may have experienced sexual or physical abuse, while others may have experienced chronic neglect. Finally, some children may have been exposed to multiple additional adverse experiences (e.g., witnessed violence, many changes in homes/schools) while others may have experienced fewer. Each of these factors, in addition to demographic differences, may influence the efficacy of the program. The heterogeneity of this population underscores the need for research that examines moderators of intervention efficacy.

In the FHF pilot study we found that (1) severity of neglect did not moderate outcomes, suggesting that children with severe neglect histories were just as likely to benefit from the intervention as those with less severe neglect histories (Taussig, Culhane, Garrido, Knudtson, & Petrenko, 2013) but that (2) level of risk (i.e., exposure to ACEs) did moderate some outcomes (i.e., posttraumatic stress and dissociative symptoms), such that maltreated children exposed to a relatively higher number of ACEs did not experience the same reduction in trauma symptoms as children exposed to fewer adversities (Weiler & Taussig, 2017). Notably, exposure to ACEs did not moderate other mental health and psychosocial outcomes (including coping skills, social acceptance, self-esteem), suggesting similar benefits were derived for participants across levels of risk. In surveying the literature on preventive interventions for youth in foster care and those for the general population, findings vary. For instance, some programs are more impactful for children with baseline behavior problems (e.g., Chamberlain, Price, Leve, Laurent, Landsverk, & Reid, 2008; Sandler et al., 2003) and others provide greater benefit to children exposed to fewer adversities (e.g., Lochman, Wells, Qu, & Chen, 2013). Due to the diversity in program type, target population, and targeted outcomes, individual interventions need to evaluate moderators based on their program’s and population’s characteristics.

Current Study

The current study sought to replicate and extend the mental health findings of the pilot FHF trial by adding data from five additional cohorts (N=270) as well as examine moderating effects. The study first examined FHF’s effects on key mental health outcomes: a multi-informant (child, caregiver, teacher) index of mental health problems, youth-reported posttraumatic stress symptoms (including dissociation), youth-reported quality of life, and caregiver- and youth-reported use of mental health treatment (including psychotropic medications). Analyses then examined whether the following baseline variables moderated the impact of FHF on outcomes: gender, race/ethnicity, type of placement (foster vs kinship care), intellectual functioning, ACEs, and mental health functioning. There were no a priori hypotheses about the direction of the potential moderating effects with the exception of the ACEs measure; based on our prior research, we hypothesized that those children with fewer ACEs would demonstrate greater program effects on trauma symptomatology.

Method

Participants

The study began in August 2002 in Denver, Colorado and expanded to four metro-area counties in 2007. Participants were recruited in 10 cohorts over 10 consecutive summers (the first five cohorts comprised the “pilot trial” and the second five, the “efficacy trial”) from a list of all children, aged 9–11, who were placed in foster care in participating counties. Children were recruited if they: (1) had been placed in any type of out-of-home care (e.g., foster care, kinship care, residential treatment) by court order due to maltreatment within the preceding year, (2) resided, at the time of recruitment, in out-of-home care within a 35-minute drive to skills groups sites, (3) had lived with their current caregiver for at least 3 weeks, (4) were not developmentally delayed and (5) demonstrated adequate proficiency in English (although their caregivers could be monolingual Spanish speaking). In the pilot trial, when multiple members of a sibling group were eligible, one sibling was randomly selected to participate in the RCT. In the efficacy trial, eligible siblings were paired for randomization and both were included in the trial (there were 22 sibling pairs included in the study). Letters introducing the study were sent to families, followed by recruitment calls a week later. Participation was voluntary and could not be court-ordered.

As the CONSORT diagram in Figure 1 shows, 90.1% percent of eligible children and their caregivers agreed to participate. After the baseline interview and prior to randomization, 16.6% of the participants were deemed ineligible for the following reasons: 38 were developmentally delayed, 33 were no longer in out-of-home care, 12 had information on their child welfare records (obtained post-interview) that made them ineligible (e.g. incorrect birthdate), and 2 were not proficient enough in English to participate in the skills groups. Of the remaining 426 youth who were randomized to treatment and control groups, 89.4% were retained at the 6-month follow-up research interviews. Children’s teachers were surveyed 10 months post-program, and 84.5% of children’s teachers completed the surveys. Of those randomized to the intervention, 95.3% of children started the intervention and of those, 92.3% completed the 30-week program.

Figure 1.

FHF Consolidated Standards of Reporting Trials (CONSORT) diagram

As shown in Table 1, the sample had good gender and racial/ethnic distribution. Half of the youth self-identified as Hispanic/Latinx, half as Caucasian, and over a quarter as Black/African American (racial/ethnic categories were not mutually-exclusive). Average IQ was 4.5 points lower than the standardization sample’s mean. According to child welfare records, two-thirds of the children’s biological mothers had a substance use history, 60% had a history of criminal activity, and 43% had a history of mental illness. Eleven percent of the children had documented sexual abuse, over a quarter had a history of physical abuse, almost two-thirds had experienced documented emotional abuse, and almost all had experienced some type of neglect. Children’s families had an average of 4.6 referrals to social services before being removed from their homes. Over three-quarters of the youth had been in therapy and one-fifth had been on psychotropic medication. At the time of the baseline interview, 42% were living in non-relative foster care, 54% were placed with kinship care providers and the remaining 4% were in some type of congregate care (i.e., group homes or residential treatment).

Table 1.

Baseline Characteristics

	Control n=193	Intervention n=233	Total N=426
Child Characteristics
Age, Mean (SD)	10.25 (.90)	10.31 (.90)	10.28 (.90)
Male, %	52.8	51.1	51.9
Hispanic, %	49.2	53.5	51.5
African American, %	25.4	31.0	28.4
Caucasian, %	49.7	51.4	50.6
IQ, Mean (SD)	94.59 (11.67)	96.42 (13.0)	95.59 (12.44)
ACEs, Mean (SD)	1.72 (1.12)	1.63 (1.04)	1.67 (1.08)
Maternal Characteristics
Substance use, %	66.5	65.7	66.0
Criminal history, %	55.4	65.2	60.3*
Mental illness, %	41.1	44.2	42.8
Maltreatment history, %	19.3	18.3	18.7
Child Welfare Characteristics
Physical abuse, %	23.8	29.6	27.0
Sexual abuse, %	13.0	9.4	11.0
Emotional abuse, %	64.2	62.2	63.1
Failure to provide neglect, %	50.3	46.8	48.4
Lack of supervision neglect, %	85.0	82.0	83.3
Educational neglect, %	23.3	28.8	26.3
Moral-legal maltreatment, %	27.5	28.8	28.2
No. referrals to social services, Mean (SD)	4.70 (5.35)	4.68 (5.20)	4.69 (5.26)
Placement type
Foster care, %	46.1	38.6	42.0
Kinship care, %	50.3	56.7	53.8
Congregate care, %	3.6	4.7	4.2
Mental Health Variables
Mental health index, Mean (SD	0.02 (1.05)	−0.02 (0.96)	0.00 (1.00)
PTSD symptoms, Mean t-score (SD)	49.69 (10.97)	49.24 (9.93)	49.44 (10.40)
Dissociation, Mean t-score (SD)	50.53 (11.35)	49.98 (10.29)	50.23 (10.78)
Quality of life, youth report, Mean (SD)	2.67 (0.31)	2.70 (0.26)	2.68 (0.29)
Mental health therapy, %	75.6	83.2	79.8*
Psychotropic medication, %	18.7	19.8	19.3

Note.

^*p<.05 for difference between intervention and control groups; racial/ethnic categories are non-exclusive which is why the numbers do not add up to 100%

Study Protocol

The current study was approved by the university institutional review board, and written informed consent and assent were obtained prior to conducting the interviews. All children who participated in the baseline interview were screened for cognitive, educational, and mental health problems using standardized tests of intellectual ability (Kaufman & Kaufman, 1990; Kaufman & Kaufman, 2004) and academic achievement (Psychological Corporation, 1992), as well as normed caregiver- and child-report measures of psychological functioning (see measures below). The findings and accompanying recommendations were summarized in reports provided to children’s caseworkers, who were encouraged to use the reports to advocate for educational and mental health evaluation and services.

Eligible children in both the “assessment only” (hereafter referred to as Control) and the “assessment plus intervention” (hereafter referred to as Intervention) groups were assessed at two timepoints: (1) Baseline, 2–3 months prior to the start of the intervention and (2) Time 2, 6-months post-intervention (17–20 months post-baseline), as were their caregivers. At both timepoints, children and their current caregivers/parents were each interviewed by separate interviewers, typically at the child’s residence. Interviewers were masked to condition at the Time 2 interview. Children and caregivers were each paid $40.00 for their participation. Current teachers of participating children were also surveyed 10-months post-intervention (T2). Following the baseline interview, children were randomized after stratifying on gender and county. All children were manually randomized, by cohort, in a single block.

Intervention

The 9-month Fostering Healthy Futures (FHF) preventive intervention consisted of two components: (1) manualized skills groups, and (2) one-on-one mentoring by graduate students in social work or psychology (FHF is described in detail in Taussig et al., 2007). The FHF program was designed to be above and beyond treatment as usual and could not be court-ordered. Although eligibility criteria required that children be in out-of-home care at the start of the intervention, if they reunified or changed placements during the intervention, their participation continued following appropriate consent.

Skills Groups.

FHF skills groups met for 30 weeks for 1.5 hours/week during the academic year and included 8–10 children and two group facilitators (clinicians and graduate student trainees). The FHF skills groups followed a project-designed manualized curriculum that combined traditional cognitive-behavioral skills group activities with process-oriented material. Units addressed topics including: emotion recognition, perspective taking, problem solving, anger management, cultural identity, change and loss, healthy relationships, peer pressure, abuse prevention, and future orientation (Taussig et al., 2007). The skills group curriculum incorporated material from other evidence-based skills group programs, including Promoting Alternative Thinking Strategies (primarily material from the emotional understanding, interpersonal problem solving, and self-control units; Kusche & Greenberg, 1994) and Second Step (primarily material from units on perspective-taking, problem-solving, communication, anger management, and dealing with peer pressure; The Committee for Children, 2001) which were supplemented significantly with project-designed exercises from multicultural sources. The skills group curriculum included weekly activities that encouraged children to practice newly learned skills with their mentors in their communities.

Mentoring.

The mentoring component of the FHF program provided 30 weeks of one-on-one mentoring for each child within the community. Mentors were graduate student interns in social work or psychology who received course credit for their work on the project. Mentors were each paired with two children with whom they spent 2–4 hours of individual time each week. Mentors were matched with children based on a host of factors that included geographical proximity, interests, demographic factors, child/family preferences, and mentors’ prior experiences. Mentors transported children to and from skills groups and joined the skills group for dinner. Mentors received weekly individual and group supervision and attended a didactic seminar, all of which were designed to support mentors as they worked with children in their communities to: (a) create empowering relationships with children, serving as positive examples for future relationships, (b) connect children with appropriate services in multiple domains and serve as a support for children as they faced challenges within various systems, (c) help children generalize skills learned in group to the “real world” by completing weekly activities, (d) engage children in a range of extracurricular, educational, social, cultural, and recreational activities, and (e) promote attitudes to foster a positive future orientation. All mentoring activities employed by mentors were individually tailored for each child, based on the children’s presenting problems, strengths, and interests, as well as their family and placement characteristics.

Program Uptake and Fidelity

On average (including data from 17 children who withdrew from the program post-enrollment), children attended 25.6 (Median=28.0, SD=6.0, Range of 1–30) of the 30 skills groups and 25.9 (Median=27.0, SD=6.4, Range of 2–41) of the 30 targeted mentoring visits. The 30 skills group sessions included 104 discrete activities. On average, across all 10 cohorts (which included 24 groups of children), 98 (94.2%) of the 104 group activities were completed.

Measures

Demographic, Maternal and Child Welfare Characteristics.

Children’s age, sex, race, ethnicity and placement type (e.g., foster care, kinship care, congregate care) were obtained from child welfare records and children’s self-reports. To determine type(s) of maltreatment experienced and maternal characteristics (e.g. substance use, criminal history), trained research assistants coded each child’s legal petition and social history (child welfare records’ narrative of the history and events preceding the legal filing that led to the child’s removal from the home) using a modified version of the Maltreatment Classification System (Barnett, Manly, & Cicchetti, 1993). The developers of the rating system report an overall kappa of .60 and adequate estimates of inter-rater agreement (.67–1.0). All records were consensus coded by at least two trained staff, and discrepancies were resolved through consultation with one of the senior investigators. Four maternal characteristics (substance use, criminal history, mental illness and history of maltreatment) and seven types of child maltreatment (physical abuse, sexual abuse, emotional abuse, failure to provide, lack of supervision, and moral-legal maltreatment) were dichotomously coded as present or absent.

Intellectual Functioning.

The Kaufman Brief Intelligence Test (KBIT and K-BIT-2; Kaufman & Kaufman, 1990; Kaufman & Kaufman, 2004) is a screening measure of intelligence that yields Verbal, Nonverbal, and Composite estimates of IQ for individuals 4–90 years of age. The KBIT was normed on a nationally-representative sample of 2,120 individuals who were stratified on the basis of gender, race/ethnicity, and geographical region. Considerably shorter than the Wechsler Intelligence Scale for Children (WISC), the KBIT takes approximately twenty minutes to administer and is highly correlated with the WISC (r =.56 to .84) and with the Wechsler Adult Intelligence Scale (rs =.79 to .89).

Adverse Childhood Experiences (ACEs)

(Raviv, Taussig, Culhane, & Garrido, 2010). ACEs were assessed with a previously-developed, project-designed 6-item published measure of ACEs for this sample, comprised of: (1) Physical Abuse; (2) Sexual Abuse; (3) Removal from a single parent household; (4) Exposure to community violence; (5) Number of caregiver transitions; and (6) Number of school transitions. The first three ACEs were coded as “present” (1) or “absent” (0) following review of child welfare records. Community violence, caregiver transitions, and school transitions were assessed through child report and dichotomized: a score of 1 was assigned to scores in the upper quartile of the sample, and a score of 0 for all others (e.g., Appleyard et al., 2005). To determine community violence exposure, an adapted version of the Things I Have Seen and Heard scale (Richters & Martinez, 1993) was used. Children were asked to indicate the number of times in the past year they had seen or heard acts such as, “guns being shot” or “seeing someone getting arrested.” Responses ranged from never (0) to four or more times (4), with the overall score representing the sum of the 8 items. The upper quartile included children with scores of 13.00 and higher (range=0–30.00, M=8.47, SD=6.70). For caregiver transitions, children reported the number of caregiver transitions (with or without child welfare involvement) experienced since birth (range=1–11, M=2.79, SD=2.07). Children in the upper quartile were those who had experienced 4 or more caregiver transitions. For school transitions, children reported the number of school transitions they had experienced (range=0–29, M=3.22, SD=2.85), with 4 or more transitions representing the upper quartile group. Children’s scores of 1 or 0 for each of the six ACE items were summed to form a composite ACE index. In this sample, ACE scores ranged from 0 to 5 (M=1.67, SD=1.08). Most children (89%) were exposed to at least one ACE.

Mental Health.

Mental health functioning was assessed using the Posttraumatic Stress and Dissociation scales of the child self-report Trauma Symptom Checklist for Children (TSCC; Briere, 1996), a widely-used symptom-oriented measure of mental health problems, as well as the Internalizing scales of the Child Behavior Checklist (CBCL) and the Teacher Report Form (TRF), both well-normed measures of child emotional and behavior problems (Achenbach & Rescorla, 2001). A multi-informant Mental Health Index was created based on principal components factor analysis of the children’s mean TSCC scores, and the Internalizing scales of the CBCL and TRF. Because teachers were only interviewed at T2, the baseline mental health index score consisted of only the TSCC and CBCL scales. The T1 factor score explained 62.8% of the variance and the factor loadings were .79; at Time 2, the factor score explained 44.8% of the variance and factor loadings were .71 for the TSCC, .67 for the CBCL, and .62 for the TRF.

Quality of Life.

Children completed the Life Satisfaction Scale (Andrews & Withey, 1976) which asks respondents to rate satisfaction in several different domains (e.g. school, home, health, friendships, leisure activity). The authors report that the original items demonstrate good internal consistency (α = .81) and construct validity. In our study, internal consistency was similar, α = .75.

Services and Medication Use:

Children’s use of mental health services and psychotropic medications were assessed based on child and caregiver reports at Time 1 and Time 2. Because of concerns related to foster parents not knowing the child’s history of therapy and medication use, we asked the caregivers to report on current use at both time points, whereas children were asked to report on lifetime use at baseline and past 6 month use at T2. Mental health service and psychotropic medication use were dichotomized at T1 and T2, such that if either the child or caregiver reported use in the timeframe, they were coded as 1; if neither informant reported use, they were coded a 0.

Analytical Methods

Equivalence between intervention and control groups on baseline characteristics and outcome measures was assessed using chi-square tests for categorical variables and t-tests for continuous variables. Chi-square tests were used to assess whether the rate of attrition varied by treatment condition. Attritted and non-attritted youth were then compared on all baseline measures for the overall sample and within treatment condition using chi-square and t-tests. When the pattern of attrition was different across treatment condition, logistic regression analyses, with interactions between baseline variables and treatment status, were run to predict attrition status. Chi-square and t-tests were also used to compare: (1) characteristics of children who refused the intervention to those who started it and (2) those who dropped out of the intervention to those who completed it.

Linear regression models were used to predict continuous outcome variables with effect sizes estimated with Cohen’s d. Logistic regression was used to examine odds ratios for dichotomous outcomes. Interaction analyses were conducted to examine whether a priori-selected demographic, ACEs, or baseline functioning variables moderated the impact of the intervention on outcomes. Significant interaction effects were further probed by testing the significance of simple slopes (i.e., the difference between intervention and control at each conditional value of the moderator variable). Conditional values of continuous variables included the mean and one standard deviation above and below the mean (Aiken & West, 1991).

All regression models adjusted for baseline scores on the corresponding outcome measure and all analyses used the intent-to-treat sample. In order to examine the impact of non-independent data from siblings, analyses were run with the full sample and then replicated with a sample that contained only one randomly selected sibling. Because there were no differences in effect sizes in any of the analyses after dropping one sibling, the results reported are for the full sample. Despite the number of analyses conducted, no method for controlling the familywise error rate was used because we wanted to examine the pattern of findings; p values are given for each analysis so the reader can make their own determination of the significance of each finding. Sample size for each analysis varied slightly due to missing data on some outcome variables (see Table 2 for Ns); no data were imputed.

Table 2.

Impact of the FHF Intervention on T2 Outcomes

		Actual Mean (SE)/%		Adj Mean (SE)/%
	n	Control	FHF	Control	FHF	Adj. Mean Difference (95% CI)	Cohen’s d/OR (95% CI)	p
Mental Health Index	346	.12(.08)	−.10(.07)	.12(.07)	−.10(.06)	−.22 [−.41, −.03]	−.25 [−.46, −.03]	.02
Posttraumatic Stress	375	44.77(.76)	43.20(.65)	44.88(.65)	43.12(.58)	−1.76 [−3.53, .01]	−.20 [−.41, .00]	.04
Dissociation	375	46.54(.83)	44.30(.63)	46.64(.65)	44.21(.58)	−2.43 [−4.15, −.71]	−.29 [−.49, −.08]	.006
Quality of Life	375	2.71(.02)	2.77(.02)	2.72(.02)	2.76(.02)	.12 [−.01, .09]	.16 [.37, −.04]	.10
Mental Health Therapy	377	69.9%	60.7%	59.2%	47.6%		.62 [.40, .97]	.04
Psychotropic Medication	378	18.0%	19.9%	7.5%	7.6%		1.01 [.53, 1.94]	.97

Results

Differences on Baseline Characteristics

Across the 25 baseline comparisons conducted, there were only two statistically significant differences. Intervention youth were more likely than control youth to have mothers with criminal histories, χ²(1,N=423) = 4.21, p < .05, and intervention youth were more likely to have a history of receiving mental health therapy, χ²(1,N=425) = 3.71, p = .05 (see Table 1).

Attrition

Chi-square analyses suggested that the rate of attrition did not differ by treatment condition, although there was a statistical trend for more intervention youth to be retained, 91.8% vs. 86.5%, χ²(1,N=426) = 3.16, p = .08. Across treatment condition, those interviewed at follow-up were compared with non-interviewed children on all baseline characteristics. There were no significant differences. When looking within treatment condition, however, youth in the control group with relatively higher baseline mental health problems were more likely to attrit. Specifically, those in the control group who attritted (compared to those who were retained) had higher scores on the T1 Mental Health Index t(191) = 2.94 p < .01, the Posttraumatic Stress scale, t(191) = 2.21 p < .05, and the Dissociation Scale, t(191) = 2.25 p < .05. Although there were no statistically significant differences between those who attritted and those who were retained within the intervention group, the pattern of findings suggested that those who were retained had higher baseline mental health scores and mothers who had a history of criminal behavior and mental illness. For this reason, interaction analyses were conducted to examine whether mental health or maternal characteristics moderated the impact of intervention status on attrition. Two of the five interaction analyses (mental health index and posttraumatic stress symptoms) were significant and two approached significance (maternal criminal history and mental illness), suggesting that the trial retained more intervention children with mental health problems and adverse maternal characteristics, whereas children with these characteristics in the control group were more likely to have attritted.

Although the ns were very small, Chi-square and independent samples t-tests were also used to compare characteristics of children who refused the intervention (n=11) to those who began it (n=222). Of the 25 comparisons, two were statistically significant. Those who experienced emotional abuse were more likely to begin the intervention (98.6% vs. 89.8%, p = .002) and those who started the intervention had more child welfare referrals than those who refused (4.8 vs. 2.4, p = .002). Comparisons were also made between those who dropped out of the intervention (n=17) and those who completed it (n=205). Those who experienced failure to provide neglect were more likely to drop out than those who did not experience this type of maltreatment (12.6% vs. 3.4%, p = .01). While none of the baseline mental health differences achieved statistical significance, there was a pattern for those who dropped out to have lower mental health problems on all indices (e.g., mental health index score for dropouts was −.41 vs. .03 for those who completed, p = .06).

Outcome Analyses

The outcome analyses are shown in Table 2. Both unadjusted and adjusted (for the corresponding T1 score) means, standard errors, and percentages are shown. Intervention youth scored significantly lower on the mental health index (β = −.11; t = −2.32; p < .05), posttraumatic stress symptoms (β = −.09; t = −2.01; p < .05) and dissociation scales (β = −.12; t = −2.76; p < .01), and they were less likely to be receiving mental health treatment at T2 (B = −.47; Wald = 4.43; p=.04). Intervention and control groups did not differ on quality of life or psychotropic medication use.

Moderating Analyses

A final set of analyses examined whether the impact of the FHF intervention on the six T2 mental health outcome variables was moderated by the following baseline variables: Gender, Hispanic, African American, IQ, Placement Type (dichotomized as foster vs. kinship care), ACEs and T1 mental health index. Forty-two analyses were conducted and three interaction terms were statistically significant. The baseline ACEs measure significantly moderated the intervention’s impact on T2 youth-reported posttraumatic stress symptoms (β = .15; t = 2.23; p < .05) and quality of life (β = −.15; t = −2.06; p < .05). In probing the interactions, we found that lower levels of ACEs were associated with stronger treatment effects on both of these outcomes. First, in predicting symptoms of posttraumatic stress, the slopes representing children with baseline ACE scores 1 SD below the mean and at the mean were significant, t(371) = −2.93, p = .004 and t(371) = −1.93, p = .05, respectively. Among children exposed to relatively few or an average number of ACEs, intervention participants reported fewer symptoms of posttraumatic stress than the control group at T2. The slope representing children exposed to high numbers of ACEs was not significant. Second, in predicting quality of life, the slope representing children with baseline ACE scores 1 SD below the mean, t(369) = 2.07, p =.04, was significant. Among children exposed to few ACEs, intervention participants reported higher quality of life at T2, as compared to children in the control group. The slopes representing children exposed to average and high numbers of ACEs were not significant.

In addition, there was a significant interaction between gender and treatment group status in predicting the use of psychotropic medication at T2 (B = −1.85; Wald = 6.27; p < .05). The percent of control girls using medication decreased (from 19.0% to 12.7%) while intervention girls had a slightly increased percentage (15.1% to 18.9%) of medication use (chi-square n.s.). For males, however, there was an opposite effect, as the percent of control boys using medications increased (15.9% to 22.7%), while it decreased for intervention boys (25.7% to 21.0%; chi-square also n.s.).

Discussion

This study sought to replicate and extend the mental health findings associated with the Fostering Healthy Futures (FHF) program, a mentoring and skills group intervention for maltreated preadolescent children in out-of-home care. As was found in the pilot trial, the FHF program demonstrated significant impact in reducing mental health symptomatology, especially symptoms associated with trauma, anxiety, and depression as well as mental health service utilization in this high-risk population, 6–10 months post-intervention. These findings are strengthened by the fact that full intent-to-treat analyses were conducted, the study controlled for baseline functioning, and multiple informants (children, caregivers, and teachers) reported on children’s mental health functioning. The small effect sizes, while lower than those obtained in the pilot trial (as is common in replication studies with larger samples), are well within the norm for studies of the impact of prevention programming that includes skills groups and mentoring (DuBois, Holloway, Valentine, & Cooper, 2002; DuBois, Portillo, Rhodes, Silverthorn, & Valentine, 2011; Leve, Harold, Chamberlain, Landsverk, Fisher, & Vostanis, 2012; Lösel & Beelmann, 2003; Sandler et al., 2014).

The positive intervention effects may actually be conservative for several reasons: (1) the study lacked a no-treatment control group (as all participants received an assessment and recommendations for services); (2) differential attrition (overall and within group) which suggested that the highest risk youth in the intervention group were retained while the control group attritted the most at-risk participants; (3) the study did not control for baseline differences between groups, which suggested the control group was less at risk on two indices, and (4) no treatment-on-the-treated or dose-response analyses were conducted. The rationale for not including a “pure” control group was due to the fact that so few children entering foster care are screened for cognitive, academic or mental health problems and we felt that all children should have critical problems identified and addressed. For example, our screening found that 26.4% of preadolescent youth in foster care had a history of suicidality, which is not routinely assessed in preadolescents (Taussig, Harpin, & Maguire, 2014). The screening assessments’ recommendations, however, were associated with greater service use post-program for children, regardless of intervention status, and therefore might have attenuated FHF program effects (Petrenko, Culhane, Garrido, & Taussig, 2011). In addition, differential attrition across and within treatment groups likely contributed to an underestimate of program effects. For example, a post-hoc analysis found an 81.4% retention rate for control group children who were above the clinical cutpoint on the CBCL internalizing scale at baseline, whereas 97.1% of intervention youth above the clinical cutpoint were retained.

One of the most important findings was that the FHF program, using a positive youth development approach, demonstrated high rates of program engagement and completion, which is rare for extremely vulnerable and mobile populations. Over 95% of children who were randomized to the intervention began it and 92% completed the 9-month program. This is striking, given that the recruitment strategy did not rely on caseworker referrals, as caseworkers may have referred youth whom they thought would most engage and persist in the program. In addition, the few number of exclusion criteria suggests that the program was engaging across gender and racial/ethnic groups, as well as for those with low and high levels of mental health problems, those living in different placement settings, and those who returned home during the program. The high engagement rate also suggests that the FHF program was contextually-sensitive. Designed specifically for children in out-of-home care, the program attended to the needs of foster parents, who have great demands on their time due to the care of multiple children, as well as the responsibility to transport them to visitation with their biological parents and to physical, mental health, and educational services. Other studies have found that it is difficult to engage this population in evidence-based treatment (e.g., Dorsey, Pullman, Berliner, Koschmann, McKay, & Deblinger, 2014; Fitzgerald et al., 2015; Ganser et al., 2017), but the FHF program provided transportation, dinner, and respite care twice a week, which likely helped retention. In addition, a large percentage of children changed placements or reunified during the program, and yet, largely due to the mentors’ collaboration with collateral adults, they were able to be retained. Over the 10 years of the study, only one biological parent withdrew their child from the program post-reunification. While it was initially felt that the lack of a parent-training component or parental involvement in the intervention would undermine the potential for achieving positive outcomes, the size of FHF’s program effects is similar to other programs for children in foster care which involve foster parents in the treatment (e.g., Leve et al., 2012). These findings are important for the field of positive youth development (PYD), as the evidence for these programs’ efficacy is just starting to be accumulated for high-risk populations.

Another primary goal of the current study was to examine moderation effects. Few studies of preventive interventions (and especially those for children in foster care) have large enough sample sizes to test moderation and several systematic reviews concluded that this is a critical next step for the field (Hambrick et al., 2016; Leenarts, Diehle, Doreleiiers, Jansma, & Lindauer, 2013; Sandler et al., 2014). Like many high-risk populations, referral and receipt of mental health services for children in foster care (after controlling for need for services) differ by sociodemographic, maltreatment, and placement factors. Studies have found that ethnic minority children in foster care are less likely to receive services, as are children in kinship care and those who have reunified with their biological parents. Some studies have shown that sexually and physically abused youth are more likely to receive services than children who have experienced neglect but not abuse (Burns et al., 2004; Garland, Hough, McCabe, Yeh, Wood, & Aarons, 2001; Leslie, Hurlburt, Landsverk, Barth, & Slymen, 2004). Despite these known differences in rates of service utilization, the field is lacking knowledge regarding whether mental health interventions engage and/or work similarly across different subgroups of children in foster care. The results of the current study suggest that the FHF program works well (across almost all outcomes) regardless of gender, racial/ethnic group membership, type of placement (i.e., non-relative foster care vs. kinship care), and for children with varying IQs and mental health functioning. There were some exceptions, one of which replicated our earlier finding that children exposed to relatively fewer ACEs tended to benefit more on some outcomes (i.e., posttraumatic stress symptoms and quality of life), but the overall pattern of findings suggest rather robust effects across several characteristics. These findings, coupled with the FHF’s few exclusion criteria and high rates of program engagement speak to the generalizability of findings.

The study was not, however, without limitations. Despite randomization, there were a few key variables on which the two groups differed at baseline and there was differential attrition both between and within treatment groups. The pattern of findings, however, suggests that these differences would lead to conservative estimates of program effects. The study would be strengthened by the use of administrative records, which would overcome some of the limitations of interview attrition. In addition, the study was conducted by the program developers and there has not been an independent replication of program effects. During program dissemination, however, FHF implementation sites have been achieving the same level of program engagement and fidelity. Despite a fairly large sample, the power to detect significant moderating effects among smaller subgroups or low-rate outcomes was more limited. Finally, the study did not examine (1) the impact of mentor/mentee match characteristics, (2) the impact of nesting children within mentors or in skills groups, (3) the impact of events that occurred (e.g. placement changes, loss of a loved one, parental incarceration) during the intervention, (4) the impact of prior or concurrent mental health treatment or diagnoses, (5) dose-response intervention effects, or (6) cost-benefit.

Conclusions and Implications

Identifying interventions that can ameliorate the pervasive deleterious effects of maltreatment, other adverse experiences, and foster care placement is no simple undertaking. Successful interventions must be grounded in theory and research, have good track records for recruiting and retaining children and families, and be contextually sensitive. The results of the current study suggest that Fostering Healthy Futures is one of a growing array of such programs. FHF differs from other evidence-based programs, however, by employing a positive youth development approach. Children are not diagnosed, there is no one “problem” the program is trying to ameliorate, and there is no treatment plan. The current study’s findings suggest that this PYD approach is both feasible and effective. Child welfare agencies have begun to discuss and implement PYD programming, but most of the programs are small-scale and untested. We know, however, that even the best intentions can have deleterious effects, especially among vulnerable populations. For example, mentoring relationships that end prematurely or involve abandonment can do more harm than good, especially for children in foster care who may have already experienced relational trauma and attachment difficulties (Grossman & Rhodes, 2002; Kupersmidt, Stump, Stelter, & Rhodes, 2017; Spencer, 2006; Spencer & Basualdo-Delmonico, 2014).

The current study’s methodology also speaks to the external validity of the study findings. Despite the fact that the participants were heterogeneous on sociodemographic factors, maltreatment history, current living situation, and cognitive, academic, emotional and behavioral functioning, there were important program main effects and few significant moderating effects. The generalizability of the findings is strengthened by the fact that participants were not referred nor did they self-select into the program (as is the case with most community-based mentoring or skills group programs in which participants sign up). This study demonstrates that FHF, a positive youth development program which is currently being implemented through community-based agencies, can engage and retain highly mobile children with significant baseline problems. The study also demonstrates that the program can be implemented with fidelity. As we scale up the manualized FHF program, we have greater assurance that the program can confer benefit for children who are heterogeneous on multiple indices.

Despite the cluster of risks associated with maltreatment, including poverty, high-risk neighborhoods, parental psychopathology, substance use, and domestic violence, this study suggests that Fostering Healthy Futures promotes better mental health functioning and reduces mental health treatment (suggesting a cost savings) for maltreated youth placed in foster care. As the child welfare system embraces new evidence-based programs and reduces structural barriers to funding and implementing innovative programming (Chaffin & Friedrich, 2004), we hope more contextually-sensitive and growth-promoting strategies to address trauma and its sequelae in vulnerable populations will be implemented.