Effects of the KEEP Foster Parent Intervention on Child and Sibling Behavior Problems and Parental Stress during a Randomized Implementation Trial

Abstract

Children in foster care are at risk for externalizing behavior problems, which can in turn increase the risk of changes in foster care placement. The KEEP (Keeping Foster Parents Trained and Supported) foster parent training intervention was designed to equip foster parents with strategies for managing externalizing behavior problems. The primary goals of this investigation were to (a) examine the effectiveness of the KEEP intervention in reducing child behavior problems, as delivered by a community agency, (b) determine if the effects of the KEEP intervention generalize to more than one child in the same home, and (c) examine the effectiveness of the KEEP intervention in reducing parental stress associated with child behavior problems. The data from 335 foster and kinship families with children between the ages of 5 and 12 years were analyzed to address these objectives. Families were randomly assigned to the intervention or control condition. The results indicated that the KEEP intervention was effective in reducing child behavior problems when delivered by a community agency. These results expanded prior research on the KEEP intervention, revealing that the intervention was effective in reducing behavior problems of more than one child in the same household and in reducing parental stress levels associated with the behavioral issues of the focal child. Thus, the KEEP intervention model holds promise for reducing the behavior problems of children in foster care and reducing stress levels of foster and kinship caregivers as it is disseminated and implemented within similar child welfare settings.

Children and youth in foster care display a range of mental health problems, at rates higher than would be expected in community samples (Landsverk, Garland, & Leslie, 2002). Among these problems, externalizing behaviors are highly prevalent. Data from the National Survey of Child and Adolescent Well-being (NSCAW) study revealed that a high proportion of children in foster care (43%, based on teachers’ reports; 50%, based on parents’ reports) display some form of externalizing behavior (National Survey of Child and Adolescent Well-being Research Group, 2003). Highlighting the importance of these findings is evidence indicating that externalizing behavior problems are associated with increased risk for placement instability in foster care (Chamberlain et al., 2006; Newton, Litrownik, & Landsverk, 2000). Furthermore, placement disruptions have been found to be related to increased rates of externalizing behavior problems (Newton et al., 2000). Thus, children who enter foster care displaying high levels of behavioral issues have an increased likelihood of experiencing a change in placement, which further increases the risk of continued—and even escalating—behavior problems.

In order to address the behavioral issues of children in foster care and to reduce the risk of the spiraling co-escalation of problem behavior and placement disruptions, the KEEP (Keeping Foster Parents Trained and Supported) intervention was developed (Chamberlain, Price, Reid, & Landsverk, 2008). Results from a series of studies indicate that the intervention is effective in reducing child behavior problems and in impacting placement changes (Chamberlain, Price, Leve, et al., 2008; Price, 2008). The primary goal of the current study was to extend this research by examining the effectiveness of the KEEP intervention in positively affecting the behavior of multiple children within a foster or kinship home, and in helping manage levels of parental stress associated with child behavior problems. We tested this as it was delivered by a community agency, utilizing a randomized design.

Based on the principles of Parent Management Training, the KEEP intervention utilizes specific components of the Multidimensional Treatment Foster Care model (Chamberlain, Leve, & DeGarmo, 2007; Eddy & Chamberlain, 2000). Foster and kinship caregivers receive 16 weeks of training and supervision in behavior management techniques and caregiver support in a group context, facilitated by trained facilitators for 90 minutes each week. Piloting of the intervention took place in Lane County, Oregon (Chamberlain, Moreland, and Reid, 1992) in a randomized trial with 72 families. Compared to the payment-only group and the control (assessment only) group, there were significantly larger improvements in child behavior, fewer failed placements, and greater foster parent retention in the parent-training group. Next, a randomized trial was conducted with 702 relative and nonrelative caregivers of children between the ages of 5 and 12 in foster care in San Diego County, California (Chamberlain, Price, Reid, et al., 2008). Caregivers in the intervention group completed the KEEP training while those in the control group received “services as usual,” which included parenting classes and support groups. In comparison to children in the control group, those in the intervention group evidenced a significant decrease in behavior problems, regardless of the age, type of placement (foster or relative), or caregiver’s language (English or Spanish). The intervention was also effective in increasing use of parenting strategies, which served to mediate the positive effects of the intervention on child behavior (Chamberlain, Price, Leve, et al., 2008). Finally, the intervention increased the number of positive exits (e.g., reunification with parents) and in mitigating the negative effects of a history of multiple placement changes on future negative placement exits, such as moving to another foster home or to a higher level of care (Price et al., 2008).

Moving the delivery of the KEEP intervention from a research-based organization (Oregon Social Learning Center - OSLC) to a community-based provider represents the next phase in the Cascading Dissemination Model (See Chamberlain, Price, Reid et al., 2008). In this model, the delivery, management, and supervision of the intervention are transitioned away from the intervention developers at each iteration and toward the implementation of the intervention by individuals that are independent of its original developers. In an implementation trial of KEEP in San Diego County (Price, Roesch, & Walsh, 2012), the delivery of the intervention was conducted by a community mental health provider. However, this implementation trial was not designed as a research study and did not involve a control group. In order to conduct a more rigorous test of the effectiveness of the KEEP intervention during implementation, the current study was conducted using a randomized design and a different cohort of caregivers.

During the course of the KEEP intervention, parents are asked to focus their efforts on applying the strategies they are learning to a particular child within their care. However, the extent to which parents apply the behavior management strategies they learn to other children in their care, and the extent to which the behavior problems of these other children are impacted, remains unknown. There is some research to suggest that generalization of the effects of parent-based interventions on multiple children in a home can and does occur.

For example, Brestan, Eyberg, Boggs, and Algina (1997) assessed the effects of parent-child interaction training for children referred for conduct problems on siblings who were not involved in treatment. Therapists focused on the referred child’s behavior problems and asked parents not to include siblings in home practice of parenting skills. Despite this, untreated siblings had lower levels of conduct problems following treatment, compared to controls. More recently, Brotman et al. (2005) evaluated the effects on older adjudicated youth siblings of a family-based intervention aimed at preventing conduct problems in preschoolers. Based on parent and teacher reports, significant improvements in antisocial behavior were found for the adolescent siblings eight months following the intervention. Finally, Hutchings et al. (2007) examined the effects of the Incredible Years intervention on a target child in the home and their closest-in-age sibling, using Eyberg’s Child Behavior Inventory. Reductions in severity scores were found for both the target children and their siblings.

Since in these studies, the reductions in behavior problems of nonreferred siblings were not dependent upon those siblings’ direct participation in the intervention, there are likely other mechanisms by which the benefits of parent-based interventions extend to siblings. One possible mechanism is the generalization of learned parenting skills, in which parents first acquire new practices with regard to the targeted or referred child, and then transfer these skills to other children in the home. In the majority of foster and kinship homes, caregivers typically care for more than one child, including their own biological children. Thus, when caregivers utilize newly acquired parenting strategies they are likely do so with all of the children in their care. Another possible mechanism for the generalization effect is the increased ability of the parent to address the behavior problems of other children. Positive and successful parenting experiences with one child might provide a parent with the time and confidence to attend to and address behavior problems in other children. This may lead to improvements in the behavior other children who might otherwise have been ignored due to an extensive focus on the child with the most serious problems. A third possible mechanism may be an overall reduction in coercive (i.e., angry and hostile) interactions within the family. Fewer coercive interactions between parents and children, and among children, should result in less problem behavior in all children in the family. Parenting strategies that result in behavior change in one child could have a systemic impact on the family, whereby a reduced display of problem behavior by one child results in fewer coercive interactions between that child and other members of the family, consequently reducing the behavior problems of other children as well.

Reductions in child behavior problems should also reduce the degree to which parents are naturally upset or stressed about these problems. Thus, equipping foster and kinship parents with the parenting skills necessary for managing child behavior problems should help to reduce how upset or stressed parents feel when encountering child behavior problems. Being less upset or stressed with a child’s behavior problems should help to increase placement stability.

The first objective of the current study was to examine the effectiveness of KEEP intervention in reducing child behavior problems as it was being delivered by a community agency not associated with the intervention’s developers. Using a randomized design, it was hypothesized that, relative to the participants in the control group, there would be a greater reduction in child behavior problems among participants in the intervention group. The second goal was to determine whether the effects of the KEEP intervention generalize to another child in the home. It was hypothesized that, relative to the sibling participants in the control group, there would be a greater reduction in child behavior problems among the siblings in the intervention group. Our final goal was to examine the effectiveness of the KEEP intervention in reducing parental stress associated with behavior problems. It was hypothesized that, relative to foster and kinship parents in the control group, there would be a greater reduction in parental stress levels linked with child behavior problems among parents in the intervention group.

Method

Participants and Recruitment Methods

Participants included foster and relative (kinship) families caring for a child received from San Diego County Child Welfare Services. Eligibility requirements were that (a) the focal child was between the ages of 5 and 12, (b) the focal child was in the current placement for at least 30 days (to minimize selecting children in temporary shelters or emergency placements), (c) the focal child was not considered to be “medically fragile” (that is, not severely physically or mentally handicapped—only one child did not meet this criterion), (d) there was at least one other child in the home, and (e) the family had had not previously received KEEP intervention services. From a pool of 609 potential participants across 4 years, a total of 354 families were recruited for participation in the study, and then randomized. The participation rate was 58%, which was similar to the 62% participation rate of the KEEP effectiveness trial (Chamberlain, Price, Reid, et al., 2008). See Table 1 for the baseline demographic characteristics of the participants. There was a higher percentage of African American participants in the intervention group than the control group, and parents in the control group had been foster parents or kinship caregivers for a longer period of time than parents in the intervention group. Three hundred and thirty five families participated in baseline interviews (intervention group = 164; control group = 171). See Figure 1 for the participant flow. Nineteen families (intervention = 15; control = 4) did not participate in the baseline interviews. The retention rates between baseline and follow-up assessments was 81% for the intervention group and 82% for the control group, rates similar to the 81% retention rate in the KEEP effectiveness trial (Chamberlain, Price, Leve et al., 2008).

Table 1.

Demographic Information on Foster and Kin Parents by Group

Demographic Information	KEEP Intervention Group (n = 164)	Control Group (n = 171)
Parent Gender
Female	94%	91%
Male	6%	9%
Mean Parent Age	45.10 (10.2)	45.75 (10.8)
Parent Ethnicity
African American	24%	9%
Asian/Pacific Islander	0%	3%
Caucasian	33%	39%
Hispanic	40%	42%
Native American	1%	4%
Mixed Ethnicity	2%	2%
Parent Preferred Language (During Interviews)
English	65%	63%
Spanish	35%	37%
Mean Months Foster Parent/Caregiver	38.22 (52)	45.79 (64)m
Mean Number of Children in Home	3.65 (1.5)	3.42 (1.4)
Relationship to Focal Child
Kinship Caregiver	45%	50%
Non-Kinship Caregiver	55%	50%
Focal Child Gender
Female	47%	49%
Male	53%	51%
Mean Age of Focal Child	7.84 (2.5)	7.32 (2.3)*
Focal Child Ethnicity
African American	23%	12%
Asian/Pacific Islander	2%	1%
Caucasian	11%	18%
Hispanic	46%	51%
Native American	1%	1%
Mixed Ethnicity	16%	17%
Relationship to Focal Sibling
Kin Caregiver	70%	69%
Non-Kinship Caregiver	30%	31%
Focal Sibling Gender
Female	48%	53%
Male	52%	47%
Mean Age of Focal Sibling	8.44 (3.9)	8.0 (4.0)

Note:

^*p < .05;

^mp < .10

Figure 1.

Flow of participants in KEEP Implementation Trial.

For this study, an age-eligible child in each household who was a dependent of Child Welfare Services was designated as the focal child (FC). Another child in the home closest in age to the FC was designated the focal sibling (FS); this child could be child in foster care, a biological or adopted child of the caregiver, or a relative in the family’s care (e.g., grandchild). If the FC left the home during the intervention, the FS was moved to the FC position (intervention = 19, control = 16). If the FS moved to the FC position or left the home, another child in the home (if available) was moved to the FS position (intervention = 16, control = 11). Since changes in placement occur this population, these substitution procedures allowed some of the FC and FS positions to be maintained throughout the intervention and assessments. Using an intent-to-treat design, intervention families that completed the baseline interview were maintained in the intervention group, even if they did not participate in any intervention sessions.

Data systems from Child Welfare Services were reviewed quarterly to identify eligible children and families. Families were contacted by phone to verify eligibility and determine level of interest. Interested parents received a home visit and were provided a description of the project. If caregivers agreed to participate, they were asked to sign a consent form and to select an envelope from among several unidentified and sealed envelopes. Each envelope contained a sheet of paper on which was either the word “Intervention” or the word “Control.” Following group assignment, the interviewer reviewed the specific portion of the consent form that addressed the procedures for that particular group assignment (i.e., intervention or control).

This study was conducted in compliance with the San Diego State University IRB. Participation was voluntary, and the decision to participate did not impact the services families received from any service agency. No solicitation or incentives were provided by Child Welfare Services for participation. Participants in both groups were provided incentives by the research project for completing baseline and termination assessments. Participants in the intervention group were given $15.00 at each group session to help compensate for travel costs. Childcare was provided for caregivers attending group sessions. Children did not attend parenting sessions or participate in any study related intervention. The parents in the intervention group received credit toward state foster care licensing requirements. Parents in the control group participated in routine parent training and group support provided by local service agencies.

KEEP Intervention Model and Delivery

The intervention was delivered by staff from Social Advocates for Youth (SAY) San Diego (a non-profit agency) who had prior experience facilitating KEEP parenting groups. Participants who were assigned to the intervention condition participated in groups comprised of 3 to 10 individuals. Parents received 16 weeks of parent training, supervision, and support in behavior management methods. The focus of the intervention was on increasing the use of positive reinforcement, consistent use of non-harsh discipline methods, such as brief time-outs or privilege removal over short time spans (e.g., not playing video games for one hour), and teaching the importance of close monitoring of the child’s whereabouts and peer associations. Strategies for avoiding power struggles, managing peer relationships, and improving success at school were also included. Sessions were structured so that the curriculum’s content was integrated into group discussions, and primary concepts were illustrated via role-playing and videotaped recordings. Although parents focused their attention on the behavioral challenges of the focal child, parents could also discuss behavioral challenges posed by other children in their care. Home practice assignments were designed to assist parents in implementing the behavioral strategies being taught. If parents missed a group session, the material was delivered during a home visit, an effective means of increasing intervention dosage for families who have missed sessions (Reid & Eddy, 1997). Eighty five percent of parents received at least one home visit. On average, across groups and sessions, the attendance rate was 67%. Intervention completion rates (including make-up sessions) were high, with 81% of parents completing all sessions.

Parenting groups were formed based on parent schedule, language preference (English or Spanish), and location. Groups met in community recreation centers, churches, or SAY facilities. Session materials were provided in English or Spanish, depending on the language used in the parenting groups. Facilitators were trained over several weeks through a series of phases involving (a) viewing video records of sessions run by experienced facilitators, (b) role-playing in mock group sessions, with the trainee as a group facilitator, and (c) co-facilitating group sessions with an experienced facilitator. The SAY personnel-led intervention groups were supervised on a weekly basis by an experienced KEEP trainer/facilitator. For each session, the supervisor reviewed the video record of the session and completed the Facilitator Adherence Rating (FAR), a 14-item rating scale assessing facilitator adherence to the structure, content, and process elements of the intervention protocol. Ratings for each item ranged from low (1) to high (5) adherence. The average item rating across the sessions and facilitators was 4.47 (SD = .36), suggesting a high degree of facilitator adherence. The supervisor then met with group facilitators weekly to provide feedback on group sessions, FAR ratings, and child behavior challenges. Consultation was also provided by the Oregon Social Learning Center, as needed.

Measures and Measurement Procedures

Assessments were completed prior to and following the intervention period. The interval range between the pre-and post-assessments was between 18 to 20 weeks. For most participants, baseline interviews started 2 weeks prior to the beginning of the intervention, and most were completed within 2 weeks following the completion of the intervention.

Child and parent characteristics

Caregiver reports of child and family characteristics and demographics were assessed at baseline via phone interviews. Caregivers knew the focal children for at least 30 days prior to the baseline assessment. Interviews were conducted in either English or Spanish, depending on the preference of the parent. The interviews began with questions about the characteristics of the focal child, followed by questions about the focal child’s behavior in the last 24 hours. Next, questions were asked about the characteristics of the focal sibling, followed by questions about the focal sibling’s behavior in the past 24 hours.

Child behavior problems and associated parental stress

The Parent Daily Report Checklist (PDR: Chamberlain & Reid, 1987) was used to assess child behavior problems and the degree of parental stress (being upset) associated with these problems. The PDR is a 31-item measure of child behavior problems, often administered via the telephone. During each call, a trained interviewer asks the parent the following question: “Thinking about (child’s name), during the past 24 hours, did any of the following behaviors occur?” Parents are then read the list of 31 behaviors and asked to indicate either “yes” or “no” to whether the behavior occurred. For each behavior that occurred, parents were asked to rate “how upset you were by that behavior,” with rating choices of 0 = Not at all; 1 = Somewhat/a little; and 2 = Quite a lot. Next, parents are asked questions about their parenting practices with this child within the last 24 hours. The PDR is structured so that parents only need to focus on recalling the past 24 hours, thus avoiding potential bias from attempting aggregate recall or estimates of frequency. Consistent with past research (Chamberlain, Price, Reid, et al., 2008), three PDR calls were administered at the baseline and termination on three different occasions across a two-week period. Scores representing levels of child behavior problems were calculated for each focal child and focal sibling at baseline and termination by summing the number of behaviors reported per day on the PDR, and dividing this by the number of calls made at each assessment period (typically three calls). Means and standard deviations for child behavior problems at baseline and termination by condition are provided in Table 2. Scores representing levels of parental stress associated with these problems were calculated for each focal child and focal sibling at the baseline and termination by summing the ratings of stress reported for each behavior reported per day on the PDR and dividing this by the number of calls made at each assessment period (see Table 2 for means and standard deviations). In previous research, the PDR has been used with families referred because of child conduct problems (e.g., McClowry, Snow, & Tamis-LeMonda, 2005) and families with children in foster care (Chamberlain, Price, Reid, et al., 2008). The concurrent validity of the PDR has been demonstrated with measures of child and family functioning, including observations of family interactions in the home (Forgatch & Toobert, 1979). For this sample, Cronbach’s Alpha coefficients were calculated for the three PDR assessments for the focal child at baseline (B1 = .82, B2 = .84, & B3 = .87; M = .84) and at termination (T1 = .86, T2 = .85, and T3 = .87; M = .86).

Table 2.

Descriptive statistics and effects sizes for child behavior problems and parental stress scores across pre-and post-test assessments.

	Pretest	Posttest	Cohen’s d
Child Behavior Problems
Focal Child
Control	4.18 (3.81)	3.59 (3.57)	.14
Intervention	5.02 (3.87)	3.56 (3.69)	.69
Same Focal Child
Control	4.28 (3.81)	3.72 (3.54)	.30
Intervention	5.03 (4.02)	3.65 (3.77)	.71
Focal Sibling
Control	3.45 (3.46)	3.22 (3.59)	.13
Intervention	4.06 (3.51)	3.11 (3.37)	.43
Same Focal Sibling
Control	3.56 (3.56)	3.30 (3.61)	.15
Intervention	4.19 (3.58)	3.10 (3.43)	.42
Parental Stress Scores
Focal Child
Control	5.92 (5.96)	5.82 (6.50)	.02
Intervention	7.07 (7.71)	5.19 (6.38)	.44
Same Focal Child
Control	5.95 (5.87)	5.80 (6.50)	.04
Intervention	7.32 (7.84)	5.59 (6.58)	.40
Focal Sibling
Control	5.67 (6.53)	5.55 (6.60)	.03
Intervention	6.60 (6.93)	5.18 (6.91)	.35
Same Focal Sibling
Control	5.62 (6.46)	5.53 (6.55)	.02
Intervention	6.55 (6.93)	4.97 (6.58)	.41

Note. (Standard Deviation)

Analytic Approach

To examine the study hypotheses, hierarchical linear modeling (HLM; Raudenbush & Bryk, 2002) was used as the primary statistical analytic tool. HLM is an appropriate analysis because of the nested (or clustered) structure of the data (i.e., repeated measures [level 1] nested within children [level 2]). Missing data common to longitudinal designs is readily incorporated in HLM if the data can be assumed to be missing at random (MAR). While there is no formal statistical test to determine MAR, there were no significant associations/differences between whether a participant had missing data and other study variables (e.g., demographics). HLM analyses retain all participants in the statistical analysis rather than implementing a list-wise deletion approach that can result in biased parameter estimates.

Specific analyses used an intent-to-treat (ITT) approach to assess the impact of the target intervention effects. Sensitivity analyses were also conducted for families that had the same sibling at the pretest and posttest. All analyses were conducted using MPlus (Muthén & Muthén, 2013). These analyses used the MLR estimation procedure in MPlus, which adjusts for missing data and non-normality of the outcome variables. Of primary interest in the analyses presented below are cross-level Group (intervention vs. control) × Time (pre-post intervention) interaction terms representing the intervention effects for both the focal child and the focal sibling. Outcome variables for the focal child and focal sibling were analyzed in separate HLM analyses. Specifically, PDR and parental stress variables at each time-point (pretest, posttest) served as the dependent variables. Potential interaction effects of relationship to caregiver, foster parent and child ethnicity, child age, language group, number of months as a foster parent, and the number of parenting sessions each foster parent received during the intervention were evaluated.

Results

Child Behavior Problems

Preliminary analyses were conducted to determine whether there were differences between the focal children and the focal siblings in age and baseline levels of behavior problems. Paired-sample t-tests were used to compare mean differences on these two variables. The results of the revealed that focal siblings (M = 8.2 years, SD = 4.0) were significantly older than focal children (M = 7.59 years, SD = 2.4), t = −3.1, p < .01. The results of the second analysis revealed that focal children had significantly higher behavior problem scores at the baseline (M = 4.59, SD = 3.87) than did focal siblings (M = 3.76, SD = 3.50), t = 4.1, p < .001. In addition, HLM analyses were used to test for baseline differences on behavior problems and in parent stress variables. As shown in Table 2, behavior problem scores were significantly higher in the focal child analyses for those in the intervention group relative to the control group, B = .835, p = .046. There were no significant differences between intervention and control participants on behavior problem scores for focal siblings, and no significant differences in parent stress for both the focal child and focal sibling analyses.

Of the 335 participants included in the analyses (see Figure 1), 62 focal children had missing data at the posttest, whereas 66 focal siblings had missing data at the posttest. (These participants included children who had changed placement during the intervention period and children from families who did not, for various reasons, complete the termination interview.) Descriptive statistics and effect sizes for the target intervention effects are presented in Table 2. The HLM analyses based on an ITT approach showed a statistically significant Group × Time interaction for behavior problem scores on the focal child (B = −0.73, p = .033) and the focal sibling (B = −0.55, p < .001). Follow-up analyses were conducted to determine the nature of the interaction. For behavior problem scores on the focal child, significantly larger decreases in scores from the pretest to posttest were found for the intervention group (B = −1.39, p < .001) relative to the control group (B = −0.66, p = .005), although scores significantly declined for both groups. Effect size estimates (see Morris & DeShon, 2002) suggest that the reduction in the intervention condition reflects a medium reduction, whereas the reduction in the control group is small. When the focal child analyses were re-evaluated using families for which the same focal child was assessed at the pretest and posttest, the statistically significant Group × Time interaction became marginally significant (B = −0.64, p = .064), although the descriptive pattern of findings was similar.

For behavior problem scores on the focal sibling, significantly larger decreases in scores from the pretest to posttest were also found for the intervention group (B = −0.95, p < .001) relative to the control group (B = −0.41, p = .055) (Table 2), although the effect size estimate in the intervention group was small-to-moderate, and the reduction in the control group was small. When the focal sibling analyses were re-evaluated using families for which the same focal sibling was assessed at the pretest and posttest, the statistically significant Group × Time interaction became marginally significant (B = −0.66, p = .066), although the descriptive pattern of findings was similar. No interaction effects were found for relationship to caregiver (i.e., kin vs. nonkin), foster parents’ or child’s ethnicity, child’s age, language group, number of months as a foster parent, and the number of parenting sessions each foster parent received (all ps > .05).

Parental Stress Levels

Descriptive statistics and effect sizes for the target intervention effects on the parental stress variables are presented in Table 2. The HLM analyses based on an ITT approach showed a statistically significant Group X Time interaction for parental stress scores on the focal child (B = −1.73, p = .021) but not for the focal sibling (B = −1.17, p = .108). Follow-up analyses were conducted to determine the nature of the interaction between parental stress scores on the focal child. Statistically significant decreases in parental stress scores from the pretest to posttest were found for the intervention group (B = −1.73, p = .003) but not for the control group (B = −0.16, p = .76). While the effect size in the control group was in effect 0, the effect size in the intervention group was small in size. When the focal child and focal sibling analyses were re-evaluated using families for which the same focal child and focal sibling was assessed at the pretest and posttest, the statistically significant Group X Time interaction for parental stress scores on the focal child was still statistically significant (B = −1.84, p < .001), as was the Group X Time interaction on the focal sibling (B = −0.98, p < .001). Similar to the findings in the full sample, statistically significant decreases in parental stress scores from the pretest to posttest were found for the intervention group (B = −1.84, p < .001) but not for the control group (B = −0.13, p = .80) for the same focal child. A similar pattern was also evident for analyses on the same focal sibling, with significant decreases in parental stress scores from the pretest to posttest were found for the intervention group (B = −1.48, p = .005) but not for the control group (B = −0.40, p = .41). However, the effect size in the intervention group was relatively small for both sets of statistically significant effects. Moreover, no interaction effects were found for relationship to caregiver (i.e., kin vs. nonkin), foster parents’ or child’s ethnicity, child’s age, language group, number of months as a foster parent, and the number of parenting sessions each foster parent received (all ps > .05).

Discussion

The primary goals of this investigation were to (a) examine the effectiveness of the KEEP intervention as it was being delivered by a community agency, using a randomized design, (b) examine the effects of KEEP on the levels of behavior problems of multiple children in the home, and (c) examine the influence of KEEP on parental stress levels associated with child behavior problems. The results of this study are consistent with findings from previous research in that the KEEP intervention was effective in reducing child behavior problems over the course of the intervention. Furthermore the intervention proved effective within a child welfare system even when it was delivered by a community agency not associated with the intervention’s developers. The results of this investigation expand prior research on KEEP by revealing that the intervention was effective in reducing behavior problems of both the child who was the focus of the intervention (focal child) and another child in the home who was closest in age to the focal child (focal sibling), suggesting that the effects of the intervention are generalizable across multiple children in the home. Finally, KEEP was found to reduce parental stress levels associated with the behavior problems of the focal child.

The findings from this investigation provide the strongest evidence to date that when the KEEP intervention is delivered by a community agency not associated with the intervention’s developers, it is effective in reducing child behavior problems. Prior research supporting the effectiveness of KEEP during implementation by a community agency utilized a quasi-experimental design with a historical comparison group (Price et al., 2012). In contrast, in the current study foster and kin families were randomized to treatment conditions, after consenting to participate. The results also revealed that the effects of the intervention did not interact with specific family or child characteristics, including the nature of the relationship between the caregiver and the child (foster parent or relative caregiver), the ethnicity of the caregiver, years of substitute caregiving experience, primary language spoken by the caregiver (English and Spanish), or ethnicity or age of the child. Thus, the intervention has the potential to generalize to foster and kinship families within a similar range of background characteristics.

Although foster parents are required to take parenting-related courses on a yearly basis in order to maintain their licenses to care for children, these courses can be quite diverse and may not specifically address the management of child behavior problems. In contrast, KEEP is designed to help parents address and manage child behavior problems by providing instruction on reinforcement of appropriate behavior, use of limit setting and consequences, and strategies for avoiding power struggles and conflicts. This instruction is provided in a supportive group context, whereby parents learn how to apply these strategies to the specific behavioral challenges that they and other caregivers are encountering daily, along with weekly home practice activities and phone calls to assist in the application of the parenting strategies.

Representing the next step in the Cascading Dissemination model outlined by Chamberlain, Price, Reid et al. (2008), the current study demonstrates the feasibility of implementing the KEEP foster parent intervention through a community agency, and doing so in a manner that maintains intervention effectiveness. In San Diego County, mental health services for children in foster are typically provided through private contractors, as is the case in many counties throughout the US. The implementation of KEEP was achieved by forming effective partnerships between various stakeholders, including child welfare personnel, mental health service providers, intervention developers, and foster parents. Such alliances have the potential to improve the behavior of children in foster care, improve children’s long-term outcomes, and decrease the levels of stress and adjustment difficulties among foster and kin caregivers.

Through successful replication of such collaborations, the KEEP intervention could be utilized as a universal intervention, as it was in the current study, and as a tool for prevention of behavioral problems leading to placement change (see Price et al., 2008). The KEEP intervention was designed to increase parent skills for managing challenging behavior problems, regardless of the level of child behavior problems. In the current study, behavior change was observed for focal siblings as well as focal children, even though focal siblings exhibited fewer behavior problems than focal children. The lower levels of behavior problems of focal siblings could be due to the fact that some of the focal siblings could have been the biological children of the caregivers and were spared the adverse experiences of the focal children who were all in foster care. In the current study, any foster or kinship family with an age-eligible child and at least one other child in the home—which is typical of most foster families—was eligible to participate. For child welfare systems that adopt the intervention, KEEP could be offered to new foster or kinship caregivers as soon as a child arrives in their home. For experienced foster parents, the intervention could be made available at any time, but may be particularly helpful at the beginning of the placement of a new child. In addition, given that the KEEP intervention has been found to be effective with children demonstrating higher levels of behavior problems as assessed by the PDR (Chamberlain, Price, Leve, et al., 2008; Price et al., 2012), the intervention could also be utilized as a more targeted intervention directed toward families with children with levels of behavior problems that have been found to place them at risk for placement disruptions (i.e., seven or more behaviors per day as assessed by the PDR; Chamberlain et al., 2006). Families attempting to manage children with challenging behavior problems could be referred by caseworkers to KEEP parenting groups. Early identification of children with elevated behavior problems could be accomplished by use of behavioral screening assessments within the first month of any new placement. For instance, caseworkers could utilize the PDR in series of random phone calls to identify children who may be presenting foster or kinship caregivers with behavioral challenges. The PDR offers an inexpensive, brief, and easy-to-administer measure of child behavior problems (Keil & Price, 2006), and has been found to be effective in identifying behavior problems associated with placement disruptions (Chamberlain et al., 2006).

As with any evidence-based intervention, implementation of the KEEP intervention within any child welfare system, as either a universal or targeted intervention, will require a thorough understanding of the culture and climate of that organization and the population it serves. Then, within that context, several key challenges need to be addressed. Among these challenges are eliciting buy-in from child welfare administrators and caseworkers, securing sustainable funding for the intervention (as either a universal or targeted intervention) and adherence monitoring, developing procedures for obtaining referral families, creating methods for maintaining a trained team of interventionists, providing supervision to maintain intervention fidelity, and identifying psychometrically-sound and practical measures of key outcomes. Outcomes most likely to be relevant to child welfare agencies would include levels of child behavior problems, frequency and nature of placement changes, and foster parent retention. Another challenge will be to identify effective strategies for eliciting and maintaining foster parent involvement in the KEEP intervention. In this study, foster and kin parents who attended the KEEP parenting sessions were provided child care, some reimbursement for travel, and credit toward state licensing requirements. Some of these incentives may not be available within child welfare systems seeking to implement the KEEP intervention. However, credit toward licensing requirement may be one incentive that will generalize across child welfare agencies.

To facilitate the implementation of the KEEP intervention, several lines of research are currently underway. First, the long-term effectiveness of the intervention is being examined in our sample using 12-and 18-month follow-up assessments of child behavior problems and placement disruptions. Second, work is planned to examine the role of the Facilitator Adherence Rating, a measure of intervention fidelity involving supervisor ratings of the delivery of the content, process, and structure of the KEEP intervention, in moderating the effects of the intervention. Additional directions for research include identification and utilization of other informants for child behavior problems. In this study, reliance was placed on parent observations of the child’s behavior. Although parents offer an important and unique perspective on child behavior, especially since their observations of child behavior problems are predictive of placement disruptions and stability (Chamberlain et al., 2006), perspectives of others who are familiar with the child in other contexts would also be beneficial. Given that children spend a great deal of time at school, both teachers and peers could provide valuable insights on the behavior of children and adolescents in foster care as they interact with others in school settings.

In this study, reliance on parent observations for study outcomes resulted in two potential limitations. First, the behavior problems of the focal child and focal sibling were assessed during the same phone interview, which decreases the independence of these assessments. To minimize this potential confound, questions about the behavior problems of the focal child and focal sibling were separated by questions about parenting practices. Also, the PDR directs parents to focus on their observations on a single child at a time. Second, the measure of parental stress was not independent of the measure of children’s behavior problems. That is, whenever a parent answered in the affirmative that a particular behavior was displayed in the past 24 hours, they were asked to rate “how upset you were by that behavior.” Thus, it would be expected that as child behavior problems decreased, the degree of parental stress (i.e., degree of being upset) associated with those problems would also decline. Independent observations of child behavior would have provided an additional measure of behavior and would have been useful in corroborating parent observations. Despite this, the PDR has already been found to be related to direct, independent observations of child behavior problems (Forgatch & Toobert, 1979).

It is also important to note other potential limitation of this study. First, for unknown reasons, the intervention and control groups differed on the representation of African Americans, with a higher proportion represented in the intervention group. In addition, there was a marginally significant difference between the two groups on the number of months caregivers had been either foster parents or kinship care givers, with those in the control group having been in their caregiver role for a longer period of time. This difference may, in part, be attributable to a parent in the control group who had been a foster parent for 29 years. However, both foster parent ethnicity and number of months being a caregiver were included as covariates in all analyses of group differences. Second, the participants were not blind to their condition. Thus, this knowledge could have influenced parents’ perceptions of the level of their children’s behavior problems and their own degree of stress associated with those problems.

As noted in the methods section, in this study if a target child or focal sibling left the home between the baseline and follow-up assessments, for whatever the reason, a substitute child was moved to that position. Thus, in several instances the follow-up assessments were conducted on a different child from the one who was assessed at baseline. Regardless of these changes and substitutions, there were still greater reductions in the behavior problems of the children in the intervention condition than the behavior problems of the children in the control condition. Thus, there is further evidence of the potential generalization of the effects of the KEEP intervention across children in the same home. When analyses were conducted using only those cases in which the focal child and focal sibling remained the same across the intervention period, the same patterns of findings resulted, although these particular results were marginally significant which may have been due to reduced statistical power associated with a smaller sample size.

The current study demonstrates the feasibility of developing partnerships among diverse stakeholders that enable and facilitate the delivery of an evidence-based intervention within a child welfare system. Furthermore, this can be done in a manner that maintains the effectiveness of the intervention and that benefits foster parents and the children in their care. The challenge for implementing the KEEP intervention in other child welfare settings is in building similar collaborative partnerships and adjusting the components of the intervention to match the unique requirements of each child welfare agency, its service providers, and the child population.