Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2013 Dec 1.
Published in final edited form as: Child Youth Serv Rev. 2012 Dec;34(12):2487–2494. doi: 10.1016/j.childyouth.2012.09.010

Effectiveness of the KEEP Foster Parent Intervention during an Implementation Trial

Joseph M Price a,c, Scott Roesch a,c, Natalia Escobar Walsh b,c
PMCID: PMC3555514  NIHMSID: NIHMS412345  PMID: 23359633

Abstract

Externalizing behavior problems are highly prevalent among children in foster care, placing them at risk for placement disruptions and later personal and social maladjustment. The KEEP foster parent intervention was designed to equip foster parents and relative caregivers with the parenting skills necessary for managing challenging behavior problems. In prior research, the KEEP intervention was found to be effective in reducing child behavior problems. In the current study, the KEEP foster parent intervention was implemented in San Diego County during a three-year trial. The intervention was delivered by paraprofessionals employed by a local community agency (Social Advocates for Youth, San Diego) to 181 foster parent and relative caregivers of boys and girls between the ages of 5 and 12. The control group from an earlier effectiveness study of the KEEP intervention that was also conducted in San Diego County was utilized as a historical comparison group. Regression analyses were used to examine the effects of the intervention on reducing levels of child behavior problems at treatment termination. Consistent with the findings from the previous KEEP effectiveness study, the intervention was found to be effective in reducing child behavior problems when delivered by a community agency. Furthermore, the KEEP intervention was found to be effective in reducing child behavior problems among children displaying various levels of initial behavior problems.

Keywords: intervention, foster parents, behavior problems

1. Introduction

Children in foster care display emotion and behavior problems requiring mental health assessments and/or intervention at a rate higher than would be expected compared to normative samples or community studies (Landsverk & Garland, 1999; Landsverk, Garland, & Leslie, 2002). Among the various problems evidenced by children in foster care, externalizing behavior problems (e.g., aggressive, disruptive, destructive, and oppositional behaviors) are highly prevalent and salient. Data from the National Survey of Child and Adolescent Wellbeing (NSCAW) study revealed that a high proportion (43% based on teacher report, 50% based on parent report) of children in foster care evidence some form of externalizing behavior problems (National Survey of Child and Adolescent Well-being Research Group, 2003). Findings from other studies reveal that the levels of antisocial behavior for children receiving child welfare services are statistically indistinguishable from children in intensive mental health treatment programs (Trupin, Tarico, Low, Jemelka, McClellan, 1993). Similarly, in their examination of the mental health of Canadian children in foster care in comparison to community and clinical samples, Stein, Evans, Mazumdar, & Rae-Grant (1996) found that both the foster and clinical samples exhibited significantly more externalizing problems than the children in the community sample, with no differences between the foster and clinical groups. Adding to the degree of impact of these findings is a body of research indicating that the risk for lifetime problems with antisocial behavior is especially high for children with an early onset of behavior problems (e.g., Patterson, DeBaryshe, & Ramsey, 1989). Not surprisingly, many youth in foster care develop serious conduct problems, including being arrested for violent crimes (Maxfield & Widom, 1996; Smith & Thornberry, 1995).

Within this population, externalizing behavior problems have been found to be linked to foster care placement instability. Not only have externalizing behavior problems been found to be predictive of placement disruptions and exits (Aarons et al., 2010; Chamberlain et al., 2006; Newton, Litrownik, & Landsverk, 2000), but placement disruptions have also been found to be predictive of increases in rates of child behavior problems (Aarons et al., 2010; Newton et al., 2000). Thus, children who enter foster care displaying high levels of behavior problems have an increased likelihood of experiencing a change in placement, which, in turn, further increases the risk for continued and even escalating behavior problems.

In response to the need for addressing the behavior problems of children in foster care and to reduce the number of changes of placement, a foster parent training intervention entitled KEEP (Keeping Foster and Kinship Parents Trained and Supported) was developed and tested. The primary goal of the current investigation was to determine whether the effectiveness of the KEEP intervention in reducing child behavior problems could be maintained when delivered by a community agency during an implementation trial conducted in San Diego County.

1.1. Development of the KEEP Intervention Model

Based on the basic tenants of Parent Management Training (Kazdin & Wassell, 2000; Patterson, 2005), the KEEP intervention represents a modified version of Multidimensional Treatment Foster Care (MTFC) which was developed by Chamberlain and colleagues (Chamberlain & Reid, 1991; Chamberlain and Reid, 1994; Eddy & Chamberlain, 2000; Leve & Chamberlain, 2004; Chamberlain, Leve, & DeGarmo 2007). An earlier version of the KEEP intervention was tested in Lane County, Oregon (Chamberlain, Moreland, and Reid, 1992). In this study, foster parents with a new child placement were randomly assigned to one of three conditions: parenting training using Parent Management Training components, payment and assessments only, and assessments only. Compared to the payment only group and the control group, parents in the parent-training group evidenced significantly greater decreases in child behavior problems, had fewer failed placements due to child behavior or emotional problems, and were significantly less likely to quit foster parenting.

The next step in this line of research involved the development and testing of the KEEP intervention in an effectiveness study in San Diego County. The intervention, which began in the fall of 2000 and was completed in early 2004, provided parent training and support to relative and nonrelative caregivers of children between the ages of 5 and 12 in regular foster care. Foster and kin caregivers in the intervention group completed a 16-week course of parenting training within the context of small facilitator-run groups. Caregivers in the control group received “services as usual,” which included yearly parenting classes and support groups for caregivers needing to meet state foster parent licensing requirements (i.e., licensed foster and kinship providers). The results of this study revealed that in comparison to the control group, children in the intervention group evidenced a significant decrease in behavior problems over the course of the 4-month intervention (Chamberlain, Price, Reid, et al., 2008). Moreover, the findings indicated that the effects of the intervention were maintained across developer-trained and non-developer-trained intervention staff (Chamberlain, Price, & Reid, et al., 2008), suggesting that with appropriate training and supervision, the intervention remained effective as it moves away from the intervention developers. The intervention was also effective in increasing parental use of targeted parenting strategies which, in turn, served to mediate the effects of the intervention on reductions in child behavior problems, especially for children displaying more than six behavior problems per day at baseline (Chamberlain, Price, & Leve, et al., 2008). Finally, the intervention resulted in increasing the number of positive exits from the home (e.g., unification with biological parents, adoptions), and mitigated the negative risk-enhancing effect of a history of multiple placements on negative exits (Price et al., 2008).

1.2. Implementation of KEEP in San Diego County

Following the completion of the original KEEP effectiveness study and publication of the initial findings, interest was generated among San Diego County Child Welfare leadership about how the KEEP intervention might be implemented into the regular in-service training offered to foster and relative caregivers. As a result, a series of meetings took place between the Child and Adolescent Services Research Center (CASRC), the Oregon Social Learning Center (OSLC) research partners, and the San Diego County Child Welfare administrators on how to proceed toward the implementation of the KEEP intervention in San Diego County. The focus of these discussions centered on three key issues: (a) identifying funding for the intervention; (b) staffing facilitators for the parenting groups; and (c) maintaining intervention fidelity. Child Welfare administrators were able to obtain supplemental funding from the State of California to conduct a pilot of the implementation of the KEEP intervention in San Diego County. Due to the workloads of caseworkers, it was determined that KEEP intervention would need to be delivered by a community-based mental health service contractor, one with a working relationship with the San Diego Child Welfare agency and currently delivering services in San Diego County. Social Advocates for Youth (SAY) San Diego, which served two of the six regions within the county, was contacted and expressed an interest in delivering the KEEP intervention. In order to maintain the fidelity of the intervention, OSLC-trained staff from the KEEP effectiveness study who were experienced in facilitating KEEP intervention groups (25 +) and in supervising other KEEP group facilitators, provided weekly supervision of SAY San Diego group facilitators.

1.3. Cascading Dissemination Model

Moving the delivery of the KEEP intervention from research-based organizations (OSLC and CASRC) to a community-based provider represents the next phase in the Cascading Dissemination Model (Chamberlain, Price, Reid et al., 2008). In this model, the delivery, management, and supervision of the intervention is moved away from the intervention developers at each iteration (in this case, OSLC) and toward the implementation of the intervention by individuals that were independent of its original developers. Phase 1 of the cascade is represented by the initial development and testing of the intervention in an efficacy study that took place in three Oregon counties (Chamberlain et al., 1992). The parent training groups were conducted by an experienced foster parent who had extensive training in the OSLC PMT model and was supervised by the treatment developer. Phase 2 of the cascade was implemented in San Diego County as the first part of the KEEP effectiveness study. In this phase, the intervention was delivered by paraprofessional staff hired by the Child and Adolescent Services Research Center (CASRC) research partners. These facilitators were supervised by an OSLC-trained on-site supervisor and an experienced OSLC clinical consultant. During Phase 3, which was the second part of the KEEP effectiveness trial, the CASRC intervention staff trained and supervised a second cohort of paraprofessional interventionists. The OSLC clinical consultant had no direct interaction with the group facilitators in this phase, but did consult with the CASRC interventionists in their supervision of this new group of facilitators. The current study represents the next phase of the Cascading Dissemination Model moving closer to large scale implementation, with delivery of the intervention through a community agency with no particular ties to OSLC and with funding provided by a Child Welfare agency rather than a research entity.

The primary goal of this investigation was to examine the effectiveness of the KEEP intervention in reducing child behavior problems as it was being delivered by a community-based mental health provider in San Diego County. Since the delivery of the KEEP intervention by SAY San Diego did not include any type of control group, data from the original KEEP effectiveness study was utilized to create a quasi-experimental design. In this design, the baseline and post-treatment data from the randomized effectiveness study of the KEEP intervention conducted in San Diego County (1999 to 2004) was integrated and analyzed with the baseline and post-treatment data collected from the implementation trial of the KEEP intervention within San Diego Child Welfare Services (2005 to 2008). Data from both the intervention and control group from the KEEP effectiveness study were utilized to in order to integrate findings from the effectiveness study with the findings from the implementation trial. Within this design, the original control condition from the effectiveness trial served as the control group when comparisons were made with the original KEEP effectiveness study intervention group. This same control group also served as a historical comparison group (nonequivalent control) when comparisons were made with the KEEP implementation trial group. The use of a historical comparison group from earlier clinical trials has been used as a component of research designs to examine the effectiveness of treatments delivered in a community setting (e.g., Costin & Chambers, 2007). This type of research design illustrates how researchers might utilize data from prior clinical trials of an intervention to examine the potential effectiveness of the intervention when it is implemented in service setting without any type of control group. It is hypothesized that relative to the children in the control group from the KEEP effectiveness study, children in the community agency administered KEEP SAY implementation trial group would also demonstrate greater reductions in child behavior problems over the course of the 4-month period of the intervention.

An additional goal of this investigation was to determine whether the KEEP intervention was effective in reducing child behavior problems at termination among children with various levels of behavior problems at baseline. That is, is the KEEP intervention effective in reducing child behavior problems regardless of the initial level of child behavior problems observed at baseline? It was hypothesized that the intervention would be effective in reducing child behavior problems at termination for children with behavior problems at or above the mean in baseline behavior problems. This hypothesis was examined using both the KEEP effectiveness study intervention group and the KEEP SAY implementation trial group.

2. Method

2.1. Participants

2.1.1. KEEP Effectiveness Study Participants

These participants were randomly assigned to either the intervention (KEEP parent training) or to the control group in an earlier effectiveness trial of the KEEP foster parent intervention (see Chamberlain, Price, Reid et al., 2008). In this earlier study, eligible study participants included all foster and relative caregivers with a child between the ages of 5 and 12 who was received from the San Diego County Health and Human Services Agency, Child Welfare services sometime between 1999 and 2004. Eligibility requirements were (a) the child was between the ages of 5 and 12, (b) the child had been in the placement for at least 30 days (in order to minimize selecting children in temporary shelters or emergency foster placements), and (c) the child was not considered to be “medically fragile” (that is, not severely physically or mentally handicapped - only one child met this criteria). The resulting sample was comprised of 700 foster families (34% kinship placements, 66% non-relative placements). California state law requires foster parents to participate in parent training and support group each year in order to be licensed. The parents in the intervention group (n = 359) were allowed to apply participation in the KEEP intervention group toward state licensing requirements. Parents in the control group (n = 341) participated in routine parent training and support provided by San Diego County services. Table 1 shows the baseline demographic characteristics of the participants from KEEP effectiveness study.

Table 1.

Demographic Information on Foster and Kin Parents by Group

Demographic Information KEEP Effectiveness Study KEEP SAY Implementation
Intervention
(n = 359)		 Control
(n = 341)		 (n = 181)
Parent Gender
     Female 94% 93% 98%
     Male 6% 7% 2%
(n = 117)
Mean Parent Age 49.86 (11.7) 47.29 (11.7) 51.28 (10.75)
Parent Ethnicity (n=154)
     African American 27% 24% 21%
     Asian/Pacific Islander 4% 2% 2%
     Caucasian 21% 34% 14%
     Latino 41% 33% 59%
     Native American 1% 1% 1%
     Mixed Ethnicity 6% 6% 3%
Parent Preferred Language (n = 159)
     English 67% 79% 49%
     Spanish 33% 21% 51%
Mean Household Income 2.3 (1.3) 2.3 (1.3) 2.0 (1.4)
Relationship to Child (n = 159)
     Kinship Caregiver 32% 36% 53.8%
     Non-Kinship Caregiver 66% 64% 46.3%
Child Gender (n=175)
     Female 50% 54% 46%
     Male 50% 46% 54%
Mean Age of Focal Child 8.8 (2.2) 8.7 (2.3) 7.9 (2.3)
Mean Number of Children in Home 3.5 (1.8) 3.5 (2.0) 3.2 (1.8)
Open in a new tab

2.1.2. KEEP SAY Implementation Trial Participants

Similar to the original KEEP effectiveness study, eligible participants included all foster and relative caregivers with a child between the ages of 5 and 12 who was received from San Diego County Child Welfare Services. In addition, because Social Advocates for Youth (SAY) also serviced relative substitute caregivers (e.g., grandparents, aunts, and uncles) who were not dependents under the care of the San Diego HHSA, participants were also recruited from eligible relative caregivers served by SAY San Diego. Similar to the KEEP effectiveness study, eligibility requirements included that (a) the child was between the ages of 5 and 12, (b) the child had been in the placement for at least 30 days, and (c) the child was not considered to be “medically fragile.” Table 1 provides the demographic characteristics of the group of participants for whom background information was available (n = 181), although the n’s varied by demographic category. For those parents who received their children from San Diego County Child Welfare services, participation in the KEEP intervention was allowed to fulfill yearly state licensing requirements. To examine potential differences between the KEEP effectiveness study participants and the KEEP SAY implementation trial group, ANOVA and Chi-Square analyses were employed. The results of the ANOVA analyses on continuous background variables revealed significant differences between the two samples on the following demographic variables: parent age, F(2,812) = 6.96, p = .001; age of the focal child, F(2,872) = 12.5, p = .001; and household income level (1 to 10 rating scale, with 1 = household income of less than $14,999 and 10 = $95,000 and up), F(2,856) = 4.33, p = .01 Chi Square analyses on dichotomous background variables revealed significant group differences on percentage of relative vs. non-relative caregivers, c2(4, N =859) = 23.7, p = .001, and primary language of foster parent, c2(2, N =859) = 47.3, p. = .001. Chi Square analyses also revealed significant group differences on percentage of ethic group composition, c2 (10, N=854) = 44.2, p. = .001. In particular, there was a higher percentage of Latino caregivers in the KEEP SAY implementation group than in either of the KEEP effectiveness study groups.

2.2. Recruitment Methods

For both sets of participants, recruitment was facilitated by use of data systems from the San Diego County HHSA that were reviewed on a quarterly basis to identify eligible children and foster and kinship families. For the KEEP SAY implementation trial group, contact information on relative caregivers from SAY San Diego was also utilized to identify eligible children and relative caregivers not served by the San Diego County Child Welfare Services. For both groups, similar recruitment procedures were used. First, eligible families were contacted by phone to determine their level of interest in the study. Next, interested families received a home visit, at which time a detailed project description, consent information and Institutional Review Board (IRB) approved consent form was provided. Those interested in participating verified their willingness to participate by signing the consent form. The investigation was conducted in compliance with appropriate IRB (San Diego State University and the Oregon Social Learning Center for the KEEP effectiveness study and from San Diego State University for the KEEP SAY implementation trial). Participation in both studies was voluntary. In addition, no solicitation or incentives were provided by San Diego County Child Welfare Services or Social Advocates for Youth for families to participate in either of these groups. However, participants in the KEEP effectiveness study were provided incentives by the research project for completing baseline and termination assessments. However, participants in the KEEP SAY implementation trial were not provided incentives for completing any assessments.

2.3. Intervention Model

Similar to parents in the KEEP effectiveness study, parents in the KEEP SAY implementation trial participated in parenting groups of 3 to 10 individuals led by a trained facilitator. Parents received 16 weeks of parent training, supervision, and support in behavior management methods. The primary focus of the KEEP intervention was on increasing use of positive reinforcement, consistent use of non-harsh discipline methods, such as brief time-outs or privilege removal over short time spans (e.g., no playing video games for one hour, no bicycle riding until after dinner), and teaching parents the importance of close monitoring of the youngster’s whereabouts and peer associations. In addition, strategies for avoiding power struggles, managing peer relationships, and improving success at school were also included. Sessions were structured so that the curriculum content was integrated into group discussions and primary concepts were illustrated via role-plays and videotaped recordings. Home practice assignments were given that related to the topics covered during sessions in order to assist parents in implementing the behavioral procedures taught in the group meeting. If foster parents missed a parent-training session, the material was delivered during a home visit. Such home visits have been found to be an effective means of increasing the dosage of the intervention for families who miss interventions sessions (Reid & Eddy, 1997).

Parenting groups were formed based on parent schedule, language preference (English or Spanish), and location. Parenting groups were conducted in community recreation centers, churches, or SAY facilities. Several strategies were used to maintain parent involvement, including (a) provision of childcare, using qualified and licensed individuals so that parents could bring younger children and know that they were being given adequate care, (b) credit was given for the yearly licensing requirement for foster care (HHSA foster parents, only), (c) parents were reimbursed $15.00 per session for traveling expenses, and (d) refreshments were provided. Group session attendance/completion rates (including make-up sessions for absences) were high, with 92% of parents completing at least 14 sessions. The language of the materials (English or Spanish) was determined by the language used in parenting groups.

As was the case with the KEEP effectiveness study, during the KEEP SAY implementation trial, the intervention was delivered by paraprofessionals. At least one of the group facilitators was bilingual in English and Spanish. Experience with group settings, interpersonal skills, and experience with diverse populations were given high priority in selecting interventionists. Interventionists were then trained over several weeks through a series of phases involving (a) viewing video records of prior sessions run by experienced facilitators from the original KEEP effectiveness study, (b) role playing in mock group sessions, with the trainee as a group facilitator, (c) and co-facilitating group sessions with an experienced facilitator. The KEEP SAY personnel-led intervention groups were supervised on a weekly basis by an experienced KEEP facilitator from the KEEP effectiveness study. Supervisors reviewed video records of group sessions and met with group facilitators on a weekly basis to provide feedback. Consultation was also provided by the Oregon Social Learning Center.

2.4. Measures

2.4.1. Child and Parent Characteristics

Foster and kin parent-report of child and family characteristics and demographics were assessed at study entry (baseline) via phone interviews. Caregivers had known the target child for at least 30 days prior to the baseline assessment. Interviews were conducted in either English or Spanish, depending on the preference of the parents.

2.4.2. Child Behavior Problems

The Parent Daily Report Checklist (PDR: Chamberlain & Reid, 1987) was used to assess child behavior problems at baseline and four months later at termination in both groups. The PDR is a 30-item measure of child behavior problems administered via telephone to parents on a series of consecutive or closely spaced days (approximately 1 to 3 days apart). During each call, a trained interviewer asked the foster/kinship parent the following question, “Thinking about (child’s name), during the past 24 hours, did any of the following behaviors occur?” Parents were then read the list of 30 behaviors and asked to indicate either “yes” or “no” as to whether the behavior had occurred in the last 24 hours. Consistent with the KEEP effectiveness study, three PDR calls were administered at baseline (prior to the intervention) and at termination (following completion of the intervention) on different occasions across a two-week period. The PDR is structured so that parents focus on recalling only the past 24 hours, thus avoiding aggregate recall or estimates of frequency thought to bias estimates (Stone, Broderick, Kaell, DelesPaul, & Porter, 2000). The PDR has been used in several previous outcome studies, including those with families referred because of child conduct problems (e.g., Kazdin & Wassell, 2000; McClowry, Snow, & Tamis-LeMonda, 2005) and families with children in regular foster care (Chamberlain, et al., 1992; Chamberlain, Price, Reid et al., 2008). The concurrent validity of the PDR has been demonstrated in connection with measures of child and family functioning, including live observations of family interactions in the home (Forgatch & Toobert, 1979; Patterson, 1976) and parents’ ratings of child behavior (i.e.., Becker Adjective Checklist; Becker, Madsen, Arnold, & Thomas, 1967). Scores representing levels of child behavior problems were calculated for each child at baseline and termination by summing the number of behaviors reported per day on the PDR (out of the possible 30) divided by the number of calls made at each assessment period (typically three calls). Means and standard deviations for baseline and termination child behavior problems by intervention and control groups are provided in Table 2.

Table 2.

Means and Standard Deviations of Baseline and Termination Behavior Problems by Group Group

Group Variable Baseline Child Behaviors
 Termination Child Behaviors
M SD M SD
KEEP Effectiveness Study
     Intervention (n = 356) 5.92 4.26 4.37 3.91
     Control (n = 341) 5.77 3.93 5.44 4.15
KEEP SAY Implementation (n = 159) 4.83 3.93 2.46 3.15
Open in a new tab

3. Results

3.1. Determination of Covariates

As reported earlier, analyses to examine potential differences between the KEEP effectiveness study groups and the KEEP SAY implementation trial group on demographic variables revealed significant group differences on several continuous variables, including the age of the focal child, age of the primary caregiver, and income level. To determine potential covariates for regression analyses, these variables were examined in relation to child behavior problems at termination using correlational analyses. The correlations between these variables and child behavior problems at termination were: r =.−.093 (p = .013), .005 (ns), and .014 (ns), respectively. Prior analyses of the demographic data also revealed significant group differences on several categorical variables, including percentages of relative vs. non-relative caregivers (with a higher proportion of parents in the KEEP SAY implementation group being relative caregivers), preferred language used by caregivers (with a higher percentage of caregivers in the KEEP SAY implementation group preferring Spanish as their primary language), and ethnic group composition. To examine the relation between these demographic variables and child behavior problems at termination a series of ANOVA analyses were employed. The results of these analysis revealed significant differences on child behavior problems for kinship type (relative vs. nonrelative), F (1,722) = 18.2, p = .001 (M = 3.5 and 4.8, respectively) and language (English vs. Spanish), F (1, 722) = 28.7, p = .001 (M = 4.9 and 3.2 respectively). In addition, significant group differences on behavior problems at termination were also found among parent ethnic groups. Post-hoc analyses, using Scheffe’s test, revealed a significant difference between Caucasian and Latino caregivers (p = .002), with children of Caucasian caregivers demonstrating significantly higher behavior problem scores at termination (M = 5.2) than children with Latino caregivers (M = 3.6). As a result of the aforementioned analyses, the following variables were entered as covariates in the subsequent regression analyses: child age, relationship of caregiver to child (kin vs. nonkin), primary language of the foster parent, and caregiver ethnicity.

3.2. Regression Analyses

3.2.1. Hierarchical Regression Analyses

A series of regression analyses were conducted in order to examine the effects of the KEEP intervention for two intervention groups (KEEP effectiveness study intervention group and KEEP SAY implementation trial group) on child behavior problems at termination and to explore the interaction between group status and level of baseline behavior problems. To begin, a hierarchical multiple regression analysis was conducted with behavior problems at termination as the criterion variable and two group status variables (KEEP effectiveness study intervention group vs. KEEP effectiveness study control group, and KEEP SAY implementation trial group vs. KEEP effectiveness study control group), baseline behavior problems, and the two interaction variables (between each group status variable and baseline behavior problems) as predictor variables. Covariates were controlled for by entering these variables on the first step of the equation (see Table 3). When entered on step 2, the two group status variables and baseline behavior problems accounted for a significant amount of overall variance in termination behavior problems, F(3, 705) = 145.03, p = .000, ΔR2 = .349. The standardized partial regression coefficient relating baseline behavior problems to termination behavior problems was statistically significant, β = .675, p = .000. The standardized partial regression coefficient relating group status 1 (KEEP effectiveness study intervention group vs. KEEP effectiveness study control group) to termination behavior problems was significant and negative, β = −.118, p = .001, indicating that children in the KEEP effectiveness study intervention group had lower behavior problem scores at termination than did the KEEP effectiveness study control group. Similarly, the standardized partial regression coefficient relating group status 2 (KEEP SAY implementation trial group vs. KEEP effectiveness study control group) to termination behavior problems was also significant and negative, β = −.246, p =.001, also indicating that children in the KEEP SAY implementation trial group had lower behavior problem scores at termination than did the KEEP effectiveness study control group. When the interaction terms between the two group status variables and baseline behavior problems was entered on step 3, a significant amount of overall variance in termination behavior problem scores was explained, F(2, 703) = 6.26, p = .002, ΔR2 = .010, and each interaction was significantly negatively associated with child behavior problems at termination, β = −.084, p = .050 (KEEP effectiveness study intervention group vs. KEEP effectiveness study control), and β = −.125, p =.001 (KEEP SAY implementation trial group vs. KEEP effectiveness study control group).

Table 3.

Hierarchical Linear Regression Predicting Child Behavior Problems at Termination

Steps B SE β
Step 1
Child Age −.097 .037 −.097**
Relationship Type .298 .074 .145***
(Kin vs. Nonkin)
Preferred Language of Parent −.435 .123 −.204***
Ethnicity Contrasts
     1. Caucasian vs. Latino .067 .128 .029
     2. African American vs. Latino −.131 .129 −.056
     3. Native American vs. Latino .833 .352 .088*
     4. Mixed Ethnicity vs. Latino −.057 .240 −.009
     5. Asian/Pacific Islander vs. Latino −.081 .197 −.017
Step 2
Child Age −.038 .029 −.038
Relationship Type .076 .060 .037
(Kin vs. Nonkin)
Preferred Language of Parent −.208 .098 −.097*
Ethnicity Contrasts
     1. Caucasian vs. Latino −.058 .101 −.025
     2. African American vs. Latino −.065 .102 −.028
     3. Native American vs. Latino .359 .279 .038
     4. Mixed Ethnicity vs. Latino .000 .190 .000
     5. Asian/Pacific Islander vs. Latino −.211 .155 −.044
KEEP (Effect.) Inter. vs. Control (Group Status 1) −.245 .065 −.121***
KEEP SAY Imp. vs. Control (Group Status 2) −.557 .081 −.229***
Centered Baseline Child Behaviors .567 .030 .567***
Step 3
Child Age −.037 .029 −.037
Relationship Type .077 .060 .037
(Kin vs. Nonkin)
Preferred Language of Parent −.223 .098 −.105*
Ethnicity Contrasts
     1. Caucasian vs. Latino −.064 .101 −.028
     2. African American vs. Latino −.083 .101 −.036
     3. Native American vs. Latino .366 .277 .039
     4. Mixed Ethnicity vs. Latino .010 .189 .002
     5. Asian/Pacific Islander vs. Latino −.227 .154 −.048
KEEP (Effect.) Inter. vs. Control (Group Status 1) −.238 .065 −.118***
KEEP SAY Imp. vs. Control (Group Status 2) −.597 .082 −.246***
Centered Baseline Child Behaviors .676 .048 .675***
Group Status 1 × Centered Baseline −.124 .063 −.084*
Group Status 2 × Centered Baseline −.290 .083 −.125***
Open in a new tab
m

p < .10;

*

p < .05;

**

p < .01;

***

p < .001

3.2.2. Simple Slope Analyses

To explore these interactions further, simple regression lines were computed and statistically evaluated, one set of analyses for the KEEP effectiveness study intervention group vs. KEEP effectiveness study control group (group status 1) and one set for the KEEP SAY implementation trial group and the KEEP effectiveness study control group (group status 2). To begin, simple regression lines were computed for the relation between number of termination behavior problems and group status 1 (KEEP effectiveness study intervention group vs. KEEP effectiveness control group) at specific values of number of baseline behavior problems: one standard deviation below the mean, at the mean, one standard deviation above the mean, and two standard deviations above the mean, and 3 standard deviations above the mean. Baseline and termination scores were standardized. Thus, the mean that was used in these analyses was zero and the standard deviation was 4.07. The simple slope between number of termination behavior problems and group status at one standard deviation below the mean, b = −.11, was not statistically significant. However, a statistically significant and negative simple slope was found between number of termination behavior problems and group status for those at the mean level of baseline behavior problems, b = −.24, p =. 001. In addition, there was a statistically significant and negative simple slope between the number of termination behavior problems and group status for those at one standard deviation above the mean, b = −.36, p = .001, those at two standard deviations above the mean, b = −.49, p = .001, and those at three standard deviations above the mean, b = −.61, p = .001. In each case, relative to the control group, there was a significant reduction in termination behavior problems for children in the intervention group. Thus, as the number of baseline behaviors increased there was a greater reduction in the behavior problems in the intervention group in contrast to the control group. Figure 1 depicts the differences between the control and intervention groups for the number of termination behavior problems at a specific value (e.g., 1 SD above the mean) for baseline behavior problems.

Figure 1.

Figure 1

Open in a new tab

Differences between KEEP effectiveness study control and intervention groups for the number of termination behavior problems at a specific value for baseline behavior problems.

Next, simple regression lines were computed for the relation between number of termination behavior problems and group status 2 (KEEP SAY implementation trial group vs. KEEP effectiveness study control group) at specific values of number of baseline behavior problems: one standard deviation below the mean, at the mean, one standard deviation above the mean, and two standard deviations above the mean, and 3 standard deviations above the mean. The simple slope between number of termination behavior problems and group status at one standard deviation below the mean, b = −.31, was statistically significant, p = .004. A statistically significant and negative simple slope was also found between number of termination behavior problems and group status for those at the mean level of baseline behavior problems, b = −.60, p =. 001. In addition, there was a statistically significant and negative simple slope between the number of termination behavior problems and group status for those at one standard deviation above the mean, b = −.89, p = .001, those at two standard deviations above the mean, b = −1.18, p = .001, and those at three standard deviations above the mean, b = −1.47, p = .001. In each case, relative to the control group, there was a significant reduction in termination behavior problems for children in the intervention group. Thus, as the number of baseline behaviors increased there was a greater reduction in the behavior problems in the intervention group in contrast to the control group. Figure 2 depicts the differences between the comparison and intervention groups for the number of termination behavior problems at a specific value (e.g., 1 SD above the mean) for baseline behavior problems.

Figure 2.

Figure 2

Open in a new tab

Differences between comparison (KEEP effectiveness study control) and KEEP SAY implementation groups for the number of termination behavior problems at a specific value for baseline behavior problems.

4. Discussion

The primary goal of the current investigation was to examine the effectiveness of the KEEP intervention in reducing child behavior problems as it was being delivered by a community-based mental health provider during an implementation trial in San Diego County. A secondary goal was to determine whether the intervention was effective in reducing child behavior problems at various levels as presented at baseline. The results from regression analyses replicated the findings of the KEEP effectiveness study in demonstrating that the KEEP intervention was effective in reducing child behavior problems in the KEEP effectiveness study intervention group (Chamberlain, et al., 2008). Furthermore, the findings from the same analyses revealed that the KEEP intervention was also effective in reducing child behavior problems when it was implemented in San Diego County by a community service provider rather than a research-based entity. This pattern of findings suggests that as the KEEP intervention moves away from the intervention developers and into real world service settings the effectiveness of the intervention can be maintained. Moreover, not only was the effectiveness of the intervention maintained as it was adopted within a child welfare setting, but it was also found to be effective with a different composition of substitute caregivers. Whereas in the original KEEP effectiveness study 34% of caregivers were relatives (e.g., grandparents, aunts & uncles), in the KEEP SAY implementation trial, 53.8% of the caregivers were relatives. Despite these differences, the cumulative findings from the current investigation and the original KEEP effectiveness study (Chamberlain, Price, Reid, et al., 2008) indicate that the KEEP foster parent intervention is effective in reducing child behavior problems, regardless of the type of caregiver relationship.

An additional goal of this investigation was to determine whether the KEEP intervention was effective in reducing child behavior problems based on various levels of initial baseline behavior problems. That is, is the KEEP intervention effective in reducing child behavior problems, regardless of the initial level of child behavior problems observed at baseline? It was hypothesized that the KEEP intervention, as delivered during both the effectiveness study and during the implementation trial, would be effective in reducing child behavior problems at termination for children with behavior problems at or above the mean in baseline behavior problems as assessed by the PDR. The results of the regression analyses revealed significant effects for the interactions between group status (intervention vs. control/comparison) and baseline behavior problems. Follow-up simple slope analyses, one set of analyses for the effectiveness study comparisons and one set for the implementation trial comparisons, revealed that the KEEP intervention was effective in reducing children behavior problems at various levels of baseline behavior problems, including at the mean level of baseline behavior problems (5.92 for the intervention group in the KEEP effectiveness study and 4.85 for the KEEP implementation study) and continuing up through levels of behavior problems as high as three standard deviations above the mean. During the implementation trial the intervention was also effective in reducing child behavior problems at levels as low as one standard deviation below the mean at baseline. The particular relevance of the findings from the simple slope analyses is that they demonstrate that the KEEP intervention is effective in reducing behavior problems at the levels that place children at risk for placement disruptions. Using data from the control group of the KEEP effectiveness study,Chamberlain et al. (2006) found that for each behavior above 6 behavior problems there was a 25% increased risk for a negative placement disruption (e.g., foster parent initiates request for change of placement because of child behavior problems or caseworker determines that the placement is no longer a good fit). Thus, by helping foster parents and relative caregivers to manage the behavior problems of the children in their care, the KEEP foster parent intervention has the potential to reduce the risk for negative placement disruptions and reduce the personal and economic burdens that result from placement disruptions.

4.1. Study Limitations

One of the limitations of current investigation was the demographic differences between the participants in the KEEP effectiveness study and those who took part in the implementation trial. Even though the participants in both groups were from the same geographic regions within San Diego County, the samples differed in several ways (e.g.., proportion of foster parent vs. relative caregivers, proportion of English vs. Spanish speakers, ethnic composition, and age of the children). As noted earlier, group differences on relative proportion of type of caregiver (foster vs. relative) was attributable to the fact that the community agency delivering the KEEP intervention in this study (SAY San Diego) serviced relative substitute caregivers caring for children who were not dependents of San Diego County Child Welfare Services. Thus, in addition to recruiting caregivers with children who were dependents of child welfare, eligible relative caregivers served by SAY San Diego with children who were not dependents were also recruited into the intervention during implementation. Consequently, a higher proportion of relative caregivers were recruited into the KEEP intervention during implementation. In contrast, in the original KEEP effectiveness study only families (both nonrelative and relative) who were caring for child welfare dependents were recruited into the study. It is possible that the inclusion of the relative givers served by SAY San Diego to the KEEP SAY implementation trial contributed to the demographic differences between the samples.

Another limitation of the current study was the internal validity threat of history (Cook & Campbell, 1979), in particular local history, in contributing to differences between the historical comparison group (KEEP effectiveness study control group) and the KEEP implementation trial group. It is possible that foster and kinship caregivers who participated in the KEEP intervention following implementation received services unavailable to the foster and kinship caregivers who participated in the earlier KEEP effectiveness study, such as expanded pre-service or in-service training. However, regular communications with our contacts at San Diego Child Welfare Services over the period of KEEP effectiveness study (1999 to 2004) and the implementation trial (2005 – 2008) did not reveal any substantial changes to basic pre-service or in-service training for foster parents.

4.2. Conclusions

First, within the context of the Cascading Dissemination Model, the results of this investigation suggest that as the KEEP intervention moves away from the intervention developers, paraprofessionals from a community agency that are given adequate training and supervision can deliver the intervention to foster and relative caregivers in a manner that is effective in reducing behavior problems of children in foster care,. As mentioned earlier, the group facilitators hired by SAY San Diego received extensive training in the KEEP intervention model, the curriculum, and the management of group processes. This training was conducted by personnel trained by the intervention developers and took place over several weeks prior to start of the first parenting groups. In addition, all group sessions were video recorded and reviewed by the group facilitator and the clinical supervisor. Also, group facilitators contacted the parents in their group each week to assess levels of child behavior problems (via the Parent Daily Report – PDR) and to discuss application of session material. Group facilitators and the clinical supervisor met each week to discuss parents’ progress in managing child behavior problems, the delivery of session material, and group processes during the prior session. It is within this context that the KEEP intervention was found to be effective in reducing child behavior problems, and at various levels of initial behavior problems. The training and supervision procedures were not burdensome for the group facilitators and likely contributed to the effectiveness of the intervention. The results of the implementation trial suggest that it is feasible to have the KEEP intervention delivered by a community mental health provider within a child welfare system of care and that it can be delivered in a manner that leads to reductions in levels of children’s externalizing behavior problems. Such reductions are likely to decrease the risk for unwanted placement changes in foster care (Chamberlain et al., 2006; Newton et al., 2000).

Second, this study illustrates a research strategy for examining the effectiveness of an evidence-based practice as it is implemented in a child welfare service setting by utilizing a research design that integrates data from a prior effectiveness study of the intervention and the data collected during the implementation of the intervention. With this design, the control group from an earlier efficacy or effectiveness study can provide a ready comparison group when it may not be possible to obtain a control or comparison group during the implementation of an intervention. Researchers and service providers might even consider collaborating in conducting a randomized effectiveness trial in conjunction with a non-randomized implementation of an intervention in a community setting. The data from this research could be analyzed together to provide results on the effectiveness of the intervention under two types of conditions; (a) one with the delivery and monitoring of the intervention carried out by researchers, and (b) the other with delivery and monitoring carried out by the community providers and/or the service agency personnel who are considering adopting the intervention. The findings generated from this type of design would address the effectiveness of the intervention and provide valuable information on the potential challenges and barriers to large scale implementation of the intervention.

Highlights.

  • Effects of the KEEP foster parent intervention examined during implementation.

  • The Cascading Dissemination model was used as the conceptual model.

  • Quasi-experimental design was utilized.

  • During implementation trial KEEP intervention reduced child behavior problems.

  • Behavior problems reduced among children displaying various levels of problems.

Acknowledgements

Support for the KEEP effectiveness trial was provided by Grant No. MH 60195 from the Child and Adolescent Treatment and Preventive Intervention Research Branchy awarded to Dr. Patti Chamberlain. Support for the KEEP SAY implementation trial was provided by a grant from the State of California awarded to San Diego County Child Welfare Services. The authors would like to thank San Diego County Child Welfare Services Directors: Yvonne Campbell, Mary Harris, and Debra Zonders-Willis; Deputy Directors: Patty Rahiser, Renee Smiley, and Roseann Myers; Social Advocates for Youth Supervisor: Shannon Throop; Research Project Directors: Courtenay Paulic, Jan Price, and Norma Talamantes; OSLC consultant JP Davis; lead interventionists Norma Talamantes, Melissa Woods, Moniesha Cole, and Sonia Miramontes; and the foster and relative caregivers who participated in these studies.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Aarons GA, James S, Monn AR, Raghavan R, Wells RS, Leslie LK. Behavior problems and placement change in a national child welfare sample: A Prospective study. Journal of the American Academy of Child & Adolescent Psychiatry. 2010;49:70–80. doi: 10.1097/00004583-201001000-00011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Becker WC, Madsen CH, Jr, Arnold CR, Thomas DR. The contingent use of teacher attention and praise in reducing classroom behavior problems. Journal of Special Education. 1967;1:287–307. [Google Scholar]
  3. Chamberlain P, Leve L, DeGarmo DS. Multidimensional treatment foster care for girls in the juvenile justice system: 2-year follow-up of a randomized clinical trial. Journal of Consulting and Clinical Psychology. 2007;Vol. 75(1):187–193. doi: 10.1037/0022-006X.75.1.187. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Chamberlain P, Moreland S, Reid K. Enhanced services and stipends for foster parents: Effects on retention rates and outcomes for children. Child Welfare. 1992;71:387–401. [PubMed] [Google Scholar]
  5. Chamberlain P, Price JM, Reid JB, Landsverk J, Fisher PA, Stoolmiller M. Who disrupts from placement in foster and kinship care? Child Abuse and Neglect. 2006;30:409–424. doi: 10.1016/j.chiabu.2005.11.004. [DOI] [PubMed] [Google Scholar]
  6. Chamberlain P, Price JM, Reid JB, Landsverk J. Cascading implementation of a foster and kinship parent intervention. Child Welfare: Journal of Policy, Practice, and Program. 2008;87:27–48. [PMC free article] [PubMed] [Google Scholar]
  7. Chamberlain P, Price JM, Leve L, Laurent H, Landsverk J, Reid JB. Prevention of behavior problems for children in foster care: Outcomes and mediation effects. Prevention Science. 2008;9:17–27. doi: 10.1007/s11121-007-0080-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Chamberlain P, Reid JB. Parent observation and report of child symptoms. Behavioral Assessment. 1987;9:97–109. [Google Scholar]
  9. Chamberlain P, Reid JB. Using a specialized foster care community treatment model for children and adolescents leaving the state mental hospital. Journal of Community Psychology. 1991;19:266–276. [Google Scholar]
  10. Chamberlain P, Reid JB. Differences in risk factors and adjustment for male and female delinquents in treatment foster care. Journal of Child and Family Studies. 1994;3(1):23–39. [Google Scholar]
  11. Cook TD, Campbell DT. Quasi-experimentation: Design and analysis issues for field settings. Boston: Houghton Mifflin Co; 1979. [Google Scholar]
  12. Costin J, Chambers SM. Parent management training as a treatment for children with oppositional defiant disorder referred to a mental health clinic. Clinical Child Psychology and Psychiatry. 2007;12:511–524. doi: 10.1177/1359104507080979. [DOI] [PubMed] [Google Scholar]
  13. Eddy JM, Chamberlain P. Family management and deviant peer association as mediators of the impact of treatment condition on youth antisocial behavior. Journal of Consulting and Clinical Psychology. 2000;68:857–63. doi: 10.1037/0022-006X.68.5.857. [DOI] [PubMed] [Google Scholar]
  14. Forgatch MS, Toobert DJ. A cost-effective parent training program for use with normal preschool children. Journal of Pediatric Psychology. 1979;4:129–145. [Google Scholar]
  15. Kazdin AE, Wassell G. Therapeutic changes in children, parents, and families resulting from treatment of children with conduct problems. Journal of the American Academy of Child & Adolescent Psychiatry. 2000;39:414–420. doi: 10.1097/00004583-200004000-00009. [DOI] [PubMed] [Google Scholar]
  16. Landsverk J, Garland A. Foster care and pathways to mental health services. In: Curtis P, Dale G, editors. The foster care crisis: Translating research into practice and policy. Lincoln, NE: University of Nebraska Press; 1999. pp. 193–210. [Google Scholar]
  17. Landsverk J, Garland AF, Leslie LK. Mental health services for children reported to child protective services. In: Myers JEB, Berliner L, Briere J, Hendrix TC, Jenny C, editors. The APSAC handbook on child maltreatment. 2nd ed. Thousand Oaks, CA, US: Sage Publications, Inc; 2002. pp. 487–507. [Google Scholar]
  18. Leve LD, Chamberlain P. Female juvenile offenders: Defining an early-onset pathway for delinquency. Journal of Child & Family Studies. 2004;13:439–452. [Google Scholar]
  19. Maxfield M, Widom C. The cycle of violence: Revisited six years later. Archives of Pediatric Adolescent Medicine. 1996;150:390–395. doi: 10.1001/archpedi.1996.02170290056009. [DOI] [PubMed] [Google Scholar]
  20. McClowry SG, Snow DL, Tamis-LeMonda CS. An evaluation of the effects of INSIGHTS on the behavior of inner city primary school children. Journal of Primary Prevention. 2005;26:567–584. doi: 10.1007/s10935-005-0015-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. National Survey of Child and Adolescent Well-Being Group. National survey of child and adolescent well-being (NSCAW One Year in Foster Care Wave 1 Data Analysis Report) Washington, DC: U.S. Department of Health and Human Services, Administration for Children and Families; 2003. [Google Scholar]
  22. Newton RR, Litrownik AJ, Landsverk JA. Children and youth in foster care: Disentangling the relationship between problem behaviors and number of placements. Child Abuse and Neglect. 2000;24:1363–1374. doi: 10.1016/s0145-2134(00)00189-7. [DOI] [PubMed] [Google Scholar]
  23. Patterson GR. The aggressive child: Victim and architect of coercive system. In: Hamerlynck LA, Handy LC, Mash EJ, editors. Behavior modification and families: !. Theory and research. 11. Applications and developments. New York: Brunner/Mazel; 1976. pp. 267–316. [Google Scholar]
  24. Patterson GR, DeBaryshe BD, Ramsey E. A developmental perspective on antisocial behavior. American Psychologist, Vol. 44(2), Special issue: Children and their development: Knowledge base, research agenda, and social policy application. 1989:329–335. doi: 10.1037//0003-066x.44.2.329. [DOI] [PubMed] [Google Scholar]
  25. Price JM, Chamberlain P, Landsverk J, Reid JB, Leve L, Laurent H. Effects of a foster parent training intervention on placement changes of children in foster care. Child Maltreatment. 2008;13:64–75. doi: 10.1177/1077559507310612. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Reid JB, Eddy JM. The prevention of antisocial behavior: Some considerations in the search for effective interventions. In: Stoff DM, Breiling J, Maser JD, editors. Handbook of antisocial behavior. New York: John Wiley & Sons, Inc; 1997. pp. 343–356. [Google Scholar]
  27. Smith C, Thornberry TP. The relationship between childhood maltreatment and adolescent involvement in delinquency. Criminology. 1995;33:451–481. [Google Scholar]
  28. Stein E, Evans B, Mazumdar R, Rae-Grant N. Then mental health of children in foster care: A comparison with community and clinical samples. Canadian Journal of Psychiatry. 1996;41:385–391. doi: 10.1177/070674379604100610. [DOI] [PubMed] [Google Scholar]
  29. Trupin EW, Tarico VS, Low B, Jemelka R, McClellan J. Children on child protective service caseloads: Prevalence and nature of serious emotional disturbance. Child Abuse & Neglect. 1993;17:345–355. doi: 10.1016/0145-2134(93)90057-c. [DOI] [PubMed] [Google Scholar]
  30. Stone AA, Broderick JE, Kaell AT, DelesPaul PAEG, Porter LE. Does the peak- end phenomenon observed in laboratory pain studies apply to real-world pain in rheumatoid arthritics? Journal of Pain. 2000;1:212–217. doi: 10.1054/jpai.2000.7568. [DOI] [PubMed] [Google Scholar]

RESOURCES