Advancing prediction of foster placement disruption using Brief Behavioral Screening

Abstract

Objective

Behavioral difficulties increase the risk that children will experience negative placement disruptions while in foster care. Chamberlain et al. (2006) found that the Parent Daily Report (PDR), a brief measure of parent-reported child behaviors, was a strong predictor of negative placement changes over one year among children receiving “usual case work” services. This paper sought to replicate and extend original findings regarding the PDR among 359 foster parents participating in a group parent-training intervention.

Methods

Foster parents of children experiencing a recent foster placement, and taking part in the KEEP parenting program, were included in analyses. Foster parents completed 16 weekly PDR calls about the behavior of a foster child in their care during the KEEP intervention and about their stress related to the child’s behaviors. Multiple strategies, including latent class analysis of weekly PDR counts and continuous moving averages of PDR counts over shorter time frames, were used to test improvements in prediction of negative placement changes.

Results

Consistent with prior findings, children with elevated PDR ratings and children living with non-relative foster parents had significantly higher levels of negative placement disruptions. Prediction improved with decision rules relying upon increased amounts of weekly PDR information, although good prediction was achieved with 3–5 weeks of PDR information. Parent-reported stress associated with behavior did not improve prediction.

Conclusions

This study confirmed the potential utility of the PDR as a predictor of negative placement changes and illustrates how longitudinal PDR information may aid in improving such prediction. Potential applications of the PDR for improving the timing, type, and quantity of services offered to help foster parents prevent placement disruptions are discussed.

Introduction

Each year, nearly half a million children reside in formal foster care as a result of maltreatment (US Department of Health and Human Services, 2008). Foster parents play a substantial role in fulfilling child welfare responsibilities to promote the safety, wellbeing, and stability of children removed from their homes. Often foster parents are faced with behavioral challenges among children in their care (Clausen, Landsverk, Ganger, Chadwick, & Litrownik, 1998; Leslie, Hurlburt, Landsverk, Barth, & Slymen, 2004; Pilowsky, 1995), which places strain on foster parents and raises risks for placement disruption. Multiple studies have found that externalizing behavioral difficulties are associated with higher levels of placement change (Chamberlain et al., 2006; Newton, Litrownik, & Landsverk, 2000) and that behavior-related placement changes constitute a significant portion of all placement changes (James, 2004). In one of the most detailed examinations of this issue, Chamberlain et al. (2006) found that the day-to-day occurrences of common child behavior problems as measured by the Parent Daily Report (PDR), a brief (5–10 minute) telephone interview of daily behavior problems, forecasted significant increases in risk for negative foster placement disruptions over a 1 year period. The frequency of behavior-related placement changes and the predictive ability of the PDR present an opportunity to identify risks for placement disruption early and to direct training and services to foster parents and children more efficiently. To explore this potential, this study sought to replicate findings regarding the strength of the PDR---placement disruption relationship and to test several methods for increasing its predictive accuracy, including making use of additional information about parent stress associated with behavior problems and longitudinal versus point-in-time behavioral information.

Placement stability is an important overarching objective for children residing in substitute care. It is a key indicator in ongoing federal Child and Family Service Reviews. Repeated placement changes frequently sever key relational ties (Barber & Delfabbro, 2005), may contribute to academic disruption and difficulty (Runyan & Gould, 1985), and may lead to less coordinated medical care (Rubin, Alessandrini, Feudtner, Localio, & Hadley, 2004). A number of independent studies have found a strong relationship between behavioral difficulties and placement changes (Barber, Delfabbro, & Cooper, 2001; Chamberlain et al., 2006; James, 2004; Pardeck, 1985). Evidence suggests that placement changes can compound already challenging behavioral problems such as aggression, low self concept, and coping problems (Harden, 2004; Herrenkohl, Herrenkohl, & Egolf, 2003; Newton et al., 2000). Children with a large number of placement changes are also at higher risk for running away from placements, spending greater time in residential facilities, and involvement with other service sectors such as juvenile justice (Harden, 2004; James, 2004; James, Landsverk, Slymen, & Leslie, 2004; Jonson-Reid & Barth, 2000). For children, many of whom have experienced extensive exposure to uncertainty and multiple family risks, placement disruptions may be experienced as a continuation of distress and lack of normative social support.

Disruptions in placements also affect the child welfare service structure. Difficulty in managing the behavioral challenges of children can place great stress on foster parents and likely on the qualitative character of foster parent-child relationships. The stress associated with behavioral concerns and movement of children into and out of foster homes can contribute to foster parent burnout and loss of foster caregivers (Boyd & Remy, 1978). Changes in children’s placements are also costly in terms of staff time, including caseworkers, placement specialists, and other caregivers who may be called upon to care for children during periods when alternative foster home care is being arranged. In their study in the UK of the costs associated with providing case work services to prepare for, find, and make placement changes, Ward and colleagues found that as children experienced more failed placements they became increasingly difficult to place and the costs of finding each new placement increased exponentially such that finding fourth, fifth, or sixth placements took 3–4 times longer than finding a first or second placement and cost an average of 6 times as much (Ward, 2007).

If child welfare systems could effectively predict placements at risk for negative disruption, this could contribute to more timely delivery of services to ameliorate such risks and reduce the frequency of repeated placement disruptions. Effective, evidence-based programs designed to support foster parents already exist. One program in particular---KEEP (Keeping Foster Parents Trained and Supported)---has been tested specifically with foster parents and resulted in reduced child behavioral problems, increased likelihood of positive placement changes (e.g., reunification), and reduction in foster home placement changes among children with high numbers of prior placements (Chamberlain, Leve, & DeGarmo, 2007; Price et al., 2008). The KEEP intervention emerges from a tradition of social-learning focused parent-training interventions (Hurlburt, 2007) such as Parent Management Training (Kazdin, Esveldt-Dawson, French, & Unis, 1987; Patterson, Chamberlain, & Reid, 1982), Parent Child Interaction Therapy (Eyberg et al., 2001), the Incredible Years (Webster-Stratton, 1998), and is adapted from Multidimensional Treatment Foster Care (Chamberlain, Moreland, & Reid, 1992; Fisher & Chamberlain, 2000). It is a group-based program that focuses on providing foster parents with skills to more effectively address child behavioral challenges as well as information about normative developmental expectations that may change foster parents’ perceptions of children’s behavior. Effective advance prediction of placement disruption could add to the public health potential of such services by targeting services to foster parents and children with need in a more timely fashion.

In the context of a large-scale randomized trial of the KEEP intervention, the relationship of the PDR with negative placement changes was examined over a 12-month period (Chamberlain et al., 2006). Analyses focused on predicting placement disruptions for 246 children in the control group using foster parent report (on the PDR) of child behavioral difficulties at study entry. Control group participants were receiving case work “as usual” services, not the KEEP intervention. Results showed that the likelihood of experiencing a negative placement change over a 12-month period was approximately 8% for children with up to 6 problem behaviors per day measured on 3 closely spaced days. Beyond 6 behaviors, the likelihood of placement disruption increased steadily and significantly with each additional reported behavior. Children reported as having 10 or 14 behavior problems in the last 24 hours (approximately 1 and 2 standard deviations above the cutoff of 6) would have an estimated odds of placement disruption of 2.4 and 6.0 times higher, respectively. The PDR obtained a specificity of .62 and sensitivity of .57 for predicting negative placement disruptions. These results suggested that the PDR could contribute to assessing risk for placement disruption, but improvements in the strength of the relationship might enhance its ability to aid in targeting services effectively.

This paper attempts to replicate the general pattern of relationships reported in Chamberlain et al. (2006), and test whether several possible additional sources of information improve prediction of placement disruption. Specifically, we examined whether PDR information, gathered over multiple weeks, improves prediction of placement outcomes among 5–12 year old children beyond initial PDR levels, and whether foster parents’ self-reported levels of stress associated with children’s daily reported behaviors improves prediction. Data for this study were collected as part of the original KEEP effectiveness trial but these analyses focus on children and foster parents who participated in the intervention condition in that study (as contrasted with those in the control group reported in Chamberlain et al., 2006).

Methods

Participants

The KEEP study was a collaboration among the San Diego County Department of Health and Human Services (DHHS), the Oregon Social Learning Center (OSLC), and the Child and Adolescent Services Research Center (CASRC). Recruitment of participants was facilitated by data systems from the social service agency that were reviewed on a weekly basis to identify eligible children and foster families. The final sample for the full study included eligible families that agreed to random assignment, completed signed informed consent procedures, and participated in a baseline assessment. Eligible participants included all foster and kinship parents receiving a new placement of a child from the San Diego County DHHS child welfare system between 1999 and 2004. Eligibility requirements were (a) the child had been in either a kin or nonkin foster care placement for a minimum of 30 days; (b) the child was between the ages of 5 and 12; and (c) the child was not considered “medically fragile” (that is, not severely physically or mentally handicapped). Once deemed eligible, families were randomly assigned to either the intervention or to the control (case worker services as usual) condition. The resulting RCT full sample comprised 700 foster child families, 359 intervention and 341 control services “as usual” (34% kinship placements, 66% non-relative placements). The sample was ethnically diverse---21% African American, 33% Latino, 22% Caucasian, 22% mixed ethnicity, 1% Asian American, and 1% Native American children. All study participants gave informed consent and the work was approved by the Institutional Review Board at San Diego State University.

Measures

The focal time series measures in the present study were obtained from the Parent Daily Report Checklist (PDR; Chamberlain & Reid, 1987). PDR data were collected at baseline to assess base rate levels of behaviors using three aggregated calls. In addition, weekly PDR data (1 day per week) were then collected for those randomly assigned to the intervention condition, during the 16-week intervention period. Analyses here will focus on the baseline levels and the repeated time series data. Models also controlled for demographic characteristics of the sample including child age measured in years, caregiver kinship status, gender of the child, and race-ethnicity.

The PDR child behavior problems index is a 30-item measure of behavior problems. PDR measures were conducted by telephone immediately after study enrollment on a series of consecutive or closely spaced days (1–3 days apart). Assessment interviewers were blind to random assignment. A trained interviewer asked the parent “Thinking about (child’s name), during the past 24 hours did any of the following behaviors occur?” Parents were asked to recall only the past 24 hours and to respond “yes” or “no” (i.e., the behavior happened at least once or did not occur). Some of the kinds of behaviors asked about in the PDR include “arguing,” “hitting,,” “whining,” “back talk,” and “swearing.”

Chamberlain and colleagues reported inter-call alpha reliability of .84 and .83 at baseline and intervention termination respectively (Chamberlain et al., 2008). A validation study found that PDR reports were correlated with observations of in-home behavior measured at a similar time (Chamberlain & Reid, 1987), however it should be noted that reports of child behavior are likely to be influenced to some degree by variations in how foster parents perceive children’s behavior in addition to the actual behaviors that occur.

Caregiver stress was assessed following the reporting of each behavior that occurred in the last 24 hours. For each behavior, caregivers were asked “How much did his/her (behavior) upset you?” Caregivers responded on a scale of 1 (not at all) to 3 (quite a bit).

Negative foster placement disruption was defined as any exit from the foster or kinship placement home that was made for a negative reason. Foster parents were telephoned at 4 and 12 months post-baseline to determine if the child remained in their home or had moved. Negative exits included foster parent requests that a child be moved due to behavior problems, caseworker or foster parent judgments that the child needed a more intensive or restrictive level of care, child runaways, or caseworker determination that the child was too difficult for the foster/kin family to manage.

Analytic strategy

Two related logistic regression approaches were utilized. The first approach utilized multivariate logistic regression to test several PDR thresholds for predicting negative placement outcomes, in the presence of demographic covariates. Initial analyses utilized a threshold of 6, comparable with the threshold identified in Chamberlain et al. (2006). It was expected that higher thresholds might also need to be studied as well because analyses were conducted in the intervention condition, where drops in reported behavior problems were expected to occur during the intervention. The second basic approach examined prediction of negative placement outcomes based on time-varying PDR trajectories. Categorical groupings of trajectories were identified with growth mixture modeling in MPlus5.2 (Muthén & Muthén, 2007). Categorical PDR trajectories identified were then tested as predictors in multivariate logistic regression models. Parental stress was added as a separate predictor. Finally, shorter algorithms for identifying risk of placement disruption were tested to understand further the potential field applicability of the PDR. All analyses focused on 304 children for whom at least 1 PDR observation was available in the first 3 intervention weeks.

Results

Across 3 closely spaced PDR calls following study enrollment, the average PDR count in the intervention condition was 6.00 (SD = 4.20). This average differs slightly from that reported in Chamberlain et al. (2008) due to small differences in the analytic sample studied. In initial logistic regressions of negative placement disruptions on the PDR, child age, child gender, child race/ethnicity, relationship of the foster parent to the child, and number of other children in the household, the PDR was not a significant predictor of negative placement outcomes as a continuous variable (OR = 1.06, p > .10) or as a dichotomous variable with a cutoff score of 6 or above to indicate risk of disruption (OR = 1.41, p > 0.10). As discussed later, this is thought to be due to a lowering of child behavioral difficulties over time that was associated with being in the KEEP intervention group (Chamberlain et al., 2008). Therefore, the multivariate model was re-run using several cutoff scores between 7 and 10. The PDR became a stronger predictor at baseline as the cutoff score was raised above 6. Model 1 in Table 1 summarizes the relationship between the PDR and negative placements with a cutoff score of 9 to indicate risk of negative placement. In the model, both the foster parent relationship to the child (non-relative vs. relative) and the PDR were significant predictors of future negative placement disruptions. Children living with non-relative foster parents had 3 times the odds of experiencing a negative placement disruption as children living with kin foster parents. Children with average PDR counts above 9 at study entry (Baseline) (22.8% of children) had 2.55 times the odds of experiencing a negative placement disruption as children with PDR counts of 9 or lower (77.2% of children). Negative placement disruptions occurred in 8.9% of cases over the 12 month follow-up for children below the threshold and 20.9% of cases above the threshold. A baseline cutoff of 9 resulted in sensitivity of .41 and specificity of .79, which was the best combination of sensitivity and specificity for cutoffs between 7 and 10.

Table 1.

Logistic Regression of Negative Placement Outcome on Parent Daily Report Counts/Trajectories and Other Case Characteristics

	Model 1 (PDR cutoff >9) (n=292)			Model 2 (3 Latent Classes) (n=294)			Model 3 (4 Latent Classes) (n=294)
	β	SE	OR	β	SE	OR	β	SE	OR
Intercept	−4.18	1.12	0.02	−4.36	1.14	0.01	−4.62	1.18	0.01
PDR	0.94	0.40	2.55*	1.56	0.39 4.77***		1.63	0.40	5.11***
Number of children in home	0.02	0.11	1.02	0.03	0.11	1.03	0.01	0.12	1.01
Age	0.07	0.09	1.07	0.06	0.09	1.06	0.08	0.09	1.08
Caregiver relationship1	1.30	0.56	3.68*	1.20	0.57	3.32*	1.23	0.57	3.41*
Gender2	−0.02	0.38	0.98	−0.05	0.39	0.95	−0.09	0.39	0.92
Child Race3
Black	1.12	0.79	3.06	1.35	0.81	3.86	1.33	0.81	3.76
Hispanic	0.14	0.47	1.15	0.02	0.48	1.02	0.36	0.48	1.44
Other	0.11	0.50	1.12	0.04	0.51	1.04	0.15	0.51	1.17
−2 Log Likelihood		194.03			183.18			182.50

Note. SE = standard error; OR = odds ratio.

¹reference category = kinship caregivers;

²reference category = female;

³reference category = White

^*p < .05,

^**p < .01,

^***p < .001

Although slightly different in terms of the level of the threshold observed, the pattern of findings conforms closely with the pattern observed in the original study control group (Chamberlain et al., 2006) with regard to risks of negative placement outcomes increasing primarily above an identified threshold of behavioral difficulty, and for children placed with non-relatives.

Behavioral trajectories

Going beyond results that included information from the average of 3 baseline PDR calls, the weekly PDR data gathered from foster parents in the intervention condition (16 calls) allowed for exploration of how behavior trajectories might improve prediction of negative placement changes. PDR behavior counts do have intra-individual variability over time. For example, children may enter a placement with substantial individual differences in reported behavior problems. Over time, reports of problem behavior may remain high for some children. In other cases, interventions may have a favorable impact, with reports of child behavior difficulties decreasing over time. Some children with low initial reported behavior problems may remain so, and in other cases there may be a reported increase in behavior problems. Focus on mean level scores masks this variation and chronology of responses.

Growth mixture models (GMM) were specified for the 16-week PDR count data. GMM is a special case of latent class analysis (Muthén & Muthén, 2007), focusing on categorical groupings or clusters of individuals exhibiting similar profiles over time. Models assessed whether variation in PDR scores over time could be meaningfully described as arising from a mixture of multiple underlying trajectories to which different individuals belonged. To assess predictive validity, latent class identifications were computed for each child and included in logistic regression models to predict negative placement changes.

To determine the number of trajectory groupings, unconditional models estimating between two and five latent class trajectories were run separately with MPlus. For each model, the first week PDR count was set as time zero, with time represented as the number of weeks from the initial week for all subsequent PDR counts. PDR counts were treated as continuous variables. Weighing both statistical and pragmatic considerations, models containing three and four latent classes were considered the most promising for prediction of negative placement outcomes. Deciding the optimal number of classes or trajectory clusters involves comparison of several criteria (Jung & Wickrama, 2008; Nylund, Asparouhov, & Muthén, 2007) including the BIC (Bayesian Information Criteria) and AIC (Akaike’s Information Criteria) and the VLMR (Vuong-Lo-Mendell-Rubin) likelihood test to determine if the specified number of classes is a better fit than a model with one fewer class. In addition to the two information criteria, we relied on the VLMR likelihood ratio test and class interpretability to decide upon an appropriate number of classes to represent the PDR trajectories.

Figure 1 illustrates estimated PDR counts for a model solution that identifies behavior count trajectories as belonging to one of four latent classes. According to our criteria for evaluating improvements in model fit, the 3 and 4 class solutions both fit the data better than a 2 class solution (VLMR, p = .067 for 3 vs. 2 classes; VLMR, p = .061 for 4 vs. 3 classes), but adding a fifth class added little to representation of the PDR data (VLMR, p = .692). For brevity, only the 4-class solution is shown in Figure 1. The 3 and 4 latent class solutions had strong similarities. First, a large percentage of children in both solutions fell into a low PDR count trajectory. Children in the low trajectory class had average PDR counts that typically started relatively low (< 6) and stayed consistently moderate to low across the 16-week period. In both solutions, a different identifiable group of children had consistently high PDR counts that averaged at least 11 behaviors per day. Between the high and low groups, the three and four class solutions differed in whether cases were identified as belonging to a single group with a relatively constant average over time, or to 2 groups, one with increasing behavior counts and 1 with decreasing counts, as in Figure 1.

Figure 1.

Four Trajectories of Parent Daily Report Counts Over 16 Weeks Identified With Growth Mixture Modeling

From the perspective of predicting negative placements, both solutions had similar effectiveness. Table 1 also shows logistic regression models (2 and 3), in which all individuals in elevated PDR classes (classes 1, 2, and 3) were combined with one-another and compared with individuals in the lowest PDR trajectory class (class 4) with respect to negative placement outcomes. Elevated classes were combined because all elevated classes were significantly more likely to experience a negative placement disruption than the lowest trajectory group. For example, the odds ratio was 4.66 for the consistently high PDR trajectory class relative to the low PDR trajectory class. Odds ratios associated with the PDR trajectories were notably higher than those utilizing only the initial PDR average and differences in log-likelihood between these models and Model 1 indicate that they represent significant improvements in prediction [X²(1) = 10.85, p < .01; X²(1) = 11.53, p < .01]. Technically models 2 and 3 have no difference in their degrees of freedom with Model 1 because they involve testing different variations of the same conceptual predictor (PDR level). Model differences observed would be significant at the p < .01 level for a single degree of freedom test, suggesting that the PDR trajectory classes clearly result in reliable improvements in prediction.

Parent reported stress

Parent reported stress related to problem behaviors was also considered for its potential additional contribution to prediction of negative placement changes. The correlation of stress with counts of problem behaviors was consistently very high, as indicated by correlations of .93, .92, and .92 in weeks 1, 2, and 3 respectively of the KEEP intervention. Scatterplots confirmed a strong linear relationship between the 2 variables in every intervention week. The high correlations suggested that stress might have little room to add to prediction of negative placement changes beyond reports of behavior counts, but several approaches were developed for including parental stress into the multivariate models.

First, a linear model of stress was developed, based on prediction from PDR behavior counts. The simple regression of stress on PDR counts suggested that each behavior resulted in an average of 1.8 points of increase on parent reported stress. The strength of this relationship was very similar to that in the control condition. A residual stress variable was calculated in each week, representing parent-reported stress above or below that expected based on the number of behaviors reported. An average residual stress score was computed across the 16 intervention weeks, with higher scores indicating higher than expected stress and lower scores indicating lower than expected stress. Model 4 in Table 2 includes residual stress (mean = 0.09 and SD = 2.28) as an additional predictor of negative placement changes; it did not result in improved prediction of negative placement changes beyond the PDR trajectories from the 4-class model [X²(1) = 0.10, p > .1].

Table 2.

Logistic Regression of Negative Placement Outcome on Parent Daily Report Trajectories, Parental Stress, and Other Case Characteristics

	Model 4 (Stress – Average residual) (n=292)			Model 5 (Stress – Proportion high stress) (n=292)
	β	SE	OR	β	SE	OR
Intercept	−4.60	1.18	0.01	−4.90	1.39	0.01
Dichotomized PDR class1	1.63	0.40	5.08***	1.61	0.41	4.98***
Number children in home	0.01	0.12	1.02	0.02	0.12	1.02
Age	0.08	0.09	1.08	0.09	0.09	1.09
Caregiver relationship2	1.20	0.58	3.33*	1.23	0.57	3.42*
Gender3	−0.09	0.40	0.91	−0.10	0.39	0.90
Child Race4
Black	1.31	0.83	3.70	1.28	0.82	3.60
Hispanic	0.36	0.48	1.43	0.33	0.49	1.40
Other	0.15	0.51	1.16	0.16	0.51	1.17
Average residual stress	−0.01	0.07	0.99
Average proportion high stress behaviors				0.60	1.47	1.82
−2 Log Likelihood		182.40			182.20

Note. SE = standard error; OR = odds ratio

¹reference category = low behavior PDR class;

²reference category = kinship caregivers;

³reference category = female;

⁴reference category = White

^*p < .05,

^**p < .01,

^***p < .001

A second formulation of parental stress focused on examining whether the types of behaviors in which children engaged added to prediction of negative placement outcome. Table 3 lists the PDR behaviors included in the treatment group version of the PDR and the percentage of occasions in which each behavior was identified as stressful given that the behavior was reported. Behaviors were divided into those with different likelihoods of generating parental stress – high (> = 65% of occasions); moderate (> = 35% and < = 64% of occasions), and low (< 35% of occasions). The average proportion of behaviors in the most stressful group was computed across intervention weeks for each child. Model 5 in Table 2 includes the average proportion of high stress behaviors as a predictor (mean = 0.38, SD = 0.17); it also did not contribute to prediction of negative placement outcomes relative to Model 2 [X²(1) = 0.30, p > .1].

Table 3.

Percentage of occasions on which PDR behaviors were identified as somewhat or very stressful by foster parents, given that the behavior occurred

Item	Behavior	%	Item	Behavior	%
1	Run away	0.0	17	Argue	64.1
2	Daydream	13.0	18	Have a school problem	65.1
3	Stay out late	15.4	19	Irresponsible	65.2
4	Sluggish	23.4	20	Tease/Provoke	65.7
5	Soil	25.0	21	Truant	66.7
6	Nervous/jittery	27.4	22	Lie	68.0
7	Competitive	27.9	23	Fight	69.4
8	Wet	28.6	24	Steal	70.0
9	Have a short attention span	31.7	25	Backtalk	71.3
10	Depressed/Sad	37.8	26	Swear/Use obscene language	71.4
11	Skip meals	38.6	27	Not mind	73.2
12	Jealous	41.4	28	Defiant	74.4
13	Complain	41.5	29	Have inappropriate sexual behavior	76.9
14	Irritable	46.7	30	Destructive	82.7
15	Boisterous/Rowdy	46.7	31	Use drugs/alcohol	na
16	Negative	58.6

Note. na = not applicable (due to no occurrences)

The PDR had 31 items in the version used with the intervention condition.

Pragmatic applications

In an applied environment, a decision maker might have access to more than an initial 1–2 weeks of PDR information, but often would not be able to make decisions about services based on long stretches of information as were utilized for estimating the PDR trajectories described above. In all of the latent class models that were tested, individuals in classes with trajectories higher than the lowest trajectory profile had substantially increased likelihood of a negative placement event. This suggested that, rather than any specific trajectory being decisive, possibly the presence of elevated PDR counts for an extended period at any point in time might be considered a signature characteristic distinguishing between cases with high and low risk for negative placement events, and that this might be detectable through an abbreviated algorithm relying on a smaller quantity of information. A final set of analyses examined several possible simple algorithms for detecting risks for negative placement events.

To begin, across the 16 weeks we examined the probability that individuals in each PDR trajectory group would have an average PDR count higher than a specific threshold during any 3-week period. To do this, we calculated 3-week moving average PDR counts for every child in the study. The probability that a child would have any consecutive 3-week PDR count average higher than 9 during the 16-week intervention period was 22.7%, 100%, 93.3%, and 100% for children in the low (1), decreasing (2), increasing (3), and high (4) trajectory classes of the four latent class solution, providing relatively high discrimination between the lowest PDR trajectory and the three elevated trajectory groups. Thus, having a 3-week average PDR count above 9 at any point was considered a signature event.

Imagining that the occurrence of such a signature event might act as a flag for considering services to aid the foster parent and child, we considered whether examining the average PDR count over the next 2 weeks after a signature event would further add to prediction of negative placements. Such information might be utilized, for example, if foster parents received a check-in call after any 3-week signature event, but were willing to observe for several more weeks before initiating services.

Table 4 summarizes 3 models for different methods of identifying signature events, with odds ratios for model parameters and overall model log-likelihood statistics. The first example focuses on average counts during the first 3 intervention weeks, the second on 3-week moving average PDR counts, and the third on 3-week moving average counts plus a 2-week follow-up average.

Table 4.

Hierarchical Logistic Regression of Negative Placement Outcome on Parent Daily Report Counts and Other Case Characteristics

	Average PDR (3 weeks) (n=294)	3-week Moving Average (n=294)	3-week Moving Average Plus 2- week Follow-up (n=294)
	OR	OR	OR
Intercept	0.02	0.01	0.01
PDR	1.84	3.27**	3.70**
Number children in home	1.02	1.04	1.02
Age	1.06	1.05	1.06
Caregiver relationship1	3.75*	3.07*	3.36*
Gender2	0.99	0.89	0.99
Child Race3
Black	3.14	4.19	4.33
Hispanic	1.12	1.20	1.28
Other	1.08	1.18	1.10
−2 Log Likelihood	197.43	190.86	188.24

Note. OR = odds ratio

¹reference category = kinship caregivers;

²reference category = female;

³reference category = White

^*p < .05,

^**p < .01,

^***p < .001

Comparison of the various models reveals a gradual increase in the strength of the relationship between behavior counts and negative placement outcomes, from an odds ratio of 1.8 obtained by averaging 1 call from 3 separate weeks together up to 3.70 obtained when utilizing a 2 stage screening process. The relatively simple algorithm utilized in the 2-stage screening process was significantly more predictive of negative placement disruption than the original Model 1 in Table 1 [X²(1) = 5.79, p < .05], but less predictive than the 4 latent class model [X²(1) = 5.74, p < .05]. Using this approach, to classification, 67.1% of children fell in the low risk group (6.6% of whom had a negative placement outcome) and 32.9% of children fell in the high risk group (21.4% of whom had a negative placement outcome). The fit of the model based on the 3-week moving average was nearly as good and may be seen as the most efficient use of multi-week PDR information. It too represented an improvement over Model 1. Sensitivity for this approach was .62 and specificity .70.

Discussion

The current study replicated findings from Chamberlain et al. (2006). Initial analyses revealed a pattern of results largely consistent with that earlier work. Higher PDR counts and living with a non-relative foster parent were associated with an increased likelihood of negative placement changes. Using only initial PDR scores taken at the time of study entry, a dichotomous PDR count of 9 or more behaviors was associated with a 2.55 times higher odds of a negative placement. Children living with non-kin caregivers again had notably higher odds (3.68) of having a negative placement change than children living with kin caregivers.

Parental stress due to children’s behavior did not contribute to prediction of negative placement events beyond reported behavior problems. Simple correlations showed that counts of child behavior had strong and linear relationships with the stress reported, illustrating directly the stress that foster parents feel when they report children having high levels of behavioral difficulties. These analyses suggest that asking about child behavior itself captures most of the information about stress caused by child behavior. The utilization of additional PDR information over time did, however, improve prediction of negative placement changes. Boundaries on the capacity of the PDR for prediction were explored by testing models that used different amounts of PDR information. In a limited information model, PDR counts taken from early weeks after study entry had some predictive power with respect to placement disruption. In contrast, utilizing up to 16 weeks of PDR data to establish a trajectory for each child resulted in the ability to identify children with 4 to 5 times higher odds of negative placement changes as children with low PDR count trajectories.

Although similar in pattern to results of Chamberlain et al. (2006), the findings of this study should be viewed as a related extension of that work. The pattern of results emerged in analysis of experiences of participants in the KEEP treatment condition. The similarity of findings increases confidence about the contribution that the PDR can make to prediction of negative placement disruptions, as it occurred in both study conditions. The exact cutoff for the PDR was observed to be slightly different, likely due in part to the fact that the KEEP intervention led to reductions in reported behavior problems over time. Still, the similarity of the pattern of findings with that from the KEEP control condition suggests that the findings do have broader relevance for families who are and are not receiving supportive interventions.

In replicating and extending the results of Chamberlain et al. (2006), this work has several policy implications with respect to a) the level of prediction that can be generated with different levels of information, b) specific definitions of risk for placement disruption, and c) how information about child behavioral difficulties might be used to inform services offered.

Models tested in this paper illustrate the incremental levels of improvement in prediction that emerge with the use of increasing amounts of information. An approach that routinely evaluates whether average PDR counts exceed a defined threshold (e.g., 6 or 9 behaviors) over several PDR calls is likely to fare significantly better than PDR information only from a single point in time.

Data from this study provide an enriched view of the variability in PDR behavior counts over time and patterns that place children at risk for negative placement change. Trajectory analyses clarified that dynamic changes in behavior counts do occur for subsets of children and highlighted methods for distinguishing between children in placement settings who have consistently low PDR counts relative to those who do not. These trajectories revealed the strongest relationships between behavior patterns and negative placement changes, however they are unlikely to be practical in application due to the amount of information used to identify them.

It is possible to imagine practical applications where decisions about delivery of services might need to be made in relatively short time frames (e.g., 3–4 weeks) in which a simple algorithm for continuous monitoring of PDR counts might alert child welfare workers to risks of placement disruption. In a simple form, elevated PDR counts above 9 represented a clear signature event indicative of heightened risk for negative placement change in this study. In fact, an abbreviated algorithm using a moving average of PDR counts over 3 weeks was a significantly better predictor of negative placement change than using only information from a single week or from only an initial 3 week period. Using an algorithm based on 5 weeks of information did not represent a further significant improvement. Therefore, this study suggests that much of the additional benefit of using PDR information over time as a predictive tool can be obtained through application of a relatively simple, easily applied algorithm that examines recent levels of reported child behavior.

From a resource allocation perspective, the ability to predict foster placements at risk for negative disruptions could result in an enhanced capacity for child welfare agencies to target services where needed. At one level, behavioral information could be used to offer and/or encourage foster parents to take part in an evidence-based parenting program such as KEEP. A number of interventions that focus on strengthening parenting skills, promoting strong parent-child relationships, and teaching parents how to address challenging behaviors have strong evidence supporting their effectiveness (Webster-Stratton, Reid, & Hammond, 2001, 2004). The KEEP intervention is one example that has been tested specifically with foster parents and been found to have multiple benefits (Price et al., 2008; Chamberlain et al., 2008).

In some child welfare environments, access to such programs may be limited. The fact that many children have consistently low PDR counts and do not exceed thresholds of behavioral risk suggests that a stepped care approach to services might be worthy of consideration for foster parent training. For example, for foster parents with children well below behavioral risk levels, it might be appropriate to consider a shorter or online-based version of a more intense evidence-based program, allowing resources to be targeted in greater alignment with foster parent and child needs. Stepped care models have been used and discussed in other health care arenas, such as diabetes (McFarland, 1997), asthma (Autio & Rosenow, 1999) smoking (Smith et al., 2001), social phobia (Andersson et al., 2006), depression (Scogin, Hanson, & Welsh, 2003), and child maltreatment prevention (Prinz, Sanders, Shapiro, Whitaker, & Lutzker, 2009).

Among foster parents already taking part in an evidence-based program, results from this study suggest that PDR counts continue to have predictive value. Even in the midst of the KEEP intervention, children with high and sustained PDR counts had substantially elevated risks for placement disruption. Children maintaining consistently high PDR counts despite the presence of an intervention could be identified through ongoing PDR information as possibly in need of additional targeted services to mitigate risks for placement disruption, and to help the foster parent and child. Depending upon the circumstance, this could take the form of further individualized services for the foster parent, child-focused problem-solving skills training, or in limited cases a specialized foster care arrangement such as Multidimensional Treatment Foster Care.

Several study limitations should be noted. Participants in this study were already engaged in a parenting intervention that had the effect of lowering overall PDR counts and risks of placement disruption. For this reason, results may not be directly comparable with findings observed in a usual service or general population group of children in foster care. For example, the threshold for predicting placement disruption (9 vs. 6) may have differed because most children were likely to have had somewhat decreasing behavioral trajectories, lowering risk of placement disruption for children who might initially have been at greater risk. Overall, however, the pattern of results was consistent with those from the KEEP control group; elevated PDR counts were still strongly related to placement changes, so it is likely that the pattern of findings has applicability outside of a treatment condition.

In a real-world application, it is not yet clear whether collecting routine PDR data from foster parents about children in their care would result in additional benefits worth the cost and effort associated with gathering such information. In the past, PDR counts have been collected using in-person calls. Currently we are examining whether automated phone and internet approaches could be implemented that would lower costs and be seen by foster parents and child welfare workers as a valuable method for enhancing communication about the stability of foster placements.

Although the PDR does have substantial predictive capacity, it is not a perfect predictor of negative placement changes. Still 38.2% of negative placement changes occurred over a year in the group of children with relatively low PDR scores. Further studies will be important to assess whether the predictive capacity of the PDR can be further enhanced and to test whether use of the PDR can contribute to more efficient, effective, and timely services for children with different levels of risk for placement disruption.