Moderators of Outcome in a Brief Family-Centered Intervention for Preventing Early Problem Behavior

Abstract

This study investigated moderators of change in an empirically supported family-centered intervention (the Family Check-Up) for problem behavior in early childhood. Participants were 731 2- to 3-year-olds (49% girls; 28% African American, 50% European American, 13% biracial) from low-income families and had been screened for risk of family stress and early-onset problem behavior. They were randomized to the Family Check-Up intervention or to a no-intervention control group. Latent growth models examined sociodemographic and parent psychological risk factors as potential moderators of change in problem behavior between ages 2, 3, and 4. Results revealed 2 moderators of intervention effectiveness. Caregivers with the lowest educational levels were more responsive to the family-centered intervention, and 2-parent families were more responsive to the intervention. Other risk factors showed no predictive effects. Overall, findings suggest that this brief family-centered intervention can be equally effective in reaching the most distressed and most disadvantaged families, compared to those who are more advantaged. However, results suggest that more attention may be needed to address the intervention needs of single parent families in reducing problem behavior in early childhood.

Numerous randomized trials show that parenting interventions are effective in preventing child problem behavior (Kazdin, 2002). These include “efficacy” trials, conducted in relatively specialist settings, as well as more recent adaptations of these interventions into “effectiveness” trials, which have the potential for widespread dissemination into real-world, community settings (Dishion & Stormshak, 2007; Gardner, Burton & Klimes, 2006; Hutchings et al., 2007; Webster-Stratton, 1998a, 1998b). To take such interventions to scale, investigators have developed modified interventions that are brief and low cost (Turner & Sanders, 2006) and that are adapted to engage the most marginalized families (Dishion, Nelson, & Kavanagh, 2003; Shaw, Dishion, Supplee, Gardner, & Arnds, 2006). It is generally accepted that such low-cost interventions, even those with quite modest effect sizes, could potentially be useful and could translate into large public health benefits (Biglan & Taylor, 2000).

A key issue underlying the potential for public health impact is whether such brief interventions are effective for the most high-risk families in society, or whether they confer greater benefit on more advantaged families, as reported in recent meta-analyses of predictors of outcome in parent training (Lundahl, Risser, & Lovejoy, 2006; Reyno & McGrath, 2006). This is an important concern, given that high-risk families are also those most likely to have children showing early onset and persistent behavior problems, which have a raised probability of escalating to more serious problem behaviors by adolescence (Farrington, 1994; Moffitt, 1993; Patterson & Yoerger, 1993). Accordingly, the present study investigated through moderator analyses whether some families and children benefit more or less from a brief intervention (Dishion et al., 2008; Shaw et al., 2006) for early prevention of problem behavior in high-risk toddlers.

The Early Steps Multisite trial (Dishion et al., 2008) is a randomized trial of a brief family-centered intervention, the cornerstone of which is the Family Check-Up (FCU), as well as linked parent management training (Dishion & Kavanagh, 2003; Dishion & Stormshak, 2007). The intervention takes an ecological approach to family intervention and treatment (EcoFIT), because it is designed for service-setting implementation and is adapted and tailored to families' ecology and needs (Dishion & Stormshak, 2007). EcoFIT was applied to early prevention of conduct problems and risk of later drug use through enhancing parenting skills and addressing domains that compromise parental functioning (e.g., social support, parent well-being, neighborhood resources). It focuses on low-income families with toddlers, who are further screened for risk factors for early-starting problem behavior. Screening is based on developmental knowledge of risk factors for early-starter conduct problems and their continuities with later risk behaviors (Moffitt & Caspi, 2001; Shaw, Bell, & Gilliom, 2000; Shaw, Gilliom, Ingoldsby, & Nagin, 2003). Engaging high-risk families in preventative interventions can, in many cases, pose challenges. These stem from the multiple problems of the families and from the fact that, unlike most clinic samples, they have not necessarily requested help with family issues. From a stages of change perspective (Prochaska & DiClemente, 1986), it is often necessary in interventions for individuals not seeking help to initially address the benefits of changing, say, parenting practices. Thus, an important feature of the FCU is the use of motivational interviewing (MI) techniques (Miller & Rollnick, 2002) to enhance engagement and motivation for change in parenting. Central to the intervention is the use of individualized feedback from a detailed home-based assessment of the child and family needs, as well as videotaped observation tasks. The shared feedback process, based on MI strategies, is a crucial part of engaging the family. This ensures that any subsequent intervention is closely tailored to the family's needs and goals and that the parents are, as far as possible, motivated and ready for change. These factors also make it more likely that a brief parenting intervention will be effective (Dishion & Stormshak, 2007). The FCU was originally developed and shown to be effective for high-risk adolescents as a preventive intervention, embedded in the public school system (Dishion et al., 2003). More recent findings on the effect of this model as implemented in middle schools reveal reductions in problem behavior 5–7 years later (Connell, Dishion, Yasui, & Kavanagh, 2007). Analyses based on uptake of intervention sessions, moreover, reveal that the highest risk families were most likely to engage in the FCU and to show reductions in adolescent problem behavior. Adaptations of the FCU to families with toddlers showed positive effects on parenting and problem behavior (Gardner, Shaw, Dishion, Burton, & Supplee, 2007; Shaw et al., 2006).

For early prevention, an accessible community service that reaches a high proportion of low-income toddlers throughout the United States is the Women, Infants, and Children program (WIC), a national program for family nutritional supplement and support. Findings from an earlier single site study (Shaw et al., 2006) and a large, multisite trial of FCU, involving 731 high-risk toddlers recruited from urban, rural, and suburban WIC sites (Dishion et al., 2008), showed modest but significant intervention effects at ages 3 and 4 on positive parenting skill and child problem behavior. In both trials, change in positive parenting appeared to predict change in the primary outcome, child problem behavior (Dishion et al., 2008; Gardner et al., 2007). Furthermore, in the multisite trial, there were also significant intervention effects on maternal depression (Shaw, Connell, Dishion, Wilson, & Gardner, in press) and on children's school readiness (Lunkenheimer et al., 2008), including language development and self-regulation. Main intervention effect sizes (Dishion et al., 2008; Shaw et al., in press) on parent-reported child problem behavior were d = 0.23 for Child Behavior Checklist Externalizing, 0.23 for the Problem scale from the Eyberg Child Behavior Inventory (ECBI), 0.18 for maternal depression, and 0.24 for positive parenting. Although these effects are modest, it is important to note that they might nevertheless have a substantial impact at a population level on problem behavior, if implemented more widely. That is, many children in the sample do not have significant levels of problem behavior, and, therefore, change is not expected. Analyses of two basic demographic factors, child gender and ethnicity (Dishion et al., 2008), revealed that there were no differences in intervention effects by gender or ethnicity.

Given these preventive intervention effects on child problem behavior, it is important to evaluate whether the FCU intervention has beneficial effects across the whole sample or whether there are subgroups of the most distressed and disadvantaged families for whom the intervention is less effective. These questions are best investigated through moderator analyses. Before examining prior literature on predictors and moderators of parent training outcome to guide choice of variables for the present study, we clarify the definition and relevance of these concepts. Moderator analyses are important for identifying those who are differentially responsive to intervention, and thus they may lead to better understanding of subgroups for whom there may be distinct causal patterns or prognoses (Hinshaw, 2002). Clinically, moderator analyses are helpful in identifying with greater precision types of clients for whom an intervention may be particularly suitable or, conversely, subgroups for whom there might be iatrogenic effects or the need for extra therapeutic effort. Findings may provide evidence to practitioners and prevention scientists that interventions can be effective for client groups traditionally thought to be hard to treat or, in the current sample, difficult to initially engage. Alternatively, results might suggest that specific interventions are less fruitful for families living in the context of high adversity.

In intervention trials, the relevant question is whether intervention effects are conditional (“moderated”) on baseline characteristics of the sample. Moderation is defined as a statistical interaction between baseline characteristic and intervention effect (Hinshaw, 2002; Kraemer, Wilson, Fairburn, & Agras, 2002). Thus, moderators are distinct from “predictors” of outcome, which may be associated with outcome equally across treatment and control groups (Beauchaine, Webster-Stratton, & Reid, 2005; Hinshaw, 2002) or tested within the treatment group only (Dumas & Wahler, 1983).

In the field of parenting interventions for problem behavior, there are relatively few analyses of moderators but many studies of predictors of outcome (Dumas & Wahler, 1983; Eyberg, Nelson, & Boggs, 2008; Kazdin & Wassell, 1999). Two recent meta-analyses (Lundahl et al., 2006; Reyno & McGrath, 2006) attempted to combine findings on predictors of outcome across multiple trials (63 and 31 trials, respectively) of parenting interventions. These meta-analyses found reasonably clear-cut results despite synthesis across a range of intervention and sample types, including clinic treatment, indicated prevention, and low-risk prevention studies. Both reviews concluded that children of parents with disadvantages (including those with low income), of single parents, and of parents with depression show poorer intervention outcomes than children facing lower levels of adversity.

Large inclusive meta-analyses are useful in telling us broadly which characteristics predict intervention success across a wide range of programs; however, their weakness lies in not being able to identify whether specific programs may be differentially successful with more troubled families. Several recent studies, not included in these reviews, reached somewhat different conclusions. Thus, Werba, Eyberg, Boggs, and Algina's (2006) study of a young clinic-referred group receiving parent–child interaction therapy (n = 81), although consistent with earlier studies in finding that maternal distress predicted poorer outcome, nevertheless found no effects of socioeconomic status on child outcome. It may be that with some recent interventions, where explicit attention is paid to client accessibility and engagement (Webster-Stratton, 1998a, 1998b), it is possible to achieve better results with high-risk families, who traditionally have been seen as “harder to reach” due to higher dropout rates and lower intervention success (Reyno & McGrath, 2006). For example, Beauchaine et al. (2005) pooled data from six treatment trials (total n = 514) of the Incredible Years program with 3- to 8-years-olds. In predictor analyses, they found that younger parents, those with a history of drug abuse, and children with comorbid problems did better in parent training than did families without such risk factors. Furthermore, in moderator analyses, mothers with poor marriages or higher levels of depression also fared better than did those who were less distressed. The same group found similar effects in a prevention study with low-income families. Thus, in two studies analyzing a large pooled sample in Head Start preschools, mothers who were depressed or who had a history of abuse or substance use were just as likely to benefit from the Incredible Years program as were those without such risk factors (Baydar, Reid, & Webster-Stratton, 2003). Using multiple measures of outcome, engagement, and attrition, Reid, Webster-Stratton, and Beauchaine (2001) found that effectiveness and satisfaction were equally strong across four ethnic groups. In an independent replication of this intervention in low-income areas in the United Kingdom (Hutchings et al., 2007), moderator analyses found stronger effects on child outcomes when mothers were more depressed, whereas family disadvantage had no impact on responsiveness to the intervention or outcomes (Gardner, Hutchings, & Bywater, 2008).

In sum, the meta-analyses present a reasonably clear picture, albeit across somewhat disparate trials, suggesting that parenting interventions in general are less successful at engaging the most distressed and disadvantaged families. In contrast, some large, more recent trials found no adverse effects of family disadvantage on outcome, in both community preventive and clinic-referred samples; it is noteworthy that, in most of these studies, the intervention approaches used paid particular attention to parent engagement and accessibility of services (Hutchings, Bywater, & Daley, 2007; Webster-Stratton, 1998b).

Our aim in the present study was to test whether there were differential intervention effects on child problem behavior, by family and parent risk factors, in the multisite Early Steps trial. After being screened as high risk for problem behavior in WIC centers, 731 low-income toddlers were randomly allocated to a brief parenting intervention, the FCU, or no intervention (for main outcomes, see Dishion et al., 2008; Shaw et al., in press). The prior literature guided our choice of potential moderator variables for the present study. Studies typically employ demographic and psychological distress variables that have repeatedly been found to predict poor behavioral outcomes in longitudinal studies (Ackerman, Brown, & Izard, 2004; Shaw, Winslow, Owens, & Hood, 1998). There is evidence about mechanisms linking these predictors to problem behavior outcome via fewer material and psychological resources (e.g., lower social support), which in turn affect parenting skill and place stress on the parent–child relationship (Larzelere & Patterson, 1990; Trentacosta et al., 2008). Metaanalyses show that the same factors traditionally appear to predict poor treatment outcome and high attrition from trials (Lundahl et al., 2006; Reyno & McGrath, 2006). Following this literature, we examined two sets of potential moderators of outcome, assessed at age 2. Family risk factors included mother a single parent, mother a teen at the birth of her first child, low maternal educational level, and maternal alcohol or drug problem. Parent risk factors, reflecting psychological problems at baseline, included depressive symptoms, perceived hassles, and partner relationship problems. Following work on cumulative family adversity and child problem behavior (Ackerman et al., 2004; Shaw et al., 1998), risk variables were also combined into a cumulative risk index. For all analyses, the dependent variable was the main trial outcome: growth in problem behavior between ages 2, 3, and 4.

Given the mixed pattern of findings in the literature, we did not have specific hypotheses about whether higher levels of individual family or parent risk factors would be associated with greater improvements in child disruptive behavior. However, on the basis of findings of prior prevention studies in which careful attention was paid to parent engagement (Baydar et al., 2003), we postulated that the FCU would tend to show as good outcomes for families with higher as with lower levels of initial family and parent adversity. It is plausible that a brief intervention that employed MI strategies intended to enhance engagement might obtain good results with the most distressed families, compared to those who are less distressed, as found in the ES pilot study (Shaw et al., 2006). The present study extends existing literature by using a larger sample (N = 731) than have most other parent intervention trials and by providing a more precise and reliable test of moderating mechanisms. It is noteworthy that of the 63 studies in Lundahl et al.'s (2006) review, 7 had Ns of over 100 but only 1 had an N of over 200. Furthermore, it is important to examine moderators rather than merely predictors of change (Hinshaw, 2002), as this allows us to distinguish predictors of general outcome from moderators that specifically predict intervention change.

Method

Participants

Ethical approval was granted by the institutional review board at all authors' institutions. Parental written consent was obtained both for screening and for trial stages of the study. Participants included 731 mother–child dyads recruited between 2002 and 2003 from WIC programs in the metropolitan areas of Pittsburgh, Pennsylvania, and Eugene, Oregon, and within and outside the town of Charlottesville, Virginia. Families were contacted at WIC sites and, following a screen to ensure that they met the study criteria by having socioeconomic, family, and/or child risk factors for future behavior problems, were invited to participate if they had a son or daughter between age 2 years 0 months and 2 years 11 months. Risk criteria for recruitment were based on earlier longitudinal work on toddlers that examined early child and family characteristics as predictors of later problem behavior in similar samples (Shaw et al., 1998, 2000, 2003). Risk criteria were defined as follows: at or above 1 SD above normative averages, derived from published standardization data, and on one or more screening measures in the following three domains: (a) child behavior (conduct problems, high conflict relationships with adults); (b) family problems (maternal depression, daily parenting challenges, substance use problems, teen parent status); and (c) sociodemographic risk (low educational achievement and low family income according to WIC criteria). For inclusion in the sample, high-risk status on at least two of the three risk domains was required. In cases where the high-risk criterion was not met for child behavior, children were required to have above-average scores on either the ECBI Intensity or Problem scale (Robinson, Eyberg, & Ross, 1980) to increase the probability that parents would be motivated to change this behavior.

Recruitment

Initially, 1,666 families with a 2-year-old were screened at WIC sites across the three study sites; of these, 879 met eligibility requirements and 731 (83%) agreed to take part. Children in the sample had a mean age of 30 months (SD 3.2) at the time of the age 2 assessment. Of the 731 families (49% girls), 37% were recruited in Pittsburgh, 37% were recruited in Eugene, and 26% were recruited in Charlottesville. Across sites, primary caregivers' self-identified ethnicity was as follows: 28% African American, 50% European American, 13% biracial, and 9% other groups. Thirteen percent of primary caregivers self-reported as Hispanic American. Over two thirds of families had an annual income of less than $20,000 in 2002–2003.

Retention

Of the 731 families who initially participated, 659 (90%) participated at the 1-year follow-up and 619 (85%) participated at the 2-year (age 4) follow-up. At ages 3 and 4, selective attrition analyses revealed no significant differences by site, race, ethnicity, or gender, levels of maternal depression, or children's externalizing behavior. Furthermore, no differences were found in the number of participants who were not retained in the control versus intervention group at ages 3 (n = 40, 32) and 4 (n = 58, 53, respectively).

Measures

All measures were administered at the age 2, 3, and 4 home visits. However, we report on risk factors assessed at baseline (age 2) and problem behavior at ages 2, 3, and 4.

Early childhood problem behavior (dependent variable)

A widely used 36-item measure of childhood problem behavior, the ECBI (Robinson et al., 1980), was administered. We used the Problem scale. A primary outcome for the trial, it asks caregivers to report whether or not the behavior is a problem for the parent. The inventory has been shown to be highly correlated with independent observations of children's behavior, to differentiate clinic-referred and nonclinic populations, and to have high test–retest reliability (.86) and internal consistency (.98; Robinson et al., 1980). In the current study, internal consistencies for the Problem factor were .84, .90, and .94 at ages 2, 3, and 4, respectively. We also used the 99-item Child Behavior Checklist (CBCL) 1.5–5, which has been found to have adequate test–retest reliability (range = 0.68–0.92) and good cross-informant agreement (parent–child care provider agreement = 0.65; Achenbach & Rescorla, 2000). We used the broad-band factor Externalizing; internal consistencies were .86, .89, and .86 at ages 2, 3, and 4.

Demographics questionnaire

A demographics questionnaire that included items about family structure and risk factors was administered to mothers. Risk factors were defined as follows: single parenthood as having no partner living in the household; teen parenthood as being less than 18 years old at the birth of the first child; and low maternal education as having completed less than high school education.

Parent substance use

Mother's current drug or alcohol problem was defined, via questionnaire (Dishion & Kavanagh, 2003), to include one or more of the following: (a) argumentative or irritable when drinking; (b) drinks every day and has 3–4+ drinks most days; (c) uses marijuana or hard drugs more than once per month; (d) uses more than one hard drug per month.

Maternal depression

We used the Center for Epidemiological Studies on Depression Scale (CES-D; Radloff, 1977), a well-validated, widely used 20-item measure of depressive symptoms. Participants report frequency of experiencing listed depressive symptoms during the past week on a scale ranging from 0 (0–1 day) to 3 (5–7 days). Items are summed to create an overall depressive symptoms score. In terms of convergent validity, clinically depressed individuals have been found to score higher on the CES-D than did nondepressed individuals (Weissman et al., 1996). Internal consistencies have been found to range from .80 to .90 and test–retest reliabilities have ranged from 0.40 to 0.70 among community samples (Devins et al., 1988; Radloff, 1977). For the current sample, internal consistencies were .76 and .75 at ages 2 and 3.

Parenting Daily Hassles (PDH)

The PDH is a measure of typical daily stressors perceived by parents and found to be associated with child behavior outcome to a greater degree than life stresses that are more global in nature (Crnic & Greenberg, 1990). The 20-item Frequency subscale of the PDH has good internal consistency (α = .81). It was highly correlated with perceived intensity of daily hassles, which indicates good convergent validity (r = .78; Crnic & Greenberg, 1990). In the present study, the 20-item PDH was administered to mothers. The scale of perceived frequency of daily stressors was used; internal consistency was .81.

Partner relationship quality

Maternal perception of the level of satisfaction in her partner relationship was assessed with the 16-item short form of the Marital Adjustment Test (Locke & Wallace, 1959), for which high scores represent higher satisfaction. Prior research shows that this measure differentiates between harmonious and disturbed marriages (Locke & Wallace, 1959; Rosenbaum & O'Leary, 1981) and also predicts child behavior problems (Emery & O'Leary, 1984). Split-half reliability of the short Marital Adjustment Test was found to be 0.90. If mothers were recently separated, they were asked to report on the period of the past year during which they were still living with their partner. If mothers were not married, they were asked to complete the scale on their most intimate adult relationship, including live-in or dating partner. The term close relationship was substituted for marriage. This strategy is sensitive to the fact that 40% of mothers in the study were single yet allowed for inclusion of important information on their close relationships. In the current study, internal consistency was .57 at age 2.

Cumulative risk index

This index was generated from the seven indicators of sociodemographic risk that were examined as moderators. These indicators were (a) single parenthood, (b) teen parent status, (c) low maternal education, (d) substance use problem, (e) maternal depression, (f) parenting daily hassles, and (g) partner relationship quality. Because maternal depression, daily hassles, and relationship quality were used as continuous variables in the primary moderator analyses, it was necessary to dichotomize them for computation of the cumulative risk index. Maternal depression was dichotomized on the basis of the standard clinical cutoff score of 16 on the CES-D (Radloff, 1977). Daily hassles and relationship quality were dichotomized based on scores greater than 1 SD above the sample mean and less than 1 SD below the sample mean on these scales, respectively. Families received a score of 1 for each risk indicator if present or of 0 if a risk indicator was absent. The cumulative risk index was the proportion of risk indicators present for each family. Possible scores on the risk index range from 0 (no risk factors) to 1 (all risk factors present).

Procedures

Assessment protocol

Parents and children who agreed to participate in the study were scheduled for a 2.5-hr home visit. Each assessment began with a series of observational tasks that included free play, cleanup, teaching tasks, meal preparation, and lunch; these parent–child interaction data were used for investigating outcomes and mediators and for parent feedback and are not further reported here. Parents also completed questionnaires during the home visit. The home visit and observation protocol was repeated at ages 3 and 4 for control and intervention groups. Families received $100 for participating in the age 2 assessment and $140 for participating in the age 4 assessment.

The randomization sequence was computer generated by a staff member, who was not involved with recruitment, and was stratified by gender. To ensure allocation concealment, the examiner opened a sealed envelope, revealed the family's group assignment only after the assessment was completed, and shared this information with the family. Examiners for follow-up assessments were not informed of families' allocation.

Intervention protocol: The FCU

Families randomly assigned to the intervention condition were scheduled to meet with a parent consultant for the FCU. The FCU (Dishion & Kavanagh, 2003; Dishion & Stormshak, 2007) is a brief, three-session intervention inspired by innovations in MI (Miller & Rollnick, 2002). The general concept is to integrate clinical skill with systematic assessments, so as to support parents' motivation to change parenting practices that may be problematic and to maintain those that are healthy. Although the typical sequence of sessions in the FCU is an initial interview, an assessment session, and a feedback session, the sequence was altered to enhance the internal validity of the current research (Dishion & Stormshak, 2007). For the first assessment, we designed the study so that families were unaware of their randomly assigned intervention status until after the initial assessment. Thus, at age 2, families completed the assessment, were randomized, and, if they were assigned to the intervention group, were then interviewed by a parent consultant and provided with feedback based on the assessment. At the end of the feedback session, families were given a $25 gift certificate for completing the FCU. The rationale for providing an inexpensive incentive for engaging with the FCU was based on the community reinforcement approach, where it has been found that engaging the highest risk clients in empirically supported interventions in community services benefits from the use of incentives (Sorensen, Rawson, Guydish, & Zweben, 2003).

Thus, the initial meeting was an assessment conducted with research staff, as described above, during which the family engaged in a variety of in-home videotaped tasks of parent–child interaction and caregivers completed several questionnaires about their own, their child's, and the family's functioning. During the assessment, staff completed ratings of parent involvement with and supervision of their child. The second session was an initial interview with the parent consultant, during which the consultant explored parent concerns with a focus on family issues that were currently most critical to the child's well being. The third meeting was a detailed feedback session, during which the parent consultant summarized the results of the assessment using MI strategies. An essential objective in the feedback session is to explore the parents' willingness to change problematic parenting practices, to support existing parenting strengths, and to identify services appropriate to the family needs. For example, caregivers often failed to realize that they were suffering from depression, and thus feedback could potentially motivate a parent to seek mental health services. The parent was also offered follow-up sessions that focused on parenting practices, other family management concerns (e.g., co-parenting), and contextual issues (e.g., day care, partner relationship, housing). The MI approach is especially prominent in the initial interview and feedback sessions and pervades later sessions as needed, depending on parent motivation and engagement (Gill et al., 2008).

All parent consultants had at least master's-level qualifications and experience in family interventions but, at the study's outset, no experience in using the FCU. Their initial training took 2.5–3 months and involved a combination of strategies that included didactic instruction and role-playing, followed by ongoing videotaped supervision of intervention activity. Before working with families, parent consultants were certified by lead consultants at each site who had been certified by the FCU intervention developer, Thomas J. Dishion, and were recertified annually. Certification was established by reviewing videotapes of feedback and follow-up intervention sessions to evaluate whether parent consultants were competent in all critical components of the intervention. Weekly supervision plus weekly cross-site videoconferences also helped to enhance fidelity and prevent drift. Finally, annual parent consultant meetings were held to update training and address issues related to the needs of families across sites.

Of families assigned to the treatment condition, 80% participated in the initial interview and feedback sessions at age 2 and 65% participated at age 3. Of those families who met with a parent consultant, the average number of sessions per family was 3.3 (SD = 2.8) at age 2 and 2.8 (SD = 2.7) at age 3 (range = 1–19), with the initial interview and feedback included as two of those sessions. Thirty-five percent of families at age 2 and 26% of families at age 3 opted to have further sessions after the feedback. Over two thirds of families chose goals related to parenting, child development, and behavior; thus, these issues were the main focus of follow-up intervention sessions. There was a modest positive correlation between level of child problem behavior and number of sessions (Shaw et al., in press), and this implied that parents of more difficult children were slightly more likely to engage. However, there were no correlations between family and parent risk factors and number of sessions at age 2. We used an intention-to-treat design and included in all analyses the 22% of families assigned to intervention who did not participate in the FCU.

Results

Table 1 shows descriptive statistics for main trial outcome variables at ages 2, 3 and 4 (as reported in Dishion et al., 2008); Table 2 shows moderator variables at Time 1. Families showed high levels of disadvantage: Nearly 80% were below the poverty line (29% below $10K), and 40% were single parent families. Furthermore, 42% of mothers were above the clinical cutoff for depression (16 on the CES-D), and 44% of children at age 2 were above the clinical cutoff on the ECBI Intensity scale (132 in the restandardization sample; Colvin, Eyberg, & Adams, 1999). Consistent with random assignment, there were no differences between the groups on baseline demographic or behavioral characteristics.

Table 1. Descriptives for Primary Trial Outcome Measures.

	Control				Intervention
Measure	N	M	SD	N (%) in clinical range	N	M	SD	N (%) in clinical range
Maternal depression, age 2	363	16.56	11.02	148 (40.8)	366	16.94	10.30	155 (42.3)
Maternal depression, age 3	320	16.26	10.84	134 (41.9)	331	14.62	11.06	123 (37.2)
ECBI Problem (t score), age 2	364	59.22	8.49	169 (46.4)	365	59.14	8.45	154 (42.2)
ECBI Problem (t score), age 3	315	60.06	10.51	163 (51.7)	327	59.18	10.22	158 (48.3)
ECBI Problem (t score), age 4	305	60.63	10.80	163 (53.4)	311	58.64	11.15	139 (44.7)
CBCL Externalizing (t score), age 2	363	59.32	7.83	179 (49.3)	367	59.65	8.57	176 (48.0)
CBCL Externalizing (t score), age 3	320	56.11	9.56	117 (32.1)	331	55.83	9.23	107 (29.7)
CBCL Externalizing (t score), age 4	306	54.67	9.97	93 (30.4)	313	52.68	10.87	82 (26.2)

Note. ECBI = Eyberg Child Behavior Inventory; CBCL = Child Behavior Checklist.

Table 2. Descriptives for Predictor Variables at Age 2.

	Intervention		Control
Variable	M	SD	M	SD
Parent psychological factors (continuous measures)
Maternal depression	16.9	10.3	16.6	11.0
Partner satisfaction (Locke & Wallace, 1959)	56.3	10.3	58.2	9.7
Parenting daily hassles	46.8	8.8	46.5	8.5
Family factors (categorical measures)
Teen parent status	22%		23%
Single parenthood	38%		42%
Low maternal education	22%		25%
Substance use problem	14%		12%

Descriptive Data on Predictor Effects

For illustrative purposes, we present descriptive data on predictors of change in the intervention group for two variables. Table 3 shows ECBI intervention change from age 2 to age 4 by risk group and, for comparison, predictors of change over the same time period in the control group.

Table 3. Illustrative Descriptive Data for Significant Moderator Effects on ECBI Problem Outcome.

	Change in ECBI Problem score from age 2 to age 4 by family risk factor
Risk variable	Intervention (n = 310)	Control (n = 304)
Single parenthood	−1.0	−1.4
Living with partner	1.6	−1.1
t test, single vs. not	t = 2.6, p = .009	t = −0.29, ns
Parent less than high school	0.59	−4.2
Parent high school or more	0.67	−0.30
t test, less vs. more than high school	t = −0.07, ns	t = −3.3, p = .001

Note. ECBI = Eyberg Child Behavior Inventory.

Moderation Analysis Plan

Moderation hypotheses were examined with latent growth model (LGM) analyses. Because only three time points of data were available, LGMs could include only intercept and linear slope growth parameters that reflected the initial level of problem behaviors and the linear rate of change in problem behaviors from age 2 to age 4. Separate LGMs examined changes in the ECBI and the CBCL Externalizing scores over time. Moderation of the effect of treatment on the rate of change in problem behaviors was captured by a series of interaction effects between intervention status and the covariates. In these models, the effect of intervention and all intervention covariates (effects on the intercept) were fixed at zero, as the intercept was a baseline measurement made prior to the introduction of treatment. However, all covariate and intervention covariate interactions were allowed to predict the slope parameter.

All of the hypothesis-testing analyses were conducted in Mplus. We used full information maximum likelihood estimation (Muthén & Muthén, 2007), which provides a method for accommodating missing data by estimating each parameter using all available data for that specific parameter. Overall model-fit indices reflect the broad correspondence between the measurement and structural paths included in the hypothesized model and the covariance structure in the observed data. In the face of acceptable overall model fit, core hypothesis tests are reflected in the statistical significance of the individual parameters of the model, such as the paths from covariates to the growth parameters.

ECBI Problem Behavior Findings

Interaction term model

LGM results for moderators of the effect of intervention on the rate of change in ECBI Problem scores are shown in Table 4. This model appeared to provide reasonable fit to the data, χ²(24, N = 723) = 55.40, p < .05, comparative fit index (CFI) = .95, root-mean-square error of approximation (RMSEA) = .04, standardized root-mean-square residual (SRMR) = .02, although the significant chi-square value is likely a reflection of the large sample size. The initial level of problem behaviors at age 2 was negatively related to maternal-reported marital quality and was positively related to mother-rated daily hassles. Overall, significant growth was seen in ECBI Problem scores over time. Intervention was significantly related to decreased growth over time. Marital quality and daily hassles were both negatively related to the rate of change, whereas low education predicted greater growth in problem behaviors. Two significant interaction effects were found. First, low maternal education interacted with intervention status to predict the rate of change in problem behaviors. The interaction effect (for an example, see Figure 1) indicates that a greater effect of intervention was seen in families with less educated mothers and that higher maternal education was associated with a smaller intervention effect. Follow-up analyses that used methods of Preacher, Curran, and Bauer (2006) indicated that significant growth in problem behaviors was seen in both high- and low-educated control groups (simple slope high-education control group = 1.21, SE = 0.40; simple slope low-education control group = 2.07, SE = 0.62). In the intervention groups, simple slopes were nonsignificantly negative but were more negative in the low-education group (simple slope low-education intervention group = −1.02, SE = 1.04) relative to the high-education intervention group (simple slope high-education intervention group = −0.41, SE = 0.41).

Table 4. LGM Interaction Model for ECBI Problem Scale.

Moderator	Intercept est (SE)	Slope est (SE)
Intervention status	fixed at 0	−1.61 (0.56)*
Partner relationship quality	−0.09 (0.03)*	−0.05 (0.02)*
Parenting daily hassles	0.31 (0.03)*	−0.08 (0.02)*
Maternal depression	0.00 (0.02)	0.02 (0.02)
Substance use problem	−0.39 (0.66)	0.20 (0.51)
Teen parent status	0.05 (0.51)	−0.10 (0.41)
Low maternal education	0.53 (0.51)	0.86 (0.40)*
Single parenthood	−0.38 (0.45)	0.52 (0.36)
Interactions: Treatment ×
Partner Relationship Quality	fixed at 0	−0.02 (0.04)
Parenting Daily Hassles	fixed at 0	−0.02 (0.04)
Maternal Depression	fixed at 0	0.01 (0.04)
Substance Use Problem	fixed at 0	−0.13 (0.95)
Teen Parent Status	fixed at 0	−1.25 (0.75)†
Low Maternal Education	fixed at 0	−1.47 (0.75)*
Single Parenthood	fixed at 0	1.35 (0.67)*
Parameter intercept	14.13 (0.38)*	1.21 (0.40)*
Parameter residual variance	16.30 (2.96)*	11.15 (1.79)*

Note. LGM = latent growth model; ECBI = Eyberg Child Behavior Inventory; est = estimate.

^*p < .05.

^†p < .10 (trend).

Figure 1.

Intervention effects on child problem behavior (Eyberg Problem scale) by parent educational level. Eyberg = Eyberg Child Behavior Inventory.

Second, single mother status interacted with intervention status to predict rate of change in problem behavior. This interaction effect indicates that stronger intervention effects were seen in two-parent families, with a smaller effect in families of single mothers. Follow-up analyses done using methods of Preacher et al. (2006) indicate that significant growth in problem behaviors was seen in both the partnered and the single-mother control groups (simple slope partnered control group = 1.21, SE = 0.40; simple slope single-mother control group = 1.36, SE = 0.56). In the intervention groups, simple slopes were both nonsignificant and negative (i.e., behavior improved) in the partnered group (simple slope partnered intervention group = −0.41, SE = 0.41) and were slightly positive in the single-mother intervention group (simple slope single-mother intervention group = 0.12, SE = 0.93).

Effect sizes for each significant moderator were calculated with Cohen's d and were based on comparing the effect of control versus intervention allocation on outcome, with families grouped by presence or absence of each moderator variable. For the Eyberg outcome, control versus intervention group, d = 0.04 for single mothers and d = 0.53 for partnered mothers, respectively. With maternal education as the moderator (see Figure 1), control versus intervention group, d = 0.18 for more educated mothers and d = 0.68 for lower educated mothers, respectively.

Cumulative risk model

The cumulative risk model provided excellent fit to the data, χ²(6, N = 723) = 8.19, p = .24, CFI = 1.00, RMSEA = .02, SRMR = .02. The cumulative risk index was significantly related to the intercept parameter (est = 2.08, SE = 0.70) and to the slope parameter (est = 3.30, SE = 1.24). The intervention effect on the slope dropped to nonsignificance in this model (est = −0.79, SE = 0.53); this change was likely due to high collinearity with the interaction term included in this model. The interaction between intervention and cumulative risk was not significantly related to the slope of problem behaviors (est = −0.41, SE = 1.68).

To verify that collinearity was the reason for the lack of main effect of intervention on the problem behavior slope in the interaction model, we ran a model that included only main effects of intervention and cumulative risk as predictors of the rate of change in problem behaviors. As expected, a significant main effect of intervention on rate of change in problem behaviors was seen (est = −0.24, SE = 0.08). This indicated that the nonsignificant effect of intervention in the interaction model was likely due to high collinearity between the main and interaction effects.

Externalizing Behavior Problem Findings

Interaction term model

LGM results for moderators of the effect of intervention on rate of change in behavior problems on the CBCL Externalizing scale are shown in Table 5. This model provided excellent fit to the data, χ²(24, N = 723) = 32.19, p = .12, CFI = .99, RMSEA = .02, SRMR = .01. The initial level of problem behaviors at age 2 was negatively related to marital quality and was positively related to daily hassles. Overall, significant declines were seen in Externalizing scores over time. Intervention was significantly related to steeper declines in Externalizing scores over time. One significant interaction effect was found. Low maternal education interacted with intervention status to predict rate of change in problem behaviors. The interaction effect (see Figure 1) shows that a greater effect of intervention was seen in families with less educated mothers, whereas higher maternal education was associated with a smaller intervention effect. Follow-up analyses (Preacher et al., 2006) indicated that slopes were significant across all four groups (Treatment × Education) but that the magnitude of the differences between the intervention and control groups was larger for the low-education groups (simple slope in control group = −1.19, SE = 0.58; simple slope in intervention group = −5.14, SE = 0.98) than for the high-education groups (simple slope in control group = −1.64, SE = 0.27; simple slope in intervention group = −3.43, SE = 0.37). Effect sizes for this moderator on CBCL growth were as follows: control versus intervention group, d = 0.15 for more educated mothers, d = 1.17 for less educated mothers.

Table 5. LGM Interaction Model for CBCL Externalizing.

Moderator	Intercept est (SE)	Slope est (SE)
Intervention status	fixed at 0	−1.72 (0.52)*
Partner relationship quality	−0.11(0.03)*	−0.04 (0.02)
Parenting daily hassles	0.26 (0.03)*	−0.03 (0.02)
Maternal depression	0.03 (0.03)	0.02 (0.02)
Substance use problem	0.39 (0.77)	−0.19 (0.48)
Teen parent status	0.36 (0.60)	−0.37 (0.38)
Low maternal education	1.39 (0.60)	0.45 (0.38)
Single parenthood	−0.36 (0.53)	0.32 (0.33)
Interactions: Treatment ×
Partner Relationship Quality	fixed at 0	−0.06 (0.04)
Parenting Daily Hassles	fixed at 0	−0.01 (0.04)
Maternal Depression	fixed at 0	−0.04 (0.03)
Substance Use Problem	fixed at 0	−0.04 (0.88)
Teen Parent Status	fixed at 0	−0.52 (0.69)
Low Maternal Education	fixed at 0	−2.16 (0.69)*
Single Parenthood	fixed at 0	0.19 (0.62)
Parameter intercept	21.08 (0.44)*	−1.64 (0.38)*
Parameter residual variance	32.08 (3.38)*	8.86 (1.68)*

Note. LGM = latent growth model; CBCL = Child Behavior Checklist; est = estimate.

^*p < .05.

^†p < .10 (trend).

Cumulative risk model

The cumulative risk model provided reasonable fit to the data, χ²(6, N = 723) = 16.64, p < .05, CFI = .99, RMSEA = .05, SRMR = .03, although the nonsignificant chi-square value likely reflects the large sample size. The cumulative risk index was significantly related to the intercept parameter (est = 3.45, SE = 0.78) and to the slope parameter (est = 3.68, SE = 1.15). The intervention effect on the slope was not significant in this model (est = −0.15, SE = 0.49); this change is likely due to high collinearity with the interaction term included in this model. The interaction between intervention and cumulative risk did not significantly predict the rate of change in problem behaviors (est = −2.77, SE = 1.57).

As above, to verify that collinearity was the reason for nonsignificance of the main effect of intervention in the interaction model, we also ran a model that included only the main effects of intervention and cumulative risk as predictors of the rate of change in problem behaviors. Consistent with a multicollinearity hypothesis, a significant main effect of intervention on growth in problem behavior was seen in this model (est = −0.85, SE = 0.29).

Discussion

Our moderator analyses found that one family risk factor, low maternal education, predicted greater improvement in child problem behavior from age 2 to age 4 in response to a brief parenting intervention. This effect on growth in problem behavior was robust across Eyberg and CBCL outcomes. Another risk factor, single parent status, on the other hand, predicted lesser improvement following intervention for the Eyberg outcome only. For both outcomes, there was a consistent picture of no moderator effects for other risk variables; parents who were very depressed or hassled, who had marital or drug problems, or who had been a teenage mother were equally likely to see improvements in child problem behavior following intervention, compared with those with lower levels of risk on these variables. When we combined predictors into a cumulative risk index, there was similarly no evidence of differential outcome by level of risk.

Our findings in the present paper complement those of Dishion et al. (2008) and Shaw et al. (in press), which demonstrated effectiveness of the FCU for preventing problem behavior in a high-risk, nonreferred sample and for addressing key risk factors, such as parenting skill and maternal depression. Effect sizes, although modest, were reasonable for a brief intervention in a preventive setting. The trial also shows that the FCU is equally likely to reduce problem behavior for boys and for girls and for families of different ethnic groups (Dishion et al., 2008). The present paper extends these findings considerably by showing that the intervention effects were generally very similar in size for families with very high levels of distress and disadvantage compared with those who were more advantaged, albeit with a sample of predominantly low-income families.

It is noteworthy that this prevention trial failed to replicate a more common pattern of findings, whereby family risk factors predict poor outcome, as found in both treatment and prevention studies in two meta-analyses (Dumas & Wahler, 1983; Reyno & McGrath, 2006). The more optimistic findings of the Early Steps trial are consistent with some other recent preschool parenting intervention trials, including those analyzing data from a rather different intervention, the Incredible Years program (Baydar et al., 2003; Beauchaine et al., 2005; Gardner et al., 2008), which found as good or sometimes better results with more distressed and disadvantaged families, in both treatment and prevention settings.

It is worth considering whether there might be common factors contributing to some interventions being more effective with very troubled or impoverished families than with less troubled families. The large meta-analyses of parenting trials included families referred for treatment and having high conduct problem symptom counts (Reyno & McGrath, 2006), as well as families in prevention trials (Lundahl et al., 2006). More recent studies that showed no moderator effects or some effects, but in the direction of more troubled parents doing better, are noteworthy in that they had much larger samples than did studies in the meta-analysis and paid strong and explicit attention to client engagement. In other ways, they are not particularly similar: The Incredible Years program is group based and much more intensive. One of its studies is a clinic treatment sample; the other is, like Early Steps, a low-income, early prevention trial but is located in Head Start preschools. Despite these considerable differences in program type, common factors likely to contribute to successful intervention with very disadvantaged families include accessibility and careful attention to client engagement and motivation (Hutchings, Bywater, & Daley, 2007). These factors are central to the FCU, with its use of shared assessment results to design an intervention led by client needs and its use of MI strategies. This explicit focus on collaborative engagement with parents is also prominent in the Webster-Stratton (1998a, 1998b) program, where it is achieved through use of a group-based program with an emphasis on nondidactic parent discussion and shared problem solving. Accessibility is addressed in the FCU by having individual meetings with families in their homes, both for assessment and for intervention, and in the Webster-Stratton program by providing food, child care, and often transport. The latter program has an impressively strong evidence base but is less brief (12 group sessions). Furthermore, a group-based format may not suit all families.

Potentially, we can speculate that the FCU, although its effect sizes may be smaller in magnitude because of the relatively small amount of contact with families, is a briefer route to engaging marginal families. Two considerations are relevant to the issue of effect sizes with a community intervention trial. First, the majority of children and families even in an at-risk community sample do not display clinically significant levels of problem behavior or dysfunctional parenting. Thus, some families simply do not need to change, and a larger number of families need to change very little. We see the most pronounced change, indeed, in the families with offspring with the highest levels of problem behavior who engage in the FCU (Connell et al., 2007; Dishion et al., 2008). The second consideration is that we are studying a developmental range where early behavior problems may be temperamentally driven, rather than representing overlearned patterns of family interaction. As some children grow in overt and covert forms of problem behavior, we may anticipate larger effect sizes over time, given the preventive nature of this effort. Findings such as those reported by DeGarmo, Patterson, and Forgatch (2004) suggest that changes in family dynamics, such as reduced depression and improved parenting, may actually produce increases in effect sizes over time. However, to resolve questions of real-world prevention effectiveness, such programs will need to be taken to scale (e.g., within a national service such as WIC) and further tested for cost-effectiveness, as some community trials have begun to do (Tudor-Edwards, Ceilleachair, Bywater, Hughes, & Hutchings, 2007).

Although it should be noted that moderator effects are relatively modest in terms of the amount of variance accounted for, another way of looking at their magnitude is to examine effect size differences by moderator group, which are quite marked. Thus, taking the CBCL outcome as an example, effect size in the intervention group is large (1.17) for less educated parents and small (0.15) for higher educated parents. It is also important to consider why parents with very low educational attainment responded better to the intervention. It might be that this risk factor is linked to more limited awareness of child development and parenting issues and that the structured feedback provided by FCU is particularly accessible and easy to understand and therefore more useful for these parents. On the other hand, in common with findings from the meta-analyses, single parents in our study appeared to find it harder to make use of the intervention, without the support of a partner. One explanation for this might be that lack of a partner to actively support change in parenting is critical. As found by Tolson et al. (1995), it might be that mothers perceived that emotional support was positively affected when another adult resided with the family, regardless of the other adult's involvement with the parenting role. The findings suggest the need to pay more attention to strategies for engaging single mothers in the FCU and addressing their goals and social support needs.

In addition to modest effect sizes, limitations of the study include that the findings may not be generalizable to higher income or lower risk families, nor to those who are referred for child behavior problems. Also, although the sample included reasonably high percentages of African American and European American families, there were proportionately fewer Hispanic families and African American families were primarily concentrated in urban rather than rural or suburban neighborhoods. It is also worth noting that, in the case of some risk variables, mothers were the informants both for the predictor and the outcome variables; mothers may also have been reluctant to disclose substance use. However, many of the risk variables were relatively objective demographic data, such as maternal age or education attainment, or tended to have little method overlap with the outcome variable, child problem behavior. Given that 40% of mothers had no live-in partner, we adapted the wording of the marital questionnaire slightly, so that it was applicable to a wider range of intimate relationships. We acknowledge that this may have affected its validity. Even though our study has randomization and an experimental design, measurement would ideally include use of multiple methods and informants. As the children transition to school, it will be possible in future waves to gather multiple informant data.

Strengths of the present study include its large and diverse sample; to our knowledge, this is the largest study to date of moderating mechanisms in a family or parenting intervention. By using three assessment points, we were able to use more powerful growth modeling techniques. The intervention is noteworthy in that it was assessment driven, with careful attention given to selection of participants likely to be at higher risk, within a low-income sample, on the basis of data from longitudinal studies of early-starting problem behavior (Shaw et al., 2000, 2003). When techniques based on MI are used, the effectiveness of a brief intervention is likely to be enhanced.

In conclusion, the FCU intervention holds much promise. When an intervention aims to be effective for at-risk families, it is critical to test whether this is indeed the case with moderator analyses. These findings suggest that the FCU is as likely to be effective with the most distressed and disadvantaged families, within an already low income sample, selected for having multiple risk factors for later problem behavior and drug use. Furthermore, although effect sizes are relatively modest, the brevity of the FCU intervention means there are possibilities for embedding it within a widespread and accessible service system, such as WIC. This means that, taken to scale, the FCU could potentially have considerable public health impact.