Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 Apr 1.
Published in final edited form as: Prev Sci. 2015 Apr;16(3):397–406. doi: 10.1007/s11121-014-0479-x

The Family Check-Up and Service Use in High-Risk Families of Young Children: A Prevention Strategy with a Bridge to Community-Based Treatment

Patty Leijten 1,✉, Daniel S Shaw 2, Frances Gardner 3, Melvin N Wilson 4, Walter Matthys 5, Thomas J Dishion 6
PMCID: PMC4169741  NIHMSID: NIHMS591308  PMID: 24643281

Abstract

Integration of empirically supported prevention programs into existing community services is a critical step toward effecting sustainable change for the highest-risk members in a community. We examined if the Family Check-Up—known to reduce disruptive behavior problems in young children—can provide a bridge to the use of community treatment services among high-risk indigent families. The study’s 731 income-eligible families with a 2-year-old child were screened and randomized to the Family Check-Up (FCU) intervention or a control condition. Families were provided yearly FCUs from age 2 through age 5. Regression analyses on families’ service use at child age 7.5 revealed increased service use, compared with that of the control group. Child disruptive behavior and socioeconomic status moderated the effect of the intervention on service use. Families who reported higher levels of disruptive child behavior and lower socioeconomic status showed more service use, suggesting the intervention increased service use among the highest-risk families. Greater use of community services did not mediate the effect of the FCU on reduced oppositional-defiant child behavior. Implications of these findings for the design and ecology of community treatment services in the context of evidence-based practices are discussed.

Keywords: Family Check-Up, Family intervention, Effectiveness, Engagement, Community-based treatment

To reduce the prevalence of child and adolescent mental health and substance use problems in communities, it is critical to effectively reach and engage those most in need of support (Biglan et al., 1990). High-risk families with few economic resources are typically less apt to engage in child mental health services and prevention programs (e.g., Prinz & Miller, 1991; Zwirs et al., 2006). Aside from the cost of engaging in interventions, socioeconomically disadvantaged families with children showing problem behavior may avoid community treatment agencies because of prior disappointing experiences, stigmatization, language and cultural barriers, and time limitations (Scheppers et al., 2006; Tolan & McKay, 1996). Fortunately, empirically supported family intervention programs have become increasingly successful in engaging families within an epidemiologically defined community (e.g., Dishion et al., 2008; Webster-Stratton, 1998). For some families, however, it may be unrealistic to assume that involvement in a single intervention program will sustain long-term improvements in outcomes, because of structural vulnerabilities in the families and the contexts families live in. In families with lower socioeconomic status (SES), short-term gains of parenting interventions tend to be maintained less well (Leijten et al., 2013). Therefore, an important next step for empirically supported family interventions is to consider how the intervention program fits within the ecology of available community treatment services. Addressing this issue of person–treatment fit would help overcome the science-to-service gap in evidence-based practice (Herschell et al., 2004; Whittaker et al., 2006).

The Family Check-Up (FCU) is an empirically supported intervention that may bridge this gap. This brief intervention is designed to support caregivers’ appraisal of existing strengths and challenges in their family management practices and to motivate families to engage in appropriate treatment services related to parenting (Dishion & Stormshak, 2007). The FCU has repeatedly been shown to decrease youth problem behavior during multiple developmental periods, such as adolescence (Connell & Dishion, 2008; Dishion & Connell, 2008) and early childhood, including oppositional behavior in young children (Dishion et al., 2008; Shaw et al., 2006). In this study, we built on this work by examining the extent to which participation in the FCU effectively engaged high-risk indigent families with services in their community when FCU services became unavailable between child ages 6 and 7.5. We expected that the FCU motivates families to use more services once the FCU becomes unavailable, because families receiving the FCU gain increased insight into their family difficulties and may model the active advocacy demonstrated by the FCU therapists in obtaining services for these difficulties. In addition, we specifically examined the extent to which the FCU motivated the highest-risk families to use services that were appropriate to their assessed needs.

Service Use in High-Risk Families

Although high-risk families may need the assistance of social services more often, their engagement in these services is typically lower (Bussing et al., 2003). Key principles that support families’ engagement are brevity, embedding services in service contexts such as schools or other agencies (Hoagwood & Koretz, 1996; Szapocznik & Kurtines, 1989), providing services in the family’s native language (Castro et al., 2004; Kumpfer et al., 2002), and using a collaborative approach (Miller & Rollnick, 2002). Empirically supported family programs, such as Head Start in the United States and Sure Start in the United Kingdom, can have positive side effects on families’ use of services outside the program, such as immunization (Love et al., 2005; Melhuish et al., 2008). These spillover effects might contribute to the cost effectiveness of family interventions if they improve children’s physical and mental health (Scott et al., 2001).

The Family Check-Up as a Bridge to Service Use

The FCU was developed to overcome barriers to seeking family support, such as fear of stigmatization, language and cultural barriers, and limited time and finances (Dishion & Stormshak, 2009). In two randomized trials with community samples, 25 to 50 % of the caregivers of middle school students who were not seeking services, engaged in the FCU (Stormshak et al, 2011). In two randomized studies of ethnically diverse, income-eligible families with toddlers enrolled in a national food supplement program, 75 to 92 % of the families randomized to the FCU engaged when the child was age 2 years (Dishion et al., 2008; Shaw et al., 2006). In the most recent and larger cohort of boys and girls from three distinct communities (i.e., rural, suburban, and urban), when followed from age 2 through 5 years, 50 % of the families randomized to the FCU engaged in the intervention at child ages 2, 3, and 4 (Dishion et al., 2014).

The FCU model involves two phases. The first is a three-session intervention involving an initial interview, a family assessment, and a feedback session (Dishion & Stormshak, 2007). In the feedback session, caregivers are engaged in a motivational interviewing process during which assessment results are shared, including a focus on both strengths and difficulties, and motivation to change and to receive additional services is discussed. The second phase, which is grounded in the Everyday Parenting curriculum (Dishion et al., 2011), involves support for specific family management practices, including positive behavior support, limit setting, monitoring, and relationship building. Families and therapists decide together which elements of the curriculum will be emphasized, depending on identified family strengths and challenges. Overall, the goal of the FCU is to identify strengths and challenges, enhance parents’ motivation for change, and specifically tailor parent training to meet the individualized needs of the child and family.

The FCU directly addresses several of the barriers to service use that high-risk families may experience. The first of these barriers is caregivers’ experience of criticism and judgment from mental health professionals when engaging with services (Owens et al., 2002; Starr et al., 2002). The FCU’s collaborative approach, motivational interviewing techniques, explicit identification of client strengths, and attention to the individual client’s needs (Miller & Rollnick, 2002; Smith et al., 2014) all work together to establish a strong therapist–client relationship. The FCU’s assessment approach, during which families receive feedback about their family management skills, is known to contribute to clients’ positive thoughts and feelings about the therapist–client relationship (Ackerman et al., 2000; Hilsenroth et al., 2004). A positive therapist–client relationship is a known strong predictor of engagement and successful outcomes of new services, and for effecting enduring change (Forgatch et al., 2005; Kerkorian et al., 2006; Lambert & Barley, 2001). Second, the FCU’s flexible and adaptive approach facilitates the therapist’s ability to adapt services to the caregiver’s culture (ethnicity and SES; Boyd-Ball & Dishion, 2006; Stormshak et al., 2011). Third, the FCU uses home visiting to reduce barriers to engagement for disadvantaged families (Szapocznik & Kurtines, 1989).

Service Use for Those Who Most Need It

Families targeted by the Family Check-Up are a community sample of high-risk indigent families. As such, the service needs of a percentage of the families far exceed those that the Family Check-Up can address within the context of this study. Examples include severe mental health difficulties of caregivers, siblings or close relatives living in the home (e.g., schizophrenia, incapacitating depression, substance abuse, domestic violence, criminal offending, and post-traumatic stress disorder). From an ecological perspective, we are aware that such ‘disruptors’ affect caregivers parenting practices, which in turn, influence children’s social and emotional development (Bradley & Corwyn, 2002; Bronfenbrenner, 1986). Thus, engagement in community treatment services would be a hopeful remedy to some family disruption.

Because of its specific emphasis on increasing families’ awareness of their strengths and difficulties and possible need for help, the FCU may be especially effective for connecting high-needs families with a wide array of services in the community. Three primary risk factors for unfavorable child outcomes in early childhood are disruptive child behavior (Tremblay et al., 1992), maternal depression (Cummings & Davies 1994; Gross et al., 2009; Kim-Cohen et al., 2005), and low SES (Bradley et al., 2001; Eamon, 2001). Increased services use among families with high levels of these risk factors would be especially relevant, both in terms of improving the lives of families that need services the most and in terms of cost effectiveness of community treatment services. In this study, we not only examined the extent to which the FCU leads to more engagement in services, but also the extent to which the FCU specifically increases the engagement of families with the highest levels of child disruptive behavior, maternal depression, and/or socioeconomic risk in these services.

To identify the most efficient use of the FCU intervention, we examined possible dose–response effects of the FCU. Each year, families in the intervention condition decide whether or not they will participate in the feedback session. Dose–response effects are repeatedly shown in family interventions (Wilson & Lipsey, 2001), and effects of the FCU on families’ engagement in other services may be stronger if families participate more frequently in the FCU. In other words, the effect of the FCU on families’ engagement in services may depend on the FCU “dosage” that families receive.

Service Use as a Mechanism of FCU Effectiveness

If the FCU increases families’ engagement in community services, then this increased use may in turn lead to more favorable child outcomes. As noted earlier, the FCU, implemented beginning at child age 2, has been established as an intervention for reliably reducing child oppositional and aggressive behavior and parental depression; these effects have been shown through early childhood and through the early school-age period, according to both parent and teacher reports (Dishion et al., 2014; Gardner et al., 2009; Shaw et al., 2009). We examined if families’ use of community services as a result of the FCU intervention mediates the effect of the FCU on reduced oppositional-defiant child behavior.

This Study

We sought to determine if the FCU is effective for engaging families in community treatment services. First, we hypothesized that families randomly assigned to the FCU would use more community services than would families in the control condition. Second, because the FCU may help parents identify their own and their child’s problematic behaviors, we hypothesized that maternal depression and disruptive child behavior would moderate the effects of the FCU on service use. Third, because the FCU targets many of the barriers to service engagement experienced by disadvantaged families, we hypothesized that family socioeconomic status would moderate the effects of the FCU on service use. Fourth, we examined the possibility of a dose–response effect in which more frequent participation in the FCU would lead to stronger effects of the FCU on service use. Finally, we examined whether increased engagement in community services as a result of the FCU mediates improvements in children’s oppositional-defiant behavior from early childhood to early middle childhood.

Method

Participants

Participants were 731 families with a 2-year-old. Children (49 % female) had a mean age of 29.9 months (SD=3.2) at the time of the age 2 assessments. Of the 731 families, 272 (37 %) were recruited in Pittsburgh, 271 (37 %) in Eugene, and 188 (26 %) in Charlottesville. Across sites, 50.1 % of the children were European American, 27.9 % were African American, 13.4 % were Hispanic American, 13.0 % were biracial, and 8.9 % represented other ethnicities (e.g., American Indian or Native Hawaiian). More than two thirds of the enrolled families had an annual family income of less than $20,000 during the 2002–2003 screening period. Forty-one percent of the mothers had a high school diploma or GED equivalence, and an additional 32 % had 1 to 2 years post-high school education.

Recruitment

Families were contacted at Women, Infants, and Children Nutritional Food Supplement (WIC) sites in Pittsburgh, Pennsylvania; Eugene, Oregon; and Charlottesville, Virginia. Families with a 2-year-old child were asked to fill out a brief screening questionnaire to measure eligibility for an intervention program. Families were invited to participate if they scored at or above 1 standard deviation above the normative average scores on the screening measures of at least two out of three domains: (a) child behavior (conduct problems, high-conflict relationship with adults), (b) family problems (maternal depression or substance-use problems, daily parenting challenges, teen parent status), and/or (c) low socioeconomic status (low education achievement, low family income). In total, 1,666 families were screened. Of the 879 families that were qualified, 731 (83 %) participated in this study (Dishion et al., 2008).

Procedure

Primary caregivers were visited yearly at home at child ages 2 to 5 years and again at age 7.5. Families in the FCU condition and the control condition received the same assessment visits, which involved questionnaires and structured interaction activities that were videotaped and later coded. After the assessment visit, only families in the FCU condition were offered a get-to-know-you (GTKY) session and a feedback session. The GTKY typically occurs prior to the assessment in clinical practice of the FCU, but was offered after the assessment in this research study so control and intervention families’ first contact was the same. The GTKY allows therapists to establish rapport with families and find out what issues are important to them. Therapists use this information in conjunction with data obtained from the formal assessment, which includes questionnaires and observed play activities, to provide feedback to families at the feedback session. At child age 2, 80 % of the families participated in the feedback session. Participation decreased over time, with 55 % participating in the feedback session at child age 5.

The feedback session of the FCU was offered to all families in the intervention condition at child ages 2, 3, 4, and 5 years. At age 6, families were notified that intervention services would not be available because of lack of funding for the study. At age 7.5, families’ engagement in services between child age 6 and 7.5 was assessed. Families received payment for participating in the 2- to 3-h-long assessments (i.e., $100 at age 2 and as much as $180 at age 7.5) and a $25 gift certificate for participating in the feedback session. Data were collected between 2003 and 2010. Parental written consent was obtained for all participants. Institutional review board approval was received.

Instruments

Service Use

Primary caregivers reported their families’ use of formal and informal helping services. Formal services included mental health counseling, substance abuse treatment, help from doctors or nurses, and agency help specifically for their children. Informal services included community support agencies, help from religious groups, and assistance from other parents or relatives. Primary caregivers responded with “yes” (coded as 1) or “no” (coded as 0) to report if their family had engaged in this form of service between child age 6 and 7.5. Responses were then summed separately into scores for formal and informal services use and together for total service use.

Disruptive Child Behavior

The intensity score of the Eyberg Child Behavior Inventory (ECBI; Robinson et al., 1980) was used to measure the child’s initial level of disruptive behavior at age 2, including symptoms of oppositional and aggressive behavior. Primary caregivers reported the extent of child disruptive problem behavior on a scale ranging from 1 (never) to 7 (always). The ECBI has been highly correlated with independent observations of children’s behavior, differentiates clinic-referred and nonclinic populations (Robinson et al., 1980), and shows high test–retest reliability (0.86) and internal consistency (0.98; Webster-Stratton, 1985). Internal consistency of the ECBI was α=.86.

Child Oppositional–defiant Behavior

A measure of child oppositional-defiant and defiant behavior was created from the Child Behavior Checklist for ages 1.5–5 and ages 6–18 to measure parent-reported oppositional-defiant behavior at age 7.5 (CBCL; Achenbach & Rescorla, 2001). As was accomplished in earlier reports from this study (Dishion et al., 2014), we computed a mean factor score for oppositional-defiant child behavior that mapped onto DSM-IV criteria for oppositional-defiant disorder and DSM-IV items about aggression relevant to conduct disorder and that was developmentally meaningful across the study’s age range (2 to 7.5 years). The oppositional-defiant items in this score include the following: is cruel to animals, destroys own things, destroys others’ things, gets into many fights, physically attacks people, is defiant, is disobedient, and has temper tantrums. Internal consistency of the CBCL was α=.78.

Child Oppositional-defiant Behavior at School

The DSM-oriented Oppositional Defiant Problems scale from the Teacher Report Form (TRF; Achenbach & Rescorla, 2001) was used to measure teacher-reported oppositional behavior at age 7.5. Internal consistency of the TRF was α=.90.

Maternal Depressive Symptoms

The Center for Epidemiological Studies on Depression Scale (CES-D; Radloff, 1977) was used as a measure of maternal depression at child age 2. The CES-D is a well-established, 20-item measure of depressive symptomology. Participants reported how frequently they had experienced listed depressive symptoms during the past week by using a scale ranging from 0 (less than 1 day) to 3 (5–7 days). Internal consistency of the CES-D was α=.74.

Socioeconomic Status

A demographics questionnaire was administered to the primary caregivers that included questions about parental education, ranging from 1 (no formal schooling) to 9 (graduate degree); annual income, ranging from 1 (<$5.000) to 13 (>$90,000); financial aid, in which 0= no, 1=yes, reversed; overcrowding in the house, in which 0= no, 1=yes, reversed; and housing, in which 1=live with a relative, 4=own your own home. SES was computed as a composite measure of the standardized scores of these five demographics. We used SES measured at both age 2 and age 7.5, because meaningful changes in families’ levels of SES were expected, for two principal reasons: a substantial number of teen mothers in the sample (23 %) were expected to finish their education and subsequently increase their SES from child age 2 to 7.5, and the economic recession that took place in the United States starting in 2008 may have substantially affected families’ levels of SES during that age span.

Analytic Strategy

Our analyses consisted of five steps that match the five hypotheses. First, we used regression analysis to test if families in the FCU condition compared with the control condition predicted greater service use. Second, we used hierarchical regression analyses to test if early disruptive child behavior and maternal depression moderated the effect of the FCU on service use. Third, we used hierarchical regression analyses to test if SES moderated the effect of the FCU on service use. Fourth, to test dose–response effects, we used regression analysis in the intervention condition to test if the number of times families participate in the FCU predicted service use. Finally, we used regression analyses to test if increased service use predicts reduced child oppositional-defiant behavior and is a mediator of the effect of the FCU on reduced child oppositional-defiant behavior.

Results

Preliminary Analyses

Caregivers reported that on average, they had engaged in one or two different forms of community services (M=1.52, SD= 1.26) in the year that the FCU was not offered (between child age 6 and 7.5). Seventy-five percent of the families across conditions had engaged in at least one form of community service. Services that families used most were agencies serving children (35 % of the families), assistance from other parents or relatives (34 %), and mental health or counseling services (31 %). Family SES at age 2 and at age 7.5 was modestly correlated (r=.13, ns), indicating low stability of family SES over time, regardless of treatment condition. More specifically, 18 % of the families increased at least 1 standard deviation in their SES between child age 2 and 7.5, and 13 % of the families decreased at least 1 standard deviation in their SES between age 2 to 7.5. Change in family SES was not affected by intervention status (β=.04, ns).

The FCU feedback session was offered yearly to families in the intervention condition between child age 2 and 5, resulting in four opportunities to participate. On average, intervention families participated three to four times (M=3.55, SD=2.03). Participation was related to families’ SES, such that families with lower SES participated more often (r=−.41). SES was therefore included as a covariate in analysis of the dose–response effect. Families’ service use was not related to children’s gender, age, ethnicity, or project site (ps>.11; see Table 1).

Table 1.

Families’ service use is higher in the FCU condition than in the control condition, and unrelated to children’s gender, age, ethnicity, or urbanicity of living location

Total services Formal services Informal services
Gender
 Boys (51 %) 1.46 .85 .61
 Girls (49 %) 1.60 .90 .70
 Age (r) −.05 −.01 −.07
Ethnicity
 European American (46 %) 1.63 .90 .73
 African American (28 %) 1.37 .88 .48
 Other/biracial (13 %) 1.51 .82 .69
Living location
 Large urban (31 %) 1.57 .94 .63
 Small urban (21.5 %) 1.56 .86 .69
 Suburban (21.5 %) 1.51 .85 .66
 Rural (26 %) 1.50 .82 .68
Intervention condition*
 FCU 1.68 .94 .72
 Control 1.40 .80 .59
Open in a new tab
*

p<.05

Primary Analyses

Consistent with the hypothesis, the FCU resulted in greater service use in the year the FCU was not offered (β=.11, p<.05, d=.21). Random assignment to the FCU was positively linked to more often seeking mental health counseling in the community (β=.09, p<.05, d=.18) or chemical dependency treatment (β=.11, p<.05, d=.22) and seeking assistance from religious groups (β=.10, p<.05, d=.19) or from community support agencies (β=.08, p<.05, d=.17).

The second step was to examine family characteristics that moderate the covariation observed between random assignment to the FCU and caregivers seeking community treatment services. Disruptive child behavior moderated the effect of the FCU on families’ service use. The effect of the FCU on service use was strongest in families with more-disruptive children, as shown by the significant condition×disruptive behavior interaction effect (β=.43, p<.05, d=.33). Whereas in the control condition families with higher levels of disruptive child behavior engaged in services at comparable rates to families with lower levels of disruptive child behavior, in the FCU condition families with higher levels of disruptive child behavior engaged in services more often than did families with lower levels of disruptive behavior (Fig. 1). The Johnson-Neyman regions of significance approach (cf. Hayes & Matthes, 2009) showed that the effect of the FCU on families’ service use became significant when children scored 120 or higher on the ECBI, which reflects a score of approximately 0.5 standard deviation above the population norm (Burns & Patterson, 2001) and 57 % of this study’s sample. In contrast to disruptive child behavior, maternal depression was unrelated to families seeking treatment in the context of the FCU (β=.03, ns).

Fig. 1.

Fig. 1

Open in a new tab

The FCU motivates families with high levels of disruptive child behavior to use more community-based service

Third, Families’ SES at age 7.5 moderated the magnitude of the association between FCU group status and services use, but only for the use of formal services. The effect of the FCU on formal service use was strongest in families with lowest SES, as shown by the significant condition×SES at age 7.5 interaction effect (β=−1.88, p<.05, d=.25). In the control condition, families with the lowest SES engaged less often in services than did families with somewhat higher SES. The FCU diminished this engagement gap, resulting in equal engagement in services for families with low and moderate SES. This effect was unique to families’ SES at age 7.5, which reflects families’ SES in the year of the assessed services. Family SES at age 2, which reflects families’ SES before the start of the intervention, was not related to the effect of the FCU on families’ service use in the year of the assessed services.

Fourth, there was a dose–response effect in the intervention condition. Families who participated more often in yearly FCU feedback sessions engaged more often in services during the intervention hiatus (β=.13, p<.05, d=.27). This effect was mainly attributable to the finding that families obtaining more feedback sessions used more formal services (β=.15, p<.05, d=.30) than informal services (β=.05, ns, d=.10).

Finally, increased engagement in services as a result of the FCU did not mediate effectiveness of the FCU on reduced oppositional-defiant child behavior. Increased service use was unrelated to oppositional-defiant behavior as rated by teachers (β=.05, ns, d=.13) and related to more (rather than less) oppositional-defiant behavior as rated by caregivers (β=.15, p<.05, d=.30). In other words, caregivers who used more community services between child age 6 and 7.5 reported more oppositional-defiant behavior in their children at age 7.5 than did caregivers who used less community services after participating in the FCU.

Discussion

We examined whether the empirically supported FCU intervention can provide a bridge to the use of community treatment services among high-risk indigent families. Our results confirm that the FCU enhances families’ service use; families in the FCU condition used more community services than did families in the control condition. In particular, they engaged more often in mental health counseling, chemical dependency treatment, assistance from religious groups, and community support agencies. This effect may be explained by the specific emphasis of the FCU on increasing family awareness of strengths and difficulties in family management and possible need for help (Dishion & Stormshak, 2007). The FCU seems to contribute to families’ motivation to search in their own community for the help they need.

We found evidence that the FCU led to more service use especially for those families most in need of assistance. Families with children who had higher initial levels of disruptive behavior and those with the lowest SES showed the greatest use of community services as a result of engagement in the FCU. The finding that families with highly disruptive children engaged in more service use after the FCU is in line with findings that families with more disruptive child behavior engage more in the FCU (Connell et al., 2007), and with both theory and empirical findings that perception about family challenges is an important precursor of help-seeking behavior (e.g., Goldberg & Huxley, 1980; Sayal, 2006; Teagle, 2002). The finding that families with the lowest SES engaged in community services more often after the FCU, relative to those in the control group, supports the notion that the FCU addresses several of the barriers that may prevent low-SES families from seeking help. The FCU addresses caregivers’ experience of criticism and judgment from mental health professionals in that it fosters strong and collaborative therapist–client relationships and emphasizes adaptation of services to caregivers’ culture (Miller & Rollnick, 2002; Smith et al., 2014).

Among study findings was a dose–response effect: families that engaged more often in the FCU showed greater engagement in community services during the year the FCU was not offered. Existing literature about dose–response effects is inconsistent. Although many studies on family interventions do not show dose–response effects (e.g., Nix et al., 2009), others suggest that dosage of treatment might play a role in the effectiveness of interventions (e.g., Baydar et al., 2003; Wilson & Lipsey, 2001). The FCU differs from many other intervention programs with respect to its time frame. The FCU is relatively brief and repeated yearly, whereas most family interventions consist of multiple meetings that are typically offered weekly or biweekly (e.g., Sanders, 1999; Webster-Stratton, 2001). It may be that dose–response effects become more salient in this brief but annual framework, much like models for pediatric check-up visits or preventive dental exams. Granted, we cannot rule out the possibility that the dose–response effect in our study reflects caregivers’ general tendency to engage in help and was not uniquely caused by engagement in the FCU. However, additional analyses in which we controlled for parents’ engagement in community services during the first years of the FCU showed that initial levels of community service use did not affect the dose–response effect of the FCU on families’ engagement in community services in the year the FCU was not offered. The dose–response effect therefore suggests that yearly repeated FCUs help optimize change in families’ engagement in community services.

Increased use of community services did not mediate the effects of the FCU on oppositional-defiant child behavior as rated by caregivers and teachers at age 7.5. In fact, engagement in community services was related to more (rather than less) oppositional-defiant child behavior in the following year, as reported by caregivers. The absence of a direct link between engagement in community services and better child adjustment is not all that surprising. It is well established that direct treatment effects on child behavior for regular community-based services for children and families are negligible (Weiss et al., 1999). Many factors limit the effectiveness of community-based services, which are likely to be compromised by limited use of empirically based practices and poor resources available for implementing interventions with fidelity. Thus, engaging families and establishing empirically supported family intervention programs is only half the battle; there is a need to design community-based mental health delivery systems that are empirically based, efficient, and implemented with fidelity. The brief and feasible FCU intervention shows that mental health services do not need to be expensive to be effective. In contrast, much of what is currently done in community treatment service may be more expensive, less effective, and can have in some cases even iatrogenic effects (e.g., Dodge et al., 2006).

The finding that randomization to the FCU mobilizes care-givers’ efforts to change and seek services builds on studies of the Drinker’s Check-Up. For example, randomization to the Drinker’s Check-Up was related to long-term changes in problem drinking, and those in the experimental condition often sought other types of addiction treatment services (Miller & Sovereign, 1989). If one assumes a self-regulatory perspective on behavior change, providing caregivers with respectful and helpful feedback about their child’s behavior and their family management prompts an individualized behavior change process that will likely involve use of resources available within the ecology of each family. It is particularly interesting that the hiatus in the delivery of FCU intervention services was the time when intervention families showed an increased level of service use, compared with that of controls. A certain wisdom is suggested on the part of caregivers, in that they shopped for services that fit their perceived needs when help from the FCU was not available. More generally, the brief FCU may increase the reach of other evidence-based programs (e.g., empirically supported treatment programs for parental mental health problems or child aggression) by supporting high-risk families to engage in treatment services.

This is not the first study that suggests that those families most in need are those that engage the most in the FCU and also benefit. In a study involving middle school youths, Connell and colleagues (Connell et al., 2007) found that the highest-risk families and youths were the most likely to engage in the school-based FCU, and correspondingly, the most likely to benefit in terms of long-term outcomes. In an earlier report about the current sample, we found that caregivers reporting high levels of disruptive behavior in their 2-year-old children were most likely to benefit from randomization to the FCU (Dishion et al., 2008). The often neglected corollary of this finding is that low-risk families often opt out of the FCU, and their children continue to show long-term positive adjustment outcomes. These patterns of self-selection suggest that caregivers’ appraisal of their family’s needs, coupled with the availability of nonpejorative, initially brief, and high-quality services, is critical when considering the prevention strategies that have high levels of reach and that potentially reduce the public health prevalence of mental health problems in children and families.

The results of our study should be interpreted in the context of its limitations. We used the naturally occurring hiatus in availability of the FCU to study its effect on families’ service use. We cannot test whether this hiatus caused families in the FCU to engage more in community services or if the same pattern of services use would appear if the FCU had been offered yearly without a hiatus. Also, because we lack information about exactly which specific programs or treatment families received in their community—and the extent to which they are empirically supported—we were unable to test the effect of the use of community services on various child outcomes, including oppositional behavior. Finally, the finding that increased use of community services did not have beneficial effects on disruptive child behavior does not exclude the possibility that community services might play an important role in mobilizing families’ efforts to seek help. Also, community services may decrease risk factors such as parental depression and parenting stress. Thus, although in our study community services were not associated with positive change in child behavior, they may still be important for encouraging families to engage in treatment. This issue reiterates the importance of having community-based mental health delivery systems that are empirically based, efficient, and implemented with fidelity.

Ours is among the first randomized studies of an empirically developed family intervention that examined the impact of an intervention on caregivers’ use of services within the ecology of the community. Our results indicate that the brief, family-based FCU intervention is effective for motivating families to engage in treatment services, especially those families with the highest needs for help, but we did not find evidence that engagement in services in turn leads to more favorable child outcomes. Future research is needed to examine which community services might be effective and which services might be ineffective or even iatrogenic. This strategy is critical to ensure that if we are to bridge the chasm between empirically supported prevention and community-based treatment, the community services families receive add to, rather than diminish, the potential for positive change.

Acknowledgments

This research was supported by National Institute of Mental Health Grant MH06291 and National Institute on Drug Abuse Grant DA016110.

Contributor Information

Patty Leijten, Email: p.leijten@uva.nl, Utrecht University, Utrecht, Netherlands. University of Oxford, Oxford, United Kingdom.

Daniel S. Shaw, University of Pittsburgh, Pittsburgh, PA, USA

Frances Gardner, University of Oxford, Oxford, United Kingdom.

Melvin N. Wilson, University of Virginia, Charlottesville, VA, USA

Walter Matthys, Utrecht University, Utrecht, Netherlands. University Medical Center Utrecht, Utrecht, Netherlands.

Thomas J. Dishion, Arizona State University, Phoenix, AZ, USA

References

  1. Achenbach TM, Rescorla LA. Manual for the ASEBA school-age forms & profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth, and Families; 2001. [Google Scholar]
  2. Ackerman SJ, Hilsenroth MJ, Baity MR, Blagys MD. Interaction of therapeutic process and alliance during psychological assessment. Journal of Personality Assessment. 2000;75:82–109. doi: 10.1207/S15327752JPA7501_7. [DOI] [PubMed] [Google Scholar]
  3. Baydar N, Reid MJ, Webster-Stratton C. The role of mental health factors and program engagement in the effectiveness of a preventive parenting program for Head Start mothers. Child Development. 2003;74:1433–1453. doi: 10.1111/1467-8624.00616. [DOI] [PubMed] [Google Scholar]
  4. Biglan A, Glasgow RE, Singer G. The need for a science of larger social units: A contextual approach. Behavior Therapy. 1990;21:195–215. doi: 10.1016/S0005-7894(05)80277-1. [DOI] [Google Scholar]
  5. Boyd-Ball AJ, Dishion TJ. Family-centered treatment for American Indian adolescent substance abuse: Toward a culturally and historically informed strategy. In: Liddle H, Rowe C, editors. Treating adolescent substance abuse: State of the science. New York: Cambridge University Press; 2006. pp. 423–448. [Google Scholar]
  6. Bradley RH, Corwyn RF. Socioeconomic status and child development. Annual Review of Psychology. 2002;53:371–399. doi: 10.1146/annurev.psych.53.100901.135233. [DOI] [PubMed] [Google Scholar]
  7. Bradley RH, Corwyn RF, Burchinal M, McAdoo HP, Garcia-Coll C. The home environment of children in the United States Part II: Relations with behavioral development through age thirteen. Child Development. 2001;72:1868–1886. doi: 10.1111/1467-8624.t01-1-00383. [DOI] [PubMed] [Google Scholar]
  8. Bronfenbrenner U. Ecology of the family as a context for human development: Research perspectives. Developmental Psychology. 1986;22:723–742. doi: 10.1037/0012-1649.22.6.723. [DOI] [Google Scholar]
  9. Burns GL, Patterson DR. Normative data on the Eyberg Child Behavior Inventory and Sutter–Eyberg Student Behavior Inventory: Parent and teacher rating scales of disruptive behavior problems in children and adolescents. Child and Family Behavior Therapy. 2001;23:15–28. doi: 10.1300/J019v23n01_02. [DOI] [Google Scholar]
  10. Bussing R, Zima BT, Gary FA, Garvan CW. Barriers to detection, help-seeking, and service use for children with ADHD symptoms. Journal of Behavioral Health Services and Research. 2003;30:176–89. doi: 10.1007/BF02289806. [DOI] [PubMed] [Google Scholar]
  11. Castro FG, Barrera M, Jr, Martinez CR. The cultural adaptation of preventive interventions: Resolving tensions between fidelity and fit. Prevention Science. 2004;5:41–45. doi: 10.1023/b:prev.0000013980.12412.cd. [DOI] [PubMed] [Google Scholar]
  12. Connell A, Dishion TJ. Reducing depression among at-risk early adolescents: Three-year effects of a family-centered intervention embedded within schools. Journal of Family Psychology. 2008;22:574–585. doi: 10.1037/0893-3200.22.3.574. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Connell A, Dishion TJ, Yasui M, Kavanagh K. An adaptive approach to family intervention: Linking engagement in family-centered intervention to reductions in adolescent problem behavior. Journal of Consulting and Clinical Psychology. 2007;75:568–579. doi: 10.1037/0022-006X.75.4.568. [DOI] [PubMed] [Google Scholar]
  14. Cummings EM, Davies PT. Maternal depression and child development. Journal of Child Psychology and Psychiatry. 1994;35:73–112. doi: 10.1111/j.1469-7610.1994.tb01133.x. [DOI] [PubMed] [Google Scholar]
  15. Dishion TJ, Connell AM. An ecological approach to family intervention to prevent adolescent drug use: Linking parent engagement to long-term reductions of tobacco, alcohol and marijuana use. In: Heinrichs N, Hahlweg K, Doepfner M, editors. Strengthening families: Evidence-based approaches to support child mental health. Münster: Verlag für Psychotherapie; 2008. pp. 403–433. [Google Scholar]
  16. Dishion TJ, Stormshak EA. Intervening in children’s lives: An ecological, family-centered approach to mental health care. Washington, DC: American Psychological Association; 2007. [Google Scholar]
  17. Dishion TJ, Stormshak EA. A family-centered intervention strategy for public middle schools. In: Bray J, Stanton M, editors. Blackwell handbook of family psychology. Malden: Blackwell Publishing; 2009. pp. 499–514. [Google Scholar]
  18. Dishion TJ, Shaw DS, Connell AM, Wilson MN, Gardner F, Weaver C. The Family Check-Up with high-risk indigent families: Preventing problem behavior by increasing parents’ positive behavior support in early childhood. Child Development. 2008;79:1395–1414. doi: 10.1111/j.1467-8624.2008.01195.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Dishion TJ, Stormshak EA, Kavanagh K. Everyday parenting: A professional’s guide to building family management skills. Champaign: Research Press; 2011. [Google Scholar]
  20. Dishion TJ, Brennan LM, Shaw DS, McEachern AD, Wilson MN, Jo B. Prevention of problem behavior through annual family Check-Ups in early childhood: Intervention effects from home to early elementary school. Journal of Abnormal Child Psychology. 2014;42:343–354. doi: 10.1007/s10802-013-9768-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Dodge KA, Dishion TJ, Lansford JE. Deviant peer influences in intervention and public policy for youth. Social Policy Report. 2006;20:3–19. [Google Scholar]
  22. Eamon M. Poverty, parenting, peer, and neighborhood influences on young adolescent antisocial behavior. Journal of Social Service Research. 2001;28:1–23. doi: 10.1300/J079v28n01_01. [DOI] [Google Scholar]
  23. Forgatch MS, Patterson GR, DeGarmo DS. Evaluating fidelity: Predictive validity for a measure of competent adherence to the Oregon Model of Parent Management Training (PMTO) Behavior Therapy. 2005;36:3–13. doi: 10.1016/S0005-7894(05)80049-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Gardner F, Connell A, Trentacosta C, Shaw D, Dishion T, Wilson M. Moderators of outcome in a brief family-centered intervention for preventing early problem behaviour. Journal of Consulting and Clinical Psychology. 2009;77:543–553. doi: 10.1037/a0015622. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Goldberg D, Huxley P. Mental illness in the community: The pathway to psychiatric care. Tavistock; London, UK/New York, NY: 1980. [Google Scholar]
  26. Gross HE, Shaw DS, Burwell RA, Nagin DS. Transactional processes in child disruptive behavior and maternal depression: A longitudinal study from early childhood to adolescence. Development and Psychopathology. 2009;21:139–156. doi: 10.1017/S0954579409000091. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Hayes AF, Matthes J. Computational procedures for probing interactions in OLS and logistic regression: SPSS and SAS implementations. Behavior Research Methods. 2009;41:924–936. doi: 10.3758/BRM.41.3.924. [DOI] [PubMed] [Google Scholar]
  28. Herschell AD, McNeil CB, McNeil DW. Clinical child psychology’s progress in disseminating empirically supported treatments. Clinical Psychology: Science and Practice. 2004;11:267–288. doi: 10.1093/clipsy.bph082. [DOI] [Google Scholar]
  29. Hilsenroth MJ, Peters EJ, Ackerman SJ. The development of therapeutic alliance during psychological assessment: Patient and therapist perspectives across treatment. Journal of Personality Assessment. 2004;83:332–344. doi: 10.1207/s15327752jpa8303_14. [DOI] [PubMed] [Google Scholar]
  30. Hoagwood K, Koretz D. Embedding prevention services within systems of care: Strengthening the nexus for children. Applied and Preventive Psychology. 1996;5:225–234. doi: 10.1016/S0962-1849(96)80014-X. [DOI] [Google Scholar]
  31. Kerkorian D, McKay M, Bannon WM., Jr Seeking help a second time: Parents’/caregivers’ characterizations of previous experiences with mental health services for their children and perceptions of barriers to future use. American Journal of Orthopsychiatry. 2006;76:161–166. doi: 10.1037/0002-9432.76.2.161. [DOI] [PubMed] [Google Scholar]
  32. Kim-Cohen J, Moffitt TE, Taylor A, Pawlby SJ, Caspi A. Maternal depression and children’s antisocial behavior: Nature and nurture effects. Archives of General Psychiatry. 2005;62:173–181. doi: 10.1001/archpsyc.62.2.173. [DOI] [PubMed] [Google Scholar]
  33. Kumpfer KL, Alvarado R, Smith P, Bellamy N. Cultural sensitivity and adaptation in family-based prevention interventions. Prevention Science. 2002;3:241–246. doi: 10.1023/A:1019902902119. [DOI] [PubMed] [Google Scholar]
  34. Lambert MJ, Barley DE. Research summary on the therapeutic relationship and psychotherapy outcome. Psychotherapy. 2001;38:357–361. doi: 10.1037/0033-3204.38.4.357. [DOI] [Google Scholar]
  35. Leijten P, Raaijmakers MAJ, Orobio de Castro B, Matthys W. Does socioeconomic status matter? A meta-analysis on parent training effectiveness for disruptive child behavior. Journal of Clinical Child & Adolescent Psychology. 2013;42:384–392. doi: 10.1080/15374416.2013.769169. [DOI] [PubMed] [Google Scholar]
  36. Love JM, Kisker EE, Ross C, Raikes H, Constantine J, Boller K, Vogel C. The effectiveness of Early Head Start for 3-year-old children and their parents. Developmental Psychology. 2005;41:885–901. doi: 10.1037/0012-1649.41.6.885. [DOI] [PubMed] [Google Scholar]
  37. Melhuish E, Belsky J, Leyland AH, Barnes J. Effects of fully-established Sure Start Local Programmes on 3-year-old children and their families living in England: A quasi-experimental observational study. The Lancet. 2008;8:1641–1647. doi: 10.1016/S0140-6736(08)61687-6. [DOI] [PubMed] [Google Scholar]
  38. Miller WR, Rollnick S. Motivational interviewing: Preparing people for change. 2. New York: Guilford Press; 2002. [Google Scholar]
  39. Miller WR, Sovereign RG. The checkup: A model for early intervention in addictive behaviors. In: Loberg T, Miller WR, Nathan PE, Marlatt GA, editors. Addictive behaviors: Prevention and early intervention. Amsterdam: Sweta and Zeitlinger; 1989. pp. 219–231. [Google Scholar]
  40. Nix RL, Bierman KL, McMahon RJ the Conduct Problems Prevention Research Group. How attendance and quality of participation affect treatment response to parent management training. Journal of Consulting and Clinical Psychology. 2009;77:429–438. doi: 10.1037/a0015028. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Owens PL, Hoagwood K, Horwitz SM, Leaf PJ, Poduska JM, Kellam SG, Ialongo NS. Barriers to children’s mental health services. Journal of the American Academy of Child & Adolescent Psychiatry. 2002;41:731–738. doi: 10.1097/00004583-200206000-00013. [DOI] [PubMed] [Google Scholar]
  42. Prinz RJ, Miller GE. Issues in understanding and treating childhood conduct problems in disadvantaged populations. Journal of Clinical Child Psychology. 1991;20:379–385. doi: 10.1207/s15374424jccp2004_6. [DOI] [Google Scholar]
  43. Radloff LS. The CES-D Scale: A self-report depression scale for research in the general population. Applied Psychological Measurement. 1977;1:385–401. doi: 10.1177/014662167700100306. [DOI] [Google Scholar]
  44. Robinson EA, Eyberg SM, Ross AW. The standardization of an inventory of child conduct problem behaviors. Journal of Clinical Child Psychology. 1980;9:22–28. doi: 10.1080/15374418009532938. [DOI] [Google Scholar]
  45. Sanders MR. Triple P–Positive Parenting Program: Towards an empirically validated multilevel parenting and family support strategy for the prevention of behavior and emotional problems in children. Clinical Child and Family Psychology Review. 1999;2:71–90. doi: 10.1023/A:1021843613840. [DOI] [PubMed] [Google Scholar]
  46. Sayal K. Annotation: Pathways to care for children with mental health problems. Journal of Child Psychology and Psychiatry. 2006;47:649–659. doi: 10.1111/j.1469-7610.2005.01543.x. [DOI] [PubMed] [Google Scholar]
  47. Scheppers E, van Dongen E, Dekker J, Geertzen J, Dekker J. Potential barriers to the use of health services among ethnic minorities: A review. Family Practice. 2006;23:325–348. doi: 10.1093/fampra/cmi113. [DOI] [PubMed] [Google Scholar]
  48. Scott S, Knapp M, Henderson J, Maughan B. Financial cost of social exclusion: Follow up study of antisocial children into adulthood. British Medical Journal. 2001;323:1–5. doi: 10.1136/bmj.323.7306.191. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Shaw DS, Dishion TJ, Supplee LH, Gardner F, Arnds K. Randomization trial of a family-centered approach to the prevention of early conduct problems: 2-year effects of the Family Check-Up in early childhood. Journal of Consulting and Clinical Psychology. 2006;74:1–9. doi: 10.1037/0022-006X.74.1.1. [DOI] [PubMed] [Google Scholar]
  50. Shaw DS, Connell AM, Dishion TJ, Wilson MN, Gardner F. Improvements in maternal depression as a mediator of intervention effects on early childhood problem behavior. Development and Psychopathology. 2009;21:417–439. doi: 10.1017/S0954579409000236. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Smith JD, Knoble NB, Zerr A, Dishion TJ, Stormshak EA. Multicultural competence and the Family Check-Up: Indirect effect on adolescent antisocial behavior through family conflict. In: Piña A, Gonzales N, editors. Journal of Clinical Child and Adolescent Psychology. 2014. The role of theory and culture in child and adolescent prevention science [Special section] Manuscript in press. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Starr S, Campbell LR, Herrick CA. Factors affecting use of the mental health system by rural children. Issues in Mental Health Nursing. 2002;23:291–304. doi: 10.1080/016128402753543027. [DOI] [PubMed] [Google Scholar]
  53. Stormshak EA, Connell AM, Véronneau MH, Myers MW, Dishion TJ, Kavanagh K, Caruthers AS. An ecological approach to promoting early adolescent mental health and social adaptation: Family-centered intervention in public middle schools. Child Development. 2011;82:209–225. doi: 10.1111/j.1467-8624.2010.01551.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Szapocznik J, Kurtines WM. Breakthroughs in family therapy with drug abusing problem youth. New York: Springer; 1989. [Google Scholar]
  55. Teagle SE. Parental problem recognition and child mental health service use. Mental Health Services Research. 2002;4:257–266. doi: 10.1023/A:1020981019342. [DOI] [PubMed] [Google Scholar]
  56. Tolan PH, McKay M. Preventing serious antisocial behavior in inner-city children: An empirically based family intervention program. Family Relations: Journal of Applied Family and Child Studies. 1996;45:145–155. doi: 10.2307/585285. [DOI] [Google Scholar]
  57. Tremblay RE, Masse B, Perron D, Leblanc M, Schwartzman AE, Ledingham JE. Early disruptive behavior, poor school achievement, delinquent behavior, and delinquent personality: Longitudinal analyses. Journal of Consulting and Clinical Psychology. 1992;60:64–72. doi: 10.1037/0022-006X.60.1.64. [DOI] [PubMed] [Google Scholar]
  58. Webster-Stratton C. Comparisons of behavior transactions between conduct-disordered children and their mothers in the clinic and at home. Journal of Abnormal Child Psychology. 1985;13:169–183. doi: 10.1007/BF00910640. [DOI] [PubMed] [Google Scholar]
  59. Webster-Stratton C. Preventing conduct problems in Head Start children: Strengthening parenting competencies. Journal of Consulting and Clinical Psychology. 1998;66:715–730. doi: 10.1037/0022-006X.66.5.715. [DOI] [PubMed] [Google Scholar]
  60. Webster-Stratton C. The Incredible Years: Parents and children videotape series: A parenting course (BASIC) Seattle, WA: Incredible Years; 2001. [Google Scholar]
  61. Weiss B, Catron T, Harris V, Phung TM. The effectiveness of traditional child psychotherapy. Journal of Consulting and Clinical Psychology. 1999;67:82–94. doi: 10.1037/0022-006X.67.1.82. [DOI] [PubMed] [Google Scholar]
  62. Whittaker J, Greene K, Schubert D, Blum R, Cheng K, Blum K, Savas SA. Integrating evidence based practice in the child mental health agency: A template for clinical and organizational change. American Journal of Orthopsychiatry. 2006;76:194–201. doi: 10.1037/0002-9432.76.2.194. [DOI] [PubMed] [Google Scholar]
  63. Wilson DB, Lipsey MW. The role of method in treatment effectiveness research: Evidence from meta-analysis. Psychological Bulletin. 2001;6:413–429. doi: 10.1037/1082-989X.6.4.413. [DOI] [PubMed] [Google Scholar]
  64. Zwirs BWC, Burger H, Schulpen TWJ, Buitelaar JK. Different treatment thresholds in non-western children with behavioral problems. Journal of the American Academy of Child and Adolescent Psychiatry. 2006;45:476–483. doi: 10.1097/01.chi.0000192251.46023.5a. [DOI] [PubMed] [Google Scholar]

RESOURCES