Do Early Childhood Interventions Prevent Child Maltreatment? A Review of Research

Abstract

We reviewed the empirical evidence on whether early-childhood primary prevention programs can reduce rates of child abuse and neglect. Fifteen studies of 14 programs for children ages birth to 5 years were completed from 1990 to 2007 and assessed impacts with methodological rigor. All but one of the programs intervened from birth to age 3 through home visits, parent education classes, or the provision of health services. The weighted average effect size of program participation was a 2.9 percentage-point reduction in maltreatment (6.6% vs. 9.5%), which is equivalent to a 31% reduction in the rate of maltreatment and a fifth of a standard deviation. Of the five programs showing significant reductions in substantiated rates of child maltreatment, three provide strong evidence of preventive effects. Only the Child-Parent Centers and the Nurse Family Partnership assessed longer-term preventive effects. Common elements of these effective programs included implementation by professional staff, relatively high dosage and intensity, and comprehensiveness of scope. The major conclusion is that the evidence base for programs in early childhood to prevent child maltreatment remains relatively weak. To advance the field, more longer term studies of a variety of intervention models are needed.

Introduction

Mounting evidence that child maltreatment has pervasive and enduring negative effects on life course development has challenged the traditional treatment focus of the field. The accumulated knowledge indicates that maltreatment victims are not only much more likely than other children to experience emotional difficulties in early childhood (Kinard, 1999; Shields, 1996) but continuing social maladjustment and school underperformance in childhood and adolescence (Cicchetti & Valentino, 2006; Eckenrode, Laird, & Doris, 1993; Shields, Cicchetti, & Ryan, 1994).

By the end of adolescence, these consequences often result in elevated rates of delinquency, substance abuse and school failure (Egeland, 1997; Kelley, Thornberry, & Smith, 1997; Mersky, 2006). It also appears that maltreatment affects adult outcomes such as parenting behavior (e.g., Haapasalo, & Pokela, 1999), mental and physical health (e.g., Briere, 1996; Cohen, Brown, & Smailes, 2001), and criminal behavior (Topitzes, 2006; Widom & Maxfield, 1996). Evidence also is increasing that the neurobiological development of maltreatment victims is impacted which can impair brain structure and function (Cicchetti & Valentino, 2006).

Moreover, the majority of maltreatment victims manifest at least one negative consequence of their victimization (e.g., Bolger & Patterson, 2003), especially when outcomes are measured later in life (e.g., Johnson-Reid, 1998; Malinosky-Rummell & Hansen, 1993). For instance, in a study investigating the long-term effects of childhood maltreatment, Egeland (1997) found that not one of the approximately 40 members of the Minnesota Longitudinal Study of Parents and Children who had been maltreated early in life escaped a negative outcome in adolescence including behavioral difficulties, drug and alcohol use, low school achievement, and psychiatric disorders.

The above findings underline the importance of preventing CAN before it occurs. In the past three decades, research on the effects of child maltreatment prevention programs has grown. Evaluation studies, however, have been generally low in methodological quality due to selective sampling, small sample sizes, short follow-up periods, and lack of actual maltreatment outcome data (see Guterman, 1997; Kaplan, Pelcovitz, Labruna, 1999). While the most recent reviews of the effects of maltreatment prevention (Geeraert, Van den Noortgate, Grietens, & Onghena, 2004; Leventhal, 2001; MacLeod & Nelson, 2000; MacMillan, 2000; Sweet & Appelbaum, 2004) indicate that parent education and home visitation programs can improve family functioning leading to reduced child maltreatment if they are intense and high in quality, two long-standing issues remain.

First, both reviews and individual studies on maltreatment prevention give less attention to changes in actual rates of reported or substantiated maltreatment. Although interventions have shown consistent positive effects on parenting practices and behavior (MacLeod & Nelson, 2000; Sweet & Appelbaum, 2004), the extent to which these impacts lead to reduced rates of CAN has not been well documented, especially for follow-up assessments. For example, of the 33 primary-prevention programs reviewed by MacLeod and Nelson (2000), only 9 assessed the general construct of maltreatment. The number of these studies that used official maltreatment and/or examined effects postprogram was undefined and is likely to be even smaller. Thus, maltreatment prevention has not been adequately assessed as a major outcome in prior reviews.

Substantiated maltreatment also has high economic costs to governments and victims, including child protective services, out-of-home placements, and lost productivity that approach $100 billion annually (Fromm, 2001). In Illinois, for example, the average cost per child for out-of-home placement accounting for total length of placement is estimated to be $49,000 (2006 dollars; Illinois Department of Child and Family Services, 2007).

Second, the effects of different types of prevention programs and the relative contributions of program components warrant more scrutiny. These would include the comparative effects of timing, duration, and intensity of services. Most if not all of the research has been on home visitation programs (Bilukha et al., 2005; Sweet & Appelbaum, 2004). Health services and center-based preschool programs have received less attention as maltreatment prevention strategies. None of the programs reviewed by Bilukha et al. (2005), MacLeod and Nelson (2000), Sweet and Appelbaum (2004) were center- or school-based preschool programs for 3- and 4-year-olds. The closest was the Parent-Child Development Center programs, which were family-centered and implemented from birth to age 3 with no measurement of maltreatment (MacLeod & Nelson, 2000).These programs target child and family behaviors that can predict later maltreatment (Guterman, 1997; McLeod & Nelson, 2000; Reynolds 2000).

The Present Study

In this study, we synthesize research on the effects of maltreatment prevention programs from birth to age 5. Three major questions were addressed:

1. To what extent do early-childhood interventions prevent child maltreatment?

2. What specific programs are effective in preventing child maltreatment?

3. What are the characteristics of programs that are effective in reducing or preventing maltreatment?

Our review differs from previous ones in a number of ways. First, we include only evaluations that measure actual maltreatment as an outcome rather than family risk or protective factors associated with maltreatment. Previous studies have considered them together, especially family processes and parenting practices. By assessing maltreatment exclusively, the impact of intervention on longer-term outcomes can be distinguished from more proximal family process behaviors. The length and age of follow-up assessment also is identified.

Second, we focus exclusively on programs implemented prior to the occurrence of maltreatment. Thus, these are primary prevention or selective prevention programs rather than tertiary or indicated prevention programs (National Advisory Mental Health Council Workgroup, 2001). For research on the assessment of interventions to reduce recidivism of maltreatment, see Toth and colleagues (2002) and Cicchetti, Rogosch, and Toth (2006).

Third, unlike Sweet and Appelbaum (2004) and McLeod and Nelson (2000), we include a variety of program types, including home visiting, parent education, health services, and center-based preschool. This enables comparison of effects across different program models.

Fourth, in contrast to some previous reviews (e.g., Geeraet et al., 2004), only studies that compared outcomes of an intervention group with that of a control group were included. Fifth, this review includes assessments of the evaluations themselves. For instance, ratings are assigned to each evaluation study for the comprehensiveness of its coverage of the prevention program theory, program details, and program implementation quality. This information contributes to the understanding of evaluation results and illuminates limitations in the field’s evaluation designs.

Finally, unlike most previous reviews (e.g., Leventhal, 2001), we include effect size estimates in percentage points and standard deviation units, which enable interpretable and direct comparisons of impacts across studies. In sum, we review relatively rigorous evaluations of maltreatment prevention programs whose results can be compared both qualitatively and quantitatively.

Review Process and Selection Criteria

We reviewed the multidisciplinary literature on the effectiveness of child and/or parent-focused interventions on reducing rates of child maltreatment. Although a wide variety of sources were consulted to identify studies, such as journals, books, and program registries, information from 28 published reviews of maltreatment prevention, home visiting, or parenting education programs provided the best source of information on studies. The majority of the studies evaluated programs specifically designed to reduce maltreatment. However, other programs evaluated had more general goals of improving child development and parenting skills but reported maltreatment as an outcome (e.g., Duggan et al., 2004; Olds et al., 1997; Reynolds & Robertson, 2003).

To be included in the review, studies must have met six criteria concerning the program design, measurement of outcomes, and research procedures.

1. The aim of the program was primary prevention, not preventing recidivism of maltreatment. These programs are also classified as universal or selective preventive interventions (Barrera & Sandler, 2006; National Advisory Mental Health Council Workgroup, 2001) (Footnote 1). However, some studies were included in which unbeknownst to the researchers at the beginning of the program, small proportions of the families had prior reports of maltreatment or were in contact with CPS (e.g., Huxley & Warner, 1993; Reynolds & Robertson, 2003.

2. The study was published or reported from 1990 to 2007. To reduce reporting bias, we searched for both published and unpublished studies. Studies published prior to 1990 were excluded because these studies had been reviewed in older reviews and meta-analysis, and the focus of this review was to examine the recent literature on child maltreatment prevention programs.

3. The program was implemented when children were under the age of 5. Most interventions with a goal of maltreatment prevention begin prenatally or no later than age 3.

4. The outcomes of abuse or neglect were measured primarily by substantiated reports of maltreatment. Involvement in the child welfare system, out of home placement, hospital records of maltreatment, and parent reports of abuse or neglect also were included if available (Footnote 2).

5. Studies included a control or comparison group in which contrasts were generally interpretable as program effects. Many studies were excluded based on this criterion (e.g., Depanfilis, 2004; Troia, 2003).

6. Adequate information about outcomes was provided. The maltreatment outcomes had to be described and the outcome metric (e.g., number of reports, rate of reports) was reported for program groups. (Footnote 3).

Description of Studies Included in the Review

Of the 48 empirical studies identified that assessed the effects of early interventions on risk/protective factors for child maltreatment or on child maltreatment outcomes, we selected 15 studies of 14 prevention programs meeting the six criteria (two studies of Hawaii Healthy Start were included). The characteristics of these studies are summarized in Tables 1. (Footnote 4).

Table 1.

Summary of Program Details of Evaluations of Maltreatment Outcomes of Early Childhood Interventions

Study	Program Evaluated	Program Category	Program Description	Progam Staff	Prog Infob	Impl Qualityb	Duration (Months)	Intensity	Actual Intensity	Child Age at Prog Start
Barth, 1991	Child Parent Enrichment Project (CPEP)	Parents/ Home visits	High risk Prenatal/postnatal Home visits	Para- professional	2	0	6	Varied	M = 11 visits	Prenatal/ birth
Brayden et al., 1993	Prenatal & Pediatric Health Services Program.	Parents/ Health Services	High risk Prenatal and postnatal Multidisciplinary team of professionals (nurses, social workers, nutritionists, psychologists) Individualized counseling/consultation n and support groups	Professional	2	0	24+ prenatal	Varied Support group: 2/mo Other contact: 1/mo	No info	Prenatal
Britner et al., 1997.	Parent Education Program for Teen Mothers.	Parents/ Parent Education Classes	High risk Postnatal Program staff Initial home visit combined with on-site parent education classes	Para- professional	2	0	3	Weekly	No info	Birth
Bugental et al., 2002	Home visitation with cognitive component	Parents/ Home visits	Moderate risk Postnatal Home visits to help educate parents and to address cognitive appraisals of caregiving problems	Para- professsional	2	2	12	20 visits/yr	M = 17 visits -72% visits included cog. component	Prenatal/ Birth
Center on Child Abuse Prevention Research, 1996.	Hawaii Healthy Start	Parents/ Home visits	High risk Postnatal Home visits	Para- professional	1	0	24	Weekly visits	No info	Birth
Duggan et al., 1999; 2004	Hawaii Healthy Start (3rd Year Follow-up)	Parents/ Home visits	High risk Postnatal Home visits	Para- professional	2	2	36–60	Weekly	−1% received weekly visits -44% of active families visited every 2 weeks -engagement varied by agency	Birth
Duggan et al., 2007	Healthy Families Alaska (HFAK)	Parents/ Home visits	High risk Postnatal Home visits	Para- professional	2	2	36–60	Varied by family functionin g level		Prenatal/ birth
DuMont et al., 2006	Healthy Families New York (HFNY)	Parents/ Home visits	High risk Prenatal/postnatal Home visits	Para- professional	2	0	60+	Varies (begins weekly)		Prenatal/ birth
Fergusson et al, 2005	Early Start	Parents/ Home visits	High risk Postnatal Trained nurses or social workers Home visits	Professional	2	0	Up to 36	Weekly	No info	Birth to 3mo
Huxley & Warner, 1993:	Community Infant Project (CIP)	Parents/ Home visits	High risk Prenatal/postnatal Nurses, mental health workers, psychiatrist, and paraprofessionals Home visits	Professional	1	0	Based on family needs (flexible)	Based on family needs (flexible)	No info	Prenatal-6 months
Marcenko et al., 1996	Home Visitation Program	Parents/ Home visits	High risk Prenatal/postnatal Nurse, social worker, and paraprofessionals Home visits	Professional/ para- professional	2	0	Birth – child age 1	First 6 weeks: weekly After: at least every 2 weeks	No Info	Prenatal
Olds et al., 1997	Elmira experimental nurse home visitation program.	Parents/ Home visits	At-risk but anyone could participate Prenatal and postnatal Trained nurses Home visits	Professional	1	1	24	Began weekly and gradually declined to every 6 weeks	M = 9 prenatal visits M = 23 visits up to age 2	Prenatal
Reynolds & Robertson , 2003	Child Parent Centers	Child/ Preschool and parent education/ School involv	Low income families Postnatal Certified teachers and parent resource teachers Preschool education for children combined with parent resources (e.g., home visits, resource center, parental educational training)	Professional	2	0	Preschool/ kindergart en + possible 3 years	Half-day preschool 5 days per week during school year; parent involveme ntb	No info	3 yrs
Stevens-Simon et al., 2001	Colorado Adolescent Maternity Program with added home visitation services	Parents/ Home visits and health services	High risk Prenatal and postnatal Multidisciplinary team of professionals (e.g. health care providers, social workers, dieticians) Pre/postnatal care and postnatal home visits	Professional	2	1	24	Weekly (for first 16 weeks) then based on need	50% of planned visits made in first 16 weeks	M = 2.2 weeks
Wagner & Clayton (1999)	Teen Parents as Teachers Intervention combined with case management	Parents/ Home visits, parent education classes, case manageme nt	Teen parents Postnatal Home visits and group parent education meetings combined with case management services	Para- professional	2	2	24	−Monthly visits -Case managem ent at least quarterly	M = 10 visits/yr M = 10 case management visits/year	Birth

^*p < .05,

^**p < .01,

^tp < .10,

^aNo information provided about significance levels.

^bThese codings were based on the articles reviewed. It is acknowledged that some of the programs have a wealth of program-related information in other empirical papers, reports, etc., but these were not reviewed for the purposes of this paper.

Note. Scale for Program information, and implementation 0 = little or no information, 1 = some information, 2 = detailed information

Program and Study Characteristics

Based on research synthesis of effective prevention practices (NIMH Work Group, 2001; Greenberg et al., 2001) and established principles of program development and evaluation research (Bickman, 1990; Rossi & Freeman, 2005), we identified a number of characteristics that describe the evidence base and may contribute to understanding differential program effects across studies. Using a stage of program development model, we assessed the program design (What are the key program features concerning target group, timing, duration, and intensity of services, staffing, and content?), implementation (Does the study include information about how or if implementation quality was assessed?), and the research procedures (What is the research design, statistical analysis, and effect sizes?). A code of 0 was assigned to a study/program characteristic if it contained little or no information, 1 if it contained some information, and 2 if it contained detailed information.

Program type and content

Program type indicates whether the program was aimed at parents or children (or both) and the general category of intervention. Program types included home visitation, parent education classes (including support groups), center-based preschool intervention, and health services (e.g., pre/postnatal health care services, nutrition consultations). Although home visitation programs describe the location of intervention, they are typically conceptualized as a distinct intervention type (Sweet & Applebaum, 2004).

We also indicated the level of risk for families participating in the intervention, and described the program content and services, and the staff who implemented the program.

Timing, duration, and intensity

Timing and duration of services refer to the age of target group when the program begins and the length (duration) of services in months or years. Several of the interventions began prenatally or in the first months of life. The intensity of intervention is the amount of time per meeting of the program and/or the frequency of meetings or classes.

Program description and implementation

We analyzed the extent to which the elements of the programs were described in the studies and the quality of implementation of the intervention as judged from the evaluation study. Given the discrepancy that often arises between the planned or ascribed dosage and the actual dosage, intervention studies were also reviewed for their documentation of actual dosage (e.g., planned number of home visits vs. actual number of home visits completed).

Research design/statistical analysis

Although one of the inclusion criterion for the review table was that the study must include a control or comparison group, given the variation across studies in how the control or comparison and program groups were formed, a category was created to document this information. To document statistical power, the sample sizes of the program and control groups also were compared. We also indicated whether the group differences were significant at the 10% level. (Footnote 5). Effect sizes in percentage points and in standard deviation units also were provided. These are independent of sample size.

Because studies vary in how outcome data are analyzed and this could potentially influence findings, this information was summarized in a separate category. Given that researchers must often control for potentially confounding factors when assessing effects in both randomized and nonrandomized studies, the number of covariates used in the statistical analysis was identified (Footnote 6). Inclusion of covariates also can increase statistical power.

Review of Findings

We summarize the main findings of the review on three major dimensions: program design, services, and implementation; methodology; and program effects.

Program Design, Services, and Implementation

Program type/content

The programs varied only modestly in approaches. Ten of the 14 interventions were home visitation programs. Two provided health services in hospitals (Prenatal and Perinatal Health Services Program [PPHS]; Brayden et al., 1993; Colorado Adolescent Maternity Program [CAMP]; Stevens-Simons et al., 2001. One conducted parenting classes (Parent Education Program; Britner & Reppucci, 1997), and another was a center-based preschool program with parent education and parental involvement (Child-Parent Center [CPC] program; Reynolds & Robertson, 2003). Some programs mixed different elements such as health services and home visits, and parenting classes (i.e., CAMP; Teen Parents as Teachers; Wagner & Clayton, 1999).

In the typical home visitation program, a trained visitor spends time visiting the homes of parents, offering parenting advice, health advice, and other services (e.g., see description of Nurse Family Partnership [NFP] in Table 5). However, programs vary within this category. For example, in some, trained nurses or other professionals conduct the home visiting (e.g., Fergusson et al., 2005; Olds et al., 1986, 1997), whereas in other programs such as Hawaii Healthy Start (Duggan et al., 1999, 2004) and Healthy Families America programs (Bugental et al., 2002; Duggan et al., 2007; DuMont et al., 2006), paraprofessionals provide the home visiting services.

Table 5.

Summary of the CPC and NFP Programs

Program Characteristics	CPC	NFP
Description	Provides high quality preschool to children ages 3 Extended intervention provides support up to age 9 (grade 3) Provides family support services	Provides nurse home visitation from pregnancy (prior to 30 weeks) up to age 2
Population served	Low-income inner city neighborhoods in Chicago	High risk moms (< 19 years old, single-parents, and/or low SES) who were pregnant with their first child Any woman bearing her first child could request to participate
Services Provided	Child-component Basic math and language skills Parent component Volunteering in the class and on class trips Parent resource room at school Educational/vocational training Outreach services (e.g., resource mobilization, home visits) Health and nutrition services	Nurse home visits: 3 main activities Educating parents about infant and fetal development Getting family and friends involved and supporting the mother Connecting the family to other social services
Staff	All teachers have a bachelors degree and are certified in early childhood	Trained nurses (bachelors degrees)
Program Evaluation	Multiple aspects of the program have been evaluated and shown long-term positive effects	Multiple aspects of the program have been evaluated and shown long-term positive effects
Benefit to Cost Ratio	$7.14:1	$5.75: 1

The other four programs had somewhat different approaches. In the parent education model (Parent Education Program for Teen Mothers; Britner & Reppucci, 1997), parents attend classes or support groups to obtain information on parenting practices. The health services programs were provided through either health clinics (Prenatal & Pediatric Health Services Program; Brayden et al., 1993) or health-based case management (CAMP; Stevens-Simons et al. 2001). The preschool program for 3- and 4-year-olds (CPC) provided a high-quality half-day program with comprehensive family services (Reynolds & Robertson, 2003).

Seven or roughly half of the interventions employed primarily professional staff as program implementers, including at least bachelor’s trained nurses, social workers, certified teachers, or health professionals. These programs included NFP, CPC, Early Start, Community Infant Project, PPHS, CAMP, and the Home Visitation Program. The other programs were implemented by paraprofessionals, although training was typically provided.

Timing, duration, and intensity

Close to half of the programs (6 of 14) began shortly after the birth of a child. Seven programs began prenatally for some participants, but for only two programs did services for all families begin prenatally (NFP, PPHS). The CPC program began at age 3 or 4.

Programs varied significantly in length, ranging from approximately 3 months (Parent Education Program) to 60 months or more (Hawaii Healthy Start, Healthy Families Alaska, Healthy Families New York). The most common profile was 12 to 24 months beginning in the first few weeks after birth with 15 to 20 total visits.

For intensity, most programs prescribed seeing the family at least once per month, but many had goals of seeing the family weekly. However, as described in the studies, many of the families did not receive the prescribed dosage. For example, in the Hawaii Health Start study (Duggan et al., 1999, 2004), only 1% of the families received weekly home visits. For many programs, the number of visits and services varied as a function of the needs or interests of the families, including the Community Infant Project, Healthy Families America, Healthy Families Alaska and New York. Moreover, the length of the visits was not consistently documented across the studies.

A few interventions had more uniform intensity and length. In NFP, families received an average of 9 prenatal and 23 infant visits over the two year period of intervention (up to age 2). In the CPC program, services were more intensive as preschool classes were 3 hours per day five days a week for the entire school year and parents participated in the parent involvement component an average of 1–2 hours per week (Reynolds, 2000; Reynolds & Robertson, 2003).

Program description and implementation quality

Studies varied in how much information was provided about the program being evaluated. Most authors provided basic information about the program itself such as the length, staffing, and intervention content, but there was much variation in the level of detail provided. The conceptual and theoretical frameworks of the interventions were not generally described in detail.

Only four studies provided specific information about the quality of implementation or fidelity of the program. These were the Teen Parents as Teachers intervention, Colorado Adolescent Maternity Program, Hawaii Healthy Start, and Healthy Families Alaska. Other programs such as Healthy Families New York, NFP and CPC described more generally with documentation of the key elements of the intervention that were implemented and/or referred to prior studies or program manuals for further information.

Methodology

Research design/statistical analysis

Most of the studies (12) employed random assignment of families to program or control groups. Three studies used either matched-group or quasi-experimental designs with evidence of comparability. In Britner and Reppucci (1997), the groups were designed according to risk status: the highest risk group received the full program, the moderate risk group received partial treatment, and the lowest risk group was the control group, which received standard level of care through the county department of social services. In the Community Infant Project (Huxley & Warner, 1993), no covariate adjustments were made. In the CPC program (Reynolds & Robertson, 2003), intervention was reserved for children at highest risk, and the matched comparison group participated in usual educational enrichment programs in preschool or kindergarten. Six covariates were included in the analyses, including prior reports of child maltreatment.

Regarding statistical analysis, many of the studies had relatively small sample sizes, including four studies with less than 100 program participants. The program groups of the Community Infant Project (Huxley & Warner) and the Healthy Families America Program with enhancements (Bugental et al., 2002) were, respectively, 20 and 35. These sample sizes limit statistical power to detect group differences, especially for maltreatment, which is an infrequent event. Program group sample sizes for CPC, Hawaii Healthy Start, Healthy Families America programs, and Early Start were relatively large, with CPC (913) and Healthy Families New York (579) being the largest.

Only six studies (Olds et al., 1997; Reynolds & Robertson, 2003; Wagner & Clayton, 1999; Duggan et al., 2004, 2007; DuMont et al., 2006) included pretest covariates to estimate program effects. To the extent that covariates are associated with program participation or maltreatment outcomes, they increase statistical power and internal validity regardless of research design.

Maltreatment measures

Most of the studies used official reports of maltreatment from child protective services as an outcome. Bugental et al. (2002) used parent self-reports on the abuse subscale of the Conflict Tactics Scale (CTS). For the Early Start program, Fergusson et al. (2005) used parental reports of contact with child protective services as well as reports of severe assault on the CTS. Finally, two studies examined children’s out of home placement as the primary outcome (Marcenko et al., 1996; Stevens-Simon et al., 2001). Hawaii Healthy Start-2 (Duggan et al., 2004) and the Healthy Families Alaska and New York (Duggan et al., 2007; DuMont et al., 2006) used both substantiated reports and parent reports of neglect/abuse on the CTS.

Maltreatment assessment period

Nearly all of the studies evaluated program effects either during and/or immediately after the program was implemented. Britner et al. (1997) and Barth (1991) assessed impacts up to two or three years postprogram (ages 3–5), but this varied by family. Only two studies had long-term follow-ups. Olds et al. (1997) conducted a follow-up evaluation of the NFP when the children were 15 years old. Similarly, Reynolds & Robertson (2003) evaluated maltreatment outcomes up to age 17. For both studies, the length of follow up was 13 years postprogram.

Program Effects

The main findings of the studies are reported in Tables 2 and 3. In the synthesis, we emphasize impacts on rates of substantiated or verified maltreatment (abuse or neglect) in percentage points and converted to effect sizes in standard deviations using the probit transformation (see Table 3; Cohen & Cohen, 1983). We also summarize effects for secondary outcomes based on the CTS subscales of neglect or abuse or official out-of-home placement (see Table 4).

Table 2.

Summary of Methods and Findings from Evaluations of Maltreatment Outcomes of Early Childhood Interventions

Study	Program Evaluated	Sample Size	Control Group	Maltx. Meas.	When Assessed	Data Analysis (Number of covariates)	Major Findings	Quantitative Findings Program/Control (Effect Sizes)a
Barth, 1991	Child Parent Enrichment Project (CPEP)	P: 97 C: 94 Total N: 191	Random assignment of high risk mothers	Substantiated reports	During and after program (2–5 year follow-up)		No program effects	15.5%/14.9% (0.03)
Brayden et al., 1993	Prenatal & Pediatric Health Services Program.	High risk P: 160 High risk C: 154 Low-risk C: 295 Total in study: 1154	High risk grps randomized into treatment and control Control: high risk and a low- risk control	Substantiated reports	Up to child age 3 years	Relative Risk w. CI	No program effect comparing high-risk program to high-risk control	CPA: 9.2% / 6.6% (0.18) Neglect: 10.6% / 4.1%* (0.49) Separation: 4%/ 1% (0.58)
Britner et al., 1997.	Parent Education Program for Teen Mothers.	P: 125 C1: 96 C2: 314 Total N: 535	Group assignment by risk level P: highest risk C1: Moderate risk, initial home visit only C2: Lowest risk, no treatment	Founded reports	3–5 year follow- up.	One-way ANOVA Chi-square	Program effect when compared to C2	Founded report: P/C1: 1.60% / 7.29% (−0.68) P/C2: 1.60% / 6.69%*(−0.64)
Bugental et al., 2002	Healthy Families America (HFA) with cognitive component	P1: 34 P2: 35 C: 27 Total N: 96	Random assignment P1: Visits (HFA) Only P2: Visits (HFA) + Cog. C: Neither	Parent self- report (CTS)	Post-program	Chi-square	Program effect when P2 compared to P1 and C combined	Percentage of children physically abused: P1(Visits)/C: 23% / 26% (−0.10) P2(Visits+cog)/C: 4% / 26% (− 1.11) P2/(P1 & C)*
Center on Child Abuse Prevention Research, 1996.	Hawaii Healthy Start	P: 147 C: 157 Total N: 304	Random assignment of high risk families	Reported maltreatment cases in hospital records	At 2 years, or immediately following program’s end	Does not report	Reported fewer and less severe maltreatment cases relative to controls. Chi square test showed no difference on rates	4% / 8%b (−0.34)
Duggan et al., 1999; 2004	Hawaii Healthy Start (3rd Year Follow-up)	P: 373 C: 270 Total N: 643	Random assignment of high risk mothers	CPS CTS	At child ages 1, 2, 3 years	−Population- Average regression -Generalized estimating equations -Adjusted odds ratios (5 covariates)	No program effects	CTS Neglect* Year 1: 22%/30% (−0.25) Year 2: 23%/28% (−0.16) Year 3: 22% / 27% (−0.16) CPS Abuse/Neglect Year 1: 0%/0.4% (−0.23) Year 2: 0.8%/1.1% (−0.12) Year 3: 0.2%/0% (0.00) Across 3 years: 1.1% / 1.5% (− 0.13) Foster care (across 3 years): 1.8% / 0.8% (0.31)
Duggan et al., 2007	Healthy Families Alaska (HFAK)	P: 162 C: 163 Total N: 325	Random assignment of high risk families	CPS reports CTS	Child age = 2 years	Multiple logistic, log- linear, and linear regression	No difference between groups	CPS Abuse/neglect Year 1: 12%/10% (0.11) Year 2: 10%/6% (0.27) Across 2 years: 16%/17% (−0.04) CTS Neglect Across 2 years: 18%/18% (0.00)
DuMont et al., 2006	Healthy Families New York (HFNY)	P: 579 C: 594 Total N: 1173	Random assignment of high risk families	CPS reports CTS	Child age = 1 years, 2 years	Logistic regression (13)	No difference in prevalence between groups	CPS Abuse/neglect Year 1: 7.90%/5.98% (0.14) Year 2: 5.08%/4.80% (0.03) CTS Abuse/neglect1 Year 1: 5.67%/7.28% (−0.13) Year 2: 6.78%/7.83% (−0.07)
Fergusson et al, 2005	Early Start	P: 220 C: 223 Total N: 446	Random assignment	CTS Parental report of contact with Child, Youth, and Family Service for CAN	12, 24, 36 mo. of program Results for 0–36 mo	Chi-square	Effect on CTS, none on contact with agency	CTS – severe assault 4.4% / 11.7%* (−0.55) In contact with agency 19.6% / 21.3% (0.06)
Huxley & Warner, 1993:	Community Infant Project (CIP)	P: 20 C: 20 Total N: 40	Matched control group not treated due to program being at capacity	Substantiated reports	Up to end of program	Z-scores of group difference	No difference in prevalence between groups (Z = 1.4)2	5% / 20% (−0.80)
Marcenko et al., 1996	Home Visitation Program	P: 125 C: 100 Total N: 225	Random Assignment	Out-of-home placement (self- report)	Baseline, 10 months of intervention, 16 months of intervention	No info	No difference in number of placements at 16 months but program group more likely to be placed with family members	Out-of-home placement 10 months: 32% / 19%b (0.41) 16 months: 26% / 23%b (0.10) Of these, percentage in foster care 10 months: 39% / 50%b (−0.28) 16 months: 24% / 37%b (−0.38)
Olds et al., 1997	Elmira experimental nurse home visitation program (NFP)	C: 184 P: 116 P2: 100 (Not used in analyses) Total N: 400	Random assignment of first time at-risk pregnant mothers	Verified reports	Child ages birth to 15 years	Different in log incidence with CI Log-linear model (6 covariates)	Fewer verified reports for program than control group	Across 15 years: Total sample: 24%/32%* (−0.24) High risk: 19%/42%*; log incidence −1.40/−0.63* (.81) (high risk sample)
Reynolds & Robertson , 2003	Child-Parent Centers (CPC)	P: 913 C: 495 Total N: 1408	Preschool students in comparable low-income neighborhood. in comparable kindergarten programs	Indicated maltreatment reports	Ages 4–17	Probit regression analysis (6 covariates)	Significant program effect on ages 4–17 indicated maltreatment reports.	7.8% / 14.7%** (−0.37) Neglect: 6.5% / 13.3%** (−0.40) Abuse: 4.1%/6.6% (−0.24)
Stevens-Simon et al., 2001	Colorado Adolescent Maternity Program with added home visitation services	P: 84 C: 87 Total N: 171	Randomly assigned Control group received CAMP only (no home visitation)	Reason for removal from home	Child age 2 years	Chi-square	No diff for abuse Program effect for neglect; No diff for combined rep	Abuse: 3.6% / 0% (0.71) Neglect: 3.6% / 15.3%* (−0.77) Abandonment: 10.9% / 4.6% (0.46) Abuse or neglect: 7.2% / 15.3% (−.44)
Wagner & Clayton (1999)	Teen Parents as Teachers Intervention combined with case management	P1: 175 P2: 177 P3: 174 C: 178 Total N: 704	Random assignment to Control group, Combined (P1) PAT only (P2), case management only (P3)	Open cases of CAN	During program	Multivariate analysis (7 covariates)	Program effect for combined vs. control	Combined/C: 0% / 2.4%* (−0.31) PAT/C: 1.3% / 2.4% (−0.08) Case/C: 2.7% / 2.4% (0.02)

¹Significant differences were found for number of incidences of self-reported physical abuse, representing a positive program effect.

²Although authors reported a p-value of .07, recalculations based on two-tailed chi-square ranged from p = .14 (Pearson uncorrected) to p = .35 (Fisher).

^tp < .10,

^*p < .05,

^**p < .01

^a(+) and (−) indicate the program group was significantly higher or lower, respectively, on the given outcome as compared to the control group. (ns) indicates the program group did not differ significantly from the control group.

^bStudy authors did not report significance levels for the group comparison. Our calculation by the Fisher chi-square test indicated no significant difference (p = .158).

Table 3.

Rates of Substantiated Child Maltreatment and Other Attributes for Reviewed Prevention Studies

Program	Program Sample	Control Sample	% CAN Program	% CAN Control	P-C	90% CI	Effect Size	Age at follow-up in years
Community Infant Program	20	20	5.0	20.0	−15.0	−.334, .034	−0.80	1–2
Child Parent Enrichment Project	97	94	15.5	14.9	0.6	−.087, .099	0.03	2–5
Colorado Adolescent Maternity Program	84	87	7.2	15.3	−8.1d	−.161, .005	−0.44	2
Nurse-Family Partnership	116	184	24.0	32.0	−8.0*	−.181, .023	−0.24	15
Child-Parent Centers	913	495	7.8	14.7	−6.9*	−.102, −.038	−0.37	17
Parent Education Program for Teen Mo.	125	314	1.6	6.7	−5.1*	−.081, −.020	−0.64	3–5
Hawaii Healthy Start-1	147	157	4.0	8.0	−4.0a	−.089, .005	−0.34	2
Healthy Families Alaska	162	163	16	17	−1.0	−.086, .063	−.04	2
Healthy Families New York2	579	594	5.1	4.8	0.3	−.019, .025	0.03	2
Teen Parents as	175 (PAT	178	0.0	2.4	−2.4*	−.041, −.004	−0.31	3
Teachers plus case managementb	+case man) 177 (PAT)		1.3	2.4	−0.9	−.034, .012	−0.08
Hawaii Healthy Start-2	373	270	1.1	1.5	−0.4	−.019, .011	−0.13	3
Prenatal & Pediatric Health Servicesc	160	154	9.2	6.6	+2.6	−.023, .081	0.18	3
Median	160	178	5.1	8.0	−2.9		−0.23
Mean (Unweighted)	241	222	7.5	11.3	−3.8		−0.23
Mean (Weighted)	−--	−--	6.6	9.5	−2.9	−.041, −.017	−0.20

²Because the data for HFNY were separated by year and it was unclear whether the same families were involved in year 1 CPS reports and year 2 CPS reports, only the year 2 data are presented in this table.

Note. Study reports since 1990. Evaluations using other measures of child maltreatment were not included.

With the exception of the Child-Parent Centers, all other interventions began either prenatally or by six months of age. Rates for the Nurse-Family Partnership are for the total sample (High risk sample: program = 19%, controls = 42%, diff = 23 percentage points).

Given the different locations of the studies, the definition of substantiated or verified reports may vary.

^*p < .10

^aSignificance values were not given for the comparison. Our calculations indicated no group differences at p < .10.

^bMaltreatment cases included open maltreated cases.

^cIt was unclear whether the physical abuse cases were separate cases from the neglect cases, so for the purposes of this table only the physical abuse cases were included as maltreatment to avoid artificially inflating maltreatment rates.

^dPhysical abuse and neglect were combined . The Fisher chi-square test by our calculation was not significant (p = .144).

Study authors reported a significant (p < .05) difference for child neglect reports.

Table 4.

Program Effects for Parent Reports of Maltreatment and Foster Care Placement

Parent Self- Report
Study	Type of Maltrea tment	Program Sample	Control Sample	Program %	Control %	P-C	90% CI	Effect Size
HFA-	Physical	34	27	23.0	26.0	−3.0	−.236, .118	−.06
Home visits/ Home visits + cognitive component	Abusea	35		4.0	26.0	−22.0	−.399, −.063	−1.11
Hawaii Healthy Start- 2	Neglectb	373	270	22.0	27.0	−5.0	−.116, .051	−.16
Healthy Families Alaska	Neglect	162	163	18.0	18.0	0.0	−.076, .078	.00
Healthy Families New York	Abuse/ Neglectb	579	594	6.8	7.8	−1.0	−.036, .016	−.08
Early Start	Severe assault	220	223	4.4	11.7	−7.3*	−.116, −.027	−.52
	In contact with agency			19.6	21.3	−1.7	−.086, .055	−.06
Foster Care
Study	Type of Maltrea tment	Program Control
Hawaii Healthy Start- 2	% in foster care	373	270	1.8	0.8	1.0	−.003, .026	+.31
Home Visitation Program	% in out of home placeme ntc	125	100	26.0	23.0	3.0	−.075, .143	+.10

Note.

^aHome-visited group with cognitive component had significantly lower rates than the combined home-visited only group and the control group (4% vs. 24%; effect size = −1.04 SD).

^bBecause the data for were separated by year and it was unclear whether the same families were involved in the separate years, only the data from the final years are presented in this table.

^cAmong those placement out of home, 24% of the program group were placed in foster care and 37% of the control group.

Substantiated reports

As shown in Table 3, cumulative rates of substantiated child maltreatment for program and control groups were available for 12 studies. Group differences in rates of CPS-reported maltreatment (abuse or neglect) index the program effect in percentage points. We also document group differences at the 10% probability level with 90% confidence intervals. Overall, systematic nonrandom variation in effects sizes in percentage points was detected (Cochrane Q chi-square (12) = 22.23; p = .035.).

The median group difference in maltreatment across the studies was −2.9 percentage points, which is in the expected direction since program participation would be hypothesized to be associated with lower levels of rates of child maltreatment. This group difference corresponds to a median rate of child maltreatment of 5.1% for the program group and 8.0% for the comparison group. The median effect size was −.23 standard deviations. Given that few programs had follow up data, these estimates reflect short-term effects.

The weighted mean group difference also was −2.9 percentage points with maltreatment rates of 6.6% for the program group and 9.5% for the comparison group. This translates to .20 standard deviations or an odds ratio of .678. Since effect sizes of .20 or greater (in absolute value) are generally considered practically significant, we interpret this effect as small to medium and practically significant. Given the cost savings in the child welfare and justice systems associated with reducing child maltreatment (Fromm, 2001; Reynolds, Temple, & Ou, 2003), an effect size of .20 could be considered relatively large. Indeed, the mean (weighted) rate of maltreatment for the program group was 31% lower than the comparison group (6.6% vs. 9.5%). We note that effect sizes for individual studies that are practically but not statistically significant should be interpreted cautiously. Their reliability is more uncertain. We interpret such effect sizes as suggestive of preventive effects.

Four of the 12 studies assessing substantiated reports (or 14 of the investigated contrasts) found that program participation was associated with significantly lower rates of substantiated or verified child maltreatment (p < .10). These were the CPC program, NFP, Parent Education Program for Teen Mothers, and the Teen Parents as Teachers (PAT) program with case management. For the latter two, other contrasts did not demonstrate significance group differences. Effects sizes ranged from −0.24 to −0.64 SD units, which are moderate to strong in magnitude. NFP (Olds, et al. 1986, 1994, 1997) and the CPC program (Reynolds & Robertson, 2003; Reynolds et al., 2003) showed long-term program effects when the children were, respectively, 15 (24% vs 32%) and 17 (7.8% vs 14.7%). Britner and Reppucci (1997) showed at the 2-year follow-up that the parenting education classes for unmarried teen mothers were effective in reducing maltreatment as compared to a lower risk control group that received no treatment (1.6% vs 6.7%). A similar pattern was found but without significance based on a home-visited control group at moderate risk. Teen PAT (Wagner & Clayton, 1999) found that PAT with added case management was associated with reduced maltreatment (0.0% vs 2.4%) but not PAT alone or case management alone.

Parent reports of maltreatment and out-of-home placement

As shown in Table 4, four studies examined impacts on parent-reported maltreatment and out of home placement. In the Healthy Families America study (Bugental et al., 2002), the standard HFA program showed no differences in rates of CTS parent-reported child abuse at the end of the program (23% vs 26%). HFA with an enhanced cognitive component, however, was associated with lower rates of child abuse (4% vs 26%) but only compared to the combined control group and standard HFA was the difference significant (4% vs 24%). Because of the small sample sizes of the groups, findings are fragile and should be interpreted cautiously. For the HFA cognitive enhancement group, 1 out the 27 cases had a rating of child abuse compared to 12 out of 46 cases for the combined groups. One or two additional cases of abuse in the enhancement group would substantially alter the findings.

In Hawaii Healthy Start (Duggan et al., 2004), no significant differences were found for parent reports of child abuse in each of the years of intervention. Rates of foster care placement also were equivalent.

The Early Start program in New Zealand (Fergusson et al., 2005) had mixed effects. The program group had a lower rate of severe assault than the control groups at the end of the program but similar rates of involvement in child protective services. The home visitation program in Marcenko et al. (1996) found no overall differences in out-of-home placement. However, among those with placement, a lower percentage was placed in foster care (24% vs. 37%; see Table 2).

Overall summary of maltreatment reports

To summarize the total evidence on maltreatment prevention, a significant relation was found between program participation and child maltreatment measured as substantiated reports, parent reports, or out-of-home placement (Cochrane Q chi-square (15) = 30.39; p = .011). This finding was consistent across a range of program/control comparisons. Consistent with findings for substantiated maltreatment, the mean group difference was −3.4 percentage points (or a 32% reduction over control groups), with effect sizes of .20 SD. One-third of the studies (5 of 15) found significant reductions in child maltreatment based substantiated or parent reports. The length of follow-up was extensive for only two studies.

Evidence of Preventive Effects

Strong evidence for preventive effects

Studies of three programs (Parent Education Program, NFP, & CPC) showed relatively strong evidence of preventive effects on child maltreatment (Britner & Reppucci, 1997; Olds et al., 1997; Reynolds & Robertson, 2003). The program and control groups were reasonably matched for assessing effects, sample sizes were relatively large, sample retention was high, analyses accounted for the influence of family circumstances, the outcome was substantiated child maltreatment, and effects were detected two or more years after the end of intervention. Moreover, the effect sizes were relatively large, especially considering the length of follow up (CPC = −.37; Parent Education Program = −.64; NFP = −.24). Among high-risk participants, the effect size of NFP based on log incidence of maltreatment by age 15 was −.80. The CPC program also showed impacts on neglect and out-of-home placement (see Appendix).

Each study has a conservative bias as well. In the Parent Education Program, the high risk group was contrasted with the low risk group, which would have been expected to have lower rates of substantiation. In the NFP and CPC programs, the comparison groups received alternative interventions. The high dosage of CPC intervention (540 to 1080 hours over two years) also substantially increased the risk of monitoring bias

Mixed or limited evidence for preventive effects

The majority of evaluations reported mixed or limited evidence of preventive effects on child maltreatment. That is, some findings indicated that program participants had significantly lower rates of child maltreatment on one or more measures and the observed group differences appeared to be reliable based on the research design or analytic approach. However, the pattern of findings was not consistent and no follow-up data were reported to provide strong support for preventive effects. Studies reporting group differences of practical but not statistical significance demonstrate limited evidence of preventive effects if the pattern of findings was consistent and if the study was methodologically strong.

For the Teen PAT program (Wagner & Clayton, 1999), findings were mixed and limited. Case management and PAT alone showed no effect but the group receiving the combined intervention had significantly lower rates of open cases of maltreatment. Although the findings suggest that intensity and comprehensiveness of services are particularly important, the absence of follow up data and whether the open cases were determined to be substantiated maltreatment warrant cautious interpretation.

CAMP with added home visitation services showed a program effect for neglect (Stevens-Simon et al., 2001). The home-visited group showed lower levels of neglect than the group without home services but both these groups did not significantly differ with respect to abuse or abandonment. The CIP home visiting program (Huxley & Warner, 1993) had an effect size of −.80 (5% vs 20%) but given sample sizes of only 20 in each group, no adjustments for confounding variables, and no follow-up assessment, group differences were not statistically reliable.

Fergusson and colleagues’ (2005) mixed findings apply to the type of information gathered on maltreatment. In their evaluation of Early Start in New Zealand, they found that the program had an effect on parental self-report of abuse on the CTS; however, there was no program effect on parental report of contact with Child, Youth, and Family Services for child abuse and neglect.

Two evaluations had multiple program groups and showed different results for the program groups. Bugental et al. (2002) evaluated the Healthy Families America (HFA) home visitation program as well as HFA with an enhanced cognitive retraining component, designed to alter parents attributions about their child’s behavior. A program effect was found for the cognitive/home visit group but not for the home-visited only group.

No evidence for preventive effects

Seven studies of six programs indicate the lack of support for preventive effects on maltreatment. In the PPHS program (Brayden et al., 1993), the program group had slightly higher rates of substantiated maltreatment, neglect, and out-of-home placement than the program group by age 3. Given the length of the program, these findings may be due to monitoring/detection bias. Marcenko et al. (1996) found that while slightly more of the home-visited mothers had children placed outside the home, these children were more likely to be placed with a family member (as opposed to foster care for example) than children of the mothers in the control group.

For Hawaii Healthy Start Program (HHS), one evaluation showed a suggestive effect on maltreatment at the end of the program based on reports in hospital records (HSS-1; Center on Child Abuse Prevention Research, 1996). The most comprehensive assessment of the program for 600 families over three years and including parent reports, official reports, and foster care, found no significant differences for any measure during each year with the exception of parent-reported child neglect at year 1 (HHS-2; Duggan et al., 1999, 2004). At the end of the program at age 3, 1.1% of the program group had substantiated reports compared to 1.5% for the control group. (Footnote 7).

Three other home visiting programs implemented by paraprofessionals and starting prenatally also found no evidence of effects. Two Healthy Families America programs (New York, DuMont et al., 2006; Alaska, Duggan et al., 2007) reported slightly higher but statistically identical rates for program versus control groups for both substantiated and parent reports of maltreatment. These findings were consistent across many different measures. They also are notable given the relatively long duration (36 to 60 months) and greater frequency of home visits compared to the other reviewed programs. The Child Parent Enrichment Project (Barth, 1991) was relatively brief at six months duration and an average of 11 visits. At the 2–5 year follow-up, roughly 15% of each randomly assigned group had substantiated maltreatment reports.

What Makes a Program Effective in Reducing Maltreatment?

Because of the strong long-term effects of the CPC and NFP programs, more detailed information about these programs is presented in Table 5. Although the two programs differed in intervention approach, timing, intensity, and duration of services, they both were based on an ecological model in which comprehensive family support services were provided through the establishment of an on-going mentoring relationship (NFP) or a school-family partnership model within a preschool setting (CPC). Each program also was largely implemented uniformly by professional staff (certified teachers or public health nurses) who were compensated relatively well and consistently followed core program features.

High-Risk Subgroups

Evidence is limited that the programs are more effective for high-risk parents. Olds et al. (1986, 1997) found that NFP had the strongest effect in reducing maltreatment for young, single mothers living in poverty. Among these high-risk families, for example, 19% of the intervention group had a substantiated report by age 15 compared to 42% of the control group. The incidence of substantiated reports also was significantly lower for the intervention group.

The CPC program showed stronger effects in reducing maltreatment for children who had four or more family risks as compared to children with lower family risk and for children who lived in the highest poverty neighborhoods as compared to children living in neighborhoods that had lower poverty levels. Although more evidence is needed, these two studies suggest that programs may be more effective for the families at greatest risk for maltreatment.

As shown in the Appendix, participation in the CPC preschool program was associated with most measures of substantiated abuse and neglect as well as out of home placement (Reynolds et al., 2007). After controlling for differences in family risk status, maltreatment in the first three years of life, later school-age intervention, and gender and race/ethnicity, 7.8% of the CPC preschool group had substantiated DCFS reports over ages 4–17 versus 14.7% of the comparison group. Rates of out-of-home placement and DCFS child neglect reports also significantly favored the program group.

High Dosage of Intervention

It would be predicted that a higher dosage of the intervention would provide the best results in reducing maltreatment. Both the NFP and the CPC programs had relatively high-dosage or high involvement. In the Elmira NFP, the nurse visits lasted approximately one hour 15 minutes. The nurses visited the families on average 9 times during the pregnancy, and weekly for 6 weeks after the pregnancy. They then visited bi-weekly until the child was 4 months, every 3 weeks until the child was 14 months, every 4 weeks until the child was 20 months, and then every six weeks until the child was 24 months. They also met with the families weekly during crisis conditions.

In the CPC program, children were involved in the preschool program daily for 3 hours per day. In addition, parental involvement of at least one half-day per week was required by the program, and parents were encouraged to become involved in multiple aspects of the program, such as classroom and class trip volunteering, vocational/educational training, and using the parental resource room.

Both NFP and CPC had relatively high dosages of the intervention, which may account for some of the positive program effects. Findings from the CPC evaluation provide some support for this idea: longer duration of preschool participation (2 years vs. 1 year) was associated with lower rates of CAN (DCFS reports) and, more broadly, lower rates of child welfare services (Reynolds & Robertson, 2003).

One finding that appears to contradict this idea that higher dosage is more effective comes from Duggan et al.’s (2004) evaluation of Hawaii Healthy Start. They found that families that received a high dose of services were more likely than controls to report severe physical abuse. However, this result must be interpreted with caution because it is unknown as to what factors caused some families to receive more services than others. It is possible that the highest risk families received the most services but that the service level was still not high enough to show program effects. Healthy Families New York and Alaska studies also did not show favorable effects. The preventive effects found in the 3-month home visiting program evaluated by Britner and Reppucci (1997) also suggest that dosage alone may be not be an essential element of effectiveness.

Comprehensiveness

Comprehensive services may be another element of program effectiveness. As described above, Teen Parents as Teachers and Healthy Families America showed effects only when components were added, such as case management or problem-solving activities for parents (e.g., Bugental et al., 2002; Wagner & Clayton, 1999). These results dovetail with the comprehensive family support services found in NFP and CPC. The latter, for example, included child education, family support, and health and community outreach services. It may be that more comprehensive programs are needed to address the issues of high-risk families in a more holistic way that acknowledges the complexity of child maltreatment and parenting practices. To the extent that different services are coordinated in meaningful ways and delivered uniformly, comprehensiveness may reinforce the effects of other features such as duration and intensity.

Service Providers

A further attribute of effectiveness may be the level of training of staff implementing the program. For example, the NFP and the CPC programs both utilize highly trained staff: the NFP have trained nurses who conduct the home visits, and the CPC have trained teachers (with Bachelors degree) with certification in early childhood. Olds and colleagues have implemented the NFP program in multiple sites and an on-going evaluation compares the effects of nurse home visitors and paraprofessional home visitors. Initial results suggest that there are more benefits for nurse-visited mothers than for mothers visited by a paraprofessional (Olds et al., 2004).

Moreover, in the review 3 of the 7 interventions implemented by professional staff demonstrated beneficial effects on child maltreatment compared to 2 of 8 for intervention implemented by paraprofessionals. Effect sizes were slightly higher for professionally-staffed interventions (.30 vs. 20 SD).

Long-Term Follow-Ups

One final important consideration is that two of the programs with strong evidence for reduction in maltreatment (NFP, CPC) conducted long-term follow-ups of the children in the program. Indeed in Olds and colleagues evaluations of the NFP program in Elmira, there was an original program effect on maltreatment at one-year follow-up (1986), the program effect disappeared at the two-year follow-up (1994), and the program effect returned at the 15-year follow-up (1997). Similarly, in the CPC program evaluation, the strongest program effect was found when looking at the age 4–17 span as compared to the age 4 to 9 group (Reynolds & Robertson, 2003). Therefore, although many studies have not examined long-term effects on maltreatment rates, it appears, based on the findings from the CPC and NFP programs, that programs aimed at reducing maltreatment may not have strong immediate effects, and it may be necessary to conduct longitudinal follow-ups to assess programs most accurately.

Why Do Some Programs Not Show Effects on Maltreatment?

Implementation Quality

Many of the studies documented poor implementation quality. For example, in the Hawaii HSP only 1% of the families received weekly visits, the recommended dosage. In other studies, the actual number of visits was often lower than the prescribed number of visits (e.g., Bugental et al., 2002; Stevens-Simon et al., 2001), and some studies had high dropout rates that reduced statistical power an may have influenced effect sizes. However, as described in Table 1, many of the studies did not include information about implementation quality, making it difficult to compare across studies.

Similarly, in large-scale studies, implementation quality may vary from site to site. For example, in Duggan et al.’s (1999) evaluation of the Hawaii HSP, they found significant differences in implementation and program effects across different sites. They also discuss how funding changes and management changes may have changed the program focus and altered the intervention (2004). These may have impacted how well the intended intervention was implemented, thus affecting its impact on reducing maltreatment. Variability in implementation across sites and families also may have influenced findings in the Healthy Families programs and the Community Infancy Program.

Service Providers

It appears that the level of training of the staff can influence program effectiveness. Home visitors in eight of the studies were paraprofessionals, including Hawaii Healthy Start and Healthy Families America studies, and only two studies showed evidence of maltreatment effects (Teen PAT and Parent Education Program). This may have contributed to the lack of significant effects, although many other factors may be at work. Also, as previously discussed, Olds et al’s (2004) study comparing paraprofessionals and nurse home visitors provides support for the idea that benefits may be greater for families visited by nurses than families visited by paraprofessionals.

Monitoring/Detection Bias

Monitoring bias is a common problem in many intervention studies, especially for those using child protective service records. That is, because families in an intervention are in more contact with some form of social service providers, they are more likely to have maltreatment detected. Bugental et al. (2002) found that of reports made on study children, 30% of physical abuse reports and 15% of the neglect reports came from the study hospitals (and all were reports on intervention families and not control families). Therefore, this monitoring bias may contribute to why some studies fail to find results. The level of bias would be expected to be a function of the length and intensity of intervention as well as services received by the control group. The longer duration interventions of Hawaii Healthy Start and Healthy Families America may have been particularly susceptible to monitoring biases, at least in part. Such bias would also be most likely to affect outcomes measured during or immediately after the program rather than in longitudinal follow-up phases. The early evaluation of NFP (Olds et al., 1986), Teen PAT, Parent Education Program, and Early Start, however, still detected significant prevention effects.

Control/Comparison Group

Another important consideration in evaluating maltreatment programs is how the control or comparison group is formed and what services they receive. In most of the studies, families were randomly assigned to program and control groups, which reduces most types of selection bias. Inspection of the groups at pretest indicated group equivalence across studies. Unfortunately, some studies had relatively small sample sizes, which raise the possibility of differences on unmeasured variables. In addition, few studies included covariates, which would have increased statistical power. Given the general comparability of program and comparison groups, the matched-group and quasi-experimental studies showed little evidence of bias. The comparison group in Community Infant Project (Huxley & Warner, 1993) was formed from a waiting list due to the intervention being at full capacity. In the Parent Education Program (Britner et al. 1997), higher risk families were assigned to the program which would likely underestimate impacts. In the CPC program, the comparison group was chosen from randomly selected schools and participated in alternative interventions, and had similar demographic characteristics prior to intervention, including maltreatment. Thus, the research designs of the studies were unlikely threats to internal validity.

A more plausible threat to the internal validity of findings is compensatory equalization, since in some studies the comparison group received alternative services and in others no information was provided. In some cases, the comparison group was simply provided with referrals to local services. In other cases, they receive some standard services. In the Duggan evaluation of Hawaii Healthy Start (1999), the authors note that many of the control group families may have been receiving care and services from other agencies. For example, at the one-year follow-up, 28% of control group mothers reported that they had received a home visit (not from a home visitor). In the Community Infant Project (Huxley & Warner, 1993), the control group families were found to use outside health services at a significantly higher rate than the program group. The authors suspected that this was because the families may have believed that the intervention covered these needs. Access to alternative services in the community by the control group also may have contributed to findings in the Healthy Families America studies. Thus, it is possible that the estimated impacts on child maltreatment for these programs are conservative.

Low Occurrence of Maltreatment

Another attribute to take into account in assessing the evidence is the incidence of maltreatment. A low rate of maltreatment for both the intervention and the control group make effects more difficult to detect or they may appear as nonsignificant. This is particularly the case with CPS reports, given that only a small proportion of actual maltreatment cases become official substantiated reports. For example, in a later trial of NFP conducted in Memphis, the rates of maltreatment were too low for both groups to conduct any meaningful analyses (Olds et al., 1999). Similarly, in the Hawaii HSP evaluation, Duggan et al. (2004) reported very low prevalence rates of substantiated maltreatment. However, many of the families showed much higher levels of abusive behavior from CTS self-reports. Therefore, when evaluating maltreatment interventions, differences may not emerge due to the low prevalence of substantiated reports of maltreatment. The greater use of multiple sources of information over longer periods of time can increase statistical power to more acceptable levels. A counterpoint to this threat was the wide range of maltreatment incidence rates across studies and that studies with higher incidence were not more likely to show significant program effects.

Length of Follow-Up

As noted above, only four of the studies assessed child maltreatment beyond the end of the intervention. Only two did this uniformly across the study sample. For most prevention programs, it may take many years before program impacts are realized in reducing maltreatment. That is, programs that are evaluated in the short-term after the program has been implemented may not show reductions in maltreatment. The Hawaii Healthy Start and Healthy Families America studies, for example, assessed maltreatment only at the end of the program for most participants (ages 2 or 3).

Although the appropriate length of follow-up may vary by intervention approach, at a minimum several years post program may be required to detect full effects on maltreatment. This was the pattern of effects for the NFP and CPC programs, as consistent effects on maltreatment were detected five or more years postprogram. This may be also related to the low occurrence of maltreatment: over longer periods of time there are more opportunities for reports to occur and for larger group differences to emerge.

Discussion

Our review finds limited evidence that early childhood interventions can prevent child maltreatment. Of the 12 interventions investigated, only 4 or one-third reported that program participants had significantly lower rates of maltreatment than comparison groups. The average effect size across studies was one-fifth of a standard deviation or a reduction in maltreatment of about 3 percentage points (or 30% reduction over control groups). While 5 studies reported reductions in either substantiated or parent-reported maltreatment, only for 3 programs was there consistent evidence of enduring effects. Among three widely implemented home visitation programs—Hawaii Healthy Start, Healthy Families America models, and Parents as Teachers, only enhanced versions of Healthy Families America and Parents as Teachers were associated with reductions in maltreatment.

Given the well-documented links between program participation and improvements in parenting practices (McLeod & Nelson, 2000; Sweet & Appelbaum, 2004), these findings suggest that even if prevention programs strengthen parenting skills and related family outcomes, reductions in actual maltreatment are much less certain, especially enduring effects.

Two programs, Nurse-Family Partnership and the Child-Parent Centers, showed strong evidence of long-term maltreatment prevention. They have relatively high intensity, are implemented by well-trained, professional staff, and provide comprehensive family services. The studies had well-conceived research designs, sufficient sample sizes for reliable inferences, and assessed maltreatment up to 13 years postprogram. The Parent Education Program (Britner & Reppucci, 1997) was the third intervention demonstrating relatively strong effects postprogram. Indeed, the high-risk intervention group had a significantly lower rate of substantiated maltreatment than the low-risk comparison group. Thus, the findings are conservative. In contrast to NFP and CPC, the program was only 12 weeks in length.

How strong is the evidence that early childhood interventions can prevention child maltreatment? Relative to the knowledge base about the impact of early childhood intervention on the prevention of school underachievement (Consortium for Longitudinal Studies, 1983; Reynolds 2000; Zigler et al., 2006), the evidence on maltreatment prevention is weak. This state of affairs can be explained by a number of factors, including differences in program scope, the number of available longitudinal studies, and the challenges of data access and collection. For the latter, maltreatment data is more difficult to obtain and assess validly than school records or measures of child well-being. Relative to other interventions or social programs such as reduced class sizes or job training, the evidence on maltreatment prevention is stronger (Temple & Reynolds, 2007). From the perspective of history, however, that there are only two studies that convincingly show long-term effects on maltreatment prevention is unexpected and should provide impetus to strengthening the evidence base on maltreatment prevention.

Limitations

We note three limitations of the review. First, we emphasized behavioral changes in actual or parent-reported rates of child maltreatment as the major outcome. Although we noted evidence of program effects for other outcomes, the interventions should not characterized as effective or ineffective on the basis of just the outcome of child maltreatment. Impacts on broader measures of child and family well-being are important in their own right and may lead to effects on later outcomes such as crime prevention and educational attainment (Topitzes, 2006). Only after a comprehensive analysis of the effects of all theoretically important outcomes as well as program costs can judgments about effectiveness be made.

Second, the review assessed relatively few studies. They were early childhood prevention programs with relatively rigorous designs and analyses. The studies are not necessarily representative of the entire field of maltreatment prevention although they are likely to be representative of the literature since 1990. The inclusion of more unpublished studies, studies of treatment programs, or of maltreatment prevention programs beginning in the school-age years would be important complements to our review. Nevertheless, our findings are consistent with Bilukha et al. (2005), which reviewed studies published from 1979 to 2001 of the effects of home visitation programs on maltreatment. Using different selection criteria and different studies than our review, Bilukha and colleagues found that roughly one-third of the 21 studies reported statistically significant reductions in general maltreatment associated with program participation. Notably, the studies varied substantially in methodological quality, which was not a selection criterion.

Because many of the studies we reviewed found no reliable group differences on child maltreatment, the inclusion of more unpublished studies would likely increase the number of studies showing “null” findings. Unpublished studies are more likely than published studies to show “no effect” findings. Consequently, the effect sizes in our review may be greater than in analyses that include unpublished studies. Nevertheless, we searched for as many studies as possible. Among the 48 studies we identified, most of the excluded unpublished studies had methodological limitations and this would likely be the case for studies we did not identify.

Third, our review emphasized impacts on child maltreatment, which combines abuse and neglect. This was done to make findings as comparable as possible across studies, as several studies did not distinguish between forms of maltreatment. Some interventions may have effects on specific types of maltreatment (e.g., educational neglect, physical abuse) that went unmeasured. Two studies, for example, showed impacts on neglect but not on physical abuse (Duggan et al., 1999; Stevens-Simon et al., 2001). We also excluded from review programs to prevent sexual abuse and for drug-addicted mothers. Impact assessments across a wider range of maltreatment measures over different intervals of time are needed. Major barriers to greater specificity of preventive effects are the frequent co-occurrence of abuse and neglect and the low base rates of particular forms of maltreatment (Mersky, 2006). Large sample sizes will be needed to assess these effects reliably. Certainly, our emphasis on substantiated reports, one measure of maltreatment, warrants cautious interpretations. The potential for monitoring bias is greater for these reports.

Implications and Recommendations

Five recommendations for research and practice are offered. First, follow-up studies beyond 1 or 2 years are needed with sample sizes that are large enough to reliably detect program group differences. In our review all but two programs had follow up periods of less than three years. Assessment of longer-term prevention effects is not only consistent with ecological and developmental psychopathological models of behavior, but can provide increased statistical power to detect effects since cumulative rates and incidence of maltreatment will be higher and more distinguishable.

Second, when evaluating preventive programs, regular collection of official data on child maltreatment is needed. After several decades of research on child maltreatment prevention, most studies of programs with a goal of preventing maltreatment do not collect official records of maltreatment or child welfare services. We identified only 13 early intervention studies that measured substantiated maltreatment or out of home placement. Two others were parent reports. Only two studies obtained records of maltreatment three or more years after the program. The collection of official reports substantiated and unsubstantiated cases of maltreatment should be routine especially in prevention studies. Given the well-known limitations of official maltreatment reports, they should be supplemented with parental and other reports. In our review, parent reports of maltreatment and records of out of home placement resulted in higher incidence rates than substantiated reports but program effects were similar across sources of report. The study of Healthy Families Alaska (Duggan et al., 2007), which had the most comprehensive measurement of maltreatment, found no evidence of prevention effects for official reports or parent reports.

Third, implemented programs may benefit by a stronger focus on key principles of effectiveness such as high intensity, longer duration, professional and well-trained staff, and comprehensive family services. As found in this review, Nurse Family Partnerships and the Child-Parent Center programs had these attributes and documented long-term prevention effects on maltreatment. The attribute of timing of intervention, however, was not strongly supported in review as the three programs showing evidence of prevention effects began at three different ages, prenatal development (NFP), birth (Parent Education Program), and in preschool (CPC). Although the Child-Parent Centers and Nurse Family Partnership show the possible benefits of a core set of principles of effectiveness implemented together, the unique contribution of each element, including professional staff and comprehensive services is not clear, and further studies of more programs are warranted. Given the relatively small number of programs reviewed and effective programs identified, generalizability of findings is limited. Moreover, the demographic attributes of program participants across the studies were limited to low-income or otherwise at-risk families.

Fourth, preschool programs should be investigated more frequently for maltreatment prevention. Only one of the reviewed studies—the Child-Parent Center Program—was a preschool program yet preschool education programs are expanding rapidly across the country as an approach to promote school readiness (Zigler, Jones, & Gilliam, 2006). Like the CPC program, early interventions that provide family services along with child educational enrichment may be most likely to prevent later maltreatment. While earlier preschool programs showed little evidence for the prevention of maltreatment (Consortium for Longitudinal Studies, 1983), on-going national evaluations of Early Head Start (Love et al., 2005) and Head Start (Puma et al., 2005) offer the potential to assess effects on child maltreatment.

Finally, research on different and innovative intervention models is needed. Our review was limited to programs implemented in the first five years of life. It is possible, of course, that home visitation programs alone may not be the most effective intervention strategy for preventing child maltreatment. Interventions that combine different elements need further investigation. For example, hybrids of the approaches assessed in this review such as preschool programs with parenting components, two-generation programs, and programs that provide more comprehensive health services along with parenting classes or preschool education could be more effective than traditional home visitation approaches to maltreatment prevention. Galano et al. (2001), for example, found suggestive evidence that a community-based program providing home visitation, parenting classes, newsletters, library resource centers, and related services can help reduce official rates of maltreatment.

Interventions that begin after age 5 or extend into school age also warrant further investigation. Reynolds et al. (2002, 2007) found that children who participated in the preschool and school-age components of the Child-Parent Center program had significantly lower rates of child maltreatment and out-of-home placements above and beyond the preschool program alone. These impacts were traceable in part to the reduced school mobility of program participants.

Moreover, interventions and policies to increase economic well-being deserve further attention. Paxson and Waldfogel (2003), in a state-level analysis of welfare involvement, found that higher welfare benefits to mothers were associated with fewer cases of child neglect and out of home placement, although these effects varied by family structure. Parental employment among single-parent families also was associated with lower rates of child maltreatment. The influence of socioeconomic experiences on maltreatment rates may interact, however, with child welfare spending and other policy resources (Malcolm, 2005). Such findings reflect the potential importance of comprehensive two-generation approaches to maltreatment prevention and related outcomes. Only after examination of a wide range of intervention approaches over longer intervals of time will a better understanding of the prevention of child maltreatment emerge.

Appendix A

Summary of Child-Parent Center Maltreatment Results

Indicated/Substantiated Abuse and Neglect (Age 4–17)	Proportion (%)
DCFS Reports Only	Total Sample (N=1411)	CPC Preschool (N=914)	None (N=497)
Any abuse/neglect (maltreatment)	10.2	7.8	14.7
Any neglect	6.7	4.7	10.3
Any physical abuse	2.1	1.4	3.4
Any sexual abuse	1.4	1.2	1.8
Any risk of physical injury	3.5	3.0	4.6
DCFS and Court Data
Any abuse/neglect	12.7	10.4	16.9
Any neglect	8.9	6.5	13.3
Any physical abuse	3.3	2.7	4.4
Any sexual abuse	1.7	1.4	2.2
More than 1 category of maltreatment	6.4	5.3	8.5
Any risk of physical injury	6.8	5.9	8.5
Out of Home Placements
DCFS Reports Only
Any out of home (incl. protective custody)	5.6	4.3	8.0
DCFS and Court Data
Any out of home (incl. temp. custody)	7.9	6.1	11.3