Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Nov 9.
Published in final edited form as: Prev Sci. 2016 Jan;17(1):52–61. doi: 10.1007/s11121-015-0600-9

Evaluation of a Community-based Approach to Strengthen Retention in Early Childhood Home Visiting

Alonzo T Folger 1, Anita L Brentley 1, Neera K Goyal 1, Eric S Hall 1, Ting Sa 1, James L Peugh 1, Angelique R Teeters 1, Judith B Van Ginkel 1, Robert T Ammerman 1
PMCID: PMC5679402  NIHMSID: NIHMS916270  PMID: 26292659

Abstract

Home visiting (HV) is a strategy for delivering services designed to promote positive parenting and prevent exposure to toxic stress during a critical period of child development. Home visiting programs are voluntary and family engagement and retention in service can influence outcomes. Most participants receive less home visits and for a shorter time than prescribed by evidence-based models. The purpose of this study was to evaluate community-based enrichment of HV (CBE-HV), an approach that was developed and implemented to increase engagement and retention in HV. CBE-HV strategies included (1) community engagement, (2) ancillary supports for families in HV, and (3) enhancements to a HV program. A retrospective, quasi-experimental study was conducted to estimate the effect of CBE-HV on the retention of families in a HV program. Comparisons of study participants were made post-implementation of CBE-HV (n=2,191) and over time (n=3,786)—pre versus post CBE-HV implementation in the study communities. The CBE-HV effect was statistically significant and protective (hazards ratio [HR] 0.77, 95% confidence interval [CI]: 0.67, 0.88), indicating that attrition from HV was 23% less in the CBE-HV group relative to the post-implementation comparison group. In the temporal comparison of study communities, CBE-HV was also associated with a significantly lower risk of HV attrition (HR: 0.71, 95% CI: 0.56, 0.89). The study demonstrated that CBE-HV is a promising approach to achieve stronger retention and engagement in HV. Further research is needed to identify the components of CBE-HV approaches that are most effective.

Keywords: home visiting, child maltreatment, retention, community engagement

Early childhood home visiting (HV) is a widely disseminated strategy to optimize child health and development. An estimated 400,000–500,000 families were served annually by HV programs leading up to 2010 (Astuto & Allen, 2009; Gomby, 2005), and a subsequent federal investment of $1.5 billion expanded the reach (Adirim & Supplee, 2013). While models vary in focus and content, HV programs share primary goals to develop parenting skills and support child development (Sweet & Appelbaum, 2004). Mounting evidence links positive early experiences to healthy brain development, and also reveals the deleterious physiological effects of inadequate nurturing (Luby et al., 2013; Shonkoff et al., 2012). The successful engagement of at-risk families in HV provides an opportunity to prevent or mitigate adverse childhood experiences and promote normative development.

Studies of HV efficacy have shown favorable effects for the prevention of adverse childhood outcomes such as delayed development and mortality (Caldera et al., 2007; DuMont et al., 2010; Olds et al., 2004; Olds et al., 2014). However, once taken to scale in a community, HV programs often have diminished effectiveness (Duggan et al., 2004). Inadequate participation in HV is partly responsible for the attenuation of program effects in ‘real-world’ settings. Most families enrolled in HV receive fewer home visits than prescribed, and nearly 50% of clients exit before the child is 12-months of age (Duggan et al., 2007; O’Brien et al., 2012). Although new evidence has revealed that the impact of participation may not follow a simple dose-response relationship (Holland, Xia, Kitzman, Dozier, & Olds, 2014), level of participation can influence program outcomes. For example, a greater frequency of HV has been associated with positive parenting behaviors (Nievar, Van Egeren, & Pollard, 2010) and more optimal gestational age at birth (Goyal et al., 2013). Enhancing participation in HV through improved engagement and retention may help generate more reliable, substantive program effects.

Determinants of program participation include characteristics of the family, service provider and community (Daro, McCurdy, Falconnier, & Stojanovic, 2003; McGuigan, Katzev, & Pratt, 2003; O’Brien et al., 2012). Maternal characteristics associated with participation are race/ethnicity (Daro et al., 2003; McCurdy, Gannon, & Daro, 2003; O’Brien et al., 2012), education (O’Brien et al., 2012), depression (McFarlane, Crowne, Burrell, & Duggan, 2014), social support (Ammerman et al., 2006; Navaie-Waliser et al., 2000), and other indicators of socio-demographic risk (Goyal et al., 2014). Some studies have shown that African American mothers have the highest rate of attrition in HV (O’Brien et al., 2012), whereas other studies have shown greater retention of African American mothers (Daro et al., 2003; McCurdy et al., 2003). Maternal depression, prevalent among mothers in HV, has been shown to increase the likelihood of enrollment (Damashek, Doughty, Ware, & Silovsky, 2011). However, mixed evidence has emerged regarding the maternal depression and retention (Girvin, DePanfilis, & Daining, 2007; Smith & Moore, 2012). Mothers with greater instrumental and emotional support needs have remained active in HV longer than mothers with less need (Navaie-Waliser et al., 2000). Similarly, greater parenting risk including low social support has been associated with increased retention (Ammerman et al., 2006). Beyond the participant-level, community violence has been associated with lower retention (McGuigan et al., 2003), as has increased provider age (Daro et al., 2003) and length of employment (O’Brien et al., 2012).

Although determinants of participation in HV have become clearer, there is little empirical evidence about how to improve engagement and retention. One challenge of participation in family-based prevention programs is often rooted in a lack of perceived need and benefits that emerge slowly (Ingoldsby, 2010). To overcome this, recent evidence suggests adapting service delivery to better align with individual clients’ needs (Ingoldsby, 2010; Ingoldsby et al., 2013; O’Brien et al., 2012; Tandon, Parillo, Mercer, Keefer, & Duggan, 2008). Cultivating perceived value and differentiating the delivery of HV service is complex and requires an approach that addresses multiple levels of ecological influence. The ecological model provides a framework to both characterize the impediments to participation and facilitate the development of strategies to improve participation (McCurdy & Daro, 2001). Therefore, successful strategies to improve participation should integrate knowledge at multiple levels including individual, provider, and community.

The need for strengthened participation in HV has been the impetus for designing approaches that improve provider-client relationships. Interventions have included flexibility in core services to better meet individual client needs and expectations (Ingoldsby et al., 2013; O’Brien et al., 2012) and providing access to ancillary services such as group support sessions (Constantino et al., 2001). There has been scant attention, however, devoted to family engagement approaches that consider the unique needs and challenges of a community. Through the same channel of influence that community violence deters retention in HV, community-level support and acceptance of HV could reduce individual ambivalence about service value and strengthen retention. This type of community-based approach was used by the Harlem Children’s Zone, Inc. Strategic community partnerships in Harlem communities and a broad reach triggered a positive shift in child academic achievement (Dobbie & Fryer Jr, 2011). Community-based interventions have also improved neonatal outcomes through community mobilization, home visiting, and support groups (Lassi, Haider, & Bhutta, 2010).

The purpose of this study was to evaluate the effectiveness of community-based enrichment of HV (CBE-HV) to strengthen participation in HV and improve program impact. CBE-HV was embedded within one high-risk, predominantly African American community known to have poor child health outcomes and lower retention in a HV program. Community-level resources were leveraged in CBE-HV to address the unique needs of families while promoting community acceptance and support for the HV program. Efforts were made to reach all new mothers in the community eligible for HV services. In an ongoing evaluation of CBE-HV, we first studied HV participation. Retention was chosen as our primary measure of participation with the reasoning that this outcome reflected engagement at multiple points of the curriculum. We hypothesized that CBE-HV would significantly improve retention in a HV program through three years of age.

Methods

Design

A retrospective, quasi-experimental study was conducted using two approaches to estimate the effect of CBE-HV on the retention of families in a HV program. The first approach was a comparison of HV retention in CBE-HV versus comparison communities following the implementation of CBE-HV, which occurred during the years 2006–2012. The second approach was a temporal comparison of retention pre (2000–2005) versus post (2006–2012) CBE-HV implementation in the CBE-HV and comparison communities.

Post-Implementation Comparison

Survival analysis was used to compare a CBE-HV group to a comparison group with respect to time (days) to discharge from HV. These families enrolled in HV during the years 2006–2012, which was after implementation of CBE-HV. The CBE-HV group contained mother-child pairs who participated in standard HV and who resided in the targeted high-risk community. Families in the CBE-HV group were exposed to the community-based strategies used to support and promote HV. Mother-child pairs in the post-implementation comparison group participated in standard HV delivered by the same program and within the same geographic HV service area. However, these families resided outside of the CBE-HV community and therefore, were not exposed to CBE-HV in their communities.

Temporal Comparison

A difference-in-differences approach was used to characterize changes in retention over time in the study communities and namely pre versus post CBE-HV implementation. This analytical approach is increasingly used in observational research to evaluate health policy implementation (Dimick & Ryan, 2014). Survival analysis was used to compare the time to discharge of mother-child pairs before (2000–2005) versus after (2006–2012) implementation within CBE-HV and comparison communities. Comparison communities were within the geographic HV service area, but outside of the CBE-HV community. All families who participated in HV prior to CBE-HV implementation received standard HV and were unexposed to CBE-HV in their communities. We hypothesized that significantly higher HV retention would be observed in the CBE-HV community after CBE-HV was implemented and that this effect would be significantly greater than temporal changes observed in comparison communities.

Study Participants and Setting

The post-implementation study population included 2,191 mother-child pairs who enrolled in HV following CBE-HV implementation (2006–2012). The CBE-HV group contained 267 families, and the post-implementation comparison group contained 1,924 families. In the temporal comparison, the study population included 3,786 mother-child pairs who received HV pre (n=1,592) and post (n=1,924) CBE-HV implementation in the study communities. Because CBE-HV participants were almost exclusively African American (95%) and race is a predictor of HV participation, the study populations were restricted to include only African American mothers. Mothers were eligible for HV if they were first-time parents and exhibited one of four risk factors believed to increase risk for child maltreatment: unmarried, low income, less than 18 years of age, and/or inadequate prenatal care.

Participants who relocated from the CBE-HV community to a comparison community (19.1%) following CBE-HV implementation continued to receive HV and CBE-HV. An estimated 15.7% of participants moved into the CBE-HV community after HV enrollment and were exposed to CBE-HV at the time of their relocation. To avoid contamination of CBE-HV within comparison communities, a primary service agency was used and a CBE-HV coordinator managed access to the CBE-HV supports and resources. The mobility of the study population represents the expected differential exposure to community-specific strategies used in HV.

All study participants received HV under the Healthy Families America model and delivered by the Every Child Succeeds program. The program is a network of 10 service agencies located in Southwest Ohio and Northern Kentucky that deliver HV to at-risk, first-time mothers and their children. The service agencies use a standard curriculum and receive common training. The families in the study cohort were enrolled prenatally or up to three months postpartum. The HV program sought to retain families until the child’s third birthday. Home visits occurred weekly for the first and last four weeks of prenatal service and every other week for the remaining time of pregnancy. Subsequently, the home visits occurred weekly and eventually tapered to monthly. The study was approved by the Cincinnati Children’s Hospital Medical Center institutional review board.

The CBE-HV community had 12,000–13,000 residents and the second highest incidence of infant mortality in Cincinnati, Ohio (Hamilton County Public Health). The 2000 US Census of Population and Housing indicated that 76.2% of children were living at or below 200% of the federal poverty level. Despite having high socio-demographic risk, the CBE-HV community has resources supportive of engagement such as 52 churches, a community council, and a coalition on economic development.

Community-based Enrichment of Home Visiting

The CBE-HV approach was an explicit effort to leverage community resources as a pathway to improve individual engagement and retention in HV. The approach was based upon an ecological framework in which social cohesion and community-centered supports are posited to influence an individual’s participation in HV (McCurdy & Daro, 2001). The strategies included (1) community engagement to support HV, (2) ancillary supports for families in HV, and (3) enhancements to the HV program.

The first strategy was to engage community stakeholders including leaders of neighborhood councils, faith-based organizations, and organizations devoted to economic development. The community stakeholders were organized into a steering committee that functioned to help shape the service climate for HV and shared ownership in transformation. Monthly meetings between the community steering committee and the HV program were forums to share program statistics, identify and secure resources (e.g., trusted and safe venues for service), and exchange ideas for program content. Similar to the paradigm of community-based participatory research (Wallerstein & Duran, 2010), input was sought from community stakeholders about health priorities and used to derive CBE-HV strategies. In one example, the community steering committee identified a need for education regarding screening and risk for sexually-transmitted infections; this topic was integrated into subsequent HV support groups.

Community engagement was persistent during CBE-HV and foundational to the design of the other CBE-HV strategies. The HV program and community steering committee worked collaboratively to identify community-centered strategies that were sensitive to local family needs and could be delivered by the HV program. Leveraging new community partnerships, HV also shifted from mainly passive recruitment (e.g., standard birth hospital referrals) to include active recruitment. In particular, the community steering committee, local churches, and schools were used to distribute promotional materials for the HV program. This change supported an effort to reach all eligible families, to increase prenatal engagement to HV, and to build community trust and recognition of HV.

The second strategy was to use knowledge gained from the community steering committee to guide development of additional supports for families in HV. These ancillary family supports included monthly maternal support group sessions and weekly instrumental support provided at a trusted, community-centered location. Support groups were designed to complement the HV curriculum and serve as a mechanism to improve participation as demonstrated in past research (Constantino et al., 2001). Support groups were 2-hour sessions that cultivated social support networks and provided exposure to content adapted to community needs (e.g., job skill development, education advancement). Regular instrumental support including clothing, diapers, and childcare items was identified by the community as a means to stem individual crises related to tangible need and to incentivize participation. A community church was used weekly to distribute the donated items.

The third strategy was to enhance the HV program by adding a community coordinator, a community liaison, and a modified discharge policy. These enhancements were to ensure the HV practices and policies were aligned with the unique needs of families. A CBE-HV coordinator was assigned to the target community and was responsible for deploying ancillary family supports and facilitating community engagement. The CBE-HV coordinator served as an added layer of support for home visitors by reaching out to mothers with excessive missed visits to attempt re-engagement. Unlike the approach used in comparison communities in which home visitors managed client discharges, the CBE-HV coordinator was required to approve discharges due to excessive missed visits. This permitted the use of a community liaison (a mother enrolled in HV), who was hired to assist in reengagement efforts. The community liaison worked directly for the CBE-HV coordinator and contacted all families at enrollment into HV and monthly thereafter to encourage participation in HV and support groups. Most (70%) home visits were delivered by a primary service agency to facilitate efficient coordination.

Comparison Condition

There were five areas in which the comparison condition differed from CBE-HV. First, there was no concerted effort in HV to form strategic community partnerships. Community-specific engagement was minimal as service agencies were active in multiple communities. Second, exposure to HV support group sessions was not organized and promoted at the community level. Third, the provision of tangible items was provided through only standard HV support including cribs, developmental materials and literature. Fourth, there was not a community-based coordinator to support and manage community engagement. Fifth, home visitors engaged clients without added supports of the CBE-HV coordinator and a community liaison. Typical procedures used to prevent disengagement from HV included a minimum of six attempts to contact families with excessive missed visits and the requirement of a 90-day window to attempt reengagement of families prior to discharge.

Measurement

Client retention was measured as the number of days between the first home visit and discharge or the last recorded home visit; families could remain active until the child was three years of age. Retention rates were also calculated at 12, 24, and 36 months post enrollment. The number of home visits received by families was also measured in each group.

Several potential covariates and confounding factors were evaluated for inclusion in a multivariable model to predict HV retention. The following factors were examined because of the empirical relevance to HV engagement and retention: maternal age, income level, marital status, adequacy of prenatal care, depression, interpersonal support, and family stress. Gestational age and weight at birth were also evaluated in model selection. All measures were (1) collected in the home by professional home visitors trained in clinical evaluation or (2) abstracted from birth records. All measures were collected at or near enrollment in HV. The inventories used to assess maternal and family risk are described below.

Edinburgh Postnatal Depression Scale (EPDS)

The EPDS was used to screen for maternal perinatal depression. The EPDS (Cox, Holden, & Sagovsky, 1987) is a 10-item inventory that was administered prenatally and at 3-months post-partum. We selected the cutoff value of ≥11 to indicate suspected major or minor depression in mothers. This cutoff was used in HV practice and is within the range of cutoffs supported by both prenatal and postnatal validation studies (Bergink et al., 2011; Matthey, Henshaw, Elliott, & Barnett, 2006).

Interpersonal Support Evaluation List (ISEL)

The interpersonal support of mothers was measured with the 12-item ISEL (Cohen & Hoberman, 1983). Because there are no clinical cutoffs recommended for the ISEL, we used the total score as a continuous scale variable.

Kempe Family Stress Inventory (KFSI)

The KFSI is a 90 minute semi-structured interview that was administered by a home visitor at enrollment to measure family stress. The KFSI is often used by HV programs to ascertain risk for child maltreatment and determine service eligibility (Korfmacher, 2000). The KFSI total score (derived from a 10-item scale) and each scale item were evaluated as reported in previous research (Ammerman et al., 2006).

In the comparison of the post-implementation study groups, all inventories were available for analyses. Only the KFSI data were available for the temporal comparison of the CBE-HV and comparison communities before (2000–2005) versus after (2006–2012) implementation.

Statistical Analysis

The differences between study groups in mother/child measures were analyzed with a Student’s t-test, Wilcoxon-Mann-Whitney test, or chi-square test. In the survival analysis, a multivariable Cox proportional-hazard regression was used to model the risk for attrition. Study participants were censored at their last home visit or at three years post-partum. Those who moved outside the service area or who had a child fatality were excluded from the calculation of retention rates. Maternal and child factors were selected for inclusion in the Cox regression model if statistically significant (p<0.05) or identified as confounders. We report hazards ratios (HR) as risk estimates for HV attrition. The proportional hazards assumption was tested by examining statistical interactions of model predictors with a log function of survival time. Stratified analyses were conducted for predictors that violated model assumptions.

In the temporal comparison of CBE-HV and comparison communities, a multivariable Cox proportional-hazard regression was used to model the risk for attrition before and after CBE-HV implementation. A difference-in-differences estimator was included in the model to test for interaction between the pre/post implementation period and the study community (CBE-HV vs. comparison). Model selection and validation was performed as stated above.

We also compared service agency retention rates to examine the potential systematic bias associated with the use of a primary service agency to deliver HV in the CBE-HV group. SAS 9.3 (SAS Institute Inc., Cary, NC) was used to perform all statistical analyses.

Data completeness

There were missing EPDS and ISEL measures for 21.6% and 14.0% of mothers, respectively. The percent missing was similar in both study groups post-implementation; these measures were not collected prior to implementation. Measures were missing when clients discharged early, provided insufficient data for scoring, or did not receive the measure because of infrequent visits. Among participants who were missing inventories, the study groups did not differ significantly on the maternal and child factors shown in Table 1. Further, the EPDS score was not shown to confound the relationship of CBE-HV with HV attrition. To preserve all study participants in the Cox regression model, we used an all-inclusive multiple imputation approach to populate missing inventory scores. The data were assumed to be missing at random and imputed using a maximum likelihood estimating technique; 50 imputed datasets were created. We included 30 auxiliary correlates (e.g., KFSI item: parents with isolation, low self-esteem, or depression) in the imputation procedure to improve missing data handling accuracy consistent with current missing data handling practice (Enders, 2010).

Table 1.

Descriptive and Participation Characteristics of Study Groups Post- CBE-HV Implementation (2006–2012) and Pre- CBE-HV Implementation (2000–2005)

Post-Implementation Period
(2006–2012)		 Pre-Implementation Period
(2000–2005)
CBE-HVa Comparison p value CBE-HVb Comparison p value
Maternal Factors
 Agec (mean) 19.5 19.9 0.07 19.4 19.1 0.45
 Low Incomed (%) 98.1 98.8 0.40 95.4 96.0 0.40
 Unmarried (%) 97.8 97.0 0.49 94.8 95.0 0.90
 Late Prenatal Caree (%) 35.2 32.9 0.45 28.7 25.5 0.45
 EPDS ≥11 (%) 23.6 22.0 0.59 NC NC –
 ISEL (mean) 29.2 29.7 0.22 NC NC –
 KFSI (mean) 35.6 36.3 0.87 37.1 37.3 0.90
Child Factors
 Female (%) 46.4 49.3 0.42 47.2 50.1 0.57
 Preterm birth (%) 14.5 11.1 0.16 NC NC –
 Low birth weight (%) 12.2 13.9 0.67 15.4 15.4 0.99
HV Participation
 Active Days (median) 461 295 <0.01* 236 228 0.56
 Total Visits (median) 24 17 0.02* 13.5 13.0 0.87
 Retentionf
  6-months (%) 69.2 61.8 0.03* 58.3 58.3 0.99
  12-months (%) 55.1 41.2 <0.01* 33.9 35.9 0.67
  24-months (%) 33.3 23.5 <0.01* 16.5 18.7 0.57
 Prenatal Enrollment (%) 70.0 60.7 <0.01* 54.3 53.3 0.83
Open in a new tab

EPDS= Edinburgh Postnatal Depression Scale, ISEL= Interpersonal Support Evaluation List, KFSI= Kempe Family Stress Inventory, NC=Not Collected

a

CBE-HV Study Group

b

CBE-HV community prior to implementation of CBE-HV approach

c

Mother’s age at the time of enrollment into HV

d

Low income was determined as 200% of federal poverty level

e

Late prenatal care was defined as the first prenatal care visit occurring after the second trimester

f

There were 33 records in the CBE-HV group and 211 records in the post-implementation comparison group excluded from the retention calculations because the women were active in the HV program at the time of analysis or the child expired during service.

*

Statistically significant difference between the study groups

Results

Participant Characteristics

The CBE-HV group did not differ significantly from the post-implementation comparison group (2006–2012) on maternal and child factors (Table 1). Mothers exposed to CBE-HV were an average of 19.5 years, approximately five months younger than the average of 19.9 years in the comparison group. These study groups were similar on measures of parenting risk, maternal depression, and perceived interpersonal support. Prior to implementation, the CBE-HV and comparison communities also were similar on maternal and child factors (Table 1).

CBE-HV Implementation

Group session attendance was 40% during the years in which attendance was recorded, 2008–2012, and mothers attended an average of six (95% confidence interval [CI]: 4.6–7.3) group sessions. There were four community liaisons active at independent times during CBE-HV. Nearly 70% of participants enrolled prenatally, and an estimated 85% of the population eligible for HV in the CBE-HV community was reached for a minimum of one home visit. During CBE-HV implementation, the 44 home visitors in the CBE-HV group had an average of 5.5 years of service, which was not significantly different than the average of 5.0 years among the 106 home visitors in the comparison group (p=0.45).

Post-Implementation Comparison

The CBE-HV group had a median of 24 home visits and was enrolled for a median of 461days (Table 1). Relative to the comparison group, this was a difference of 166 additional enrollment days and 7 additional home visits for mothers in the CBE-HV group. The 6, 12, and 24 month post-enrollment retention rates were significantly higher in the CBE-HV group ranging from 69.2% vs. 61.8% (p=0.03) at 6-months to 32.3% vs. 23.5% (p<0.01) at 24-months post-enrollment (Table 1).

The final multivariable model included the main effect of CBE-HV exposure, EPDS score (≥ 11), and maternal age at enrollment (Table 2). Time of enrollment in HV (prenatal vs. post-natal) was identified as a significant predictor, but violated the proportional hazards assumption; therefore, parameter estimation was stratified by enrollment time. The CBE-HV group effect was statistically significant and protective (HR 0.77, 95% CI: 0.67, 0.88), indicating that the incidence of attrition was 23% less in the CBE-HV group. This was equivalent to 30% higher retention in the CBE-HV group relative to the comparison group throughout service. Elevated depressive symptoms at enrollment were associated with a 22% higher risk of attrition (HR 1.22, 95% CI: 1.08, 1.37). Preterm birth approached statistical significance (p=0.051), but was excluded from the model in favor of model parsimony.

Table 2.

Adjusted Risk Estimates of Attrition using Multivariable Cox Proportional Hazards Models

Models Hazards ratio 95% confidence interval p value
CBE-HV Post Implementation (2006–2012)
 CBE-HV Study Group 0.77 0.68, 0.89 <.01
 EPDS ≥11 1.22 1.08, 1.37 <.01
 Maternal Age 0.97 0.96, 0.98 <.01
Time & Community Interaction
 †Post vs. Pre: CBE-HV Community 0.59 0.48, 0.74 <.01
 Post vs. Pre: Comparison Communities 0.84 0.78, 0.90 <.01
 Maternal Age 0.98 0.97, 0.99 <.01
Open in a new tab

Note: The interaction between the time (2000–2005 / 2006–2012) and community (CBE-HV / control communities) variables was modeled by adjusting for only maternal age. Because the EPDS was not administered prior to 2006, the data were unavailable for inclusion in the Cox regression model for time*community interaction.

†

Study participants in the CBE-HV study group (2006–2012)

EPDS= Edinburgh Postnatal Depression Scale

Temporal Comparison

A statistically significant (p=0.004) interaction (i.e., difference-in-differences estimator) was observed between the pre/post implementation period and the study community (CBE-HV vs. comparison). Accounting for underlying temporal changes in comparison communities, exposure to CBE-HV was associated with a significantly lower risk (29%) of HV attrition in the CBE-HV community (HR: 0.71, 95% CI: 0.56, 0.89). Relative to pre-implementation, the post implementation period was associated with significantly lower probabilities of HV attrition in CBE-HV and comparison communities (Table 2); however, the effect was appreciably stronger in the CBE-HV community. Survival plots depict the retention probabilities as similar across communities prior to CBE-HV implementation with a subsequent increase greatest in the CBE-HV community (Figure).

Figure.

Figure

Open in a new tab

Survival plots showing the probability of home visiting retention for mothers in the CBE-HV and comparison communities, 2000–2005 vs. 2006–2012

Service Agency

The primary service agency used to deliver HV to the CBE-HV group had a non-CBE-HV retention rate similar to that of other service agencies in the program.

Discussion

Home visiting is a strategy for delivering services designed to promote positive parenting and prevent childhood exposure to toxic stress during a critical period of development. The degree of success in HV is believed to be dependent on the level of family engagement (Nievar et al., 2010). As the practice of HV evolves, practitioners have begun to consider novel strategies to strengthen the investment and engagement of families in service. Adapting a HV program to be more commensurate with clients’ needs may improve retention in HV during early childhood. In CBE-HV, the adaptation was deployed at the community-level, following that perceived need and acceptance of HV can spread from the community-level to the individual. This rationale stems from the ecological model and is underpinned by the promise of translational research methods that rely heavily on community participation (Wallerstein & Duran, 2010).

In this study, CBE-HV was associated with significantly higher HV retention among low income, African American mothers in a community with high socio-demographic and health disparities. The findings supported CBE-HV as influential to individual participation in HV and suggested this approach elicited community-level support and acceptance of HV. The positive effect in a high risk community was encouraging, because previous research has shown this risk can inhibit maternal follow-through with service (Alonso-Marsden et al., 2013). This effect was above and beyond other positive influences on retention that accrued over time including program maturity, initiation of quality improvement, and collaborations with obstetricians.

There have been few interventions developed at the “systems-level” to improve engagement in prevention programs (Ingoldsby, 2010), and this study is the first known evaluation of a community-based approach to improve HV retention. In clinical trials of medical interventions, community-based approaches have been successful in recruiting and retaining minority participants (Fouad, Johnson, Nagy, Person, & Partridge, 2014; Las Nueces, Hacker, DiGirolamo, & Hicks, 2012). Community health advisors, who were residents of a target community, were used to improve retention and adherence of minority women in a multi-center, randomized clinical trial (Fouad et al., 2014). The women with community health advisors had 80% adherence to study visits, whereas the comparison group women had 65% adherence. The positive impact of community health advisors on participation provides further support for the CBE-HV approach, in which community partners were integral to design and implementation.

In a provider-level intervention to increase retention in HV, attrition was 43% lower at study sites during a retention intervention (Ingoldsby et al., 2013). Nurse home visitors adapted service frequency and content to meet client need and capacity. In contrast, CBE-HV was focused on community adaptations to promote and complement standard HV. Although CBE-HV produced a more modest gain in retention, CBE-HV was implemented in only one high-risk community as opposed to full program sites that encompassed multiple communities of varying deprivation. Indeed, the possibility that there could be synergy between provider- and community-level interventions to achieve optimal retention warrants further investigation.

The positive CBE-HV effect could be attributed to the unique alignment of community-based strategies designed to mitigate poor retention among high-risk minority families. Congruent with evidence of successful recruitment and retention of minority participants in health research (Yancey, Ortega, & Kumanyika, 2006), community-based organizations, mainly churches, were used in CBE-HV to support service (e.g., support groups) and promote trust. The high percentage (85%) of HV eligible families reached in the CBE-HV community supports the use of community organization such as churches to engage these populations. Culturally accepted staff assisted in engagement as evidenced in past research (Yancey et al., 2006), and family needs including social and instrumental support were addressed.

In addition to the CBE-HV effect, we observed that elevated depressive symptoms were associated with lower retention. Although consistent with Smith and Moore (2012), this result conflicts with research that has linked depressive symptoms to higher retention (Girvin et al., 2007). Girvin et al. examined a HV program with a short duration (3-months), which may account for the differential findings. Intervening with depression should be considered as a way to improve retention (Ammerman, Putnam, Teeters, & Van Ginkel, 2014). Sustained participation in a long-term prevention program could be too difficult for depressed mothers.

The study had several strengths. First, we used a large sample of families from an established HV program that uses a widely-disseminated model. Second, routine program data collection allowed for the control of baseline maternal risks known to predict engagement. Third, the CBE-HV effect was demonstrated in a ‘real-world’ setting within an existing HV practice. Fourth, the study was conducted with comparisons across communities and over time using difference-in-differences to isolate the CBE-HV effect in the context of other temporal trends.

There were also important limitations to this study. First, the study design was quasi-experimental. Although the study participants were restricted to high-risk African American mothers in the same urban core and who were motivated to enroll in HV, the potential exists that the study groups differed in motivation for long-term participation. Diminishing concerns of a biased effect was the equivalence of the study groups on key baseline maternal and child factors, statistical adjustment for known predictors of engagement, and positive effects demonstrated in both post-implementation and temporal comparisons of the target community. The findings are generalizable to only predominantly African American communities with high levels of health disparity, albeit a critical segment of the population to engage in prevention programs. The CBE-HV approach should be tested further in a cluster randomized controlled trial.

Second, a primary service agency was assigned to deliver most HV to the CBE-HV study group. This component was ‘by design,’ but precluded adjustment for agency clustering that could have contributed to the outcome of retention. However, an analysis of non-CBE-HV client retention within the primary service agency revealed similar retention to other service agencies. On the provider-level, the tenure of home visitors did not differ significantly between the study communities. The HV program provided standardized training for all home visitors, hosted frequent meetings for service agency leadership, and required a common curriculum. Together, the findings and program features reduce the likelihood that service agency or provider was the significant driver of the observed retention effect.

Third, 14% and 22% of study participants in both groups had missing baseline EPDS and ISEL measures, respectively. To retain the participants in the study analysis, we used current missing data techniques with auxiliary correlates to impute EPDS and ISEL scores. The inclusion of potential correlates of missingness has been shown to improve the plausibility of the ‘missing at random’ assumption and reduce bias (Enders, 2010); a chance remains, however, that the assumption used for imputation led to a biased estimate of EPDS effect.

Fourth, the attendance of support groups was recorded to be less than half of mothers in the CBE-HV group. However, mothers were still exposed to the other components of CBE-HV such as the presence of a community liaison. Nevertheless, better participation in support groups may have resulted in a larger effect size. Subsequent research should be conducted to disentangle the components of the CBE-HV and elucidate the independent contribution of each strategy.

This study supports CBE-HV as a promising approach to achieve stronger retention in HV. Community-based partnerships can serve to identify the needs of families and cultivate a culture of trust and value for prevention programs. This research has implications for practice including integrating and advancing the evaluation of novel strategies to improve HV retention.

Acknowledgments

The authors acknowledge the participation and support of the United Way of Greater Cincinnati, Kentucky H.A.N.D.S., and Ohio Help Me Grow.

Funding: This study was supported in part by Grant R01MH087499 from the National Institute of Mental Health to Ammerman.

Footnotes

Compliance with Ethical Standards

Disclosure of potential conflicts of interest: The authors declare that they have no conflict of interest.

Ethical approval: All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. This article does not contain any studies with animals performed by any of the authors.

Informed consent: For this type of study (retrospective) formal consent is not required.

References

  1. Adirim T, Supplee L. Overview of the federal home visiting program. Pediatrics. 2013;132(Suppl 2):S59–S64. doi: 10.1542/peds.2013-1021C. [DOI] [PubMed] [Google Scholar]
  2. Alonso-Marsden S, Dodge KA, O’Donnell KJ, Murphy RA, Sato JM, Christopoulos C. Family risk as a predictor of initial engagement and follow-through in a universal nurse home visiting program to prevent child maltreatment. Child Abuse & Neglect. 2013;37:555–565. doi: 10.1016/j.chiabu.2013.03.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Ammerman RT, Putnam FW, Teeters AR, Van Ginkel JB. Moving beyond depression: A collaborative approach to treating depressed mothers in home visiting programs. Zero to Three. 2014;34(5):20–27. [Google Scholar]
  4. Ammerman RT, Stevens J, Putnam FW, Altaye M, Hulsmann JE, Lehmkuhl HD, Van Ginkel JB. Predictors of early engagement in home visitation. Journal of Family Violence. 2006;21:105–115. [Google Scholar]
  5. Astuto J, Allen L. Home visitation and young children: An approach worth investing in? Society for Research in Child Development, Social Policy Report. 2009;23:3–22. [Google Scholar]
  6. Bergink V, Kooistra L, Lambregtse-van den Berg MP, Wijnen H, Bunevicius R, van Baar A, Pop V. Validation of the Edinburgh Depression Scale during pregnancy. Journal of Psychosomatic Research. 2011;70:385–389. doi: 10.1016/j.jpsychores.2010.07.008. [DOI] [PubMed] [Google Scholar]
  7. Caldera D, Burrell L, Rodriguez K, Crowne SS, Rohde C, Duggan A. Impact of a statewide home visiting program on parenting and on child health and development. Child Abuse & Neglect. 2007;31:829–852. doi: 10.1016/j.chiabu.2007.02.008. [DOI] [PubMed] [Google Scholar]
  8. Cohen S, Hoberman HM. Positive events and social supports as buffers of life change stress1. Journal of Applied Social Psychology. 1983;13:99–125. [Google Scholar]
  9. Constantino JN, Hashemi N, Solis E, Alon T, Haley S, McClure S, Carlson VK. Supplementation of urban home visitation with a series of group meetings for parents and infants: Results of a “real-world” randomized, controlled trial. Child Abuse & Neglect. 2001;25:1571–1581. doi: 10.1016/s0145-2134(01)00292-7. [DOI] [PubMed] [Google Scholar]
  10. Cox JL, Holden JM, Sagovsky R. Detection of postnatal depression. Development of the 10-item Edinburgh Postnatal Depression Scale. The British Journal of Psychiatry. 1987;150:782–786. doi: 10.1192/bjp.150.6.782. [DOI] [PubMed] [Google Scholar]
  11. Damashek A, Doughty D, Ware L, Silovsky J. Predictors of client engagement and attrition in home-based child maltreatment prevention services. Child Maltreatment. 2011;16:9–20. doi: 10.1177/1077559510388507. [DOI] [PubMed] [Google Scholar]
  12. Daro D, McCurdy K, Falconnier L, Stojanovic D. Sustaining new parents in home visitation services: Key participant and program factors. Child Abuse & Neglect. 2003;27:1101–1125. doi: 10.1016/j.chiabu.2003.09.007. [DOI] [PubMed] [Google Scholar]
  13. Dimick JB, Ryan AM. Methods for Evaluating Changes in Health Care Policy: The Difference-in-Differences Approach. JAMA. 2014;312:2401–2402. doi: 10.1001/jama.2014.16153. [DOI] [PubMed] [Google Scholar]
  14. Dobbie W, Fryer RG., Jr Are high-quality schools enough to increase achievement among the poor? Evidence from the harlem children’s zone. American Economic Journal: Applied Economics. 2011;3:158–187. [Google Scholar]
  15. Duggan A, Caldera D, Rodriguez K, Burrell L, Rohde C, Crowne SS. Impact of a statewide home visiting program to prevent child abuse. Child Abuse & Neglect. 2007;31:801–827. doi: 10.1016/j.chiabu.2006.06.011. [DOI] [PubMed] [Google Scholar]
  16. Duggan A, McFarlane E, Fuddy L, Burrell L, Higman SM, Windham A, Sia C. Randomized trial of a statewide home visiting program: Impact in preventing child abuse and neglect. Child Abuse & Neglect. 2004;28:597–622. doi: 10.1016/j.chiabu.2003.08.007. [DOI] [PubMed] [Google Scholar]
  17. DuMont K, Kirkland K, Mitchell-Herzfeld S, Ehrhard-Dietzel S, Rodriguez ML, Lee E, Layne C, Greene R. A randomized trial of Healthy Families New York (HFNY): Does home visiting prevent child maltreatment. 2010 Retrieved from http://www.ocfs.state.ny.us/main/reports.
  18. Enders CK. Applied missing data analysis. Guilford Publications; 2010. [Google Scholar]
  19. Fouad MN, Johnson RE, Nagy MC, Person SD, Partridge EE. Adherence and retention in clinical trials: A community‐based approach. Cancer. 2014;120(S7):1106–1112. doi: 10.1002/cncr.28572. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Girvin H, DePanfilis D, Daining C. Predicting program completion among families enrolled in a child neglect preventive intervention. Research on Social Work Practice. 2007;17:674–685. [Google Scholar]
  21. Gomby DS. Home visitation in 2005: Outcomes for children and parents. Washington, DC: Committee on Economic Development; 2005. [Google Scholar]
  22. Goyal NK, Hall ES, Jones DE, Meinzen-Derr JK, Short JA, Ammerman RT, Van Ginkel JB. Association of maternal and community factors with enrollment in home visiting among at-risk, first-time mothers. American Journal of Public Health. 2014;104:S144–S151. doi: 10.2105/AJPH.2013.301488. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Goyal NK, Hall ES, Meinzen-Derr JK, Kahn RS, Short JA, Van Ginkel JB, Ammerman RT. Dosage effect of prenatal home visiting on pregnancy outcomes in at-risk, first-time mothers. Pediatrics. 2013;132(Suppl 2):S118–S125. doi: 10.1542/peds.2013-1021J. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Holland ML, Xia Y, Kitzman HJ, Dozier AM, Olds DL. Patterns of visit attendance in the nurse–family partnership program. American Journal of Public Health. 2014;104:e58–65. doi: 10.2105/AJPH.2014.302115. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Ingoldsby EM. Review of interventions to improve family engagement and retention in parent and child mental health programs. Journal of Child and Family Studies. 2010;19:629–645. doi: 10.1007/s10826-009-9350-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Ingoldsby EM, Baca P, McClatchey MW, Luckey DW, Ramsey MO, Loch JM, Smith BJ. Quasi-experimental pilot study of intervention to increase participant retention and completed home visits in the Nurse–Family Partnership. Prevention Science. 2013;14:525–534. doi: 10.1007/s11121-013-0410-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Korfmacher J. The Kempe Family Stress Inventory: A review. Child Abuse & Neglect. 2000;24:129–140. doi: 10.1016/s0145-2134(99)00115-5. [DOI] [PubMed] [Google Scholar]
  28. Las Nueces D, Hacker K, DiGirolamo A, Hicks LS. A systematic review of community‐based participatory research to enhance clinical trials in racial and ethnic minority groups. Health Services Research. 2012;47:1363–1386. doi: 10.1111/j.1475-6773.2012.01386.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Lassi ZS, Haider BA, Bhutta ZA. Community‐based intervention packages for reducing maternal and neonatal morbidity and mortality and improving neonatal outcomes. The Cochrane Library. 2010 doi: 10.1002/14651858.CD007754.pub2. [DOI] [PubMed] [Google Scholar]
  30. Luby J, Belden A, Botteron K, Marrus N, Harms MP, Babb C, Barch D. The effects of poverty on childhood brain development: The mediating effect of caregiving and stressful life events. JAMA Pediatrics. 2013;167:1135–1142. doi: 10.1001/jamapediatrics.2013.3139. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Matthey S, Henshaw C, Elliott S, Barnett B. Variability in use of cut-off scores and formats on the edinburgh postnatal depression scale–implications for clinical and research practice. Archives of Women’s Mental Health. 2006;9:309–315. doi: 10.1007/s00737-006-0152-x. [DOI] [PubMed] [Google Scholar]
  32. McCurdy K, Daro D. Parent involvement in family support programs: An integrated theory. Family Relations. 2001;50:113–121. [Google Scholar]
  33. McCurdy K, Gannon RA, Daro D. Participation patterns in home‐based family support programs: Ethnic variations. Family Relations. 2003;52:3–11. [Google Scholar]
  34. McFarlane E, Crowne SS, Burrell L, Duggan A. Home visiting service delivery and outcomes for depressed mothers. Zero to Three. 2014;34(5):53–60. [Google Scholar]
  35. McGuigan WM, Katzev AR, Pratt CC. Multi-level determinants of retention in a home-visiting child abuse prevention program. Child Abuse & Neglect. 2003;27:363–380. doi: 10.1016/s0145-2134(03)00024-3. [DOI] [PubMed] [Google Scholar]
  36. Navaie-Waliser M, Martin SL, Campbell MK, Tessaro I, Kotelchuck M, Cross AW. Factors predicting completion of a home visitation program by high-risk pregnant women: The North Carolina Maternal Outreach Worker Program. American Journal of Public Health. 2000;90:121–124. doi: 10.2105/ajph.90.1.121. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Nievar MA, Van Egeren LA, Pollard S. A meta‐analysis of home visiting programs: Moderators of improvements in maternal behavior. Infant Mental Health Journal. 2010;31:499–520. doi: 10.1002/imhj.20269. [DOI] [PubMed] [Google Scholar]
  38. O’Brien RA, Moritz P, Luckey DW, McClatchey MW, Ingoldsby EM, Olds DL. Mixed methods analysis of participant attrition in the Nurse-Family Partnership. Prevention Science. 2012;13:219–228. doi: 10.1007/s11121-012-0287-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Olds DL, Kitzman H, Cole R, Robinson J, Sidora K, Luckey DW, Holmberg J. Effects of nurse home-visiting on maternal life course and child development: Age 6 follow-up results of a randomized trial. Pediatrics. 2004;114:1550–1559. doi: 10.1542/peds.2004-0962. [DOI] [PubMed] [Google Scholar]
  40. Olds DL, Kitzman H, Knudtson MD, Anson E, Smith JA, Cole R. Effect of home visiting by nurses on maternal and child mortality: Results of a 2-decade follow-up of a randomized clinical trial. JAMA Pediatrics. 2014;168:800–806. doi: 10.1001/jamapediatrics.2014.472. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Shonkoff JP, Garner AS, Siegel BS, Dobbins MI, Earls MF, McGuinn L, Wood DL. The lifelong effects of early childhood adversity and toxic stress. Pediatrics. 2012;129:e232–e246. doi: 10.1542/peds.2011-2663. [DOI] [PubMed] [Google Scholar]
  42. Smith SA, Moore EJ. Health literacy and depression in the context of home visitation. Maternal and Child Health Journal. 2012;16:1500–1508. doi: 10.1007/s10995-011-0920-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Sweet MA, Appelbaum MI. Is home visiting an effective strategy? A meta‐analytic review of home visiting programs for families with young children. Child Development. 2004;75:1435–1456. doi: 10.1111/j.1467-8624.2004.00750.x. [DOI] [PubMed] [Google Scholar]
  44. Tandon SD, Parillo K, Mercer C, Keefer M, Duggan AK. Engagement in paraprofessional home visitation: Families’ reasons for enrollment and program response to identified reasons. Women’s Health Issues. 2008;18:118–129. doi: 10.1016/j.whi.2007.10.005. [DOI] [PubMed] [Google Scholar]
  45. Wallerstein N, Duran B. Community-based participatory research contributions to intervention research: The intersection of science and practice to improve health equity. American Journal of Public Health. 2010;100:S40–S46. doi: 10.2105/AJPH.2009.184036. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Yancey AK, Ortega AN, Kumanyika SK. Effective recruitment and retention of minority research participants. Annual Review of Public Health. 2006;27:1–28. doi: 10.1146/annurev.publhealth.27.021405.102113. [DOI] [PubMed] [Google Scholar]

RESOURCES