Service Engagement and Retention: Lessons from the Early Childhood Connections Program

Abstract

The high attrition rates found in studies of early childhood home visitation create barriers to measuring the effectiveness of such programs. Most studies examine attrition at program completion. This practice may mask important differences in characteristics between families that end participation at various time points. This study helps address this gap by examining factors associated with percent attrition for early drop out (before three months) compared to the program midpoint (nine months or more) and program completion (18 months) using data from the treatment arm of a small feasibility study of enhanced referral to home visitation among child welfare-involved families (n = 64). Caregivers who identified as White tended to leave by the program midpoint and caregivers who had better social support were more likely to stay at the end of the program. This study is the only published study to date of participation in a community-based home visitation program by child welfare-involved families but several trends identified were consistent with prior studies with other populations. Given the very small sample size, both statistically significant and near significant trends are discussed in the context of existing literature. The practical variation found has implications for continuing to build knowledge of attrition in early childhood home visitation.

1. Introduction

Many models of home visitation for families with young children exist and this approach is frequently endorsed for preventing child maltreatment as well as promoting infant health and developmental outcomes (Brown & Sturgeon, 2004; Stagner & Lansing, 2009; Howard & Brooks-Gunn, 2009). Recently Congress reauthorized $400 million per year for 2016 and 2017 through the Maternal, Infant, and Early Childhood Home Visiting Program (MIECHV) to strengthen and expand programs across the United States (Medicare Access and CHIP Reauthorization Act, 2015). There has been increasing interest in testing these approaches with families involved in child welfare system (Chaffin, Hecht, Bard, Silovsky, & Beasley, 2012; Wald, 2014), despite some questions about whether or not these approaches prevent maltreatment (Reynolds, Mathieson & Topitzes, 2009).

Understanding the effect of such services with vulnerable populations, however, is complicated by high attrition rates. Across various studies of different home visitation program models, attrition rates range from 30% to 70% (Drotar, Robinson, Jeavons, & Lester, 2009; Foulon et al., 2015; Wagner, Clayton, Gerlach-Downie, & McElroy, 1999; Wagner, Spiker, & Linn, 2002). It has been suggested that measuring whether or not home visitation can be used to prevent child maltreatment (and other longer-term outcomes) is impossible as long as the attrition rates remain so high (Daro, McCurdy, Falconnier, & Stojanovic, 2003). This makes improving our understanding of attrition key to advancing work in home visitation (Ammerman, 2016).

While many studies note attrition as a problem, most studies of early childhood home visitation examine program drop out at a single point (typically at the close of a study or the end of a manualized intervention) instead of considering whether the family discontinued services rapidly or stayed for a significant portion of the program. This practice may mask important differences that could be used to inform efforts to promote longer-term retention. Further, only one program, SafeCare (Chaffin et al., 2012), has been subject to significant testing with a child welfare population, but this program is atypical in its short duration (i.e., about three months compared to the two-year expectation for most MIECHV programs). The present study helps fill this gap by looking at the timing of attrition from a community-based early childhood home visitation program by participant characteristics within a child welfare involved population.

1.1. Background

Home visitation programs are promoted as effective means of preventing child maltreatment as well as promoting infant health and development. Many, if not most, early childhood home visiting models are based upon the idea of longer term service provision, ideally 2–3 years (U.S. Department of Health & Human Services, 2013). Sweet and Appelbaum (2004) examined 60 home visiting programs and found that most were to be delivered for 12 to 24 months (30%), followed by 24 to 36 months (23.3%), and a few for 9 to 12 months (18.3%). While families are expected to stay for a long period to benefit, reported attrition rates suggest that relatively few families complete the desired dosage (Ammerman, 2016). It is unclear if attrition contributes to conflicting results regarding outcomes like the prevention of maltreatment (LeCroy & Whitaker, 2005; Duggan, et al., 1999; Filene, Kaminski, Valle & Chachat, 2013; Reynolds et al, 2009). For example, Holland and colleagues (2014) found mothers who attended at least 50% of recommended visits had better child health outcomes than mothers with less frequent attendance. Filene and colleagues (2013) found that studies that measured outcomes during treatment were stronger than those measuring post-test outcomes-though it was not clear if attrition played a role.

Most home visitation studies have high attrition rates. An early review of various home visitation programs reported that 8% to 51% of the families left their programs within the first 12 months (Guterman, 2000). These findings persist across program models, locations, and time. A study of Hawaii’s Healthy Start Program reported attrition rates of 30% by six months, 44% by nine months, and 51% by 12 months (Duggan et al, 1999). In a study of a Parents as Teachers model, attrition rates reached 47%–49% by the end of the first year and were close to 60% by the end of year two (Wagner, Spiker, and Linn, 2002). In a study of the Healthy Families America model, one-third of the families enrolled in a two-year program reached that milestone (Daro et al., 2003). Foulon and colleagues (2015) reported attrition rates of 17% by three months and 63% by 24 months in a French perinatal home-visiting program. While early trials of the Nurse Family Partnership program reported retention rates of 60–70%, later replications in Colorado were more consistent with other home visiting studies (O’Brien et al, 2012). A recent evaluation of SafeCare in Georgia reported a completion rate of 43% (Bolt, 2015).

1.2. Factors associated with attrition

Variations in program models and target populations complicate attempts to better understand attrition. Programs vary according to professional or paraprofessional delivery (Daro et al, 2003; Filene et al., 2013) though the impact of this is unclear. Programs also target very different populations and have varying requirements for when a caregiver enrolls. Some models/studies focus on programs designed for universal or lower risk groups (McGuigan, Katzey, & Pratt, 2003; O’Brien et al, 2012; Wagner, Spiker & Linn, 2002). For example, the Nurse Family Partnership model is designed to serve low-income mothers but also limits the population to first-time mothers who are already involved in prenatal care and engage in home visiting during the prenatal period (California Clearinghouse, n.d.). The Parents as Teachers model is designed to be a universal model and encourages prenatal enrollment but will enroll at any stage prior to kindergarten entry (Parents as Teachers, n.d.). In contrast other models focus on more vulnerable families. The Healthy Families of America model is designed to target families who have a number of risk factors and may be enrolled prenatally or within 3 months of birth (The Healthy Families of America, n.d.). Early Head Start is designed for children in families living at or below the federal poverty line and enrolls from pregnancy to three years of age (Home Visiting Evidence of Effectiveness, n.d.). Only a couple of programs have targeted those already contacting or at significant risk of child welfare involvement (Damashek et al, 2011; Girvin, DePanfilis, & Daining, 2007). Neither of these is specific to very early childhood, nor do they follow the same practice of long-term service. For example, the evidence-based model designed to serve child welfare involved families (e.g., SafeCare) serves families with children ages birth to five but is typically limited to 22 weeks total (Home Visiting Evidence of Effectiveness, n.d.). While some of the existing long-term early childhood home visitation models described above may serve families involved in child welfare, retention and outcomes specific to this population have not been reported.

1.2.1 Race and poverty

The association of service participation and race/ethnicity may be confounded by poverty. Our understanding of this interaction among child welfare involved populations, is related to child welfare contact rather than community-based services participation. Given the focus of the present study on home visitation with child welfare involved families, this interaction is still relevant. Under-represented minority populations are generally much more likely to both live in families with lower incomes and live in lower income neighborhoods (Drake & Jonson-Reid, 2009). There are recent data suggesting that maltreatment may be both more common, both in the self-report (Slopen et al, 2016) and in official reports which lead to child welfare contact (Kim & Drake, 2018) among poor White than poor Black children. While Hispanic populations are similarly poor, there rates of system contact for both child welfare and other health indicators is often more similar to White populations (Kim & Drake, 2018; Putnam-Hornstein, Needell, King & Johnson-Motoyama, 2013). On the other hand, accessing services needed to address issues that cause contact with child welfare may be complicated by lack of resources in high poverty communities (Lorthridge, McCroskey, Pecora, Chambers, & Fatemi, 2012). Further lower resourced neighborhoods may also increase exposure to risk factors that are associated with a greater risk of maltreatment reports (Friesthler & Weiss, 2009).

Of course, access to home visitation and retention once enrolled in services are different. Most studies examining home visiting attrition (leaving a program after enrolled) and race have found a higher retention rate for Black mothers (Daro et al, 2003; McCurdy, Gannon & Daro, 2003; Navaie-Waliser et al, 2000). Daro and colleagues (2003) also found longer term participation for Hispanics compared to white mothers; a more recent study using data from multiple countries found higher retention among Hispanic women compared to all others (McGuigan & Gassner, 2016). A study of Early Head Start programs found no significant differences in early leaving by race (Caronongan, Moiduddin, West, & Vogel, 2014). In regard to initial engagement compared to follow-through on appointments, one study was found that examined the possible confound of demographic risk (i.e., poverty and age) with race. Alonso-Marsden and colleagues (2013) found that Black and Hispanic women, as well as women with higher demographic risk (poverty), were more likely to enroll in a very brief nurse home visiting program. The positive association for Black women on follow through with appointments, however, was almost entirely mediated by the demographic risk that was negatively associated with keeping appointments. The relationship between high demographic risk and program dropout was also found in an evaluation of Early Head Start (Caronongan et al., 2014). Again, program and population variation make it difficult to compare findings across studies.

1.2.2. Age

The findings related to retention and mothers age are more consistent. Most studies (Brand & Jungman, 2014; Daro et al., 2003; Fifolt, Lanzi, Johns, Strichik, & Preskitt, 2016; Josten et al., 2002; McGuigan, Katzey, & Pratt, 2003) agree that older women stay longer. Not all studies, however, find equally strong associations. In addition, the age range of mothers involved in the studies vary. As an example, Damashek and colleagues (2011) reported that the likelihood of SafeCare program completion increased 10% per year of mother’s age, but the majority of the mothers in the study were in their 20’s.

1.2.3 Mental health

There are a number of inconsistent findings regarding the association between mental health and attrition. Comparing findings is complicated by the fact that different studies measure mental health very differently and sometimes in combination with other factors. For example, some studies found mothers with more significant mental health or stress issues (e.g., depression, emotional partner violence, high stress) stayed longer (e.g., Ammerman et al., 2006; Girvin et al., 2007). Other studies found that mothers with fewer mental health/health behavior issues (e.g., not isolated, non-smoking, no substance abuse problems) stayed longer (e.g., Barlow et al., 2015; Damashek et al., 2011; McGuigan & Gassner, 2016; Navaie-Waliser et al., 2000). In a study of a nurse home visiting program, both mothers with very low levels of need (a composite including mental health) and very high levels of need were more likely to drop out of the program (Holland et al., 2014).

1.2.4. Relationship to home visitors

Some work suggests that the relationship between home visitors and mothers is key to program retention. One study (Ammerman et al., 2006) found that the first few home visits were key to developing a relationship between the caregiver and visitor that promoted longer-term participation. The enthusiasm of participants at the start of home visits may be related to service providers’ behaviors (Duggan et al., 1999). Duggan and colleagues (1999) argued that service providers showed more interest in working with clients who demonstrated enthusiasm during the initial visits, but providers were less likely to continue to reach out to indifferent clients. Other factors such as the level of experience, a match with the mother in regard to community or racial/ethnic background and quality of supervision have all been noted a potential factor influencing participant engagement but empirical data are limited and sometimes conflicting (Korfmacher et al., 2008).

1.2.5. Factors associated with early engagement or later retention

Despite the widespread concern about attrition and the acknowledgement that families drop out at different times, relatively few studies have purposively examined program leaving at different time points. Within this relatively scant literature, there is great variation in how longer or shorter periods are defined and analyzed. A large scale evaluation study reported a retention by 3, 6 and 12 months for five home visiting programs (HFA, NFP, PAT, SafeCare and Triple P). Three-month retention ranged from 80 to 91.5%; PAT=89.4%. Given that SafeCare and Triple P are shorter term programs, retention for 6 and 12 months was very low. Retention among the three programs typically designed to continue service for 2+ years (HFA, NFP and PAT), six-month retention ranged from 76.5 to 82.3% and 12-month retention ranged from 57.6 to 73% (Boller et al., 2014).

Only a few studies have examined factors associated with varying lengths of stay in a program with a wide range of measures of participation. One study of a nurse home visitation program found that participants who were married, enrolled in the program during an early stage of their pregnancy, and who had greater social support needs were more likely to stay longer than 10 months as compared to less than 10 months following birth (Navaie-Waliser et al., 2000). Holland et al (2014) looked at participation in terms of adherence to visits; low attenders, increasing attenders and high attenders and found that the lower and higher risk groups left earlier. Foulon and colleagues (2015) found that women who reported a prior history of abortion, not working or studying, feeling insecure in their attachment, and no tobacco use during pregnancy were more likely to drop out of the home visitation program early (by 3 months). Daro and colleagues (2003) found Black and Hispanic women, older women and employed women were more likely to stay longer, but measured duration as a continuous number of months. An analysis of home visiting across 12 different countries found that older mothers, Hispanic compared to white mothers and the quality of supervision given to home visitors were all positively associated with their participants remaining in home visitation at least 12 months compared to dropping out earlier (McGuigan & Gassner, 2016). In a study of American Indian adolescent mothers, most attrition occurred prior to 12 months (Barlow et al., 2015). Brand and Jungman (2014) studied a two-year nurse home visiting program in Germany and looked at retention in terms of 25% (about 6 months) as compared to completing 75% (about 18 months) of the possible time period. Early leavers were younger and had fewer pregnancy risks. In a very brief home visiting intervention program, Alonso-Marsden and colleagues found that Black and lower-income women were more likely to enroll but less likely to follow through on a scheduled nurse home visit (2013). In qualitative studies, relationship factors, as well as the provision of information and resources individualized according to a caregiver’s need, have been found to predict longer-term retention (Beasley et al., 2017).

1.3. Theoretical framework

Prior work on factors associated with home visiting retention is consistent with an ecological perspective. Bronfenbrenner’s (1992) model can be clearly seen in the way microsystem factors (e.g. child and family characteristics) interact with mesosystem factors (e.g. neighborhood, program worker and program characteristics). While generally not measured directly, macrosystem factors (e.g. child welfare system policies and functions) have an underlying role to play. Indeed, McCurdy and Daro (2001) proposed an integrated theory of parent involvement that closely aligns with the ecological framework suggesting that individual, provider, program and neighborhood context all impact participation. They also suggested that higher parent perception of need would predict initial enrollment. In at least one study of very short-term engagement, however, the higher demographic risk was predictive of enrollment, but not infant health risk or needs (Alonso-Marsden et al, 2013). One limitation of the prior research is the limited attention paid to another key aspect of Bronfenbrenner’s model, the chronosystem. The chronosystem includes transitions and changes over time. The present study builds on prior work assessing micro-system and limited meso-system characteristics and seeks to refine our understanding of the chronosystem by purposively addressing retention at multiple time points.

1.4 Research questions

Despite the recognition of the importance of both initial service engagement and retention, few studies have examined factors related to when (what stage of the program) participation ends as compared to reporting an overall attrition rate at the close of the study. Additionally, only one study has focused on attrition and home visitation with child welfare and that program is a much shorter term in nature (Chaffin et al., 2012). The present analysis helps fill this gap using data from a small effectiveness study of participation in a para-professional home visitation program among low-income, intact families reported to child protective services. The goal was to explore participation according to stages related to early, midpoint and final completion to see if variation in predictive factors could be found.

For the current study, variable selection was guided by prior research within an ecological framework exploring individual demographics, select risk and protective factors and service provider factors that may influence initial engagement, longer-term retention and study completion. The time periods selected [left prior to 3 months (initial engagement consistent with Boller et al., 2014; Foulon et al., 2014), 9 months (midpoint) and 18 months(completion)] were chosen to mimic the few prior studies that have reported retention at specific time periods as closely as possible. The research questions were as follows:

• Question 1. What factors are associated with caregiver early engagement (at least 3 months) of the program?

• Question 2. What factors are associated with caregiver retention (9 months or more) of the program?

• Question 3. What factors are associated with caregiver completion (18 months or more) of the program?

While there is some consensus in regard to older age predicting participation, most other variables tested have inconsistent findings. Further given the variation in how participation has been measured and lack of information specific to this population, it was unclear if similar findings would hold. Therefore, specific hypotheses were not advanced. Although small, this exploratory analysis is the first to focus on retention in community-based early childhood home visitation as a factor of time using a child welfare-involved sample.

2. Methods

The present data were drawn from a randomized controlled trial of a specialized engagement approach (Early Childhood Connections) conducted in a large Midwest metropolitan region. This program was designed to connect intact families with young children involved with child protective services (CPS) to an existing community resource, Parents as Teachers (PAT). Early Childhood Connections (ECC) was developed through a university-field partnership adding specialized engagement strategies and additional referral supports to the regional PAT program. This home visitation model was selected because of its inclusion on the list of effective programs that can be funded through MIECHV (U.S. Department of Health & Human Services, 2013) and its’ ready availability in the study area.

Programmatic changes to the regional PAT program due to the recession (that occurred just as the grant was implemented) caused many of the previously full-time staff to be reduced to part-time. This restricted their ability to take new cases and therefore part-time ECC PAT parent educators were employed directly by the university to participate in the current project outside of their regular hours. These home visitors received additional training in how CPS works, updated information on local resources for families, and were asked to suspend the common practice of terminating families who missed three scheduled appointments. This latter request was due to the innately high risk and mobile nature of our study population and because of the desire to explore whether the population was willing to participate in such services if barriers to access were removed [a more detailed discussion of the study implementation and results can be found in f (Jonson-Reid et al., 2018; Stahlschmidt et al., 2018)].

2.1. ECC home visitors

All parent educators at the time were initially hired through school districts, had to have a high school diploma, some prior relevant prior training in child development, and had to complete the PAT basic national training program (Parents as Teachers, n.d.). Two of three of the home visitors who began the study were Black and the other White; all were female. All lived within the study target region. During the study, the White home visitor took another position and was replaced by a Black home visitor. Only one home visitor had a substantial college education and had become a PAT educator after retiring from another position. The study had human subjects approval from Washington University in St. Louis as well as the participating agency or administrative review processes.

2.2. Sample and Data Collection

Because CPS must assess and/or serve families as they are hotlined, families are assigned to units on a space available basis. Therefore, it was necessary for randomization to occur at the CPS worker level. Workers in units randomly selected to the treatment condition included hotline and voluntary in-home service workers. Participation was voluntary and consenting workers received additional presentations on early childhood development and home visitation. Treatment condition CPS workers (n=57) offered eligible families a referral to Parents as Teachers (PAT), but otherwise provided no services outside usual care. Control unit CPS workers (n=65) provided usual care only and were asked to refer eligible families to a study of child development.

Families were referred following an investigation or assessment that either resulted in no ongoing services or in-home voluntary case management services. None of the families had CPS cases that involved mandated services or family court involvement at the time of referral. If families whose workers offered PAT were interested in a visit (assented) to find out more, ECC assisted in sending a worker to the home. Once there, the PAT worker explained their program and offered to enroll them in voluntary home visiting services. Once a family in the treatment group agreed to PAT, they were informed of the study by the PAT worker. Interested families were contacted by a member of the research team (trained graduate students) who went to the home to explain the study and obtain written consent. Usual care families (not included in the present paper) were alerted to the possibility of participating in a study by their CPS workers and contacted by the research team if interested.

Eligible families had at least one CPS hotline referral prior to the study and had at least one child (not placed into foster care) between the ages of two months to 2.5 years. The age range was set to assure the child was old enough for the planned developmental screen and young enough so that the family could participate in PAT prior to the transition to preschool. Due to IRB requirements regarding how families were referred to the study (one group by PAT and the other directly from CPS), it was impossible to blind the interviewers to a family’s experimental condition. Maternal risk indicators measured in interviews over time and attrition, however, did not vary significantly by treatment and control group (see Jonson-Reid et al., 2018) making it unlikely that this significantly biased results.

2.2.1. ECC Sample Description and Data Collection

Because of the interest in attrition within home visitation, only the treatment arm of the study is referenced in the remaining paper. A total of 73 families (or 93% of families referred) assented to the treatment arm of the study, but only 64 families completed the baseline interview and in-person formal consent and began PAT. The loss of families between assent and written inhome consent was primarily due to being unable to locate families who had suddenly changed residence, as is not atypical for this population (Fowler, Henry, Schoeny, Landsverk, Chavira & Taylor, 2013; Pandey & Guo, 2007; Sidebotham, Heron & ALSPAC Study Team, 2006). Because these families were lost prior to completing the in-home consent and baseline interview, no further analysis of the families that assented but did not begin the study could be completed. Families were not compensated for participating in home visitation, but they were compensated for participating in study interviews in the form of a $20 grocery card and children’s books (number of books per interview varied according to the size of the family), provided at each interview. Interviews were conducted at baseline, nine and 18 months. PAT participation was logged by home visitors on tally sheets which were provided to the study team. The average parental age at the start of the ECC was 26.05 years (SD = 6.72). Consistent with the demographics of the CPS and lower income population in the region, close to three-fourths (73.44%) of the caregivers were Black.

2.3. Measures

2.3.1. Dependent variable

The dependent variable for this analysis was the length of stay with PAT measured in months ranging from zero (staying less than 1 month) to 18 (completing the entire study period). Exact dates for enrollment and program leaving were recorded by the parent educators and provided to the study team. The number of visits were also available and are reported. Program participation time was collapsed into three periods: (1) three-month marking early engagement (following Girvin et al., 2007; McGuigan & Gassner, 2016); nine-month marking longer-term retention or the midpoint (following Girvin, et al., 2007; Navaie-Waliser et al, 2000), and program completion at 18-month (stayed until the end of the study). The percentage of families by case characteristics at each stage was constructed so that differences in the proportion of the sample remaining in a given time period could be examined. As a supplement, the time in months until one-third and one half of the sample were lost was also reported.

2.3.2. Predictor variables

Predictor variables were chosen based on prior studies and informed by an ecological framework. Meso-system variables included the PAT worker characteristics, the CPS system contact, and neighborhood characteristics. While relationships between a family and home visitor have been noted as important, there were only three PAT home visitors working with families and satisfaction was so high across workers there was insufficient variability to explore this effect statistically. There was also insufficient variability in PAT staff by race or gender. While there was a turnover with one PAT home visitor, families were enrolled on a rolling basis so this only impacted a few families. We did, however, examine retention by PAT worker as a nominal indicator variable to account for unmeasured variation. For ease of presentation we refer to the worker as one, two or three as there were not enough families solely served by the worker that left to break out further.

Originally ECC was designed to be tested with families at the time of their first contact with CPS, but later linkage to CPS records revealed that several families had reports or a record of an older child placed into foster care prior to the one triggering the referral to ECC. CPS report history” was coded as one if a family had two or more reports prior to the study and zero otherwise. “Foster care history” was coded as one if the family had ever had a child placed in care prior to the study and zero otherwise. Hypothetically retention could have been impacted by CPS in two ways. It is possible that families with prior history would perceive their need to be greater and be more likely to accept services (e.g., McCurdy & Daro, 2001). Hypothetically, it was possible that families with different caseworkers would have different levels of engagement. However, many of the families received no services following the hotline and nearly all of the cases were closed to child welfare within four months of beginning PAT. Further, several CPS workers had only 1 family in the study. This made any CPS worker level effect on retention unlikely and difficult to assess.

Nearly all of the families were very low income. At baseline nearly 75% of families lived in zip codes with between 22% and 53% of families living at or below the poverty line. The material need was not measured until the three-month check-in (at which time all but 17% of the sample either lived in a high poverty neighborhood or reported material needs). Because material need was not available at baseline, living in an area with high levels of family poverty was the only variable tested in the present analysis -- a nominal variable based on the median value (low<26% v. high=26% or more).

Microsystem. Because of the nature of the study population (prenatal CPS referrals did not occur in the study region unless the concern was in regard to an older child), we were unable to control for prenatal as compared to later enrollment. Remaining predictors included maternal demographics (coded as ‘Black’ or ‘White (including Hispanic or Middle Eastern)’ according to self-identification), high school completion, marital status, and maternal age.

Child related measures of need for PAT included developmental concerns and capacity of the parent to support development. Child development as a measure of infant risk (similar to Alfonso-Marsden and colleagues (2013), but not accounting for physical health per se) was measured at baseline using the Ages and Stages Questionnaire (ASQ) (Bricker, 1988; Squires et al, 1999) which was the screening instrument most frequently used by PAT in the region. The scales for communication, fine and gross motor, social-emotional development and personal and social development were dichotomized according to a borderline or clinical score versus not. Having a home environment supportive of child development was measured with selected questions from the Home Observation Measurement of the Environment (Bradley, & Caldwell, 1979) that indicated the presence of activities and stimulation (books, toys) provided by the family. It was coded one for high (scored at or over 85% of supports possible were noted) and zero otherwise.

Maternal risk and protective factors were measured at each interview. Only baseline and nine-month measures are included in this paper given the interest in predicting attrition according to early, mid and final stages. Mental health indicators included depression, isolation, and stress. “Maternal depression” was measured with the Center for Epidemiology Studies Depression Scale (Radloff, 1977) and coded as one when the score was at or above the clinical cutoff (scored 16) and zero otherwise. “Maternal stress” was measured with the Parenting Stress Index (Abidin, 1990) and coded one for high scores (five to 12 stressors) and zero otherwise. “Social contact” was measured by the Maternal Social Support Index (MSSI) (Pascoe, Ialongo, Horn, Reinhart, & Perradatto, 1987) and coded one for more than one social contact per month and zero otherwise. Protective factors included social support. “Formal social support” referred to the number of service systems (e.g., daycare, schooling, and other services for child rearing) around the family based on the Family Support Scale (Dunst, Jenkins & Trivette, 1984). This was coded one if high (3–5 supports) or zero as low (two or fewer services). “Partner communication” was also measured by the MSSI and coded as one for self-reported good quality communication with her partner and zero otherwise. The presence of a partner was defined by the mother and not based on cohabitation.

2.4. Analytic strategy

2.4.1. Bivariate survival analyses

Data were cleaned and coded using SAS 9.4. Bivariate survival analyses were used to identify associations between participants’ characteristics and both initial engagement and long-term retention. The interclass correlation for possible clustering effect by zip code was tested using SAS PROC CANCORR. The resulting correlations were less than r=.09 with a non-significant Wilk’s Lambda of .51 suggesting any clustering effect by zip code was inconsequential (Khattree & Naik, 2000). The interclass correlation for possible clustering effect by PAT worker was tested using SAS PROC CANCORR. The resulting correlations were moderate (r=.31) but significant with a Wilk’s Lamda of .01 suggesting a possible consequential effect (Khattree & Naik, 2000).

2.4.2. Kaplan-Meier estimate of the survival functions

The Kaplan-Meier method was used to estimate survival functions (i.e., the percentage of participants staying in the program) of PAT participants by the predictor variables (Kaplan & Meier, 1958). Due to the small sample size, confidence intervals for survival functions were estimated by Greenwood's method (Greenwood, 1926) using the log-log transformation of the survival function (Hosmer, Lemeshow & May, 2008). To provide more informative estimates with regard to initial engagement (three months), long-term retention (nine months), and program completion (18 months) attrition rates (i.e., the percentage of participants retained) are presented by caregiver and family characteristics. Significant tests for the bivariate survival functions used the log-rank test and the Peto-Peto modification of the Gehan-Wilcoxon test (Peto & Peto, 1972). With the exception of the control variable for prior CPS reports (because like foster care history it was a measure of prior child welfare involvement), only characteristics with significant (p<.05) or near significant tests of equality of strata over time (.05<p<.10) not necessarily at specific intervals. Only these variables are presented in tables and figures. We chose to report factors that trended toward significance given the exploratory nature of the analyses of retention with this population and the desire to support future research. In other words, given the small sample size, it may be that variables trending toward significance have practical value for future research with larger samples.

2.4.3. Joint effects and changes over time

In order to test the joint effects and changes over time, an exact logistic regression (Allison, 2012) was applied for small samples to examine the relative contribution of factors that appeared significant predictors of retention. PAT worker was used as a stratifying variable for this analysis due to the significant ICC. Finally, the sample size limitation precluded testing of interaction terms in multivariate analyses or the potential for changes in parent risk scores or CPS contact over time. Instead, post-hoc analyses using Chi-square/Fisher Exact and Cochran-Mantel-Haenszel Chi-square tests with standardized residuals (Agresti, 2007) were used to further explore changes in risk between baseline and nine months and caregiver characteristics that predicted program completion (retained for 18 months). Standardized residuals over +/−1.96 indicate significant difference for individual cells compared to marginal proportions.

3. Results

Less than 20% of the sample had a child that scored in the borderline or clinical range for a developmental concern across subscales and no new concerns emerged over time. Nearly 38% of the caregivers were in the clinical range for depression at baseline and about 55% of them reported a large (over five) number of life stressors (e.g., death in the family, new child, illness, loss of work, lost relationship…) in the past 12 months. Among participants, most (62.50%) of the caregivers reported having high levels of formal social support. About half reported regular informal social support contacts. Over 75% reported having good communication with a partner.

On average, families stayed in the program for 13.85 months (SD= 7.57), but only 48.44% stayed with PAT for the full 18 months. Only about 14% left PAT by 3 months with an average of 1.8 visits (SD=1.9). Among those that left at midpoint, another 11%, had an average of 5.6 visits (SD 3.9). Among families that stayed at least nine months, the average number of visits was 17 (SD=10.9). As can be seen from the figures, there was significant variability in frequency and generally fairly low dosage of visits. The number of appointments made as compared to appointments kept was also explored and followed a pattern of increased length of participation associated with increased percentage of appointments kept (27% among those leaving at 3 months, 58% among those who stayed at least 9 months, and 71% among families who completed 18 months). Because visit data were collected every three months according to tally sheets it was not possible to accurately time order visits to look at increasing or decreasing frequency. There was no significant or near significant association between child development measures, number of life stressors, or maternal age and time retained in the program and all caregivers were female.

3.1. Factors affecting dropout rates

The survival function is expressed in percent still enrolled by predictors. For example, 89.4% of Black caregivers were retained in the first three months, 85.1% of Black caregivers remained by nine months, and 55.3 % of Black caregivers were retained at 18 months. Table 1 presents the data from Figures 1–3 in tabular form and illustrates the demographic, risk and social support variables that showed significant or near significant differences for the at least one-time period.

Table 1.

Survival Analyses Results by Baseline Family Characteristics at 3, 9, and 18 months with Adjusted Confidence Intervals

		Survival Function (95% CI)
Name	n (%)	3-Month	9-Month	18-Month
Race (p=.009)
White	17 (27%)	0.76 (0.49, 0.90)	0.47 (0.23, 0.68)	0.29 (0.11, 0.51)
Black	47 (73%)	0.89 (0.76, 0.95)	0.85 (0.71, 0.93)*	0.55 (0.40, 0.68)
High % Family Poverty in Zip (p=.02)
Yes	30 (46%)	0.93 (0.76,0.98)	0.90 (0.72, 0.97)	0.63 (0.44, 0.78)
No	34 (54%)	0.82 (0.65,0.92)	0.62 (0.43, 0.76)	0.35 (0.20, 0.51)
Foster care history1(p=.08)
Yes	15 (23%)	0.80 (0.50, 0.93)	0.67 (0.37, 0.85)	0.27 (0.08, 0.50)
No	49 (77%)	0.90 (0.77, 0.96)	0.77 (0.63, 0.87)	0.55 (0.40, 0.68)
Multiple report history2
Yes	38 (59%)	0.87 (0.56, 0.96)	0.80 (0.50, 0.93)	0.47 (0.21, 0.69)
No	26 (41%)	0.88 (0.75, 0.94)	0.73 (0.59, 0.84)	0.49 (0.34, 0.62)
Home environment3(p=.03)
High	38 (59%)	0.84 (0.68, 0.92)	0.63 (0.46, 0.76)	0.42 (0.26, 0.57)
Low	26 (41%)	0.92 (0.73, 0.98)	0.92 (0.73, 0.98)	0.58 (0.37, 0.74)
Maternal depression4(p=.09)
High	24 (38%)	0.90 (0.74, 0.96)	0.87 (0.71, 0.94)	0.53 (0.36, 0.67)
Low	40 (62%)	0.85 (0.64, 0.93)	0.58 (0.37, 0.74)	0.42 (0.23, 0.60)
Formal social support6(p=.02)
High	40 (62%)	0.85 (0.70, 0.93)	0.80 (0.64, 0.89)	0.62 (0.46, 0.75)*
Low	24 (38%)	0.92 (0.71, 0.98)	0.67 (0.44, 0.82)	0.25 (0.10, 0.43)
Social contact7(p=.02)
High	32 (50%)	0.94 (0.77, 0.98)	0.84 (0.66, 0.93)	0.62 (0.43, 0.77)
Low	32 (50%)	0.81 (0.63, 0.91)	0.66 (0.47, 0.79)	0.34 (0.19, 0.51)
Partner communication8(p=.09)
Good	48 (76%)	0.89 (0.77, 0.95)	0.79 (0.65, 0.88)	0.54 (0.39, 0.67)
Poor	15 (24%)	0.80 (0.50, 0.93)	0.60 (0.32, 0.80)	0.33 (0.12, 0.56)
PAT worker (p=.01)
1	25 (41%)	0.88 (0.67, 0.95)	0.76 (0.54, 0.88)	0.68 (0.46, 0.82)
2	16 (23%)	0.87 (0.58, 0.97)	0.62 (0.35, 0.81)	0.31 (0.11, 0.54)
3	23 (36%)	0.87 (0.65, 0.95)	0.83 (0.60, 0.93)	0.39 (0.20, 0.58)
Total	64 (100%)	0.86 (0.75, 0.92)	0.75 (0.63, 0.84)	0.48 (0.36, 0.60)

Note.

^*p<=.05;

¹Foster care history was coded as 1 if the family had ever had a child placed in foster care and 0 otherwise.

²CPS report history was coded as 1 if a family had two or more prior reports and 0 otherwise.

³Home environments was measured with selected questions that indicated the presence of activities and stimulation (books, toys) provided by the family. It was coded 1 for high (scored at or over 0.85 of supports noted) and 0 otherwise.

⁴Maternal depression was coded as 1 when the score was at or above the clinical cutoff (scored 16) and 0 otherwise.

⁵The total numbers of stressors recoded as 1 for high scores (scored 5 to 12) and 0 otherwise.

⁶Social support referred to the amount of service systems (e.g., daycare, schooling, and other services for child rearing) around the family. It was coded as 1 as high (3–5 supports) or 0 for low (2 or less services).

⁷Social contact was coded as 1 for more than one social contact per month and 0 otherwise.

⁸Partner communication was coded as 1 for good quality of mother’s communication with her partner (was not required to live with the mother).

Fig. 1.

The percentage (%) of participants retained by the end of 3 months.

Fig. 3.

The percentage (%) of participants retained by the end of 18 months.

3.2. Caregiver and CPS history factors and early engagement (at least 3 months)

Table 1, column 4, and Figure 1 illustrate how many families remained at the three-month stage. Attrition at this stage was very low, only five families left before 3 months and none of the predictors were significant.

3.3. Caregiver and CPS history factors and longer term retention (at least 9 months)

Table 1, Column 4 and Figure 2 illustrate how many families remained at nine-months. White caregivers were significantly more likely to exit PAT services by nine months. Less than half (47.1%) of the White caregivers remained in PAT for at least nine months compared to over 85% of Black caregivers. As shown in Figure 4a, this difference became statistically significant around 8 months and remained so (with some brief overlap in confidence interval about 11 months) until nearly 15 months (Log-rank test: chisq = 6.6, df = 1, p = 0.010 and Peto-Peto modification of the Gehan-Wilcoxon test: chisq = 7.9, df = 1, p = 0.005). While not statistically significant by time period, there was a trend toward greater retention rates among caregivers with depressive symptoms in the clinical range (87% compared to 58%), and lower scores for environment supportive of child development (HOME) (92% v. 63%). Also, not statistically significant by time period, caregivers remaining with PAT reported more formal (80% v 67%) or informal social contacts in their lives (84% v. 66%) at baseline.

Fig. 2.

The percentage (%) of participants retained by the end of 9 months.

Fig. 4.

a. Survival function of Parents as Teachers (PAT) participants by race. The shaded areas are 95% confidence intervals. b. Survival function of Parents as Teachers (PAT) participants by social support. The shaded areas are 95% confidence intervals.

3.4. Caregiver and CPS history factors and program completion (at least 18 months)

Figure 3 illustrates the participation rates by the end of the study. The difference between having high (higher retention) or low formal supports became a significant predictor only for program completion. At this stage, only baseline formal social support discriminated between retention at 18 months or not. Caregivers reporting at least 3 formal supports at baseline were more likely to remain at 18 months (62% v 25%). Although not statistically significant due to wide confidence intervals, the retention rates trended toward significance for Black caregivers (55% v 29%), families living in higher poverty zip codes (63% v 35%), families with no prior record of a child in foster care (55% v 27%), and those with higher numbers of social contacts per month (62% v 34%).

3.5. Participant/PAT worker relationship and retention

There were no statistically significant differences in caregiver characteristics and assignment to workers as families were assigned due to space available on a rolling basis. There were no differences in 3-month engagement rates by PAT worker. At the nine-month stage, there was a non-significant trend toward higher retention for families served by Worker 3, but by 18 months there were more families retained from the caseload of Worker 1. None of the differences were statistically significant. The average number of visits per PAT home visitor was 11.6 (SD=8.5), 13.8 (SD=7.2) and 16.6 (SD=15) respectively. These differences were not significant.

3.6. Post-hoc analyses: program completion by race and social support

To further explore significant race and social support effects, an exact logistic regression (Allison, 2012) for small samples was used while stratifying by PAT worker due to the significant interclass correlation. Black caregivers were 8.6 times (Exact Odds Ratio) more likely to remain in the program until the end of the study (CI 1.89, 49.39; p=.003), controlling for formal social support. Formal social support was not significant nor was an interaction between the two variables significant.

3.7. Post-hoc analyses: caregiver characteristics and change in risk over time

The present sample was too small to test possible effects related to changes in caregiver risk or protective factors over time. Given the importance of ethnic/racial categorization, baseline depression, social contact and social support, additional post-hoc descriptions of the mothers within these groups were completed.

3.7.1. Caregiver risk characteristics by racial category

Because Black women were more likely to be retained overall (significant at the nine-month period), a post-hoc analysis of baseline characteristics was conducted. There were no statistically significant differences in demographics or baseline measures of risk between Black and White caregivers with the exception of a near significant different in self-reported positive partner communication. Black women were more likely to report good communication with a partner (Fisher=.09). Though not statistically significant, a lower proportion of Black women endorsed having 3 or more formal social supports at baseline line (59.6 v 70.6%) or having at least 3 people they could call on in a time of need (48.9 v. 68.7%). Black women were more likely to live in zip codes with more than 26% of families at the poverty line (51.1 v 35.3%) but this was also not statistically significant. PAT worker “one” was initially a White female, but after leaving the study for another position was replaced by a Black female. The percentage of Black families at baseline was quite similar among the “three” workers (72%, 81%, and 70%), and there was no significant difference among Black families staying until the study ends by worker (84%, 90%, and 79% respectively).

3.7.2. Change over time

The selection of 3, 9 and 18-month intervals for survival analyses was made to correspond to the prior available literature and the length of the present program. Another way to view retention over time is to look at how many months it takes to lose a given proportion of the sample group. Table 2 shows the number of months it took until a third or 50% of a given sample group was lost. As shown in Column 2, most subgroups with the exception of White families remained through month 8. In our study, several subgroups never reached 50% attrition and those that did often remained for at least a year.

Table 2.

Number of Months to 33% and 50% Attrition by Case Characteristics.

Percent Lost to Attrition	33%	50%
Race Category
Black	16	NA
White	5	8
High % Family Poverty in Zip
Yes	17	NA
No	8	15
Foster care history
Yes	12	14
No	14	NA
CPS report history
Yes	14	17
No	10	16
Home environment
High	8	14
Low	16	NA
Maternal depression
High	8	16
Low	16	NA
Social support
High	14	NA
Low	10	15
Social contact
High	8	14
Low	16	NA
Partner communication
Good	16	NA
Poor	8	12
PAT worker
1	NA	NA
2	15	16
3	14	16

Note: NA =Not applicable because that attrition threshold was never reached for that subgroup.

Hypothetically, changes between baseline and nine-month values on maternal risk and protective variables such as new contacts with CPS, change in caregiver or depression levels, or social supports could impact program completion (retained through 18 months). There was no change in the mean or a median number of social supports reported between baseline and nine months. The values for depression and stress did change (both declined) though the difference between timepoints was not statistically significant. The change in caregiver depression, while not statistically significant, did indicate a shift from above or below the clinical cutoff and is further explored. Because some caregivers had CPS contact following study start this was also further examined.

A depression change variable was created indicating whether a caregiver: (A) remained in the non-clinical depression symptom range at both points (27.4%), (B) changed from non-clinical to clinical (9.8%), (C) changed from clinical to non-clinical (41.2%), or (D) remained in the clinical range for depression (21.6%). The CMH chi-square Fisher Exact was near significant (p=.07) so standardized residuals were further explored to see which cells had more powerful deviations from expected counts. A higher proportion of women in the non-clinical group at both time points (group A) had left PAT by nine months compared to the overall marginal proportion (28.6% v 9.8%; STD residual=2.77) and a lower proportion remained until the close of the study (57.1% v. 82.3%; STD residual=−2.90) compared to the overall marginal proportion. It should be noted that 27% of women who scored in the non-clinical range and 16% of those that scored in the clinical range at baseline discontinued PAT by nine months and could not be located for the nine-month interview.

A similar coding was completed for CPS contact prior to as compared to during the study period grouping families according to: (A) only a single hotline immediately prior to the study and no CPS hotline during the study (48.4%), (B) multiple CPS reports prior to the study but no report during (10.9%), (C) a single CPS report prior but additional CPS contact during the study (28.1%), and (D) both multiple prior reports and reports during the study (12.5%). The overall CMH chi-square (exact) was not significant (p=.21). However, there was an interesting trend according to the level of CPS contact. Fewer women in the low CPS contact (group A) remained in the study until completion compared to the overall marginal proportion (61.3% v. 75%). Women in CPS contact group C were more likely to be among those remaining in the program until completion than the overall marginal completion rate (94.4% v. 75%). Three families among CPS group C had at least one child enter foster care for a brief period and then return home during the study. All of these families remained with PAT for the entire study period.

4. Discussion

Overall, ECC successfully engaged 86% of participating families for at least three months and retained 75% at least nine months. These rates are consistent with those reported in a recent national evaluation of five different home visiting models that included PAT (Boller et al., 2014). There is a reason to believe that families with CPS contact have significant needs in regard to life stressors as well as parenting (e.g., Fowler et al., 2013; Jonson-Reid, Drake & Kohl, 2009: Putnam-Hornstein & Needell, 2011). Despite the fact that PAT is not designed specifically for this population, our final program completion rate of 48.4% was similar to the 49.5% completion rate found in the study of SafeCare designed for families with prior child welfare referrals (Damashek, et al., 2011). SafeCare, however, is a much shorter program. Our findings, if replicated, suggest that retaining child welfare involved families in longer-term community-based home visitation is possible if barriers to initial enrollment and continued participation (e.g., terminating families for missed appointments) are removed.

Consistent with some of the scant prior work that looked at attrition at various time points (Ammerman et al, 2006; Brand & Jungman, 2014; Damashek, et al., 2011; Daro et al, 2003; Girvin, et al., 2007), there were differences, albeit few statistically significant, in factors associated with midpoint or program completion. The engagement through three months, however, was high across all groups with participation by demographic, risk and protective factors ranging from 76 to 94%. This again is consistent with a recent national evaluation suggesting similarities across populations in regards to remaining beyond three months once beginning services both for community-based home visiting and for programs like SafeCare which are designed to target child welfare involved families (Boller et al., 2014). Given the variation in how program participation has been measured, more work is needed to explore the best means of measuring participation and drop out that can best inform ongoing engagement strategies.

Race category and attrition

Consistent with three prior studies (Daro et al.,2003; McCurdy et al., 2003; and Navaie-Waliser et al., 2000), mothers who identified as Black were more likely to be retained. It is unclear whether this association reflects unmeasured differences in risk (e.g. Kim & Drake, 2018, Slopen et.al., 2016). Although Black caregivers were less like to report formal social supports and more likely to live in higher poverty neighborhoods at baseline, there were no racial/ethnic differences in mental health and stressor risk measures. It is possible that the greater contextual risks increased the perception among Black caregivers that they needed the service-consistent with the McCurdy and Daro theory. While Alonso-Marsden and colleagues (2013) found that greater contextual risk reduced follow-through on home visits for Black caregivers, their focus was on beginning program participation not retention after the engagement. On the other hand, it is possible that White families in the present study had significant barriers to participation that were unmeasured. If so, the lower retention would support the idea that families with greater numbers of problems are more likely to leave programs early (Damashek et al., 2011; McGuigan & Gassner, 2016; Navaie-Waliser et al., 2000). Of course, to our knowledge, no studies of retention in “usual care” community-based early childhood home visiting with a child welfare-involved population exist.

Although not statistically significant, there was a trend toward longer retention among caregivers with higher levels of depression symptoms, and/or living in zip codes with higher numbers of families in poverty, more contact with CPS, and/or having lower resources to support cognitive development of their children at baseline. This trend is again consistent with the ideas of McCurdy and Daro (2001) regarding greater retention for mothers with higher perceived need. This is also consistent with at least two other studies that found that mothers with higher levels of mental health and/ or social isolation stayed longer (e.g., Ammerman et al., 2006; Girvin et al., 2007). In some prior work, home visitation alone was found to moderate maternal depression (Landsverk et al., 2002) but this was not true for other studies (Ammerman et al., 2009; Duggan et al., 2004; Duggan et al., 2007; Mitchell-Herxfeld et al., 2005). While not statistically significant, slightly over 41% of women in our study reported baseline symptoms of depression in the clinical range but were not in the clinical range at nine months while another 21% remained in the clinical range at both timepoints. In the present study, only the absence of maternal depression across timepoints (albeit there was some loss to follow-up at 9 months) appeared associated with retention; less depressed mother were more likely to leave early. Again this is consistent with higher needs mothers remaining in home visitation longer (e.g., Ammerman et al., 2006; Girvin et al., 2007).

The trend toward greater retention of mothers with higher levels of informal and formal social support was consistent with one review and an early study (Navaier-Waliser et al, 2000; McGuigan & Gassner, 2016). Initially this seemed counterintuitive in relation to McCurdy and Daro’s (2001) ideas regarding risk and retention. Social isolation is noted as a risk factor for maltreatment (Berlin, Appleyard, & Dodge, 2011; MacMillan et al., 2009). On the other hand, it may be that some of the skills that provide the foundation for connecting to informal supports also increase the likelihood of engagement with the home visitors. This may be consistent with Duggan and colleagues (1999) who suggested that early caregiver enthusiasm interacted with agency behavior to produce retention.

4.1. Strengths and limitations

While contributing to a better understanding of the dynamics of attrition in community-based early childhood home visitation, there were significant limitations to the present study. First and foremost, the small sample size limited the power to detect statistical significance and precluded the ability to conduct theoretically informed multivariate survival analyses. Second, time enrolled in a program may not be the same as program content delivery due to the possible variation in the number of visits for each family in a given time period. Nonetheless, several findings in regards to factors associated with attrition were consistent with prior work as discussed above. Examination of racial differences (although not statistically significant) in some meso system baseline measures suggested that Black women faced more contextual challenges. It is possible that these factors might have mediated or moderated retention results with a larger sample. Future work should explore these relationships. Further because of the demographics of the region, it was not possible to assess racial/ethnic differences across other groups.

Due to concerns about record keeping burden for the PAT workers, visits and appointments made were recorded by tallies rather than dates, making it difficult to assess time ordering. Further the flexibility in content delivery that is part of the PAT model, made it difficult to compare what a given family received according to dose. Home visiting programs in which visits are logged into administrative data by date and content delivered may offer an opportunity to study this more carefully. Additionally, PAT like many home visiting programs also provides referrals and assistance accessing other resources. Because other needs and services received were not measured until later time points in the study it was not possible to account for the time ordering of referrals to resources that may have been provided by home visitors. Nor was data available in regard to which other services families accessed as a result of these referrals. To our knowledge, outside references in qualitative work (e.g., Beasley et al., 2017), this is poorly researched area. More work is needed to understand how home visitation may link families to additional services and to what extent this may impact retention separate from specific program content.

While not just a limitation of this study per se, there is significant variation across home visitation programs in terms of eligibility criteria as well as the use of para-professionals compared to nurse or other professionals (Dodge, et al., 2014; Duggan et al., 2007; Filene, Kaminski, Valle, & Cachat, 2013; Gomby, 2007; Raikes, et al., 2014). Our study was limited to a paraprofessional program and offered to a particular population, families with contact with child protection. Much more research is needed to know if engagement and retention factors vary across program models and populations. Some qualitative work suggests that caregivers who perceive getting tailored support for their needs are more likely to be retained in home visitation (Beasley et al., 2017). Parents as Teachers does by design encourage flexible delivery of curricula as well as referrals to outside resources (Parents as Teachers, n.d.). Additionally, participants in the program received children’s books along with the compensation for interviews. ECC PAT workers were also supported with updated lists of community resources and free or low-cost family recreational activities. Further, typical program practices of closing cases for families that missed a set number of appointments was greatly relaxed allowing some families to remain in contact with their PAT worker that would normally have been terminated. It is unclear if continued participation was enhanced by the direct link between family need and the program components or the flexibility in enrollment criteria.

4.2. Implications

This study attempted to further disentangle early engagement and retention to help shed light on factors that can be targeted to improve program completion. While a small sample, our study adds to the emerging evidence that the import of factors related to engagement change over time. In other words, what may work to attract a family to enroll in home visitation may be different than what is needed for longer term retention. Olds (2003) and Duggan and colleagues (2000) both emphasized careful program monitoring to detect when attrition begins and employ retention strategies. Applying survival analysis such as Kaplan-Meier (KM) estimate and the survival function in this study may be a useful approach for other long-term home visitation programs to examine the dynamics of attrition at various key stages in a program. The displays of how these factors develop or lose importance over time (refer back to Figure 4a may be of particular use to program evaluation and research-especially with larger samples where results may be more stable). Because programs may vary in length and content, the best means of identifying stages of the retention warrant further study. Future work is needed to look at qualitative indicators of engagement and their correspondence to specific time periods of program participation as well as looking at how engagement, retention, and completion are associated with program outcome.

Engaging and retaining child welfare involved parenting programs is typically thought of as a difficult task (Chaffin et al., 2009). While it makes intuitive sense that greater risk factors may impede participation in programs like home visitation, our work is consistent with prior studies that find a mixed and complex pattern between risk and protective factors and retention among child welfare involved parents. There are several potential reasons for the seemingly counterintuitive rates of involvement. First, ECC was offered to the parents and arrangements made to visit with a PAT home visitor without proactive enrollment and all engagement occurred within the home. This process may have ameliorated some of the barriers to engagement experienced by mothers living in low resource areas (e.g., Alonso-Marsden et al., 2013). More research is needed in understanding how child welfare-involved families participate in existing community-based services. Second studies of engaging child welfare involved families in parenting programs (not necessarily home visiting) often involve a cross-section of families that may have varying numbers of prior contacts and service episodes with child welfare and serve families with older children (e.g., Chaffin et al, 2009; Damashek et al., 2011). ECC was designed to assist in connecting families at their earliest known contact with CPS; the majority of the families in the study had only one CPS report prior to enrollment. While there was a trend toward families with more CPS contacts staying longer in our study, most of the ECC families stayed longer than the total length of the programs focused on child welfare populations (Chaffin et al, 2012; Damashek et al., 2011). More work is needed to explore whether or not retention is higher by moving connections to preventive services to earlier stages of child welfare involvement. It is also possible that participation in community-based home visitation is somehow less stigmatizing than participating in programming that is directly tied to their child welfare involvement, but there is insufficient research to support or reject this hypothesis.

Beyond simple retention in a program, more work is needed to understand how participation and dose relate to outcomes. Although the national evaluation found relatively high overall program retention rates at six months, the dosage across programs was not greater than about two visits per month regardless of program model (Boller et al., 2014). If program models require longer-term participation to achieve positive benefits, then more work is needed to elucidate the mechanisms involved in retaining families. It is also important, however, to link the study of engagement and attrition to program outcomes so we can understand what dosage is needed for whom to achieve what benefit (Azzi-Lessing, 2011). It may be that some populations benefit sufficiently from lower program dosages and could be considered ‘completers’ as compared to drop outs. Likewise, it is not clear that longer participation is always associated with more positive outcomes. If need predicts retention but a program is unable to ameliorate that need, then longer-term retention may actually be associated with poor outcomes. Only as it becomes clearer what factors are associated with what level of participation and how that impacts outcomes will we be able to match target strategies to engage various subgroups of families to participate long enough to benefit from services.

Home visitation is viewed as one of the most promising platforms for intervening early in the life of a child to promote safety and positive development. Very little work, however, has been done to understand the use of home visitation by certain population such as CPS-involved families. Even fewer published studies specifically to our knowledge, only a few studies explore attrition in home visitation with this population (Damashek, et al., 2011; Girvin et al, 2007). While primary prevention of maltreatment is clearly preferable, exploring ways of providing preventive programming to the more than millions of families (U.S. Department of Health & Human Services, 2017) who are already referred to child protection is a moral and cost imperative. Given the emphasis on expanding early childhood home visitation services, it is critical that we understand more about the participation in and outcomes of such services with families contacting child protective services.

Highlights.

• One of the few studies that examined the participation in a community-based home visitation program by child welfare-involved families

• White caregivers tended to leave in the middle of the program (nine-month) stage

• Caregivers who had better social support were more likely to stay in the end of the program

• Given the very small sample size, both statistically significant and near significant trends are discussed in the context of existing literature

• Practical variation found has implications to build knowledge of attrition in home visitation programs