Addressing Maternal Depression, Substance Use, and Intimate Partner Violence in Home Visiting: A Quasi-Experimental Pilot Test of a Screen-and-Refer Approach

Abstract

This quasi-experimental pilot study describes preliminary impacts of the “Home Visitation Enhancing Linkages Project (HELP),” a pragmatic screen-and-refer approach for promoting identification of and linkage to treatment for maternal depression (MD), substance use (SU), and intimate partner violence (IPV) within early childhood home visiting. HELP includes screening for MD, SU, and IPV followed by a menu of motivational interviewing and case management interventions aimed at linking clients to treatment, designed for delivery within routine home visiting. HELP was piloted within four counties of a statewide home visiting system that were implementing Healthy Families America. HELP clients (N = 394) were compared to clients in five demographically matched counties that received usual Healthy Families services (N = 771) on whether their home visitors (1) identified MD, SU, and IPV risk; (2) discussed MD, SU, and IPV during home visits; and (3) made referrals for MD, SU, and IPV. All data were extracted from the program’s Management Information System. A significant impact of HELP was found on discussion of risk in home visits for all three risk domains with large effect sizes (MD OR = 4.08; SU OR = 15.94; IPV OR = 9.35). HELP had no impact on risk identification and minimal impact on referral. Findings provide preliminary support for HELP as a way of improving discussion of client behavioral health risks during home visits, an important first step towards better meeting these needs within home visiting. However, more intensive intervention is likely needed to impact risk identification and referral outcomes.

Early childhood home visiting (HV), a strategy for delivering voluntary preventive services to optimize parent and child outcomes across the life course, is the primary supportive intervention offered to high-risk families in the perinatal period. HV capacity was significantly expanded under the Affordable Care Act, with more than 160,000 of the nation’s highest risk families served by the federal HV program in 2016 (Health Resources and Services Administration, 2016), and recent federal re-investment supports its continued growth. HV has shown favorable impacts on a host of maternal and child outcomes, including parenting skills and parent-child interactions, and long-term impacts on child development, school readiness and performance, and family economic self-sufficiency, though effects are generally small (Michalopoulos et al., 2019). HV impacts on child maltreatment prevention, a primary goal of many HV models, is mixed, with some studies finding lower maltreatment rates among families in HV (Chaiyachati, Gaither, Hughes, Foley-Schain, & Leventhal, 2018), and others finding no impacts on maltreatment (Duggan et al., 2007).

A potential contributor to the lack of consistent HV impact on child maltreatment is program capacity to address maternal behavioral health risks that are linked to child maltreatment, such as maternal depression (MD), substance use (SU), and intimate partner violence (IPV) (Teti, Cole, Cabrera, Goodman, & McLoyd, 2017). Families with these risk factors are prevalent in HV; According to recent national data, 34% of HV clients reported clinically significant depression symptoms, 39% reported binge drinking or illicit drug use, and 17% reported experiencing IPV prior to HV enrollment (Michalopoulos et al., 2015). Families with more complex risk profiles may be more difficult to engage in and less likely to benefit from HV (Beasley et al., 2018; Folger et al., 2015), though engagement is a complex process that is impacted by provider, program, and community factors in addition to client characteristics (Latimore et al., 2017). Tailoring HV services to clients’ needs, including those related to behavioral health, has been suggested as a way to improve engagement and outcomes (Supplee, Parekh, & Johnson, 2018).

Historically, HV programs have demonstrated low rates of identification of MD, SU, and IPV, and, even when identified, were often not able to link clients with treatment (Duggan et al., 2004; Tandon, Parillo, Jenkins, & Duggan, 2005). In a recent study by the authors, home visitors in one state reported infrequently addressing client MD and SU, and reported less self-efficacy and more client-level barriers related to SU compared to MD (Dauber, Ferayorni, et al., 2017). As HV continues to expand nationally, equipping home visitors with tools for addressing maternal behavioral health risks may improve engagement and outcomes for the highest-risk families.

In recognition of this potential, federally funded HV programs are now mandated to screen clients for depression and IPV (HRSA Maternal and Child Health, 2016). There is also a growing body of literature on enhancements to routine HV that target maternal behavioral health. Most of this research has focused on depression, and supports the delivery of cognitive behavioral therapy by trained mental health clinicians during home visits for reducing depression symptoms in HV clients (Ammerman et al., 2013). A group behavioral intervention for depression delivered by clinicians as an adjunct to HV has also demonstrated success in reducing depressive symptoms and preventing new onset of major depression (McFarlane, Burrell, Duggan, & Tandon, 2017). This intervention has recently been adapted for delivery by home visitors, and initial feasibility and outcomes are favorable (Tandon, Ward, Hamil, Jimenez, & Carter, 2018). Additionally, a home visitor delivered structured empowerment intervention for IPV demonstrated reductions in IPV reports over 24 months (Sharps et al., 2016). These efforts are limited by (1) their focus on a single behavioral health risk (depression or IPV), despite high rates of co-occurrence of multiple behavioral health risks in perinatal women; and (2) the need for mental health clinicians to deliver the interventions (except for (Sharps et al., 2016; Tandon, Leis, et al., 2018), an extramural resource that may be out of reach for many local HV programs.

There is a need for complementary pragmatic approaches that can be feasibly integrated into routine HV and are sustainable within the existing resources of state HV systems. Screen-and-refer models, which include screening followed by brief motivational interventions aimed at linking clients with external treatments, represent a potential pragmatic approach for simultaneously addressing multiple behavioral health risks within HV. Home visitors routinely implement standardized developmental screenings and make referrals to health and early intervention services, supporting the feasibility of extending existing screen-and-refer procedures to encompass maternal behavioral health. To test this approach, we partnered with administrators, program staff, and HV training and technical assistance providers in one state to develop and pilot test the “Home Visitation Enhancing Linkages Project (HELP)” within four counties of a statewide HV system that was implementing Healthy Families America. HELP was designed to be maximally pragmatic, and thus able to be implemented within routine HV services, by home visitors, using resources that are typically available within the HV setting (Loudon et al., 2015).

The current quasi-experimental pilot study examined the impact of the HELP intervention on three pragmatic outcomes that are routinely tracked by home visitors in the course of HV service delivery: (1) identification of SU, MD, and IPV risk in HV clients; (2) discussion of SU, MD, and IPV during home visits; and (3) referrals to treatment for SU, MD, and IPV provided by home visitors. We hypothesized that HV clients in the HELP condition would be more likely than a statistically matched comparison group of HV clients who did not receive HELP to (1) have their SU, MD, and IPV risks identified by their home visitor, (2) have these risks discussed in home visits, and (3) receive referrals from their home visitor in these domains. In addition, to inform future efforts to tailor HV services more specifically to client needs, we examined the impact of home visitor and client baseline characteristics on study outcomes as an exploratory analysis.

Method

Study Design and Procedures

HELP was piloted within the Healthy Families America program in four counties of a statewide HV system that were selected due to their strong leadership structure; mix of urban and less densely populated areas; and suspected high rate of target risk domains. The study used a quasi-experimental intent-to-treat design, whereby clients in the four HELP counties were compared to Healthy Families clients in five counties in the same state that did not receive HELP, applying weighted propensity scores. To select the comparison counties, demographic characteristics of Healthy Families clients in each of the 17 counties that did not implement HELP were compared to those of Healthy Families clients aggregated across the four HELP counties. The five counties that represented the closest demographic match to the HELP counties were selected for the comparison sample. Due to high rates of missing data (15–25%) on client race and ethnicity, we selected counties for which the racial/ethnic distribution of clients fell within 10% of the average distribution of the HELP counties to attempt to balance groups on these characteristics prior to propensity score modeling. The HELP counties received Healthy Families plus HELP; the comparison counties received Healthy Families only. Propensity weights were applied in all analyses to achieve balance between study groups, as described below. All study data were culled from the Healthy Families Management Information System.

Study investigators recruited home visitors in the HELP counties and obtained informed consent following completion of HELP training. All Healthy Families home visitors in the HELP counties who completed the HELP training requirements were eligible. Clients of participating home visitors who met eligibility criteria (18 years or older, biological mother of target child, and newly enrolling in Healthy Families from September 2013 - May 2015) and had data available in the Management Information System were included in the study. The comparison condition included all home visitors and clients who enrolled in Healthy Families in the five comparison counties during the HELP enrollment period, met HELP eligibility criteria, and had data available. All Healthy Families clients entered into the Management Information System provided informed consent for the use of their data for research. The governing IRBs approved our use of these data.

Participants

Clients

394 HELP clients and 771 non-HELP clients met eligibility criteria and are included in study analyses (see Figure 1 for CONSORT diagram). Due to the larger size of the comparison counties, their HV programs were contracted to serve more families, accounting for the disparity in sample size across conditions. Table 1 presents baseline descriptive statistics on the HELP and non-HELP clients prior to propensity score analysis. HELP clients were older (t (711.93) = −3.46, p < .01), less likely to have a high school diploma or GED (χ²(1) = 13.72, p < .001), more likely to have a college degree (χ² (1) = 21.53, p < .001), and less likely to be identified as having a mental health risk at baseline (χ² (1) = 5.98, p < .05). After limiting the comparison sample to five counties, there were no significant group differences on race or ethnicity.

Figure 1.

CONSORT diagram

Table 1.

Comparison of HELP clients and Non-HELP clients on baseline characteristics prior to (unweighted) and after (weighted) propensity score analysis.

	Unweighted		Weighted
	Non-HELP N (%)	HELP N (%)	Non-HELP N (%)	HELP N (%)
Total N	771	394	762	387
Enrolled in HFA prenatally	223 (29%)	130 (33%)	175 (30%)	177 (31%)
Client age in years (M/SD)**	25.93 (5.34)	27.18 (6.02)	26.35 (.20)	26.37 (.29)
Education
Less than high school	195 (17%)	105 (27%)	149 (26%)	151 (26%)
High school diploma or	326 (43%)	122 (31%)	221 (38%)	220 (38%)
GED***
Some post high school	208 (27%)	114 (29%)	161 (28%)	158 (28%)
education
Associate or college degree***	38 (5%)	49 (13%)	44 (8%)	44 (8%)
Currently employed	132 (17%)	78 (20%)	101 (18%)	108 (19%)
Marital status single	527 (46%)	247 (64%)	383 (67%)	376 (66%)
Primary language is English	575 (76%)	292 (76%)	424 (75%)	444 (78%)
Hispanic ethnicity	293 (47%)	154 (46%)	217 (47%)	215 (45%)
Race
White	260 (40%)	135 (42%)	196 (40%)	201 (42%)
Black	265 (40%)	120 (37%)	193 (39%)	179 (37%)
Multiracial	42 (6%)	21 (7%)	31 (6%)	33 (7%)
Other race	90 (14%)	49 (15%)	72 (15%)	68 (15%)
Uninsured	58 (8%)	23 (6%)	45 (8%)	28 (5%)
Has Medicaid	628 (83%)	327 (85%)	465 (82%)	489 (87%)
Receiving any public benefits	709 (93%)	346 (90%)	527 (92%)	513 (90%)
Receiving TANF	305 (40%)	162 (41%)	220 (38%)	249 (43%)
Child welfare system involved	101 (13%)	53 (14%)	72 (13%)	82 (14%)
Target child only child under 5	512 (67%)	248 (63%)	385 (67%)	369 (64%)
years
Received prenatal care	731 (97%)	377 (98%)	546 (97%)	551 (98%)
SU risk identified at baseline	87 (11%)	42 (11%)	62 (11%)	66 (11%)
MH risk identified at baseline*	245 (32%)	98 (25%)	169 (29%)	167 (29%)
IPV risk identified at baseline	79 (10%)	47 (12%)	57 (10%)	76 (13%)
Number of home visits completeda,***	22.07 (.69)	28.97 (1.00)	22.43 (.70)	29.11(100)

Note.

^*p<.05

^**p<.01

^***p<.001.

Significant differences refer to unweighted comparisons.Total N’s prior to weighting range from 1136 to 1165 due to missing data; N’s for Hispanic ethnicity and race are 955 and 982 respectively. After weighting, total N’s range from 1123 to 1149 due to missing data, with total N’s of 945 for Hispanic ethnicity and 972 for race. Propensity score weights were not computed for 16 cases due to missing data on variables included in the propensity model.

^aNumber of home visits was not included in the propensity score model. It is presented in the table for descriptive purposes.

Propensity score analysis, a set of techniques used to correct for selection bias in quasi-experimental studies, was conducted following the procedures of Guo and Fraser (Guo & Fraser, 2010). First, univariate logistic regressions determined significant predictors of group membership (HELP vs. non-HELP), with p < .15 used as a conservative threshold for significance (Guo & Fraser, 2010). Predictors of group assignment at p < .15 included: age (B (SE) = 1.24 (.35), p < .001); prenatal enrollment into HV (OR (SE) = 1.21 (.16), p = .14); high school education or GED (OR (SE) = .62 (.08), p < .001); college education or greater (OR (SE) = 2.76 (.62), p < .001); single marital status (OR (SE) = .79 (.10), p = .07); and baseline mental health risk status (OR (SE) = .71 (.10), p = .02). Next, a multivariate probit regression model including significant univariate predictors was used to estimate propensity scores for each client using Stata’s pscore procedure (Stata Corp., 2009). Note that 16 clients were excluded due to missing data on covariates. The final multivariate propensity score model fit the data well (χ² (6) = 42.61, p < .001; Hosmer-Lemeshow χ²(8) = 7.04, p = .53), and nearly all baseline characteristics were balanced between groups within blocks defined by propensity scores. After applying propensity score weights, there were no significant baseline differences between HELP and non-HELP clients (see Table 1).

Home Visitors

Home visitors (HELP N = 25; non-HELP N = 46) were all female, ranging in age from 24 to 71 years (M (SD) = 40.71 (12.01)), with a range of less than 1 to 21 years HV experience (M (SD) = 5.74 (4.89)). Racial/ethnic breakdown was 28% White, 33% Hispanic/Latina, 25% Black/African American, 3% Native American, and 10% Multiracial. 43% of home visitors had more than a high school education. There were no significant demographic differences between groups.

Study Conditions

Comparison Condition: Healthy Families America

The comparison condition received the Healthy Families HV model. Serving families prenatally through 5 years, Healthy Families is an evidence-based, voluntary HV program designed to promote positive parenting, enhance child health and development, and prevent child maltreatment. Home visits (approximately 1 hour) occur weekly for the first 6 months, then taper in frequency depending on the family’s level of functioning, and follow a structured curriculum that is flexibly tailored to families’ needs. Services include assessment of family strengths and needs, parenting education and support, developmental screenings, and referrals and service coordination. In the state system under study, families are eligible if they are prenatal or within two weeks of the baby’s birth (or up to 12 months if on Temporary Assistance for Needy Families), and if they have two or more risk factors associated with child maltreatment (e.g., inadequate income, unstable housing, history of substance misuse, no prenatal care, history of depression or psychiatric care).

Intervention Condition: Healthy Families America plus HELP

The intervention (HELP) condition received Healthy Families plus the HELP intervention. The development of HELP is described in a prior publication (Dauber, John, Hogue, Nugent, & Hernandez, 2017) and Online Resource 1 contains a detailed overview of the HELP intervention and training model. HELP includes three core evidence-based interventions: screening for MD, SU, and IPV followed by motivational interviewing (MI) and case management (CM) interventions aimed at linking clients to treatment. HELP was designed as a pragmatic intervention for delivery within routine HV by the HV workforce and has three phases. Identify includes standardized screening for MD, SU, and IPV during the first three months of HV and again six months later using the Edinburgh Postnatal Depression Scale (Cox, Holden, & Sagovsky, 1987), the UNCOPE for SU (Campbell, Hoffman, Hoffmann, & Gillaspy, 2005), and the Relationship Assessment Tool for IPV (Smith, Earp, & DeVellis, 1995). Connect includes a menu of MI and CM interventions applied as needed by the home visitor to link clients to treatment. Support is a maintenance phase aimed at supporting retention in treatment by providing encouragement and resolving ongoing barriers. HELP home visitors and supervisors were trained via a series of informational webinars on SU, MD, and IPV followed by an in-person workshop that included role-play exercises focused on distinct HELP interventions, and optional annual booster trainings. To maintain consistency with the pragmatic approach, the HELP model encouraged supervisors to provide ongoing support to home visitors in HELP implementation, however a formal coaching protocol was not applied, as this was outside of routine HV practice at the time of the study.

Fidelity

Analysis of fidelity to the HELP protocol was completed on a subsample of 116 HELP clients who participated in the more intensive HELP implementation evaluation, which included examination of screening results and HELP fidelity checklists (see (Dauber, John, et al., 2017). Nearly all clients in the implementation subsample were screened (97%) using the validated screening tools described above, and 22% screened positive on at least one risk domain. Among positive-screening clients, 44% received any MI intervention and 32% received any CM intervention. Compared to published fidelity standards for implementation of evidence-based interventions in community settings that report adherence between 60 and 80% (Durlak & DuPre, 2008), fidelity to the MI and CM interventions was low. However, 100% of positive-screening clients received general supportive interventions, consistent with the HELP protocol.

Measures

All study outcome measures were extracted from the Healthy Families Management Information System. Trained family assessment workers or home visitors collected data in clients’ homes using the state’s standardized forms and subsequently entered the data into the Management Information System. Note that the study outcome measures are distinct from the screening tools implemented as part of the HELP intervention. The risk identification study outcome was operationalized as home visitor recognition of client risk related to MD, SU and IPV, as indicated by HV program data documented by home visitors in the Management Information System.

Baseline characteristics

Baseline characteristics were self-reported by clients at intake into Healthy Families and included basic demographics (age, race/ethnicity, education, marital status, employment), public system involvement (receipt of Medicaid, TANF, child welfare system involvement), whether the client enrolled into HV prenatally, received prenatal care, and had additional children under 5 years. Baseline risk identification variables for MD, SU, and IPV were also collected at intake (see below for how these variables were coded).

Risk Identification

Home visiting clients were considered risk identified for MD, SU, or IPV if their data record indicated either a current issue with or current service receipt for mental health, SU, or IPV. This information was gathered from the Healthy Families intake form and follow-up interview, which is conducted every six months. At each interview, home visitors indicate (1) whether the client has a “current issue” with alcohol abuse, substance abuse, depression, other mental illness, or domestic violence as defined by client report, report by a family member or other person, or the home visitor’s observation, and (2) whether the client is currently involved in services for mental health, substance use, or IPV. While these administrative indicators of risk identification are broad and lack the sensitivity and specificity of validated screening tools, they are pragmatic measures that were not intended to identify risk in a diagnostic way, but rather to document whether home visitors recognized a need within the three risk domains. Note that while the mental health component of the HELP intervention was specific to maternal depression, the HV program collected information on mental health generally. It is thus possible that the study outcomes reported for MD actually pertain to another mental health problem. However, in practice, MD is the most commonly encountered mental health concern in the HV context, and it is likely that the majority of mental health outcomes reported pertain to MD.

Risk Discussion

Healthy Families home visitors complete logs after each visit in which they indicate topics discussed via check boxes. Discussion of MD in a home visit was coded as having occurred if the home visitor checked one of the following: “administered mental health screening,” “shared mental health screening results with family,” or “discussed mental health issues.” Discussion of SU was coded as having occurred if the home visitor checked one of the following: “administered SU screening,” “shared SU screening results with family,” or “discussed substance and alcohol use issues.” Discussion of IPV was coded as having occurred if the home visitor checked one of the following: “administered domestic violence screening,” “shared domestic violence screening results with family,” “based on domestic violence screening, developed safety plan,” or “discussed domestic violence issues and safety planning.” The final variables analyzed documented any discussion of each risk domain in any home visit occurring during the HELP study period (coded as yes/no).

Referral for Services

Home visitors tracked the date and type of all referrals provided to clients in the Management Information System. Any referral made during the HELP study period for psychiatric or psychological treatment (MD), other mental health counseling (MD), domestic violence services (IPV), or substance abuse services (SU) was included in the Referral outcome variables (coded as yes/no to indicate whether any referral was made).

Statistical Analysis

As described above, propensity weights were used in all analyses to account for potential imbalance between conditions (Guo & Fraser, 2010). Weighted logistic regressions were conducted to compare HELP and non-HELP clients on the three study outcomes (risk identification, risk discussion, referral) in each of the three risk domains (MD, SU, and IPV). Next, we conducted multi-level mixed effects models in Mplus 7.31 (Muthen & Muthen, 1998–2017) to test for potential HELP impacts on outcomes, including propensity weights, and controlling for client and home visitor covariates. All analyses were conducted as intent-to-treat, including all clients in the HELP condition regardless of dosage. Data for the outcome analyses have a three-level nested structure: clients within home visitors within counties. We modeled home visitor and client as random effects, and used the sandwich variance estimator to adjust standard errors as appropriate to account for nesting within counties (Asparouhov, 2005). Home visitor years of experience (calculated as years from hire date to end date of HELP period or date of termination from HV program) and education (any education beyond high school) were included as covariates at the home visitor level. Client-level covariates included: age, education, marital status, primary language, prenatal enrollment into home visiting, TANF status, child welfare involvement, the number of home visits completed, and baseline risk status. Nonsignificant client covariates were trimmed in final models. Effects associated with study condition (HELP vs. non-HELP) were indicated by a statistically significant B coefficient and tested by the pseudo-z test, calculated by dividing the coefficient by its standard error. Odds ratios are presented as measures of effect size.

Results

Missing Data and Sensitivity Analysis

Risk identification and referral outcomes were missing for 31% and 39% of the HELP and non-HELP samples respectively. As described above, risk identification outcomes were coded from the Healthy Families follow-up assessments, conducted every six months. Of the 356 clients who were missing risk identification outcomes, 347 did not complete a Healthy Families follow-up assessment during the HELP study period [258 due to dropping out of Healthy Families before six months and 98 for unknown reasons]. The remaining 9 clients were missing risk identification outcomes for unknown reasons. Thus, analyses on risk identification outcomes excluded the 356 clients with missing data. Referral outcomes were coded from referral forms that could be completed during any home visit and 452 clients who had at least one home visit had no referral forms completed. Consultation with the state administrators who monitor data integrity for Healthy Families suggested that missing referral forms for clients who attended at least one home visit likely indicates that no referrals were made, and that values of zero could legitimately be entered for the referral outcomes. We conducted a sensitivity analysis in which we ran the mixed effects models for each referral outcome in two ways: (1) coding all missing referral outcomes as zero, and (2) excluding clients with missing referral outcomes. Results were equivalent and we report the findings from the models with missing referrals coded as zero.

Bivariate Weighted Group Differences in Outcomes

Propensity score-weighted logistic regressions were conducted to examine bivariate group differences on study outcomes (see Table 2). HELP clients had significantly higher rates of risk identification and discussion for all risk domains, and higher referral rates for SU and IPV (very low numbers in some cells and the large confidence interval for SU suggest that significance should be interpreted with caution).

Table 2.

Weighted bivariate group comparisons on study outcomes.

	Non-HELP	HELP
	Weighted N (%)	Weighted N (%)	OR (95% CI)
Risk Identificationa
Weighted Total N	382	410
MD	57 (15%)	110 (27%)	2.08 (1.44, 3.01)***
SU	17 (5%)	36 (9%)	2.07 (1.12, 3.81)*
IPV	16 (4%)	32 (8%)	1.96 (1.04, 3.67)*
Discussionb
Weighted Total N	576	573
MD	164 (29%)	312 (55%)	2.80 (2.16, 3.63)***
SU	59 (10%)	280 (49%)	7.63 (5.37, 9.84)***
IPV	49 (8%)	238 (41%)	6.46 (4.71, 8.87)***
Referralc
Weighted Total N	576	573
MD	50 (9%)	55 (10%)	1.11 (.72, 1.72)
SU	1 (0.13%)	13 (2%)	16.79 (2.04, 138.21)**
IPV	13 (2%)	31 (5%)	2.49 (1.27, 4.91)**

Note.

^aClient identified as risk positive in the MIS at any follow-up timepoint

^bHome visitor noted discussion of risk on any home visit log during the study period

^cHome visitor noted a referral in the MIS at any time during the study period. ATE weights were applied to all analyses.

^*p < .05

^**p < .01

^***p < .001.

HELP Impacts on Outcomes

Multi-level mixed effects analyses were conducted to examine the impact of HELP on study outcomes, controlling for home visitor and client characteristics. Significant findings are reported here, and full tables of all parameter estimates and p-values for both full and trimmed models are available in Online Resource 2. HELP clients were significantly more likely than non-HELP clients to have their home visitor discuss MD [B (SE) = 1.41 (.57), pseudo-z = 2.46, p = .014, OR = 4.08], SU [B (SE) = 2.77 (.48), pseudo-z = 5.81, p = .000, OR = 15.94], and IPV [B (SE) = 2.24 (.37), pseudo-z = 6.03, p = .000, OR = 9.35]. These effects were relatively large based on the odds ratios, particularly for SU and IPV. There were no significant effects of HELP on risk identification or on referral for MD or IPV. A large effect was found for SU referrals [B (SE) = 2.67 (1.20), pseudo-z = 2.23, p = .026, OR = 14.41], however the very small number of SU referrals warrants caution in interpreting this finding.

Home Visitor and Client Characteristics Associated with Outcomes

To inform future research on service tailoring, we examined whether home visitor and client characteristics were related to study outcomes. Home visitors with more years of experience were significantly more likely to identify client SU [B (SE) = .06 (.02), pseudo-z = 2.48, p = .013], but less likely to discuss MD [B (SE) = −.06 (.02), pseudo-z = −2.39, p = .017], or to make referrals for IPV [B (SE) = −.13 (.06), pseudo-z = −2.03, p = .042]. Note that Mplus does not provide odds ratios for covariates at the home visitor level. Clients with less education [B (SE) = −.39 (.08), pseudo-z = −4.71, p = .000, OR = .68] and whose primary language was English [B (SE) = .83 (.31), pseudo-z = 2.71, p = .007, OR = 2.29] were more likely to be risk- identified for MD. Prenatally enrolling clients were more likely to have their home visitors discuss MD [B (SE) = .43 (.17), pseudo-z = 2.58, p = .010, OR = 1.53]. Child welfare involved clients were more likely to be risk-identified for SU [B (SE) = 1.31 (.44), pseudo-z = 2.99, p = .003, OR = 3.69] and to have their home visitor discuss MD [B (SE) = .68 (.24), pseudo-z = 2.83, p = .005, OR = 1.96] and SU [B (SE) = .94 (.47), pseudo-z = 1.99, p = .046, OR = 2.55].

The number of home visits attended and baseline risk status predicted nearly all outcomes. Specifically, clients who attended more home visits were more likely to be risk- identified for MD [B (SE) = .02 (.01), pseudo-z = 3.21, p = .001, OR = 1.02] and IPV [B (SE) = .01 (.01), pseudo-z = 2.06, p = .040, OR = 1.01], to have their home visitor discuss MD [B (SE) = .04 (.01), pseudo-z = 7.56, p = .000, OR = 1.04], SU [B (SE) = .04 (.01), pseudo-z = 8.04, p = .000, OR = 1.04], and IPV [B (SE) = .04 (.01), pseudo-z = 6.96, p = .000, OR = 1.04], and to receive a referral for MD [B (SE) = .03 (.01), pseudo-z = 6.13, p = .000, OR = 1.03] or IPV [B (SE) = .02 (.01), pseudo-z = 2.03, p = .0 42, OR = 1.02]. Clients who were risk-identified at baseline were more likely to be risk-identified at follow-up for MD [B (SE) = 2.03 (.34), pseudo-z = 5.92, p = .000, OR = 7.62], SU [B (SE) = 3.17 (.45), pseudo-z = 7.07, p = .000, OR = 23.79], and IPV [B (SE) = 1.92 (.43), pseudo-z = 4.47, p = .000, OR = 6.79], more likely to have their home visitor discuss MD [B (SE) = 1.16 (.11), pseudo-z = 10.94, p = .000, OR = 3.18] and SU [B (SE) = 1.87 (.30), pseudo-z = 6.19, p = .000, OR = 6.49], and more likely to receive a referral for MD [B (SE) = .78 (.16), pseudo-z = 4.90, p = .000, OR = 2.17] and SU [B (SE) = 2.02 (.68), pseudo-z = 2.97, p = .003, OR = 7.56].

Discussion

This study examined the impact of HELP, a pragmatic screen-and-refer approach for addressing prevalent maternal behavioral health risks in HV. The primary impact of HELP was on increasing the extent to which home visitors discussed MD, SU, and IPV with clients during home visits. Effect sizes were large, with the largest effects found for SU. However, HELP had no impact on risk identification and minimal impact on referral rates. Rates of risk identification and referral were low in both the intervention and comparison groups, with less than 30% of the overall sample risk-identified for at least one risk domain at follow-up, and less than 10% receiving a referral. These rates of risk identification are lower than expected, given national data on prevalence of behavioral health risks in HV clients (Michalopoulos et al., 2015). The low rates of risk identification may reflect HELP’s lack of impact on improving home visitors’ ability to recognize behavioral health risks in their clients, as well as clients’ reluctance to disclose concerns in these areas to home visitors. Given the operationalization of risk identification as including clients who were known to be involved in services for the three risk domains, the low rates of risk identification may reflect clients’ difficulties accessing and remaining engaged in treatment. Finally, it is possible, though not probable, that the sample included in this study represents a lower risk sample than the typical population served by the Healthy Families program.

We suggest three potential explanations for the lack of impact of HELP on risk identification and referral. First, as documented in our prior study (Dauber, John, et al., 2017), fidelity to the HELP protocol was low, with less than half of positive-screening clients receiving any MI or CM intervention. Thus, it is possible that the dose of HELP was high enough to prompt more general discussions about risk factors—an intervention within easier reach for home visitors—but not high enough to have a meaningful impact on risk identification and referral, which are more technically demanding. Given the small scope of this pilot study, resources were targeted at training home visitors, and implementation supports aimed at other aspects of the implementation context (e.g., supervision guidelines, coaching, service coordination efforts) were not instituted, which likely contributed to low fidelity (Powell et al., 2012). Training alone is not likely to lead to sustained practice change, as noted in a recent study that trained home visitors in MI techniques to improve communication around sensitive topics, including mental health and SU (West, Gagliardi, et al., 2018). This study found that initial improvements in MI skill following training were largely attenuated over time. More intensive reinforcement of MI and CM through ongoing coaching and support to both home visitors and supervisors is likely required to promote greater uptake of these interventions within HV (Fixsen, Blase, Naoom, & Wallace, 2009). The extent to which formal coaching models can be feasibly implemented and sustained long-term within routine HV is currently being tested. Second, stigma and fear of child removal frequently lead HV clients to conceal MD, SU, and IPV from home visitors, contributing to low rates of risk identification. Many pregnant and postpartum women avoid seeking help for behavioral health concerns for these reasons (Stone, 2015). Finally, pregnant and parenting women experience a host of barriers to accessing treatment systems, including lack of transportation and child care, long waiting lists, payment difficulties, cultural or language barriers, and prior negative experiences with treatment (Abrams, Dornig, & Curran, 2009). Anticipated barriers to service access may have caused home visitors to be less likely to offer referrals, or made clients less likely to accept them.

Home visitor experience and client characteristics including education, primary language, prenatal enrollment, child welfare system involvement, and clients’ baseline risk status also impacted study outcomes. These findings are consistent with recent national data showing that home visitors with more training in sensitive topics discussed them with clients more frequently (Michalopoulos et al., 2019) and also reinforce the complexity of families served by HV programs and the need to tailor service delivery to families’ unique profiles (Supplee et al., 2018). Future studies with larger samples are needed to test whether HELP is differentially effective for subgroups of clients. Additionally, the number of home visits attended was an important predictor of nearly all study outcomes, with clients who attended more visits more likely to have their behavioral health risks identified and addressed. As the home visitor-client relationship is strengthened over time, clients may become more willing to reveal their behavioral health needs.

A primary strength of the study is its pragmatism: the study was conducted in the context of a widely implemented HV model and leveraged data collected as part of routine HV operations. However, there were several important limitations. First, propensity score analysis does not account for the influence of unmeasured differences between groups (Guo & Fraser, 2010), and the discrepancy in size between the conditions may have introduced unexplained variability. A true randomized controlled trial is necessary to confirm study findings. Second, given the low fidelity to HELP, it is difficult to disentangle whether the lack of HELP impact is due to the intervention itself being ineffective or to clients receiving an insufficient dosage of the intervention. Third, the administrative outcomes, while pragmatic, lack sensitivity and specificity. Risk identification and referral within HV are complex processes that are not adequately captured by simple dichotomous variables (Goldberg, Greenstone Winestone, Fauth, Colon, & Mingo, 2018). Finally, generalizability is limited by the focus on a single state and HV model.

Our findings suggest that training home visitors in a screen-and-refer intervention such as HELP can lead to increased discussion of client behavioral health risks during home visits, which can serve as an important first step towards better meeting these needs within HV. However, discussion of behavioral health risks alone, in the absence of adequate follow-up on identified concerns, may have negative effects. Further investigation into the specific content of discussions of risk during home visits as well as the extent of follow-up is warranted. Screening for depression and IPV is now mandated for federally-funded HV programs (HRSA Maternal and Child Health, 2016); however, screening alone often does not lead to receipt of needed services (Yonkers et al., 2009). Recent research on referral practices within HV supports the complexity of the referral process, and referrals often do not lead to successful engagement in services (Goldberg et al., 2018). Moving the needle on risk identification, referral, and most importantly, access to treatment will require more intensive intervention than what was provided in HELP. We suggest two avenues that should be explored in future studies to see whether they have greater success in effectively addressing maternal behavioral health risks within HV. The first is bolstering implementation supports for screen-and-refer models such as HELP, including empirically-supported implementation strategies such as coaching and fidelity monitoring that support the transfer of training into practice (Burke & Hutchins, 2007), as well as improved service coordination between HV and treatment providers to maximize successful engagement in treatment (West, Duggan, Gruss, & Minkovitz, 2018). There is precedent and support for integrating these strategies into HV. For example, a model enhancement focused on emerging literacy skills showed higher quality implementation when fidelity monitoring was integrated into HV supervision (Manz et al., 2017). In addition, a recent study supported the use of formal policies and protocols related to care coordination with primary medical care (Paradis, Belknap, O’Neill, Baggett, & Minkovitz, 2018). Service coordination is of primary importance in HV; it is a MIECHV benchmark (HRSA Maternal and Child Health, 2016), and a framework for service coordination within HV was recently published (West, Duggan, et al., 2018). The state that participated in the current study is engaged in ongoing efforts to strengthen HV systems and improve service coordination.

The second avenue that may have potential for addressing behavioral health within HV includes developing enhancements to HV models that deliver evidence-based behavioral health treatment during routine home visits. Model enhancements for depression and IPV are currently being tested, and initial results are favorable (Sharps et al., 2016; Tandon, Ward, et al., 2018). For example, the first RCT testing a home visitor-delivered preventive intervention for postpartum depression showed significant reductions in depressive and anxiety symptoms at six months (Tandon, Ward, et al., 2018). Our group is developing a similar HV enhancement for SU.

Both avenues are likely required to meet the needs of the full spectrum of risk seen in HV clients—those at lower levels of risk who may be well-served by the integration of brief interventions into HV, and those with complex co-occurring behavioral health concerns who need more intensive interventions provided by specialty treatment providers. Additionally, both are in line with the precision prevention science framework, which emphasizes tailoring service delivery to subgroups of clients based on particular characteristics or areas of need, and has been suggested as a way of improving the efficiency and effectiveness of HV programs as they continue to be scaled up nationally (Supplee et al., 2018).