Innovative Research Methods to Advance Precision in Home Visiting for More Efficient and Effective Programs

Abstract

Home visiting during early childhood can improve a range of outcomes for children and families. As evidence‐based models are implemented across the nation, two questions have emerged. First, can home visiting improve outcomes more efficiently? Second, can overall effects be strengthened for specific subgroups of families? For the past several decades, research focused on testing the average effects of home visiting models on short‐ to long‐term outcomes has found small impacts. These effects are not the same for all families. The field needs new evidence produced in new ways to overcome these challenges. In this article, we provide an overview of the evidence in this field, including what works and for whom. Next, we explain precision approaches to various fields and how this approach could be used in home visiting programs. Research on precision home visiting focuses on the ingredients of home visiting models, collaborating with practitioners to identify the ingredients and testing them on near‐term outcomes, and using innovative study designs to learn more quickly what works best for which families. We conclude by proposing four pillars of research that will help achieve precision home visiting services.

Early childhood home visiting is a service‐delivery strategy for expectant families and families with young children to achieve outcomes ranging from positive parenting to healthy child development. Home visiting programs achieve outcomes through education, family support, and referrals to community‐based services. They are designed based on decades of research on developmental science to determine how to support healthy family functioning, parenting, and child development. Home visiting is available in every state in the United States and in many U.S. tribal communities (National Home Visiting Resource Center, 2018), which serve some of the most vulnerable families (Duggan et al., 2018).

Communities use many different home visiting models. Home visiting programs vary in the families they target, the specific outcomes they aim to achieve, and the services they provide (Duggan et al., 2018). Many of the most widely disseminated models are complex, addressing a broad range of outcomes in frequent visits over 2 or more years. These models provide training and support to local programs, as well as some form of accreditation and ongoing monitoring of program quality (Duggan et al., 2018).

Overview of Evidence

The Home Visiting Evidence of Effectiveness (HomVEE) review is one of the most comprehensive assessments of empirical evidence on home visiting. HomVEE has identified 28,927 studies, including 363 randomized control trials (RCT) of 46 home visiting models (Sama‐Miller et al., 2018), with some impacts 20 years after intervention (Olds et al., 2014). In the past 10 years, the diversity of evidence‐based models has increased to include not only intensive, long‐term models, such as the Nurse Family Partnership (NFP; Olds et al., 2013), but also short‐term models, such as the Family Check‐Up (Moore et al., 2012); models focused on one specific outcome, such as Attachment and Biobehavioral Catchup (Dozier & Bernard, 2019); and universal models, such as Family Connects (Dodge et al., 2014).

While a rich body of research over the last 20 years shows impacts on important child and family outcomes, these studies reveal a pattern of small effect sizes, about 0.10 on average (see Table 1). A national randomized trial of the four models used most commonly in the United States found similar effect sizes (Michalopoulos et al., 2019). This suggests that the field needs a radically different approach if we are to achieve important outcomes for the most vulnerable families.

Table 1.

Average Effect Sizes from Impact Studies of Home Visiting over Children’s First 2 Years

Domain	Effect size range	Average effect size
Measures of parenting	−0.11 to 0.88	0.09 to 0.11
Measures of child maltreatment	−0.08 to −0.20	−0.02 to −0.03
Measures of child health	−0.24 to 0.39	−0.08 to 0.04
Measures of child development	−0.36 to 0.14	−0.08 to 0.06

The current state of the research raises two questions. First, can home visiting improve outcomes more efficiently? Many evidence‐based models aim to serve families over several years (Duggan et al., 2018), but many families drop out of home visiting after 1 year (Duggan et al., 2018). The long‐term effectiveness of alternative models of shorter duration, such as Family Connects (Dodge et al., 2014) and Attachment and Biobehavioral Catchup (Dozier & Bernard, 2019), suggests the possibility of improving the efficiency of home visiting. While some studies point to correlational evidence of the length of time home visiting has to last to achieve outcomes (e.g., Goyal et al., 2013; Holland, Xia, Kitzman, Dozier, & Olds, 2014), we lack tests that determine the optimal length of programs needed to achieve certain outcomes.

Second, can overall effects be strengthened for vulnerable families who enroll but fail to benefit from current models? Most research on these models has been designed to test average effects by analyzing subgroups after the study is complete. The modest overall effects of home visiting suggest that programs achieve stronger impacts for some families than others (e.g., Berlin, Martoccio, & Jones Harden, 2018; McFarlane et al., 2013). One way to improve overall impacts is to target services more narrowly, focusing on serving only the types of families shown to benefit. A more inclusive option is to determine how to tailor services to extend effects to families that do not benefit. An inherent tension exists between targeting specific families and delivering universal models to strengthen impacts across all families (Rose, 1985). The field needs a range of models, but to achieve stronger impacts overall, we need a more thorough understanding of what works best for whom.

What Works?

While it remains important to examine the overall effectiveness of specific models, we also need to consider the effectiveness of specific ingredients of models. We need to determine which components of home visiting are the active ingredients that drive improvements in outcomes. To our knowledge, only one meta‐analysis has examined the research to understand which ingredients of home visiting programs spur impacts (Filene, Kaminski, Valle, & Cachat, 2013), but results were limited due to the lack of specificity of model ingredients in the literature. Models often include specific components theorized to change an intended outcome based on empirical research on child development or parenting (e.g., Olds, Kitzman, Cole, & Robinson, 1997). Some ingredients may be essential across all home visiting models or families, while others might be necessary for only some models or families. While we can infer some causal links between components and outcomes by testing model enhancements (e.g., Haire‐Joshu et al., 2018), most models have not causally tested which ingredients drive intended outcomes.

For Whom?

Often, tests of moderation conducted after a study is complete focus on family characteristics that are easy to measure, such as race or socioeconomic status, and are subject to sampling bias (Supplee, Kelly, MacKinnon, & Barofsky, 2013). Some evidence suggests that considering more nuanced characteristics may help us understand variation in impacts. For example, in a large clinical trial of Healthy Families America, a home visiting model focused on preventing child maltreatment, maternal depression interacted with type of maternal‐avoidant attachment. The program improved children’s behavioral outcomes in families where the mother was either depressed or avoidant, but not where the mother was both depressed and avoidant (McFarlane et al., 2013). Research on the Nurse Family Partnership, a home visiting model focused on child health outcomes, suggested that impacts on child maltreatment for women experiencing low levels of intimate partner violence occur only by changing short‐term outcomes of fertility and economic self‐sufficiency (Eckenrode et al., 2017). Home visitors tailor content to individual family needs, but it is unclear if or how they use empirical evidence to guide this tailoring based on family characteristics (Duggan et al., 2018). Rigorous, replicated analysis of moderation across different samples is needed to advance our understanding of how impacts vary across subgroups.

Precision Home Visiting

The fields of medicine and public health offer a rationale and methods to broaden and strengthen impacts through greater precision in interventions. Medical researchers increasingly use innovative designs to help determine which treatment works best for which patients, using findings from basic science to tailor treatments for individual patients’ genomic and environmental characteristics (Tsao et al, 2016). For example, rather than treating all cases of lung cancer the same way, doctors can now offer treatments for specific types of cancer identified through genetic research, dramatically improving outcomes (Tsao et al., 2016). Similarly, in precision public health, practitioners use surveillance methods to align interventions with communities’ specific needs for more efficient and cost‐effective investment (Sujan & Eckenrode, 2017; Weeramanthri et al., 2018), such as offering less costly approaches to individuals with fewer risk factors and more costly, intensive programs to high‐risk individuals (e.g., Sujan & Eckenrode, 2017). Recently, those involved in prevention science have called for refining implementation of evidence‐based programs by offering services tailored to specific clients and adapting them based on clients’ responses (August & Gerwitz, 2019).

Drawing from the concepts and methods of precision medicine and public health, home visiting can use research to identify what works best and for whom to improve outcomes more efficiently across diverse families. While not all precision medicine or public health methods may apply to home visiting, we can adopt methods that do apply, adapt methods that may apply, and forego methods that do not apply.

Innovative research methods are emerging to hasten progress in achieving precision home visiting. This approach shifts the focus from average effects of a complex intervention toward a deeper understanding of how family characteristics moderate the effects of specific ingredients and the implications of using this understanding to refine theory and approaches to intervention. Drawing from other work (e.g., breakthrough impact research; Center on the Developing Child at Harvard University, 2016), we propose that implementing four pillars of precision home visiting research will advance the field.

Pillar 1: Authentic Research‐Practice Partnerships

Strong, authentic partnerships among diverse stakeholders are critical for translating research into practice and policy (Finnigan & Daly, 2014). A high level of interaction across stakeholder groups is essential to identify the highest‐priority issues and achieve a shared vision for designing an intervention that, if successful, could be taken to scale quickly and easily. Following the lead of the Patient Centered Outcomes Research Initiative, we define stakeholders broadly, including families, home visitors, local or state program administrators, funders, and researchers (Patient‐Centered Outcomes Research Institute, 2014). Input from key stakeholders can highlight the most important questions to test that align with what providers see in the field, and stakeholders can help design innovations that prevent challenges to implementation when scaling (Supplee, Parekh, & Johnson, 2018). For example, the NFP tested an innovation to improve family retention, which was effective, but interviews after the study suggested that the method was too difficult to implement (Olds et al., 2015). Having this kind of information at the start of a trial would have helped the study be scalable more quickly. (For an example of how to involve stakeholders in study design, implementation, data interpretation, and dissemination, see Jensen et al., 2018.)

Pillar 2: Focusing on Active Ingredients

In interventions, active ingredients are the parts of the program responsible for impacts on key outcomes. If a complex intervention fails to purposefully test the active ingredients and tests only overall effects, the intervention is likely to be ineffective. The concept of active ingredients is not new, and is similar to other concepts, such as common factors (Chorpita et al., 2017), evidence‐based kernels (Embry & Biglan, 2008), or common factors (Wampold, 2015). In one study, researchers analyzed complex highly structured interventions to promote adolescent mental health to identify the interventions’ target populations, common ingredients, and intended outcomes (Chorpita et al., 2017). They trained practitioners to implement common ingredients with fidelity, monitored practitioners’ use of them, and measured changes in adolescent patients’ mental health functioning. The slope of improvement was steeper among adolescents who received services from practitioners using core components than for youth who participated in full, complex interventions.

The field will need to define and test active ingredients using rigorous designs to move toward more efficient models and broaden benefits across diverse subgroups. A key step in testing active ingredients is to test near‐term mediators to refine models without waiting for long‐term outcomes. In many cases, we lack strong empirical evidence that near‐term mediators result in long‐term outcomes of interest. However, we have strong evidence for some mediators. For example, improved parent–child interaction is a mediator that is strongly linked to many long‐term outcomes (e.g., cognitive development, Harris & Almutairi, 2016; child maltreatment, Green et al., 2018), as is parental mental health (National Research Council & Institute of Medicine Committee on Depression, Parenting Practices, & the Healthy Development of Children, 2009). This suggests that the field can begin more rapid testing of active ingredients with mediators with strong empirical evidence while building empirical evidence between other mediators and long‐term outcomes. Partnerships between developmental scientists and prevention scientists can leverage findings across bodies of research.

Research on home visiting includes several examples of testing active ingredients. One is an online guide for home visitors that offers information on selecting strategies to teach parents how to promote early literacy based on their children’s expressive language score (Buzhardt et al., 2018). In this work, researchers first tested the impact of each strategy to ensure that it changed the mediator of specific aspects of children’s expressive language. Another example is SafeCare, a multimodule home visiting model, which was developed using a series of single‐case designs testing each module’s impact on a specific short‐term mediator (Lutzker & Rice, 1984) before testing the full model’s effectiveness in preventing the long‐term outcome of child maltreatment (Silovsky et al., 2011).

Pillar 3: Coherent, Well‐Specified Models

In a coherent model, the ingredients of an intervention align with the target outcome (Segal, Opie, & Dalziel, 2012). A well‐specified model avoids overly broad terms like provide parent support, and instead articulates the specific active ingredients and strategies home visitors should use to motivate, enable, and reinforce behavior change. Some of the ingredients of complex programs may not change the desired outcome. Some models have overly broad theories of change (Hebbeler & Gerlach‐Downie, 2002). Others cover a wide variety of content, some of which is not aligned with target outcomes, an issue that has worsened in recent years (Duggan et al., 2018).

A review of home visiting programs to reduce child maltreatment provides evidence of the importance of coherence (Segal et al., 2012). In this work, all the home visiting programs whose activities and content aligned with the intended outcome—child maltreatment—had changes in child maltreatment. Sixty percent of programs in which the content and activities aligned only partially with the outcome of child maltreatment had impacts in that outcome. None of the programs that lacked alignment of content and activities with child maltreatment showed impacts in maltreatment. In a recent national study of home visiting, researchers found small average effects across models but indications that individual models had stronger impacts on outcomes that were aligned with the individual model (Michalopoulos et al., 2019). This indicates that aligning activities to desired outcomes and precisely identifying the ingredients of models tied to an outcome of interest could produce more streamlined and effective models. The work of refining models so they are coherent and well‐specified should be done with current models and does not require new models to be developed.

Pillar 4: Innovative Research Designs that Accelerate Building Evidence

Science typically follows a progression from basic research to establish the prevalence and causes of a phenomenon to highly controlled efficacy studies of interventions, then to less tightly controlled experimental studies to establish effectiveness under usual circumstances and dissemination and implementation studies of how models can be scaled up (Knox, Hill, & Berlin, 2018). Conventional randomized trials of interventions usually answer questions of average effects across families and can take years to complete. They generally involve a fixed intervention, that is, an intervention that is not modified as the trial unfolds, even in the face of knowledge that suggests modifications would be wise. In contrast, precision home visiting will need to use rapid‐cycle, iterative study designs for evaluating and improving interventions in actual settings to increase empirical information on active ingredients. Next, we briefly describe three of these innovative designs. (For more information about each method, see https://www.hvresearch.org/.)

Multiphase Optimization Strategy

MOST is a research framework that uses engineering principles to optimize interventions. Researchers name specific ingredients of an intervention’s active ingredients, and then test the effectiveness of the ingredients through methods like fractional factorial designs (Collins, Murphy, & Stretcher, 2008). The framework drives researchers to account for key constraints on the system in which the intervention will be used (e.g., time, money) to ensure that what is developed can be scaled to and implemented in actual settings. Once the most efficient ingredients of a program are identified, researchers carry out a standard RCT on the whole program. For example, a strong relationship between the mother and the home visitor is critical for effective services and outcomes. A model may identify the ingredients thought necessary to establish such a strong relationship, and then test them by assigning families randomly to one or more of the ingredients. Using a fractional factorial design, the investigator would identify which ingredients are tied directly to strengthening the relationship, and then use those together to test an enhancement for a program targeted at building a relationship between mothers and home visitors.

Sequential Multiple Assignment Randomized Trial

In a SMART, researchers examine the effectiveness of different sequences of treatment options based on predetermined decision points according to time or outcome (e.g., if the child is not responding after 3 weeks, a new treatment option would be rerandomized; Collins et al., 2008). The design allows researchers to examine the effects of different treatment pathways tailored to fit the unique needs of the participants. SMARTs can help home visitors identify the most effective ways to tailor programs based on family characteristics and can ensure that families get services that fit their needs. Researchers can use information from SMARTs when designing interventions, determining follow‐up activities, and identifying target participant groups.

Idiographic Clinical Trials

ICTs use the participant’s own data as the comparison to test changes that occur within the individual over time (Ridenour, Wittenborn, Raiff, Benedict, & Kane‐Gill, 2016). ICTs can serve as an alternative to conventional RCTs because they allow for smaller samples, shorter durations, and lower study costs without compromising internal validity. However, their findings cannot be generalized to larger samples or to whole models. For example, if a home visiting program identifies a need to address maternal bonding with children more effectively, researchers could study individual lessons to see if they are effective at getting mothers to follow the child’s lead or provide care and comfort. After evidence accumulates about each individual component, the effective components can be put together into a lesson, which can then be tested as a full package in a more conventional RCT.

Stronger Research, Stronger Programs

Home visiting is a promising mechanism for supporting positive outcomes for vulnerable children and families. The current system is complex and costly, while producing modest average effects. The field is poised to improve efficiency and broaden impacts, with new research methods to hasten the transition from recognized need to intervention design to studies, ultimately scaling up what works best for whom. We need to bring together the fields of developmental science and prevention science to test active ingredients, develop well‐specified and coherent theories of change, and use more efficient research methods. We believe implementing the pillars of research on precision home visiting that we have proposed can transform home visiting programs, as well as advance other prenatal and early childhood services.

We acknowledge the complexities and challenges of our proposal to shift the research paradigm. Authentic research‐practice partnerships take substantial time and are often costly to execute. Identifying active ingredients is a simple concept in theory, but isolating ingredients in complex programs and then testing them will be challenging. Analyses of studies after they are complete suggest that some outcomes may be achieved for one specific subpopulation along two mediated pathways (Eckenrode et al., 2017) or that a moderator may increase impact for one outcome while attenuating it for another (Loughlin‐Presnal & Bierman, 2017). Efforts to strengthen the alignment of models between activities and intended outcomes will face political pressure amid calls for home visiting programs to add more content and address more outcomes. For example, in 2012, Congress proposed moving lead screening activities from the Centers for Disease Control and Prevention to home visiting programs, adding activities to home visiting programs that may not align with intended outcomes (National Safe and Healthy Housing Coalition, [Link]). Finally, given that few researchers are comfortable with rapid‐cycle designs, these efforts will require multidisciplinary partnerships. Yet despite this complexity, we believe the potential benefits outweigh the challenges in the long‐term.

We acknowledge that not all aspects of precision medicine may apply to precision home visiting. Interventions tested in medicine often are not as complex as behavioral interventions. As the field develops, we need to consider which concepts and methods can be adopted, which can be adapted, and which should be abandoned. These methods should be revisited as advances in science, such as emerging research on biomarkers and parenting (e.g., Ulmer‐Yaniv et al., 2018), highlight connections between fields.

By understanding what services work best for which families, we can strengthen impacts more efficiently. Our research and development center is charged with building awareness of precision research methods, supporting researchers to study precision home visiting, and identifying opportunities to conduct and fund studies on precision. We hope the idea of precision research spreads across the research community to support more efficient, effective programs that meet family needs to improve critical social outcomes.