Universal Reach at Birth: Family Connects

Summary

How do we screen all families in a population at a single time point, identify family-specific risks, and connect each family with evidence-based community resources that can help them overcome those risks—an approach known as targeted universalism? In this article, Kenneth A. Dodge and W. Benjamin Goodman describe Family Connects, a program designed to do exactly that.

Developed and tested in Durham, NC, Family Connects—now in place at 16 sites in the United States—aims to reach every family giving birth in a given community. The program rests on three pillars. The first is home visiting: trained nurses (or other program representatives) welcome new babies into the community, typically at the birthing hospital, then work with the parents to set up one or more home visits when the baby is about three weeks old so they can identify needs and connect the family with community resources. The second pillar, community alignment, is an assembly of all community resources available to families at birth, including child care agencies, mental health providers, government social services, and long-term programs for subgroups of families with identified needs, such as Healthy Families and Early Head Start. The third pillar, data and monitoring, is an electronic data system that acts as a family-specific psychosocial and educational record (much like an electronic health record) to document nurses’ assessments of mother and infant, as well as connections with community agencies.

In randomized clinical trials, Family Connects has shown promising results. Compared to control group families, families randomly assigned to the program made more connections to community resources. They also reported more positive parenting behaviors and fewer serious injuries or illnesses among their infants, among other desirable outcomes. And in the first five years of life, Family Connects children were significantly less likely to be subject to Child Protective Services investigations than were children in a control group.

Since University of Colorado pediatrician Henry Kempe first identified battered child syndrome in 1962, most of the nation’s efforts in this area have been directed toward protecting and treating children after the fact of maltreatment. This is a never-win situation because the maltreated children keep on coming. More recently, researchers, service providers, and policy makers have been shifting toward prevention of maltreatment. But most of these efforts have been underfunded or have targeted a small number of children in a community. Frustrated by the modest (at best) overall impact of child protection programs that serve only a small number of families in a community, policy makers and scholars have called for new approaches that seek population impact—that is, lowering the maltreatment rate for all children and families in a particular area, such as a state, county, city, or school system.¹

Achieving population impact requires more than simply scaling up a proven intervention that had been delivered to only a small number of children. It requires an approach that involves the entire community of service providers, policy makers, and population of families from the outset. To illustrate this point, in this article we identify challenges that occur when attempting population impact by developing and scaling up programs targeted only to small subgroups. Next, we introduce the theoretical model for Family Connects, a program that seeks to overcome these challenges and to improve population indicators of infant health, wellbeing, and child maltreatment through collaboration with the community of intervention providers, brief postnatal home visits to all birthing families, and family-specific connections between families and community resources. We then describe findings from three independent evaluations of Family Connects. These findings suggest that the program can be implemented with broad reach, high quality, and positive impacts for infants and families. We conclude by discussing possibilities and challenges for disseminating and sustaining the model, as well as future opportunities for innovation.

Challenges to Scaling Up Targeted Interventions

Several major early intervention programs, such as Nurse Family Partnership and Early Head Start, are long-term, intensive, and expensive.² To save money, they limit eligibility to a subpopulation based on demographic characteristics. The logic behind child abuse prevention programs that target a demographic subgroup is efficiency: if we can identify a subgroup that is known to be at high risk for maltreatment, then restricting intervention to that subgroup should save money because the intervention won’t be “wasted” on a low-risk group that would most likely have a healthy outcome even without costly intervention. This is a sensible strategy when three criteria are met:

1. The targeted subgroup includes a high proportion of the problem outcome cases;

2. Intervention can encompass a high proportion of this targeted group; and

3. Intervention effectively lowers the rate of problem outcomes in this group.

These criteria are not easily achieved in child maltreatment prevention programs. The first problem with targeting subgroups of participants in child maltreatment prevention is that risk exists across all demographic groups.³ Even though relative risk may be higher in some subgroups than in the rest of the population, risk still accrues in supposedly lower-risk groups. And because these lower-risk groups are larger, they account for most child maltreatment cases. This is an example of the “prevention paradox” first noted by epidemiologist Geoffrey Rose in 1981, in which the majority of cases of a negative outcome occur in populations at low or moderate risk for that outcome, because those at highest risk represent only a small portion of the population.⁴ In the case of child maltreatment, imagine a targeted subgroup (for example, low-income, first-time mothers) that constitutes perhaps 20 percent of the full population and has a two-fold higher risk for child maltreatment than the rest of the population. In this scenario, two-thirds of all maltreatment cases will occur in the nontargeted 80 percent that will never receive intervention. Even if an intervention with the targeted subgroup is highly effective (say, cutting maltreatment in half), the full effect would be to reduce population-level maltreatment by only 17 percent. Many problem cases will be missed if the intervention is restricted to one subgroup and the impact on the population as a whole will be small.

Because lower-risk groups are larger than higher-risk groups, they account for most child maltreatment cases.

The second problem is that intervention with targeted subgroups rarely has a high penetration rate (the proportion of all families in the targeted group who actually receive the intervention). Although randomized controlled trials (RCTs), in which a group of families who receive the intervention are compared to a control group of families who don’t, often report a high participation rate, families in such trials represent a unique subgroup of the targeted population—they are the ones who have provided prior written consent to get into the study. Nonconsenting families never get into the study, but they are still part of the targeted population. When targeted interventions are rolled out and scaled up in a community, the participation rate is typically lower than during the trials, a loss described as the “scale-up penalty” by Northeastern University criminologist Brandon Welsh and Nurse-Family Partnership developer David Olds.⁵ One reason for the low penetration rate is that stigma makes some families hesitant to participate. If the targeting factor is high risk for abuse, some families might not want to be identified as being in that group.

Another challenge in reaching all families in the targeted group is that funding rarely allows for saturation of the targeted population, partly because the cost would be prohibitively high.⁶ Penetration rates will drop even further because targeted programs rarely have the funds to saturate the eligible population. Even more worrisome, limited funding opens the door to cherry-picking participants so that the families that enroll in the program are at relatively low risk because they are high in compliance or motivation, and the highest-risk and most needy families are left out.

Another problem with interventions that target a small subgroup is that the intervention must have a large impact to achieve population-level outcomes. Although numerous targeted programs achieve a statistically significant impact compared with a control group, the size of the intervention effect may not be large enough to affect the full population.

Attractiveness and Challenges of Universality

The problems with targeted interventions we describe don’t mean that they fail to help a subgroup of families promote their infants’ healthy development. A targeted strategy has advantages, of course. Risk among the target group is indeed higher, by definition. Also, the content and focus of the intervention can be tailored to a relatively homogeneous group of families that may have common needs and may respond in a similar way. Further, not all families in a community are likely to need the long-term, intensive services that such programs provide. Some families may respond favorably to short-term programs or alternate interventions that meet their particular needs. The challenge is to reach all families in a population and then quickly triage and match interventions to a family’s clinically assessed needs. Rather than force communities to choose between universal and targeted intervention, the best strategy may be to embed targeted interventions in a universal strategy that reaches the entire population while offering intensive interventions for targeted subgroups. A useful analogy is our health care system for young children: pediatricians see children universally for well-baby visits, during which they screen and triage children to identify subgroups that could benefit from specialized services, such as neurosurgery, speech therapy, or ear tubes. Candidates for neurosurgery are not selected based on demographics but on clinically assessed need. Targeted interventions like the Nurse-Family Partnership are the neurosurgery of child abuse prevention, and universal approaches are the well-baby pediatric care. We need both to achieve a comprehensive system of developmental psychosocial-educational care.

A universal approach to early intervention has several advantages.⁷ Programs restricted to the poor are, rightly or wrongly, popularly considered “poor programs.” Unless participation brings outright cash payments, potential participants might may not join at high rates. Popular and political support is more likely if a program is offered to everyone in the community giving birth, rather than a select few, and such support increases the likelihood of funding. Over time, universal reach means that popular support is more likely to continue and funding is more likely to be sustained. Because of economies of scale, universal reach means that the per-family cost can be reduced and recruitment can be less complicated, less ambiguous, and less awkward and stigmatizing.

A universal goal leads to a very important shift in aspiration and focus. If an intervention is directed at an individual family or only a small subset of families, it necessarily prioritizes helping the family respond to a fixed system of community resources: such an intervention teaches a parent how to be first in line for the best child care facility, cash payments from government, and health services. The intervention makes no effort to improve the overall quality of childcare in the community or the efficiency with which families in general can connect to community resources. If an evaluation of the intervention’s effectiveness compares a small number of participating families to families in a control group, it is plausible that the intervention could be shown to improve the lot of these families. What happens when this program is brought to scale and every family tries to be first in line? Only when universality and population impact become the goals will program developers find it cost-effective to direct intervention not only at individual families but also—to encourage better-coordinated efforts to support children and families—at community agencies.

Whether child policy should be targeted or universal was the central question when public schooling was first considered two centuries ago. Some advocates argued that middle-class families would find ways to get tutoring and other schooling for their children even without government support, and that tax dollars should be conserved. Other advocates argued that universal public education would be of higher quality. Universal public education won out, of course. It’s difficult to imagine a well-functioning public education system that doesn’t include children from all backgrounds, even recognizing that many affluent families send their children to private schools.

Universal reach doesn’t necessarily mean a single program with a single funding source and mandatory attendance by all. Pediatric care again provides an analogy and an example of near-universal reach in a voluntary system in which cost and funding are challenges. In the United States, pediatric care is supported with multiple funding sources, both public (like Medicaid and CHIP) and private (like health insurance). For universal programs that aim to prevent child maltreatment and promote healthy development, funding will be an important issue, pushing clinicians to develop approaches that are both cost-beneficial and that minimize total cost to make the approach palatable in challenging financial and political times.

In addition, universal reach doesn’t mean that every family receives the same intervention program and the same dollar expenditure. Returning to the analogy of pediatric care, universal reach is achieved by matching every child with a primary care physician at birth. If families don’t voluntarily identify a pediatrician, the birthing hospital typically matches them with a provider, even if that means a clinic. Then the family is encouraged to attend a series of age-related well-baby visits that include physical examination and assessment; brief, universal, developmentally appropriate interventions (for example, “Have your baby sleep on her back”); brief, family-specific interventions (for example, discussion of breast-feeding and its challenges); and referral to specialists when problems are identified (for example, ear tubes for otitis media or an oncology referral for leukemia). Of course, not every child gets neurosurgery, which is reserved for the few whose clinical diagnoses show it is needed.

What Families Need at Birth

It would be fortuitous if developmental science could identify a single environmental or family factor at birth that predicts the majority of variance in important child outcomes. But empirical evidence indicates that many factors, ranging from family financial instability to parents’ mental health problems, provide unique incremental predictions. One factor, such as a mother’s substance abuse, might account for a maltreatment outcome in one family, whereas a different factor, such as financial instability, might account for maltreatment in another.⁸

The task becomes one of engineering: How do we screen all families at a single time point, such as birth, identify family-specific risks and needs for intervention, and connect each family with evidence-based community resources to address that family’s risk?

These findings suggest that no single intervention can successfully resolve risk in all families.⁹ Instead, different interventions will be necessary for different at-risk families. Some prevention programs rely only on demographic characteristics to identify a subgroup for targeted intervention (for example, low-income first-time mothers), but the evidence indicates that clinical characteristics such as maternal depression or parental substance abuse provide a stronger basis for targeted interventions.

The task becomes one of engineering: How do we screen all families at a single time point, such as birth, identify family-specific risks and needs for intervention, and connect each family with evidence-based community resources to address that family’s risk? This sort of targeted universalism has been pioneered in other areas. For example, it describes the state of optimal health care delivery in which a general pediatric practitioner sees every family, assesses child-specific risk, and then refers the child to a specialist, such as a urologist or surgeon who can deliver targeted intervention when needed. This model forms the basis for Family Connects.

The Family Connects Model

Family Connects aims to reach every family giving birth in a community so that it can identify family-specific risks and needs and then connect each family with the community resources to meet those needs, strengthen and enhance the parent-child relationship, and improve parent and child wellbeing.¹⁰ For communities to afford implementing the model universally, the per-infant cost must be modest. Costs are contained by guidelines that restrict the number of intervention contacts (in-person visits, phone calls, etc.) to a maximum of seven and by limiting the time period to the first 12 weeks (except in unusual circumstances, such as a long-term stay in a neonatal intensive care unit). The program isn’t a continuous intervention or a case management system. Instead, it consists of time- and cost-limited outreach in the spirit of a public health model. In implementations thus far, the total cost has averaged between $500 and $700 per infant birth, an amount we believe is affordable in communities where the cost of public education totals more than $8,000 per older child annually. Of course, funding for public education comes from a combination of local, state, and federal sources; we believe funding for public health approaches like Family Connects can also come from combined local, state, and federal sources if community leaders can figure out how to braid these resources. Because the program’s fixed-cost infrastructure expenses are relatively large, it can’t be implemented at modest cost with only a small subgroup of families in a community.

The program rests on three pillars. The first pillar, home visiting, is a system to reach all families giving birth in a community, typically at the birthing hospital. A trained public health nurse (or other program representative) welcomes the baby into the community. The nurses invite themselves to one or more home visits (when the baby is about three weeks old) so that they can identify needs and connect the family with community resources. The second pillar, community alignment, is an assembly of all community resources available to families at birth, including child care agencies, mental health providers, government social services, and programs for subgroups of families, such as Healthy Families, Parents as Teachers, and Early Head Start. The third pillar, data and monitoring, is an electronic data system that acts as a family-specific psychosocial and educational record (much like an electronic health record) to document nurses’ assessments of mother and infant, as well as connections with community agencies.

Nurse Home Visits

Nurses or program representatives greet the mother at the birthing hospital to congratulate the family and to welcome the baby into the community. They deliver the message that the community wants to partner with the parents to support their child’s long-term success in health, education, and wellbeing. They also tell the parents that research shows every parent can be successful but that at the same time, every parent can benefit from support. In that spirit, the nurse would like to visit the parents in their home when the infant is about three weeks old to understand and help with their family-specific needs. The visits are flexibly timed to avoid disrupting community standards of care, such as well-baby visit schedules. The nurse also aims to accommodate family situations, for example, by delaying a visit because of extended stays by relatives or by speeding up a visit to attend to urgent needs. The nurse promises to bring “goodies,” such as free diapers, to make the visits more attractive.

Ideally, both the mother and her partner (usually the father) are present during the visit, although the nurse takes the mother’s lead in deciding whether the partner’s presence is appropriate. During one pilot of Family Connects, the nurses assumed that partners would participate and went to great lengths to invite them. But this strategy sometimes led the mother to withdraw, presumably because of conflict between the parents or difficulty in scheduling a convenient time. Because of this, the protocol was changed: now the nurse listens to the mothers’ advice on whether to include their partners, who participate about half the time.

The initial home visit typically lasts between 90 and 120 minutes. The nurse is trained to conduct a structured clinical interview that includes several hundred scored items and covers a diverse set of topics in a conversational tone. The topics aren’t covered in a preset order; typically, the nurse follows the mother’s interests. The oral interview responses are supplemented by standardized screening for particularly sensitive or high-risk circumstances, such as substance abuse. The nurse assesses risk in 12 key domains (see table 1) that predict adverse outcomes among children. Consistent with an ecological approach to health and wellbeing, these domains encompass not only the needs of individual family members, but also the family’s needs within its environment.¹¹ The domains include child characteristics, such as temperament or health risk, which may make an infant more challenging to care for; family characteristics, such as parents’ substance use, parent-child relationship quality, or household safety; and community characteristics, such as neighborhood violence or access to resources.

Table 1.

12 Risk Factors Assessed and Scored in Family Connects Nurse Home Visits

Domain	Risk Factor
Parenting and child care	Child care plans Parent-infant relationship Management of infant crying
Family violence and safety	Material supports Family violence Mother’s past experience of being maltreated
Parent mental health and wellbeing	Depression and anxiety Substance abuse Social and emotional support from others
Health care	Parent health Infant health Health care plan, medical home, and insurance

Risk in each domain is scored quantitatively on a simple four-point scale: 1 indicates no risk; 2 indicates risk that can be resolved through a brief intervention by a nurse (for example, if the parent has no knowledge about how to select out-of-home childcare, the nurse educates her and helps her identify childcare plans); 3 indicates significant ongoing risk that requires a connection with a community resource to resolve, such as a mother’s substance abuse problem requiring professional treatment; and 4 (used in less than 1 percent of cases) indicates an emergency requiring crisis intervention (for example, imminent risk to the infant’s health or infant maltreatment).

At the end of the interview, the nurse summarizes the findings with the parents, and together they develop a course of action based on the needs identified. The plan may include follow-up home visits, phone calls, or contact with external agencies. This approach encourages parent buy-in and protects them from stigma in several important ways. First, the course of action is grounded in the needs identified by the parents and nurse during the home visit, rather than presumed needs based on family demographic characteristics. The experience is similar to, say, having high blood pressure identified during a routine physical and working with a doctor on a course of action to treat the problem. Second, because the plan is collaborative rather than directive, the parents are active participants in determining what’s best for their family.

After developing a course of action and gaining parental consent, the nurse communicates in writing with the infant’s pediatrician and the mother’s primary care provider to create a bridge to ongoing care after the nurse’s work ends. Four weeks after closing the case, the program makes a follow-up phone call to check on the family’s progress and to determine whether referrals to community agencies were successful.

When families report an unsuccessful community connection, Family Connects staff members either make another attempt to get the family and the agency to connect (if desired) or to help the family access alternative community resources. If an agency has high rates of missed connections, program staff also work with it directly to reduce systemic barriers.

Alignment with Community Agencies

To make efficient referrals, nurses need to understand the array of community agencies that serve families with young children. The second pillar of the program is alignment with these agencies. A Family Connects staff member reaches out to as many agencies as possible to recruit their participation; to document agency goals, service capacity, eligibility criteria, fees, wait-list time, and evidence of effectiveness; and to assemble agencies into an annotated electronic directory for the nurses to use. In Durham, NC, for example, the program’s directory includes over 400 agencies, ranging from childcare facilities and volunteer parent groups to professional intervention services. Staff prioritizes identifying evidence-based programs. And although in many communities the array of agencies includes a wide variety of government and professional services, as well as nonprofits, identifying informal resources, such as faith-based giving closets or food pantries, is equally important, especially in underfunded and rural communities. In communities with fewer formal resources, or communities in which formal services lack high quality, informal resources may be a critical source of support for families with young children.

Coordination with community agencies is helped along by one (or more) advisory boards. One board consists of community leaders and agency directors who guide program direction, and a second board consists of frontline agency workers who work to solve problems, such as long wait lists and misunderstandings, as they arise.

Integrated Data System

Family Connects staff document all their work in a family case record created from birth reports. The record begins with attempts to schedule a home visit; it also includes the nurse’s clinical interview, screening instrument responses, scoring of risk, referrals made to community agencies, interventions received at the agencies, and parents’ “consumer satisfaction” responses to follow-up calls.

The case record serves three important functions. First, it’s a key clinical tool that integrates information from many sources to guide intervention decision-making. The family-specific case record helps the nurse match family needs with known evidence-based interventions in the community. It incorporates information from the agency, such as whether the family entered intervention and made progress. Much of this information is communicated to the infant’s pediatrician at the end of the program. However, all information sharing requires the parents’ consent, so that they remain in control.

Second, the case records are scrubbed of identifying information and aggregated to provide a summary about each community agency. If the case records include almost all births in a community, these agency records will be fairly comprehensive. They can be used to document agency service to families, the quality of that service, and parents’ satisfaction with the agency. Family Connects uses the aggregated information to communicate with agencies about their performance and to solve problems like chronic long waiting lists or excessive dropouts.

Third, the family case records are aggregated to create population-level indicators of family needs, and the agency records are aggregated to map community assets and resources. The collective family-needs information is then matched with the aggregated community resources to identify gaps in a community’s ability to serve all its families. For example, in one community the case records indicated that 11 percent of all birthing mothers had a substance abuse problem meriting external intervention. But the asset map indicated that the community had resources to serve only 3 percent of mothers. The program passed this information on to county commissioners, who were able to fund additional substance abuse treatment for mothers who needed to be home with a newborn.

The aggregated information also provides population-level indicators of family functioning at birth and can be used to track community progress across cohorts of parents and children. In this way, the integrated data system not only serves the needs of individual families and agencies but also advances public health.

Training, Supervision, and Fidelity

The Family Connects program requires training of each staff member, certification of nurses in the protocol, ongoing supervision, and documentation that implementation adheres to the program model (known as fidelity). After qualified nurses are hired, their training begins with reading detailed manuals. Then they observe expert nurses during home visits and conduct practice home visits that are observed by a supervisor. They are tested on how well they adhere to the way questions are posed to parents and how reliably they score risk in each of the 12 domains. Finally, they receive certification that they’re ready to implement the program. Nurses from remote areas travel to the national site, in Durham, NC, for several days of training. Back in their home communities, an onsite nurse supervisor works with a national site overseer.

Nurse supervisors conduct quarterly fidelity checks with all home visiting nurses at their site, in which the supervisor accompanies nurses to parents’ homes and evaluates adherence to 62 components of the visit. These items include properly administering health assessments for the mother and baby; completing all assessment queries across the 12 domains of family risks and needs; teaching about infant care and safety; and developing of a course of action with the family. Supervisors also evaluate family risk in each of the 12 domains and compare their ratings with those of the home visiting nurse to ensure consistency. After initial scoring, all records are stored electronically at the national office in Durham so that rates of adherence to the model can be tracked over time both within and across program sites.

Nurse supervisors conduct quarterly checks with all home visiting nurses, accompanying the nurses to parents’ homes and evaluating adherence to 62 components of the visit.

Family Connects Evaluation

To be eligible to receive funds from the Maternal, Infant, and Early Childhood Home Visiting (MIECHV) program, or to become known as “evidence-based,” a program must undergo a rigorous RCT to demonstrate its impact when implemented in a research setting, and we report findings from two such community-level RCTs. Less commonly completed but just as important is an evaluation of how well a program is implemented and how much of an impact it has when it’s scaled up and brought to new communities. We offer such a field evaluation of Family Connects.

In these three trials, we were interested first in whether the program could be implemented at scale with high quality. Indicators of quality included:

• penetration rates, that is, the proportion of all families giving birth that were recruited into and successfully completed the program;

• fidelity of implementation, defined by the portion of families for whom quantitative scores were completed for nurses’ adherence to the protocol (as a percentage of all possible items) and the reliability of nurses’ scoring of risk factors (computed as chance-corrected agreement with an independent supervisor’s scoring of the same protocol), and;

• connection rates, defined as the portion of all families for whom need was identified, the portion of all families for which an external agency connection was proposed, and then the portion for whom a successful connection was established.

We were also interested in impact, which was determined by comparing outcomes for intervention families to those for control families in the domains of connectedness to the community, parenting quality, parent mental health, infant maltreatment, and use of emergency health care.

First RCT

The first RCT included all 4,777 resident children born at the two hospitals in Durham County, NC, between July 1, 2009, and December 31, 2010.¹² About 40 percent were European American, 37 percent were African American, and 23 percent belonged to other groups or were multiracial. Twenty-six percent reported Hispanic ethnicity, 62 percent received Medicaid or had no health insurance, and 56 percent were not married.

Families of babies born on even dates were assigned to receive the intervention and those born on odd dates were assigned to be controls. The evaluation was based on intent-to-treat status, meaning that nonparticipants were included and that recruitment into the intervention was part of the intervention itself. The intervention began at birth and almost always ended by the time the child was 12 weeks old. For a subset of analyses, interviews and in-home observations of parenting and the home were completed when the infant was about six months old, at least three months after the Family Connects program had been completed. To contain costs, a randomly selected, representative sample of 664 intervention and control families were recruited for data collection; 549, or 80 percent, agreed to participate. In addition, administrative records were retrieved from emergency departments for counts of emergency room visits and overnight stays in a hospital, as well as from Child Protective Services for counts of child maltreatment investigations and substantiations.

Evaluation of Implementation

Penetration rates.

As table 2 shows, of the 2,327 families that gave birth on an even date, 1,863 (80 percent) were successfully recruited into the program, which required that they listen to the goals and framing of the program, interact with a Family Connects staff member (usually at the birthing hospital), and schedule an initial home visit. Of the recruited families, 86 percent successfully completed the program, including receiving referrals to community resources when appropriate, yielding a population-wide full completion rate of 69 percent.

Table 2.

Implementation and Impact Findings across Three Trials

	RCT I	RCT II	Field Trial
Population	All resident county births at two Durham County, NC hospitals (July 2009–Dec. 2010)	All resident county births at one Durham County, NC hospital (Jan. 2014–June 2014)	All resident county births in four low-income, rural counties in eastern NC (Feb. 2014–Dec. 2015)
FC Implementation sample	2,327	456	994
% minority	70%	72%	67%
% Medicaid or no health insurance	61%	68%	69%
Penetration rates
Initial participation rate	80%	76%	84%
Of those initiating, completion rate	86%	84%	77%
Program completion rate	69%	64%	64%
Fidelity
Adherence to protocol	84%	90%	87%
Agreement in assessing risk factors	k=.69	k=.75	k=.78
Community connection rates
% of families with at least one referral	45%	42%	54%
% of referred families for initiating service	79%	83%	94%
Impact
Connectedness
Number of community connections	I > C**	I > C**	I > C**
Parenting and parent mental health
Number of positive parenting behaviors	I > C**	I = C	I = C
Father-infant relationship quality	I = C	I = C	I > C**
Maternal anxiety/depression rate	I < C**	I < C**	I = C
Infant health and wellbeing
Number of emergency care episodes
Birth to 6 months, administrative records	I < C**	I = C	Not assessed
Birth to 6 months, mother’s reports	I < C**	I = C	I < C**
6–12 months, administrative records	I < C**	I = C	Not assessed
Birth to 24 months, hospital records	I < C**	I > C**	Not assessed
Child protective services investigations	I < C**	Not assessed	Not assessed

^*Cohen’s kappa coefficient is a statistic that measures agreement between raters after accounting for the possibility of agreement by chance. Kappa values greater that 0.60 are considered to be substantial.

^**Intervention group (I) average differs from the control group (C) average. There is a greater than 95% likelihood that this difference is not due to random chance or error (p<.05).

Fidelity of implementation.

Adherence to the protocol manual is essential for program quality. In this trial, an independent quality-control expert accompanied nurses on 116 randomly selected home visits and independently documented whether a nurse correctly completed each of 62 model elements. Overall, nurses adhered to 84 percent of all elements, which is considered high. The quality-control nurse also independently scored each of the 12 risk factors to ensure consistent family risk assessment. Because each of the 12 factors was rated on a scale with only four possible entries (1 through 4), the expert and the home visiting nurse might have had a high rate of agreement on factors score by chance, as they would be expected to agree some of the time simply by guessing. To overcome this problem, the nurses’ reliability in assessing risk was evaluated using Cohen’s kappa coefficient, a statistic that measures agreement between raters after accounting for the possibility of agreement by chance.¹³ Kappa values greater than 0.60 are considered to be substantial. For the current trial, chance-corrected agreement across the 116 observed home visits was 0.69.

Risk and connection rates.

Ninety-four percent of all families were scored as having at least one need that merited intervention. In 1 percent of families, the need was a crisis requiring immediate emergency intervention; 49 percent had modest to moderate needs that the nurse could resolve through additional home visits and brief counseling (for example, through breastfeeding consultation or education about how to find high-quality child care); and 44 percent had serious needs requiring connection with a community resource, such as substance abuse treatment, depression treatment, or social services. One month after the nurse terminated her involvement with the family, Family Connects staff telephoned each family to find out whether they had successfully made a community resource connection. Of the families referred to a community resource, 79 percent reported they had indeed followed through to initiate the connection.

Satisfaction with Family Connects.

During the phone call, family members were asked whether they would recommend Family Connects to another new mother, and 99 percent said yes.

Evaluation of Impact

To evaluate the impact of Family Connects, interviewers visited a representative subset of intervention and control families when the infant was six months old. To avoid potential bias, interviewers weren’t told which families had been eligible to receive Family Connects, and participating families weren’t told that the primary study goal was to evaluate Family Connects. Consent was obtained to access administrative records, and hospital and Child Protective Services records were accessed then and later. Findings are summarized in table 2.

Connectedness.

Six months after the birth, intervention mothers reported 16 percent more community connections than did control mothers.

Parenting and parent mental health.

Intervention mothers reported more positive parenting behaviors than did control mothers, and a higher-quality father-infant relationship. Screening indicated that intervention mothers were 28 percent less likely than control mothers to exhibit signs of clinical anxiety.

Infant health and wellbeing.

Compared to control mothers, intervention mothers reported 35 percent fewer serious injuries or illnesses among their infants that required emergency department care or hospitalizations. Administrative records from the two community hospitals indicated that from birth to age six months, intervention infants had 59 percent fewer emergency medical episodes than did control infants. Between six and 12 months, intervention infants had 31 percent fewer such episodes.

Follow-up analyses examined whether Family Connects had a positive impact for various subgroups. Even if it had a favorable impact on the population as a whole, it might have had no impact on some groups of families. Instead, the additional analyses, which focused on emergency care episodes through age 12 months, found positive impact for every group studied. Though both groups benefited, infants with one or more birth risks had stronger intervention impact, defined as the difference between intervention and control infants, than did infants with no birth risks. Similarly, infants with Medicaid or no insurance experienced stronger intervention impact than infants with private insurance. The program’s impact was stronger for majority than for minority families, and stronger for boys than for girls, but in both cases significantly positive for each group.

Recent analyses have explored the program’s impact on children’s involvement in the Child Protective Services system over the children’s first five years.¹⁴ After accounting for demographic risk factors, including birth complications, Medicaid status, minority status, and single parent status, results indicate a 39 percent reduction in the rate of total child protective service investigations for suspected child abuse or neglect (see figure 1).

Figure 1.

Child Protective Services Investigations through Age 60 Months: RCT I.

Second RCT

Often, early-years intervention programs initially show positive impacts that aren’t replicated when a second independent trial is conducted. We initiated a second RCT in Durham in 2014.¹⁵ Because one of the two original hospitals changed its policies and now prohibits patient contact with non-hospital staff, births at this hospital were excluded from the new trial. As a result, the trial was conducted with all resident births at one hospital from January to June of 2014. This time, the 456 families giving birth on an odd date were assigned to the Family Connects intervention, and the 479 families giving birth on an even date were assigned as controls. Interviewers completed outcome assessments with a subsample when the children were four to eight months old. Again, to avoid potential bias, interviewers weren’t told which families were eligible to receive Family Connects, and participating families weren’t told that the primary study goal was to evaluate Family Connects.

Evaluation of Implementation

Penetration rates.

As table 2 shows, of the 456 families who gave birth on an odd date, 349 (77 percent) were successfully recruited into the program; 84 percent of these successfully completed the program, yielding a population-wide completion rate of 64 percent. These figures are slightly lower than those for the first RCT, but they show that the program reached a large percentage of families across trials and time.

Fidelity of implementation.

Nurses adhered to 90 percent of all elements, considered very high. And when the quality control nurse scored each of the 12 risk factors, chance-corrected agreement as assessed by Cohen’s kappa was found to be high at 0.75.

Risk and connection rates.

Ninety-five percent of all families had at least one family need meriting intervention, according to nurses’ scores. One percent required emergency intervention, 52 percent had modest to moderate needs that could be resolved by the nurse, and 42 percent had serious needs requiring connection with a community resource. In follow-up phone calls, 83 percent of the families referred to a community resource reported that they had initiated the connection.

Evaluation of Impact

Connectedness.

As in the first trial, intervention mothers reported more community connections than did control mothers—in this case, 17 percent more.

Parenting and parent mental health.

Unlike in the first trial, in which intervention mothers fared better, in the second trial intervention and control mothers had similar levels of positive parenting behaviors and father-infant relationship quality. As in the first trial, intervention mothers were less likely than control mothers to exhibit signs of clinical anxiety—in this case, 20 percent less likely.

Infant health and wellbeing.

Unlike the first trial, in which intervention infants fared better, in the second trial intervention and control infants had similar rates of serious, emergency medical care episodes between birth and six months. In the first trial, the intervention group had a mean of 1.5 episodes per family by 24 months of age, and the control group had 2.4. In the second trial, the intervention group’s mean was 1.1, which was lower than the mean in the first trial. Yet the control group mean was lower still, at 0.9 episodes. We have no explanation for the precipitous drop in these episodes among the control group. Involvement with child protective services hasn’t yet been evaluated for the second RCT.

Field Trial

The first two RCTs were conducted in Durham, NC, where the program was developed. It’s plausible that implementation quality could be higher at this site than in other places, and that its impact elsewhere could be lower.¹⁶ We sought to complete a rigorous evaluation of the program’s implementation and impact when it was brought to new sites and implemented by local staff members.¹⁷

After winning an Early Learning Challenge Grant from the US Department of Education’s Race to the Top program, North Carolina allocated funds to Family Connects. Beginning in 2014, Family Connects was introduced in four low-income, rural counties in northeast North Carolina (Beaufort, Bertie, Chowan, and Hyde). Conducting an RCT didn’t meet the grant’s goal of delivering services to all children and families in the four counties, so we evaluated program impact through what’s called a natural comparison design. Specifically, we compared outcomes for families of infants born from February 1, 2014, through July 31, 2014—before Family Connects came to their county—with outcomes for families of infants born during implementation, from September 1, 2014, to December 31, 2015. To reduce potential participation and response bias, all intervention group families participating in the impact evaluation were recruited without regard for their participation status. The evaluation was also double-blind: families didn’t know that the survey’s primary goal was to examine how Family Connects affected child and family wellbeing, and interviewers didn’t know which families actually had completed the Family Connects program. Because we were comparing families of infants born in different time periods, we had to consider the possibility that time-related factors, such as the state of the global economy, could account for any differences that would otherwise be attributed to Family Connects.

The four rural counties in the trial had relatively few institutional community resources upon which nurses could draw. However, the community alignment organization phase of the implementation revealed many informal resources, such as an intervention program administered by the Rotary Club, social groups at the Veterans of Foreign Wars, and church-related support. Because nurses had to drive great distances to some families’ homes, we anticipated that the program would have a lower penetration rate. Thus, the field trial presented an important opportunity to advance public health, public policy, and early childhood home visiting by evaluating the dissemination of a low-cost, universal home visiting program to rural communities characterized by very low resources and chronic poverty.

Evaluation of Implementation

Penetration rates.

As table 2 shows, of the 994 families with a child born during the implementation period, 770, or 77 percent, were successfully recruited into the program. Of these families, 83 percent successfully completed the program through the step of making connections with community resources, yielding a population-wide full completion rate of 64 percent. These figures are similar to those for the first two RCTs, again showing a high penetration rate across trials and time.

Fidelity of implementation.

Nurses adhered to 87 percent of all elements, and chance-corrected agreement in scoring risk using Cohen’s kappa was high at 0.78.

Risk and connection rates.

Nurses scored 99.5 percent of all families as having at least one need meriting intervention. Less than one percent of families required emergency intervention, 45 percent had modest to moderate needs that could be resolved by the nurse, and 54 percent had serious needs requiring connection with a community resource. In follow-up calls, 83 percent of the families referred to a community resource reported that they had initiated a connection.

Evaluation of Impact

The comparison group of 343 infants born from February 1, 2014, to June 30, 2014, was close in demographic and community characteristics to the Family Connects group born from September 1, 2014, to December 31, 2015

Connectedness.

As in the first two RCTs, intervention mothers reported greater community connectedness than did control mothers.

Parenting and parent mental health.

Intervention and control-group mothers reported similar levels of positive parenting behaviors, but father-infant relationship quality was significantly higher for intervention families. No significant differences were observed for possible clinical anxiety among mothers, although intervention mothers reported 18 percent lower levels of possible clinical depression.

Infant health and wellbeing.

The Family Connects intervention had consistent, positive impacts on whether infants received emergency medical care. Specifically, intervention mothers reported that between birth and infant age six months, they had sought emergency medical care for their infants 25 percent less often.

Dissemination, Innovation, and Policy Engagement

Family Connects recently established a national office to support three missions: broad dissemination, research and innovation, and policy engagement.

Dissemination Challenges

The Family Connects model is now being disseminated in over two dozen communities across the United States. These communities range from midsize cities to small rural communities. Almost all of them requested that Family Connects come to their community, rather than responding to marketing. Recently, though, the program has adopted a new strategy of reaching out to selected communities to help them consolidate community support for Family Connects and find sustainable funding.

A major challenge of most of these communities is finding a way to sustain funding over time.

Communities have found financing for the Family Connects program in many ways. Sources include local public funding, state grants, private philanthropy, for-profit health care organizations, Medicaid reimbursement, and federal funding awarded to states through MIECHV. Most communities use funding from multiple sources.

A major challenge for most of these communities is finding a way to sustain funding over time. Philanthropic funding, especially, is typically awarded in the short term or year to year, making long-term planning difficult and slowing the process of getting community agencies to buy in to the communal effort. One strategy is to take advantage of numerous federal funding mechanisms, such as the new Family First Prevention Services Act. The long-term answer, though, may be policy change at the federal and state levels to make funding for early childhood programs as much of a priority as it is for later childhood and elder care.

Research and Innovation

The Family Connect program’s second mission is research and development. All dissemination sites are required to evaluate the implementation as part of initial certification and ongoing monitoring, and plans are under way to aggregate data across sites to understand natural patterns in implementation quality over time and across sites. Coupled with this is a new study of implementation cost that could help explain variation in quality. Each new site presents an important opportunity for impact evaluation and continued learning. Rigorously designed impact evaluations won’t be possible at all sites, but alternate designs can be used, and several sites could offer opportunities for new RCTs.

Continued evaluation faces challenges, however. First, communities often lack funding to conduct rigorous evaluation. Second, their willingness to do so may also suffer due to difficulties with partners or funders, or the fear that negative findings could result in loss of funds for services that they believe are helping families in their community. Ultimately, we need the political will to increase funding to support continued implementation of additional program replication trials, to require evaluation as a condition of receiving funds for new programs, and to establish evaluation as a tool for continued learning and quality improvement.

Some innovation plans derive from findings from the three trials already conducted. For example, positive impact on fathers was found in two trials but not the third; thus one innovation will be to focus more on fathers and evaluate what happens rigorously.

At several sites, innovation is related to topics that a community wants to prioritize. For example, one community received funding as part of a broader effort to improve trauma-informed services, that is, services for children and families that have experienced various forms of trauma; in this community, a module of enhanced training of staff members, assessment of trauma, and intervention is being planned. This additional component will be layered on top of Family Connects so that implementation isn’t compromised. Other topical modules may target nutrition and early literacy.

Another innovation—conducted collaboratively with other nationally known home-visiting models—is an effort to apply a similar rationale for universal screening in the prenatal and postnatal periods. And one more goal is to understand how Family Connects can collaborate more closely with pediatric care.

Policy Engagement

The concept of universal reach at a critical point in the lifespan, coupled with assessment and individualized connection with community resources to support a young child’s healthy behavioral development and to prevent child maltreatment, has a long way to go before it becomes routine for every family in the United States. To push this movement forward, program developers and researchers will need to work with policy makers at the local, state, and federal levels. They need to understand what drives policy so that they can frame their case in a way that reaches attentive ears. They will need to conduct benefit-cost analyses and analyses of financing models, and they will need to keep providing rigorous empirical evidence about program implementation and impact. Finally, they will need to understand how state and federal funding streams, such as Medicaid, MIECHV, and the Title V maternal and child health services block grant program, may be used to help communities make such programs financially sustainable.

Conclusions

The story of Family Connects offers promise for population-level impact on the prevention of child abuse and neglect. But the history of innovation and rigorous evaluation in this field shows a mixed pattern: some implementations are positive, and some are disappointing; some evaluations show positive impacts, and some show none. To make true headway, evaluators and program developers need to acknowledge the full range of findings about a program’s implementation and impact, and they must embrace continued research rather than cutting it off after favorable findings emerge. At the same time, funders of research and implementation in the public, philanthropic, and private sectors should exercise patience, thus easing any pressure to produce favorable findings. With a committed partnership between programs and funders, we are optimistic that we’re not far from achieving population impact in reducing child abuse and neglect.