Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 Dec 1.
Published in final edited form as: Am Psychol. 2018 Dec;73(9):1117–1129. doi: 10.1037/amp0000393

Toward Population Impact From Early Childhood Psychological Interventions

Kenneth A Dodge 1
PMCID: PMC6416783  NIHMSID: NIHMS1012438  PMID: 30525790

Abstract

Acting alone, psychologists rarely achieve population impact on important mental health and well-being outcomes for families and young children. The traditional Institute of Medicine model of moving from efficacy trials to effectiveness trials to scaling up has not succeeded, partly due to degradation of program quality and impact during scale-up and partly due to a failure to consider system-context issues at the outset. Analysis of barriers to population impact leads to the proposal of a new comprehensive system of care that includes both top-down coordination among community agencies providing services and bottom-up outreach to every family to connect them with services. The North Carolina Smart Start Initiative is a top-down approach to improving the community-level quality of early childcare and education services. A natural experiment demonstrates that it improves population indicators of children’s education outcomes. Family Connects is a bottom-up approach that reaches all families giving birth in a community through brief home visits to assess needs and connect families with community resources. A randomized controlled trial reveals increased community connectedness, lower maternal anxiety, reduced emergency health episodes, and lower rates of investigations for child abuse. These initiatives point toward the promise of population impact through psychological interventions in early life that are delivered in a collaborative system of care.

Keywords: prevention, children, population health, intervention


Developmental psychologists now know that the first 3 years of life witness an explosion of millions of new synaptic connections per second, laying the foundation for an infant’s working models of social relationships and conditioned responses that guide future experiences, behaviors, and outcomes. They have also discovered that self-regulation of impulses and executive function develop rapidly over the following 3 years when brain energy peaks. Prevention psychologists have translated these discoveries into scientifically tested interventions for young children and families that improve behavioral and emotional development. But these discoveries have not yet achieved population improvements in the health, education, and well-being of our youngest children (Shonkoff, 2017). As shown in Figure 1, the Duke Child Well-Being Index (Land, 2017) indicates modest American population-level declines in children’s health, well-being, and social relationships over the past 40 years. Educational attainment has been flat, and family economic well-being has fluctuated with the overall economy. Furthermore, and embarrassingly, the United States has experienced growing (not diminishing) disparities in health, education, and well-being across income levels over the past 40 years.

Figure 1.

Figure 1.

Selected indicators of child well-being over time, from the Duke Child Well-Being Index, 2017 (Land, 2017). See the online article for the color version of this figure.

The slow progress has not been for lack of federal public spending on children, which has increased to a level exceeding $500 billion a year, mostly on health care, cash assistance, and early care and education (Dodge & Haskins, 2015). Spending on children living in poverty has also increased to over $14,000 annually per low-income child (Chaudry, Morrissey, Weiland, & Yoshikawa, 2018). Nonetheless, Land’s (2017) findings show we are making insufficient progress in improving our children’s health and well-being and virtually no progress for children living in poverty.

The dual challenges of improving overall health and well-being while reducing income-group disparities are particularly acute during early childhood. The National Household Education Survey (Child Trends, 2015) indicates only modest average gains in children’s kindergarten readiness skills over the past 25 years and huge and continuing disparities by poverty status, parents’ education level, and race and ethnicity. Over the last 20 years, the rate at which infants are born small for their gestational age has actually increased, and the increase is greatest among infants born to African American women (Catov, Lee, Roberts, Xu, & Simhan, 2016).

Psychologists have discovered empirically supported ways to intervene to help families with very young children, but critics have lamented the failure of these interventions to improve population rates of childhood mental health and well-being, and they call for a different order. Kazdin and Blasé (2011, p. 21) stated, “Despite advances, mental health professionals are not likely to reduce the prevalence, incidence, and burden of mental illness without a major shift in intervention research and clinical practice.” Weisz, Krum- holz, Santucci, Thomassin, and Ng (2015) suggested that to shrink the gap between large laboratory-based effect sizes and small real-world impact, psychologists will need to shift their focus to real-world settings and population indicators.

In this essay, I analyze our failures and propose that psychologists join with policymakers to create a new comprehensive system of care for children birth-to-5 that combines a top-down approach of improving community capacity to serve families with a bottom-up approach of reaching every young child to assess child-specific needs and to connect them with targeted community resources. Just as our health care system provides primary care through well- baby visits to the pediatrician and tertiary care when needed through specialists, we need a psychosocial health care system that provides universal reach and assessments at touchpoints across the early life span with connections to specialized resources when needed. Just as health care has become coordinated through the advent of the portable electronic health record, I propose a cumulative electronic psychosocial record for each young child that begins prenatally and becomes merged into the school record. I report initial empirical studies that evaluate the impact of these systemwide approaches and conclude with optimism about the potential of psychological interventions to “move the needle” on population outcomes for young children and families; that is, if policymakers and psychologists buy in to a new system of psychosocial care.

Why and How Have We Failed?

Scale-Up Failure

Following the Institute of Medicine model (Mrazek & Haggerty, 1994), psychologists often assume that progress moves unidirectionally from laboratory science to small randomized controlled efficacy trials to community-based effectiveness trials to impact through community-wide scaling up. This model has not succeeded. We now know that volunteer children studied in the laboratory or even in controlled community environments do not represent the larger population, challenging the ecological validity of the science itself. When we try to scale up these interventions in communities with different populations, participation rates (called penetration rates) fall, and access to programs is biased toward advantaged families who know how to work the “system” but have less to gain (Chaudry et al., 2018).

Furthermore, program quality often gets degraded when programs are scaled up because per-child funding levels go down and fidelity to the original intervention model is compromised. Welsh, Sullivan, and Olds (2010) have coined the term scale-up penalty to refer to the combined decrements in impact that accrue when we scale up programs that had been hatched in the psychologist’s university laboratory. Simply put, the scale-up model is not succeeding.

Poor Incentives

One reason that this strategy persists but fails is that policymakers and funders have evaluated interventions using incremental rather than transformational metrics. We fund programs at such low levels that population impact is never possible, but we claim success because a program has been put into place. We reward based on immediate improvements in getting closer to a goal rather than true progress. If we had used that criterion to incentivize NASA in its race to put a person on the moon, we would have rewarded climbing mountains.

In children’s services, our emphasis on immediate improvement leads us to focus on short-term “quick fixes,” such as school dropout prevention programs for failing high schoolers rather than programs that help parents prepare their child for kindergarten, and homeless shelters rather than affordable housing. When we do provide funding, it often comes as a modest pilot project for a small number of families just to show something has been put into place. This focus on immediate incremental improvement sometimes actually worsens the long-term problem by allowing us to ignore the population need, and it has made effective interventions subject to the whims of politics and funding fads because the core infrastructure has not changed. Our research funding streams suffer similarly: They require immediate success, which inhibits creative ideas.

Family Context Matters

By starting in a university laboratory, psychologists sometimes develop interventions without sufficient regard to a young child and family’s working and community context. Some of this myopic perspective is a carryover from the 1960s when legacy interventions for very young children, such as Abecedarian (Campbell & Ramey, 1994) and Perry Preschool (Schweinhart et al., 2005), were tested with small numbers of stay-at-home low-income mothers. These ground-breaking infant home-visiting programs deserve our admiration but were created in an era when most low-income mothers stayed home and were available to receive a home visitor every week. The Perry Preschool Project explicitly excluded mothers who planned to work outside the home because its home visitors required that visits occur during daytime hours. Today, almost all mothers, including low-income mothers, are trying to work outside the home. Sixty-one percent of all children under age 5 in the United States are in nonparental child care, and they average 36 hr a week in this care (Chaudry et al., 2018, p. 43). Many programs to improve a child’s parenting have not kept pace with trends in family ecology and are therefore constrained in their ability to achieve population impact.

Peer Context Matters

Many laboratory-based individual-focused interventions fail to appreciate the dynamic nature of a child’s social interactions. Consider interventions based on the developmental science finding that a child’s academic and behavioral functioning is affected by the functioning level of a child’s peers (Justice, Petscher, Schatschneider, & Mashburn, 2011). A young child placed with peers who are slightly more advanced tends to benefit, whereas the same child, if placed with peers who are less advanced, tends to decline in achievement. If savvy parents place their children with like-minded well-functioning peer families in segregated preschool settings, and policies require separate (but equal?) Head Start classrooms for low-income children only, the effect will be better outcomes for the savvy families and increased disparities across groups.

Resource Context Matters

Psychologists who implement family interventions usually assume a fixed sum of available community resources. As a clinical psychologist working with Head Start parents years ago, I could not control funding levels to improve service capacity, so I turned my attention to teaching a parent how to navigate the existing system of resources, that is, how to be first in line for the best quality summer childcare program and how to apply for housing subsidies before funding runs out. Evidence-based parenting programs often follow this “fixed-sum assumption.” If we tried to scale up these programs without improving the overall community resource context, they would implode with no net gain in population impact.

A Need to Change the System

One concludes that to reach population impact, we need to “change the system” as much as we need to change families. But what system are we talking about? There is no organized system of psychosocial care for children in their early years. Tolan and Dodge (2005) pointed out that most young children’s mental health needs go unnoticed because no organized system exists to identify, treat, or prevent mental health problems. So, too, it is with psychosocial and educational needs. Children come to kindergarten with vastly different levels of academic and behavioral readiness because there is no organized system to prepare them. The lack of a system contributes to lower mean levels of population well-being and large disparities across groups.

We do have a comprehensive system of care for children from kindergarten to age 18, called public schools, funded by tax dollars. If a child has head lice, the school nurse will detect the problem and facilitate treatment. If a child is abused, the school social worker will notice and investigate. If a child cannot read, the schoolteacher assesses and remediates. Schools provide nutrition and afterschool care. The public school system is not without problems, of course. Critics lament its ineffectiveness and mismanagement, but few serious critics propose doing away with public schooling altogether. We also have a comprehensive system of care for the elderly, through Medicare and Social Security, also funded by tax dollars.

Between birth and kindergarten, we have no system of care at all. We do not even know who they are. We know how many children are born based on birth records, and we know how many children begin kindergarten; but they are lost in a black hole in between those years. We need to create a system of tracking and caring for all children in their early years.

Our lack of a system of care for young children is not merely an oversight; rather, it is based on historically rooted cultural beliefs and outdated developmental science. Our culture has long assumed that families should, can, and will provide the best care for their own young children and that the community should stay out of the family’s business. Developmental science used to reinforce this cultural belief with now-outdated assertions that only a stay-at-home mother can offer an infant what that infant needs. The message has been that mothers should stay at home with their young children and government should not intrude. These messages mistakenly assume that mothers of young children have sufficient financial resources to stay at home and are able to provide all the care and stimulation that their children need.

Our public policies and spending priorities are also based on outdated developmental science. When the United States created the world’s first universal public education system almost 200 years ago, formal schooling began at age 7, based on then-contemporary scientific understanding that children younger than 7 cannot learn and that young children could be neglected or mistreated without consequence because their brains do not remember. Age 7 was, after all, the “age of reason,” when children first learn to reason and think.

Today, we continue a public expenditure model that increases per-child spending as children get older, in sharp contrast with contemporary scientific understanding that neuronal development, brain plasticity, and the likelihood of intervention success are at their zenith in early life. Ironically, even though parents are poorest when their children are youngest, our government provides more financial support for families at older than younger ages, mostly through public education. According to Chaudry et al.’s (2018, p. 49) figures, we spend $12,401 per year on each school-aged child but only $2,566 per year on each child birth-to-age-5. We spend only .3% of our gross domestic product on young children, much lower than virtually all other developed countries. We need to bring funding levels for young children up to existing levels for older children to provide resources when they have greatest impact.

What Young Children Need

If we aim to build a system to provide services, what community services do our young children and families need, and what services do we afford them? Developmental science tells us that in the first year of life, infants need competent and confident adults, hopefully parents, to be with the infant full-time to care for the infant’s physical and emotional needs, unencumbered by the stress of out-of-home work demands so that the infant can develop security of attachment, relational health (Willis, 2013), and social—emotional competence (Domitrovich, Durlak, Staley, & Weissberg, 2017), which lay the foundation for mental health (Kazdin & Blasé, 2011) and well-being. This science suggests we need policies and funding to enable new parents to stay at home and to parent with the highest quality; we also need the highest quality out-of-home infant care for families that prefer to work. By the time a child learns to locomote, say words, and play with others, that child needs ongoing cognitive stimulation, provided either in a financially secure and stress-free home environment or through well-staffed center-based care (Tolan & Dodge, 2005). As the child grows closer to kindergarten age, the child needs to begin the transition to formal schooling with structured play-based curricula that teach phonics, numerosity, social—emotional skills, and executive function of self- regulation. The child needs experiences interacting with well-functioning peers to develop social competence (Domitrovich et al., 2017). We also know that needs vary across families (e.g., one child needs professional training in communications skills, whereas another family needs psychiatric intervention for maternal postpartum depression), so the child needs a community that has aligned resources to serve families.

Chaudry et al. (2018) showed empirically that upper middle class parents are able to find nurturing experiences for their children and effective specialized programs when needed. They stay home after birth, at least for a period of time, by taking advantage of government- and employer-supported family leave policies, which are common in almost every developed country except the United States. When they go back to work, they are able to afford nannies or high-quality center-based child care. If their toddler has unique developmental needs, they are able to find private services to screen, assess, and intervene effectively. In the year before kindergarten, they are likely to place their child in a high-quality prekindergarten program.

For low-income families in America, it is an entirely different story. Parents are forced to go back to work almost immediately after giving birth. Although they need housing loans, food stamps, childcare subsidies, and job training, parents know how disorganized the so-called system is, and they are not able to access fully the limited resources available. Even agency leaders know this disarray. I recently met with the director of social services in a North Carolina community, who told me that he administers 38 different social services for families birth-to-5, but his data system is so antiquated that he does not know whether a family that receives food stamps is also receiving a housing loan. No one knows. The programs are disjointed, and their total effect is less than the sum of all parts. In the child’s first year of life, low-income families use nonparental care at higher rates than wealthier families do, whereas wealthy families can afford to stay home. After the first year, the pattern reverses, when low-income families struggle to afford high-quality center-based care that wealthy families hoard (Chaudry et al., 2018). Because no one watches over, tracks, and screens the full population of children, low- income families are not able to diagnose whether their toddler has developmental needs, let alone access services to address those needs. In the year before kindergarten, despite Head Start and public programs, wealthier families are more likely to place their child in prekindergarten, and the ones they choose are of higher quality. We need a system that will restore equity.

More than several leading thinkers have proposed top-down plans to remedy these problems. Chase, Emarita, Carlson, and Giovanelli (2016) proposed a policy plan to reduce racial inequity for Minnesota young children that includes family resource centers, economic investments, flexible funding streams, and holistic health services. Chaudry and colleagues (2018) proposed an ambitious plan to combat inequity in the early years that includes enhanced paid parental leave, affordable childcare and education, universal preschool, and a reinvention of the Head Start program. The National Governors Association proposed “Building Ready States” to support high-quality early childhood state systems (Demma, 2010).

These proposals will improve community capacity, but none includes a bottom-up plan to reach all children and match them with community resources. Commendable efforts have been implemented to match community services to a child’s needs and to “wrap them around a child” (Stroul, Blau, & Friedman, 2010, p. 4; Stroul & Friedman, 1986), but these efforts have been confined to children with serious mental disorders and funded by extraordinary means, such as the military’s program at Fort Bragg, NC (Behar, 1997), and a court order known as “Willie M.” (Dodge, Kupersmidt, & Fontaine, 2000). The wrap-around approach has not yet been applied to full populations. We need a universal system that reaches all children from all income brackets.

What Would a New System of Care Look Like?

A comprehensive system of care must support diverse families at varying levels of risk. Psychologists distinguish among primary (before the fact of disorder), secondary (early intervention for those at high risk or in early stages of disorder), and tertiary (slowing progression of disorder after diagnosis) prevention programs (Institute of Medicine, Committee on Assuring the Health of the Public in the 21st Century, 2003) and among universal (population-wide), selected (directed toward a high-risk group), and indicated (targeted after diagnosis) services (Mrazek & Haggerty, 1994). Designing a system requires including all these levels and matching and delivering services according to a child’s individualized needs. It must address several important questions, discussed in the next sections.

What Are the Goals?

Developmental science shows that children follow diverse pathways to success, such that a model of numerous unique incremental risk factors will account for most outcomes, leading to the conclusion that different interventions are needed for different children. One goal of a coordinated system of care will be to match a child’s particular needs with child-specific interventions to mitigate long-term expensive outcomes such as rehabilitative health care, remedial education, and dependency on government supports that accrues from low well-being. The problem is as much an engineering challenge as a scientific one.

What Kinds of Programs and Policies Should be Included?

We are in an evidence-based era, so a system of care must provide interventions that have been proven to be effective through randomized controlled trials and other rigorous evaluation methods. Because children have different needs at different ages, we need a system that repeatedly reaches children from the prenatal period through school entry. This system must address biological, psychological, and social factors in both the child and the family. Primary pediatric care offers a useful analogy: Every child in a community (hopefully) attends well-baby health care visits at which a pediatrician screens for problems, provides brief interventions (e.g., nutrition advice, admonishment to have a proper car seat, and “back to sleep” messages), and refers some families to needed specialized services such as otolaryngology or surgery.

A psychosocial system of care needs both primary care providers to screen all children and specialists to serve particular groups. Once we identify a child’s needs, we need to connect that family with evidence-based developmentally appropriate interventions. A community needs a comprehensive menu of specialized services that includes policies to help families with particular needs (e.g., childcare subsidies, housing loans, WIC, SNAP, CHIP, and family medical leave); group-based interventions that combine individualized home visiting with high-quality child care and education (e.g., Abecedarian and Early Head Start); and individually targeted psychological interventions such as Nurse Family Partnership and Attachment Behavioral Catch-Up (Roben, Dozier, Caron, & Bernard, 2017), which might be called “psychosocial neurosurgery.”

Who Finances the System of Care?

A recent National Academy of Sciences panel report (Allen & Backes, 2018) concluded that “despite the great promise of early care and education, it has been financed in such a way that high-quality early care and education has only been available to a fraction of the families needing and desiring it” (p. 1). This must change.

Top-down tasks of growing community capacity should be driven only partly by the for-profit marketplace; if essential services are lacking, public funding can shore up gaps in service but is not likely to be the sole source of reform in children’s services. Philanthropy has played a critical role in innovation and filling gaps and will likely continue to drive change in this country. But our culture also values family responsibility and contribution to a child’s health care, education, and development. Much of our early childcare and education infrastructure depends on family payment through fees for services. Government-supported sliding scales could ensure universal coverage. A successful system of care for young children will combine contributions from public, philanthropic, private-for-profit, and family resources in a coordinated way.

How Should a System of Care Be Administered?

A system of care will garner the broadest public support if it reaches universally to every child. A new bureaucracy is not likely to be supported, and so a system of care will be most successful if it is embedded in current practices. Social services reach only low-income and troubled families, and the childcare industry reaches only those who can afford care. The service system that is most closely aligned to the principles articulated here and most likely to reach all children over time is pediatric health care. Although adherence to recommended care is sporadic, most children get immunized and see their pediatrician annually. Many pediatricians voice a desire to serve their young patients more comprehensively, and pediatric practices are increasingly incorporating screening for developmental delays and autism, social determinants of health, and adverse child experiences. Broadening the scope of pediatric care will require more than the paltry 12 min per visit reimbursed by health insurance. Public-private partnerships could cofund the expansion. But no amount of reimbursement will enable an individual pediatrician who sees only a small portion of children in a community to grow a comprehensive system of community resources for families; pediatricians need a top-down communal infrastructure in which to embed their bottom-up efforts. Better funding and support could lead pediatricians to embrace a new system of care that tends to children’s psychosocial, as well as physical, health needs.

Parents must participate readily as recipients of services, and so the system must be “consumer-friendly” through an information system that tracks services, assessments, and functioning across time. The system will depend on providers of evidence-based services to participate, and so the system must be able to track services by agencies in efficient electronic ways. The system must incorporate an efficient way to document services received by a child, track those services and outcomes over time, monitor implementation fidelity, hold providers accountable, and evaluate impact of individual services on individual children as well as evaluate the impact of the system itself on population outcomes.

Key Components of a System of Care for Young Children

A well-functioning system of care for early childhood needs both top-down and bottom-up structures. A bottom-up structure is one that reaches almost every child in a community one by one, through screening, assessment, and intervention to address each child’s developmental needs. Universal screening must address a wide range of needs that span family financial stability; access to health care; parenting and childcare; and the child’s cognitive, emotional, and behavioral development. It must be repeated at key intervals across development, because new needs arise as new environmental demands are placed on a child. This screening system needs to reach universally to every child in a community, partly because risk can be found at every income level and partly because a program that is exclusively for the poor tends to be a poor program, without community political support.

A universal screening system is only as good as the array of community resources that are available and accessible to families that are screened. So, the top-down component of a system of care is an aligned array of evidence-based community services that are poised and ready to serve children. The services include professional treatments, such as those for maternal depression, parental substance abuse, and domestic violence. They also include a network of licensed childcare agencies that serve the diverse needs of families, including part-time care, whole-day care, workplace-based care, credentialed in-home care, emergency care, and respite care. The services must also address a family’s financial instability through emergency housing loans, WIC, food stamps, and the array of services in a community Division of Social Services.

The collective impact movement is a top-down systems approach that supports service providers as a way to move the needle on population outcomes. Strive Together is a prominent example through which software, leadership, and joint public commitment are leveraged to motivate a community (Strive Together, 2018). Although powerful, this approach has not yet provided hard empirical evidence of its success. Another example is the Harlem Children’s Zone (Dobbie & Fryer, 2011), a place-based approach that provides massive resources for all children in a neighborhood. Its success may be hard to replicate because of the resources it requires. Some communities have tried to build a System of Care, a now-common term first articulated by Stroul and Friedman (1986), pioneered at Fort Bragg (Behar, 1997), and carried today by the National Technical Assistance Center for Children’s Mental Health (Stroul, Dodge, Goldman, Rider, & Friedman, 2018). This approach has been confined to after-the-fact treatments in the mental health domain. In this article, I call for expansion to prevention and treatment in broader psychosocial health and well-being.

Finally, the top-down and bottom-up elements of a system of care need to be tied together through an integrated data system. We need a way to identify service agencies, code the evidence-based services they provide, align services with children’s needs, and make evidence-based services readily available to children whose individual needs indicate they need these services. A universal data system will provide information on every child in a community, thus providing a map of children’s needs and status; it will also provide information on every community service, thus providing a map of community capacity to serve children. Comparing children’s needs with community capacity to respond affords yet another crucial component of a system of care, which is to grow community services in a targeted way to address unmet needs.

Innovative Models

The remainder of this article is devoted to empirical studies of the impact of systemwide interventions to change population-level outcomes for young children. The interventions are consistent with the principles of systems change articulated here and are implemented across an entire community. Community implementation must be evaluated through creative designs and data-analytic methods. The first example is a top-down approach for improving the infrastructure of early childcare and education services in a community. The second example is a bottom-up approach to reach every family in a community at the time of an infant’s birth. Both approaches will be necessary to achieve population impact on child development outcomes.

The Top-Down Approach: North Carolina’s Smart Start

In the early 1990s, Governor Jim Hunt of North Carolina listened to developmental psychologists, especially those at Frank Porter Graham Child Development Center at the University of North Carolina, and formulated a vision to provide state resources to local communities so that they could grow high-quality daycare and preschool programs. He called it Smart Start (Ponder, 2011). In most communities, childcare is unregulated, and high-quality care is accessible by only the wealthy (Chaudry et al., 2018, p. 47, Figure 3–3). The goals of the Smart Start initiative are to improve the health, school readiness, and well-being for all children in the community. This is a top-down approach that attempts to change the infrastructure of childcare services. It is not implemented with merely a few children but is instead a systemwide policy.

The initiative has several components. First is state-level support for local professionals to help childcare agencies improve their quality up to standards that the state developed through its innovative five-star rating system. These standards address curricula, child–teacher ratios, physical building characteristics, attention to child health, and so forth. Second is support for professionals to help childcare agency leaders address the family functioning, behavioral, and social– emotional development of their child clients. Consultants help childcare leaders deal with behavior problems, developmental delays, immunizations, family stress, and related issues. Third is a focus on all children, not just low-income children. Fourth is encouragement of collaboration among local agencies. Fifth is great flexibility at the local level to improve childcare and education as local leaders see fit, to enable them to tailor programs to the families they serve and develop a sense of ownership and commitment to their own effort. Over time, restrictions have been added that require financial support for income qualifying children to attend high-quality childcare. Funding for the initiative has been substantial but time-varying, peaking at about $400 per child capita in an entire county with an annual statewide budget of over $250 million.

How could one evaluate the impact of such a community-wide initiative? By design, the initiative is intended to help every young child within a community, so the unit of evaluation must be the entire cohort of births in a community. Fortunately for evaluators, Smart Start was started in 1993 as a pilot in 18 of the 100 North Carolina counties. Counties were selected to represent North Carolina’s diversity and geography, with at least one county from each congressional district. The program increased each year to more than 50 counties by 1997 and to all 100 in 1999. Because funding was uncorrelated with any child characteristic, this funding pattern thus qualifies as a natural experiment.

To evaluate impact, Dodge, Bai, Ladd, and Muschkin (2017; (also Ladd, Muschkin, & Dodge, 2014; Muschkin, Ladd, & Dodge, 2015) merged three large administrative data sets for over 1.3 million children: birth records to identify the county of a child’s residence at birth, school records to identify county of residence at school age and a child’s outcomes in academic test scores and other educational variables, and records of the state’s per child capita funding allocation to each county in each of 13 years of the study. They successfully matched 74% of all births to a public school record. Because about one fifth of children in North Carolina never attend public school, they estimated that they matched about 90% of all eligible children. The independent variable was the number of dollars (in hundreds) allocated to a child, dependent variables were child education outcomes, and many child-level covariates such as race and birth risk were drawn from birth records. The model included county fixed effects that control for any stable unmeasured differences across counties and year fixed effects that control for differences across cohorts of children.

The authors found that compared with a child who resided in a county at a time when that county received little or no funding, a child who was lucky enough to reside in a county with relatively high Smart Start funding over the first 5 years of life has, on average, about a 10th of a standard deviation higher standardized achievement test scores in both reading and math in Grade 3. The gains for this child extend to a lower probability of being placed in special education and a lower probability of being retained in grade. The gains continued without any fadeout in Grade 4 and Grade 5. Effect sizes were substantial: A child who lived in a county that received the average state funding for Smart Start experienced about two more months of learning than did a child who lived in a county with no funding. Furthermore, when they examined impact on subgroups of children, they found that Smart Start funding levels had a positive impact on all subgroups, with the greatest impact on the group of children who qualified for free or reduced-price lunch, as shown in Figure 2 for fifth-grade math scores.

Figure 2.

Figure 2.

Relation between Smart Start funding levels in early childhood and children’s standardized math scores in fifth grade (Dodge, Bai, Ladd, & Muschkin, 2017). Lower line is for children receiving free or reduced lunch, and upper line is for children not receiving free or reduced lunch. Error bars indicate standard error of the means. CI = confidence interval. See the online article for the color version of this figure.

The Smart Start Initiative continues today, albeit with funding threats. The evaluation shows convincingly that investment in top-down infrastructure of community agency support for developmental-science-based early childcare and education can improve population-level scores while reducing income-based disparities in children’s outcomes.

The Bottom-Up Approach: Importance of Individual Assessment and Matching

The challenge of supporting early child development would be easier if one “silver bullet” intervention could satisfy every child’s needs. Indeed, the assumption of many early childhood programs is that a single generic intervention, targeted at children from low-income backgrounds, could remediate these children’s backgrounds and erase disparities in development that are present at kindergarten entry. Instead, children have diverse needs that must be individually identified and met. Colleagues and I have conducted two prospective studies of families to test the dual hypotheses that children’s needs (and risk) go beyond demographic characteristics and vary across families.

The goal of the first study (Berlin, Appleyard, & Dodge, 2011) was to identify predictors of child abuse and failure-to-thrive so that we could identify targets for preventive intervention. The study followed a community-representative sample of 500 pregnant women from the second trimester of pregnancy through infant age 26 months. Predictor variables came from a clinical interview during pregnancy, and the child maltreatment outcome came from child protective service records. In support of the first hypothesis, we found that although demographic variables of maternal age, education, and income did indeed predict child maltreatment outcomes, even after these demographic variables were entered, clinically assessed indicators for maternal social isolation, depression, substance use, and parenting skills provided significant increments in the prediction. Specifically, maternal substance abuse and depression assessed in the prenatal period predicted later child maltreatment even after controlling for demographic factors (Appleyard, Berlin, Rosanbalm, & Dodge, 2011), and a mother’s early parenting beliefs (specifically, her inaccurate biased attributions that her infant can act intentionally to hurt her) predicted her infant’s later presence in a child maltreatment registry (Berlin, Dodge, & Reznick, 2013). Furthermore, these clinically assessed variables provided unique increments in the prediction; that is, some children were at risk due to parental substance abuse or domestic violence, whereas other children were at risk due to family financial instability or poor childcare quality or parenting skills deficits. These findings tell us that risk extends beyond demographic disadvantage and that individual clinical assessments can improve the identification of children at risk and point toward matching interventions to a family’s specific need.

The second study (Dodge, Goodman, & Bai, 2018) included a representative sample of 280 infants followed from birth through 60 months of age, as part of the Durham Connects intervention program. Predictor variables were obtained by interview at age 3 weeks, and outcomes were obtained from hospital and child protective service records across the next 60 months. As in the first study, demographic variables significantly predicted poor outcomes (both emergency medical visits and child protective services investigations), but clinically assessed variables provided important increments in this prediction beyond demographic variables.

We interpret these findings as evidence of the added value of screening every family to identify clinically important factors to match community services to families according to their specific needs. Children follow diverse pathways to achieving these competencies, leading to the conclusion that different interventions are needed for different children. Although some programs target children based on demographic factors alone (e.g., Head Start for low-income children, Nurse Family Partnership for low-income first-time mothers), longitudinal empirical analyses (Dodge, Goodman, & Bai, 2018) have indicated that clinically assessed needs provide a stronger prediction of outcomes and could enhance matching of children with interventions.

Family Connects

These findings inspire a bottom-up population-wide intervention designed to reach every child in a community at birth in a systematic way akin to public health. The program started as Durham Connects but became Family Connects when disseminated to other communities. It represents a paradigmatic shift in delivery of community services to families at birth (Dodge, Goodman, Murphy, O’Donnell, & Sato, 2013b). The program assumes that any and every family is vulnerable at birth. All families are eligible, although the program is voluntary. The assertion is that the route to community change is universal delivery of a system of care. These universal efforts do not replace more targeted approaches. In fact, a goal of Family Connects is to identify which families would benefit most from more cost-intensive services, just as a primary care pediatrician identifies which families require more expensive specialized surgical care.

The program aims to assess each family’s needs in a comprehensive array of domains, intervene when possible, and match the family with evidence-based community services to meet the family’s specific needs. From the family’s perspective, the program has three steps.

First, a trained nurse greets the mother in the birthing hospital, celebrates the birth, and communicates that research has shown that every child and parent can be successful but that no one is successful alone. Nurses tell the mother that the community wants to support her specific needs. She then invites herself to one to three home visits, at which she uses a structured clinical interview to assess family needs in each of 12 domains that developmental science has shown are risk factors for child maltreatment and poor development (Dodge et al., 2013b). These domains cover health care, family safety and financial stability, parenting, and parent mental health. The nurse identifies a profile of family needs. When possible, the nurse resolves any problem through brief intervention. The second step is to connect the family with community resources to solve more ongoing problems such as maternal substance abuse, financial crises, and depression. The third step is to ensure that the mother can continue to connect with her baby by “passing the baton” of the family to the infant’s pediatrician and mother’s primary health care provider through notes and meetings.

Family Connects also grows a strong top-down infrastructure of community agencies that are committed to supporting families in the first 5 years of life. One limit of individual-oriented psychological interventions home-visiting programs is that the interventionist is left with the task of teaching the client to “battle the system.” Family Connects, in contrast, creates a new system. Nurses understand the eligibility criteria that an agency uses, its goals, and its evidence base. The program assembles community services in an electronic directory, which the nurse uses to match families to resources. In Durham, the directory now has 400 professional, paraprofessional, and volunteer agencies serving families. By assessing family needs in almost the entire population and cataloging the available community resources to address those needs, the program can report to community leaders where gaps exist. For example, in Durham, program leaders learned early on that the need for maternal in-home substance abuse treatment was far greater than the community capacity to address this problem, and they were able to present empirical evidence to elected officials, who then financed capacity-building over time.

Through phenomenal support from The Duke Endowment, we were able to implement and evaluate Family Connects through randomized controlled trials. The first trial included all 4,777 births in Durham over an 18-month period, regardless of their interest in participating in a clinical intervention trial. Because nurses went to the hospital to recruit mothers, evaluators did not want “control” mothers to learn that other mothers had been selected for the program, so we randomly assigned infants based on the date of birth, with even-numbered births randomly assigned to intervention and odd-numbered births to control. No mother knew that this was a randomized trial, providing an experience closer to true community services, albeit with ethical concerns that were deliberated successfully with the institutional review board. At infant age 6 months, evaluators independently approached a random subset of families to assess parenting and infant development. The parent was blind to the evaluation goal, and the interviewer was blind to the family’s experimental condition, which was determined from the infant’s birth date. Randomization was successful in yielding groups that did not differ significantly in preintervention characteristics, and follow-up retained over 90% of participants.

In the first randomized controlled trial (Dodge et al.,2014), evaluation showed that Family Connects successfully recruited 80% of all even-birthdate births to participate in the program, and 86% of that figure went on to complete all phases of the program. Fidelity of implementation was strong, and Family Connects nurses were successful in identifying risk in a reliable way. A nurse supervisor accompanied the nurse on 10% of visits and found strong reliability in coding of risk and adherence to the protocol.

Postintervention indicated that 99% of all parents liked their experience and would recommend it to others. It is easy to like the idea that a nurse would visit the home to support a mother, especially when it is offered to everyone in the community.

Moving to evaluation of impact, compared with control families, families that had been randomly assigned to the Family Connects intervention showed better outcomes at infant age 6 months in domains that had been directly targeted by the intervention. They had more connections to community resources, more positive parenting as rated by mothers and independently observed by the interviewer, more father involvement, lower maternal anxiety, and more home safety. There were no differences in parents’ tendency to use out-of-home childcare (which is reassuring because the nurse did not try to sway mothers to use out-of-home childcare or to stay home), but if a mother did use childcare, its quality was rated as higher by the North Carolina five-star rating system.

Dodge, Goodman, Murphy, O’Donnell, and Sato (2013a) examined hospital administrative records of admissions and billing and found that infants randomly assigned to Family Connects had fewer emergency department admissions, fewer overnights in the hospital, and lower emergency care costs than did control infants. These differences emerged shortly after the intervention and grew across the first 12 months of life, showing a real preventive effect, as shown in Figure 3. Furthermore, this preventive effect held for each subgroup analyzed, including both low-income and middle income families, suggesting the potential for reducing disparities in outcomes across groups.

Figure 3.

Figure 3.

Mean cumulative number of emergency health care episodes for groups randomly assigned to Durham Connects or control (Dodge et al., 2013a). See the online article for the color version of this figure.

Finally, Goodman, Dodge, Murphy, and O’Donnell (2018b) examined the official child abuse registry and found that by 60 months of age, the cumulative rate of child-abuse-investigated cases was significantly lower for the Family Connects group than the control group.

These encouraging findings led to a replication randomized control trial (RCT). Dodge, Goodman, Murphy, & O’Donnell (2018) found generally consistent results regarding high penetration rates, strong fidelity of implementation and reliability of scoring, increases in community connections, and lower parental mental health concerns such as anxiety. The emergency room finding for infants was not significant, however.

Program developers then moved on to ask whether they could move the needle for entire communities by implementing the program in four rural counties in eastern North Carolina. They used a quasi-experiment with intent-to-treat analysis by comparing population-level outcomes for infants born during the 6-month period just prior to implementing the program with outcomes for infants born during the implementation period. Goodman, Dodge, Murphy, and O’Donnell (2018a) found that infants born during the period of Family Connects implementation had significantly fewer emergency department visits than did the control group, replicating the first RCT.

The Family Connects program has been accredited and approved for federal funding from the Maternal, Infant, and Early Childhood Home Visiting Program and has been implemented, or plans for implementation are under way, in over 30 communities nationwide. In each of these locations, an entire population, not just a subgroup, is being targeted.

Financing of population-level interventions is a challenge. Families cannot be expected to pay if universal participation is anticipated. Funding for Family Connects comes from a variety of sources, including for-profit health care systems that see the potential benefit in saved costs, Medicaid, philanthropy, public health departments, and county and state tax dollars and grants.

Through phenomenal foundation support, program developers are now piloting innovations in how to implement the program in diverse community contexts and how to grow a broader and more developmentally continuous system of care for children that would begin prenatally and last through at least the beginning of kindergarten. Specifically, as part of the Get Ready Guilford Initiative, Family Connects is collaborating with the Nurse-Family Partnership and local community leaders in designing a universal system to screen and support all pregnancies during initial prenatal care visits and with the Healthy Steps program to “pass the baton” for a family after they complete the Family Connects program to the pediatrician’s office to continue following the family across development. These teams are piloting new universal screening and intervention protocols prenatally and at ages 12, 24, and 36 months, to create a developmentally matched ongoing system of care.

By reaching families at these critical life points, we have learned that one of the major challenges for families is navigating the silos of services that their infant utilizes across early life, including home-visiting programs, social services, childcare, developmental interventions, Head Start, and parental psychological interventions. To help families and agencies track these services so that interventions can be designed to address child-specific histories and needs, we are designing an electronic family-level information system that could become a child’s psychosocial record of screenings, intervention services, and developmental assessments. This record will enable a mother to bring a child’s developmental history to new intervention settings, for community service agency to communicate with each other, and for providers to chart a child’s progress across development. The record would be the psychosocial equivalent of one’s electronic health record. If the record system includes almost the entire population, aggregated measures could chart a community’s progress.

Closing

It would be naive to assert that the system of care proposed here should be implemented in the same way in all communities across the United States. Just as context differences across families within a community require tailoring of interventions to a family’s needs, so, too, context differences across communities within the country suggest that aspects of the system must be tailored to a community. The Smart Start initiative was developed to suit the needs of southern families and was implemented in North Carolina’s state-funded, county-administered system of governance: What changes might be needed to implement this initiative in other states and nations? Could the Family Connects program, which was developed in a midsized southern community, be implemented effectively in a large metropolitan community such as New York, rural communities of the Midwest, or sub-Saharan Africa? Would cultural differences in norms about hosting a relative stranger in one’s home affect the efficacy of Family Connects when implemented in new communities? These questions do not dampen enthusiasm for dissemination of these programs, but they highlight the need for continued rigorous evaluation as these programs are disseminated in new contexts.

Is it possible to go further to create an entire new system of psychosocial care for young children from birth to school age based on top-down and bottom-up principles similar to those I propose here? Several communities are trying. The Duke Endowment is leading a breath-taking attempt to improve outcomes for the population of young children in Guilford County, North Carolina. The George Kaiser Family Foundation is leading an effort with similar goals in Tulsa, Oklahoma. These efforts are in early stages and will be evaluated over time.

Our nation was successful in creating a universal public education system for older children. We created a system of care for the elderly through Social Security and Medicare. Can we create a comprehensive system of psychosocial care for young children? I believe it is possible, and necessary, to ensure the next generation’s success and well-being.

Acknowledgments

The author acknowledges and is grateful to the numerous funders of the research reported here, particularly The Duke Endowment.

References

  1. Allen LR, & Backes EP (Eds.). (2018). Transforming the financing of early care and education 10.17226/24984 [DOI] [PubMed] [Google Scholar]
  2. Appleyard K, Berlin LJ, Rosanbalm KD, & Dodge KA (2011). Preventing early child maltreatment: Implications from a longitudinal study of maternal abuse history, substance use problems, and offspring victimization. Prevention Science, 12, 139–149. 10.1007/s11121-010-0193-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Behar L (1997). The Fort Bragg evaluation: A snapshot in time. American Psychologist, 52, 557–559. 10.1037/0003-066X.52.5.557 [DOI] [Google Scholar]
  4. Berlin LJ, Appleyard K, & Dodge KA (2011). Intergenerational continuity in child maltreatment: Mediating mechanisms and implications for prevention. Child Development, 82, 162–176. 10.1111/j.1467-8624.2010.01547.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Berlin LJ, Dodge KA, & Reznick JS (2013). Examining pregnant women’s hostile attributions about infants as a predictor of offspring maltreatment. Journal of the American Medical Association Pediatrics, 167, 549–553. 10.1001/jamapediatrics.2013.1212 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Campbell FA, & Ramey CT (1994). Effects of early intervention on intellectual and academic achievement: A follow-up study of children from low-income families. Child Development, 65, 684–698. 10.2307/1131410 [DOI] [PubMed] [Google Scholar]
  7. Catov JM, Lee M, Roberts JM, Xu J, & Simhan HN (2016). Race disparities and decreasing birth weight: Are all babies getting smaller? American Journal of Epidemiology, 183, 15–23. 10.1093/aje/kwv194 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Chase R, Emarita B, Carlson E, & Giovanelli A (2016). Prenatal to age 3: A comprehensive, racially-equitable policy plan for universal healthy child development Retrieved from http://www.buildinitiative.org/Portals/0/Uploads/Documents/Work/Recent%20Events/PrenatalToAge3_Plan_9-16.pdf
  9. Chaudry A, Morrissey T, Weiland C, & Yoshikawa H (2018). Cradle to kindergarten: A new plan to combat inequality New York, NY: Russell Sage Foundation. [Google Scholar]
  10. Child Trends. (2015). Early school readiness: Indicators of child and youth well-being Retrieved from https://www.childtrends.org/indicators/early-school-readiness
  11. Demma D (2010). Building ready states: A governor’s guide to supporting a comprehensive, high-quality early childhood state systems Washington, DC: National Governors’ Association, Center for Best Practices. [Google Scholar]
  12. Dobbie W, & Fryer RG Jr. (2011). Are high-quality schools enough to increase achievement among the poor? Evidence from the Harlem Children’s Zone. American Economic Journal Applied Economics, 3, 158–187. 10.1257/app.3.3.158 [DOI] [Google Scholar]
  13. Dodge KA, Bai Y, Ladd HF, & Muschkin C (2017). Impact of North Carolina’s early childhood programs and policies on educational outcomes in elementary school. Child Development, 88, 996–1014. 10.1111/cdev.12645 [DOI] [PubMed] [Google Scholar]
  14. Dodge KA, Goodman WB, & Bai Y (2018). Clinical versus demographic prediction of child maltreatment Unpublished manuscript, Center for Child and Family Policy, Duke University, Durham, NC. [Google Scholar]
  15. Dodge KA, Goodman WB, Murphy R, & O’Donnell K (2018). Replication of Family Connects Unpublished manuscript, Center for Child and Family Policy, Duke University, Durham, NC. [Google Scholar]
  16. Dodge KA, Goodman WB, Murphy RA, O’Donnell K, & Sato J (2013a). Randomized controlled trial evaluation of universal postnatal nurse home visiting: Impact on emergency care. Pediatrics, 132, S140–S146. 10.1542/peds.2013-1021M [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Dodge KA, Goodman WB, Murphy R, O’Donnell K, & Sato J (2013b). Toward population impact from home visiting. Zero to Three, 33, 17–23. [PMC free article] [PubMed] [Google Scholar]
  18. Dodge KA, Goodman WB, Murphy RA, O’Donnell K, Sato J, & Guptill S (2014). Implementation and randomized controlled trial evaluation of universal postnatal nurse home visiting. American Journal of Public Health, 104(Suppl. 1), S136–S143. 10.2105/AJPH.2013.301361 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Dodge KA, & Haskins R (2015). Children and government. In Lerner RM, Bornstein MH, & Leventhal T (Eds.), Handbook of child psychology and developmental science: Vol. 4 Ecological settings and processes; (7th ed., pp. 654–703). 10.1002/9781118963418.childpsy417 [DOI] [Google Scholar]
  20. Dodge KA, Kupersmidt J, & Fontaine R (2000). The Willie M. Program: Report to the state of North Carolina department of mental health administration Unpublished manuscript, Center for Child and Family Policy, Duke University, Durham, NC. [Google Scholar]
  21. Domitrovich CE, Durlak JA, Staley KC, & Weissberg RP (2017). Social-emotional competence: An essential factor for promoting positive adjustment and reducing risk in school children. Child Development, 88, 408–416. 10.1111/cdev.12739 [DOI] [PubMed] [Google Scholar]
  22. Goodman WB, Dodge KA, Murphy R, & O’Donnell K (2018a). Evaluation of Family Connects in rural communities Unpublished manuscript.
  23. Goodman WB, Dodge KA, Murphy R, & O’Donnell K (2018b). Prevention of child injuries and maltreatment through the family connects universal nurse home-visiting program Unpublished manuscript.
  24. Institute of Medicine, Committee on Assuring the Health of the Public in the 21st Century. (2003). The future of the public’s health in the 21st century Washington, DC: National Academies Press. [Google Scholar]
  25. Justice LM, Petscher Y, Schatschneider C, & Mashburn A (2011). Peer effects in preschool classrooms: Is children’s language growth associated with their classmates’ skills? Child Development, 82, 1768–1777. 10.1111/j.1467-8624.2011.01665.x [DOI] [PubMed] [Google Scholar]
  26. Kazdin AE, & Blasé SL (2011). Rebooting psychotherapy research and practice to reduce the burden of mental illness. Perspectives on Psychological Science, 6, 21–37. 10.1177/1745691610393527 [DOI] [PubMed] [Google Scholar]
  27. Ladd HF, Muschkin C, & Dodge KA (2014). From birth to school: Early childhood initiatives and third-grade outcomes in North Carolina. Journal of Policy Analysis and Management, 33, 162–187. 10.1002/pam.21734 [DOI] [Google Scholar]
  28. Land K (2017). The Duke Child and Youth Well Being Index (CWI) report, 2016 Durham, NC: Duke University Center for Child and Family Policy. [Google Scholar]
  29. Mrazek PJ, & Haggerty RJ (Eds.). (1994). Reducing risks for mental disorders: Frontiers for preventive intervention research Washington, DC: National Academies Press. [PubMed] [Google Scholar]
  30. Muschkin CG, Ladd HF, & Dodge KA (2015). Impact of North Carolina’s early childhood initiatives on special education placements in third grade. Educational Evaluation and Policy Analysis, 37, 478–500. 10.3102/0162373714559096 [DOI] [Google Scholar]
  31. Ponder K (2011). The Smart Start story Unpublished manuscript.
  32. Roben CKP, Dozier M, Caron EB, & Bernard K (2017). Moving an evidence-based parenting program into the community. Child Development, 88, 1447–1452. 10.1111/cdev.12898 [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Schweinhart LJ, Montie J, Xiang Z, Barnett WS, Belfield CR, & Nores M (2005). Lifetime effects: The High/Scope Perry Preschool study through age 40 (High/Scope Educational Research Foundation Monograph No. 14). Ypsilanti, MI: High/Scope Educational Research Foundation. [Google Scholar]
  34. Shonkoff JP (2017). Rethinking the definition of evidence-based interventions to promote early childhood development. Pediatrics, 140(6): e20173136 10.1542/peds.2017-3136 [DOI] [PubMed] [Google Scholar]
  35. Strive Together. (2018, October). 8 million students, ONE VISION Retrieved from https://www.strivetogether.org/wpcontent/uploads/2017/10/StriveTogether_Overview_Brochure_Oct2017.pdf
  36. Stroul B, Blau G, & Friedman R (2010). Updating the system of care concept and philosophy Washington, DC: Georgetown University Center for Child and Human Development, National Technical Assistance Center for Children’s Mental Health. [Google Scholar]
  37. Stroul B, Dodge J, Goldman S, Rider F, & Friedman R (2018). Expanding the system of care approach toolkit Retrieved from https://gucchdtacenter.georgetown.edu/the-soc-approach.html
  38. Stroul B, & Friedman R (1986). A system of care for children and youth with severe emotional disturbances (rev. ed.). Washington, DC: Georgetown University Child Development Center, National Technical Assistance Center for Children’s Mental Health. [Google Scholar]
  39. Tolan PH, & Dodge KA (2005). Children’s mental health as a primary care and concern: A system for comprehensive support and service. American Psychologist, 60, 601–614. 10.1037/0003-066X.60.6.601 [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Weisz JR, Krumholz LS, Santucci L, Thomassin K, & Ng MY (2015). Shrinking the gap between research and practice: Tailoring and testing youth psychotherapies in clinical care contexts. Annual Review of Clinical Psychology, 11, 139–163. 10.1146/annurevclinpsy-032814-112820 [DOI] [PubMed] [Google Scholar]
  41. Welsh BC, Sullivan CJ, & Olds DL (2010). When early crime prevention goes to scale: A new look at the evidence. Prevention Science, 11, 115–125. 10.1007/s11121-009-0159-4 [DOI] [PubMed] [Google Scholar]
  42. Willis DW (2013, October). The home visiting and early childhood systems: The national perspective Presentation at the California Home Visiting Summit, Sacramento, CA. [Google Scholar]

RESOURCES