Promoting Prenatal and Early Childhood Health: Evaluation of a Statewide Materials-Based Intervention for Parents

Abstract

Objectives. There is a critical need for effective, large-scale health communication programs to support parents of children aged 0–5 years. We evaluated the effectiveness of the Kit for New Parents, a multimedia health and parenting resource now distributed annually to 500000 parents in California.

Methods. In this quasi-experimental study, 462 mothers in the intervention group and 1011 mothers in the comparison group, recruited from prenatal and postnatal programs, completed a baseline interview about health-relevant parenting knowledge, and mothers in the intervention group received the kit. Both groups were reinterviewed 2 months later. At 14-months postbaseline, 350 mothers in the intervention group and a sample of 414 mothers who had equivalent demographic characteristics (comparison group) were interviewed about parenting knowledge and practices.

Results. Of the mothers in the intervention group, 87% reported using the kit within 2 months after receiving it, and 53% had shared it with their partner. At both follow-ups, mothers in the intervention group showed greater gains in knowledge and reported better practices at 14 months than did mothers in the comparison group. Gains were greater for prenatal recipients and for Spanish speakers. Providers considered the kit a valuable resource for their parenting programs.

Conclusions. The kit is an effective, low-cost, statewide health intervention for parents.

A critical public health challenge is to translate research findings into effective, large-scale, and sustainable interventions.1^–3 Interventions during pregnancy and early childhood are especially important because they have shown some of the greatest impacts on children’s long-term health and development.4^–8

Parents’ knowledge and practices related to health care, bonding with their baby, nutrition, smoking, safety, and other factors can greatly affect children’s healthy development.9^–13 Although parenting education should be a priority public health investment, this has not been achieved at the population level.6 A national survey found that first-time, adolescent, single, and low-income parents cited parenting information as one of their greatest needs.14 In a California survey, the majority of parents (including 70% of Spanish-speaking parents) believed they did not have adequate knowledge to care for their young children.15

Most of the effective interventions reported to date have involved relatively intensive, expensive, person-to-person programs.7^,16^,17 Affordable mass coverage will require greater reliance on less expensive, materials-based approaches. These interventions must address the traditional concern that such approaches have weak effects, especially for underserved and low-literate groups who may show relatively low levels of knowledge both before and after interventions, i.e., “knowledge gaps.”18^–20

We found 16 published experimental or quasi-experimental studies that tested materials-based parenting education interventions. Although most of these studies were small and focused on limited topics in fairly homogeneous populations, they generally showed positive outcomes on parents’ knowledge, attitudes, or practices.21

Research on communication, adult learning, and motivation indicates that educational interventions are more effective when the intended beneficiaries participate in their design and when the approaches are interactive, adaptable to different learning styles, build on people’s self-identified needs, fit within their social contexts, and connect them with local resources.21^–25

In 2001, the First 5 California Children and Families Commission (“First 5”) took up the challenge to provide health and parenting education to all new parents in the state by developing the Kit for New Parents; the kit was primarily funded by a tobacco tax. Using guidance from past research, the commission worked with experts and providers to develop and test content in the following priority areas: prenatal care; child safety, nutrition, and health care; early childhood learning and development; childcare; and access to services. Examples of content include ways to stop smoking, childproof homes, feed infants healthy foods, and encourage early learning. Each kit contains 6 videos (featuring celebrities, experts, and diverse parents and children), a 60-page resource guidebook, topical brochures, and a book to read to babies (Figure 1 ▶). Printed materials are written at about the 6th grade level. The kits are produced and distributed in English and Spanish at a unit cost of $17.50. They are now distributed free to approximately 500 000 expectant and new parents each year through prenatal care providers, delivery hospitals, home visits, childcare, a toll-free telephone number, and other means. The kit program is the nation’s largest statewide health-promotion intervention to support parents during pregnancy and their child’s early years.

FIGURE 1—

The original First 5 Kit for New Parents.

To determine whether to launch the kit program statewide, First 5 commissioned an evaluation to answer the following questions: 1) Do parents use the kit and find it helpful? 2) Do mothers who use the kit show greater knowledge gains in the short and long term than mothers who do not receive a kit? 3) Do mothers make positive changes in parenting practices because of the kit? 4) Do Spanish-speaking mothers benefit from the kit as much as English-speaking mothers? And 5) What changes could improve the kit and its distribution?

METHODS

Outcome Evaluation

Study design and sample.

We used a quasi-experimental longitudinal design (Figure 2 ▶) to evaluate the kit’s effectiveness. The study began with recruitment of the intervention group. Quota samples of women were selected from urban and rural areas in 9 counties to meet the following criteria: half Spanish-speakers, half English-speakers, at least half on Medicaid, half prenatal, and half postnatal. Pregnant women were recruited from prenatal programs, and postnatal women either at hospital discharge or at a home visit—common venues planned for kit distribution.

FIGURE 2—

Timeline of data collection among mothers in intervention (who received the Kit for New Parents) and comparison groups: California, October 2000 to May 2002.

^aFirst 5 began distributing the Kit for New Parents statewide in November, 2001.

^bBecause the original comparison group was recruited 6 months after the intervention group was, there was concern that mothers in the comparison group might receive the kit during the 6-month wait for their 14-month interview, thereby contaminating the findings. Therefore, a new comparison sample of mothers who could be interviewed during the same time period that the intervention mothers had their 14-month follow-up interviews was recruited through a snowball sampling method.

Upon recruitment, baseline interviews were conducted with 542 women who comprised the intervention group. After the interview, mothers received a kit and most were briefly oriented on its use. Two months later, 462 of these mothers (85%) completed a follow-up telephone interview.

To better assess the kit’s effectiveness before statewide distribution, First 5 subsequently funded the addition of a comparison group of mothers who did not receive a kit. Six months after the mothers in the intervention group completed follow-up interviews, we used the same quotas to recruit a comparison group from pre- and postnatal programs that were carefully selected to be similar to those from which the mothers in the intervention group were recruited. Of the 1236 mothers in the comparison group who were recruited and interviewed at baseline, 1011 (82%) completed the 2-month follow-up telephone interview. First 5 later commissioned a 14-month follow-up interview. Of the original 542 women in the intervention group who completed the baseline interview, 350 (65%) completed this follow-up.

First 5 released the kit for statewide distribution just before the scheduled 14-month intervention group follow-up. Because the original comparison group was recruited 6 months after the intervention group was, we were concerned that mothers in the comparison group might receive the kit during the 6-month wait for their 14-month interview, thereby contaminating the findings. Therefore, we recruited a new comparison sample of mothers who could be interviewed during the same time period that the intervention mothers had their 14-month follow-up interviews. The new comparison group was recruited through a snowball sampling method26^–28; that is, the mothers in the original comparison group provided names of other mothers who had babies approximately 6 months older than theirs. This sample of 414 (82% of those referred and eligible) mothers in the new comparison group completed the 14-month interview.

Overall, mothers in the baseline intervention and comparison groups had similar background characteristics. In both samples, approximately two thirds of the mothers were enrolled in Medicaid, half chose Spanish as the language for their interview, half were pregnant, half had a newborn, and 16% were adolescents. However, relative to the comparison group, more mothers in the intervention group were aged 29 years or younger (76% vs 70%) and were first-time mothers (49% vs 41%). Mothers in the intervention group and those in the snowball-sampled comparison group who were interviewed at 14 months also had similar background characteristics. There were no significant differences related to age, income, or enrollment in Medicaid and the Special Supplemental Nurtitional Program for Women, Infants, and Children (WIC). However, fewer mothers in the intervention group spoke Spanish compared with the comparison group (46% vs 59%), and more mothers in the intervention group were first-time mothers compared with the comparison group (48% vs 37%).

Assessment instruments.

The baseline questionnaire included demographic items (age, race, Hispanic/Latino origin, mother’s Medicaid enrollment status, and information about pregnancy or birth status); an item regarding the infant’s health insurance coverage; and 8 knowledge items about priority issues addressed in the kit materials. These 8 knowledge items were developed by an expert panel to cover key areas of health and parenting knowledge cited in the literature21^,29 and included in the kit. Five items, in multiple-choice format, related to sleep safety, infant feeding and nutrition, and early learning. The remaining 3 items, in yes or no format, were scenario questions on accessing resources for childcare, low-cost medical care for babies, and smoking cessation.

The 2- and 14-month follow-up interviews included the 8 knowledge items, additional demographic and health insurance items, and for kit recipients only, multiple-choice and open-ended questions about kit use, helpfulness, and satisfaction since the previous interview. The 14-month interview contained additional multiple-choice items about mothers’ practices in caring for older babies related to child development, reading to infants, feeding, home safety, childcare, and discipline. Two of these items were intended to explore practices for which there was little information in the kit: co-sleeping and infant bedtime routines. Two Spanish translators reviewed initial instrument translations, and pretests were conducted in English and Spanish.

The 8 knowledge items were each scored 1 for correct answers (otherwise 0), then summed, divided by 8, and multiplied by 100, yielding a scale score with a range of 0 to 100. Cronbach’s coefficient alpha, calculated to estimate internal consistency reliability of the knowledge scale at baseline, was .49. A test–retest study of the stability of the knowledge scale was conducted with a separate sample. Pregnant women (N = 191; half Spanish speakers and half English speakers) were recruited at prenatal programs (similar to those in the outcome study) and interviewed by telephone. One week later, 180 (94%) were reinterviewed. The test–retest correlation coefficient was .83.

Data analyses.

Two independent coders grouped open-ended responses into categories. Interrater agreement was 90%. We used cross tabulations to display categorical variables by intervention versus comparison group and other variables, and the χ² test was used to evaluate group differences. We used repeated measures analysis of variance to analyze the primary outcome—mother’s knowledge score. We computed effect sizes (standardized mean differences) at 2 months and at 14 months to compare knowledge-scale differences between all mothers who received the kit and all mothers in the comparison groups. We also compared knowledge-scale differences for language and pregnancy status subgroups of mothers in the intervention and comparison groups.

Process Evaluation

After the outcome study, we conducted a statewide process evaluation of the program. Interviews were conducted with First 5 officials at the state level, administrators in all 58 counties, and with service providers in 10 urban and rural counties in diverse geographic regions. Interviews included questions about kit recipients, providers’ kit use and customization, and suggestions for improvements. Results were tabulated across issue areas.

RESULTS

Kit Use and Helpfulness

At the 2-month interview, 87% of mothers reported using the kit—defined as having watched any of the kit videos or having read any of the written resources (except for the baby book). Kit usage among Spanish-speaking mothers was 95%, compared with 82% among English-speaking mothers. Overall, 53% of mothers reported that their partner also used the kit—61% among Spanish speakers and 43% among English speakers. Mothers who were briefly oriented to the kit by a provider (78%) used more of the 4 kit components (means of 2.1 vs 1.8 components, P<.05) than mothers who were not oriented. Mothers consulted the kit for both general knowledge and specific issues. For example, mothers commented:

I wanted to know how to feed the baby. How they grow.

My cousin gets the baby and throws her up in the air and catches her. I looked in the brochures, and it showed me it was dangerous. So I told him that it can cause convulsions or even death. He never did it again.

At the 2-month interview, 94% of mothers said that the kit was helpful to them. At the 14-month follow-up, 60% of mothers reported using the kit since their 2-month interview. Mothers found the kit helpful for specific issues: child safety practices (76%), infant learning (72%), feeding solid food (71%), breastfeeding (58%), smoking (43%), health care (35%), and childcare (21%).

Spanish-speaking mothers, mothers recruited during pregnancy, and mothers who shared the kit with their partners found it helpful for more issues (P < .01) than did English-speaking mothers, mothers who received the kit postnatally, or mothers who did not share it with a partner. There were no significant differences in kit helpfulness between first-time and experienced mothers, or between adolescent and older mothers.

Parenting Knowledge

At the 2-month follow-up, mothers in the intervention group showed significantly greater knowledge gains than did mothers in the comparison group (P < .001; effect size of .48); both groups started with the same baseline score (52 points on a 100 point scale). The groups’ baseline scores were not significantly different even after we controlled for mother’s language, pregnancy status, age, and parity. At the 14-month follow-up, the scores of mothers in the intervention group were higher than those of the mothers in the snowball-sampled comparison group (P < .001; effect size of .51). Although mothers in both intervention groups—those recruited prenatally and postnatally—showed significantly greater gains in knowledge than mothers in the comparison group, gains were higher for the prenatal group (P < .001, effect sizes of .67 vs .27). The small percentages of mothers in the intervention group who did not use their kit (13% at baseline; 6% at 14 months) had significantly lower knowledge gains than those who had used it (P < .05).

Figure 3 ▶ compares knowledge-scale scores at baseline, 2-month, and 14-month follow-ups. For both language groups, mothers in the intervention group made significantly greater knowledge gains than their counterparts in the comparison group, even when we controlled for mother’s age and parity. Results for Spanish-speaking mothers in the intervention group showed nearly twice the effect size compared with that for English-speaking mothers in the intervention group (effect sizes at 2-month follow-up of .62 vs .38, P < .001). The baseline “knowledge gap” between English- and Spanish-speaking mothers was reduced by 40%.

FIGURE 3—

Knowledge-scale scores among English- and Spanish-speaking mothers in the intervention (who received the Kit for New Parents) and comparison groups, by time of data collection: California, October 2000 to May 2002.

Note. Dotted lines refer to the snowball-sampled comparison group. See “Methods” section for information on knowledge assessment.

Knowledge gains were greater with higher family use of the kit (P < .001, Figure 4 ▶). Composite scores were calculated to reflect the level of the mother’s and her partner’s use of the kit during the periods from baseline to 2 months and from 2 to 14 months. Gains were highest when both the mother and her partner used the kit before the 2-month interview and again before the 14-month interview (composite score of 4).

FIGURE 4—

Mothers’ average knowledge-score increases, by composite family use of the Kit for New Parents: California, October 2000 to May 2002.

Parenting Practices

At the 2-month follow-up, 48% of mothers who received the kit reported having changed their thinking or having done something differently because of the kit. Verbatim responses included:

I learned that I need to talk to him, spend time with him, read to him, because he’s not there just to lie in his crib. Now I see him respond to me when I talk to him like he wants to talk, and he smiles.

At the 2-month follow-up, mothers who received the kit prenatally were much more likely to report the subsequent enrollment of their infants in health insurance than mothers who did not receive the kit (adjusted odds ratio of 2.37, P < .001). There was no significant difference in enrollment between mothers who received the kit postnatally and mothers in the comparison group.

At the 14-month interview, there were statistically significant differences in parenting practices, with mothers who received the kit reporting a greater (1) number of childproofing steps taken (P < .01), (2) frequency of reading to infants (P < .01), (3) frequency of consistent medical care for infants (P < .01), and (4) frequency of safer bottle-feeding practices (P < .05). One mother reported:

It gave us ideas, so we bought covers for the plugs, a gate so he won’t get out, and a lock on the bathroom toilet so he won’t fall inside. When you just hear about it, you don’t really pay attention to it. But once you see it [in the kit] you pay more attention to it.

There were no significant differences for breastfeeding initiation or duration, age of first solid foods, co-sleeping practices, talking aloud to the baby, or having a bedtime routine for the baby.

Process Evaluation

One year after the statewide launch, approximately 500 000 kits were distributed. Kit administrators and providers were highly enthusiastic about it as a resource for new parents. Providers commented that the kit enhanced their classes and counseling with parents. They reported that it was an excellent addition to their health and parenting programs with underserved groups, including military families, migrant workers, incarcerated women in halfway houses, single fathers, and adolescents in family-life classes.

Providers offered important recommendations to improve the kit program, including organizing the content around the child’s age, rather than by topic; adding more information about breastfeeding, disabilities, co-parenting for shared custody, and other topics; offering the videos in a DVD format; reducing the size of the kit box; producing kits in more languages (especially Asian languages); and distributing the kit to parents of older children (aged 1–5 years).

DISCUSSION

There is increasing evidence that what occurs during pregnancy and early childhood profoundly affects children’s long-term health. Although research findings strongly suggest that interventions to support parents and their young children can be powerful investments, it has been challenging to translate these findings into successful, large-scale, affordable, and sustainable programs. Evidence of efficacy under highly controlled conditions often is not sufficient for policymakers who need confidence that programs will work in diverse, real-world settings. Green and Glasgow argued that public health research tends to neglect such assessment of external validity.30 They recommended that studies examine representative participants, realistic settings, and differential effects among subgroups over at least 1 year; dose–response effects; comparisons with results of other studies; diffusion; customization; realistic costs; and the intervention’s institutionalization and sustainability. These were also factors of interest to California policymakers, and each received favorable assessment in our study.

Kit Effectiveness for Diverse Groups

Our study provides evidence for the effectiveness of the kit in diverse community contexts. A high percentage of mothers—87%—reported using the kit within the first 2 months, and over half of them shared it with their partners. Nearly all users (94%) found the kit useful for important health and parenting issues. The literature suggests that the perinatal period is a time of increased learning for parents.21 The comparison group showed modest knowledge gains prenatally and no significant gains postnatally. By contrast, gains for kit recipients were significantly greater than gains for nonrecipients in both the short and long term. Knowledge gains were particularly impressive among Spanish-speaking mothers. Although they had substantially lower baseline knowledge scores than English-speaking recipients, the “knowledge gap” present among Spanish- speaking mothers was significantly reduced within 2 months after receiving the kit. Parenting knowledge gains showed a dose–response group effect with the level of family kit use over time.

Knowledge gains in this study compared favorably with results from other studies. The effect sizes for the kit recipients’ short- and long-term knowledge gains were approximately twice as large as those found in a national meta-analysis of mostly more intensive and more expensive interventions.16

The kit also affected important parenting practices. Mothers who received the kit reported higher rates of early health insurance enrollment for their infants and more positive practices related to infant safety, feeding, health care, and early reading than did mothers in the comparison group. Nonsignificant differences for breastfeeding and feeding solid food may reflect “ceiling effects”—more than 90% of both groups showed the desired practices. Nonsignificant differences related to co-sleeping safety and infant bedtime routines were expected because these items were included to explore practice areas about which there was little advice in the kit.

These results challenge traditional thinking about the weaknesses of materials-based health promotion. By combining multiple media and low-literacy formats with a mix of didactic content, emotional support, and behavioral modeling, the kit potentially leverages the findings of newer communication models that emphasize interactivity, flexibility for different styles of learning, and participatory design.22^,31^,32 The kit is an “active” communication approach that engages parents to define their own issues and preferred times and ways of using the materials.

The kit model is guided by public health’s socioecological framework, which posits that people learn and make changes within the context of family, community, and culture.23^,33 Providers use the kit as a complement to their existing programs. Current research indicates that such interpersonal connections should enhance the impact of this mass communication strategy.34 As a home-based resource, the kit promoted interaction among family members, and it was more effective when shared by the mother and her partner.

Refining and Extending the Kit Model

First 5 adopted many of the study’s recommendations by developing additional information; producing DVDs with the content organized by the child’s age (0–5 years); reducing the size of the kit box; and developing kits in Mandarin, Cantonese, Vietnamese, and Korean. The kit program has been customized into statewide initiatives in Alabama, Arizona, Kentucky, and Pennsylvania.

There are several limitations to this study. First 5’s iterative decisions to fund the study’s components precluded a randomized design and required recruitment of the comparison group 6 months after the intervention group. However, the 2 groups were equivalent on key demographic variables and yielded identical average baseline knowledge scores. For the long-term follow-up, a separate comparison group had to be recruited through snowball referrals from the original comparison group. The snowball comparison group was also equivalent to the intervention group on key background variables. Unlike the intervention group, the snowball comparison group had no previous experience with the assessment.

Because the study did not involve a representative sample of parents in California, the results cannot be statistically generalized statewide. However, the study involved diverse mothers from different state regions, from typical pre- and postnatal venues in which the kit is currently distributed. The range of domains and items represented in the knowledge scale was limited by the constraints of conducting baseline interviews that fit into the routine of busy providers in natural settings. It would be useful to explore additional knowledge areas with parents in future studies. Finally, although there were important self-reported practice results that favored the mothers in the intervention group in the cross-sectional analysis at 14 months, more study is needed to longitudinally evaluate kit-related practice changes and effects on child health outcomes.

Overall, the study has addressed key threats to internal validity, including potential selection, history, maturation, and attrition effects.35 The patterns of results found across groups, times, and dosages provide converging evidence that supports the overall construct and discriminant validity of the measures employed and of the evaluation design. The external validity of the study’s findings is further supported by favorable assessments of the 10 criteria proposed by Green and Glasgow.30

The kit program in California–and those adapted in other states—is an innovative, large-scale, health communication initiative for new parents. The evaluation results suggest that the kit has value as an effective and low-cost statewide resource to educate parents of children aged 0 to 5 years.