Effectiveness of a Community Coalition for Improving Child Vaccination Rates in New York City

Abstract

We used a retrospective, matching, birth cohort design to evaluate a comprehensive, coalition-led childhood immunization program of outreach, education, and reminders in a Latino, urban community. After we controlled for Latino ethnicity and Medicaid, we found that children enrolled in the program were 53% more likely to be up-to-date (adjusted odds ratio = 1.53; 95% confidence interval = 1.33, 1.75) and to receive timely immunizations than were children in the control group (t = 3.91). The coalition-led, community-based immunization program was effective in improving on-time childhood immunization coverage.

The most effective strategies for improving community-wide childhood immunization rates combine reminders, tracking, and outreach.^1,2 Most evidence about these strategies derives from provider-driven programs, with very little from community-driven programs.^3–6 Our immunization program, Start Right, is community driven, but until recently, we have not had community-specific data for demonstrating its effectiveness, relying instead on comparisons to national data.^7,8 In this study, we re-examined the program's effectiveness with a comparison cohort in our own community.

METHODS

Prior to the intervention, our Latino, low-income community in New York City had childhood immunization rates of 57%—well below city and national rates.⁷ To address this problem, Start Right, our 23-partner coalition, adapted national and citywide materials for its own package of bilingual and community-appropriate immunization-promotion materials; trained peer health educators; implemented personalized immunization outreach and promotion within social service and educational programs; provided outreach, education, and reminders to parents; and supported provider immunization delivery^1,5,6,9–19 (Table 1). Enrollment, tracking, and accountability were shared among partners.^20,21 Participants were recruited through the Special Supplemental Nutrition Program for Women, Infants, and Children (27%), facilitated State Children's Health Insurance Program enrollment program (20%), childcare and Head Start centers (20%), parenting assistance programs (19%), and housing and tenant associations (9%). The refusal rate was 2%.

TABLE 1.

Community-Based Immunization-Promotion Program Components

Strategy	Implementation	Result
Membership	A total of 23 community organizations: social services, housing, faith-based, childcare providers, primary care providers, the city health department.	Broad-based constituency.
Leadership	Shared leadership, academic, 2 community organizations.	Enhanced coordination and community ownership.
Program design	Used community organization needs and experiences and provider perspectives to select evidence-based best practices and to develop bilingual and community-specific outreach and educational materials.	Blend of own experience and national evidence-based insights; enhanced community ownership and relevance of immunization materials.
Integration into existing services	Immunization promotion designed as an “add-on” to 5 basic community social service and education programs: parenting assistance, childcare, facilitated enrollment for SCHIP, housing assistance, and WIC. Implementation guide tailored the intervention to each type of program.	Immunization promotion not stand-alone, but incorporated into ongoing programs working with parents of young children.
Staff/peer training	Social service and educational staff or peers trained in immunization outreach, education, reminders and tracking with our own bilingual materials and curriculum, including Immunization 101, Card Reading, Tips on Educating Parents, and Methods for Reminding and Tracking Families.	A total of 998 community staff trained in 2000 to 2007, with 150 to 200 active at any given time; enhanced outreach and sustainability through the long-term presence of trained staff in the community.
Parental empowerment	Parents learn about immunizations, immunization cards, and the immunization schedule from peers or trusted staff through one-on-one and group sessions. Use of coalition-developed, bilingual, low-literacy materials.	A total of 10 251 parents participated, with 60% receiving one-on-one education, and 40% receiving both group and one-on-one education. Parents like the program and recommend it to friends.
Reminders and recall	Parents given calendar dates for when shots are due, reminder phone calls and cards before each immunization due date, and recall phone call, card, or visit if missed.	Average of 2.7 calls, reminders, or visits per child.
Tracking	Decentralized immunization tracking at each member organization, consolidating data from parents’ cards and electronic registries.	A total of 184 896 immunization records in the consolidated coalition database, including 24% recorded only on the child's vaccination card.

Note. SCHIP = State Children's Health Insurance Program; WIC = Special Supplemental Nutrition Program for Women, Infants, and Children.

Study Design

We used a quasi-experimental, retrospective, birth cohort design^22,23 with 10 857 children born between April 1999 and September 2003 at the primary community hospital (76% of community births) who resided in the community's zip codes. Following National Immunization Survey methodology, we created 4 annual cohorts of children, aged 19 to 35 months as of April 1 of each year, 2002 to 2005 (n = 2879, 2788, 2653, and 2577, respectively). The annual cohorts were divided into intervention and control groups. The study was conducted retrospectively in 2006 to 2007.

The hospital database was used for demographics; the New York Citywide Immunization Registry (CIR) for immunization records. CIR is a population-based registry with mandated provider reporting. Most children (88%) had a record in CIR (n = 9511; 93.9% of the Start Right group, 87.0% of the control group).

Outcome measures were up-to-date immunizations for the 4:3:1:3:3 series (4 diphtheria-tetanus-pertussis [DTaP]; 3 polio; 1 measles-mumps-rubella; 3 Haemophilus influenza b; and 3 hepatitis B²⁴) and timeliness of the last DTaP dose, known as DTaP4, measured by elapsed days between the date a child became overdue and the immunization date for children with a DTaP4 dose (n = 5059).²⁵ Significance of differences in coverage and timeliness were assessed with the χ² test and the unpaired 2-sample Student's t test. Logistic regression was used to estimate adjusted odds ratios (AOR) for the intervention effect on immunization status, controlling for Latino ethnicity and Medicaid enrollment (n = 10 231).²⁶ We used Stata 9.0 (StataCorp, College Station, TX) for statistical analysis.

RESULTS

Across all birth cohorts, 8.2% (n = 895) enrolled in Start Right. Compared with control groups, children in Start Right were similar in age (mean = 27.4 months) but were more likely to have Medicaid (85.1% vs 78%; χ² = 27.8; df = 2; P < .001) and be Latino (92% vs 85.1%; χ² = 39.1; df = 2; P < .001).

Children in Start Right achieved significantly higher (11.1%) immunization coverage than did control children (χ² = 44.6; P < .001; Figure 1) and completed the immunization series earlier, by 11 days (t = 3.91). After controlling for ethnicity and Medicaid, children in Start Right were 53% more likely to be up-to-date than were control children (AOR = 1.53; 95% confidence interval [CI] = 1.33, 1.75). Neither Latino ethnicity (AOR = 1.07; 95% CI = 0.93, 1.24) nor Medicaid (AOR = 1.05; 95% CI = 0.95, 1.16) significantly influenced immunization coverage.

FIGURE 1.

Differences in registry-reported immunization coverage rates from the community average: New York City, 2002–2005.

DISCUSSION

Despite similarities at birth and control for ethnicity and Medicaid insurance, less than 3 years later, the children in the Start Right intervention had significantly higher immunization coverage rates than did the rest of their birth cohort. The community birth cohort control further validates our earlier findings regarding the effectiveness of our community-driven, comprehensive immunization-promotion intervention.^7,8 The observed increased coverage is not the result of a higher community-wide immunization level, but of the immunization-promotion program year after year. This improvement in coverage is well within the range expected for reminder–recall interventions and other community-based programs.⁵

The major limitation to this study was incomplete data reporting to the CIR.^27,28 In contrast to our previous reports, which included parent-held vaccination card data, we only included registry data in this study. Incomplete provider reporting of immunizations to the CIR, estimated at 85%, may have downwardly biased immunization coverage.²⁹ As indicated in Table 1, including only immunizations reported to the CIR excluded 24% of immunizations recorded on the intervention children's vaccination cards.

We attribute the success of the program to a number of factors, including community ownership of the program, integration of immunization promotion into social service and educational programs, training of a large cadre of peer educators, intense parental education and empowerment, and culturally appropriate reminders arising out of the context of our intervention. We hope that this study helps others seeking ways to embed health care promotion into community programs.

Peer Reviewed

Contributors

All authors except R. Andres-Martinez (who joined the team later) participated in the design of the study. M. Irigoyen and M. S. Stockwell obtained the control sample data, and S. Chen and R. Andres-Martinez assembled and analyzed the data. L. Guzman helped develop the Start Right database and, with O. Pena, assured high-quality data inputs from partners. The article was drafted by S. E. Findley, M. Irigoyen, M. Sanchez, and M. S. Stockwell, with assistance in developing the program summary in Table 1 from M. Mejia and R. Ferreira, who also reviewed the final draft of the article.