Assessment of Kindergarten Immunization Rates in Colorado: School Self-Reports vs. Health Department Audits, 2004–2005

SYNOPSIS

In 2005, the Colorado Department of Public Health and Environment audited a sample of kindergarten school records to determine vaccination coverage at school entry. In addition to the audit, the traditional method of collecting immunization data by self-reports from schools continued through that school year. The results of the two surveys were compared. The audit results indicated that 76.3% (n=1,776; 95% confidence interval 73.2, 79.4) of Colorado's kindergarteners received all required vaccines. In contrast, the series coverage estimated from school self-reports for the same time frame was 89.4% (n=46,559). Self-reports by school staff in Colorado appear to overestimate the immunization status of children entering kindergarten. Because more than three-quarters of U.S. states use some form of school self-report to assess immunization status, this finding has significant implications for most state health departments.

The enactment of school immunization laws has significantly increased the proportion of school-aged children vaccinated and, consequently, reduced the incidence of vaccine-preventable diseases (VPDs) in the United States.¹ Laws concerning vaccination date from 1855, when Massachusetts made smallpox vaccination obligatory for school entry.² Over time, more states passed laws requiring certain vaccinations as a prerequisite for children entering school. Kindergarten school entry laws now exist in all 50 states and the District of Columbia (DC).³

To ensure high coverage levels are maintained, the Centers for Disease Control and Prevention (CDC) requests that state immunization programs monitor vaccination rates in schools. Each state has a system for determining the immunization coverage of children entering school. In 2005, the CDC conducted a survey to identify the assessment techniques used by states to monitor these rates. Thirty-nine (76.5%) of the states, including DC, reported that school staff reviews immunization records to determine how many children have received required vaccinations. The remainder of the states (23.5%) use health department staff or some combination of school and health department staff to audit records and determine children's up-to-date (UTD) status.³

Methods for monitoring vaccination rates vary greatly among the 39 states using school self-reports. Some states rely on school nurses to review immunization records while others use school administrative staff to review the records and provide a summary report to the immunization program. Documentation that has been signed by a medical professional is required by some states and not by others. Even within a single state, different schools may approach their assessment and data collection procedures differently. Furthermore, the antigens assessed vary from state to state. These variations lead to questions of data quality and comparability across states.

Prior to the 2004–2005 school year, the immunization program in Colorado relied on schools' self-reports to determine students' vaccination status. The number of children UTD and not UTD at school entry, the number with a medical, personal, or religious exemption, and the number that were in the process of becoming UTD were reported by the schools. Concern about the accuracy of self-reported data led the Colorado Department of Public Health and Environment (CDPHE), in consultation with the National Immunization Program at CDC, to evaluate the quality of information from self-reports and to test a new data collection protocol for assessing coverage levels.

The Colorado Board of Health requires that children attending kindergarten have documentation of having received five doses of diphtheria/tetanus/pertussis vaccine or four if the fourth dose was received after the fourth birthday; four doses of polio vaccine or three if the third dose was received after the fourth birthday; two doses of measles/mumps/rubella (MMR) vaccine; three doses of hepatitis B vaccine; and either one dose of varicella vaccine or a history of having had varicella disease (series 4:3:2:3:1).⁴ The rules also require that schools have on file an official Certificate of Immunization or an approved alternate document for every enrolled student. Each of these must include dates of vaccination or the appropriate documentation for medical, religious, and philosophical exemptions. Children who are not documented as UTD at school entry are given 14 days to resolve the issue by submitting the proper documentation or a written plan demonstrating that the child is in the process of becoming UTD within the minimally recommended time intervals.

METHODS

During the 2004–2005 school year, the CDPHE audited school records to determine whether the regulations were being followed and whether the self-reports by schools were accurate. The audit was conducted by local and state public health staff to determine UTD status. The state was divided into five regions (Denver Metro, East Central, Northeast, Southeast, and Western Slope) and a two-stage cluster sample design was employed. A simple random sample of schools was selected from each of the five regions.

For each region, the size of this sample was dependent upon the variation in enrollment from school to school, the regional coverage level estimated from a survey conducted in 1992 (the last year in which there was a regional audit), and the number of records that the CDPHE chose to audit at each school. A total of 123 schools were sampled throughout the state. Twenty children were randomly selected from each school; if the school had less than 20 enrolled children, all were sampled. Local or state public health staff audited each child's immunization record. Audit information collected included age, gender, race, ethnicity, country of birth, vaccination date, exemption status, type of school immunization staff, and the date that health department staff visited the school to perform the audit. The type of documentation used to record vaccination status was also collected. This documentation included an official Certificate of Immunization or an alternate document. Data were collected between January 1 and March 31, 2005.

These data were analyzed to determine whether (1) each child had the required vaccination documentation in the school record, (2) each child had all required vaccines and had completed the 4:3:2:3:1 series, and (3) there were any factors significantly associated with a child being UTD. Children were classified as UTD if the school record indicated that they had received the complete series of required vaccinations. UTD status was calculated by region as well as for the entire state. State estimates were weighted to ensure that each region was represented proportionally. The statistical analysis was performed using SAS.⁵ The SAS procedure PROC SURVEYMEANS was employed to account for the complex survey sample design.

In addition to the audit, the traditional method of collecting immunization data by self-reports from schools continued through the 2004–2005 school year. Schools were mailed a form and asked to report the number of students who were UTD, those who had taken an exemption, and those in the process of becoming UTD. The aggregate kindergarten data were summed in Microsoft® Excel⁶ and compared to the results obtained from the audit of the school records.

RESULTS

Documentation of vaccination status

The majority of the kindergarteners in the sample (62.7%, or 1,531) had approved certificates of immunization in their school records; 35.0% (713) had an unapproved document on file, and a small percentage had no immunization document in their school record (2.3%, or 42). Although an approved document of immunization is required according to the Rules of the Colorado Board of Health, the percentage of children with an approved document on file varied across regions from a low of 36.6% in the Northeast region to 98.7% in the Western Slope region.

Coverage

Vaccination coverage estimates are shown in Table 1. About 76.3% (n=1,776; 95% confidence interval [CI] 73.8, 78.8) of Colorado's kindergarteners received the required vaccines as indicated by school records. In contrast, the series coverage estimated from school self-reports for the same time frame was 89.4% (n=46,559). The series coverage based on school self-reports of 89.4% is 10 percentage points higher than the upper 95% CI based on the audits. The audit found that coverage on the individual vaccines ranged from 85.1% for MMR to 92.2% for varicella. No significant difference was observed in UTD status by month of data collection (p>0.05).

Table 1.

Vaccination coverage in Colorado kindergarteners for the 2004–2005 school year

^aFive doses of diphtheria/tetanus/pertussis (DTaP) vaccine or four if the fourth dose was received after the fourth birthday

^bFour doses of polio vaccine or three if the third dose was received after the fourth birthday

^cTwo doses of measles/mumps/rubella (MMR) vaccine

^dThree doses of hepatitis B vaccine

^eEither one dose of varicella vaccine or a history of having had varicella disease

^fFive doses of DTaP vaccine or four if the fourth dose was received after the fourth birthday; four doses of polio vaccine or three if the third dose was received after the fourth birthday; two doses of MMR vaccine; three doses of hepatitis B vaccine; and either one dose of varicella vaccine or a history of having had varicella disease

UTD = up-to-date

CI = confidence interval

Reasons for non-UTD status

Overall, 6.1% of the kindergarteners took some kind of exemption. A small percentage (2.8%) took exemptions for all vaccines (0.2% medical, 0.2% religious, and 2.4% personal) (Table 2). An additional 3.3% of kindergarteners took exemptions for at least one vaccine in the series but not all vaccines. Kindergarteners who were in the process of becoming UTD accounted for another 0.2%. For 17.4% of kindergarteners, there was no recorded reason for being not UTD. Of these, about 1.3% of kindergarteners were categorized as not UTD because there were no records in the school file. For the remaining children who were not UTD (16.1%) and did have records in their school file, no explanation for their status was apparent. Those children had not taken an exemption nor were they in the process of becoming UTD.

Table 2.

UTD status and reasons not UTD

^aFive doses of diphtheria/tetanus/pertussis vaccine or four if the fourth dose was received after the fourth birthday; four doses of polio vaccine or three if the third dose was received after the fourth birthday; two doses of measles/mumps/rubella vaccine; three doses of hepatitis B vaccine; and either one dose of varicella vaccine or a history of having had varicella disease

UTD = up-to-date

Factors associated with vaccination status

No association was detected between UTD status and a child's gender, race, or ethnicity. However, there was an association between country of birth and vaccination status in two of the five regions. In the Northeast, 77.5% (334 of 431) of the children born in the U.S. were UTD, compared to 59.6% (31 of 52) who were born out of the country (χ²=8.03, p<0.01). In the Western Slope region, 82.2% (194 of 236) of children born in the U.S. were UTD, while only 72% (121 of 168) of the non-U.S.-born children were UTD (χ²=5.92, p=0.02).

The composition of the school personnel who monitor immunization records was also significantly related to UTD status for the same two regions. Children in these regions were more likely to be UTD if they attended a school with a school nurse. In the Northeast region, 81.8% (148 of 181) of kindergarteners in schools with a nurse were UTD, while other schools had a coverage rate of 71.6% (217 of 303) (χ²=6.30, p=0.01). In the Western Slope region, those figures were 81.4% (301 of 370) and 68.8% (55 of 80), respectively (χ²=6.32, p=0.01).

DISCUSSION

The results of this survey reveal that data self-reported by school staff in Colorado differ significantly from the data collected by auditing school records and overestimate the immunization status of children entering kindergarten. The school audit data indicated that most of the children who were not UTD at kindergarten entry had no information in their record to explain why they were not in compliance with the school law; they were not in process, nor had they taken an exemption. A possible explanation is that school staff believed these children were UTD for all required vaccinations and reported them as such, resulting in higher coverage estimates for the self-reported rates than the rates from the school record audits.

Knowing the correct immunization status of a child is important to assure healthy school settings, programmatic decision making, identifying pockets of need, and identifying vulnerable children during an outbreak situation. It is probable that some of the children classified as not UTD did not have their current immunization information in their school records and, thus, were truly UTD. A subsequent study in progress in Colorado involves collecting immunization histories from providers to compare with and investigate the accuracy of immunization data in school records.

Healthy People 2010 objectives state the goal for individual vaccine coverage is to maintain kindergarten rates at the 95% level.⁷ Without accurate estimates of coverage, there is no way to determine whether those goals are being met or if progress is being made to reach them. While UTD status for the required series in Colorado, as estimated by the school audit, was found to be only 76.3%, coverage for each of the individual vaccines exceeded 85%. The difference between the coverage for individual vaccines and the series occurs because the children missing one vaccine are not the same children who are missing another vaccine; thus, combining them amplifies the perception of undervaccination. Although series coverage estimates can be used to identify individuals for follow-up, they may overrepresent risk and do not necessarily provide a clear picture of the susceptibility of children to specific VPDs.

Factors associated with a kindergartener being UTD included having a school nurse on site and being born in the United States. Both of these findings were true only for certain regions and thus the study should be replicated to determine if these associations are found in subsequent school years.

The data from this project are being used in Colorado to make school personnel aware of the current percentage of children not UTD who were reported as UTD, to educate them about the importance of complying with the school law, and to develop resources for school personnel to evaluate the immunization status of students. Health departments should work to facilitate collaboration with state and local departments of education, in addition to school personnel responsible for monitoring vaccination status. Developing a collaborative approach to monitoring vaccinations may increase the support for this activity in both public health and educational settings.

Health departments that continue to use data from school self-report should consider communicating the importance of accurate coverage estimates to the people reviewing immunization records. Additionally, school staff needs to be aware of the public health importance of reporting accurate school coverage data. Because of the close contact children in school have with other children, ensuring that they are fully immunized is imperative. In the event of an outbreak, the only way for the state health department to identify susceptible children may be through the reports provided by the schools.