The Effect of Different Definitions of a Patient on Immunization Assessment

Abstract

Objectives. In this report, the authors compare immunization assessment using 2 definitions of a patient.

Methods. Two Clinical Assessment Software Application (CASA) assessments were performed. The first sampled 200 two-year-olds seen at least once since birth. The second sampled 200 two-year-olds seen in the previous year. Children with incomplete immunizations in the first sample were contacted.

Results. In the second sample, 72% of children had complete immunizations, compared with 46% in the first sample. In the first sample, 78% of children with incomplete immunizations had not been seen during the past year. Of 134 children in the first sample seen in the past year or successfully contacted, 75% had complete immunizations.

Conclusions. The CASA assessment's definition of a patient underestimates immunization rates.

Immunization is the most recognized method of clinical disease prevention. In 1998, 80.6% of children aged 19 to 35 months had received 4 doses of diphtheria and tetanus toxoids and pertussis, 3 of oral polio vaccine, and 1 of measles–mumps–rubella.1 Immunization rates are used as key indicators of quality of care provided by physicians and health care plans. High immunization rates are associated with appropriate use of preventive health care services and represent evidence of a medical home.2 Factors contributing to deficient immunizations include inadequate immunization history, the cost of vaccines, lack of insurance, inability of families to get appointments, lack of transportation, lack of an identified source of care, missed opportunities to immunize, and lack of knowledge by the parent.3^–10

In 1992, the Clinical Assessment Software Application (CASA) tool was designed by the Centers for Disease Control and Prevention as a standardized method of assessing documented immunization rates in a practice and determining causes of low immunization rates. The CASA assessment defines a patient as a child younger than 36 months seen at least once in the practice who has not been documented to have left the practice.

A CASA assessment performed in a hospital-based primary care practice showed unexpectedly low immunization rates. The medical staff of the practice hypothesized that most of the children with incomplete immunizations were no longer patients of the practice. This report describes the results of attempting to contact families of children with incomplete immunizations who had not been seen in the practice in the prior year to determine their current immunization status.

METHODS

In January and March 1998, 2 CASA assessments were performed on samples of 200 children from the pediatric practice of a Midwestern urban children's hospital. Continuity of care is provided by pediatricians, supervised residents, and nursepractitioners for approximately 27 000 visits a year. Approximately 95% of the children are eligible for free vaccines. The first assessment (unofficial CASA assessment) evaluated immunization rates of children aged 24 to 35 months who had been seen in the practice at least once in the previous year (December 1, 1996–November 30, 1997). The second assessment (official CASA assessment) evaluated children aged 24 to 35 months who had been seen at least once since birth (December 1, 1994–November 30, 1997). The CASA software generated the sample from a list of children obtained from the hospital billing system. Medical records personnel from the practice obtained the records for the assessment team from the health department, who reviewed charts and entered data into the CASA program. Children were considered completely immunized when they had received 4 doses of diphtheria and tetanus toxoids and pertussis, 3 of oral polio vaccine, and 1 of measles–mumps–rubella.

Children from the official CASA assessment with incomplete immunizations who had been seen at least once in the prior year were defined as being active patients and having deficient immunizations. Children with incomplete immunizations who had not been seen in the prior year were defined as “missing children.” The Institutional Review Board of University Hospitals of Cleveland approved the study. Informed consent was obtained from all contacted families.

Missing children's identifiers (child's hospital medical record number, mother's name, and child's social security number if available) and last known address and telephone number were obtained from the hospital computer system. In attempting to contact patients by telephone or letter, members of the health department (H.P., B.M., and C.M.) used immunization registry information, telephone directories, birth certificate data, and automobile registration data. Contact was attempted at all addresses and, if unsuccessful, a home visit was made to the last known address. Identifiers of children who were not located were given to the county Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) and to Cuyahoga County Health and Nutrition (the agency coordinating Medicaid enrollment) to determine if the child was presently enrolled.

Statistical analysis used χ² or the Fisher exact tests in Epi Info version 6.01 (Centers for Disease Control and Prevention, Atlanta, Ga).

RESULTS

The results of the 2 CASA assessments were significantly different for all factors except the potential increase in completion rate by eliminating missed opportunities (Table 1 ▶). In the unofficial CASA assessment, 131 children (66%) were insured by a Medicaid health maintenance organization (HMO), 29 (15%) by fee-for-service Medicaid, and 10 (5%) by commercial insurance; 29 (15%) were uninsured.

TABLE 1—

Results of Unofficial and Official Clinical Assessment Software Application (CASA) Assessments Performed on the Pediatric Practice of a Midwestern Urban Children's Hospital

	No. of Children With Complete Immunizations (%)
	Unofficial CASA Assessment	Official CASA Assessment	P
DTP4, OPV3, MMR1	144 (72)	92 (46)	.0000001
DTP3, OPV3, MMR1	170 (85)	114 (57)	.0000000
DTP1	188 (94)	170 (85)	.003
MMR1	176 (88)	116 (58)	.0000000
HIB3	178 (89)	140 (70)	.000003
HEP3	180 (90)	136 (68)	.0000001
Potential increase in completion rate through elimination of missed opportunities, %	(2)	(4)	.68
Dropout rate between DTP1 and DTP4, %	(17)	(37)	.001

Note. DTP = diphtheria and tetanus toxoids and pertussis; OPV = oral polio vaccine; MMR = measles–mumps–rubella; HIB = Haemophilus influenzae type b; HEP = hepatitis B. Numbers after abbreviations indicate numbers of doses administered.

Of the 108 children with incomplete immunizations in the official CASA assessment, 24 (22%) had been seen in the practice within the past year and were considered active patients. The remaining 84 (78%) had not been seen in the practice in the past year and were considered missing. Of these 84 children, 27 (32%) were not located and 39 others (12 who were living out of the county and 27 who were located only by the WIC and Cuyahoga County Health and Nutrition) were not contacted. Of the 18 children (21%) who were contacted, 8 had complete immunizations, 9 did not, and the parents of 1 child refused to participate. Of the 134 children who were active patients or had been contacted, 100 (75%) had complete immunizations. Of 188 children who might be active in the practice (200 minus 12 children who had moved out of the county), 100 (53%) had complete immunizations. Assuming that the 54 children not located had complete immunizations at the same rate as the 8 of 18 who were located, 66% of 188 children would have complete immunizations.

DISCUSSION

The CASA assessment software helps physicians gain an understanding of their immunization practices. The low potential increase in immunization rates brought about by eliminating missed opportunities in both assessments shows that when children are immunized in the practice, they receive all needed immunizations. CASA defines a patient as any child younger than 36 months seen at least once in the practice who has not been documented to have left the practice. Of the 108 children who had incomplete immunizations, 84 (42% of the sample of 200) had not been seen in the practice in the prior year. The CASA assessment requires that children leaving a practice be documented by progress notes or release of records. These children are eliminated from the sample. When a healthy child transfers care, the parents often carry the child's immunization records with them but do not request that records be forwarded to the new physician. Parents do not always notify their child's physician about transfer of care. Of the 84 missing children, 12 were living out of the county.

What defines a practice's patients for accurate assessment? Stokley et al. found that immunization rates in 7 pediatric practices varied greatly depending on whether the child made 1 visit, 2 or more visits, or a visit in the past 6 months, or was a patient of the practice by parental report. They recommended that there should be a consistent definition but offered no best definition.11 Massoudi et al., who found excellent immunization rates with the CASA assessment, defined a patient as a child making at least 3 visits to the practice. They had no data on immunization rates for children making at least 1 visit to the practice.12 Darden et al. evaluated 3 population definitions for immunization assessment: consecutive visits of children aged 2 and 3 years (consecutive method), a chart audit with a phone call to the family to verify patient status (active method), and a chart audit without a phone call (chart method). The consecutive (81% complete) and active (80% complete) methods were significantly different from the chart method (72% complete).13

Morrow et al. used the CASA assessment in 10 practices and found immunization rates for samples of patients meeting the following definitions: at least 1 well-child or immunization visit (70% complete), active patients on the basis of parental report (85% complete), at least 1 visit in the past year (79% complete), and consecutive patient visits (78% complete). In practices where more than 50% of patients were insured by Medicaid, the completion rates were 54%, 77%, 72%, and 70%, respectively.14 Irigoyen et al. reported evidence linking poor health outcomes in children to “disengagement,” or dropout from medical homes. They defined children as being disengaged when they did not visit their medical home within 1 month of a preventive health care visit recommended by the American Academy of Pediatrics. When they excluded children who had disengaged from medical care, the immunization rates of practices in New York City rose from 51% to 80%.15

The use of 2 different definitions of an active patient in our practice produced significantly different results for a CASA assessment (72% with complete immunizations for children seen in the past year vs 46% for children in the official CASA assessment). The 84 children in the latter assessment who had not been seen in the past year met Irigoyen and colleagues' definition of disengagement. By this definition, our immunization rate would have been 79% (92 of 116 children). Our data are similar to those of Irigoyen et al. and Morrow et al. In our study, contacting families whose child had deficient immunizations and had not been seen in the past year yielded immunization rates of 54%, 66%, and 75%, depending on the method of accounting for missing data. Rodewald et al. discussed the effect of the use of appropriate data sources for immunizations and the effect of missing records on immunization rates. They did not discuss the importance of the definition of a practice's patients in the calculation of the immunization rates.16 Our study shows this.

Appropriate immunization rates are a marker of good preventive pediatric care. Accurate assessment of immunization rates is important. A best definition of a practice's patients must be agreed on for immunization rates to be accurate. The definition in the CASA assessment—being seen once since birth—underestimates the immunization rates of a practice.