Vaccine adverse events in a safety net healthcare system and a managed care organization

Abstract

Background:

The Institute of Medicine, in a 2013 report, recommended that the Vaccine Safety Datalink (VSD) expand collaborations to include more diversity in the study population. Kaiser Permanente Colorado (KPCO), an established VSD site, partnered with Denver Health (DH), an integrated safety net healthcare system, to demonstrate the feasibility of integrating DH data within the VSD. Prior to incorporating the data, we examined the identification of specific vaccine associated adverse events (VAEs) in these two distinct healthcare systems.

Methods:

We conducted retrospective cohort analyses within KPCO and DH to compare select VAEs between the two populations. We examined the following associations between January 1, 2004 and December 31, 2013: Measles, Mumps, and Rubella (MMR) vaccine and febrile seizures in children 2 years and younger, intussusception after rotavirus vaccine in infants 4–34 weeks, syncope after adolescent vaccines (Tetanus, Diphtheria, acellular Pertussis; Meningococcal and Human Papillomavirus) in adolescents 13–17 years and medically attended local reactions after pneumococcal polysaccharide (PPSV23) vaccine in adults 65 years and older. Both sites used similar data procurement methods and chart review processes.

Results:

For seizures after MMR vaccine (KPCO – 3.15 vs. DH – 2.97/10,000 doses) and syncope after all adolescent vaccines (KPCO – 3.0 vs. DH – 2.37/10,000 doses), the chart confirmed rates were comparable at the two sites. However, for medically attended local reactions after PPSV23, there were differences in chart confirmed rates between the sites (KPCO – 31.65 vs. DH – 14.90/10,000 doses). For intussusception after rotavirus vaccine, the number of cases was too low to make a valid comparison (KPCO – 0 vs. DH – 0.13/10,000 doses).

Conclusion:

We demonstrated that data on important targeted VAEs can be captured at DH and rates appear similar to those at KPCO. Work is ongoing on the optimal approach to assimilate DH data as a potential safety net healthcare system in the VSD.

1. Background

The Institute of Medicine (IOM), in its 2013 report on childhood immunization schedule and safety, recognized the Vaccine Safety Datalink (VSD) as the nation’s premier vaccine safety data source for studying the safety of the childhood immunization schedule [1]. VSD is a collaborative project between the Immunization Safety Office of the Centers for Disease Control and Prevention and nine integrated healthcare plans (IHPs) across the United States. The VSD uses vaccine registry and electronic health record (EHR) data from each participating site to conduct postmarketing surveillance of vaccine safety. The participating MCOs comprise a population of more than 9 million members annually (3% of the US population) [2–4].

The IOM, however, noted that VSD may have limited socioeconomic diversity, although there are no significant differences in gender, race, ethnicity, and educational attainment between the VSD population and the general U.S. population [5]. Since the IHPs that contribute data to the VSD are largely private insurance plans, its population has a smaller percentage of low-income individuals and may not be representative of the population of the United States. The IOM recommended that VSD expand collaborations with new health plan partners to include more diversity in the study population. In response to this recommendation, Kaiser Permanente of Colorado (KPCO), a VSD study site for more than 14 years, collaborated with Denver Health (DH), an integrated safety net healthcare system and explored the feasibility of integrating DH data into the VSD.

The feasibility of obtaining demographic, vaccine and medical encounter data from DH and incorporating that data into the VSD is described elsewhere [6]. Prior to incorporating DH data into the VSD and using that data for vaccine safety studies, we examined the identification of vaccine associated adverse events (VAEs) in these two distinct healthcare systems. Our primary objective was to capture and compare select known established vaccine-adverse event pairs between the two populations (KPCO and DH) in different age groups in pre-defined risk windows identified in previous studies. Specifically, we looked at the following known VAE associations:

1. MMR (Measles, Mumps, and Rubella) vaccine and febrile seizures in children 2 years of age and younger.

2. Rotavirus vaccine and intussusception in infants 4–34 weeks of age.

3. Tdap (Tetanus, Diphtheria, acellular Pertussis), MCV4 (Quadrivalent Meningococcal Conjugate), and HPV (Human Papillomavirus) vaccines and syncope in adolescents 13–17 years of age.

4. PPSV23 (Pneumococcal Polysaccharide 23-valent) vaccine and medically attended local reactions in adults 65 years of age and older.

2. Methods

We conducted retrospective cohort analyses within the two healthcare systems, KPCO and DH. KPCO primarily is a closed panel, group model IHP with a membership of over 600,000 individuals in the Denver metro area. KPCO joined the national VSD project in 2001. As part of the VSD, KPCO prepares weekly and yearly electronic data files containing information on demographics, vaccination history, and medical encounters in the outpatient, urgent care, emergency department (ED), and inpatient settings. Since joining the VSD, KPCO has participated in more than 100 vaccine safety studies.

DH is an integrated urban safety net health system providing full spectrum health care to socioeconomically disadvantaged and racially diverse populations in the Denver metropolitan area. It provides care to over 200,000 unique patients a year; over 147,000 of these receive primary care at DH. DH has a robust immunization program and registry that serves as a repository for all vaccine inventory and administration in the DH system. It has an advanced set of information services to support clinical care and research. The socioeconomic, racial and ethnic constitution of DH’s population and its ability to capture clinical and vaccine data for research purposes make it an ideal potential candidate for integration into the VSD.

For all VAEs studied, we included vaccinations received between January 1, 2004 and December 31, 2013. Vaccines licensed after January 1, 2004 were included in the analyses when they began routine use in either system. We did not assess the accuracy of the vaccination status in this study as data quality analyses for the electronic vaccination records within the VSD and DH have found high levels of accuracy when compared with medical records [7,8].

The VAEs identified electronically were confirmed using chart reviews. KPCO and DH used similar data procurement methods and chart review processes in examining the vaccine-adverse event associations. For each outcome, a list of potential cases with their medical record number, vaccination date and outcome date were identified electronically. All electronically identified cases were chart reviewed if the number was less than 50, or else a random sample of 25 charts or 10% of the charts was reviewed (whichever was greater). Trained medical chart abstractors used a standard chart review tool to confirm the electronically derived data using clinical information from the EHR. Brighton Collaboration case definition was used for reviewing medical records for local reactions and intussusception [9,10] (Brighton Collaboration definitions are not available for febrile seizures and syncope). For each VAE, the methods are described below and summarized in Table 1.

Table 1.

Criteria for vaccine-adverse events.

Vaccine-adverse event	Age group	Vaccines	Time period	ICD-9-CM codes for outcome	Risk interval	Setting	Chart review
MMR vaccine and febrile seizures	2 years and younger	MMR	Jan 1, 2004 -Dec 31, 2013	345.x, 780.3x	7–10 days	ED/IP/UC	Evidence of fever along with seizure, in the absence of intracranial infection, metabolic disturbance or history of afebrile seizures [25]
Rotavirus vaccines and intussusception	4–34 weeks	Rotarix, RotaTeq	Aug 1, 2006 - Dec 31, 2013	543.9, 560.0	1–7 days	ED/IP/UC	Evidence of intestinal obstruction, intestinal invagination, and venous congestion [9]
Adolescent vaccines and syncope	13–17 years	Tdap	Sep 1, 2005 - Dec 31, 2013	780.2	Day 0	OP/ED/UC	Evidence of transient loss of consciousness associated with the inability to maintain postural tone [26]
		MCV4	Jun 1, 2005 - Dec 31, 2013
		HPV	Aug 1, 2006 - Dec 31, 2013
PPSV23 and medically attended local reactions	65 years and older	PPSV23	Jan 1, 2004 –Dec 31, 2013	682.3, 682.6, 682.9, 729.81, 729.5, 995.3, 709.8, 709.9, 289.3, 683, 785.6, 999.3, 999.5, 999.9, 995.2	1–4 days	OP/ED/UC	Description of morphological or physiological changes at or near the injection site [10]

Vaccines: MMR - Measles, Mumps, and Rubella, Tdap - Tetanus, Diphtheria, acellular Pertussis, MCV4 – Meningococcal, HPV - Human Papillomavirus, PPSV23 - Pneumococcal Polysaccharide 23-valent. ICD-9-CM - International Classification of Diseases, 9th Revision.

Setting: ED – Emergency Department, IP – Inpatient, UC – Urgent care, OP – Outpatient.

2.1. MMR vaccine and febrile seizures

At each site, all children ages 2 years and younger who had received their first dose of MMR vaccine between January 1, 2004 and December 31, 2013, and had a primary care visit between 6 months and 2 years of age were eligible for inclusion. Primary care was defined as a visit in a pediatrics, family practice, or internal medicine clinic. We examined febrile seizures in the 7–10 days following the administration of the MMR vaccine using International Classification of Diseases, 9th Revision (ICD-9) codes 345.x (epilepsy) and 780.3x (convulsions) in the ED, inpatient or urgent care setting [11]. All electronically identified cases were chart confirmed at their respective sites.

2.2. Rotavirus vaccine and intussusception

The cohorts at both sites included infants between the ages of 4 and 34 weeks (inclusive) who had received a rotavirus vaccine (monovalent or pentavalent) between August 1, 2006 and December 31, 2013. We identified intussusception events electronically in the ED, inpatient or urgent care setting within 1–7 days of rotavirus vaccination using ICD-9 codes 543.9 (other and unspecified disease of the appendix including intussusception) and 560.0 (intussusception) [12]. Any electronically identified cases were chart reviewed.

2.3. Adolescent vaccines and syncope

For all adolescent vaccine associated syncopal events, we examined separate cohorts for each of the three vaccines recommended by the Advisory Committee on Immunization practices (ACIP) during adolescence—Tdap, MCV4, and HPV. The cohorts included all adolescents 13–17 years of age who had received the respective vaccine from the time the vaccine was first routinely used in either system (Tdap—September 1, 2005; MCV4—June 1, 2005; HPV—August 1, 2006) until December 31, 2013. Given that multiple vaccines can be administered on the same day, we also looked at the syncope rates associated with receiving different combinations of the three vaccines. The cases of syncope were identified using ICD-9 code 780.2 (syncope and collapse) in the outpatient, ED, or urgent care setting on the day of vaccination [13]. Electronically identified cases were chart reviewed and confirmed as vaccine associated if the syncope occurred within 15 minutes of receiving the vaccine. At KPCO, due to the high number of electronically identified cases (n = 239), a random sample of 73 charts were reviewed (25 each after MCV4 and HPV, and 23 after Tdap).

2.4. Pneumococcal polysaccharide vaccine 23-valent (PPSV23) and medically attended local reactions

Inclusion criteria were adults ages 65 years or older who had received a dose of PPSV23 vaccine between January 1, 2004 and December 31, 2013. We identified any local reactions electronically in the outpatient, ED, or urgent care setting within 1–4 days of vaccination using the following ICD-9 codes: 682.3, 682.6, 682.9 (cellulitis), 729.81 (limb swelling), 729.5 (pain in limb), 995.3 (allergy unspecified), 709.8 (other unspecified disorder of skin), 709.9 (unspecified disorder of skin and subcutaneous tissue), 289.3, 683, 785.6 (lymphadenitis), 999.3 (infection following infusion or vaccination), 999.5 (serum reaction), 999.9 (complications of medical care), 995.2 (adverse effect of a medication or biologic substance) [14]. We chose to exclude the outcomes on the day of vaccination (day 0) because diagnosis codes assigned on the day of vaccination often represent prevalent conditions and as such are not true adverse effects of the vaccine [14,15]. Electronically identified cases were chart-reviewed at DH; at KPCO a random sample of 37 cases (of 376 total) were chart reviewed.

3. Results

At KPCO, of the 44,416 children 2 years and younger who received the MMR vaccine within the 10 year period, 15 cases of febrile seizures were identified electronically within 7–10 days of vaccination. Fourteen were confirmed on chart review for a rate of 3.15 events/10,000 doses. From the 30,256 DH children who received the MMR vaccine, we identified 11 cases of febrile seizures electronically. On chart review, 9 were confirmed febrile seizure cases for a rate of 2.97 events/10,000 doses (Table 2).

Table 2.

Febrile seizures 7–10 days following MMR vaccination in children 2 years and younger.

	Kaiser Permanente, Colorado			Denver Health
	Electronically identified	Chart-reviewed	Chart-confirmed	Electronically identified	Chart-reviewed	Chart-confirmed
MMR doses	44,416			30,256
Febrile seizures	15	15	14	11	11	9
Events/10,000 doses (95% CI)	3.38 (2.04–5.60)		3.15 (1.87–5.32)	3.64 (2.01–6.56)		2.97 (1.55–5.72)

MMR - Measles, Mumps, and Rubella.

Following the 121,031 doses of rotavirus vaccine received at KPCO, we did not identify any cases of intussusception in the risk period. At DH, following the 79,610 doses, one case of intussusception was identified electronically and confirmed through chart review (Table 3).

Table 3.

Intussusception 1–7 days following Rotavirus vaccine in infants aged 4–34 weeks.

	Kaiser Permanente, Colorado			Denver Health
	Electronically identified	Chart-reviewed	Chart confirmed	Electronically identified	Chart-reviewed	Chart-confirmed
Rotavirus doses	121,031			79,610
Intussusception	0	0	0	1	1	1
Events/10,000 doses (95% CI)	0		0	0.13 (0.02–0.89)		0.13 (0.02–0.89)

For syncopal events after individual adolescent vaccines, the rate of electronically identified events was higher at KPCO (19.87–28.93/10,000 doses) versus DH (4.18–11.72/10,000 doses). On chart review, the rates of confirmed syncopal events following vaccination were lower at both sites (KPCO – 0–4.63/10,000 doses; DH – 0–2.79/10,000 doses). For the sum of all syncopal events after individually administered adolescent vaccines, KPCO and DH had comparable chart confirmed rates/10,000 doses (KPCO – 3.0 vs. DH – 2.37/10,000 doses) (Table 4). For syncopal events following different combinations of adolescent vaccines, low numbers of chart confirmed incident syncopal cases made comparisons between KPCO and DH challenging (data not shown).

Table 4.

Syncope within 15 minutes following individually^a administered vaccines in adolescents 13–17 years of age.

	Kaiser Permanente, Colorado			Denver Health
	Electronically identified	Chart-reviewed	Chart-confirmed	Electronically identified	Chart-reviewed	Chart-confirmed
Tdap doses	11,575			7173
Syncope	23	23	0	3	3	2
Events/10,000 doses (95% CI)	19.87 (13.20–29.90)		0	4.18 (1.35–12.97)		2.79 (0.70–11.15)
MCV4 doses	25,042			3453
Syncope	72	25	1	4	4	0
Events/10,000 doses (95% CI)	28.75 (22.82–36.22)		1.15 (0.36–3.65)	11.58 (4.35–30.87)		0
HPV Doses	49,774			27,299
Syncope	144	25	4	32	32	7
Events/10,000 doses (95% CI)	28.93 (24.57–34.06)		4.63 (3.08–6.96)	11.72 (8.29–16.58)		2.56 (1.22–5.38)
All dosesb	86,391			37,925
Syncope	239	73	5	39	39	9
Events/10,000 doses (95% CI)	27.66 (24.37–31.40)		3.00 (2.04–4.41)	10.28 (7.51–14.07)		2.37 (1.23–4.56)

Vaccines: Tdap - Tetanus, Diphtheria, acellular Pertussis, MCV4 – Meningococcal, HPV - Human Papillomavirus.

^aThe only administered vaccine on the day they received the vaccine.

^bSumming the individually administered vaccines (Tdap, MCV4, and HPV).

For medically attended local reactions following PPSV23 vaccine in adults, DH’s electronically identified and chart confirmed rates were lower compared to KPCO data (Electronic – KPCO 61.65/10,000 doses vs. DH 34.06/10,000 doses; Chart-confirmed – KPCO 31.65/10,000 doses vs. DH 14.90/10,000 doses) (Table 5).

Table 5.

Medically attended local reactions in the 1–4 days following Pneumococcal polysaccharide 23 valent vaccine in adults 60 years and older.

	Kaiser Permanente, Colorado			Denver Health
	Electronically identified	Chart-reviewed	Chart confirmed	Electronically identified	Chart-reviewed	Chart- confirmed
PPSV23 doses	60,987			9396
Medically attended local reactions	376	37	19	32	32	14
Events/10,000 doses (95% CI)	61.65 (55.73–68.21)		31.65 (27.48–36.44)	34.06 (24.08–48.16)		14.90 (8.82–25.16)

PPSV 23 - Pneumococcal Polysaccharide 23-valent.

4. Discussion

In this study, we demonstrated the feasibility of capturing VAEs at DH and compared their individual rates with those at KPCO, an established VSD study site. Overall, the number of vaccine doses received at DH was lower than that at KPCO (Tables 2–5), due to the smaller population base, but vaccination rates are high in both systems [6]. We found that for serious adverse events such as febrile seizures, the rates at the two sites were comparable. Due to the rarity of intussusception after rotavirus vaccine, we did not find enough cases to make a meaningful comparison. For syncope after adolescent vaccines, total chart confirmed VAEs (i.e., sum for all individual vaccines) were similar at KPCO and DH, although electronically identified cases at KPCO were almost three times higher than at DH. For medically attended local reactions after PPSV23, there were differences in chart confirmed rates between the two study sites.

MMR vaccination is known to be associated with an increased risk of febrile seizures in children, probably due to vaccine induced fever [16]. We compared the rates of febrile seizures after MMR vaccine in children at the two sites and found the rates to be comparable. Moreover, these rates were comparable to the rates of febrile seizures after MMR vaccine in previously published VSD studies [17,18]. The similar rates of febrile seizures after MMR vaccine at both DH and KPCO suggest that care seeking behavior and data capture were similar for a serious VAE in these two distinct systems.

Rotavirus vaccination is associated with a small increase in risk of intussusception, a type of intestinal blockage [12,19]. However, since it is a rare adverse event with an attributable risk of about 1.5–5.3 per 100,000 doses [12,19], we were unable to find any rotavirus vaccine associated intussusception cases at KPCO among the 121,031 doses of vaccine administered. At DH, we were able to confirm 1 case among the 79,610 doses administered (not statistically different from KPCO). Though there are no known studies on vaccine associated intussusception and socioeconomic factors, it is known that the risk of natural intussusception is higher among children living in lower socioeconomic conditions compared to those living in high socioeconomic settings [20]. In our study, we did not find an evidence of a difference between the two sites; however, there were very few observed cases to make a valid comparison.

Syncope following vaccine administration especially in adolescents is often discussed, but rarely studied using electronic data and chart review [21–23]. We saw a high rate of electronically identified syncopal events on the day of vaccination that were not confirmed on chart review, especially at KPCO. Chart review indicated that for the majority of the electronically identified cases, the diagnosis code assigned on the day of vaccination was for a prevalent event and not associated with the vaccine. Although there was some variability in the rates of chart-confirmed syncope between the 2 health systems, when all syncopal events after individual adolescent vaccines were combined, the rates were similar. Analysis of syncopal episodes after different combinations of adolescent vaccines was limited by low numbers of chart confirmed cases. Overall, syncopal rates after adolescent vaccines appear lower than those anecdotally reported by clinic staff. These VAEs may occur after the nurse administers the vaccines when the doctor has already seen the child, and the chart may have been closed. In the event that the patient recovers without incident, the chart may not be reopened to document this new information. It is unclear why there is a significant difference in electronically identified cases between KPCO and DH, but one possible explanation is differences in coding practices. Such differences are important to quantify before embarking on a large scale study of vaccine safety.

Local reactions are the most common adverse events after PPSV23 vaccine, but are usually mild [24]. We compared the rates of medically attended local reactions after PPSV23 at both sites, and found higher rates at KPCO compared to DH for both electronically identified and chart-confirmed cases. These results suggest that there may be socioeconomic and cultural differences in health care-seeking behavior between members at the two sites. It is possible that DH members may be less likely to use the healthcare system when they contract minor illnesses, but may seek medical care for a more serious condition. Such differences should be considered when conducting multi-site vaccine safety studies.

5. Limitations

In this study, we only examined acute events following vaccination in members who were vaccinated. The potential for misclassification of the outcome due to the vaccinated persons going outside their respective healthcare system in the event of an outcome is low. However, this may not be true when we look at long-term adverse effects of a vaccine or while studying the safety of the entire immunization schedule. Families may switch health plans, resulting in limited information on their vaccinations and follow-up information after their vaccinations. In that case, to reduce the potential of misclassification of the outcome (adverse event) and misclassification of the exposure (vaccination status), we would need members to be enrolled within the system and using the system. Current VSD sites utilize data from members enrolled in their health plans and using their system. Since the patients at DH are not entirely in a managed care environment, we used healthcare utilization to define the patient populations. In our previous work, [6] we found that the empaneled population at DH (those seen at least once in a primary care clinic in the past 18 months) provided the best comparison to KPCO’s enrolled population and could be used to define populations for vaccine safety studies. We explored this further when we examined the association between MMR vaccine and febrile seizures, where we required children to have a primary care visit between 6 months – 2 years of life and found the adverse event rates to be comparable between the two systems.

6. Conclusion

We have demonstrated that VAEs can be captured in a meaningful way in a safety net health system which unlike the other VSD sites is not a closed healthcare system. For serious VAE such as febrile seizures after MMR, rates appear quite similar at KPCO and DH. The very low numbers of intussusception cases after rotavirus vaccine made comparison between the two sites inconclusive. For syncope after adolescent vaccines, the rate of chart-confirmed cases for all vaccines is similar, although KPCO appears to have a substantially higher rate of syncope cases identified by using the electronic diagnosis code only. There appear to be lower rates of chart-confirmed local reactions after PPSV23 vaccine at DH than KPCO, which may suggest socioeconomic and cultural differences in health care-seeking behavior. It is these differences which point to the importance of incorporating safety net healthcare systems into adverse event reporting systems. That these selected vaccine-adverse events can be captured at DH is reassuring for the incorporation of data from this safety net healthcare system into the VSD and to use that data for future vaccine safety studies. However, it is important to understand the socioeconomic and cultural differences in presenting for care, especially for less serious VAEs such as local reactions, and define patient populations appropriately prior to conducting specific vaccine safety studies.