Abstract
Background:
In 2013 the Institute of Medicine suggested that the Vaccine Safety DataLink (VSD) should broaden its population by including data of more patients from low income and racially and ethnically diverse backgrounds. In response, Kaiser Permanente Colorado (KPCO) partnered with Denver Health (DH), an integrated safety net health care system, to explore the integration of DH data.
Methods:
We compared three different methods (reference date of September 1, 2013): “Empanelment” (any patient who has had a primary care visit in the past 18 months), “Proxy-enrollment” (two health care visits in 3 years separated by 90 days), and “Enrollment” in a managed care plan. For each of these methods, we compared cohort size, vaccination rates, socio-demographic characteristics, and health care utilization.
Results:
The empaneled population at DH provided the best comparison to KPCO. DH’s empaneled population was 111,330 (57,173 adults; 54,157 children), while KPCO had 436,290 empaneled patients (336,462 adults; 99,828 children). Vaccination rates in both health care systems for empaneled patients were comparable. Two year-old up-to-date coverage rates were 83.2% (KPCO) and 86.9% (DH); rates for adolescent Tdap and MCV4 were 85.5% (KPCO) and 90.6% (DH). There were significant differences in the two populations in age, gender, race, preferred language, and % Federal Poverty Level (FPL) (DH 70.7% < 100% FPL; KPCO 17.4%), as well as in healthcare utilization – for example pediatric emergency department utilization was twice as high at DH.
Conclusions:
Using a cohort of “empaneled” patients, it is possible to integrate data from a safety net health care system that does not have a uniform managed care population into the VSD, and to compare vaccination rates, socio-demographic characteristics, and health care utilization across the two systems. The KPCO-DH collaboration may serve as a model for incorporating data from a safety net healthcare system into the VSD.
Keywords: Vaccine safety, Safety net health system, Population surveillance
1. Background
The Vaccine Safety Datalink (VSD) is an established collaboration of nine integrated healthcare delivery systems and health plans which uses electronic health record (EHR) data to conduct observational studies of vaccine safety [1–5]. In January 2013, the Institute of Medicine published a report entitled “The Childhood Immunization Schedule and Safety: Stakeholder Concerns, Scientific Evidence, and Future Studies.” [6]. The report encouraged the VSD to expand collaborations with new partners in order to increase study population diversity, in particular by increasing the number of patients with low incomes or from racial and ethnic minority backgrounds. In response, Kaiser Permanente Colorado (KPCO), a VSD site for over 14 years, collaborated with Denver Health (DH), an integrated safety net health care system that provides services to socioeconomically disadvantaged populations primarily in the Denver, Colorado, metropolitan area. Safety net health care institutions offer care to all patients regardless of ability to pay, and a substantial share of their patients are uninsured or have Medicaid [7]. An integrated health care delivery system is an organized, coordinated network that provides a vertical continuum of services to a particular patient population or community [8].
Below we discuss different approaches to identifying comparable patient cohorts at DH and KPCO, and compare vaccination rates and socio-demographic characteristics in the two systems. Finally, we examine baseline health care utilization rates. Utilization patterns may differ between a managed care organization and a safety net health care system, and as utilization of emergency department (ED) and hospital services are often settings for identifying important outcome diagnoses in vaccine safety studies, it is critical to understand these differences and their potential impact prior to incorporating safety net data into the VSD.
2. Methods
2.1. Setting and data sources
KPCO is an integrated health care delivery system that provides health care to over 600,000 members in Colorado’s Front Range [9]. For routine patient management KPCO utilizes HealthConnect, the Hyperspace version of the Epic Systems Corporation’s outpatient, fully integrated, electronic health record (EHR). KPCO’s EHR infrastructure captures all vaccines administered in KPCO clinics and contracted facilities across the Colorado region as well as historical vaccinations reported by members. All vaccine data within the KPCO EHR is pulled into the HealthTrac registry, which is then programmed to identify gaps in coverage and recommend specific vaccines at medical visits. The HealthTrac registry has a two-way interface with Colorado’s state immunization registry, the first in Colorado. The data collected includes vaccine type, date of vaccine administration, vaccine manufacturer and lot number. KPCO VSD data were used to identify health care utilization, vaccine uptake, pregnancy information, gender, and age. Patient race, preferred language, and percent federal poverty level were estimated from the KPCO Geographically Enriched Member Socio-Demographics (GEMS) Member and Population report, which utilizes geocoding to calculate these variables based on residence address. KPCO has been a leader in the field of vaccine safety with the VSD, contributing to over 100 articles in the field, and routinely has the highest immunization rates in the state for all health plans.
DH [10–13] is an integrated urban safety net health system that includes a 477-bed hospital, three urgent care centers, seventeen school-based clinics, and nine federally qualified community health centers (FQHCs). DH’s Community Health Services provide services to 25% of Denver residents, 35% of Denver’s children, and a large proportion of Denver’s indigent and minority populations. One third of the births in Denver each year occur at DH (3000–3500 per year). Although a substantial proportion of DH’s patients are in public capitated insurance plans such as Medicaid Managed Care and the Child Health Insurance Program, a significant number of patients are covered by fee-for-service programs such as non-managed care Medicaid. Like Kaiser Permanente, DH is an integrated system that provides full spectrum health care including primary, specialty, urgent, and emergency department (ED) care, as well as hospitalization services. Unlike KPCO, DH owns its own hospital.
DH implemented its first electronic vaccine registry in 1996, and updated the registry in 2003 to a web-based application named VaxTrax. Until April 2016, the registry served as a repository for all vaccine inventory and administration in the DH system, for both children and adults. Data from VaxTrax are transmitted nightly via secure electronic communication to the Colorado Immunization Information System (CIIS). All DH clinics have capability to query CIIS to obtain vaccine data for immunizations given outside of the DH system. DH has a robust immunization program that consists of an agency-wide immunization policy and quality committee, a lead immunization nurse and team of outreach nurses, immunization advocates at every clinic and in every hospital unit, and a customized immunization registry. Due to the impact of the comprehensive immunization program and registry [14], DH’s immunization rates typically exceed national averages by substantial margins [15–17].
Historically, DH had a partial EHR with robust patient registries but hand-written medical notes. On April 9th, 2016, DH switched to Epic Systems as its EHR in the ambulatory, ED/urgent care, and hospital settings, and as the platform for its immunization application. As an early adopter of Healthcare Information Technology (HIT), DH is a Level 5 HIT enterprise, and one of the nation’s “most wired” health systems [18,19].
2.2. Cohort creation
Currently, all VSD sites including KPCO utilize data from members enrolled in their health plans. As DH has a mix of capitated (i.e., payment to a physician or group of physicians is a set amount for each enrolled person assigned to them, per period of time, whether or not that person seeks care) and fee-for-service patients they frequently use health care utilization to define their patient populations. We compared KPCO and DH populations using three different methods of defining patient cohorts. For each method, we chose September 1, 2013, as our reference date (details below).
Empanelment: A patient who accessed any primary care service at least once in the 18 months prior to the referent date (from 3/1/2012 to 9/1/2013) was considered to be an empaneled or active patient [20]. A primary care visit was defined as any encounter in the Family Medicine, General Internal Medicine, or General Pediatrics divisions.
Proxy-Enrollment: This cohort included patients with two ambulatory visits (excluding visits in the Emergency Department and urgent care) in the 3 years prior to Sept 1, 2013 (from 9/1/2010 to 9/1/2013) separated by at least 90 days, with at least one of the visits being a primary care visit [21].
Enrollment: Long used as the standard definition of an active patient in the VSD, “enrollment” in one of KPCO’s health plans defines membership in this cohort. This group included all members enrolled at KPCO as of September 1, 2013. At DH, this group included members with managed care payer sources including Medicare Managed Care, Medicaid Managed Care, Child Health Plan (CHP) and Employee Health Plan (DHMP). However, due to data sharing restrictions between DH’s delivery system and its health plan, we could only obtain managed care enrollment data from those patients who were represented in the empaneled or the proxy-enrollment cohorts.
2.3. Statistical analysis
Demographic characteristics were compared using Chi Square and approximate Z score testing. Health care utilization rates were compared using approximate Z score testing.
3. Results
We first compared 3 different methods of identifying a population cohort in each of the health care systems: empanelment, proxy-enrollment, and enrollment in managed care. Empaneled patients (those seen at least once in a primary care clinic in the past 18 months) numbered 426, 603 at Kaiser Colorado, and 111,330 at Denver Health (Table 1). DH did have a much higher percentage of children and adolescents: 48.6% compared to 22.9% at KPCO. The proxy-enrollment populations were smaller in size compared to the empaneled populations, with the pediatric populations showing the biggest decline using this cohort definition: 7% less children at KPCO and 10% less at DH (Table 1). DH has a smaller enrolled managed care population than KPCO. In addition, not all of DH’s managed care patients receive healthcare at Denver Health. Therefore we only examined DH’s managed care populations who met criteria for either enrolled and empaneled (37,744 patients) or enrolled and proxy-enrolled (37,237). Because of the low number of enrolled managed care patients at DH, and because of the smaller number of children using the proxy-enrollment method, we chose to use empanelment as the best way to define the comparative cohorts in the two health care systems.
Table 1.
Composition of different cohorts, September 1, 2013.
| Cohort | Health care system |
|||||
|---|---|---|---|---|---|---|
| Kaiser Permanente Colorado: N (%) |
Denver Health: N (%) |
|||||
| ≤18 years | >18 years | Total | ≤18 years | >18 years | Total | |
| Empaneleda | 97,793 (22.9) | 328,810 (77.1) | 426,603 | 54,157 (48.6) | 57,173 (51.4) | 111,330 |
| Proxy-enrollmentb | 91,107 (22.7) | 309,865 (77.3) | 400,972 | 48,594 (45.3) | 58,664 (54.7) | 107,258 |
| Enrolledc | 97,731 (21.2) | 362,709 (78.8) | 460,440 | |||
| Enrolled and empaneled | 83,651 (23.8) | 268,358 (76.2) | 352,009 | 24,933 (66.1) | 12,811 (33.9) | 37,744 |
| Enrolled and proxy-enrollment | 74,745 (23.7) | 240,579 (76.3) | 315,324 | 23,650 (63.5) | 13,587 (36.5) | 37,237 |
Empaneled = seen for at least one primary care visit in past 18 months From 9/1/2013.
Proxy-Enrollment = two ambulatory care visits (excluding ED and urgent care) in 3 years prior to 9/1/2013, separated by at least 90 days, with at least one visit being a primary care visit.
Enrolled = enrolled in managed care plan as of 9/1/2013.
We next compared vaccination rates in the two systems. Both KPCO and DH had high levels of pediatric and adolescent vaccination rates, although DH had higher levels of human papillomavirus vaccination (Table 2). Both systems had similar rates of Tdap administration to pregnant women (54.6% and 54.8%), but KPCO had higher rates of administration during 27–36 weeks of pregnancy, which is the preferred time for vaccination [22]. DH had higher rates of vaccination for pneumococcal vaccine in patients 65 years of age and older. Vaccination rates were quite similar across the different patient cohorts, although KPCO’s enrolled cohort had somewhat lower pediatric and adolescent vaccination rates than the empaneled and proxy-enrollment cohorts (data not shown), likely reflecting enrolled patients who received health care outside of KPCO.
Table 2.
Vaccination rates for pediatric, adolescent, pregnant and senior populations in two health care systems.
| Kaiser Permanente Colorado N (%) | Denver Health N (%) | ||
|---|---|---|---|
| Pediatrics | Vaccine(s) | n = 5491 | n = 3295 |
| Combo 3a | 4569 (83.2) | 2862 (86.9) | |
| Hepatitis A | 4560 (83.0) | 2817 (85.5) | |
| Adolescents | n = 27,986 | n = 13,557 | |
| Tdap | 24,682 (88.2) | 12,568 (92.7) | |
| MCV4 | 24,584 (87.8) | 12,334 (91.0) | |
| Tdap & MCV4 | 23,915 (85.5) | 12,279 (90.6) | |
| HPV 1 dose | 17,875 (64.0) | 11,774 (86.9) | |
| HPV 3 doses | 10,117 (36.2) | 8069 (59.5) | |
| Pregnant Women | Tdap: Timing in relation to pregnancy | n = 5677 | n = 2197 |
| 27–36 weeks | 2436 (42.9) | 331 (15.1) | |
| Any other time During pregnancy | 484 (8.5) | 238 (10.8) | |
| At delivery | 183 (3.2) | 635 (28.9) | |
| Seniors | Pneumococcal | n = 70,826 | n = 6946 |
| Vaccine | 31,305 (44.2) | 5899 (84.9) |
Pediatric vaccines reflect rates for children aged 3 years as of the September 1st, 2013, reference date.
Adolescent vaccines reflect rates for patients aged 13–17 years as of the reference date.
Pregnancies were identified for women who became pregnant in the 18 months prior to the reference date. The timing of Tdap administration was determined based on last menstrual period.
Pneumococcal vaccine status was determined for every patient age 65 years or older as of the reference date.
DTaP = Diphtheria/tetanus?(acellular) pertussis; IPV = inactivated poliovirus; MMR = measles/mumps/rubella; Hib = Haemophilus influenza type B; Hep B = hepatitis B; VZV = varicella zoster virus; Tdap = Tetanus/diphtheria/acellular pertussis; MCV4 = meningococcal conjugate vaccine (quadrivalent); HPV = human papillomavirus; pneumococcal vaccine = 23-valent polysaccharide pneumococcal vaccine.
All vaccines reflect empaneled cohorts in respective health care systems.
Combo 3 series includes 4 DTP/DTaP, 3 IPV, 1 MMR, 3 Hib, 3HepB, 1VZV, and 4 Pneumococcal Conjugate vaccines.
The socio-demographic characteristics of patients in the two systems were quite different (Table 3). At DH, the majority of the empaneled patients were of Hispanic ethnicity (60.6%), with 34.8% of the cohort self-identified as preferring to speak Spanish. Only 1.5% of the population had income that was above 200% of the federal poverty level (FPL) ($22,980 for an individual or $47,100 for a family of four in 2013). At KPCO, 16.3% of empaneled patients were Hispanic, 1.5% preferred to speak Spanish, and 62.2% were above 200% of the federal poverty level.
Table 3.
Empaneled cohort demographic characteristics in two health care systems.a
| Characteristic | Kaiser Permanente Colorado |
Denver Healthb |
|||
|---|---|---|---|---|---|
| N (%) Total (N = 426,603) | N (%) <18 years | ≥18 years | Total (N = 111,330a) | ||
| Age | |||||
| <18 years | 97,793 (22.9) | 55,435 (49.8) | |||
| ≥18 years | 328,810 (77.1%) | 55,892 (50.2) | |||
| Sex | |||||
| Female | 234,205 (54.9) | 27,398 (49.4) | 36,157 (64.7) | 63,555 (57.1) | |
| Male | 192,398 (45.1) | 28,037 (50.6) | 19,735 (35.3) | 47,772 (42.9) | |
| Racec | |||||
| Hispanic | 69,536(16.3) | 39,191 (70.7) | 28,284 (50.6) | 67,475 (60.6) | |
| Black | 20,903 (4.9) | 7594 (13.7) | 9946 (17.8) | 17,540 (15.7) | |
| White | 312,273 (73.2) | 6043 (10.9) | 14,912 (26.7) | 20,955 (18.8) | |
| Other | 23,889 (5.6) | 2093 (3.8) | 2568 (4.6) | 4661 (4.2) | |
| Unknown | 4266 (0.01) | 514 (0.9) | 182 (0.3) | 696 (0.6) | |
| Preferred languagec | |||||
| English | 310,994 (72.9) | 31,459 (56.8) | 37,239 (66.6) | 68,698 (61.7) | |
| Spanish | 6399 (1.5) | 21,267 (38.4) | 13,505 (24.2) | 34,772 (31.2) | |
| Other | 4266 (1.0) | 2624 (4.7) | 3983 (7.1) | 6607 (5.9) | |
| Unknown | 104,944 (24.6) | 85 (0.15) | 1165 (2.1) | 1250 (1.1) | |
| %Federal Poverty Levelc | |||||
| 0–100 | 74,228 (17.4) | 40,706 (73.4) | 37,999 (67.9) | 78,705 (70.7) | |
| 101–150 | 39,674 (9.3) | 2265 (4.1) | 5066 (9.1) | 7331 (6.6) | |
| 151–200 | 47,353 (11.1) | 835 (1.5) | 2485 (4.4) | 3320 (2.7) | |
| >200 | 265,347 (62.2) | 814 (1.5) | 2218 (3.9) | 3032 (2.7) | |
| Unknown | 0 (0) | 10,815 (19.5) | 8124 (14.5) | 18,939 (17.0) | |
All comparisons between DH and KPCO totals are significant (p < 0.05) using Chi Square and approximate Z score testing.
3 DH members have some missing data.
KPCO Race, Preferred Language, and Percent Federal Poverty Level categories are estimates from the GEMs (Geographically Enriched Member Socio-demographics) Member and Population Demographics Report.
We next examined health care utilization patterns in the two systems. Adult patients at DH had much higher rates of ED and urgent care utilization than patients from KPCO (Table 4). Primary care utilization rates at DH were 19.5% greater than KPCO, and inpatient utilization was 54.5% higher; specialty care utilization was 6.5% lower than at KPCO. Pediatric utilization rates were more similar across the two systems, but rates of ED and urgent care use were still twice as great at DH (Table 4). Pediatric primary care rates were slightly lower at DH (2.01 visits per year per patient compared to 2.18 at KPCO), while pediatric specialty visit rates were 86.5% greater at KPCO (all differences significant at p < 0.05). Pediatric inpatient utilization was similar across DH and KPCO.
Table 4.
Health care utilization rates of the empaneled cohorts by type of care.
| Kaiser Permanente Colorado (N = 426,603) |
Denver Health (N = 111,330) |
|||||||
|---|---|---|---|---|---|---|---|---|
| >18 Years Type of care | N = 328,810 # of Visits | # of Pts. | Proportion of Pts usingservices | Utilization ratea | >18 Years # of visits | N = 57,173 # of Pts. | Proportion of Pts using services | Utilization ratea |
| Primary care | 1,009,979 | 328,810 | 100.00% | 2.05 | 209,755 | 57,173 | 100.00% | 2.45 |
| Emergent care | 2936 | 2471 | 0.75% | 0.01 | 20,248 | 11,951 | 20.90% | 0.24 |
| Urgent care | 42,119 | 33,728 | 10.26% | 0.09 | 27,087 | 15,640 | 27.36% | 0.32 |
| Medical specialty | 1,135,090 | 222,133 | 67.56% | 2.30 | 184,681 | 33,424 | 58.46% | 2.15 |
| Inpatient | 54,069 | 31,306 | 9.52% | 0.11 | 14,895 | 9017 | 15.77% | 0.17 |
| ≤18 Years Type of care | N = 97,793 # of Visits | # of Pts. | Proportion of Pts using services | Utilization rate | 18 Years # of Visits | N = 54,157 # of Pts. | Proportion of Pts using services | Utilization rate |
| Primary care | 319,238 | 97,793 | 100.00% | 2.18 | 163,243 | 54,157 | 100.00% | 2.01 |
| Emergent care | 3726 | 2892 | 2.96% | 0.03 | 4545 | 3807 | 7.03% | 0.06 |
| Urgent care | 17,843 | 13,557 | 13.86% | 0.12 | 19,894 | 12,246 | 22.61% | 0.24 |
| Medical specialty | 100,915 | 29,885 | 30.56% | 0.69 | 30,137 | 9755 | 18.01% | 0.37 |
| Inpatient | 8622 | 7477 | 7.65% | 0.06 | 5839 | 5344 | 9.87% | 0.07 |
Utilization rates represent average use per patient per year for that type of clinical service. Primary care utilization is 100% in both health care systems by design, as “empaneled” cohorts have at least one primary care visit in the past 18 months.
All comparisons between DH and KPCO utilization rates are significant (p < 0.05) using approximate Z score testing.
4. Discussion
Given the encouragement from the Institute of Medicine to expand populations within the VSD to include populations of lower income and higher percentages of racial and ethnic minorities, safety net institutions such as DH could be natural partners with existing VSD sites. In preparation for incorporating data from a safety net health system into the VSD, we compared two health care systems in Denver, Colorado: DH and KPCO. We found that using a cohort of “empaneled” patients was the most meaningful way to compare data between DH and KPCO. In addition, the concept of empanelment [20] is central to defining patient populations in safety net health care systems across the U.S., and therefore key in comparing vaccination and health care utilization rates in these organizations. In general, both systems had high vaccination rates (in fact, DH and KPCO routinely have the highest reported Health-care Effectiveness Data and Information Set (HEDIS) measures for childhood vaccination in the state of Colorado) (DH internal data), but very different socio-demographic characteristics in their patients. Importantly, baseline health care utilization patterns differ at DH and KPCO.
It is important to note the challenges in defining an “enrollment” cohort in health care delivery systems that are not traditional MCOs. At Kaiser and within the VSD in general, enrollment in the health plan has long been used to define patient cohorts. However, in an integrated safety net institution such as DH, only a portion of patients are enrolled in a managed care health plan. For this reason, even though DH manages its own health plan, there are patient privacy regulations in place to prevent the sharing of data between the health plan and the health care delivery system except for patients who are active patients at DH. Therefore a health system such as DH must rely on “empanelment” rather than “enrollment” to define its patient cohorts. For this project, we therefore also defined an empaneled cohort at KPCO in order to compare the two systems, and would suggest that empanelment provides a rational approach to incorporate data from safety net systems into the VSD.
It does appear that, despite differences in socio-demographic characteristics, immunization rates in general are quite similar across DH and KPCO. However, health care utilization rates are quite different, especially given the high rates of ED and urgent care utilization at DH. Because health care utilization is often used as an outcome metric in vaccine safety studies, it is critical to understand these baseline patterns if data from safety net institutions are to be included in making the VSD data more generalizable.
The higher adult inpatient utilization rates may well reflect the increased morbidity and chronic disease complexity that comes with poverty, as well as the higher levels of substance abuse and behavioral health diagnoses in a safety net population. The much higher ED and urgent care utilization patterns at DH likely reflect the lack of financial barriers to these levels of service. Patients with Medicaid, for example, have no co-pay for ED or urgent care; at Kaiser in contrast co-pays for urgent care can run $50–$100, and for emergency care $150–$300.
The implications of this work are as follows. First, it is possible to define a cohort of patients in a safety net health care system which has a majority of non-capitated patients and that can be used to compare vaccination rates and health care utilization rates with other health care systems. Second, vaccination rates in the safety net system were at least as high as at KPCO. Third, health care utilization rates are different in key areas used in vaccine safety studies, especially adult ED and hospitalization rates. These differences must be accounted for when using data from a safety net system in larger vaccine safety studies. Of note, a recent study compares rates for specific vaccine adverse events between DH and KPCO, and finds similar rates for more serious events such as febrile seizures after measles-containing vaccines in children, but different rates for less serious events such as local reactions after pneumococcal vaccination in adults [23].
In addition to defining comparable patient cohorts and understanding differences in vaccination rates, socio-demographic characteristics, and health care utilization, it is important to consider how to make DH data accessible to investigators in the VSD. Because of the differences noted above, we have elected not to merge DH data with data from KPCO. Rather, our plan in the next phase of this project to create vaccine tables for a Virtual Data Warehouse (VDW). The VDW would permit federated queries for data via the pre-existing VSD server “Hub”. In this way DH data will be accessible to the VSD, but separately identified from KPCO data.
There are a number of limitations to this study. First, the method of empanelment may not identify all “active” patients, and results could vary if different time frames (e.g., 12 or 24 months) were used instead of 18 months. However, it does provide a rational method to compare populations across health care systems, and is used by safety net systems across the U.S. Second, geographic variables at KPCO were based on geocodes and not individual-level information. Third, health care utilization does vary between safety net institutions and integrated health care systems such as KPCO, and this variation creates challenges for establishing outcomes for vaccine safety studies. Understanding this variation was one of the goals of this project. Strengths of this study include defining differences in socio-demographic characteristics and health care utilization, and defining a method to compare populations, at a safety net system (DH) and an integrated health care system (KPCO).
In summary, we have identified a method (empanelment) to compare patient cohorts at DH, a safety net health care delivery system, and KPCO, a long-time participant in the VSD. DH provides a high level of patient diversity, and has good capture of immunization data. It is important to understand differences in baseline health care utilization rates across the two systems prior to embarking on joint vaccine safety studies.
Acknowledgments
Funding source
The work was funded by contract 200-2012-53582 from the Centers for Disease Control and Prevention Vaccine Safety Datalink “Increasing the Diversity with Respect to Underserved Populations” task order.
Findings and conclusions of this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.
Footnotes
Financial disclosure statement
No financial disclosures were reported by the authors of this paper.
Conflict of interest statement
No conflicts of interest are reported by the authors of this paper.
References
- [1].Fahey KR. The pioneering role of the Vaccine Safety Datalink Project (VSD) to advance collaborative research and distributed data networks. eGEMs (Generating Evidence & Methods to improve patient outcomes), vol. 3(1); 2015, Article 21. Available at: <http://repository.edm-forum.org/egems/vol3/iss1/21> [accessed May 8, 2016]. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [2].McNeil MM, Gee J, Weintraub ES, Belongia EA, Lee GM, Glanz JM, et al. The Vaccine Safety Datalink: successes and challenges monitoring vaccine safety. Vaccine 2014;32(42):5390–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [3].Baggs J, Gee J, Lewis E, Fowler G, Benson P, Lieu T, et al. The Vaccine Safety Datalink: a model for monitoring immunization safety real-time vaccine safety surveillance for the early detection of adverse events. Pediatrics 2011;127: S45–53. [DOI] [PubMed] [Google Scholar]
- [4].Lieu TA, Kulldorff M, Davis RL, Lewis EM, Weintraub E, Yih K, et al. Med Care 2007;45:S89–95. [DOI] [PubMed] [Google Scholar]
- [5].Chen RT, Glasser JW, Rhodes PH, Davis RL, Barlow WE, Thompson RS, et al. Vaccine Safety Datalink project: a new tool for improving vaccine safety monitoring in the United States. The Vaccine Safety Datalink Team. Pediatrics 1997;99(6):765–73. [DOI] [PubMed] [Google Scholar]
- [6].IOM (Institute of Medicine, now the Academy of Medicine). The childhood immunization schedule and safety: Stakeholder concerns, scientific evidence, and future studies The National Academies Press: Washington, DC; 2013. <http://www.nationalacademies.org/hmd/Reports/2013/The-ChildhoodImmunization-Schedule-and-Safety.aspx> [accessed December 23, 2016]. [PubMed] [Google Scholar]
- [7].Lewin ME, Baxter RJ. America’s health care safety net: revisiting the 2000 IOM report. Health Aff 2007;26(5):1490–4. [DOI] [PubMed] [Google Scholar]
- [8].Enthoven AC. Integrated delivery systems: the cure for fragmentation. Am J Manage Care 2009;15:S284–90. [PubMed] [Google Scholar]
- [9].McCarthy D, Mueller K, Wrenn J. Kaiser Permanente: bridging the quality divide with integrated practice, group accountability, and health information technology. The Commonwealth Fund Case Study: Organized Health Care Delivery System, June 2009. <http://www.commonwealthfund.org/~/media/Files/Publications/Case%20Study/2009/Jun/1278_McCarthy_Kaiser_case_study_624_update.pdf> [accessed December 23, 2016].
- [10].Gabow P, Eisert S, Wright R. Denver Health: a model for the integration of a public hospital and community health centers. Ann Intern Med 2003;138 (2):143–9. [DOI] [PubMed] [Google Scholar]
- [11].Nuzum R, McCarthy D, Gauthier A, Beck C. Denver Health: a high-performance public health care system: the commonwealth fund; July 2007. <www.commonwealthfund.org/publications/fund-reports/2007/jul/denverhealth–a-high-performance-public-health-care-system> [accessed December 23, 2016].
- [12].Gabow PA, Mehler PS. A broad and structured approach to improving patient safety and quality: lessons from Denver Health. Health Aff 2011;30:612–8. [DOI] [PubMed] [Google Scholar]
- [13].Gabow PA, The PL. The lean prescription: powerful medicine for our ailing healthcare system Boca Raton (FL): CRC Press; 2015. [Google Scholar]
- [14].Davidson A, Melinkovich P, Beaty B, Chandramouli V, Hambidge SJ, Phibbs SL, et al. Immunization registry accuracy: improvement with progressive clinical application. Am J Prev Med 2003;24:276–80. [DOI] [PubMed] [Google Scholar]
- [15].Melinkovich P, Hammer A, Staudenmaier A. Improving pediatric immunization rates in a safety-net delivery system. In: Schilling L, editor. Implementing and sustaining improvement in health care Oakbrook Terrace (Illinois): Joint Commission Resources Inc; 2009. [chapter 9]. [Google Scholar]
- [16].immunization rates. In: Schilling L, editor. Implementing and sustaining improvement in health care Oakbrook Terrace (Illinois): Joint Commission Resources Inc; 2009. [chapter 9]. [Google Scholar]
- [17].Farmar AM, Love-Osborne K, Chichester K, Breslin K, Bronkan K, Hambidge SJ. Achieving high adolescent HPV vaccination coverage. Pediatrics 2016;138: e20152653. [DOI] [PubMed] [Google Scholar]
- [18].Miliard M Most Wired 2014 hospitals big on data Healthcare IT News, July 9, 2014. <http://www.healthcareitnews.com/news/most-wired-2014-named> [accessed May 8, 2016]. [Google Scholar]
- [19].Health Information Management Systems Society. Two organizations honored with davies public health award of excellence: Denver Public Health, Denver Public Health Information Service (DPH-IS); 2009. <http://www.himss.org/News/NewsDetail.aspx?ItemNumber=5556> [accessed May 8, 2016].
- [20].Safety Net Medical Home Initiative. Empanelment: Establishing patientprovider relationships; 2013. <http://www.safetynetmedicalhome.org/change-concepts/empanelment> [accessed May 8, 2016].
- [21].Haller I, Hitz P, Bredfeldt C, Butani A, Pardee R, Johnson B. Evaluation of the utilization-based population denominators in the HMORN context. Clin Med Res 2013;11(3):176. [Google Scholar]
- [22].Centers for Disease Control and Prevention (CDC). Updated recommendations for use of tetanus toxoid, reduced diphtheria toxoid, and acellular pertussis vaccine (Tdap) in pregnant women—Advisory Committee on Immunization Practices (ACIP), 2012. MMWR Morb Mortal Wkly Rep 2013;62(7):131–5. [PMC free article] [PubMed] [Google Scholar]
- [23].Narwaney KJ, Breslin K, Ross CA, Shoup JAA, Wain KF, Weintraub ES, et al. Vaccine adverse events in a safety net healthcare system and a managed care organization. Vaccine 2017;35(9):1335–40. [DOI] [PMC free article] [PubMed] [Google Scholar]