Comparative Health Systems Research among Kaiser Permanente and Other Integrated Delivery Systems: A Systematic Literature Review

Abstract

Context:

Because of rising health care costs, wide variations in quality, and increased patient complexity, the US health care system is undergoing rapid changes that include payment reform and movement toward integrated delivery systems. Well-established integrated delivery systems, such as Kaiser Permanente (KP), should work to identify the specific system-level factors that result in superior patient outcomes in response to policymakers’ concerns. Comparative health systems research can provide insights into which particular aspects of the integrated delivery system result in improved care delivery.

Objective:

To provide a baseline understanding of comparative health systems research related to integrated delivery systems and KP.

Design:

Systematic literature review.

Methods:

We conducted a literature search on PubMed and the KP Publications Library. Studies that compared KP as a system or organization with other health care systems or across KP facilities internally were included. The literature search identified 1605 articles, of which 65 met the study inclusion criteria and were examined by 3 reviewers.

Results:

Most comparative health systems studies focused on intra-KP comparisons (n = 42). Fewer studies compared KP with other US (n = 15) or international (n = 12) health care systems. Several themes emerged from the literature as possible factors that may contribute to improved care delivery in integrated delivery systems.

Conclusions:

Of all studies published by or about KP, only a small proportion of articles (4%) was identified as being comparative health systems research. Additional empirical studies that compare the specific factors of the integrated delivery system model with other systems of care are needed to better understand the “system-level” factors that result in improved and/or diminished care delivery.

Introduction

Rising health care costs,¹ wide variation in quality,² and increased patient complexity led to passage of the Affordable Care Act,³ which has resulted in the US health care system undergoing rapid changes. These changes include payment reform (ie, value-based purchasing, bundled payments)⁴ and movement toward integrated delivery systems, such as accountable care organizations and patient-centered medical homes.^5,6 Because the current US health care system performs poorly relative to those of other countries, alternative models of care delivery have been proposed.^5,7

Some of the inefficiencies of the current US health care delivery system stem from the growth of new and expensive medical technologies and the fee-for-service payment of physicians.⁸ Although physicians aim to provide patient care on the basis of scientific evidence, financial considerations may influence their treatment decisions. Replacement of fee-for-service with capitated payment has been proposed as one way to improve the efficiency of health care delivery. However, changing the physician payment structure by itself may not be enough to achieve the desired outcome. Previous research has shown that although prospective payment has slowed the growth of health care spending at the medical group level, similar results have not been achieved among individual or small practices.⁸ Physician practices therefore may need to be reorganized and integrated across multispecialty groups and hospitals to be responsive to new payment methods. Thus, health care reform efforts also may need to focus on redesigning integrated systems of care.⁷

Integrated delivery systems are a model of health care involving an organized, coordinated, and collaborative network that brings together various physicians to deliver coordinated care and a continuum of services to a given patient population.⁷ Integrated delivery systems are clinically and fiscally accountable for the health status and outcomes for the population served, and they have systems to manage and to improve clinical outcomes. Key attributes of successful integrated delivery systems have been suggested.⁷ These attributes include: 1) shared values and goals, 2) patient-centeredness and a focus on population health, 3) coordination of care across a continuum of health care services and settings, 4) physician financial incentives that are aligned with patients’ goals, 5) use of evidence-based practices, 6) electronic health records (EHR) that are accessible and shared by all physicians to track patients across a continuum, 7) the right mix of primary care and specialist physicians and appropriate medical equipment to serve the given population, and 8) continuous innovation and learning to improve the value of care.

A previous report highlighted that tightly integrated delivery systems with their own health plan may serve as a potential model of high-performing health care systems because the insurance function of these systems allows for greater flexibility and aligned incentives, and helps deliver high-value care.⁹ More specifically, Kaiser Permanente (KP), the largest nonprofit integrated delivery system in the US, may serve as a model of a high-performing health care system because of several unique aspects. KP serves vari ous geographic populations in the US, including California, Colorado, District of Columbia, Georgia, Hawaii, Maryland, Oregon, Virginia, and Washington, with more than nine million active members in 2013. Patients in the KP system receive comprehensive, multidisciplinary health care, including all medical and surgical specialties as well as pharmacy, radiology, and laboratory services. In many geographic Regions, KP owns its hospitals; in the other Regions, KP has contracts with preferred hospitals. The population in KP is representa tive of the states they serve; data indicate that members overall are similar to the general population regarding age, sex, and race/ethnicity, with only slight underrepresentation of those in lower and higher income and educa tion categories.^10–12

Despite these key features, important questions remain about the “best practices” of integrated delivery systems that achieve superior outcomes. Policymakers are increasingly demanding high-quality research regarding which specific aspects of the integrated delivery model result in superior patient outcomes. For example, there are essential questions regarding how integrated systems are able to coordinate care among different specialties and how the use of information technology and clinical decision support systems are able to support transformational care delivery.⁷ Consequently, well-established integrated delivery systems, such as KP, should work to identify the specific system-level factors or confluence of factors that improves such services, as well as access, quality, and other such outcomes in an integrated delivery system. The answer to these fundamental questions may serve as a platform to inform and to guide emerging models of care delivery such as accountable care organizations and patient-centered medical homes.

Through a better understanding for which key systems and processes in integrated delivery systems work and the mechanisms by which they function, this knowledge may be translated and disseminated to the larger US health care delivery system. Comparative health systems research involves a comparison of the different approaches used by systems to organize and deliver health care services for a given population. Thus, comparative health systems is one area of research that may be able to provide valuable insights to policymakers and practitioners regarding which particular aspects of the integrated delivery system model result in improved care delivery and patient outcomes.

In response to policymakers’ growing interest in this area, KP has embarked on a research agenda for comparative health systems. Therefore, the objective of this literature review was to examine the existing published studies on comparative health systems that relate to integrated delivery systems and KP, to obtain a baseline understanding of the state of comparative health systems research that can provide foundational knowledge. We also sought to identify, to quantify, and to classify the literature in this area.

Methods

To gain an understanding of the universe of research studies published on comparative health systems, we conducted a literature search on PubMed and the internal KP Publications Library. The KP Publications Library is a unique, full text searchable database of publications authored or coauthored by KP staff, including investigators, clinicians, and administrators, regardless of journal. The database contains 10,000 records describing journal articles, book chapters, books, letters, and commentaries. The library does not include posters, presentations, or published abstracts. In both literature searches, we sought to include existing studies that compared KP as a system or organization with other health care systems or organizations, or across KP facilities internally, in any topic area. We defined comparative health systems research as any study that compared KP as a system with another health system; any study that compared KP’s performance with a state or national benchmark; and any study that compared KP’s innovations in care delivery with old or previous models of care. We also included intra-KP studies that compared a system of care in or between another KP Region or in or between KP facilities. Any such types of these comparisons between different systems or models of care were defined as the systems of comparison. The search was inclusive of all subject areas, ranging from quality to information technology. We looked for explicit comparisons between KP and similar health care systems. Studies that included aggregated data from KP and other health care systems were excluded because there were no direct system-level comparisons that would allow us to disentangle the different health care systems.

In PubMed, the MeSH (Medical Subject Headings) search terms included Kaiser Permanente and comparative health services (n = 258), comparative health systems (n = 34), comparative integrated systems (n = 14), health services benchmarking (n = 10), health system benchmarking (n = 7), and quality benchmarking (n = 7; Table 1). From the 330 publications, we identified 16 studies involving direct system comparisons, which we categorized into KP to Other US, KP to Other International, and KP to KP (interregional or intraregional KP comparisons). The PubMed search yielded 16 relevant articles that met the criteria of comparative health systems research involving KP (Figure 1).

Table 1.

Search terms used in review of Kaiser Permanente Publications Library

Set	Search term
1	Compar*[all fields] AND Formulary[all fields]
2	“Kaiser Permanente”[title/abstract] AND Compar*[title/abstract/subject/publication type] {447}
3	Compar*[all fields] AND “Formulary Therapeutics”[all fields] {2}
4	Compar*[all fields] AND Pharmacy[all fields] AND technology[all fields] {217}
5	“Comparative quality”[all fields] {14}
6	Compar*[all fields] AND quality[subject] AND assurance[subject] {63}
7	Compar*[all fields] AND Quality[subject] AND Improvement[subject] {1}
8	Compar*[all fields] AND quality[subject] AND health[subject] AND care[subject] {252}
9	Compar*[all fields] AND “Information Technology”[all fields] {171}
10	Compar*[all fields] AND electronic[subject] AND health[subject] AND record[subject] {33}
11	Compar*[all fields] AND “Clinical integration”[all fields]{7}
12	“Comparative Performance”[all fields] {28}
13	Inter-Regional collection AND “Kaiser Permanente”[title/abstract] AND compar*[title/abstract/subject/publication type] {54}
14	Compar*[all fields] AND “Care Models”[all fields] {177}
15	Compar*[all fields] AND “Care Team”[all fields] {193}
16	Compar*[all fields] AND “Care Team Approach”[all fields] {1}
17	Compar*[all fields] AND “Multidisciplinary Care Team”[all fields] {4}
18	“Comparative Effectiveness Research”[subject] {13}
19	“Comparative Treatment Effectiveness”[all fields] {5}
20	“Comparative Mortality” [all fields] {20}

Symbols: Asterisk = wildcard search; brackets = field searched; [all fields] = searching the full text of the publication; braces (at end of rows) = Publications Library total number of articles; and quotation marks = phrase search.

Figure 1.

Identification of articles for inclusion in systematic review.

KP = Kaiser Permanente.

On the basis of PubMed results and additional refinement, we expanded our search to 20 keywords and topics to discover both external and internal comparisons from our KP Publications Library. The KP Publications Library search was used to find additional articles that may have been missed through the PubMed search because of differences in tagged words or keywords, articles that are not indexed, or delays in indexing.

We limited the final results from the KP Publications Library to the following criteria: 1) publication type: journal article only (no editorials, letters, and commentaries); 2) abstract: no publications without an abstract unless published in the last two months; and 3) date: no publications before 1995 because of the likelihood of lesser relevance.

The comprehensive search using the same 20 search terms in the KP Publications Library generated 1271 unique citations (Table 1). After limiting the search set to the previously stated criteria, 1132 citations required closer review. These studies were manually reviewed, and articles that were previously identified from PubMed were removed. We examined the results, first considering the study title and abstract and then reviewing the full text article, if necessary, to make a determination of appropriateness. Our examination of 1117 KP abstracts and full publications yielded 45 more publications relevant to the topic of comparative health systems research. We also identified 4 publications that were not found through our literature search (because they did not have a KP author listed or had no keyword hits from our search sets) and were provided to us by KP authors or identified through press releases because they were found to be relevant to the overall topic. We reviewed the final set of articles for agreement on inclusion.

Results

In total, the literature search from the 2 comprehensive databases, in addition to publications identified outside our systematic review, resulted in 65 publications for inclusion in this analysis. Table 2 summarizes the included studies of comparative health systems. For each of the studies reviewed, we evaluated the system of comparison, topical area, and condition type. The topical area was the areas of comparison, the condition type was the disease or diseases of study, and the outcomes were the system outputs. The following topical areas were examined: resource use (ie, cost of care, utilization, length of stay); quality (ie, quality-improvement programs, quality performance, processes of care, patient outcomes of care, patient satisfaction); health information technology (ie, management of health information across computer systems); EHR (ie, electronic health information about patients); clinical decision support (ie, system that assists physicians with decision making related to patients); computerized physician order entry (ie, electronic entry of physician treatment orders); telemedicine (ie, telecommunications systems that provide health care across distances); health system performance (ie, health system delivery of care); self-care (ie, patient self-management of condition); disease management (ie, interventions to help patients cope with a condition); pharmacy consultation (ie, pharmacist counseling of patients regarding their medications); care delivery/care coordination (ie, provision and coordination of health care services); registries (ie, collection of data on patients with a specific condition); clinical integration (ie, integration of clinical information and health care services from different entities); patient safety (ie, prevention of medical errors); medication adherence (ie, patients taking medications as prescribed); and team performance (ie, team functioning).

Table 2.

Studies by systems of comparison

Author, Year	Systems of comparison	Topical area	Condition	Outcomes	Conclusions
Thorner,55 1978	KP to US military health care system	Resource use	NA	Ambulatory care and hospital utilization rates among civilian beneficiaries of the military health care system, members of KP, and noninstitutionalized population	Civilian beneficiaries of the military health care system were generally not found to be taking full advantage of their entitlement to health care.
Rubenstein,57 2002	KP to VA	Quality	MHSA	Team success in developing depression care improvement programs	A high degree of local support and expertise was needed from primary care and mental health clinicians for the local team’s approach to quality improvement. However, the central team approach was more likely to succeed than the local team’s approach when local practice conditions were not optimal.
Kerr,60 2004	KP to VA and other HMO	Quality	Diabetes mellitus	Quality of diabetes mellitus care	Diabetes processes of care and 2 of 3 intermediate outcomes were better for patients in the VA system than for patients in commercial managed care, but both VA and commercial managed care had room for improvement.
Magid,66 2011	KP to VA and US	Quality	CVD	BP control among patients with uncontrolled BP	A multimodal intervention of patient education, home BP monitoring, BP measurement reporting to an interactive voice response system, and clinical pharmacist follow-up achieved greater reductions in BP compared with usual care.
Fishman,59 2004	KP to US	Health system performance; resource use; quality	NA	Cost, quality, and effectiveness of US health care system	There needs to be a revision in how health services research approaches analyses of cost, production, and output, and one must consider alternative notions of final goods. Also, there needs to be a review of the availability and quality of data necessary to conduct this research.
Kim,61 2004	KP to US	Health system performance	Diabetes mellitus	Diabetes mellitus processes of care	Group/network models provided better diabetes processes of care than did independent practice association (IPA) models. This may be caused by the clinical infrastructure available in group models that is not available in IPA models.
Stiefel,64 2008	KP to US	Resource use	NA	Hospital and hospice use for end-of-life care	Geographic variation in hospital use in KP appears to be correlated with variation in the surrounding communities; this suggests that KP resource use may be influenced, at least in part, by broader community practices.
Horberg65 2011	KP to US	Quality	HIV	Quality performance for patients with HIV	KP’s results compared favorably with those of other organizations.
Wisdom,67 2011	KP to US	Data management	MHSA	Data management capacity at substance abuse treatment programs	An infusion of expertise, training, and funding is needed to improve substance abuse treatment programs’ IT-related systems and data management processes.
Schroeder,69 2012	KP to US	Quality	Diabetes mellitus; CVD	Simultaneous control of diabetes mellitus, hypertension, and hyperlipidemia	Individuals who simultaneously achieve multiple treatment goals may provide insight into self-care strategies for individuals with comorbid health conditions.
Hazelhurst,68 2012	KP to US FQHCs	Quality, HIT	Asthma	Asthma care quality	Automated measures of asthma care quality performed well in the HMO, where practice is more standardized.
DeCoster,56 1997	KP to Manitoba, Canada, and US	Resource use	NA	Ambulatory care visit rates and procedure rates for 3 surgical procedures	For the surgical procedures studied, US rates were higher than those in either KP or Manitoba. US system led to more surgical intervention, and removal of financial barriers led to higher use of primary care services, where more preventive and ameliorative care can occur.
Smith,63 2006	KP to Canada and US	HIT/EHR/CPOE/CDS/patient safety	NA	Potentially contraindicated agents in elderly patients	Alerts in an outpatient EHR aimed at decreasing prescribing of medications in elderly persons may be an effective method of reducing prescribing of contraindicated medications.
Ham,58 2003	KP to UK NHS and US Medicare program	Resource use	NA	Hospital utilization of different health systems	NHS can learn from KP’s integrated approach, the focus on chronic diseases, effective management, the emphasis placed on self-care, the role of intermediate care, and the leadership provided by physicians in developing and supporting this model of care.
Profit, 62 2006	KP to UK and US	Resource use	Premature infants	Gestational age at discharge among moderately premature infants	Infants of 30–34 weeks’ gestation at birth admitted and cared for in hospitals in California have a shorter length of stay than those in the UK. Certain characteristics of the integrated health care approach pursued by the HMO of NICUs in California may foster earlier discharge.
Towill,72 2006	KP to UK NHS	Health system performance	NA	Health care supply chain performance	Health care “best practice” is readily related to conventional supply chain performance metrics and engineering change model attributes. Much of the apparent success of KP was caused by cultural and organizational factors.
Feachem,70 2002	KP to UK NHS	Resource use; health system performance	NA	Cost and health system performance	KP achieved better performance at roughly the same cost than the NHS did because of integration throughout the system, efficient management of hospital use, the benefits of competition, and greater investment in information technology.
Séror,71 2002	KP to UK NHS	HIT	NA	Ideologic differences in health care market infrastructure	Telecommunications technologies and the Internet may contribute significantly to health care system performance in a context of ideologic diversity.
Frølich,73 2008	KP to Danish Health Care System	Health system performance	NA	Utilization, quality, and costs of different health systems	Compared with the Danish Health Care System, KP had a population with more documented disease and higher operating costs, while employing fewer physicians and resources such as hospital beds. Observed quality measures also appeared higher in KP.
Strandberg-Larsen,74 2010	KP to Danish Health Care Service	Clinical integration	NA	Primary care clinicians’ perceptions of clinical integration in two health care systems	More primary care clinicians in KPNC reported clinical integration than did general practitioners in the Danish Health Care System.
Schiøtz,76 2011	KP to Danish Health Care System	Health system performance	Ambulatory-care-sensitive conditions; chronic conditions	Preventable hospitalizations and readmissions for ambulatory-care-sensitive conditions and chronic conditions	There were substantial differences between the Danish Health Care System and KP in the rates of preventable hospitalizations and subsequent readmissions associated with chronic conditions, which suggest much opportunity for improvement in the Danish Health Care System.
Schiøtz,77 2012	KP to Danish Health Care System	Quality	Diabetes mellitus	Self-management behaviors among patients with Type 2 diabetes mellitus	Despite better self-management support in KP compared with the Danish Health Care System, self-management remains an undersupported area of care for people receiving care for diabetes in the two health systems.
Paxton,75 2011	KP to Norway	Registries; quality	Knee arthroplasty	Choice of implants, techniques, and outcomes for knee arthroplasty	Cumulative survival of the patients receiving a total knee prosthesis was 94.8% for the arthroplasties performed in Norway and 96.3% for those performed at KP. Patient characteristics, selection of implants, surgical techniques, and outcomes differed between the cohorts.
Domurat,13 1999	Intra-KP comparison	Disease management; HIT/EHR; quality	Diabetes mellitus	Laboratory and BP screening rates for patients with diabetes mellitus	Computer-supported care management by a dedicated team appeared to reduce the number of hospitalizations and to improve screening rates and glycemic and BP control.
Horberg,14 1999	Intra-KP comparison	Care delivery/coordination; health system performance	HIV	Antiretroviral therapy adherence for HIV-positive patients	Various multidisciplinary care teams were associated with improved adherence, including ones that did not include an HIV specialist but included primary care plus other health professionals.
Brown,15 2000	Intra-KP comparison	Disease management	Diabetes mellitus	Mortality, change in comorbidity, rate of uptake of preventive measures, use of pharmaceuticals, and hospital utilization for patients with diabetes mellitus	A comprehensive management program was associated with substantial improvements in the process and outcomes of care in a large population of HMO members with diabetes.
Johnston,16 2000	Intra-KP comparison	Home health; HIT/telemedicine; quality; resource use	CVD, COPD, cancer, diabetes mellitus, anxiety, wound care	Effectiveness of remote video technology in home health care setting	Remote video technology in the home health care setting was shown to be effective, well received by patients, and capable of maintaining quality of care, and to have the potential for cost savings.
Merenich,17 2000	Intra-KP comparison	Disease management	CVD	Lipid screening and treatment rate, patient satisfaction among patients with CVD	HMOs have tremendous incentive and the unique opportunity and ability to develop systems to better manage large numbers of individuals with CVD.
Nichols,18 2000	Intra-KP comparison	Quality	Diabetes mellitus	Glycemic control among patients with Type 2 diabetes mellitus	Busy clinicians with heavy workloads can improve their management of diabetes by identifying patients whose glycemic control could be improved through a change in medication or simple adjustment in dosage.
Perry,19 2000	Intra-KP comparison	Quality	Cancer	Breast cancer screening rate in a managed care organization	Several innovations demonstrated the ability to integrate improved care management into evolving service delivery at KP, such as use of call center technologies and redesign of primary care delivery.
Lorig,20 2001	Intra-KP comparison	Self-care	Chronic diseases	Health behavior,self-efficacy, health status, and health care utilization among patients with chronic disease	One year after exposure to a chronic disease self-management program, most patients experienced statistically significant improvements in a variety of health outcomes and had fewer ED visits.
Thompson,21 2001	Intra-KP comparison	Quality	NA	Patient satisfaction, self-rated knowledge of prevention, acceptance of Health Plan guidelines, and likelihood of remaining with the Health Plan among recent adult enrollees of KP	Group visit attendees stood out as experiencing the greatest benefits and were especially likely to report avoiding a telephone call or visit to their physician by using a self-care handbook.
Vollmer,22 2001	Intra-KP comparison	HIT; medication adherence	Asthma	Medication adherence to inhaled corticosteroids among asthmatics	An HIT-based adherence intervention showed potential for supporting medication adherence in patients with chronic diseases such as asthma.
Shafer,23 2002	Intra-KP comparison	Quality; team performance	Chlamydia infection	Chlamydial screening rates among adolescent girls	Implementation of a clinical practice intervention in a large HMO significantly increased the C trachomatis screening rates for sexually active adolescent girls during routine checkups.
Yuan,24 2003	Intra-KP comparison	Resource use; quality; pharmacy consultation	NA	Patient survival and hospitalization among KP enrollees	Intensive outpatient consultation with a pharmacist targeting high-risk patients would improve survival and decrease hospitalization rates.
Taylor,25 2003	Intra-KP comparison	Disease management; quality	Diabetes mellitus	Clinical laboratory values, annual physician visits, and psychosocial factors in patients with complicated diabetes mellitus	A nurse care management program can significantly improve some medical outcomes in patients with complicated diabetes without increasing the number of physician visits.
Finley,26 2003	Intra-KP comparison	Care coordination	MHSA	Medication adherence, patient satisfaction, and physician satisfaction among patients with depression	Clinical pharmacists had a favorable effect on multiple aspects of patient care.
Garrido,27 2005	Intra-KP comparison	HIT/EHR; quality; resource use	NA	Utilization and quality of ambulatory care	Readily available, comprehensive, integrated clinical information reduced use of ambulatory care while maintaining quality and allowed physicians to replace some office visits with telephone contacts.
Patel,28 2005	Intra-KP comparison	Interdisciplinary care team; quality	NA	Time from arrival to physician assessment, percentage of patients who left without being seen, and satisfaction among patients in the ED	Implementation of a team assignment system in an ED was associated with reduced time to physician assessment, a reduced percentage of patients who left without being seen, and improved patient satisfaction.
Lorig,29 2005	Intra-KP comparison	Self-care	Chronic diseases	Effectiveness of dissemination of chronic disease self-management program	Six years after the beginning of the dissemination process, the peer-led chronic disease self-management program was integrated into most of the KP Regions and was being offered to several thousand patients each year.
Stubbings,30 2005	Intra-KP comparison	HIT/telepharmacy	NA	Quality of care and cost among patients in a telepharmacy program	Clinical pharmacy call center has used telephonic, electronic, and other means of communication in an effort to reduce costs and improve the quality of care.
Grypma,31 2006	Intra-KP comparison	Quality	MHSA	Depression severity, treatment contacts, use of antidepressants, and costs among patients with depression	An adapted version of the Improving Mood-Promoting Access to Collaborative Treatment (IMPACT) program implemented at a large HMO achieved similar clinical improvements in depression as the clinical trial despite a lower number of intervention contacts.
McFarland,32 2006	Intra-KP comparison	Resource use	MHSA	Initiation and duration of substance abuse treatment after index event	State Medicaid policies may make it difficult for clients to obtain suitable chemical dependency treatment services.
Palen,33 2006	Intra-KP comparison	HIT/CPOE/CDS; patient safety	NA	Physician compliance with guidelines for laboratory monitoring at time of therapy initiation	There was no significant difference between the control and intervention group physicians (CPOE reminders) in the overall rate of compliance with ordering the recommended laboratory monitoring for patients prescribed study medications.
Simon,34 2006	Intra-KP comparison	HIT/CDS; patient safety	NA	Rates of dispensing potentially inappropriate medications in older adults	Age-specific alerts sustained the effectiveness of drug-specific-alerts to reduce potentially inappropriate prescribing in older people and resulted in a considerably decreased burden of the alerts.
Vollmer,35 2006	Intra-KP comparison	HIT; resource use; quality	Asthma	Acute health care utilization and quality of life among asthmatics	This study did not find improved health outcomes between patients who received an automated telephone outreach or usual care.
McConnell,36 2006	Intra-KP comparison	Disease management; quality	CVD	Hypertension control among patients with CVD	A pharmacist-managed, physician-supervised population management approach in patients with coronary artery disease significantly improved BP control.
Hornbrook,37 2007	Intra-KP comparison	HIT	Pregnancy	Detection of pregnancy episodes and maternal morbidities using automated data	A pregnancy episode grouper algorithm takes advantage of databases readily available in IDSs and has important applications for health system management and clinical care.
Humphries,38 2007	Intra-KP comparison	HIT/CDS; patient safety	NA	Rate of codispensing of critically interacting drug combinations	Employment of an intervention system that limits electronic alerts regarding drug interactions to those deemed critical but that also requires pharmacist intervention and collaboration with the prescriber decreased the number of critical drug interactions dispensed.
McGaw,39 2007	Intra-KP comparison	Care coordination; patient safety	Chronic diseases	Reductions in medical errors, follow-up with care plans	Identification of unintended medication discrepancies and potential drug-related problems and increased follow-up during care transitions can improve patient safety and quality of care while saving health care resources.
Merenich,40 2007	Intra-KP comparison	Disease management; quality	CVD	All-cause mortality in patients with CVD	Compared with those not enrolled in the comprehensive cardiac care (CCC) program, patients enrolled in the early CCC program were 89% less likely to die.
Neuwirth,41 2007	Intra-KP comparison	HIT, quality, resource use	NA	Impact of panel management on patients, physicians, and staff	Spread of panel management should be informed by lessons and findings from early adopters and should include continued monitoring of the impact of this rapidly developing approach on quality, patient satisfaction, primary care sustainability, and cost.
Raebel,42 2007	Intra-KP comparison	HIT/CDS; patient safety	NA	Proportion of ambulatory elderly patients prescribed potentially inappropriate medications	Coupling data available from information systems with the knowledge and skills of physicians and pharmacists can improve safety of prescribing for patients aged 65 years and older.
Graetz,43 2009	Intra-KP comparison	Care coordination; HIT/EHR	NA	Clinicians’ perceptions of care coordination	EHR use is associated with aspects of care coordination involving information transfer and communication of treatment goals.
Schmittdiel,44 2009	Intra-KP comparison	Quality	Diabetes mellitus	Effectiveness of diabetes mellitus care management	In a population with improving control of risk factors, patients entering diabetes care management experienced slightly greater improvement.
Sterling,45 2009	Intra-KP comparison	Quality; care delivery	MHSA	Chemical dependency and mental health treatment outcomes among adolescents	A chemical dependency treatment episode resulting in good 1-year chemical dependency outcomes may contribute significantly to both chemical dependency and mental health outcomes 3 years later.
Bowman,46 2010	Intra-KP comparison	HIT/EHR; quality	NA	Patient satisfaction and service quality of a call center technology	DirectConnect system has resulted in statistically significant improvement in key service quality measures.
Delate,47 2010	Intra-KP comparison	Disease management; resource use	CVD	Total health care expenditures after incident acute cardiovascular event	Comprehensive and aggressive implementation of secondary cardiac prevention strategies and close monitoring and follow-up of patients with coronary artery disease provided by the Collaborative Cardiac Care Service were associated with reduced health care expenditures.
Smith,48 2010	Intra-KP comparison	Resource use; disparities; HIT/EHR	NA	Validity of race and ethnicity data from Health Plan administrative records	Quality of racial information obtained from administrative records may benefit from additional supplementation by birth certificate data.
Baer,49 2011	Intra-KP comparison	HIT/EHR	NA	Office visit utilization, quality of care, and patient satisfaction with secure message	Secure messaging has been associated with a decrease in office visits, an increase in measurable quality outcomes (at least in primary care), and excellent patient satisfaction.
Moiel,50 2011	Intra-KP comparison	Quality	Cancer	Colorectal cancer screening rates, stage at diagnosis, percentage of patients with colon cancer who were screen-detected, survival	Early detection of colorectal cancer saves lives when a program tests the most at-risk people. Using a good test (fecal occult blood test/fecal immunochemical test) that is able to reach more people, rather than the “perfect test” that reaches fewer people, transforms an ineffective program into a successful one. A critical element was the transition of the individual testing to population screening.
Naber,51 2011	Intra-KP comparison	Care delivery; quality; resource use	Vestibular dysfunction	Health outcomes and utilization for patients with vestibular dysfunction	Interdisciplinary treatment of vestibular dysfunction improved patient coping, functionality, and satisfaction and decreased overall health care utilization in patients with vestibular dysfunction.
Nelson,52 2011	Intra-KP comparison	Resource use	Palliative care	Hospital readmission among patients referred to inpatient palliative care service	Readmissions to the hospital per patient per 6 months after consultation decreased from 1.15 to 0.7 admissions per patient with the postteam consultation.
Roblin,53 2011	Intra-KP comparison	Care coordination	NA	Short-term future health of Medicare beneficiaries	Medicare beneficiaries empaneled to relatively high functioning primary care teams had significantly better physical and emotional health at 2 years after baseline assessment than those empaneled to relatively low functioning patient care teams.
Ferrara,54 2012	Intra-KP comparison	Quality	Diabetes mellitus	Risk of macrosomia, and postpartum and glucose testing among women with gestational diabetes	Receiving care at the centers with higher referral frequency to telephonic nurse management for gestational diabetes mellitus was associated with decreased risk of a macrosomic infant and increased rate of postpartum glucose testing.

BP = blood pressure; CDS = clinical decision support; COPD = chronic obstructive pulmonary disease; CPOE = computerized physician order entry; CVD = cardiovascular disease; ED = Emergency Department; EHR = electronic health record; FQHCs = federally qualified health centers; HIT = health information technology; HIV = human immunodeficiency virus infection; HMO = health maintenance organization; IDS = integrated delivery system; IT = information technology; KP = Kaiser Permanente; KPNC = Kaiser Permanente Northern California; MHSA = mental health and substance abuse; NA = not applicable; NHS = National Health Service; NICU = neonatal intensive care unit; UK = United Kingdom; VA = Veterans Affairs (Veterans Health Administration).

Of all studies published by or about KP, only 4% of articles were identified as being comparative health systems research. The comparative health systems studies that were reviewed tended to focus mostly on quality of care (n = 30) and health information technology/EHR/clinical decision support/telemedicine (n = 18). Diabetes mellitus was also a common focus of the studies reviewed (n = 11).

Most studies identified in the literature search that met the criteria of comparative health systems research were intra-KP studies (interregional or intraregional that were in or between different KP Regions; n = 42).^13–54 These studies either compared one KP Region with another for a particular care topic or compared a system of care in a KP Region that had heterogeneous processes among its different medical centers. Fewer studies (n = 15) were identified that compared KP with another US health care system (ie, fee-for-service, health maintenance organization, and/or Veterans Affairs).^55–69 In addition, there were 12 studies that compared KP with international health care systems.^{55,58,62,63,70–77}

Among the different topical areas that the comparative health systems studies covered, the most frequently studied topic was quality of care (n = 30)^a and articles that related to health information technology/EHR/clinical decision support/telemedicine (n = 18).^b Other commonly studied topics included resource use (cost/utilization; n = 16),^c health systems performance (n = 7),^{14,59,61,70,72,73,76} and disease management (n = 7).^{13,15,17,25,36,40,47}

On the basis of disease or type of condition, diabetes mellitus was the most frequently studied (n = 11).^d Other conditions that were commonly studied included cardiovascular disease (n = 7),^{16,17,36,40,47,66,69} mental health and substance abuse (n = 6),^{26,30,32,45,57,67} and asthma/chronic obstructive pulmonary disease (n = 4).^16,22,35,68

Several themes emerged from the literature as possible factors that may contribute to improved care delivery in integrated delivery systems. Seven studies suggested clinical integration as a possible reason for better performance.^{19,27,58,61,62,70,74} The use of technology (ie, electronic alerts, health information technology, EHR, secure messaging, remote video technology) was another common attribute cited across studies.^e Last, a comprehensive approach to care delivery (ie, multidisciplinary care teams, comprehensive care management, interdisciplinary treatment, multimodal interventions) and self-management were other themes highlighted as possibly improving patient outcomes.^f

Discussion

In our review of the literature on comparative health systems research involving one or more KP entities, we found that most studies to date have focused primarily on intra-KP comparisons. Fewer studies compared KP with other US health care systems or international health care systems. One possible reason for this gap in the literature could be the lack of recognition of an integrated delivery system’s ability to deliver high-quality services, the paucity of comparative performance data, and the unwillingness of organizations to share performance data. Furthermore, because most of the US health care system operates under a fee-for-service model, there are a limited number of other similar integrated delivery systems with a health plan component that may serve as suitable comparisons to KP. As a result, there remains much room for growth and additional research in comparative health systems that compare the KP model of care with other US health care systems, including more traditional fee-for-service care models, academic medical centers, Veterans Affairs medical centers, other integrated delivery systems (ie, Intermountain Healthcare based in Salt Lake City, UT; Geisinger Health System headquartered in Danville, PA; Group Health in Seattle, WA), the safety net (federally qualified health centers, community health centers, and free clinics), and international health care systems. Additional research in this area could examine which systems or processes work in improving care delivery and how different systems are able to achieve these outcomes. Improved performance is evidence that key processes contribute to better care. Further investigations into the types of best practices would lead to a more comprehensive understanding of which models or systems of care are most effective.

We also found that the comparative health systems studies we reviewed tended to focus on quality of care and health information technology/EHR/clinical decision support/telemedicine. In addition, some key attributes of integrated delivery systems emerged from the literature as possibly contributing to a higher performance. Clinical integration, the use of technology, comprehensive care, and patient self-management were consistent themes identified as being associated with improved care delivery. These have been areas of emphasis in practice and research in KP, and thus such findings are not surprising. The EHR and population health management programs are considered essential elements of an integrated approach to care that promotes a consistent and reliable care experience.^7,9 However, additional research that examines other factors hypothesized to lead to a higher health system performance, such as physician financial incentives, patient-centeredness, and continuous innovation, should be further investigated. Our finding by type of condition revealed that diabetes mellitus and cardiovascular disease were the most commonly studied, likely because these are common and prevalent conditions and areas of research emphasis. Because most of the comparative health systems studies tended to focus on a limited number of condition types, studies that examine other common types of conditions, such as cancer, gastrointestinal diseases, and joint diseases, would further contribute to the body of literature.

There are a few limitations to this systematic review. First, our review of the literature from the KP Publications Library was restricted to studies published after 1995 until the most currently available at the time of the literature search. There may have been additional studies that were published before our study period or after our literature search was conducted. Second, we focused only on studies of comparative health systems that compared KP with other systems of care. There may be other comparative health systems studies that did not explicitly include KP as a comparator. We excluded studies that aggregated KP data with other health systems. Despite our best efforts, we acknowledge that we may have missed some articles in our literature search. However, we also asked other researchers at KP, as part of our systematic review, to ensure a complete and comprehensive literature search. Furthermore, this literature review did not attempt to examine or compare the outcomes of the comparative health systems studies. Rather, we sought only to identify, to classify, and to quantify the studies to help guide future research among large integrated delivery systems.

Conclusion

We found that studies published by or about KP rarely included comparative health systems research. Given the changing health care landscape and movement toward integrated care, additional empirical studies that compare the specific factors of the integrated delivery system model with other systems of care (or in KP if there is heterogeneity of such care) may identify the system-level factors that result in more efficient care delivery. Additionally, more work must be done in partnership with similar health care organizations to demonstrate the benefits of integration toward quality, affordability, accessibility, and effectiveness. Such investigations could seek to understand how systems work to improve clinical outcomes and examine what are the key characteristics of successful systems. By developing the capacity to conduct and communicate the outcomes of comparative health systems research, the health care industry will be able to disseminate and translate the best practices that are able to address issues of quality, affordability, access, and effectiveness. It is important for all to gain organizational commitment to address the research questions that compare each different system’s performance with rigor and transparency. The knowledge gained from comparative health systems research will enable the dissemination and translation of best practices that can be adopted by the larger US health care delivery system and ensure high-quality, effective care for all.

Editor’s note:

For more on research and comparative health systems, please see the editorial on page 94: A Commentary on “Comparative Health Systems Research among Kaiser Permanente and Other Integrated Delivery Systems: A Systematic Literature Review” available at: www.thepermanentejournal.org/issues/2014/summer/5693-kaiser-commentary.html.

A Superior Form of Service

Group medicine is a superior form of service. The best way to make full use of the present technology of medicine is to organize medical groups, teams that will practice in health centers. These must be close to the people, in industrial centers, residential neighborhoods, or farms.

— Henry E Sigerist, MD, 1891–1957, medical historian and social visionary