Ten-Year Evaluation of the Population-Based Integrated Health Care System “Gesundes Kinzigtal”

Abstract

Background

The population-based integrated health care system called “Gesundes Kinzigtal” (Integrierte Versorgung Gesundes Kinzigtal, IVGK) was initiated more than 10 years ago in the Kinzig River Valley region, which is located in the Black Forest in the German state of Baden-Württemberg. IVGK is intended to optimize health care while maximizing cost-effectiveness. It consists of programs for promoting health and for enabling cooperation among service providers, as well as of a shared-savings contract that has enabled resources to be saved every year. The goal of the present study was to investigate trends in the quality of care provided by IVGK over the past ten years in comparison to conventional care.

Methods

This is a non-randomized observational study with a control-group design (Kinzig River Valley versus 13 structurally comparable control regions), employing data collected by AOK, a large statutory health-insurance provider in Germany, over the period 2006–2015. Quality assessment was conducted with the aid of a set of indicators, developed by the authors, that was based exclusively on claims data. The statistical analysis of the trends in these indicators over time was conducted with preset criteria for the relevance of any observed changes, as well as preset mechanisms of controlling for confounding factors.

Results

For 88 of the 101 evaluable indicators, no relevant difference was seen between the trend over time in the region of the intervention and the average trend in the control regions. Relevant differences in favor of the IVGK were observed for six indicators, and negatively divergent trends compared to the controls were observed for seven indicators. In the main summarizing statistical analysis, no positive or negative difference was found between the Kinzig River Valley and the other regions with respect to trends in the health-care indicators over time.

Conclusion

An evaluation based on 101 indicators derived from health-insurance data did not reveal any improvement of the quality of care by IVGK and the totality of the programs that were implemented under it. However, under the conditions of the shared-savings contract, no relevant diminution in the quality of care was observed over a period of 10 years either, compared with structurally similar control regions without an integrated care model.

Health care in Germany is provided—historically grown—in sectors that are largely distinct from one another, at the interfaces of which recurrent losses of information and “breaks in care” occur, leading to negative effects on the quality and cost of care (1, 2). One approach to improving the effectiveness and efficiency of health care is seen in “integrated care” with horizontal and vertical networks of actors. A concept of this kind was developed for a rural area by Optimedis AG and the “Kinzigtal Medical Quality Network Medical Initiative” (MQNK), which undertook a joint venture to form “Gesundes Kinzigtal GmbH.” The selective contract “Gesundes [healthy] Kinzigtal Integrated Care” (Integrierte Versorgung Gesundes Kinzigtal, IVGK) was entered into with two health insurance companies (box).

BOX. “Healthy Kinzigtal integrated care”.

The selective contract for integrated care “Gesundes Kinzigtal” (“Healthy Kinzigtal”) (IVGK) was entered into in 2006 between the management company “Gesundes Kinzigtal GmbH” and the German General Local Health Insurance Fund (Allgemeine Ortskrankenkasse, AOK) Baden-Württemberg. At the end of 2006, the German Social Insurance Fund for Agriculture, Forestry and Horticulture (Sozialversicherung für Landwirtschaft, Forsten und Gartenbau, SVLFG) also signed up to the contract. The partners in the GmbH (limited liability company) are the Kinzigtal Medical Quality Network—Medical Initiative Kinzigtal (MQNK), which hold a two-thirds share, and Optimedis AG, which holds a one-third share.

The IVGK pursues a population-based approach with the goal of improving the health status of the population, patient orientation, and quality of care, at the same time as reducing per capita costs. The aim is to achieve this through a care program for the chronically ill, programs of target group-specific prevention and health promotion, as well as patient activation and optimized management of intersectoral interfaces (37, 38).

Persons insured with the two health insurance funds in the catchment area covered by the selective contract can sign up for the IVGK, while retaining their free choice of physician and hospital. Service partners include primary care physicians, pediatricians, other specialists, hospitals, nursing homes, and other health care professions. In 2020, a total of 45 service partners were involved, including 24 primary care physicians. The network has 115 cooperation partners, including physiotherapists, care homes, pharmacies, and fitness studios. In all, 8300 persons insured with AOK Baden-Württemberg and 305 persons insured with the SVLFG (representing in each case approximately 28% of the region‘s insured persons) were enrolled in 2020. Furthermore, 1,160 persons insured with other health insurance funds also used the services offered by “Healthy Kinzigtal.”

A core element of the IVGK is the shared savings contract agreed between the health insurance funds and the management company (37, e7– e9). The annual savings achieved since 2011 have totaled approximately 5 million euros. This amount is divided between the health insurance funds and the management company (e10). In this way, the IVGK finances the model: for example, the extrabudgetary services agreed with the service partners are remunerated (39). The basic remuneration of services is carried out using the standard care process.

The population-based IVGK is considered to be a best-practice example of integrated care in Germany as well as internationally (3– 6). The early years saw extensive accompanying research and process evaluation, for example, by means of surveys among members (e1), as well as surveys among service partners regarding satisfaction, cooperation, and their commitment to integrated care (e2), health promotion among the elderly (e3), patient satisfaction, and preference of insured persons in terms of their participation in decision-making (e4). Based on routine data, a survey on underuse, overuse, and misuse of care was conducted, involving four authors of the study reported here. The survey investigated the quality of care only for the period 2005–2011 compared to care in Baden-Württemberg (random sample)—using 18 indicators for selected diseases that formed the focus of the IVGK (7).

No external outcome evaluation based on routine statutory health insurance data for the period after 2011 has been carried out as yet. This represents a serious gap in evidence—especially if one assumes that undesirable trends such as underuse, for example, as a result of managed care elements like the shared savings contract (e5, 8), would only become demonstrably evident after a number of years. It also appears conceivable that unintended side effects as a result of focusing integrated care management on specific programs only emerge after years. Although evaluation studies, albeit with inconsistent results, from other countries are available (4, 9– 17), the differences in health care as well as in the understanding of integrated care make it impossible to conclude from this that a care model such as the IVGK is effective in Germany. Therefore, the present study evaluates the quality of care provided by the IVGK over a 10-year period using a methodological approach developed specifically for this study. The study investigates whether the quality of care in the Kinzig River Valley has decreased, remained constant, or increased compared to structurally similar control regions.

Methods

The methodology of the study—development of indicators (18), selection of comparison regions, data base, and statistical approach—is presented in the eMethods Section. In accordance with the population-based contract, the evaluation relates to all persons insured with the AOK (a large statutory health-insurance provider in Germany) living in the Kinzig River Valley and not only to AOK-insured persons enrolled in the integrated care system. For the period 2006–2011, the evaluation uses raw data that are comparable to the previous study (7), but chooses a fundamentally different methodological and statistical approach, as defined in advance in a study protocol (19). The following elements are central to this approach:

• A comprehensive set of indicators—based on the literature, developed in a structured manner, and agreed upon using a Delphi consensus process—that focuses both on the health care of insured individuals who participated in one of the 14 IVGK programs offered during the observation period as well as on the prevention and treatment of common chronic conditions with high potential for prevention or coordination (“tracers”) (18)

• A comparison with 13 structurally similar control regions (etable 1)

• An evaluation of the difference between the trend in the intervention region and the mean trend in the control regions for the individual indicator prevalences by means of a grading procedure, i.e., weakly/moderately/strongly positive and negative hints (eMethods Section)

• The main summarizing statistical analysis using a permutation test (e6) to determine the overall evidence across all indicators.

eTable 1. Characteristics of the populations in the Kinzig River Valley region, the control regions, and the comparison region Baden-Württemberg.

	Number of insured persons			Age: mean (median)			Sex (% females)			Charlson index*1 (mean)			GISD (mean)
Year	KT	CR 1–13	BW*2	KT	CR 1–13	BW*2	KT	CR 1–13	BW*2	KT	CR 1–13	BW*2	KT	CR 1–13	BW*2
2006	29 226	400 033	311 062	44.0 (44)	43.3 (44)	44.0 (45)	52.4	53.5	53.8	0.43	0.50	0.52	7.51	7.49	7.10
2007	28 698	391 142	307 434	44.3 (45)	43.8 (45)	44.3 (45)	52.2	53.4	53.7	0.39	0.46	0.48	7.36	7.33	6.96
2008	28 331	384 816	301 916	44.7 (45)	44.2 (45)	44.7 (46)	52.2	53.4	53.7	0.36	0.43	0.45	7.21	7.18	6.81
2009	28 208	381 251	299 955	45.0 (46)	44.6 (46)	44.9 (46)	52.1	53.4	53.6	0.34	0.40	0.42	7.09	7.05	6.69
2010	28 716	384 212	302 893	44.9 (46)	44.6 (46)	44.8 (46)	51.9	53.2	53.5	0.31	0.38	0.39	6.97	6.91	6.57
2011	29 041	390 794	308 033	45.0 (46)	44.6 (46)	44.7 (46)	51.8	53.1	53.4	0.29	0.35	0.36	6.84	6.78	6.45
2012	28 965	388 369	304 050	44.9 (46)	44.5 (46)	44.7 (46)	51.7	52.8	53.2	0.28	0.33	0.34	6.72	6.64	6.33
2013	28 649	384 713	300 982	44.9 (46)	44.7 (46)	44.8 (46)	52.2	53.1	53.4	0.26	0.31	0.32	6.59	6.51	6.21
2014	29 073	393 337	309 921	45.2 (47)	44.8 (47)	44.8 (46)	52.1	53.1	53.4	0.24	0.30	0.30	6.47	6.37	6.08
2015	29 206	397 845	314 269	45.2 (47)	44.9 (47)	44.7 (46)	52.1	52.9	53.1	0.23	0.28	0.28	6.34	6.24	5.96

*¹ Charlson comorbidity index in the year of individual entry into the study, averaged over the study population per region and year

*² The comparison region „BW“, i.e., the BW sample without insured persons in Kinzig River Valley, but with insured persons in the control regions.

BW, Baden-Württemberg (here: random sample); GISD, German Index of Socioeconomic Deprivation; CR 1–13, control regions; KT, Kinzig River Valley region

Results

Depending on the year, 28 208–29 206 AOK-insured persons in the intervention region and 381 251–400 033 AOK-insured persons in the control regions were included (etable 1). The study was initially based on the analysis of 119 quality indicators measurable via routine data. For 13 indicators, the number of cases was too small for the statistical analysis (1 x no patient numbers meeting the denominator criterion, 12 x no meaningful estimation of prevalence possible due to all or none of the patients being considered as fulfilling the numerator criterion due to small group sizes). A further five indicators are descriptive in nature, meaning that the development of the respective indicator prevalence cannot be definitively classified as an improvement or a deterioration. Thus, the evaluation results are based on 101 indicators (etable 2).

eTable 2. Indicators that could be operationalized and evaluated (n = 101).

ID	Indicator name (short)	Indicator statement (long)	Supply need			Donabedian
			Acute	Chronic	Prevention	Structure	Process	Outcome
1 – Heart failure
1.1	Treatment with an ACE inhibitor or AT1 blocker (> 90 DDD)	Proportion of heart failure patients prescribed at least 90 daily doses (DDD) of an ACE inhibitor or AT1 blocker in the previous 12 months		X			X
1.2	Treatment with class-I and -III antiarrhythmic drugs	Proportion of heart failure patients prescribed class-I and -III antiarrhythmic drugs (excluding amiodarone), calcium channel blockers (except amlodipine, felodipine), and non-steroidal anti-inflammatory drugs in the previous 12 months		X			X
1.3	Treatment with anti-inflammatory drugs	Proportion of heart failure patients prescribed non-selective NSAID (e.g., diclofenac, ibuprofen, naproxen) or selective Cox-2 inhibitors (coxibs) in the previous 12 months		X			X
1.4	Treatment with beta-blockers I	Proportion of heart failure patients prescribed at least 90 DDD of a beta-blocker in the previous 12 months		X			X
1.6	Assessment of electrolyte disturbance	Proportion of heart failure patients diagnosed with an electrolyte disorder (hyponatremia, hypokalemia, and hyperkalemia) in the previous 12 months		X			X
1.8	Hospital admission rate due to heart failure	Proportion of heart failure patients requiring inpatient admission for heart failure in the previous 12 months		X				X
1.9	Patient counse ling	Proportion of heart failure patients who received instruction on managing their chronic disease in the previous 12 months		X			X
1.10	Digoxin level monitoring	Proportion of heart failure patients prescribed digoxin who had their serum digoxin level checked in the previous 6 months		X			X
1.11	Medication monitoring/ medication plan	Proportion of heart failure patients with drug monitoring		X			X
1.13	Renal function and electrolyte monitoring	Proportion of heart failure patients with serum electrolyte and renal function monitoring at least every 6 months		X			X
1.15	Stroke in heart failure	Proportion of heart failure patients that required inpatient admission for ischemic stroke in the previous 12 months		X				X
1.16	Tricyclic antidepressants	Proportion of heart failure patients with depression and an indication for treatment with antidepressants who were prescribed tricyclic antidepressants		X			X
2 – Osteoporosis
2.1	Bisphosphonate	Proportion of osteoporosis patients treated witha bisphosphonate within the previous 12 months		X			X
2.2	Medication (calcium/vitamin D) for a fracture	Proportion of osteoporosis patients with a pathologic fracture who received a combination of calcium and/or vitamin D supplementation in the previous 12 months		X			X
2.3	Physiotherapy after fracture	Proportion of patients with osteoporosis and pathological fractures who received physiotherapy within the last 12 months.		X			X
3 – Metabolic syndrome/diabetes mellitus
3.1	Treatment with antihypertensive drugs I	Proportion of patients with metabolic syndrome who were prescribed antihypertensive drugs in the previous 12 months		X			X
3.2	Diabetic eye screening	Proportion of patients with diabetes mellitus who have had an annual eye exam		X			X
3.4	Amputations in diabetes	Proportion of patients with diabetes mellitus who underwent a lower extremity amputation in the previous 12 month		X				X
3.5	Exercise ECG in obesity	Proportion of obese patients that underwent an exercise ECG from their primary care physician to assess training performance in the ‧previous 12 months		X			X
3.6	Diabetic retinopathy	Proportion of diabetic patients with newly diagnosed diabetic retinopathy in the previous 12 months		X				X
3.7	Participation in the type-1 and type-2 diabetes mellitus DMP	Proportion of patients with type-1 and type-2 diabetes mellitus participating in the diabetes mellitus type 1 and type 2 DMP		X			X
4 – Chronic obstructive pulmonary disease (COPD)
4.1	Respiratory therapy	Proportion of COPD patients that received respiratory therapy in the previous 12 months		X			X
4.4	Participation in the COPD DMP	Proportion of COPD patients participating in the COPD DMP		X			X
4.6	Inpatient admission due to COPD	Proportion of COPD patients that required inpatient admission for exacerbations in the previous 12 months		X				X
5 – Depression
5.1	Follow-up care	Proportion of patients with depression who received follow-up care (quarterly visits to a psychiatrist or psychotherapist) in the previous 12 months		X			X
5.2	Inpatient admission for moderate depression	Proportion of patients diagnosed with moderate depression who required inpatient admission in the previous 12 months		X				X
5.3	Psychotherapy II	Proportion of patients with depression who received psychotherapy in the previous 12 months		X			X
5.4*	Hospital readmission for depression	Proportion of patients treated as inpatients for depression who needed to be readmitted to hospital within 30 days, 90 days, or 1 year		X				X
6 – Rheumatoid arthritis
6.1	Functional diagnostics	Proportion of rheumatoid arthritis patients aged 18 years and over undergoing regular and comprehensive disease monitoring		X			X
6.2	Renal function monitoring	Proportion of rheumatoid arthritis patients in whom renal values were monitored		X			X
6.3	Medication (disease-modifying antirheumatic drugs)	Proportion of patients aged 18 years and over with newly diagnosed rheumatoid arthritis in the previous 12 months who received at least one prescription of a disease-modifying antirheumatic drug in the ‧previous 12 months		X			X
7 – Chronic pain
7.1	Screening for depression	Proportion of chronic pain patients that were screened for depression		X			X
7.2	Constipation prophylaxis	Proportion of patients on chronic pharmacological pain management who received a prescription for laxatives in the previous 12 months			X		X
7.3	Psychotherapy III	Proportion of chronic pain patients with depression who received psychotherapy in the previous 12 months		X			X
7.4	Pain analysis	Proportion of chronic pain patients who received a pain analysis with differential diagnosis in the previous 12 months		X			X
7.5	Peptic ulcer under pain medication	Proportion of chronic pain patients who developed peptic ulcer under pain medication in the previous 12 months		X				X
8 – Back pain
8.1	Incapacity for work	Proportion of patients with low back pain in the previous 12 months and unable to work for more than 14 days		X			X
8.2	Specialist physician contact I	Proportion of patients with back pain treated by a specialist physician in the previous 12 months		X			X
8.3	Opioids	Proportion of patients with back pain who were prescribed opioids in the previous 12 months		X			X
9 – Antibiotic therapy
9.1	Cystitis pathogen detection	Proportion of patients with hematuria due to urinary tract infection for whom a midstream urine culture was performed	X				X
9.2	Fluoroquinolones and cephalosporins	Proportion of uncomplicated cystitis patients prescribed fluoroquinolones and/or cephalosporins as first-line antibiotics	X				X
9.3	Prescription of reserve antibiotics	Proportion of urinary tract infection patients that were prescribed reserve antibiotics in the previous 12 months	X				X
10 – Pediatric preventive medicine
10.2	HPV vaccinations	Proportion of girls aged 12–17 years who received their first and second vaccinations against cervical cancer (HPV)			X		X
10.3	Hip screening	Proportion of children aged < 6 weeks who underwent screening for developmental dysplasia of the hip			X		X
10.4	Visual impairment	Proportion of children aged < 36 months who underwent an eye examination for impaired vision			X		X
10.5	ADHD	Proportion of children aged > 4 years who presented again 30 days after an initial prescription for ADHD medication			X		X
11 – Age-specific indicators
11.1	Flu vaccination	Proportion of patients aged ≥ 60 years who received a flu vaccination in the previous 12 months			X		X
11.3	Polymedication I	Proportion of patients aged ≥ 65 years that were concomittantly prescribed five and more active substances in the previous 12 months		X			X
11.4	Adverse drug reactions (ADRs)	Proportion of patients aged ≥ 65 years who required inpatient admission due to ADRs in the previous 12 months		X				X
11.5	Long-term prescription of benzodiazepines	Proportion of patients aged ≥ 65 years with long-term prescription (longer than 8 weeks) of benzodiazepines in the previous 12 months		X			X
11.6	Prescription of long-acting benzodiazepines	Proportion of patients aged ≥ 65 years prescribed a long-acting benzodiazepine		X			X
12 – Multiple sclerosis
12.1	Physiotherapy	Proportion of patients diagnosed with multiple sclerosis who received physiotherapy		X			X
13 – Coronary heart disease
13.1	Prescription of antihypertensive drugs II	Proportion of CHD patients treated with ACE inhibitors or ARB (AT2 antagonists) in the previous 12 months		X			X
13.2	Prescription of acetylsalicylic acid (ASA)	Proportion of CHD patients prescribed ASA in the previous 12 months		X			X
13.3	Prescription of beta-blockers II	Proportion of CHD patients prescribed a beta-blocker in the previous 12 months		X			X
13.4	Participation in the CHD DMP	Proportion of CHD patients participating in the CHD DMP		X			X
13.5	Contact with a specialist physician II	Proportion of CHD patients with a comorbidity such as hypertension and/or diabetes mellitus and/or depression who were referred to a specialist physician in the previous 12 months		X			X
13.6	Prescription of nitrates/calcium antagonists due to absolute contraindications to beta blockers	Proportion of CHD patients prescribed nitrates and/or calcium antagonists in the previous 12 months due to absolute contraindications to beta blockers		X			X
13.9	Rehabilitation	Proportion of CHD patients post acute coronary syndrome, post coronary revascularization, and/or with stable angina pectoris and limiting symptoms and/or with chronic heart failure who received rehabilitation		X			X
13.10	Prescription of statins I	Proportion of CHD patients prescribed statins in the previous 12 months		X			X
14 – Myocardial infarction
14.1	Prescription of ACE inhibitors or ARBs for left ventricular systolic dysfunction	Proportion of post-myocardial infarction patients with left ventricular systolic dysfunction that received a prescription for ACE inhibitors or ARBs (angiotensin receptorblockers)		X			X
14.2	Prescription of beta-blockers III	Proportion of patients that suffered a myocardial infarction and received a prescription for beta-blockers		X			X
14.3	Prescription of statins II	Proportion of patients that suffered myocardial infarction and received a prescription for statins		X			X
14.4	Prescription of antiplatelet drugs	Proportion of patients that suffered myocardial infarction and received a prescription for an antiplatelet drug		X			X
15 – Stroke
15.2	Duplex ultrasound	Proportion of patients with stroke/TIA who underwent duplex ultrasound of the carotid arteries			X		X
15.3	Antiplatelet drugs	Proportion of stroke patients that received an antiplatelet drug		X			X
15.5	Hospital readmission for stroke	Proportion of stroke patients that required hospital readmission for stroke in the year following discharge		X				X
15.6	Antidepressants	Proportion of patients with stroke and depression who received a selective serotonin reuptake inhibitor (SSRI)		X			X
16 – Dementia
16.1	Prescription of acetylcholinesterase inhibitors	Proportion of patients with Alzheimer’s disease who received an acetylcholinesterase inhibitor		X			X
16.2	Imaging	Proportion of with newly diagnosed dementia patients that underwent brain imaging at diagnosis		X			X
16.4	Laboratory tests	Proportion of newly diagnosed dementia patients that underwent the following laboratory blood tests in the previous 12 months: blood count, TSH, electrolytes, glucose, vitamin B 12		X			X
16.6	Polymedication II	Proportion of dementia patients that were concomitantly prescribed ≥ five active substances		X			X
16.7	Test methods in suspected dementia	Proportion of dementia patients in whom a dementia-specific testing procedure was performed		X			X
16.8	GP-led geriatric assessment in the primary care setting	Proportion of patients that received a basic geriatric assessment by their GP		X			X
17 – Drug safety
17.1	Dementia and prescription of piracetam	Proportion of dementia patients that were prescribed piracetam in the previous 12 months		X			X
18 – Screening/prevention
18.1	Skin cancer screening	Proportion of patients aged ≥ 35 years who underwent skin cancer screening in the previous 12 months			X		X
18.2	MMRV vaccine (11–23 months)	Proportion of children (11–23 months) that received all four measles, mumps, rubella, and chickenpox vaccinations			X		X
18.3	Measles, mumps, rubella, and chickenpox vaccine (11–23 months)	Proportion of children (11–23 months) that received at least one of the measles, mumps, rubella, or chickenpox vaccines			X		X
18.5	Colon cancer screening	Proportion of patients aged ≥ 55 years that underwent colonoscopy for colon cancer screening			X		X
19 – Asthma
19.1	Allergy history	Proportion of patients with newly diagnosed bronchial asthma whose allergy-focused clinical history was taken		X			X
19.2	Long-term medication with inhaled glucocorticosteroids	Proportion of patients with chronic bronchial asthma (stage 2 and higher) receiving long-term medication with inhaled glucocorticosteroids (ICS)		X			X
20 – Regional: demography
20.1	Live births	Live births per 1000 women aged 15–49	Not classified					X
20.5	Deaths	Deaths per 100,000 inhabitants	Not classified					X
20 – Regional: utilization (inpatient)
20.7	Overall hospital cases	Insured persons hospitalized per 100,000 inhabitants			X			X
20.8	Myocardial infarction hospital cases	Insured persons hospitalized due to myocardial infarction per 100,000 inhabitants			X			X
20.9	Ischemic heart disease hospital cases	Insured persons hospitalized due to ischemic heart disease per 100,000 inhabitants			X			X
20.10	COPD hospital cases	Insured persons hospitalized due to COPD per 100 000 inhabitants			X			X
20.11	Diabetes mellitus hospital cases	Insured persons hospitalized due to diabetes mellitus per 100 000 inhabitants			X			X
20.12	Stroke hospital cases	Insured persons hospitalized due to stroke per 100 000 population			X			X
20.13	Knee and hip replacements	Insured persons undergoing a knee or hip replacement per 100 000 inhabitants		X				X
20.14	Cesarean section deliveries	Proportion of cesarean section deliveries relative to all deliveries	Not classified					X
20.15	Mental illness hospital cases	Insured persons hospitalized due to mental illness per 100 000 inhabitants			X			X
20.16	Malignant neoplasms hospital cases	Insured persons hospitalized due to malignant neoplasms per 100 000 inhabitants			X			X
20 – Regional: cancer incidence
20.17	New cancer cases	New cancer cases per 100 000 inhabitants	Not classified					X
20 – Regional: nursing care
20.19	Care-dependent persons	Persons in need of long-term care per 100 000 inhabitants	Not classified			Not classified
20 – Regional: utilization (outpatient)
20.40	Outpatient medical and psychotherapeutic treatment	Proportion of insured persons receiving medical and psychotherapeutic treatment	Not classified			X
20 – Regional: pharmacotherapy
20.43	Generic drugs	Proportion of generic drug prescriptions relative to all prescriptions of generically available drugs	Not classified				X
20.44	Analog preparations	Proportion of prescriptions of drugs classified as analog preparations with the potential for high savings	Not classified				X
20.45	RSCA	Proportion of prescriptions of RSCA-compliant antidiabetic drugs relative to all antidiabetic drugs	Not classified				X
Total			3	70	19	1	74	25

* Indicator has three forms (5.4.30, 5.4.90. 5.4.365).

IACE inhibitors, angiotensin converting enzyme inhibitors; ADHD, attention deficit hyperactivity disorder; ARB, angiotensin II receptor blockers; ASA, acetylsalicylic acid; AT1 blockers, angiotensin II receptor antagonists; AT2 antagonists, angiotensin II antagonists; COPD, chronic obstructive pulmonary disease; DDD, defined daily dose; DMP, disease management program; ECG, electrocardiogram; GP, general practitioner; HPV, human papillomavirus; ICS, inhaled corticosteroids; CHD, coronary heart disease; NSAID, non-steroidal anti-inflammatory drug; RSCA, German Risk Structure Compensation Act; SSRI, selective serotonin reuptake inhibitors; TIA, transient ischemic attack; TSH, thyroid stimulating hormone; ADR, adverse drug reactions

The contents of the table were taken from the detailed compilation of all approved indicators in the supplement to the publication by Geraedts et al. 2020 (18), with kind permission from the Zeitschrift für Evidenz, Fortbildung und Qualität im Gesundheitswesen (ZEFQ).

Overview of indicators

For 88 of these indicators, a comparison between the trend in indicator prevalence in the Kinzig River Valley region and the mean trend in the control regions did not indicate any relevant difference (efigure 1). A total of 13 indicators fulfilled the evaluation criteria defined prior to the data analysis with regard to significance and clinical relevance (Tables 1, 2; for temporal course, see eFigures 2, 3), while six showed a relevant difference in trend in favor of and seven to the disadvantage of the Kinzig River Valley region (Tables 1, 2).

eFigure 1.

Estimation of the trend difference between intervention region and control regions. The Figure shows the results regarding the trend difference for all directed indicators; to ensure uniform presentation across indicators, these were re-coded in such a way that a numerically positive trend difference always means an advantage for the Kinzig River Valley region compared to the control regions (“directed trend difference”). The individual indicators are plotted on the x-axis; the y-axis shows the re-coded differences in trend estimates between the Kinzig River Valley and the control regions with a 95% confidence interval. The result was sorted according to the size of the re-coded trend difference.

Table 1. Indicators showing a relevant difference in the trend difference of indicators in favor of the Kinzig River Valley region compared to 13 control regions.

No.	Indicator statement	2006 Baseline prevalence	Trend difference [CI]*1	z-Score [CI]*2	Evaluation (hints)*3
1.3	Proportion of heart failur patients prescribed non-selective NSAID (e.g., diclofenac, ibuprofen, naproxen) or selective Cox-2 inhibitors (“coxibs”) in the previous 12 months Program: Strong Heart (since 2007)	35.6 %	−3.79 [−5.52; −2.07]	−3.25 [−9.41; 1.62]	Moderately positive
3.7	Proportion of patients with type-1 and type-2 diabetes mellitus participating in the Diabetes Mellitus Type 1 and Type 2 DMP	43.0 %	7.67 [2.80; 12.53]	1.14 [0.40; 2.32]	Weakly positive
5.1	Proportion of patients with depression who received follow-up care (quarterly visits to a psychiatrist or psychotherapist) in the previous 12 months Program: Better Mood (since 2010)	57.7 %	7.15 [2.60; 11.70]	1.25 [0.43; 2.54]	Weakly positive
5.2	Proportion of patients diagnosed with moderate depression who required inpatient admission in the previous 12 months Program: Better Mood (since 2010)	5.65 %	−2.06 [−3.46; −0.67]	−1.78 [−5.31; 1.04]	Weakly positive
8.1	Proportion of patients with low back pain in the previous 12 months and unable to work for more than 14 days Program: S tronger Back (since 2011)	7.79 %	−0.83 [−1.36; −0.31]	−1.88 [−5.55; 1.06]	Weakly positive
20.45	Proportion of prescriptions of RSCA-compliant antidiabetic drugs relative to all antidiabetic drugs	91.4 %	2.88 [1.78; 3.99]	1.76 [0.98; 3.21]	Weakly positive

*¹ Trend difference in % (Δ_KT-CO): difference between the 5-year trend in the Kinzig River Valley region and the corresponding mean in the control regions

*² z-score: t rend difference divided by the standard deviation of trends in the control regions (this measures the extent to which the Kinzig River Valley region holds an extreme position as measured by the variation between the control regions)

*³ Evaluation (hints): evaluation according to a previously defined classification of outcomes (see eMethods Section)

DMP, disease management program; CI, 95% confidence interval; NSAID, non-steroidal anti-inflammatory drugs; RSCA, German Risk Structure Compensation Act

Table 2. Indicators showing a relevant difference in the trend difference of indicators to the disadvantage ofthe Kinzig River Valley region compared to 13 control regions.

No.	Indicator statement	2006 Baseline prevalence	Trend difference [CI]*1	z-Score [CI]*2	Evaluation (hints)*3
4.6	Proportion of COPD patients that required inpatient admission for exacerbations in the previous 12 months	3.71 %	0.94 [−0.02; 1.91]	1.39 [−0.03; 3.38]	Weakly negative
7.1	Proportion of chronic pain patients that were screened for depression Program: Well Advised (since 2013)	26.7 %	−10.32 [−15.01; −5.62]	−2.71 [−7.84; 1.35]	Moderately negative
10.3	Proportion of children aged < 6 weeks who underwent screening for developmental dysplasia of the hip Preventive pediatric health care services	70.8 %	−8.55 [−13.09; −4.00]	−1.58 [−4.60; 0.83]	Weakly negative
11.4	Proportion of patients aged ≥ 65 years who required inpatient admission due to adverse drug reactions in the previous 12 months Program: Active Health Promotion in Old Age (since 2007)	0.17 %	0.33 [0.13; 0.52]	1.82 [0.69; 3.67]	Weakly negative
12.1	Proportion of patients diagnosed with multiple sclerosis who received physiotherapy	50.7 %	−6.09 [−16.25; 4.06]	−1.42 [−5.23; 1.77]	Weakly negative
14.1	Proportion of post-myocardial infarction patients that received a prescription for ACE inhibitors or ARBs for left ventricular systolic dysfunction Program: Strong Heart (since 2007)	82.9 %	−5.45 [−9.77; −1.12]	−2.39 [−7.45; 1.54]	Moderately negative
15.6	Proportion of patients with stroke and depression who received an SSRI	30.5 %	−10.32 [−20.17; −0.47]	−2.35 [−7.47; 1.73]	Moderately negative

*¹ Trend difference in % (Δ _KT-CO): difference between the 5-year trend in the Kinzig River Valley region and the corresponding mean in the control regions

*² z-Score: t rend difference divided by the standard deviation of trends in the control regions (this measures the extent to which the Kinzig River Valley region holds an extreme position as measured by the variation between the control regions)

*³ Evaluation (hints): evaluation according to a previously defined classification of outcomes (see eMethods Section)

ACE, angiotensin converting enzyme; ARB, angiotensin II receptor blocker; CI, 95% confidence interval; SSRI, selective serotonin reuptake inhibitor

For example, with indicator 1.3, the proportion of heart failure patients with prescribed anti-inflammatory drugs is expected to decrease. The decline in the intervention region was far more pronounced compared to the average decline in the control regions, leading to a “moderately positive hint” in the evaluation in terms of the magnitude of change (difference in trend) and conspicuousness in the range of variation (z-score) compared to the control regions (Figure 1a). Tables 2 summarizes the indicators with a negative trend. With regard to the prevalence of treatment with ACE inhibitors or sartans in patients following myocardial infarction and left ventricular systolic dysfunction, the initially high level in the Kinzig River Valley region could not be maintained over time compared to the other regions (Figure 1b). In this regard, the Kinzig River Valley region fulfills the criteria for a “moderately strong negative hint” compared to the average for the control regions (Figure 1a, Figure b).

Figure 1.

Temporal trends of two indicators showing a relevant difference in a comparison of the Kinzig River Valley (KT) region with the control regions

a) An example of a positive trend in KT: heart failure (indicator 1.3)—Prescription of anti-inflammatory drugs: moderately positive hint

b) An example of a negative trend in KT: myocardial infarction (indicator 14.1)—ACE inhibitors/ARBs for left ventricular dysfunction: moderately negative hint

ACE, angiotensin converting enzyme; ARB, angiotensin II receptor blocker; KT, Kinzig River Valley; BW, Baden-Württemberg

Indicators with a trend classified as “inconclusive” could well have developed in the desired direction during the observation period, but then the trend in the Kinzig River Valley showed no conspicuousness compared to the mean and variation of trends in the control regions (20).

Tables 3 shows the results broken down according to disease groups and topics, with different numbers of indicators in each category (21). The points with which the respective indicator groups contribute to S_hint (total points average across all indicators) with weak or moderately strong hints is shown (Figure 2). It is apparent that there is no concentration of indicators with a positive or negative hint for specific disease groups or issues. Areas that appear to be particularly amenable to integrated care (high level of coordination required between primary care physicians and specialists/hospital, for example, coronary heart disease, myocardial infarction, and stroke) do not contrast positively to control regions (table 3).

Table 3. Overview of the indicator groups and overall result.

Indicators	Number of indicators	Number with a positive/negative hint and strength of the hint (points)
		Positive	Negative
By topic
Age-specific indicators/geriatrics	5	0	1 × Weak (−1)
Antibiotic therapy	3	0	0
Drug safety	1	0	0
Asthma	2	0	0
Chronic obstructive pulmonary disease	3	0	1 × Weak (−1)
Chronic pain	5	0	1 × Moderate (−3)
Dementia	6	0	0
Demography (live births/mortality) (RI)	2	0	0
Depression	6	2 × Weak (2)	0
Metabolic syndrome/diabetes	6	1 × Weak (1)	0
Screening/prevention	4	0	0
Heart failure	12	1 × Moderate (3)	0
Utilization (inpatient) (RI)	10	0	0
Utilization (outpatient) (RI)	1	0	0
Cancer incidence (RI)	1	0	0
Coronary heart disease	8	0	0
Pediatric preventive medicine	4	0	1 × Weak (−1)
Multiple sclerosis	1	0	1 × Weak (−1)
Myocardial infarction	4	0	1 × Moderate (−3)
Osteoporosis	3	0	0
Nursing care (RI)	1	0	0
Pharmacotherapy (RI)	3	1 × Weak (1)	0
Rheumatoid arthritis	3	0	0
Back pain	3	1 × Weak (1)	0
Stroke	4	0	1 × Moderate (−3)
Total indicators	101	8 Points	−13 Points
Shint (total points average across all indicators)		− 0,05
According to need for care
Supply need: acute	3	0	0
Supply need: chronic	70	4 × Weak (4) 1 × Moderate (3)	3 × Weak (−3) 3 × Moderate (−9)
Supply need: prevention	19	0	1 × Weak (−1)
Unclassified	9	1 × Weak (1)	0
According to quality dimensions*
OECD/HCQI project: patient safety/patient orientation	3	0	0
Structural quality according to Donabedian	1	0	0
Process quality according to Donabedian	74	4 × Weak (4) 1 × Moderate (3)	2 × Weak (−2) 3 × Moderate (−9)
Outcome quality according to Donabedian	25	1 × Weak (1)	2 × Weak (−2)

* Multiple entries possible

HCQI, health care quality indicators; OECD, Organisation for Economic Co-operation and Development; RI, regional indicator

Figure 2.

Summarizing result of the indicator analysis. Representation of the evidence for the summary statistic S_hint with a two-sided p-value under the null hypothesis that S_hint in the Kinzig River Valley region is equal to the mean in the control regions

KT, Kinzigtal; S_hint, total points for “positive hints” and “negative hints” (Table 3, eTable 4, eMethods)

Mortality

Improved health status of the population, as formulated in the IVGK as a goal, can be represented, among other things, by means of the indicator „deaths per 100 insured persons.“ The analysis conducted to this end revealed no difference between the Kinzig River Valley region and the control regions (efigure 4).

eFigure 4.

Mortality: change in deaths per 100 insured persons over 5 years

Bell curve showing the variance in trends in the control regions

KT, Kinzig River Valley intervention region;

CR, control regions;

BW, Baden-Württemberg (random sample without the Kinzig River Valley)

Preventable hospital cases

The indicator set includes typical integrated care indicators, such as potentially preventable hospital (re-)admissions for heart failure, chronic obstructive pulmonary disease (COPD), depression, diabetes, and adverse drug reactions. The results are inconsistent, indicating an advantage for IVGK only in relation to inpatient admissions for moderate depression (eFigure 2). Inpatient admissions due to COPD for patients with COPD, as well as hospital admissions for adverse drug reactions in old age, revealed negative trends compared to control regions (eTable 3, eFigure 3).

eTable 3. Hospital (re-)admissions and ambulatory-care–sensitive conditions.

Indicator statement	2006 Baseline prevalence	Trend difference [95% CI]*1	z-Score [95% CI]*2	Evaluation (hints)*3
Proportion of heart failure patients that required inpatient admission for heart failure in the previous 12 months	9.80%	−1.27 [−2.90; 0.35]	−0.97 [−3.24; 0.92]	Inconclusive
Proportion of heart failure patients that required inpatient admission for ischemic stroke in the previous 12 months	1.75%	0.12 [−0.33; 0.58]	0.38 [−1.08; 1.99]	Inconclusive
Proportion of insured persons hospitalized due to ischemic heart disease at least once in the calendar year*4	0.60%	−0.00 [−0.03; 0.03]	−0.00 [−0.59; 0.59]	Inconclusive
Proportion of stroke patients that required inpatient readmission for stroke in the year following discharge	14.80%	2.61 [−2.47; 7.69]	0.66 [−0.65; 2.23]	Inconclusive
Proportion of COPD patients that required inpatient admission for exacerbations in the previous 12 months	3.71%	0.94 [−0.02; 1.91]	1.39 [−0.03; 3.38]	Weakly negative
Proportion of insured persons hospitalized at least once due to COPD*4	0.13%	0.04 [0.03; 0.05]	1.99 [1.25; 3.49]	Inconclusive
Proportion of insured persons hospitalized at least once due to diabetes mellitus*4	0.26%	0.01 [−0.01; 0.02]	0.21 [−0.35; 0.85]	Inconclusive
Proportion of patients diagnosed with moderate depression who required inpatient admission in the previous 12 months	5.65%	−2.06 [−3.46; –0.67]	−1.78 [−5.31; 1.04]	Weakly positive
Proportion of patients treated as inpatients for depression who needed to be readmitted to hospital within 30 days, 90 days, or 1 year	30 Days: 13.94% 90 Days: 16.49% 365 Days: 23.78%	30 Days: 2.42 [−2.48; 7.32] 90 Days: 1.86 [−3.51; 7.23] 365 Days: –4.01 [−11.24; 3.22]	0.82 [−0.89; 2.90] 0.41 [−0.83; 1.83] −0.66 [−2.49; 0.90]	Inconclusive Inconclusive Inconclusive
Proportion of patients aged ≥ 65 years who required inpatient admission due to adverse drug reactions in the previous 12 months	0.17%	0.33 [0.13; 0.52]	1.82 [0.69; 3.67]	Weakly negative

*¹ Trend difference in % (Δ_KT-CO): difference between the 5-year trend in the Kinzig River Valley region and the corresponding mean in the control regions

*² z-score: t rend difference divided by the standard deviation of trends in the control regions (this measures the extent to which the Kinzig River Valley region holds an extreme position as measured by the variation between the control regions)

*³ Evaluation (hints): evaluation according to a previously defined classification of outcomes (see eMethods)

*⁴ The indicator “Insured persons hospitalized due to ischemic heart disease/COPD/diabetes mellitus per 100 000 population” was modified according to the data basis.

COPD, chronic obstructive pulmonary disease; 95% CI, 95% confidence interval

Selection of antibiotics

Two indicators relate to the prescribing of antibiotics (fluoroquinolones/cephalosporins for uncomplicated cystitis, and reserve antibiotics for urinary tract infection). When considering the time course, all regions showed a clear increase and no advantage for the Kinzig River Valley population (eFigures 5a– c, 6a– c).

eFigure 5a.

Proportion of patients with uncomplicated cystitis that were prescribed fluoroquinolones and/or cephalosporins as first-line antibiotics

a) Prevalence of treatment with fluoroquinolones and/or cephalosporins

KT, Kinzig River Valley intervention region;

CR, control regions;

BW, Baden-Württemberg (random sample without the Kinzig River Valley)

eFigure 5c.

Proportion of patients with uncomplicated cystitis that were prescribed fluoroquinolones and/or cephalosporins as first-line antibiotics

c) The forest plot shows the change in the prevalence of treatment with fluoroquinolones and/or cephalosporins in patients with uncomplicated cystitis over 5 years (trend in %)

95% CI, 95% confidence interval

eFigure 6a.

Proportion of urinary tract infection patients that were prescribed reserve antibiotics* in the previous 12 months

a) Prevalence of treatment with reserve antibiotics

* See eTable 5 for list

KT, Kinzig River Valley intervention region;

CR, control regions;

BW, Baden-Württemberg (random sample without the Kinzig River Valley)

eFigure 6c.

Proportion of urinary tract infection patients that were prescribed reserve antibiotics* in the previous 12 months

c) The forest plot shows the change in the prevalence of treatment with reserve antibiotics in patients with urinary tract infection

* See eTable 5 for list

95% CI, 95% confidence interval

Overall evidence

The central research question was answered with a summarizing statistical analysis (permutation analysis) that looked at trends across all 101 indicators: none of the three pre-defined statistical key performance indicators (hints, difference in trend, and z-score) revealed any statistical abnormality with regard to the role of the Kinzig River Valley region as an intervention region. The graphical representation of results (Figure 2, eFigure 7a, b) clearly shows that the key performance indicator calculated for the Kinzig River Valley (red) is in each case within the range that results from the corresponding calculation for the control regions (light green). The values should be assessed in each case in comparison to the mean value of the control regions, the value at which the bell curve reaches its highest point. The S_hint measure, which evaluates the number and extent of indicators rated as a “positive hint” or a “negative hint” according to a point scheme (table 3), was slightly less favorable in Kinzig River Valley (-0.05) than the mean of the control regions (etable 4), without being statistically significant. The results on the other measurements also provide no evidence of an extreme position for the values in the intervention region and thus no evidence of a difference in the development of quality of care as a result of the IVGK compared to the control regions (eFigure 7a, b). There are also no noteworthy differences between the 53 program-specific indicators and the 48 non-program-specific indicators.

eFigure 7a.

Overall evidence for summary statistic S_diff and summary statistic S_IzI

a) Representation of the overall evidence for the summary statistic S_diff with a two-sided p-value under the null hypothesis that S_diff in the Kinzig River Valley region is equal to the mean in the control regions

KT, Kinzig River Valley intervention region;

S_diff, the size of the trend differences

eFigure 7b.

Overall evidence for summary statistic S_diff and summary statistic S_IzI

b) Overall evidence for the summary statistic S_IzI; the dotted reference lines relate to the properties of absolute z-scores.*

* A value above 1 for the absolute value of an indicator-specific z-score is expected for 32 % of regions, above 1.96, for 5% of regions. Bell curve and p-value are not available for S_IzI.

KT: Kinzig River Valley intervention region; S_|z|, the size of the absolute trend difference taking into consideration the variation between the control regions in order to answer the question regarding an extreme position in the Kinzig River Valley region

eTable 4. Representation of the overall evidence for summary statistics*.

Region	Shint	Sdiff	S|z|
Kinzig River Valley	−0.05	−0.02	0.93
Control region 1	0.09	−0.01	0.81
Control region 2	−0.05	0.03	0.71
Control region 3	0.12	0.02	0.89
Control region 4	−0.09	−0.05	0.91
Control region 5	0.04	−0.03	0.77
Control region 6	0.02	0.04	0.67
Control region 7	0.10	−0.02	1.07
Control region 8	0.17	0.06	1.18
Control region 9	−0.02	−0.01	0.78
Control region 10	−0.10	−0.02	0.82
Control region 11	−0.10	−0.06	0.76
Control region 12	−0.06	0.06	1.21
Control region 13	0.04	−0.02	0.73

*Data (on Figure 2, eFigures 7a, eFigures b) on the three key figures: results according to region for all 101 indicators

Discussion and conclusion

Overall, the evaluation of the IVGK over a 10-year period shows no evidence of a relevant (positive or negative) difference in terms of the trend in the quality of care for the period 2006–2015 compared to the 13 structurally similar control regions. The improvement in quality of care aimed for by the IVGK could not be demonstrated. Thus, the slight trend toward an improvement in the quality of care observed for the first few years—based on different methodology and only a handful of indicators (7)—cannot be confirmed. One might suspect that the IVGK achieved an improvement ahead of the other regions due to its spirit of optimism in the early years. However, the trajectories of the indicators provided no evidence for this.

The question was whether a deterioration in the quality of care could be observed under the conditions of a shared-savings contract. Our results suggest that this is not the case. This is consistent with investigations conducted on the Accountable Care Organizations (ACO) in the USA: for example, the systematic review by Kaufman et al. (2019) (17) on the impact of ACOs on utilization, outcomes, and cost also indicates that there was no evidence to suggest that cost-saving incentives lead to negative outcomes.

If one considers the results of our evaluation in comparison to the international literature, one must first of all point out that there is no standard concept for integrated care and, therefore, different implementation strategies exist (5, 22– 24). Failure to take this into consideration could lead to incorrect conclusions and generalizations about the effectiveness of integrated care (24). The available international studies evaluating integrated care programs report inconsistent results (4, 9– 17), for example, a reduction in costs but no improvement in quality (25). The fact that quality improvements often cannot be shown is attributed by the authors of the Nuffield Report (12) to the fact that integrated care was not implemented to levels of 90% or 100% as expected, but at most to a level of 30%. We have no data at our disposal on the basis of which we could assess this in relation to the IVGK. It was only possible to evaluate the integrated care contract independently of the extent of its implementation.

There is a consensus that groups at high risk for interventions need to be identified in order to achieve measurable effects of an intervention. According to the international literature, hospitalization and readmission are key indicators for assessing health care (14, 24). One starting point are diseases for which it is assumed that preventive measures and early and effective outpatient care can prevent hospitalization (26, 27). The indicators presented on this (etable 3) show—compared with the control regions for the Kinzig River Valley—comparable trends for seven out of 10 indicators, slightly negative trends for two indicators, and a slightly positive trend for one indicator. Mortality is another key indicator for the assessment of the effectiveness of integrated care (24) and the health status of the population (OECD indicator) (28). According to a systematic review, there are mixed results in this regard for integrated care (24). We cannot confirm the survival benefits reported by Schulte et al. (29) in 2014 for the insured population enrolled on the IVGK for the overall AOK population in the Kinzig River Valley region.

Since the prevention of antibiotic resistance is of great public health relevance, the monitoring of antibiotic prescriptions provides hints on the implementation of recommendations on antibiotic selection (30). Recent nationwide analyses show that prescriptions have declined slightly (31). Following a significant increase, the IVGK has shown a slight decrease in the prescription of reserve antibiotics (not quinolones) since 2012, possibly due to expert pharmaceutical consultations.

The strengths of our evaluation lie in its long observation period of 10 years. International investigations often only consider extremely short time periods, and the results are described as preliminary (5). In contrast to many international evaluations, we used a comprehensive, systematically developed set of indicators that can be mapped with routine data (18) and which also take into account non-program-specific areas of care in order to identify possible neglect of other areas of care; however, this was not the case here. A further strength lies in the inclusion of multiple control regions that are structurally similar to the intervention region to minimize biases that cannot be eliminated from the analysis. The observed variation between regions highlights the fact that a simple comparison with a single control population or with nationwide averages (32– 34)—as has been widely conducted—is insufficient (15, 35, 36). To the best of our knowledge, this is also the first time that a cross-indicator summarizing analysis that allows the evaluation result to be described using a small number of key indicators has been conducted.

When evaluating the results, a number of limitations need to be considered. Results of the evaluation for the period 2005–2011 were known prior to conducting the study. Due to previous experience, we chose a different approach and statistical methodology in the present study. One must bear in mind that indicator prevalence depends not only on the numerator, but also on the denominator of the reference population for the indicator. Significant changes in the denominator over time, for example, due to modified diagnosis documentation, as observed for depression, affect indicator prevalence—and thus also the calculated trend difference—without necessarily changing the treatment provided by physicians or therapists (effect on the numerator of the indicator). Since these changes are not necessarily equally pronounced in all regions, artifacts cannot be ruled out in individual cases. One can also not exclude the possibility that, despite adjustment, bias exists due to unrecorded confounding variables. However, a critical review of the indicator-specific evaluations gave no indication that important differences between the Kinzig River Valley and the control regions would not have been identified using the methodology employed. Finally, limitations also lie in the routine data themselves: for example, some indicators rated as relevant and important could not be mapped (medication plan, palliative care, nursing care, emergency) (18). It was also not possible to consider important parameters such as patient-reported outcomes and experiences with health care or, for example, possible delayed onset of disease due to health-promoting programs offered by the IVGK. Other questions of interest include efficiency of care, which could not be examined as part of this study. Therefore, a wide field of research remains for future evaluations.

Supplementary Material

eMethods

Information on methodology

The study was approved by the Ethics Committee of the Medical Faculty of the Philipps University of Marburg and registered in the German Clinical Trials Registry (DRKS00012804).

Indicator development (see Geraedts et al., 2020 [18])

In order to develop indicators, the following were conducted: extensive searches for quality indicators in the literature and indicator databases; a document analysis of the IVGK programs (IVGK, Integrierte Versorgung Gesundes Kinzigtal, “Healthy Kinzigtal Integrated Care”) for program-specific indicators; and focus groups with patients, physicians, and IVGK stakeholders to potentially complement the quality indicators (QIs) identified thus far.

The literature search yielded 239 QIs and the qualitiy indicator database search 293 QIs, which were complemented by 21 QIs obtained from focus groups with patients, physicians, and stakeholders in the IVGK. Of the 553 QIs overall, 251 QIs remained after duplicate removal and alignment with the basis of data. The consensus process reduced this preliminary qualitiy indicator set to 101 QIs. In addition, 45 QIs from German Federal Health Reporting (e11)—referred to here as regional indicators—were also approved through a consensus process and included in the qualitiy indicator set; these were intended to serve as a means of integrating regional quality outcomes. Thus, the final set consisted of 146 indicators, which—as far as possible—were operationalized (regarding the number of indicators that could be ultimately operationalized and evaluated, see details below).

Selection of regions

The following rationale was adopted for the selection of comparison regions as control groups in order to achieve a quasi-experimental study design: the control regions should resemble the intervention region in as many parameters as possible, i.e., be comparable to the Kinzig River Valley in terms of its river, physician network, size of districts and towns, proximity to maximum-care hospitals, socioeconomic factors such as unemployment rate, income tax revenue, commuter balance, and educational levels, among others. For the selection process, an iterative approach was required that included subjective assessments (for example, rating some very small medium-sized towns as “small medium-sized towns that more closely resemble a small town”) as well as inclusion and exclusion criteria agreed upon by the project team through a consensus process.

Data basis and populations

Data basis

The Scientific Institute of the AOK (Wissenschaftliches Institut der AOK, WIdO) provided data on persons insured by the AOK Baden-Württemberg. This database includes, among other things, master data on insured persons, data on services delivered in the hospital and rehabilitation sectors, care provided by office-based physicians, drug prescriptions, therapeutic products, and incapacity for work. The data is made up of different data sets that can be linked via various characteristics, for example, using the ID of the internal project-specific insured person, the encrypted identification number of the German statutory health insurance-accredited physician—in this case, the operating site (BSNR) and lifelong physician number (LANR)—or institution. Observation period: 2006–2015; the start originally planned for 2005 was not possible due to a lack of sufficiently differentiated information on insurance periods for that year.

Definition of the populations:

The following target populations needed to be defined for the study on an annual basis: the population of the Kinzig River Valley region as the intervention region, the populations of 13 control regions, and the population of the Baden-Württemberg (BW) sample, comprising insured persons in BW excluding the Kinzig River Valley region. The populations were defined in several steps, starting from an INTEGRAL pool of insured persons that included all those persons insured by the AOK Baden-Württemberg on at least one day in the period 2006–2015 and who either resided in the Kinzig River Valley region or one of the 13 control regions, or belonged to the BW sample region without the Kinzig River Valley (drawn as a random sample). Inclusion and exclusion criteria were based on information relating to insured periods and place of residence (zip code).

Persons who were insured with the AOK Baden-Württemberg:

a) For the entire year in the respective calendar year

b) From the start of the year up to death

c) From birth up to the end of the year

were included.

The following persons were excluded:

a) Insured individuals who were not continuously observable, that is to say, with an incomplete insurance period due to a change of insurance with or without a change in place of residence between the 13 regions in Baden-Württemberg and the rest of Baden-Württemberg or out of Baden-Württemberg

b) Insured individuals that were consistently observable but not resident in a region.

This could apply to insured persons in the Kinzig River Valley region, the control regions, and the BW sample

Operationalization of the indicators

Based on the set of indicators, a written programing instruction, expressed in words, was included in an Excel spreadsheet for each indicator. The corresponding inclusion and exclusion variables were specified for the definition criteria. Cases of doubt regarding services to be considered or ICD codes to be included were discussed with the lead author of the indicator consensus process.

If services on the uniform value scale (Einheitlicher Bewertungsmaßstab, EBM) that were listed in the range of codes for GP-centered care (hausarztzentrierte Versorgung, HzV) and thus billed via HzV remuneration items were used to operationalize an indicator, insured persons who participated in HzV were excluded from the evaluations of this indicator, since the service could no longer be identified via the uniform value scale (EBM). This exclusion process needed to be carried out for 15 of 90 operationalizable indicators.

It was possible to operationalize 119 indicators, and a total of 101 indicators were evaluated. For two indicators, the operationalization was subsequently corrected on the basis of the discussion held at the knowledge transfer workshop in Hausach on December 3, 2019, and the new version was used in the statistical analysis. This related to indicator 1.3: prescription of anti-inflammatory drugs (exclusion of metamizole), and indicator 15.5: inpatient readmission due to stroke (use of discharge and not admission diagnosis). This did not lead to any change in the results.

Statistical analysis

Procedure overview

For each indicator, rates standardized to the population of Baden-Württemberg (“indicator prevalence”) and the average change (the trend) in the corresponding indicator prevalences during the observation period were determined, controlling for age, sex, morbidity (Charlson index) (e12, e13), and socioeconomic status (e14, e15). The 5-year difference in trend between the intervention region (IVGK) and the mean of the control regions was calculated as a statistical key figure with a two-sided 95% confidence interval. As a further statistical key figure, the z-score was determined: this describes the extent to which the Kinzig River Valley region holds an extreme position as measured by the variation between the control regions. Criteria for the relevance of differences in trend were defined beforehand; consequently, the results per indicator were classified according to a prespecified scheme as more or less strong indications of a positive or negative development of the IVGK compared to the control regions: “strongly/moderately/weakly positive (or negative) hint” or “inconclusive.” In a first step, this approach was intended to provide a descriptive assessment of the individual indicators taken in isolation. Secondly, it provided the basis for the subsequent main analysis to determine the overall evidence. On the one hand, this was intended to cover a wider range of care in terms of content considering the indicator set in terms of content, while at the same time avoiding the potential problem of false-positive signals due to multiple tests: therefore, to determine the overall evidence, a permutation approach (e10) was used across the indicators to examine the position held by the Kinzig River Valley region compared to the control regions in terms of hints, difference in trend, and absolute z-score (key figures: S_hint, S_diff, and S_|z|) when, instead of the Kinzig River Valley region, the 13 control regions are successively considered as “intervention regions.”

Approach to the statistical analysis of individual indicators:

Definitions used here:

Main analysis—summary and overall evidence across indicators:

The main results of the respective indicator-specific analyses were first summarized in table and graph form in general overviews (e16). The cross-indicator analyses are based on three statistical key figures that reflect different aspects of the Kinzig River Valley’s development. These include:

To this end, a permutation approach (e6) was used across the directed indicators to examine the position held by the respective results for the Kinzig River Valley region in comparison to the control regions if, instead of the Kinzig River Valley (KT), the 13 control regions are successively considered as the intervention region “KT.” For each directed indicator and control region, Δ“_KT”_-CO and z-score were calculated as above, and per “KT” region as well as for the Kinzig River Valley, the following measures were calculated as an average across all directed indicators:

The reports on the statistical analyses performed using Stata 15 were automated and also generated as LaTeX code using Stata 15.

• All operationalized indicators were binary (criterion fulfilled, yes/no).

• For each operationalized indicator, a structured individual report was prepared, which, in the descriptive section, outlines the indicator-specific population and the corresponding raw rates of fulfilled criteria observed in reality per region and calendar year. Furthermore, the corresponding rates standardized to the population of Baden-Württemberg are shown, referred to hereafter as “prevalence.” These were determined using logistic regression, adjusted for—in addition to age and sex—differences in morbidity (Charlson index) (e12, e13) and socioeconomic status, operationalized by an indicator for the mean German Index of Socioeconomic Deprivation (GISD) (e14, e15).

• For each indicator and region, the average change (the trend) in the corresponding prevalences during the observation period (2006–2015) was investigated. The inclusion of multiple comparison regions allows for a relatively robust estimation of global temporal trends, taking into consideration possible structural breaks caused, for example, by administrative changes in the health care system that impact all regions in equal measure. Thus, we avoid the assumption of linearity of the temporal trend in each region by including the calendar year as a categorical variable and only assume that the deviation from the global trend is linear.

• In the part on indicator-specific statistical inference, the trend difference and z-score (see definitions) were calculated as statistical key figures with a two-sided 95% confidence interval. Whenever it was also possible in terms of content for the respective indicator to indicate a desirable direction of trends in the sense that increasing/decreasing prevalences could be interpreted as improvement or deterioration (“directed indicators”), the magnitude of the trend difference was classified as clinically relevant/not clinically relevant (see definitions below).

• Based on the trend difference, the z-score, and an assessment of clinical relevance, the results for each indicator were classified according to a prespecified scheme as more or less pronounced hints (see definitions below) of a positive or negative trend in Kinzig River Valley compared to the control regions.

• Trend: For the Kinzig River Valley region, the control regions, and as a complement also for Baden-Württemberg, respectively, the average absolute change in standardized prevalences over a 5-year period (in percentage points) was estimated as the trend. The reference period of 5 years applies to all indicators irrespective of the actual observation period and is intended to improve the readability and comparability of the estimated trends.

• Trend difference: This refers to the number of percentage points by which the 5-year trend in the Kinzig River Valley region differs from the corresponding mean value in the control regions. The trend difference is given by the trend in the Kinzig River Valley region minus the mean of the trends in the control regions. Additionally, another trend difference is considered, whereby the random sample of Baden-Württemberg (outside the Kinzig River Valley) is used for comparison instead of the control regions.

• The variation (standard deviation) in trends between control regions provides information on the magnitude of changes in the prevalence of an indicator that may result from local health system differences alone.

• z-score: This describes the extent to which the Kinzig River Valley region holds an extreme position measured in terms of the variation between control regions. The z-score is defined as the trend difference divided by the standard deviation of the trends in the control regions.

• Clinical relevance: The specification according to which a trend difference could be considered as a clinically relevant difference between intervention and control regions was guided by the potential for improvement in order to catch ceiling and floor effects. This potential was defined as the difference between the baseline prevalence and theoretical optimum of 100% and 0%, respectively. A trend difference was deemed to be relevant if it was at least 10% of the potential for change and totaled at least 0.1 percentage points.

• Examples (desired: increasing prevalence):

  • A baseline prevalence of 90% can be improved by at most 10 percentage points; that means that a difference Δ_KT-CO of one percentage point is clinically relevant.

  • A baseline prevalence of 40% can be improved by at most 60 percentage points; that means that from six percentage points, the difference Δ_KT-CO of six percentage points is clinically relevant.

• For each indicator, the prevalence in the sample of insured persons in Baden-Württemberg (excluding the Kinzig River Valley) in the first year of the observation period was chosen as the baseline prevalence.

• Hints: The results per indicator were classified as “strongly/moderately/weakly positive hints,” “strongly/moderately/weakly negative hints,” or “inconclusive.” This was done by specifying requirements of varying strictness on the trend difference (clinically relevant? statistically significant?) and the z-score (Is the Kinzig River Valley region conspicuous compared to the control regions?). eTable 6 shows which specification was made before the evaluations began.

• The number of indicators rated as “positive hints” or “negative hints” (S_hint) (for weighting using points)

• The size of the trend differences (S_diff)

• The size of the absolute trend difference taking into consideration the variation between the control regions in order to answer the question regarding an extreme position in the Kinzig River Valley (S_|z|, see below).

• S_hint: Here, an indicator with a “strongly positive/negative hint” received ± 5 points, with a “moderately positive/negative hint” ± 3 points, with a “weakly positive/negative hint” ± 1 point, and with “inconclusive” 0 points. High S_hint values indicate a positive development compared to the other regions.

• S_diff: Here, in order to establish comparability between indicators with different prevalence levels, the trend difference of an indicator was placed in mathematical relation to the prevalence of the Baden-Württemberg random sample and transformed into a log odds ratio. High S_diff values indicate a positive development compared to the other regions.

• S_|z|: Here, the absolute value of the z-score was calculated for each indicator. High S_|z| values indicate an extreme (positive or negative) development compared to the other regions, while S_|z| values close to zero indicate a mid-position compared to the other regions. A value above 1 for the absolute value of an indicator-specific z-score is to be expected for 32% of regions; above 1.96, 5% of the regions are expected.

eTable 5. Reserve antibiotics.

ATC	Name
J01CF	Staphylococcal penicillins
J01CG	Staphylococcal penicillins
J01CR	Staphylococcal penicillins
J01DB	Oral cephalosporins
J01DC	Oral cephalosporins
J01DD	Oral cephalosporins
J01DE	Oral cephalosporins
J01FA06	Newer macrolides
J01FA09	Newer macrolides
J01FA10	Newer macrolides
J01FA15	Newer macrolides
J01M	Quinolones (gyrase inhibitors)
J01FF01	Lincosamides
J01FG	Streptogramins
J01AA12	Tigecycline
J01BA01	Chloramphenicol
J01DF01	Aztreonam
J01DH	Carbapenems
J01G	Aminoglycoside antibacterials
J01XA	Glycopeptide antibacterials
J01XB	Polymyxins
J01XX01	Fosfomycin iv
J01XX08	Oxazolidinones
J01XX09	Daptomycin

ATC, anatomical therapeutic chemical classification

eTable 6. Specifications for the classification of results.

Trend difference	z-Score	Hint
If	And	Then
● ΔKT−CO is clinically relevant and ● 95% confidence interval is “far” above 0 (above 0.5 x the relevance threshold)	● z > 1.96 (in the upper 2.5% range of the bell curve) and ● 95% confidence interval is above 1.00 (“far” above)	Strongly positive hint
● ΔKT−CO is clinically relevant and ● statistically significant (95% confidence interval above 0)	● z > 1.96 (in the upper 2.5% range of the bell curve)	Moderately positive hint
● ΔKT−CO is clinically relevant	● z > 1.00 (“far” up the bell curve)	Weakly positive hint
correspondingly for strongly/moderately/weakly negative hints
Otherwise		Inconclusive

eFigure 5b.

Proportion of patients with uncomplicated cystitis that were prescribed fluoroquinolones and/or cephalosporins as first-line antibiotics

b) Bell curve showing the variance in trends in the control regions: change in prevalence of treatment with fluoroquinolones and/or cephalosporins over 5 years

KT, Kinzig River Valley intervention region;

CR, control regions;

BW, Baden-Württemberg (random sample without the Kinzig River Valley)

eFigure 6b.

Proportion of urinary tract infection patients that were prescribed reserve antibiotics* in the previous 12 months

b) Change in treatment prevalence over 5 years; bell curve showing the variance in trends in the control regions

*See eTable 5 for list

KT, Kinzig River Valley intervention region;

CR, control regions;

BW, Baden-Württemberg (random sample without the Kinzig River Valley)