Skip to main content
PMC home page
Health Services Research logoLink to Health Services Research
. 2021 Jun 18;56(Suppl 1):1045–1056. doi: 10.1111/1475-6773.13674

Outcomes of a randomized quality improvement trial for high‐risk Veterans in year two

Evelyn T Chang 1,2,3,✉, Jean Yoon 4,5, Aryan Esmaeili 4, Donna M Zulman 6,7, Michael K Ong 2,3,8, Susan E Stockdale 1,9, Elvira E Jimenez 1,10, Karen Chu 1, David Atkins 11, Angela Denietolis 12, Steven M Asch 6,7; PACT Intensive Management (PIM) Demonstration Sites, PIM National Evaluation Center, and PIM Executive Committee
PMCID: PMC8515223  PMID: 34145564

Abstract

Objective

The Veterans Health Administration (VHA) conducted a randomized quality improvement evaluation to determine whether augmenting patient‐centered medical homes with Primary care Intensive Management (PIM) decreased utilization of acute care and health care costs among patients at high risk for hospitalization. PIM was cost‐neutral in the first year; we analyzed changes in utilization and costs in the second year.

Data sources

VHA administrative data for five demonstration sites from August 2013 to March 2019.

Data sources

Administrative data extracted from VHA's Corporate Data Warehouse.

Study design

Veterans with a risk of 90‐day hospitalization in the top 10th percentile and recent hospitalization or emergency department (ED) visit were randomly assigned to usual primary care vs primary care augmented by PIM. PIM included interdisciplinary teams, comprehensive patient assessment, intensive case management, and care coordination services. We compared the change in mean VHA inpatient and outpatient utilization and costs (including PIM expenses) per patient for the 12‐month period before randomization and 13‐24 months after randomization for PIM vs usual care using difference‐in‐differences.

Principal findings

Both PIM patients (n = 1902) and usual care patients (n = 1882) had a mean of 5.6 chronic conditions. PIM patients had a greater number of primary care visits compared to those in usual care (mean 4.6 visits/patient/year vs 3.7 visits/patient/year, p < 0.05), but ED visits (p = 0.45) and hospitalizations (p = 0.95) were not significantly different. We found a small relative increase in outpatient costs among PIM patients compared to those in usual care (mean difference + $928/patient/year, p = 0.053), but no significant differences in mean inpatient costs (+$245/patient/year, p = 0.97). Total mean health care costs were similar between the two groups during the second year (mean difference + $1479/patient/year, p = 0.73).

Conclusions

Approaches that target patients solely based on the high risk of hospitalization are unlikely to reduce acute care use or total costs in VHA, which already offers patient‐centered medical homes.

Keywords: Veterans Health Services, multimorbidity, Patient Care Team, case management

What is known on this topic

  • Health care systems have implemented interventions for multimorbid patients at the highest risk for adverse outcomes, such as hospitalizations and uncoordinated care, in hopes of reducing costs and improving outcomes.

  • Randomized controlled trials of intensive management programs to improve care for complex, multimorbid patients, have not demonstrated decreased acute care utilization or costs in the first 6 to 12 months of implementation.

What this study adds

  • During the second year of a VHA randomized quality improvement trial among multimorbid Veterans at the highest risk for hospitalizations, intensive management was not associated with a meaningful difference in hospitalizations, emergency department visits, or health care costs, compared to usual team‐based primary care.

  • Intensive management programs for high‐risk patient management may not provide additional utilization or cost reduction beyond that provided by patient‐centered medical homes.

  • Expectations that the benefits of intensive management might increase over time were not supported by longer follow‐up.

1. INTRODUCTION

Health care systems have implemented interventions for patients at the highest risk for adverse outcomes, such as hospitalizations and uncoordinated care, in hopes of reducing costs and improving outcomes. 1 These patients with complex needs are costly to health care systems 2 , 3 , 4 and have intense use of medical care and behavioral health services. They often have multiple chronic medical and psychiatric conditions, functional limitations, and may require a high number of medications. 2 , 5 Furthermore, their medical complexities are compounded by social needs, including social isolation, health literacy, financial insufficiency, and unstable housing. 6 , 7 Many approaches to improve care have involved multidisciplinary teams that aim to provide more proactive and coordinated management of a combination of medical, behavioral and social problems that often interact to produce poor health (hereafter called “intensive management”). Thus far, randomized controlled trials of intensive management programs to improve care for complex, high‐risk patients, including those conducted in the VHA, 8 , 9 , 10 , 11 , 12 , 13 have not demonstrated that these programs decrease acute care utilization or costs in the first 6 to 12 months of implementation.

Little is known about how intensive management for complex, high‐risk patients affects health care costs and outcomes beyond the first year. One of the few published randomized clinical trials, the Geriatric Resources for Assessment and Care of Elders (GRACE), found that providing evidence‐based care for low‐income seniors with multiple chronic conditions at high risk of hospitalization significantly decreased acute care utilization for intervention patients in the second year. 14 We hypothesized that it may take longer than 1 year to observe the full effects of intensive case management on high‐risk patients, as it takes time for case management teams to develop trusting relationships and to observe the effects of behavioral change, such as medication adherence, treatment adherence, and depression or substance use treatment, all of which were described in our formative interviews with patients and providers. 15 , 16 , 17

In this study, we report on the effects of an intensive case management program in VHA primary care on health care utilization and costs during the second year of implementation using a large randomized cohort (n = 3783). We previously reported on the outcomes from the first year of implementation separately. 8 This study was conducted as a randomized quality improvement study in the VHA, the largest integrated health care delivery system in the United States.

1.1. Methods

1.1.1. Setting

The VHA National Office of Primary Care partnered with health services investigators to test whether intensive case management programs in primary care, or Primary care Intensive Management teams (PIM), could decrease acute care utilization, such as emergency department (ED) visits and hospitalizations among high‐risk Veterans enrolled in VHA primary care. 18 The Office of Primary Care leaders selected five geographically diverse sites (Georgia, Ohio, Wisconsin, North Carolina, and California) in a competitive process to implement interdisciplinary case management teams which augmented primary care, or replaced primary care, as in one of the five sites. The PIM teams were led by primary care physicians and generally consisted of nurses, social workers, and psychologists. Three teams also included peer support specialists. The teams performed comprehensive assessments, preventative home visits, transitional care management, medication management, care coordination, health coaching, patient and caregiver education, case management for social needs, and advance care planning. 19 In 4 of 5 programs, PIM teams actively followed patients and provided patients with case management services alongside the patients' primary care teams, while in the fifth program, primary care management was transferred to the PIM team from the primary care team.

Usual care consisted of primary care delivered through VHA patient‐centered medical homes, or Patient Aligned Care Teams (PACT). The PACT model specifies that teamlets ideally include a primary care provider (physician, nurse practitioner, or physician's assistant), a registered nurse care manager who performs proactive care management and supports care coordination, a licensed vocational or practical nurse, and an administrative clerk. 20 In addition, a social worker, pharmacist, integrated mental health provider, and dietitian are available to support two PACT teamlets. PACTs routinely provide telephone and in‐person visits for all patients and offer electronic secure messaging between patients and their PACT team.

1.1.2. Study design

The full details of the evaluation protocol are described elsewhere. 18 This study is registered in Clinical Trials under study record NCT03100526. Briefly, we performed a randomized quality evaluation on high‐risk Veterans during the VHA intensive management demonstration enrollment period (August 2014‐March 2017). The outcomes of the first year were reported elsewhere using the initial cohort (n = 2210), 8 which is smaller than the full cohort reported here. Of note, a randomized quality improvement evaluation differs from traditional randomized controlled trials in that patients are not individually consented, and outcomes rely on routinely collected data. 21

1.1.3. Sample

Patients were identified as “high‐risk” if they were in the top 10% of patients at highest risk for hospitalizations by using a VHA risk prediction algorithm 22 based on prior VHA outpatient and inpatient utilization, demographics, comorbidities, and clinical variables, such as medications prescribed, lab results, and vital signs. Inclusion criteria included enrollment to VHA general primary care or women's health primary care and six‐month history of VHA hospitalization or emergency department use. Exclusion criteria included enrollment in another VHA program with case management services, (eg, homeless primary care, home‐based primary care, palliative care, hospice, or mental health intensive case management), or receipt of institutional long‐term care (ie, nursing home facility) (Figure 1).

FIGURE 1.

FIGURE 1

Open in a new tab

Participant inclusion flowchart. ED, emergency department; PIM, Primary care Intensive Management; VHA, Veterans Health Administration [Color figure can be viewed at wileyonlinelibrary.com]

We randomly assigned 1923 patients to the intervention group and 1935 to usual primary care by site and gender (to ensure representation of 10% women for future policy), according to a stratified design, using a random number generator. 18 The sample size exceeded the target sample size of 1052 (526 per evaluation group) to provide 90% power to detect a small effect (Cohen d = 0.2) on emergency department visits or hospitalizations with a 2‐sided significance level of 0.05. PIM teams reviewed the patients assigned to the intervention group and offered participation to patients who they thought would benefit the most from their services. All patients who were assigned to the intervention group were included in the analysis, regardless of whether they received the intervention.

To allow high‐risk patients in usual care the opportunity to participate in the intervention after 12 months of assignment to usual care, high‐risk Veterans in the usual care group were no longer excluded from being randomly selected to the intervention group. If they still met eligibility criteria in the second year, high‐risk Veterans could be randomized to cross over into the intervention or could remain in usual care. 18 In our analysis, we excluded the 75 patients who were initially assigned to usual care and then re‐randomized into the intervention group or usual care in the second year (Figure 1).

1.1.4. Data sources

Data were obtained from the VHA Corporate Data Warehouse, which includes the VHA outpatient National Patient Care Database, VHA Inpatient Treatment files for utilization, VHA Managerial Cost Accounting files for VHA health care costs, VHA Fee Basis files for VHA‐covered outpatient/inpatient costs and utilization in the community (hereafter referred to as “VHA‐covered” community care), and VHA Vital Status files for mortality and date of death. Medicare costs were not included in this study, as we found that the inclusion of Medicare data did not change health care utilization or costs of prior studies examining this same high‐risk cohort due to the low proportion of Medicare outpatient and inpatient utilization. 23

1.1.5. Measures

We investigated intervention effects on four primary outcomes: VHA outpatient utilization, VHA inpatient utilization, inpatient length of stay, and VHA health care costs as recorded in VHA administrative data. We also included non‐VHA outpatient (in‐person and phone visits) and inpatient utilization and costs that were provided as part of a VHA benefit for Veterans, called “Community Care.” 24 We included the PIM services as part of the outpatient utilization and costs. Mortality was a secondary outcome.

Models included patient demographics, including age, gender, race, and marital status. We also assessed chronic medical and mental health conditions (listed in Table 1) during the year prior to randomization using ICD‐9 and ICD‐10 Health Economics Resource Center definitions. 25

TABLE 1.

Baseline characteristics of patients assigned to intervention (n = 1901) and usual care (n = 1882)

Intervention (N = 1901) Usual care (N = 1882) Total (N = 3783)
Patient characteristic N (%) N (%) N (%) p value
Male, n (%) 1707 (89.79) 1686 (89.59) 3393 (89.69) 0.832
Mean age, years (SEM) 63.22 (0.30) 62.98 (0.30) 63.10 (0.21) 0.571 a
Age, n (%)
≥65 years old 925 (48.66) 908 (48.25) 1833 (48.45) 0.8
Race, n (%)
White, non‐Hispanic 918 (48.60) 880 (47.21) 1798 (47.91) 0.599
African American 815 (43.14) 843 (45.23) 1658 (44.18)
Hispanic 46 (2.44) 41 (2.20) 87 (2.32)
Other 110 (5.82) 100 (5.36) 210 (5.60)
VHA enrollment priority, n (%)
Group 1: Service connected ≥50% 727 (38.24) 707 (37.57) 1434 (37.91) 0.668
Marital status, n (%)
Married 614 (32.33) 581 (30.90) 1195 (31.62) 0.246
Divorced/separated/widowed 924 (48.66) 895 (47.61) 1819 (48.13)
Single 361 (19.01) 402 (21.38) 763 (20.19)
Unknown 2 (0.11) 4 (0.21) 6 (0.16)
Charlson comorbidity index, n (%)
0 358 (18.83) 360 (19.13) 718 (18.98) 0.632
1 367 (19.31) 384 (20.4) 751 (19.85)
2‐6 1176 (61.86) 1138 (60.47) 2314 (61.17)
Jen Frailty Index (JFI), n (%)
≥6 606 (31.96) 593 (31.64) 1199 (31.80) 0.834
Chronic medical conditions
Acid related diseases 604 (31.77) 571 (30.34) 1175 (31.06) 0.341
HIV/AIDS 43 (2.26) 36 (1.91) 79 (2.09) 0.453
Alzheimer's 16 (0.84) 20 (1.06) 36 (0.95) 0.484
Arthritis 557 (29.3) 548 (29.12) 1105 (29.21) 0.902
Asthma 166 (8.73) 161 (8.55) 327 (8.64) 0.846
Any cancer 385 (20.25) 380 (20.19) 765 (20.22) 0.963
Colorectal cancer 31 (1.63) 40 (2.13) 71 (1.88) 0.262
Congestive heart failure 271 (14.26) 266 (14.13) 537 (14.2) 0.915
Chronic renal failure 353 (18.57) 346 (18.38) 699 (18.48) 0.884
Chronic obstructive pulmonary disease 452 (23.78) 447 (23.75) 899 (23.76) 0.985
Dementia 48 (2.52) 72 (3.83) 120 (3.17) 0.022
Diabetes 732 (38.51) 748 (39.74) 1480 (39.12) 0.435
Headache 252 (13.26) 246 (13.07) 498 (13.16) 0.866
Hepatitis C 211 (11.1) 208 (11.05) 419 (11.08) 0.963
Hypertension 1390 (73.12) 1371 (72.85) 2761 (72.98) 0.851
Ischemic heart disease 463 (24.36) 451 (23.96) 914 (24.16) 0.778
Low back pain 790 (41.56) 763 (40.54) 1553 (41.05) 0.526
Lung cancer 34 (1.79) 42 (2.23) 76 (2.01) 0.331
Multiple sclerosis 8 (0.42) 8 (0.43) 16 (0.42) 0.984
Parkinson's disease 24 (1.26) 28 (1.49) 52 (1.37) 0.552
Prostatic hyperplasia 329 (17.31) 350 (18.6) 679 (17.95) 0.301
Peripheral vascular disease 242 (12.73) 221 (11.74) 463 (12.24) 0.354
Spinal cord injury 14 (0.74) 10 (0.53) 24 (0.63) 0.427
Stroke 186 (9.78) 177 (9.4) 363 (9.6) 0.692
Chronic behavioral health conditions
Any drug dependence/abuse 403 (21.2) 406 (21.57) 809 (21.39) 0.779
Anxiety 100 (5.26) 82 (4.36) 182 (4.81) 0.194
Depression 798 (41.98) 828 (44) 1626 (42.98) 0.210
Tobacco/nicotine dependence 625 (32.88) 642 (34.11) 1267 (33.49) 0.421
Post‐traumatic stress disorder 474 (24.93) 477 (25.35) 951 (25.14) 0.771
Schizophrenia 89 (4.68) 70 (3.72) 159 (4.2) 0.140
Mean number of chronic conditions (of those listed above), n (SEM) 5.58 (0.05) 5.60 (0.05) 5.59 (0.04) 0.736 a
Open in a new tab

Abbreviations: DID, difference in differences; PIM, Primary care Intensive Management; SEM, SE of the mean; VHA, Veterans Health Administration.

a

Chi‐squared used to compare all patient characteristics; except for age and mean number of chronic conditions.

1.1.6. Data analytic approach

We analyzed patients based on their original randomized assignment and excluded the 75 patients who were re‐randomized during the second year. We estimated mean inpatient and outpatient utilization and costs (including PIM program expenses) per patient for 12 months before and 13 to 24 months after randomization. We also compared unadjusted and adjusted costs (adjusted for patient fixed effects and interaction between intervention group and time, Charlson Comorbidity Index score). We used difference‐in‐differences (DID) estimated from ordinary least squares regression models such that DID = (Post − Pre)Intervention − (Post − Pre)Usual_Care. We compared utilization by type, inpatient, and outpatient, using negative binomial models. In all regression models, we adjusted standard errors for clustering within the facility.

We also performed a subgroup analysis for patients who were ≥65 years old vs <65 years old and patients who were calculated to have a frailty measure called the Jen Frailty Index (JFI) 26 ≥6 (which represents Veterans with significant functional impairment and approximates needs in three or more activities of daily living) vs JFI < 6. We hypothesized that older, more frail adults would be more likely to have significant impacts from PIM.

1.1.7. Ethics

The evaluation design was reviewed by representatives of the VHA Office of Research and Development and the Office of Research Oversight under a memorandum from the Office of Primary Care Services. Because the evaluation was designed to support VHA missions, intended to be used by and within VHA for operations or quality improvement purposes, and not designed to contribute to generalizable knowledge outside of the VHA, it met the criteria for operations quality improvement without Institutional Review Board (IRB) approval or waiver 27 , 28 conducted under the authority of the VHA national Office of Primary Care. 29 To further guide the evaluation, we also created an Ethics Committee as part of the study. 30

2. RESULTS

The study sample included a total of 3783 high‐risk Veterans who were randomly assigned: 1901 to the intervention group and 1882 to the usual care group. The groups were similar across all demographics and chronic medical/psychiatric comorbidities (Table 1). Both groups were predominantly male (90%), an average of 63 years old, mostly White (48%) or Black (44%), and unmarried (68%). We assessed 30 different chronic medical and mental health conditions and found no significant differences between groups. Both groups had an average of 5.6 chronic conditions among the ones assessed in our study.

The patients randomly assigned to PIM had, on average, 3.05 (SD 8.51) visits with the PIM team. Of patients randomly assigned to PIM, 787 (41%) did not have any visits with the PIM team during the intervention period based, because PIM team chart review determined the patient would not benefit from additional service or the patient declined, and 774 (41%) had three or more visits with the PIM team (Figure 1).

During the second year of the PIM intervention, we observed that the high‐risk patients assigned to the PIM intervention group had significantly more VHA and VHA‐covered community care outpatient encounters than the patients assigned to the usual care group (IRR 1.08; p = 0.005), regardless of the number of visits they had with the PIM team (Table 2). In addition, patients assigned to PIM had more visits in primary care (IRR 1.16; p < 0.001); mental health/substance use (IRR 1.14; p = 0.02); home visits (IRR 2.58; p < 0.001); housing services (IRR 1.33; p = 0.01); case management (IRR 3.11; p < 0.001); and telehealth for remote monitoring (IRR 1.55; p = 0.01) in Year 2. There were no differences in the number of emergency department visits (IRR 1.03; p = 0.4); urgent care (IRR 1.31; p = 0.5); or VHA‐covered community outpatient encounters (IRR 1.03; p = 0.8).

TABLE 2.

Differences in outpatient utilization for Primary care Intensive Management (PIM) relative to usual primary care

Primary care Intensive Management (PIM) (N = 1901) Usual primary care (N = 1882) Incidence rate ratio (SEM) a
Types of outpatient care, visits per year 12 months pre‐randomization Mean (SD) 13‐24 months post‐randomization Mean (SD) 12 months pre‐randomization Mean (SD) 13‐24 months post‐randomization Mean (SD) D‐D Estimate a p value
Primary care Intensive Management 0 (0) 3.05 (8.51) 0 (0) 0 (0) NA
In person VHA encounters
Emergency Department 2.72 (2.59) 1.81 (2.87) 2.73 (2.41) 1.78 (2.68) 1.03 (0.04) 0.427
Triage 1.63 (2.95) 1.31 (2.6) 1.44 (2.66) 1.28 (2.48) 0.89 (0.05) 0.031
Observation 0.17 (0.44) 0.09 (0.36) 0.14 (0.47) 0.08 (0.32) 0.98 (0.15) 0.91
Urgent care 0.02 (0.2) 0.03 (0.33) 0.02 (0.16) 0.01 (0.18) 1.31 (0.57) 0.529
Primary care 4.79 (5.86) 4.58 (7.15) 4.81 (4.52) 3.73 (4) 1.16 (0.04) <0.001
Women's health 0.22 (1.1) 0.23 (1.13) 0.28 (1.35) 0.22 (1.1) 1.25 (0.16) 0.079
Geriatrics 0.12 (0.83) 0.13 (0.77) 0.13 (1.08) 0.14 (1) 0.9 (0.22) 0.678
Home‐based primary care 0.28 (2.38) 0.79 (5.6) 0.27 (2.16) 0.59 (3.83) 1.18 (0.32) 0.549
Medical/surgical specialty 10.64 (14.3) 9.49 (14.33) 11.14 (16) 10.43 (18.53) 1.02 (0.03) 0.537
Mental health, substance use treatment 8.78 (22.64) 7.62 (21.12) 9.37 (24.26) 7.86 (20.86) 1.14 (0.06) 0.021
Mental health intensive case management 0.08 (2.36) 0.2 (4.08) 0.05 (1.49) 0.18 (2.95) 0.26 (0.19) 0.061
Social work 0.2 (1.16) 0.15 (0.72) 0.17 (0.78) 0.19 (1.25) 0.92 (0.15) 0.622
Housing services 2.23 (9.3) 1.78 (8.07) 2.45 (9.44) 1.78 (6.88) 1.33 (0.15) 0.009
Case management 0.08 (0.4) 0.07 (0.37) 0.08 (0.37) 0.02 (0.19) 3.11 (0.78) <0.001
Home visits 0.09 (0.65) 0.3 (1.76) 0.07 (0.55) 0.1 (1.09) 2.58 (0.7) <0.001
Rehabilitation therapies 2.43 (5.75) 2.2 (5.49) 2.66 (6.31) 2.37 (5.66) 0.95 (0.06) 0.453
Palliative care, hospice 0.01 (0.16) 0.03 (0.39) 0.03 (0.46) 0.05 (0.39) 1.58 (1.12) 0.519
Clinical pharmacy 2.37 (15.26) 1.66 (9.26) 1.72 (7.98) 1.32 (6.15) 1.02 (0.07) 0.764
Long‐term care (nursing home, adult day health care) 0.23 (5.02) 0.29 (5.73) 0.11 (3.79) 0.08 (0.87) 1.37 (0.65) 0.508
Other (dental, procedures, labs, diagnostic imaging) 17.98 (18.27) 14.15 (15.71) 17.58 (15.38) 13.9 (15.15) 1.02 (0.03) 0.503
Telephone VHA encounters
Primary care 3.46 (4.02) 3.5 (5.17) 3.55 (4.13) 2.91 (4.2) 1.18 (0.05) <0.001
Geriatrics 0.06 (0.48) 0.25 (1.49) 0.07 (0.54) 0.12 (0.88) 1.92 (0.48) 0.009
Mental health, substance use 0.65 (1.95) 0.59 (2.05) 0.71 (2.08) 0.66 (2.36) 1.08 (0.1) 0.431
Medical/surgical specialty 0.35 (0.93) 0.33 (1) 0.36 (1.01) 0.35 (1.16) 1.05 (0.11) 0.631
Housing services 0.48 (2.46) 0.48 (2.69) 0.82 (4.04) 0.6 (2.86) 1.14 (0.18) 0.401
Case management 0.04 (0.52) 0.28 (1.58) 0.04 (0.52) 0.06 (0.79) 5.12 (1.84) <0.001
Telehealth 1.52 (8.03) 1.25 (5.55) 1.35 (5.76) 0.91 (4.48) 1.55 (0.25) 0.007
Total VHA outpatient counts 60.39 (50.76) 52.68 (50.5) 61.22 (48.54) 50.88 (48.74) 1.08 (0.03) 0.002
Total VHA‐covered outpatient counts in community 8.96 (37.01) 10.22 (41.77) 7.87 (32.79) 8.43 (37.02) 1.03 (0.1) 0.775
Total VHA and VHA‐covered outpatient counts 72.39 (64.7) 62.9 (68.74) 72.1 (59.87) 59.31 (63.38) 1.08 (0.03) 0.005
Open in a new tab

Abbreviations: DID, difference in differences; PIM, Primary care Intensive Management; SEM, SE of the mean; VHA, Veterans Health Administration.

a

Difference‐in‐differences (DID) comparing 12 months before vs 13 to 24 months after randomization. The estimates were obtained from negative binomial models predicting utilization with covariates for year, interaction between intervention group and time, Charlson Comorbidity Index score, and patient fixed effects. Incidence rate ratio represents the difference in utilization rate for the intensive management group relative to the usual group.

Significant p values (< 0.05) are in bold.

High‐risk patients assigned to the PIM intervention had no significant difference in a number of VHA inpatient admissions (IRR 1.06; p = 0.4) for all types of inpatient admission, VHA‐covered hospitalizations in the community (IRR 1.21; p = 0.3), and total VHA and VHA‐covered community inpatient admissions (IRR 1.07; p = 0.3) relative to the usual care group during the second year (Table 3). We did not observe any significant differences in length of stay for each type of inpatient admission. There was no significant difference in VHA nursing home admission (IRR 1.07; p = 0.9).

TABLE 3.

Differences in inpatient utilization for Primary care Intensive Management (PIM) relative to usual primary care

Types of inpatient admissions, number of hospitalizations per year and length of stay (LOS) Primary care Intensive Management (PIM) (N = 1901) Usual primary care (N = 1882) ncidence rate ratio (SEM) a
12 months pre‐randomization Mean (SD) 13‐24 months post‐randomization Mean (SD) 12 months pre‐randomization Mean (SD) 13‐24 months post‐randomization Mean (SD) D‐D estimate a p value
Acute medical/surgical Counts 0.55 (0.02) 0.4 (0.02) 0.55 (0.02) 0.4 (0.02) 1.06 (0.09) 0.537
LOS 3.11 (0.28) 1.99 (0.18) 2.75 (0.21) 1.8 (0.15) 1.07 (0.1) 0.489
Psychiatry Counts 0.09 (0.01) 0.06 (0.01) 0.1 (0.01) 0.08 (0.01) 0.95 (0.18) 0.795
LOS 0.77 (0.1) 0.71 (0.16) 0.93 (0.12) 0.85 (0.15) 0.99 (0.21) 0.97
Substance use Counts 0 (0) 0 (0) 0.01 (0) 0.01 (0) 1.09 (0.79) 0.903
LOS 0.14 (0.05) 0.13 (0.05) 0.14 (0.05) 0.21 (0.06) 0.84 (0.57) 0.795
Domiciliary Counts 0.02 (0) 0.02 (0) 0.02 (0) 0.02 (0) 1.09 (0.38) 0.805
LOS 1.3 (0.3) 0.99 (0.22) 1.19 (0.28) 1.05 (0.24) 1.07 (0.39) 0.855
Rehabilitation Counts 0.01 (0) 0 (0) 0 (0) 0.01 (0)
LOS 0.1 (0.04) 0.11 (0.07) 0.11 (0.08) 0.11 (0.04)
Nursing home Counts 0.02 (0) 0.03 (0) 0.02 (0) 0.03 (0.01) 1.07 (0.39) 0.853
LOS 0.83 (0.21) 1.4 (0.32) 0.72 (0.19) 1.21 (0.3) 1.03 (0.39) 0.939
Total VHA inpatient Counts 0.69 (0.02) 0.48 (0.02) 0.69 (0.02) 0.47 (0.02) 1.06 (0.08) 0.443
Total VHA‐covered inpatient in the community Counts 0.07 (0.01) 0.09 (0.01) 0.07 (0.01) 0.07 (0.01) 1.21 (0.24) 0.33
Total VHA and VHA‐covered inpatient Counts 0.76 (0.03) 0.57 (0.03) 0.76 (0.03) 0.54 (0.03) 1.07 (0.08) 0.332
Open in a new tab

Abbreviations: DID, difference in differences; LOS, Length of stay; PIM, Primary care Intensive Management; SEM, SE of the mean; VHA, Veterans Health Administration.

a

Difference‐in‐differences (DID) comparing 12 months before vs 13 to 24 months after randomization. The estimates were obtained from negative binomial models predicting utilization with covariates for year, interaction between intervention group and time, Charlson Comorbidity Index score, and patient fixed effects. Incidence rate ratio represents the difference in utilization rate for the intensive management group relative to the usual group.

Overall, we did not observe significant differences in mean VHA and VHA‐covered community health care costs, even accounting for program costs (Table 4). There was a trend towards increased VHA outpatient costs (DID estimate $928; p = 0.053) and combined VHA and VHA‐covered community outpatient costs (DID estimate $1057; p = 0.1) among the patients assigned to PIM compared to those assigned to usual primary care. There were no significant differences in mean ED or inpatient costs. The difference in total mean VHA and VHA‐covered community costs for inpatient and outpatient utilization was not significant (DID estimate $1479; p = 0.7). Comparing costs at the 25th, 50th, and 75th percentiles, we found costs were similar between the PIM and usual care groups during the second year ($6330 vs $5845, $15 105 vs $14 581, $36 449 vs $34 311, respectively).

TABLE 4.

Differences in healthcare costs for Primary care Intensive Management (PIM) relative to usual primary care

Primary care Intensive Management (PIM) (N = 1901) Usual primary care (N = 1882)
Costs, $ 12 months pre‐randomization Mean (SD) 13‐24 months post‐randomization Mean (SD) 12 months pre‐randomization Mean (SD) 13‐24 months post‐randomization Mean (SD) D‐D estimate† p value
Total VHA inpatient costs 14 084.9 (751.2) 14 090.5 (1428.4) 13 492.0 (803.4) 13 290.9 (1139.1) 245.4 (4036.3) 0.974
Total VHA‐covered inpatient costs in the community 990.5 (159.2) 1518.1 (301.2) 767.8 (109.8) 1133.2 (164.9) 175.67 (208.07) 0.446
Total VHA and VHA‐covered inpatient costs 15 075.4 (786.4) 15 608.5 (1482.2) 14 259.8 (821.9) 14 424.1 (1171.3) 421.1 (3874.9) 0.919
Total VHA outpatient costs 17 976.7 (16 085.8) 16 912.9 (18 081.1) 18 578.0 (17 480.2) 16 953.4 (21 431.5) 928.3 (340.6) 0.053
Total VHA‐covered outpatient costs in the community 1433.6 (5960.0) 1559.5 (6897.9) 1501.5 (6321.4) 1494.5 (6937.5) 139.4 (235.7) 0.586
Total VHA and VHA‐covered outpatient costs 21 523.6 (18 696.9) 18 472.5 (20 169.6) 22 171.8 (20 095.7) 18 447.9 (22 938.9) 1057.0 (554.6) 0.129
VHA ED costs 2018.7 (59.0) 1418.7 (2544.3) 2000.3 (52.0) 1367.1 (2568.3) 50.1 (47.3) 0.349
VHA‐covered ED costs in the community 75.8 (8.3) 86.2 (11.04) 81.4 (18.4) 117.2 (20.35) −26.6 (24.0) 0.328
Total VHA and VHA‐covered ED costs 2113.3 (61.0) 1520.2 (61.6) 2092.3 (56.5) 1496.6 (65.2) 23.4 (43.7) 0.621
Total VHA and VHA‐covered costs 36 599.0 (973.0) 34 081.0 (1621.1) 36 431.6 (1001.3) 32 872.0 (1390.0) 1478.7 (3943.2) 0.727
Open in a new tab

Note: DID comparing 12 months before vs 13 to 24 months after randomization. The estimates were obtained from regression models predicting average costs with covariates for year, interaction between intensive management group and time, Charlson comorbidity index score, and patient fixed effects with considering cluster effect of stations.

Abbreviations: DID, difference in differences; ED, emergency department; PIM, Primary care Intensive Management; SEM, SE of the mean; VHA, Veterans Health Administration.

There were no differences in mortality between the two groups. Almost 10% (184/1901) died within the first 2 years after randomization among the patients assigned to the intervention, and 10% (196/1882) among the patients assigned to usual care died (p = 0.5).

In subgroup analyses, there were no significant differences in outpatient, inpatient, or total VHA utilization or costs among Veterans who were ≥65 years old or had a Jen Frailty Index ≥6 during the second year of implementation (Appendix S1). There was an insignificant decrease in outpatient encounters among PIM patients age <65 relative to older patients (p = 0.095).

3. DISCUSSION

This study indicates that augmenting VHA's patient‐centered medical home with an intensive management intervention did not reduce acute care utilization or costs in the second year after the intervention began. We found significant increases in VHA outpatient utilization (eg, primary care, mental health, home visits, case management, telehealth) in the second year among the PIM group that were also observed in the first year. 8 Both intervention and usual care groups experienced similar decreases in inpatient admissions and length of stay in the second year relative to the baseline year. We did not observe any significant differences in second‐year utilization and costs for VHA‐covered community care. There were also no significant differences in VHA utilization or costs for Veterans older than 65 years old or those more frail and functionally impaired (JFI ≥6), unlike the GRACE study, 14 although there was a small but insignificant decrease in outpatient encounters among PIM patients age < 65 relative to older patients. Overall, the intervention was cost‐neutral, even after accounting for costs of encounters with the intensive management team.

There are several reasons why we may not have observed any overall changes in health care utilization or costs among the patients who were assigned to the intervention. First, our results may be diluted, since our analysis includes all patients assigned to intensive management, regardless of whether they received the intervention. Not all patients assigned to the intervention group received intensive case management services, similar to other intensive management programs. 31 Because the PIM teams only contacted the patients who might benefit the most from their services, 41% of the patients assigned to the intervention did not receive any services, while 41% received three or more services.

Furthermore, the high‐risk patient population is heterogeneous. 32 , 33 It includes vulnerable older adults, patients with housing instability; patients with serious mental illnesses; patients with active substance use, and patients with terminal diagnoses at the end of their life. For many of these, the VHA has existing programs to provide needed services. In addition to clinical subgroups, the literature has hypothesized that subgroups of high‐risk patients can be also identified through utilization clusters, 34 quality of life, 35 and trajectory subgroups. 23 These different phenotypes of high‐risk patients may require specialized services or a more tailored approach, so a general program such as PIM may not be able to impact outcomes for a heterogeneous high‐risk patient population. Programs that focus on a smaller subset of patients who may benefit, such as the VHA home‐based primary care program, may be more likely to yield cost savings. 36 , 37 , 38

Lastly, another potential explanation is that the targeted outcomes of decreased inpatient utilization may not be modifiable in the short‐term for many high‐risk patients. 39 , 40 , 41 For instance, increased outpatient utilization could actually be appropriate based on the patient's context; PIM could have improved quality by addressing underutilization. A previous VHA case management study 42 found that hospitalized Veterans with a chronic medical condition not assigned to primary care were more likely to have increased readmissions with increased primary care services. In our study, we found that teams tended to select patients who were not already engaged in the VHA (data not published). Patients assigned to the intensive case management group had sustained increased outpatient engagement with primary care and mental health, which may be desirable for an integrated delivery system, such as the VHA, even if it does not reduce costs.

Outcomes in addition to decreased inpatient utilization may be desirable. We also found that intensive management had a modestly positive effect on patient experience outcomes, including trust and perceived care coordination. 43 Qualitative studies performed as part of this evaluation also found that patients who received intensive management services reported improved quality of care 39 and behavior changes, such as increased medication adherence and eating a healthy diet. 17 , 39 These findings suggest that intensive management might benefit patient experience in ways that could influence long‐term engagement with health care and quality of care, even if short‐term utilization is not affected.

Since the VHA Primary care Intensive Management demonstration's end in September 2018, the demonstration has provided the VHA with lessons to incorporate into patient‐centered medical homes. Most high‐risk Veterans are already managed in general primary care, 44 so building on care in existing medical homes to provide case management functions may be feasible. 45 The VHA Office of Primary Care leadership acknowledges that providing intensive case management services, such as risk stratification, comprehensive assessment, and transitional care management, in addition to routine primary care and panel management within patient‐centered medical homes will be challenging. Other practices in the Comprehensive Primary Care Plus Medicare program 46 , 47 and those described in a policy report for the Robert Wood Johnson Foundation 48 have also found similar challenges with performing case management in addition to routine primary care and panel management. Rather than relying on the teamlet, which consists of the primary care provider, registered nurse, licensed vocational nurse, and clerk, the VHA can leverage the entire patient‐centered medical home's interdisciplinary team (which ideally includes a social worker, integrated mental health provider, pharmacist, and dietitian) to assess and develop treatment plans for the few high‐risk patients who may need more services beyond what the typical medical home provides. To further equip primary care staff in their management of high‐risk patients, the VHA will develop an internal repository of tools useful for high‐risk patient management that can be quickly and easily accessed by primary care staff. The VHA also plans to organize its nurse and social work case managers in specialty care (eg, housing services, congestive heart failure, transplant services, mental health) to support high‐risk patients and to reduce duplication of case management efforts.

Limitations of this study include that the evaluation took place in the VHA setting, which may not be generalizable to health care systems that do not offer patient‐centered medical homes or to fee‐for‐service health care settings that may incentivize increased outpatient/inpatient utilization. 49 In addition, while VHA enrollees have higher rates of psychosocial conditions than non‐VHA populations, 49 , 50 high psychosocial condition rates are typical for high‐risk patients in most care systems. Our study, however, was designed specifically to inform VHA primary care operations and policy and may have implications for any large system that intends to implement similar intensive case management programs for high‐risk patients who also have multiple chronic conditions. Lastly, although our study was rigorously designed to avoid regression to the mean, 51 the benefits of the intervention could have been masked by including patients who were randomly assigned to the intervention but did not receive the intervention in our analysis.

In conclusion, the VHA found that offering an intensive case management program in addition to routine primary care services for multimorbid patients at the highest risk for hospitalizations increased outpatient utilization during the second year of implementation; however, it did not significantly decrease inpatient utilization or health care costs, even when taking VHA‐covered community care costs into account. Given the resources required to implement these intensive case management programs, the VHA plans to incorporate lessons learned from this study to further strengthen primary care's foundation of patient‐centered medical homes. Future VHA evaluations will study the population effects of incorporating intensive case management services for patients with multiple chronic conditions into patient‐centered medical home practices.

CONFLICT OF INTEREST

None of the authors have disclosed a conflict of interest. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs, or the US government, or other affiliated institutions. Preliminary results were presented at the Society of General Internal Medicine Annual Meeting in May August 11, 2019 in Washington, DC, and at the VHA HSR&D/QUERI National Conference in October 29‐31, 2019 in Washington, DC.

Supporting information

Appendix S1. Supporting information.

Click here for additional data file. (38.8KB, docx)

ACKNOWLEDGEMENTS

The authors thank the following participating investigators, who served as scientific advisors and provided and cared for study patients: Brook Watts, MD, Melissa Klein, MD, Jessica Eng, MD, Neha Pathak, MD, Deborah Henry, MD, Parag Dalsania, MD, Sudip Roy, MD, and Jeffrey Jackson, MD. We would like to thank Lisa Rubenstein, MD, MSPH, Gordon Schectman, MD, and Kathryn Corrigan, MD for their leadership in the VHA on this initiative; Carrie Patton and Belinda Velazquez for their implementation support; Edward Post, MD, PhD, Traci Solt, RN, and Ami Shah, MD on their insights on how the data may guide VHA primary care policy; Bruce Kinosian, MD, and Orna Intrator, PhD, for their collaboration on the subgroup analyses for age and Jen Frailty Index; Rong Guo, MS, and Caroline Yoo, PhDc, for programming support; Martin Lee, PhD for statistical support; Kelsey Cummings, MS for her assistance with manuscript preparation. This study was funded by the VHA Office of Primary Care (Washington DC), funding # XVA 65‐54. The funder was not involved in collection, management, and analysis of the data; preparation or final approval of the manuscript; and decision to submit the manuscript for publication; however, the funder was involved in the design of the study, the interpretation of the data, and review of the manuscript.

Chang ET, Yoon J, Esmaeili A, et al. Outcomes of a randomized quality improvement trial for high‐risk Veterans in year two. Health Serv Res. 2021;56(S1):1045–1056. 10.1111/1475-6773.13674

REFERENCES

  • 1. Bodenheimer T, Berry‐Millett R. Follow the money‐controlling expenditures by improving care for patients needing costly services. N Engl J Med. 2009;361(16):1521‐1523. [DOI] [PubMed] [Google Scholar]
  • 2. Zulman DM, Pal Chee C, Wagner TH, et al. Multimorbidity and healthcare utilisation among high‐cost patients in the US veterans affairs health care system. BMJ Open. 2015;5(4):e007771. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3. Stanton M, Rutherford M. The High Concentration of US Health Care Expenditures. Rockville, MD: Agency for Healthcare Research and Quality; 2005. [Google Scholar]
  • 4. Cohen SB, Yu W. The Concentration and Persistence in the Level of Health Expenditures over Time: Estimates for the U.S. Population, 2008–2009. Rockville, MD: Agency for Healthcare Research and Quality; 2012. [Google Scholar]
  • 5. NICE . Multimorbidity: Clinical assessment and Management, Vol 20; 2016.
  • 6. Loeb DF, Bayliss EA, Candrian C, de Gruy FV, Binswanger IA. Primary care providers' experiences caring for complex patients in primary care: a qualitative study. BMC Fam Pract. 2016;17:34. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7. Mautner DB, Pang H, Brenner JC, et al. Generating hypotheses about care needs of high utilizers: lessons from patient interviews. Popul Health Manag. 2013;16(Suppl 1):S26‐S33. [DOI] [PubMed] [Google Scholar]
  • 8. Yoon J, Chang E, Rubenstein LV, et al. Impact of primary care intensive management on high‐risk Veterans' costs and utilization: a randomized quality improvement trial. Ann Intern Med. 2018;168(12):846‐854. [DOI] [PubMed] [Google Scholar]
  • 9. Zulman DM, Pal Chee C, Ezeji‐Okoye SC, et al. Effect of an intensive outpatient program to augment primary care for high‐need veterans affairs patients: a randomized clinical trial. JAMA Intern Med. 2017;177(2):166‐175. [DOI] [PubMed] [Google Scholar]
  • 10. Finkelstein A, Zhou A, Taubman S, Doyle J. Health care Hotspotting‐a randomized, controlled trial. N Engl J Med. 2020;382(2):152‐162. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11. Dhalla IA, O'Brien T, Morra D, et al. Effect of a postdischarge virtual ward on readmission or death for high‐risk patients: a randomized clinical trial. JAMA. 2014;312(13):1305‐1312. [DOI] [PubMed] [Google Scholar]
  • 12. Englander H, Michaels L, Chan B, Kansagara D. The care transitions innovation (C‐TraIn) for socioeconomically disadvantaged adults: results of a cluster randomized controlled trial. J Gen Intern Med. 2014;29(11):1460‐1467. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13. McWilliams A, Roberge J, Anderson WE, et al. Aiming to improve readmissions through InteGrated hospital transitions (AIRTIGHT): a pragmatic randomized controlled trial. J Gen Intern Med. 2019;34(1):58‐64. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14. Counsell SR, Callahan CM, Clark DO, et al. Geriatric Care Management for low‐Income SeniorsA randomized controlled trial. JAMA. 2007;298(22):2623‐2633. [DOI] [PubMed] [Google Scholar]
  • 15. Katz ML, Stockdale S, Zulman D, et al. Understanding high risk patients' barriers to care: the detective work of VA intensive primary care teams. Paper presented at: Academy Health ARM2017; New Orleans, LA
  • 16. Stockdale S, Katz M, Bergman A, Zulman D, Denietolis A, Chang E. What do patient‐centered medical home (PCMH) teams need to better manage care for their patients at high‐risk for hospitalization or mortality? Paper presented at: Society of General Internal Medicine Annual Meeting; May August 11, 2019; Washington, DC.
  • 17. Bergman A, Stockdale S, Katz M, Zulman D, Chang E. Strategies for short and long‐term patient engagement among high‐need patients: reflections of key stakeholders in VA's primary care intensive management (PIM) teams. Academy Health Annual Research Meeting; June February 4, 2019; Washington, DC
  • 18. Chang ET, Zulman DM, Asch SM, et al. An operations‐partnered evaluation of care redesign for high‐risk patients in the veterans health administration (VHA): study protocol for the PACT intensive management (PIM) randomized quality improvement evaluation. Contemp Clin Trials. 2018;69:65‐75. [DOI] [PubMed] [Google Scholar]
  • 19. Chang ET, Raja PV, Stockdale SE, et al. What are the key elements for implementing intensive primary care? A multisite veterans health administration case study. Healthcare. 2018;6(4):231‐237. [DOI] [PubMed] [Google Scholar]
  • 20. Rosland AM, Nelson K, Sun H, et al. The patient‐centered medical home in the veterans health administration. Am J Manag Care. 2013;19(7):e263‐e272. [PubMed] [Google Scholar]
  • 21. Horwitz LI, Kuznetsova M, Jones SA. Creating a learning health system through rapid‐cycle, randomized testing. N Engl J Med. 2019;381(12):1175‐1179. [DOI] [PubMed] [Google Scholar]
  • 22. Wang L, Porter B, Maynard C, et al. Predicting risk of hospitalization or death among patients receiving primary care in the veterans health administration. Med Care. 2013;51(4):368‐373. [DOI] [PubMed] [Google Scholar]
  • 23. Wong ES, Yoon J, Piegari RI, Rosland AM, Fihn SD, Chang ET. Identifying latent subgroups of high‐risk patients using risk score trajectories. J Gen Intern Med. 2018;33(12):2120‐2126. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24. Greenstone CL, Peppiatt J, Cunningham K, et al. Standardizing care coordination within the Department of Veterans Affairs. J Gen Intern Med. 2019;34(Suppl 1):4‐6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25. Yoon J, Chow A. Comparing the Measurement of Chronic Conditions in ICD‐9‐CM and ICD‐10‐CM in VA Patients, FY2014‐FY2016. Menlo Park, CA: VA Palo Alto Health Economics Resource Center; July 2017 2017.
  • 26. Kinosian B, Wieland D, Gu X, Stallard E, Phibbs CS, Intrator O. Validation of the JEN frailty index in the National Long‐Term Care Survey community population: identifying functionally impaired older adults from claims data. BMC Health Serv Res. 2018;18(1):908. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27. VHA Handbook 1058.05: VHA operation activities that may constitute research. Department of Veterans Affairs, ed. Washington, DC: Veterans Health Administration; 2011.
  • 28. Puglisi T. Reform within the common rule? Commentary. Hastings Cent Rep. 2013;43(s1):S40‐S42. [DOI] [PubMed] [Google Scholar]
  • 29. Veterans Health Administration . Program Guide: 1200.21. VHA operations activities that may constitute research. VHA Handbook. Vol 13. Department of Veterans Affairs, Operations; 2019. [Google Scholar]
  • 30. Bottrell MM, Simon A, Geppert C, Chang ET, Asch SM, Rubenstein L. Facilitating ethical quality improvement initiatives: design and implementation of an initiative‐specific ethics committee. Healthcare. 2020;8(2):100425. [DOI] [PubMed] [Google Scholar]
  • 31. Garcia ME, Uratsu CS, Sandoval‐Perry J, Grant RW. Which complex patients should be referred for intensive care management? A mixed‐methods analysis. J Gen Intern Med. 2018;33(9):1454‐1460. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32. Prenovost KM, Fihn SD, Maciejewski ML, Nelson K, Vijan S, Rosland AM. Using item response theory with health system data to identify latent groups of patients with multiple health conditions. PLoS One. 2018;13(11):e0206915. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33. Davis AC, Shen E, Shah NR, et al. Segmentation of high‐cost adults in an integrated healthcare system based on empirical clustering of acute and chronic conditions. J Gen Intern Med. 2018;33(12):2171‐2179. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34. Lee NS, Whitman N, Vakharia N, Taksler GB, Rothberg MB. High‐cost patients: hot‐spotters don't explain the half of it. J Gen Intern Med. 2017;32(1):28‐34. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35. Rijken M, van der Heide I. Identifying subgroups of persons with multimorbidity based on their needs for care and support. BMC Fam Pract. 2019;20(1):179. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36. Edwards ST, Prentice JC, Simon SR, Pizer SD. Home‐based primary care and the risk of ambulatory care–sensitive condition hospitalization among older veterans with diabetes mellitus. JAMA Intern Med. 2014;174(11):1796‐1803. [DOI] [PubMed] [Google Scholar]
  • 37. Edwards ST, Saha S, Prentice JC, Pizer SD. Preventing hospitalization with veterans affairs home‐based primary care: which individuals benefit Most? J Am Geriatr Soc. 2017;65(8):1676‐1683. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38. Edes T, Kinosian B, Vuckovic NH, Nichols LO, Becker MM, Hossain M. Better access, quality, and cost for clinically complex veterans with home‐based primary care. J Am Geriatr Soc. 2014;62(10):1954‐1961. [DOI] [PubMed] [Google Scholar]
  • 39. Wong MSKM, Stockdale SE, Ewigman N, Jackson JL, Chang ET. High‐needs, high‐cost patient perspectives on their experiences with VHA intensive primary care teams; June, 2018; Seattle, WA.
  • 40. McWilliams JM, Schwartz AL. Focusing on high‐cost patients—the key to addressing high costs? N Engl J Med. 2017;376(9):807‐809. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41. Atkins D, Kansagara D. Reducing readmissions‐destination or journey? JAMA Intern Med. 2016;176(4):493‐495. [DOI] [PubMed] [Google Scholar]
  • 42. Weinberger M, Oddone EZ, Henderson WG. Does increased access to primary care reduce hospital readmissions? Veterans affairs cooperative study group on primary care and hospital readmission. N Engl J Med. 1996;334(22):1441‐1447. [DOI] [PubMed] [Google Scholar]
  • 43. Zulman DM, Chang ET, Wong A, et al. Effects of intensive primary care on high‐need patient experiences: survey findings from a veterans affairs randomized quality improvement trial. J Gen Intern Med. 2019;34(Suppl 1):75‐81. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44. Chang ET, Zulman DM, Nelson KM, et al. Use of general primary care, specialized primary care, and other veterans affairs services among high‐risk veterans. JAMA Netw Open. 2020;3(6):e208120‐e208120. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45. Stockdale SKM, Bergman AA, Zulman DM, Denietolis A, Chang ET. What do patient‐centered medical home (PCMH) teams need to improve care for patients with complex needs? Located at: accepted for publication. J Gen Intern Med. https://pubmed.ncbi.nlm.nih.gov/33511564/. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46. Peikes D, Anglin G, Taylor EF, et al. Independent Evaluation of Comprehensive Primary Care Plus (CPC+) First Annual Report. Princeton, MJ: Mathematic Policy Research; 2019. [Google Scholar]
  • 47. Peikes D, Anglin G, Dale S, O'Malley AS, Brown R, Genevro J. Independent Evaluation of Comprehensive Primary Care Plus (CPC+) Second Annual Report. Princeton, NJ: Mathematica; 2020. [Google Scholar]
  • 48. Bodenheimer TS, Berry‐Millett R. Care management of patients with complex health care needs. Synth Proj Res Synth Rep. 2009;19:52372. [PubMed] [Google Scholar]
  • 49. Oliver A. The veterans health administration: an American success story? Milbank Q. 2007;85(1):5‐35. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50. Watkins KE, Pincus HA, Smith B, et al. Program evaluation of VHA Mental Health Services Capstone Report. RAND Health; 2011.
  • 51. Katz MH. Trust but Verify (ideally with a randomized clinical trial). JAMA Intern Med. 2017;177(2):162‐163. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Appendix S1. Supporting information.

Click here for additional data file. (38.8KB, docx)

Articles from Health Services Research are provided here courtesy of Health Research & Educational Trust

RESOURCES