Title: High-risk patient experiences associated with an intensive primary care management program in the Veterans Health Administration

Abstract

Intensive management programs may improve healthcare experiences among high-risk and complex patients. We assessed patient experience among 1) prior enrollees (n = 59) of an intensive management program (2014–2018); 2) non-enrollees (n = 356) at program sites; and 3) non-program site patients (n =728), using a patient survey based on the Consumer Assessment of Healthcare Providers and Systems in 2019. Outcomes included patient ratings of patient-centered care; overall healthcare experience; and satisfaction with their usual outpatient care provider. In multivariate models, enrollees were more satisfied with their current provider vs. non-enrollees within program sites (aOR 2.36 [95% CI 1.15–4.85]).

Introduction

Patients at increased risk of adverse health outcomes and hospitalizations are a growing focus for health care systems (Blumenthal et al., 2016). Ambulatory intensive management programs targeting high-need, high-risk patients have shown mixed results in improving health system and clinician-focused outcomes, such as utilization, cost, or health outcomes (Delaney et al., 2020; Yoon et al., 2018; Zulman et al., 2017). Fewer studies have examined impacts on patient care experiences, despite the importance to high-quality care. Limited evidence suggests increased patient satisfaction with the programs (Zulman et al., 2017), though other studies demonstrated minimal impact (Peikes et al., 2018) or focused benefits, such as to perceived access (Boult et al., 2011). Little is known of intensive management program effects on high-risk patient experiences, including on perceptions of care coordination, self-management support, and comprehensiveness, or effects on non-enrolled patients who may indirectly benefit from program influence on primary care.

Patient experience, including satisfaction with providers, has been described as a critical component of care quality delivered to high-need, high-cost patients (Zulman et al., 2014). Understanding the short- and long-term effects of intensive management programs on patient experience through relationship-building, trust, or patient engagement would help clarify the effects of intensive management programs outside the traditional return on investment or costs (Chang et al., 2017; Hsu et al., 2019; Zulman et al., 2019).

In addition to impact on enrollee experience, intensive management programs theoretically impact a high-risk patient population through dissemination of processes, trainings, recommendations, or tools, changing provider behavior and knowledge. This may lead to affecting non-enrolled high-risk patients through altered interactions between primary care and their remaining patients (Greenhalgh, 2005). Intensive management programs may also offload workload expended on the most complex patients, with potential positive repercussions for high-risk patients cared for by less-burdened primary care teams (Okunogbe et al., 2017; Schuttner et al., 2022). To our knowledge, no studies have assessed indirect effects on non-enrolled high-risk patients for intensive management programs, though evidence supports “spillover” effects from care model implementation (Einav et al., 2020). Clarifying changes in longer-term patient experiences (direct and indirectly affected by the program) would deepen understanding of cumulative effects.

VHA uses a patient-centered medical home model (Patient Aligned Care Team, PACT) to deliver coordinated, continuous primary care through multidisciplinary teams. To better care for high-risk patients, the VHA Office of Primary Care piloted a five-site program from 2014–2018: the PACT-Intensive Management (PIM) demonstration program. PIM provided individualized case management for primary care patients, augmenting the usual medical home services. Details have been previously described (Zulman et al., 2014, 2017). Briefly, PIM teams consisted of primary care and mental health providers, nurses, and social workers directly caring for a small panel of enrolled high-risk patients. PIM enhanced the PACT model by providing enrollees with case management, care coordination, medication management, patient and caregiver education, and care navigation services. PIM also supported other primary care providers through training, performing chart reviews, and providing care plan suggestions for non-enrolled patients.

We aimed to examine patient experience arising from PIM, including perceptions of overall healthcare experiences, patient-centered care, and satisfaction with providers after PIM’s conclusion. We hypothesized high-risk patients enrolled in PIM would have better experiences than non-enrollees, and high-risk patients overall at PIM facilities would be more likely to have better experiences than those at facilities without PIM.

Methods

Study design

This retrospective cohort study examined associations between enrollment in PIM and patient experiences among high-risk veterans. We used patient survey responses to identify three experience domains: overall healthcare experience; patient-centered care; and satisfaction with recent outpatient providers. We examined associations between these domains and two PIM exposures: 1) direct personal history of PIM enrollment, and 2) indirect exposure to PIM for non-enrolled patients through support for primary care at facilities offering PIM.

Experience measures were drawn from the VHA Survey of Healthcare Experiences of Patients (SHEP) in fiscal year 2019 (FY19, October 1, 2018-September 30, 2019), a routine mail-in survey of patients with outpatient encounters in the prior month, adapted from the Consumer Assessment of Healthcare Providers and Systems (CAHPS) (Wright et al., 2006). For this survey, patients provide unique responses without duplication within the year. Covariates included facility characteristics from VHA’s Site Tracking System and Support Service Center Capital Asset databases (VSSC), and patient characteristics from the Corporate Data Warehouse or the SHEP survey (U.S. Department of Veterans Affairs, 2014). This study was conducted as an operational evaluation supported by the VHA Office of Primary Care and was considered quality improvement rather than research. It was not subject to institutional review board approval nor waiver.

Survey sample

For FY19 SHEP, a purposeful oversample of high-risk patients from select sites were included in the routine sampling frame, which was further divided into two subsampling frames. The first included any high-risk patients with prior PIM enrollment that met SHEP sampling criteria (enrollment in VHA primary care or women’s health primary care with a visit within 10 months). Patients were considered PIM-eligible if they had a validated risk prediction score (Care Assessment Need, CAN) ≥ 90^th percentile in 1 year for hospitalization or death (Fihn et al., 2014), ≥ 1 ambulatory care-sensitive chronic medical condition, a history of VHA emergency room use or facility hospitalization in the prior 6 months, and not admitted for substance use treatment, residing in nursing homes, enrolled in a specialized medical home for homeless patients, or receiving home-based primary care. The second subsampling frame included high-risk patients without prior PIM enrollment, but who met both PIM eligibility and SHEP sampling criteria at the time of the survey. Within these two sampling frames, patients were randomly sampled from five PIM sites and ten matched comparison facilities without PIM. Sites were matched based on region, hospital or community-affiliation, academic affiliation, urban or rural status, patient-centered medical home implementation performance (Nelson et al., 2014), and size. Of 5,681 patients surveyed, 1,526 (26.9%) responded overall and 1,143 (20.1%) responded to the questions of interest and were included in the analysis.

Patient experience measures

Survey questions asked patients about ambulatory care experiences for the past 6 months. We assessed three experience domains based on standard VHA question composites: a) overall healthcare experience; b) patient-centered care dimensions of comprehensiveness, care coordination, and self-management support; and c) patient satisfaction with their recent outpatient provider (details, Table e1, Supplemental Digital Content). The latter asked patients about satisfaction with a specific outpatient provider (physician, nurse practitioner, or Physician’s Assistant) seen in the past 6 months and asked the patient to self-identify if this was their primary care provider. As some patients had been with PIM until late 2018 (overlapping with the survey window), this question could refer to providers that may have been PIM-affiliates. All responses were on a minimum of a 4-point Likert scale (e.g., “never” to “always”), except for self-management support and comprehensiveness (binary yes/no) and patient satisfaction with provider (1–10 numeric rating). For this study, we derived composite binary outcomes representing responses in the best or top two-best categories across component questions (details, Table e1, Supplemental Digital Content). Similar outcomes (i.e., “top-box”) have been used previously (Schuttner et al., 2020).

PIM exposures

PIM exposure was assessed across three levels. First, PIM teams directly managed a small number of patients at PIM sites (i.e., “enrollees”). We considered patients with any length of PIM enrollment (2014–2018) as enrollees. Second, PIM supported primary care with resources and provider trainings at PIM sites, potentially indirectly affecting high-risk patients not enrolled in PIM (i.e., “non-enrollees”). Last, we included high-risk patients from non-program sites (i.e., “usual care”). Comparing enrollees to non-enrollees estimates the direct effects of PIM on enrollees at PIM sites. Enrollees compared to high-risk patients at non-program sites represents indirect and direct effects of PIM. Non-enrollees at PIM sites versus non-PIM site usual care patients clarifies the indirect effects of PIM (Figure 1).

Figure 1.

At five program sites, an intensive management program (PIM) managed some high-risk patient enrollees (reference group for direct effects of PIM) and supported primary care (PC) serving non-enrolled patients (reference group for indirect effects of PIM). At ten sites without PIM, patients continued to receive usual primary care (reference group without effect of PIM).

Covariates

Patient and clinic characteristics from the final quarter of FY18 (July 1-September 30, 2018) were included a priori in adjusted models: facility affiliation (hospital vs. community); size (full-time equivalent providers per 10,000 patients); proportion of fully staffed PACT teams; patient age; self-identified gender; race and ethnicity (Asian/other, non-Hispanic Black, Hispanic, non-Hispanic White); education (≤ vs. > high school diploma); marital status (married vs. not); neighborhood socioeconomic index (Merkin et al., 2009); self-rated physical and mental health (excellent/very good vs. all others); and Elixhauser comorbidity count (van Walraven et al., 2009), and 2019 survey quarter.

Statistical analysis

Summary statistics were reported for outcomes and patient/facility characteristics, showing mean (SD) or n (%) as appropriate. Multivariate logistic regression was conducted to assess associations of interest. Potential correlation of responses for patients within the same facility were accommodated using the analysis method of stratification for survey data (Heeringa et al., 2017). Hypothesis testing was two-sided with an alpha of 0.05. To account for potential bias from survey non-response, we developed study-specific survey weights for survey non-response and included these weights in the final adjusted models. Weights were generated using propensity scores from models that considered patient characteristics (age, sex, marital status, copayment status (a VHA marker of disability and income), number of hospitalizations and emergency department visits in prior year, Elixhauser comorbidity count, CAN risk scores, neighborhood socioeconomic index, and general vs. specialized primary care team enrollment), facility characteristics (rural vs. urban location, hospital vs. community-based clinic affiliation, and size), and survey quarter. Analyses used SAS Version 9.4 (SAS Institute Inc., Cary, NC).

Results

Of 1,143 respondents, 64% (n = 728) were “usual care” patients from non-PIM sites, 31% (n = 356) were “non-enrollees” from PIM sites without enrollment, and 5% (n = 59) were “enrollees” from PIM sites with prior enrollment. Respondents were on average 71.6 (SD=10.0) years old, mostly male (96%) and non-Hispanic White (58%). Without nonresponse weighting, PIM enrollees had worse self-rated health, higher comorbidity burden, less frequently were married, and were fewer White non-Hispanic than other groups (Table 1). Age and gender distribution were similar between groups. PIM sites were more likely to be hospital affiliated than non-PIM sites; sites were otherwise well balanced. Overall, 22% (n = 228) of respondents reported “top” experience, 38% (n = 339) reported top care coordination, 55% (n = 558) reported top self-management support, 45% (n = 456) reported top comprehensiveness, and 53% (n = 594) reported top satisfaction with recent providers. Reweighting for non-response reduced imbalances particularly in comorbidity burden across groups, although some imbalances in patient race, marital status, education, and site staffing levels remained (Table e2, Supplemental Digital Content).

Table 1.

Demographics of high-risk survey respondents according to exposure to a VHA intensive management program (PIM). Unweighted mean (SD) shown, except where noted.

	All N = 1143	Enrolled, PIM sites n = 59	Non-enrolled, PIM sites n = 356	Usual care, non-PIM sites n = 728
Age, years	71.6 (10)	72.2 (11.6)	72.2 (9.8)	71.3 (9.9)
Female, n (%)	51 (4)	3 (5)	15 (4)	33 (5)
Race / ethnicity, n (%)
Asian/Other	80 (7)	17 (29)	26 (7)	37 (5)
Black, non-Hispanic	310 (27)	11 (19)	113 (32)	186 (26)
Hispanic	90 (8)	1 (2)	6 (2)	83 (11)
White, non-Hispanic	663 (58)	30 (51)	211 (59)	422 (58)
Married, n (%)	486 (43)	21 (36)	148 (42)	317 (45)
Education ≤ high school, n (%)	439 (40)	17 (41)	157 (45)	265 (37)
Elixhauser	4.2 (2.6)	5.1 (2.8)	4.1 (2.4)	4.2 (2.6)
Physical health, n (%)a	199 (18)	6 (14)	60 (17)	133 (19)
CAN score, 1y	90.7 (6.5)	93.3 (6.5)	90.6 (6.4)	90.5 (6.6)
Mental health, n (%)a	413 (38)	14 (33)	118 (34)	281 (39)
Socioeconomic index	0.8 (0.1)	0.7 (0.1)	0.7 (0.1)	0.8 (0.1)
CBOC, n (%)	125 (11)	3 (5)	22 (6)	100 (14)
Clinic FTE / 10k patients	12.8 (2.4)	13 (3)	13 (2.7)	12.7 (2.1)
Fully staffed teams	54.0 (24.5)	64.8 (16.8)	69.7 (17.1)	45.4 (23.9)
Unadjusted responses, n (%)
Overall experienceb	228 (22)	16 (31)	65 (20)	147 (23)
Care coordinationc	339 (38)	10 (30)	94 (34)	235 (40)
Self-mgmt. supportc	558 (55)	18 (51)	166 (52)	374 (56)
Comprehensivenessc	456 (45)	16 (43)	141 (44)	299 (46)
Satisfied with providerc	594 (53)	27 (64)	176 (49)	391 (54)

^aSelf-rated health, very good/excellent.

^b“Top-two” (two best) box respondents.

^cTop-box (best) respondents.

CAN = Care Assessment Need, a validated risk prediction of hospitalization or death in 1 year. CBOC = community-based outpatient clinic. FTE = full time equivalent clinicians.

In multivariate models adjusted for patient and facility characteristics, PIM enrollees were more likely to report being highly satisfied with recent providers than non-enrollees at PIM sites (aOR 2.36 [95% CI 1.15–4.85]). PIM non-enrollees were less likely to be highly satisfied with providers than usual care patients at non-PIM sites (aOR 0.70 [95% CI 0.50–0.97]). Satisfaction with providers was similar between non-enrollees at PIM sites and usual care patients at non-PIM sites (aOR 1.64 [95 % CI 0.81–3.33]). There were no significant associations between PIM exposure and the remaining experience measures, including overall healthcare experience or patient-centered care (Table 2).

Table 2.

Adjusted associations (aOR [95% CI]) between patient experience measures and exposure to intensive program management (PIM) care for high-risk patients with (enrolled) or without prior PIM enrollment (non-enrolled) at program sites, and those at non-program sites without exposure to PIM (usual care).

	Enrolled (vs. usual care) aOR (95% CI)	Enrolled (vs. non-enrolled) aOR (95% CI)	Non-enrolled (vs. usual care) aOR (95% CI)	P a
Overall experience	0.86 (0.38–1.97)	1.23 (0.53–2.83)	0.70 (0.46–1.09)	0.29
Care coordination	0.52 (0.23–1.20)	0.71 (0.31–1.66)	0.73 (0.50–1.06)	0.12
Self-mgmt. support	1.05 (0.49–2.25)	1.27 (0.58–2.76)	0.83 (0.58–1.18)	0.55
Comprehensiveness	0.80 (0.36–1.77)	0.88 (0.39–1.97)	0.91 (0.63–1.31)	0.79
Satisfied w/ provider	1.64 (0.81–3.33)	2.36 (1.15–4.85)	0.70 (0.50–0.97)	0.02

ORs estimates likelihood of patients responding in top- or top-two best responses for all questions in the domain. Models are adjusted for patient socio-demographics, comorbidities, self-rated health, time of survey response, clinic staffing ratios, hospital/community affiliation, and size.

^aP-values are for the comparison across three groups.

Discussion

We assessed patient care experiences associated with prior implementation of an intensive management program in VHA primary care, the PIM demonstration project, among patients at high risk of adverse events. High-risk patients previously enrolled in PIM reported higher satisfaction with outpatient providers (who could have been their usual primary care provider or PIM provider) after PIM program completion compared to non-enrolled high-risk patients at the same site. We were also interested in exploring PIM’s indirect effects on patient experience, as PIM supported primary care through resources and trainings. We found little difference in high-risk patient experiences between PIM and non-PIM sites. Contrary to our hypothesis, non-enrolled patients at PIM sites were less satisfied with recent outpatient providers than usual care patients at non-PIM sites. PIM did not appear to create longer-term improvement on patient perceptions of care coordination, care comprehensiveness, self-management support, or overall healthcare experience.

Our current study builds on prior work from the VHA. A previous survey of a small number (n = 56) of high-risk patients enrolled in a randomized trial of a similar VHA intensive management program found that after 6 months, patients reported high satisfaction with case management services, improved perception of communication, and overall VHA care satisfaction (Zulman et al., 2017). A subsequent study, using data from a 2016 survey of high-risk patients at PIM sites, showed that enrolled patients were more likely to be satisfied with primary care, reported higher trust in providers, had better perceptions of care coordination, and overall were more satisfied with VHA care compared to non-enrollees (Zulman et al., 2019). Our study adds three unique aspects to these works. First, we assessed a longer follow-up window for previously enrolled patients. Second, we calculated a propensity score-based survey nonresponse weight, allowing improved comparisons between groups. Finally, we examined the potential indirect effects on non-enrolled high-risk patient populations by sampling patients across PIM and non-PIM sites.

Despite initial promising benefits (Zulman et al., 2017, 2019), PIM influence on broader patient care experiences, including perceptions of care coordination, comprehensiveness, or overall VHA experience, may not have persisted after program conclusion. These dimensions may pertain to more concrete care dimensions such as ancillary primary care staff activities (e.g., comprehensive screening, health education, or care navigation), or wider VHA factors outside primary care or PIM, such as pharmacy services, that may have little to no durability without the active case management and resources within PIM. In contrast, PIM may have led to sustained relationships or greater trust in the VHA among enrollees, translating to greater satisfaction with other outpatient providers. Primary care providers for enrollees may have alternatively had an increased knowledge of high-risk patient management or improved practice style, persisting beyond the conclusion of PIM. While we were unable to track characteristics of the providers reflected in our patient experience survey (i.e., if PIM-affiliated), we speculate that PIM could have provided sufficient training or resources to outpatient providers to sustain satisfaction of prior enrollees. The lower satisfaction with providers for non-enrolled patients compared to usual care patients at non-PIM sites suggests PIM was less beneficial for patient-provider interactions for patients not directly enrolled in PIM. However, we note high satisfaction in the sample (on average, 53% were “10/10” satisfied with their providers) may have led to a ceiling effect.

These findings have take-aways for organizations interested in the generalizable aspects of intensive management programs. These programs require significant resource investment. The limited effect on patient experience, combined with mixed or null findings for clinical and quality outcomes (Delaney et al., 2020; Yoon et al., 2018; Zulman et al., 2017), suggests the return on investment may be inadequate for systems offering patient-centered medical homes. Our findings may be higher yield for systems where patient satisfaction with providers is paramount, e.g., within fee-for-service models where satisfaction may impact care-seeking behavior. Findings suggest that satisfaction with providers may be better in the long-term after patient enrollment in an intensive management program. In contrast, the other dimensions of patient experience showed no longer-term associations with the prior presence of the intensive management program. These dimensions may relate to PIM case management services, which may have not been possible to sustain in primary care after PIM ended (Stockdale et al., 2021). These null findings are important; the immediate positive benefits on patient experience from intensive management programs (Zulman et al., 2017, 2019) beyond satisfaction with providers, may be short-lived and limited to when a patient is enrolled.

Limitations

Turnover and death likely reduced the number of prior enrollee patients available for the survey (previously, this group had 12.1% mortality and 8.6% facility attrition over 16 months) (Zulman et al., 2017). Small sample size for prior enrollees meant we could not account for the timing of PIM enrollment, limiting our understanding if patient experiences evolved after leaving the program. As an observational study, we acknowledge residual confounding may still be present. We attempted to address this by propensity score-weighting for survey non-response (though we were unable to resolve all imbalances in sociodemographics and facility staffing), and by adjusting our models for observable confounders. We were unable to link provider characteristics to the survey, so were unable to parse if outpatient providers for patients asked about in the survey were or had been PIM staff, and were unable to account for potential correlation of patient-level responses by providers. Finally, results may not generalize to programs with alternative structures, outside the VHA, or in caring for non-veterans.

Conclusions

A VHA intensive management program for high-risk patients was associated with better satisfaction with outpatient providers in the year following the program’s conclusion among program enrollees. However, there was no impact on perceptions of patient-centered care or overall healthcare experience after program conclusion. Given the resources required to implement high-risk patient intensive case management programs, the absence of lasting effects on many facets of patient experience further supports continuing to provide and strengthen comprehensive PCMH-based primary care for patients across all risk levels.

Data Availability:

Data for this study was collected as part of a national evaluation effort through the Veterans Health Administration Office of Primary Care. A deidentified limited dataset that support the findings of this study are available from the corresponding author upon reasonable request.