Linking Practice Adoption of Patient Engagement Strategies and Relational Coordination to Patient‐Reported Outcomes in Accountable Care Organizations

Abstract

Policy Points.

• Accountable care organizations (ACOs) have incentives to promote the adoption of patient engagement strategies such as shared decision making and self‐management support programs to improve patient outcomes and contain health care costs.

• High adoption of patient engagement strategies among ACO‐affiliated practices did not improve patient‐reported outcomes (PROs) of physical, emotional, and social function among adult patients with diabetes and/or cardiovascular disease over a one‐year time frame, likely because implementing these strategies requires extensive clinician and staff training, workflow redesign, and patient participation over time.

• A dominant focus on improving clinical measures to meet external requirements may crowd out time needed for care team members to address other outcomes that matter to patients, including PROs.

• Payers and policy‐makers should explicitly incentivize the collection and use of PROs when contracting with ACOs.

Context

Adult primary care practices of accountable care organizations (ACOs) are adopting a range of patient engagement strategies, but little is known about how these strategies are related to patient‐reported outcomes (PROs) and how relational coordination among team members aids implementation.

Methods

We used a mixed‐methods cohort study design integrating administrative and clinical data with two data collection waves (2014‐2015 and 2016‐2017) of clinician and staff surveys (n = 764), surveys of adult patients with diabetes and/or cardiovascular disease (CVD) (n = 1,276), and key informant interviews of clinicians, staff, and administrators (n = 103). Multivariable linear regression estimated the relationship of practice adoption of patient engagement strategies, relational coordination, and PROs of physical, social, and emotional function. The mediating role of patient activation was examined using cross‐lagged panel models. Key informant interviews assessed how relational coordination influences the implementation of patient engagement strategies.

Findings

There were no differential improvements in PROs among patients of practices with high vs. low adoption of patient engagement strategies or among patients of practices with high vs. low relational coordination. The Patient Activation Measure (PAM) is strongly related to better physical, emotional, and social PROs over time. Relational coordination facilitated the implementation of patient engagement strategies, but key informants indicated that resources and systems to systematically track treatment preferences and goals beyond clinical indicators were needed to support effective implementation.

Conclusions

Adult patients with diabetes and/or CVD of ACO‐affiliated practices with high adoption of patient engagement strategies do not have improved PROs of physical, emotional, and social function over a one‐year time frame. Implementing patient engagement strategies increases task interdependence among primary care team members, which needs to be carefully managed. ACOs may need to make greater investment in collecting, monitoring, and analyzing PRO data to ensure that practice adoption and implementation of patient engagement strategies leads to improved physical, emotional, and social function among patients.

The patient protection and affordable care act empowered the Centers for Medicare and Medicaid Services to create accountable care organizations (ACOs) charged with being accountable for both the costs and quality of care for a defined group of patients.1, 2 Almost all ACOs in the United States include physician groups and hospitals, with some involving other organizations such as postacute care facilities and community health centers as well,3, 4 which support the provision of coordinated patient care across the continuum of care settings. ACOs provide a unique opportunity to examine fundamental changes in how health care services are delivered and whether these changes improve patient engagement in their own health and health care. ACOs incentivize their affiliated practices to improve patient engagement because ACO contracts with health plans and payers involve capitated or global payment and financial risks and rewards.1, 2 As a result, there is growing interest in accelerating the adoption of patient engagement strategies among health care systems and adult primary care practices.5 ACOs, however, face strong incentives to improve quality and decrease costs of care in the short run. Performance feedback, population health management initiatives, and utilization management are much easier strategies for ACOs to implement compared to the disruptive operational changes associated with adopting and implementing patient engagement strategies, which require clinician and staff training, workflow redesign, and patient participation.

Diabetes and/or cardiovascular disease (CVD) combined generate annual US health care costs of $353 billion, contributing to rising costs in the United States.6, 7, 8 Treatment adherence and lifestyle changes remain low among adult patients with diabetes and/or CVD, and physicians often do not know how to better activate patients as part of routine chronic care management activities.9, 10, 11 The ability of ACOs to succeed under the new payment models that incentivize the provision of patient‐centered care will depend on improving engagement among adult patients with diabetes and/or CVD in primary care settings. There is a growing evidence base that such engagement is associated with better patient outcomes and how to better care for adults with diabetes and/or CVD,12, 13, 14, 15, 16, 17, 18, 19, 20 but little is known specifically about what practices are doing to better engage their patients in their own health and health care.21, 22, 23 Adult primary care practices affiliated with ACOs are beginning to adopt and learn from a range of patient engagement strategies,24 such as motivational interviewing, shared decision making, shared medical appointments, and health risk assessments, as well as including patients in quality improvement and clinic governance, but little is known about the connection of these efforts to patient‐reported outcomes (PROs) of care, including physical, social, and emotional function.

A foundational change in US health care delivery promoted by ACOs is the use of team‐based primary care to more effectively manage chronic conditions and to engage patients in their own health and health care.25 Nonphysician primary care clinicians and staff may be better positioned to uncover and address social and nonmedical issues that impede self‐management of chronic conditions.26, 27 Relational coordination among primary care team members has been associated with quality of care, improved efficiency, and higher patient and staff satisfaction,28, 29, 30 and may also support patient engagement. Relational coordination is the “mutually reinforcing process of communicating and relating for the purpose of task integration”31 and includes shared goals, team communication, and team coordination. Shared goals and more accurate, timely, and frequent communication may put primary care teams in a better position to engage patients and their families. In cross‐sectional analyses, however, relational coordination was not significantly associated with better PROs of physical, emotional, and social functioning.32, 33 Given the documented challenges of implementing patient engagement strategies,34 it may be that the benefits of relational coordination among team members accrue over time. Implementing patient engagement strategies can be disruptive to operations and invoke a high level of task interdependence among care team members. A strong foundation of relational coordination may enable teams to manage increased task interdependencies to engage patients and improve PROs over time because shared goals and effective problem‐solving communication enable primary care teams to overcome hurdles faced when implementing patient engagement strategies.

The development of ACOs provides an opportunity to examine the connections between practice adoption of patient engagement strategies, relational coordination among primary care team members, and PROs. Using a cohort of adults with diabetes and/or CVD, we examined the cross‐sectional and temporal relationships of practice adoption of patient engagement strategies, relational coordination among primary care team members, and PROs of physical, emotional, and social function. Key informant interviews of clinicians and staff assessed the ways in which relational coordination influenced the implementation of patient engagement strategies. A logic model (Figure 1) visually depicts our conceptualization of the connections among the study constructs and our four research hypotheses:

• Hypothesis 1: Patients of practices with high adoption of patient engagement strategies will be more activated and engaged in their own care and have better PROs of physical, emotional, and social function over time compared to patients of practices with low adoption of patient engagement strategies.

• Hypothesis 2: A foundation of high relational coordination will enable the adoption and implementation of patient engagement strategies at the practice level. Patients of practices with high relational coordination among team members will be more activated and engaged in their own care and have better PROs of physical, emotional, and social function over time compared to patients of practices with low relational coordination.

• Hypothesis 3: Patient activation will partially mediate the relationship of high practice adoption of patient engagement strategies and better PROs of physical, emotional, and social function, as well as partially mediate the relationship of high relational coordination and better PROs.

• Hypothesis 4: Consistent with evidence highlighting the benefits of patient activation on health care utilization and outcomes,13, 15, 18, 19 more highly activated patients will have better PROs of physical, emotional, and social function at baseline and follow‐up compared to less activated patients.

Figure 1.

Logic Model: Linking Practice Adoption of Patient Engagement Strategies, Relational Coordination, and Patient‐Reported Outcomes

^* Patient engagement in care is conceptualized as a response to improved patient activation. In the current study, patient engagement is an unmeasured variable.

Methods

Setting

The study includes clinicians, staff, leaders, and patients of two established ACOs—Advocate Health Care (AHC) in Chicago, Illinois, and HealthCare Partners (HCP) in Los Angeles, California—with a strong commitment to use performance feedback from the research to inform the future dissemination of patient engagement strategies among affiliated practices. Each ACO is a large and long‐established health care organization participating in the Medicare Shared Savings Program35 and other risk‐bearing contracts that create incentives to better involve patients in their care to achieve better outcomes and reduce the costs associated with emergency department visits and preventable hospital admissions and readmissions.

Practice Selection

A multistage sampling approach was used to maximize variation in the adoption of patient engagement strategies among selected practices. Based on a literature review and prior research assessing the patient engagement strategies used by ACOs,24 we developed a 39‐item primary care practice survey to assess practice adoption of patient engagement strategies. Clinician leaders (n = 77) of adult primary care practices at one of two large ACOs in the greater Chicago and Los Angeles metropolitan areas were surveyed. Reliability information of the practice survey scales are detailed elsewhere.33

The practice survey included six domains: (1) patient care outreach in regard to disease prevention and health promotion (8 items); (2) changes in the clinician‐patient relationship in the areas of communication, motivational interviewing, and patient involvement in treatment care plans (11 items); (3) shared decision making (12 items); (4) patient self‐management of their condition(s) (1 item); (5) end‐of‐life/advanced serious illness care patient engagement and family involvement (3 items); and (6) patient involvement in the overall design of care and in organization‐wide efforts to improve the quality of care (4 items). Items used Likert scales and were scored as follows: 0 for “none,” 33.3 for “yes, but not regularly,” 67.7 for “yes, partially implemented,” and 100 for “yes, fully implemented.” If practice leaders reported not knowing whether a patient engagement strategy was adopted, we classified their response as “no,” because the strategies entail noticeable changes to practice workflows that impact the roles and responsibilities of care team members.

We calculated the mean of responses to the 39 items for each of the 77 practices to summarize practice‐level adoption of patient engagement strategies. Based on their mean score, practices were rank ordered within their ACO, and the practices were then divided into quartiles. Eight practices were randomly selected from each of the two ACOs: four that scored in the top quartile and four that scored in the bottom quartile of the distribution. Selected practices were categorized as “high” (n = 8) or “low” (n = 8) adoption of patient engagement strategies. Practice surveys were conducted from October to November 2014 (Time 1) and March to July 2016 (Time 2) for the 16 selected practices (response rate [RR] = 100%). Practices had a median of 5.0 primary care physicians, with an overall range of 1 to 17 physicians, and an interquartile range (IQR) of 3.5 to 11.5 physicians.

Patient Sampling

Adult patients were included in the sampling frame for the research study if they (1) received care from 1 of the 16 practices identified for study inclusion (at least one primary care visit to the practice during calendar year 2014); (2) had a diagnosis of diabetes (International Classification of Diseases codes [ICD] 9 250.XX) and/or CVD (ICD‐9 410‐14, 426‐9, or 430‐8); (3) were between 18 and 82 years of age; and (4) spoke English or Spanish for the purposes of completing the patient survey. Patients older than 82 years of age were excluded because the population is more likely to be frail, which requires a more specialized focus as it relates to patient engagement.

Patient Surveys

Two waves of patient surveys were conducted to assess patient activation and PROs of physical, emotional, and social function over time. The surveys were fielded between April and September 2015 (Time 1) and between May and August 2016 (Time 2). Patients were mailed a paper copy of the survey along with a $10 Target gift card. Surveys were mailed in English and Spanish, according to the recipient's language preference as indicated by each ACO's electronic health record data. An additional $10 Target gift card was mailed to patients who completed and returned the survey. Two additional mailings were sent to nonrespondents, along with a reminder postcard. Patients who did not return a mailed survey within six weeks were contacted by phone and given the option of completing a telephone interview in English or Spanish. Up to 10 phone call attempts were made to contact nonrespondent patients. In addition to direct contact of patients, fliers were posted in waiting rooms of each practice, advertising the survey and encouraging patients who received it to respond. A response rate of 51% (n = 2,176) was achieved for the first wave of the patient survey.

Updated patient clinical and administrative data from each ACO were recorded and reviewed to assess whether patients remained eligible for the follow‐up survey. A total of 407 Time 1 survey respondents (18.7%) were excluded from the Time 2 survey sample because they were no longer ACO members or switched practice sites within the ACO. Using the same survey data collection protocol as the Time 1 patient survey, 1,769 patients (eligible respondents to the Time 1 survey) were surveyed. A total of 1,291 returned the Time 2 survey, yielding a 73% response rate. The most common reasons for nonresponse ascertained by phone contacts were time constraints, not wanting to identify as having a “chronic condition,” and dissatisfaction with care. Patients with missing Patient Activation Measure (PAM) scores (n = 15) were excluded from the analysis because patient activation is a mediator of interest. The final analytic sample includes a cohort of 1,276 patients.

Measures

Well‐validated and widely used measures of physical function (PROMIS Short Form 12a), emotional function (PHQ‐4), and social function (PROMIS Short Form 8a) were used.36, 37, 38, 39 The physical function score ($\alpha =0.92$) is the average of responses to 13 questions assessing how often the patient has encountered difficulty with physical activities, each ranging from 1 = unable to do to 5 = no difficulty. The emotional function score ($\alpha =0.88$) is the average of responses to four questions assessing how often the patient experienced problems, each ranging from 1 = nearly every day to 4 = not at all. The social function score ($\alpha =0.96$) is the average of the responses to eight questions assessing how often the patient has encountered difficulty with participating in social roles and activities, each ranging from 1 = always to 5 = never. Patient surveys also assessed patient activation using the 13‐item Patient Activation Measure.16 PAM ($\alpha =0.92$) is scored as the average of responses to positively phrased statements of patient activation, each ranging from 1 = disagree strongly to 4 = agree strongly. PAM measures self‐efficacy for behavior change and provides clinicians with feedback to tailor their work to increase or reinforce patient activation.40

All survey composite measures were scored as continuous measures using the half‐scale rule,41 whereby respondents had to complete at least half of the items comprising the composite measure for a score to be calculated.

Patient Administrative and Clinical Data

Electronic health record and administrative data comprised clinical outcome measures including blood pressure, low‐density lipoprotein cholesterol (LDL‐C), and glycated hemoglobin (HbA1c) values; comorbidity information; and demographic data throughout the study period. All covariates for the patient cohort were used to impute missing values into 10 separate data sets using multiple imputation with chained equations. The resulting values were averaged across the 10 imputed data sets and the resulting averaged values were imputed into the analytic data set in place of missing values.42

As secondary measures, we analyzed patient‐level data on intermediate clinical outcome measures. Healthcare Effectiveness Data Information Set comprehensive diabetes care measure definitions were used to construct dichotomous outcome measures of control: HbA1c ˂ 8.0%, blood pressure ˂ 140/90 mmHg, and LDL‐C ˂ 100 were considered “controlled.”

Clinician and Staff Surveys

All adult primary practice members of the 16 practices were surveyed at Time 1, from January to March 2015, and at Time 2, from January to April 2016. The response rates to the clinician and staff surveys were 86% (n = 353) for Time 1 and 84% (n = 411) for Time 2. The survey assessed relational coordination among adult primary clinicians and staff,43 including primary care clinicians (MD, NP, or PA), nurses (registered nurse [RN] and RN care manager), medical assistants, diabetic educators (RN/health/peer), nutritionists, receptionists, and social workers. These roles were selected because they were the most common adult primary care team members contributing to the care of patients with diabetes and/or CVD within the 16 selected practices. The relational coordination measure ($\alpha =0.87$) includes assessments of shared goals, shared knowledge, and mutual respect, as well as four dimensions of communication (frequent, timely, accurate, problem solving). Practices were dichotomized as having high (n = 8) vs. low (n = 8) relational coordination based on the median Time 1 practice score of 4.1. Five of the eight practices with low adoption of patient engagement strategies had low relational coordination scores and five of eight practices with high adoption of patient engagement strategies had high relational coordination.

Qualitative Data

To qualitatively examine the role of relational coordination in supporting effective implementation of patient engagement strategies, two site visits were conducted involving hour‐long interviews with primary care physicians and staff at each of the 16 ACO‐affiliated practices: once in May 2015 and again in May 2016. A total of 68 clinicians and staff were interviewed; 35 individuals were interviewed in both Time 1 and Time 2, 18 were interviewed in Time 1 only, and 15 were interviewed in Time 2 only for an overall total of 103 interviews. Some interviews (n = 7) were also conducted with physician leaders of each ACO to gain insight about strategic goals of the ACOs related to the use of primary care teams to improve patient engagement. The research team scheduled site visits when patient advisory group meetings were occurring to enable in‐person discussions between the research team and patient advisors. Quarterly patient advisory group meetings at each ACO enabled the research investigators to integrate patient feedback on survey instruments, blinded comparison of primary care practices on key survey questions, and formative study results.

The Institutional Review Board approved a waiver of informed consent for the self‐administered surveys of patients and clinicians. Informant consent was required and obtained for clinician, staff, and patient interviews. All interviews were recorded and transcribed with the interviewee's permission; key informants were assured their individual responses would remain confidential and not attributed to specific practice sites in feedback reports or publications.

Statistical Analyses

For Time 1 and Time 2, patient survey data were linked with clinical and administrative data (level 1) and then merged with practice survey data aggregated to the practice (level 2). To assess potential selection effects and the generalizability of our findings, we examined the patient‐level predictors of patient attrition over the course of a three‐year measurement period (2014‐2016).

Multivariate regression models were used to examine the relative effect of patient‐level predictors of patient attrition, accounting for the clustering of patients within practices using generalized estimating equations (GEE). To address hypothesis 1, we separately examined the association of Time 1 practice‐level adoption of patient engagement strategies (high vs. low) and PROs at Time 1 and Time 2. To address hypothesis 2, we examined the association of Time 1 practice‐level adoption of patient engagement (high vs. low) and PROs. Multivariable linear regression models were estimated to assess these relationships and controlled for patient age, education, sex, comorbidity count, and practice size and accounted for the clustering of patients within practices using GEE. Our general statistical model used to assess differential changes in PROs for practices with high vs. low adoption of patient engagement strategies, PES (hypothesis 1) is:

$$\begin{array}{ccc}\hfill \mathrm{PRO}& =& {\mathrm{b}}_0+{\mathrm{b}}_1\ast \mathrm{PES}+{\mathrm{b}}_2\ast \mathrm{Time}+{\mathrm{b}}_3\ast \mathrm{PES}\ast \mathrm{Time}\hfill \\ & & +{\mathrm{b}}_4\ast \mathrm{age}+{\mathrm{b}}_5\ast \mathrm{education}+{\mathrm{b}}_6\ast \mathrm{sex}\hfill \\ & & +{\mathrm{b}}_7\ast \mathrm{comorbidities}+{\mathrm{b}}_8\ast \mathrm{PAM}+{\mathrm{b}}_9\ast \mathrm{practice}\mathrm{size}+\epsilon \hfill \end{array}$$

where the primary coefficient (b) of interest is b₃ . To assess differential changes in PROs for practices with high vs. low relational coordination, RC (hypothesis 2), the general statistical model was:

$$\begin{array}{ccc}\hfill \mathrm{PRO}& =& {\mathrm{b}}_0+{\mathrm{b}}_1\ast \mathrm{RC}+{\mathrm{b}}_2\ast \mathrm{Time}+{\mathrm{b}}_3\ast \mathrm{RC}\ast \mathrm{Time}+{\mathrm{b}}_4\ast \mathrm{age}\hfill \\ & & +{\mathrm{b}}_5\ast \mathrm{education}+{\mathrm{b}}_6\ast \mathrm{sex}+{\mathrm{b}}_7\ast \mathrm{comorbidities}\hfill \\ & & +{\mathrm{b}}_8\ast \mathrm{PAM}+{\mathrm{b}}_9\ast \mathrm{practice}\mathrm{size}+\epsilon \hfill \end{array}$$

To address our hypotheses related to mediating (hypotheses 3) and direct effects of PAM (hypothesis 4), cross‐lagged panel models were estimated. The path models simultaneously estimate the mediating role of PAM on the relationship between practice adoption of patient engagement strategies and each of the PROs. The path model includes adjustments for patient age, sex, comorbidities, educational attainment, and practice size. Correlated residuals for PAM and each PRO at Time 1 and Time 2 were included. Paths from relational coordination and patient engagement strategies to Time 1 patient activation and PROs at Time 2 were estimated, accounting for the impact of Time 1 PROs on Time 2 PROs. Clustered robust standard errors were used to account for patients sampling within practice sites. Path models were estimated using structural equation model procedure in STATA 14.0 with maximum likelihood estimators that account for missing values.

Qualitative Analyses

To qualitatively assess the connections between implementing patient engagement strategies and relational coordination, we used template coding of qualitative interviews of key informant interviews to analyze variation in implementation of patient engagement strategies and implementation experiences of primary care team members.44 A codebook based on the interview guide was used to code the interview data related to implementing patient engagement strategies. Given that a subset of interview questions used the terms “patient activation” and “patient engagement” and referred to “teams,” two team members used the autocode feature of Atlas.ti to code every instance of the terms “engag^*,” “activat^*,” and “team^*” and associated text across all 103 interview transcripts.45 The keyword search approach was the first step used to identify areas in the transcripts that pertained to our topics of interest. This approach was used in conjunction with the template coding of transcripts using the interview guide to sufficiently capture context.

Clusters (or “families”) of transcripts were generated to classify interview participants of practices with high vs. low adoption of patient engagement strategies. Coding practices were compared and discrepancies were addressed through discussion and clarification at regular research team meetings. A researcher not involved in the initial coding process then identified uncoded instances of patient engagement barriers in the coded transcripts. The third researcher reviewed a subset (approximately 15%) of coded transcripts from each ACO, finding this amount of review to be sufficient for validating the coding scheme used by the two main researchers, as minimal discrepancies and inconsistencies were identified in the third review. Using the final coded data, Atlas.ti analysis features were used to examine implementation experiences and perceptions of team member contributions to patient engagement among primary care clinicians and staff of practices and compare differences and similarities in experiences of key informants of practices with high vs. low adoption of patient engagement strategies.

Results

The average scores for each patient engagement strategy for practices with high and low adoption of strategies for both time periods are summarized in Table 1. Overall, implementation of strategies increased over time for practices with low adoption at Time 1. Practices with high adoption of patient engagement strategies generally did not expand their use over time. Activities that were adopted more over time by practices with low adoption include clinician and staff training and the use of motivational interviewing techniques, telehealth for diabetes and CVD, shared medical appointments for diabetes, programs to improve family participation and support for patients with diabetes, and patient advisory groups. Despite the overall changes observed over time, the categorization of practice adoption of patient engagement strategies as high vs. low was the same at Time 1 and Time 2.

Table 1.

Patient Engagement Strategies Used by ACO‐Affiliated Practices, by High vs. Low Practice Adoption

	All Practices		High‐Adoption Practices		Low‐Adoption Practices
	n = 16		n = 8		n = 8
Patient Engagement Strategy	Time 1 Mean	Time 2 Mean	Time 1 Mean	Time 2 Mean	Time 1 Mean	Time 2 Mean
39‐item patient engagement survey score (range: 0‐100); mean (SD)	54.9 (27.7)	64.5 (21.1)	79.1 (11.9)	76.2 (15.5)	30.6 (12.5)	52.7 (19.9)**
Patient Care Outreach Strategies
1. Conducts a health risk assessment (HRA) survey with patients	75.0	81.3	91.8	95.9	58.3	66.6
2. Provides patients feedback on their HRA results	72.9	79.2	95.9	95.9	50.0	62.5
3. Provides ongoing monitor of HRA results (assessing changes over time)	64.6	79.2	95.9	95.9	33.3	62.5
4. Refers patients to a disease prevention or health promotion program as a result of the HRA	62.4	72.9	95.9	95.9	29.0	50.0
5. Encourages relevant patients to participate in a healthy eating program	81.3	87.5	100.0	95.9	62.6	79.1
6. Encourages relevant patients to participate in a physical activity program	83.4	85.4	95.9	95.9	71.0	75.0
7. Encourages relevant patients to participate in an employee health promotion/prevention/wellness program	72.9	70.9	100.0	87.6	45.9	54.1
8. Sponsors or participates in school health clinic interventions	33.4	41.7	66.8	70.9	0.0	12.5
Patient Communication, Motivational Interviewing, and Involvement in Treatment Care Plans
9. HRA results are available electronically to care team members (through the electronic medical record) at the point of care	62.5	79.3	95.9	91.8	29.1	66.8
10. Clinicians are trained in motivational interviewing techniques	60.4	72.9	91.8	83.4	29.1	62.5*
11. Clinicians consistently use motivational interviewing techniques in communicating with patients (eg, encourage patients to ask questions)	60.4	79.2	91.8	83.4	29.1	75.0*
12. Clinicians consistently encourage patients to discuss their work, home life, and social situation	79.3	79.3	91.8	91.8	66.8	66.8
13. Staff are trained in motivational interviewing techniques	39.6	60.4	70.9	66.8	8.4	54.1*
14. Staff consistently use motivational interviewing techniques in communicating with patients (eg, encourage patients to ask questions)	41.7	66.7*	70.9	70.9	12.5	62.5***
15. Staff note patient preferences for treatment in the patient's record	62.6	70.9	75.1	75.1	50.0	66.8
16. Select staff serve as “health coaches” for patients seeking to modify their lifestyle	47.9	56.3	70.9	62.6	25.0	50.0
17. Patients can routinely provide information on their care and their health via patient portal (not just access)	72.9	81.3	91.8	91.8	54.1	70.9
18. Telehealth is consistently made available to patients with diabetes	37.5	58.4	70.9	62.5	4.1	54.3*
19. Telehealth is consistently made available to patients with cardiovascular disease	43.7	56.3	75.0	62.5	12.4	50.0*
Shared Decision Making
20. Clinicians consistently involve patients in developing treatment goals	75.0	85.5	95.9	91.8	54.1	79.3
21. Clinicians or staff review goal setting for behavioral changes with patients as a result of their HRA	58.3	77.1	95.9	91.8	20.8	62.5*
22. Practice provides eligible patients with shared‐decision‐making videos	20.9	25.0	41.8	29.1	0.0	20.9
23. Physicians consistently have follow‐up discussions with patients regarding their treatment options and preferences	68.8	81.4	95.9	91.8	41.6	71.0*
24. There is a formal evaluation of the impact of shared decision making on patient care choices, outcomes of care, and patient experience with their care	54.2	56.3	87.6	70.9	20.8	41.8
25. There exists an organized follow‐up program to assist patients in managing their medications at home, such as pharmacist‐led medication management	62.5	62.5	95.9	83.4	29.1	41.6
26. Shared medical appointments (group visits) are available for patients with diabetes	41.6	66.7*	79.1	87.5	4.1	45.9*
27. Shared medical appointments (group visits) are available for patients with cardiovascular disease	29.2	35.4	58.4	58.4	0.0	12.5
28. Peer‐to‐peer (patient‐to‐patient) programs are available for patients with diabetes	41.6	56.3	70.9	70.9	12.4	41.8
29. Peer‐to‐peer (patient‐to‐patient) programs are available for patients with cardiovascular disease	29.1	33.3	45.9	45.9	12.4	20.8
30. Programs exist to improve family participation and support for patients with diabetes	58.3	68.8	87.5	87.6	29.1	50.0*
31. Programs exist to improve family participation and support for patients with cardiovascular disease	41.6	54.3	58.3	62.6	25.0	45.9
Patient Self‐Management
32. Provision of at‐home monitoring devices and/or tools to assess medication management, blood pressure, blood sugar, and lipids	70.9	70.8	87.6	75.0	54.1	66.6
End‐of‐Life/Advanced Serious Illness Care Patient Engagement and Family Involvement
33. Clinicians consistently discuss the importance of patient advanced directives	79.3	79.3	87.6	87.6	71.0	71.0
34. Clinicians consistently discuss hospice care options with patients	75.1	79.3	79.3	91.8	71.0	66.8
35. Clinicians consistently discuss the availability of both hospital‐based and community‐based palliative care with patients	77.1	81.3	87.5	91.8	66.8	70.9
Patient Involvement in Design of Care and Organization‐wide Efforts to Improve Quality of Care
36. Patient advisory councils exist for patients with diabetes	22.9	33.4	45.9	37.5	0.0	29.3*
37. Patient advisory councils exist for patients with cardiovascular disease	16.7	29.2	33.4	37.5	0.0	20.9
38. Patients consistently participate in quality improvement teams	31.3	39.7	62.5	54.3	0.0	25.1
39. Patients are involved in helping to govern the clinic or practice	31.2	39.6	50.0	50.1	12.4	29.1

^* p < 0.05, ^** p < 0.01, ^*** p < 0.001. Numbers followed by asterisks show mean scores change statistically significantly from Time 1 (2015) to Time 2 (2016).

Relational coordination was high across practices (mean = 4.09) and mean values did not differ for practices with high vs. low adoption of patient engagement strategies at Time 1. By Time 2, relational coordination scores were higher for practices with high adoption than practices with low adoption (mean = 4.29 vs. 4.03, p < 0.001, data not shown).

Table 2 compares Time 1 patient demographics, PROs, intermediate clinical outcomes, and patient activation for patients of practices with high vs. low adoption of patient engagement strategies. Except for race/ethnicity and systolic blood pressure, Time 1 patient characteristics were similar for patients of practices with high vs. low adoption of patient engagement strategies. Attrition analyses indicate that our cohort includes a higher proportion of older patients and patients with better controlled LDL‐C and HbA1c compared to those who were lost to follow‐up (Appendix 1).

Table 2.

Baseline Characteristics of the Patient Cohort, by High vs. Low Practice Adoption of Patient Engagement Strategies

Key Variables	All Patients	Patients of Practices with High Adoption	Patients of Practices with Low Adoption
Overall patient cohort (n)	1,276	610	666
Patient‐level variables
Age, no. (%)
18‐44	69 (5.41%)	34 (5.57%)	35 (5.26%)
45‐54	150 (11.76%)	68 (11.15%)	82 (12.31%)
55‐64	335 (26.25%)	158 (25.9%)	177 (26.58%)
65‐74	498 (39.03%)	248 (40.66%)	250 (37.54%)
75+	224 (17.55%)	102 (16.72%)	122 (18.32%)
Sex, no. (%)
Female	726 (56.9%)	348 (57.05%)	378 (56.76%)
Male	550 (43.1%)	262 (42.95%)	288 (43.24%)
Spanish‐language survey, no. (%)	230 (18%)	105 (17%)	125 (19%)
Race/Ethnicity, no. (%)
White	538 (42.16%)	199 (32.62%)	339 (50.9%)***
Hispanic/Latino	405 (31.74%)	201 (32.95%)	204 (30.63%)
Black/African American	150 (11.76%)	132 (21.64%)	18 (2.7%)
Other	183 (14.34%)	78 (12.79%)	105 (15.77%)
Patient Activation Measure, mean (SD)	3.27 (0.45)	3.26 (0.46)	3.28 (0.44)
Social function, mean (SD)	3.65 (1.04)	3.65 (1.03)	3.65 (1.05)
Physical function, mean (SD)	3.99 (0.87)	3.98 (0.87)	3.99 (0.87)
Emotional function, mean (SD)	3.49 (0.72)	3.46 (0.73)	3.52 (0.70)
Systolic blood pressure, mean (SD)	131.67 (13.79)	133.08 (14.48)	130.36 (12.99)***
Diastolic blood pressure, mean (SD)	75.52 (8.27)	75.97 (8.73)	75.11 (7.80)
Low‐density lipoprotein cholesterol (LDL‐C), mean (SD)	89.93 (31.82)	90.04 (32.33)	89.82 (31.33)
Glycated hemoglobin (HbA1c), mean (SD)	7.13 (1.45)	7.19 (1.52)	7.06 (1.38)
Practice‐level variables	n = 16	n = 8	n = 8
Relational coordination, mean (SD)	4.1 (0.2)	4.2 (0.2)	4.0 (0.2)
Primary care physicians (FTEs), mean (SD)	6.8 (4.8)	9.6 (5.4)	4.0 (1.3)
All primary care clinicians and staff, mean (SD)	30 (21)	38 (24)	22 (16)
Total patients attributed to physicians in the practice, mean (SD)	8,261 (5,575)	11,223 (5,753)	5,484 (3,775)

Abbreviations: FTE, full‐time equivalent; SD = standard deviation.

^***

p < 0.001

Asterisks in the last column indicate that patients of practices with high adoption of patient engagement strategies are statistically significantly different from patients of practices with low adoption of patient engagement strategies.

Appendix 1.

A Comparison of Patient Characteristics of the Cohort vs. Nonrespondents to Time 2 Survey

	Nonrespondents to Time 2	Patient Cohort (Responded to Time 1 and Time 2)
Baseline Characteristics	(n = 885; 41%)	(n = 1,291; 59%)	p‐value
Age (N = 2,168)
18‐24	7 (0.8%)	6 (0.5%)	<0.0001
25‐34	20 (2.3%)	5 (0.4%)
35‐44	66 (7.5%)	52 (4.0%)
45‐54	124 (14%)	154 (12%)
55‐64	219 (25%)	324 (25%)
65‐74	296 (34%)	502 (39%)
75+	146 (17%)	247 (19%)
Sex
Male	429 (48%)	556 (43%)	0.013
Female	456 (52%)	735 (57%)
Race
White	392 (47%)	635 (49%)	0.3
Black	114 (14%)	153 (12%)
Asian	73 (8.7%)	123 (10%)
Pacific Islander	6 (0.7%)	3 (0.2%)
First Nation	4 (0.5%)	9 (0.7%)
Other	252 (30%)	361 (28%)
Education (N = 2,154)
Eighth grade or less	115 (13%)	153 (12%)	0.7
Some high school	69 (8.0%)	103 (8.0%)
High school	165 (19%)	280 (22%)
Some college	281 (32%)	412 (32%)
College	122 (14%)	176 (14%)
More than college	114 (13%)	164 (13%)
Insurance
Private	411 (46%)	522 (40%)	0.045
Medicaid	17 (1.9%)	27 (2.1%)
Medicare	398 (45%)	638 (49%)
Medicaid and Medicare	59 (6.7%)	104 (8.1%)
Comorbidity count (mean, SD)	5.0 (4.0, 7.0)	5.0 (4.0, 8.0)	0.063
Patient Activation Measure	3.2 (3.0, 3.6)	3.2 (3.0, 3.6)	0.6

Patients with diabetes and/or CVD had stable PRO scores over one year. The median PRO score changes were 0.0 points (IQR = −0.3, 0.2), 0.0 points (IQR = −0.5, 0.4), and 0.0 points (IQR = −0.2, 0.2) for physical, social, and emotional function, respectively (data not shown).

There were no mean differences in PROs and PAM between practices with high vs. low adoption of patient engagement strategies at either point in time, and there was no differential change over time in PROs and PAM for practices with high vs. low adoption (Table 3). Regarding intermediate clinical outcomes, there were no statistically significant differences in LDL‐C, HbA1c, or diastolic blood pressure levels for patients of practices with high vs. low adoption of patient engagement strategies, and there were no differential changes between patients of practices with high vs. low on any intermediate clinical outcome measure over time. Compared to patients of practices with low adoption of patient engagement strategies, patients of practices with high adoption had slightly higher mean systolic blood pressure at both Time 1 and Time 2, but there were no differential changes over time.

Table 3.

Adjusted Comparison of Patient‐Reported and Clinical Outcome Changes Over Time (n = 1,276), by Practices with High vs. Low Adoption of Patient Engagement Strategies

	Time 1				Time 2				Difference‐in‐Difference
	Overall	High PES	Low PES	Dif	Overall	High PES	Low PES	Dif
Measure	Mean (SE)	Mean (SE)	Mean (SE)	p‐value	Mean (SE)	Mean (SE)	Mean (SE)	p‐value	Coefficient (p‐value)
Patient‐Reported Outcomes
Physical function	4.24 (0.05)	4.28 (0.07)	4.21 (0.06)	0.4	4.19 (0.05)	4.22 (0.07)	4.16 (0.06)	0.5	−0.01 (0.9)
Emotional function	3.59 (0.03)	3.58 (0.04)	3.60 (0.04)	0.7	3.59 (0.03)	3.55 (0.04)	3.62 (0.04)	0.15	−0.06 (0.3)
Social function	3.90 (0.05)	3.96 (0.08)	3.86 (0.07)	0.2	3.87 (0.05)	3.93 (0.08)	3.83 (0.07)	0.4	0.00 (1)
Intermediate Clinical Outcomes
Low‐density lipoprotein cholesterol	97.13 (1.44)	97.81 (1.89)	96.58 (1.78)	0.4	93.76 (1.34)	93.50 (1.85)	94.08 (1.61)	0.6	−1.80 (0.5)
Glycated hemoglobin (%)	7.13 (0.08)	7.27 (0.11)	7.04 (0.10)	0.072	7.23 (0.08)	7.31 (0.11)	7.18 (0.10)	0.5	−0.10 (0.4)
Systolic blood pressure (mm Hg)	131.09 (0.87)	133.30 (1.12)	129.69 (0.94)	0.010	130.58 (0.87)	133.04 (1.12)	128.94 (0.95)	0.002	0.49 (0.6)
Diastolic blood pressure (mm Hg)	76.76 (0.48)	77.27 (0.71)	76.44 (0.59)	0.3	75.61 (0.48)	76.11 (0.71)	75.29 (0.59)	0.5	−0.01 (1)
Patient Activation Measure	3.33 (0.02)	3.33 (0.03)	3.34 (0.02)	0.9	3.35 (0.02)	3.34 (0.02)	3.35 (0.02)	0.5	−0.00 (1)

Abbreviations: High PES, practices with high adoption of patient engagement strategies; Low PES, practices with low adoption of patient engagement strategies; SE, standard error.

There are no statistically significant differences between patients of practices with high vs. low adoption of patient engagement strategies over time.

Multivariable regression analyses indicate that there were no mean differences in PROs and PAM between practices with high vs. low relational coordination at either point in time and there was no differential change over time in PROs and PAM for practices with high vs. low relational coordination (Table 4). Regarding intermediate clinical outcomes, only HbA1c at Time 1 (7.2% vs. 7.0%, p = 0.01) was different for patients of practices with high vs. low relational coordination. There were no differential changes between patients of practices with high vs. low relational coordination on intermediate clinical outcome measures over time.

Table 4.

Adjusted Comparison of Patient‐Reported and Clinical Outcome Changes Over Time (n = 1,276), by Practices with High vs. Low Relational Coordination

	Time 1				Time 2				Difference‐in‐Difference
	Overall	High RC	Low RC	Dif	Overall	High RC	Low RC	Dif
Measure	Mean (SE)	Mean (SE)	Mean (SE)	p‐value	Mean (SE)	Mean (SE)	Mean (SE)	p‐value	Coefficient (p‐value)
Patient‐Reported Outcomes
Physical function	4.24 (0.05)	4.29 (0.06)	4.19 (0.06)	0.2	4.19 (0.05)	4.20 (0.06)	4.17 (0.06)	0.7	−0.06 (0.3)
Emotional function	3.59 (0.03)	3.62 (0.04)	3.56 (0.04)	0.2	3.59 (0.03)	3.59 (0.04)	3.60 (0.04)	0.8	−0.06 (0.2)
Social function	3.90 (0.05)	3.94 (0.07)	3.87 (0.07)	0.3	3.87 (0.05)	3.87 (0.07)	3.87 (0.07)	1	−0.07 (0.3)
Intermediate Clinical Outcomes
Low‐density lipoprotein cholesterol	97.13 (1.44)	98.24 (1.73)	95.97 (1.88)	0.2	93.76 (1.34)	93.54 (1.64)	93.98 (1.71)	0.8	−2.72 (0.3)
Glycated hemoglobin (%)	7.13 (0.08)	7.26 (0.10)	7.00 (0.10)	0.01	7.23 (0.08)	7.27 (0.10)	7.18 (0.10)	0.6	−0.18 (0.14)
Systolic blood pressure (mm Hg)	131.09 (0.87)	131.28 (1.14)	130.90 (1.15)	0.7	130.58 (0.87)	130.39 (1.14)	130.78 (1.15)	0.8	−0.77 (0.5)
Diastolic blood pressure (mm Hg)	76.76 (0.48)	76.74 (0.62)	76.79 (0.63)	1	75.61 (0.48)	75.34 (0.62)	75.89 (0.63)	0.4	−0.49 (0.4)
Patient Activation Measure	3.33 (0.02)	3.33 (0.02)	3.34 (0.02)	0.5	3.35 (0.02)	3.34 (0.02)	3.36 (0.02)	0.4	−0.01 (0.8)

Abbreviations: High RC, practices with relatively high relational coordination; Low RC, practices with relatively low relational coordination; SE, standard error.

There are no statistically significant differences between patients of practices with low vs. high relational coordination over time.

Path analyses confirmed that there were no significant associations between (1) high practice adoption of patient engagement strategies and patient activation scores and (2) high relational coordination and patient activation scores. Patient activation at Time 1 did not have a mediating role; instead, patient activation at Time 1 had direct effects on each of the three PROs at Time 2 (Figure 2). The strongest direct effect of patient activation at Time 1 was on social function at Time 2 (β = 0.26, p < 0.001), while patient activation had more modest effects on physical function (β = 0.09, p < 0.05) and emotional function (β = 0.10, p < 0.01) at Time 2.

Figure 2.

Cross‐Lagged Panel Model Results: Patient Activation Impacts Patient‐Reported Outcomes Over Time

^* p < 0.05, ^** p <0.01, ^*** p < 0.001

The panel model includes adjustments for patient age, sex, comorbidities, education, and practice size (not shown). Correlated residuals for Patient Activation Measure scores and each patient‐reported outcome (PRO) at Time 1 and Time 2 are also included. Paths from relational coordination and patient engagement strategies to PROs were estimated but not statistically significant and therefore are not depicted, for legibility.

Qualitative analyses revealed that clinician and staff key informants from practices with high vs. low adoption of patient engagement strategies differed in their perception of team boundaries and relationships, experiences of managing low patient activation as a barrier to implementing patient engagement strategies, and experiences of implementation under time and resource constraints. Appendix 2 summarizes the key differences in implementation experiences of practices with high vs. low adoption using illustrative quotes from care team members. Importantly, key informants of practices with high adoption more often described implementing intensive approaches, such as shared medical appointments and shared decision making, as continuous improvement processes with expected implementation challenges to be overcome. They were more likely to internalize responsibility for supporting patient engagement compared to key informants of practices with low adoption, who tended to repeatedly blame low patient activation and limited time and resources as primary reasons why patient engagement strategies were not possible to implement.

Appendix 2.

Key Informant Experiences of Relational Coordination and Implementing Patient Activation and Engagement Activities

	Practices with High Adoption of Patient Engagement Strategies	Practices with Low Adoption of Patient Engagement Strategies
Perception of team boundaries and relationships	Working relationships were generally described as high functioning. Frequently referred to bounded teams or formal/high‐frequency interactions. Teams often work together to address a range of social issues (transportation, finances, social support, etc.). Team members are connected through shared medical appointments and other PAE activities. Example Quotes: PCP: “Patients’ priorities are often completely different from the medical professions’ priorities, in a way. So we took it to another level, which is why we hired a patient coach as part of our care teams here. And we also got a social worker that's on‐site for them. And we also have a case manager for them on‐site. So if they have issues running the systems from referrals to hospitalizations, getting notes from the hospital? Or somebody they want to just cry with you because they're lonely because their husband passed away or something. So that's all on staff here.” MA: “ It's all about teamwork—even though some of the MAs in the back office might not be assigned to a particular doctor, but if I have a problem or something I can't figure out and I let them know, they'll come and they'll help me. So we all work as a team.” PCP: “I think what would explain our clinic's performance is staff engagement, having proactive staff, having enthusiastic staff, staff that can motivate the patients, staff that can encourage the patients and then get them interested in controlling their diabetes and their clinical conditions better.”	Working relationships were generally described as high functioning. Inconsistent understanding of care team member boundaries, including whether or not the following people and processes were considered inside or outside of the care team: Phone follow‐up by nurses and medical assistants Family members as extended care team members Referrals to related services as team activities Only one mention of addressing nonmedical needs. Some concerns about low physician acceptance of expanded roles for medical assistants. Example Quotes: Nurse: “A couple of years ago, when we developed these teams we were really strict about it and we were working really well, and then we got this centralized call center, and it was just too hard for them to determine to make the appointment and who goes on what team and where, and so it kind of got swept underneath the carpet a little bit.” PCP: “You know, I've tried, and there has been a lot of resistance to expanding MA roles. MAs are not comfortable putting or appending orders for the provider. I think it would be so helpful if they could, but they're just not comfortable with it.”
Low patient activation as a barrier to PAE implementation	Mentioned low patient activation when asked to name challenges to implementing patient engagement strategies, but generally not when discussing other interview topics. Compared to low adoption practices, high adoption practice key informants more frequently described shared decision making and addressing motivation as strategies to overcome low patient adherence. Example Quotes: PCP: “And lately, he got hospitalized for very poorly controlled diabetes. He started opening up about how many family members had bad experiences with doctors, which is why he never really wanted to see a doctor. And he started talking about the struggles in his life, stress at his home, his upcoming divorce, his disappointments in life, and after talking to me about this then he started listening to me about what we could do to help him. And now he's doing everything; he even joined my weight loss clinic.” PCP: “So if we have a really complicated patient, they'll send me messages and I'll review treatments and then we'll refer the patient to Health Enhancement, and then we'll send them to our care management department where they manage home referrals like if they have diabetic needs or diabetic supplies that need to be sent or just anything like that—that's our care management department. Then we do Thursday calls.… All these high‐risk patients and we're routinely checking up on them, making sure that they don't have any questions or any concerns that we might need to address.”	Often mentioned low patient engagement as a challenge when discussing other topics in the interview, which was more pervasive compared to high PAE practices. Infrequently mentioned low engagement in connection with financial, social, or cultural barriers faced by patients. Example Quotes: MA: “We call them and tell them, ‘We can send you the lab requisition so you can go to the lab,’ yet they still don't go. So it's those kind of situations that makes it a little difficult for us to be able to track their care and see what exactly it is that they need, because we're providing them with the resources. We're giving them the options but they're just not taking them or taking them for whatever reason.” Nurse: “I had to say, ‘Look, you're 30 years old, you're a diabetic, you have all of these problems and if you don't stop, if you don't start getting this under control, your kidneys are going to fail. Is that what you want?’ And then after that, then she finally started doing her meds correctly and being a little more compliant.”
Implementing PAE under time and resource constraints	Generally recognized constraints, but were more likely to internalize responsibility for supporting patient engagement. Example Quotes: PCP: “I run the weight loss clinic here … and, we really need to expand. We really need to grow more. Because one of the biggest factors that causes diabetes is obesity.” PCP: “We bust our butts to make sure that they're getting what they want and what they need. We're adding more resources. We're trying our best to reach out to them and give them what they need to have that continuity of care and be satisfied with their care. That's definitely something that we do here.” Nurse: “Everything that I do in the morning [huddle] is to improve access right now. It's a knife in my heart. (46:46).”	Had more limited exposure to PAE, but indicated resource constraints and time as the reasons PAE implementation was low. Example Quotes: PCP: “I really felt I had all these new tools to break through resistance and everything. In truth when you're seeing patients every 15 to 30 minutes, it was not practice.” Nurse: “They're expecting to have them do an additional 100 things, so it's not so easy to care about the patients’ outcomes when you are expected to do so much.”

Abbreviations: MA, medical assistant; PAE, patient activation and engagement; PCP, primary care physician

Key informants also discussed the importance of team member contributions to improving patient engagement, including what medical assistants and nurses do to prepare patients for encounters through agenda setting to prioritize discussion with clinicians and providing intensive phone follow‐up to patients not receiving recommended care. Clinicians of practices with low adoption of patient engagement strategies, however, were more likely to indicate the challenges being related to patients themselves and frequently did not acknowledge medical assistant or nurse contributions to implementation. Compared to clinicians of practices with high adoption, clinicians of practices with low adoption more frequently expressed concerns and skepticism about expanded roles for medical assistants in support of patient engagement.

Key informants at all practices described the focus of shared decision making to be on improving clinical metrics directly targeted by the ACOs through performance‐based financial incentive programs. Clinicians from both high‐ and low‐adoption practices sometimes misunderstood what specific patient engagement strategies entailed. For example, some indicated that shared decision making for diabetes and CVD treatment was accomplished by occasionally handing out pamphlets to patients to consider the pros and cons of their treatment decisions at home. Moreover, when efforts were made to engage patients with written information, follow‐up was infrequently described.46

Both ACOs provided routine motivational interview training to aid in shared decision making. One ACO provided a two‐day on‐site motivational interviewing training session for all primary care teams and offered ongoing web‐based refresher courses on the topic, while the other ACO provided a half‐day motivational interviewing training as part of an ongoing performance improvement initiative. Clinicians and staff sometimes noted that the communication skills imparted in the trainings were impractical to implement without additional organizational support. For example, communication techniques such as goal setting were perceived by clinicians to be challenging to implement in routine clinical interactions without an organized system to systematically track treatment preferences and goals.

As part of the research study, practice leaders and administrators were provided feedback reports that summarized scores on patient survey and clinician/staff survey measures at two points in time and provided recommendations for supporting the implementation of patient engagement strategies. ACO leaders confirmed the dissemination of feedback reports to clinicians and staff. The Time 2 team survey results, however, indicate that only 32% of adult primary care team members recalled receiving feedback and recommendations from the research study, and key informant interviews confirmed that performance feedback data were not generally shared with care team members, highlighting the important role that middle managers play in sharing performance data and supporting the implementation of disruptive innovations into routine practice.47

Discussion

Although more activated patients have better PROs of physical, social, and emotional function over time, our findings indicate that practice efforts to better engage patients are difficult to implement without a robust system that extends beyond clinical measures to support patient‐centered goal setting. Simply doing more activities appears not to be related to higher patient activation or better PROs among patients with chronic conditions without decision support for clinicians working with patients to set and achieve goals related to PROs and broader life goals that may be achieved through better engagement and self‐management. Evidence indicates that the most effective strategies clinicians can use to improve patient activation and support behavioral changes are very interactive strategies that require dedicated clinician time and strong organizational support of their routine use in primary care encounters.48

Contrary to hypotheses 1 and 2, patients of practices with relatively high adoption of patient engagement strategies and/or relational coordination did not report better PROs compared to patients of practices with relatively low adoption of patient engagement strategies and/or low relational coordination at Time 1 or Time 2. Importantly, clinical outcomes were largely no different for patients of practices with high vs. low adoption of patient engagement strategies. The one‐ to two‐point difference in systolic blood pressure we found between patients of practices with high and low adoption was statistically significant, but does not represent a clinically meaningful difference. Instead, variation on the PRO measures is largely attributable to patient characteristics, particularly how activated they are.

Given the lack of associations found when testing hypotheses 1 and 2, contrary to hypothesis 3, patient activation did not mediate the relationships. The positive relationship of PAM and each of the three PROs over time, however, supports hypothesis 4 and adds to the existing evidence base about the importance of PAM on patient outcomes over time.12, 13, 15, 18, 19 The strong pathway between patient activation at Time 1 and social function at Time 2 is noteworthy. The social function PRO measure assesses patient satisfaction with social roles such as work and family responsibilities, as well as more discretionary social activities such as leisure activity and relationships with friends; improving patient activation may improve social connectedness, which is largely outside the control of primary care practices,49 but addressing social participation can aid self‐management of chronic conditions.50

Our mixed‐method approach provides insight into the lack of expected relationships between practice adoption of patient engagement strategies, relational coordination, and PROs. Mean relational coordination scores were generally high and not significantly different for practices with high vs. low adoption of patient engagement strategies at Time 1. By Time 2, practices with high adoption of patient engagement strategies improved relational coordination, highlighting that improving task coordination among team members is foundational to implementing patient engagement strategies as part of routine care. Indeed, the key informants from practices with high adoption highlighted the close connection between relationships between care team members and efforts to engage adult patients with diabetes and/or CVD in their own health and health care. Implementing patient engagement strategies may improve relational coordination among primary care team members because implementation of innovations requires problem solving and adaptation, which are facilitated by effective team communication and coordination. Relational coordination also supports the reorganization of team member roles and responsibilities to patient activation and engagement.

Implementing the disruptive practice changes to support patient engagement requires increased task interdependence among primary care team members that needs to be carefully managed. The lack of change in PROs over time and the lack of association of high practice–level adoption of patient engagements strategies and better PROs highlight the challenges primary care teams face as they continue to be incentivized by ACOs and other risk‐based contracts to improve patient outcomes in the short run. Even though most ACOs have a strong commitment to improving patient engagement to achieve improved outcomes and patient experience at reduced costs, primary care teams are challenged with balancing quality, access, and cost goals that can sometimes be at odds with one another. Other organizational goals, such as improving clinical performance measures, were thought to crowd out the adoption and implementation of patient engagement strategies. As a result, primary care clinicians and staff had a superficial understanding of what patient engagement entailed, and those who were aware of specific strategies indicated that they inconsistently applied the approaches as part of routine care.46

Both ACOs had strong incentives for practices to have their patients meet biomedical targets, such as having blood pressure or blood sugar levels under control, which may contribute to less engagement with patients to improve areas that matter most to them. PROs, by contrast, continue to be underappreciated and are rarely systematically collected to support patient engagement and chronic care management. ACOs and other organizations incentivizing value‐based care may need to make the investment in collecting and monitoring PRO data to ensure that patient engagement strategies have their intended effect on physical, emotional, and social outcomes that sometimes matter more to patients and families than intermediate clinical outcomes incentivized by ACOs.

While this intensive mixed‐method practice‐based research study has strengths, the results should be considered in the context of some limitations. The study results may not generalize to other diabetes or CVD patient populations, as they are limited to English‐ and Spanish‐speaking patients receiving care from two ACOs selected for study. The analytic sample included a higher proportion of older adults and patients with better HbA1c control compared to patients excluded due to nonresponse and attrition. To address potential selection bias, regression analyses adjusted for a range of patient characteristics. Another limitation is that a single leader at each practice completed the practice survey, interrater reliability of the survey was not assessed, and we are unable to independently validate each response. It is possible that practice leaders overreported their level of adoption and implementation. The results may also not generalize to safety‐net settings, which have different resources and face different incentives. Examining patient activation and engagement in safety‐net settings is an important area for future inquiry. Importantly, the relatively small degree of change between Time 1 and Time 2 in the PRO measures, PAM, and relational coordination combined with the imperfect alignment of the various surveys across time limits our ability to draw stronger inferences by leveraging the naturally occurring variation in change over time between the two time periods. It also may be that a longer period than one year is required to detect effects on PROs because even if practices made progress in implementing patient engagement strategies, patients may not be exposed to a strategy for months into the year.

Our cohort study was designed to take advantage of naturally occurring variation in implementation of patient engagement strategies in the two study sites as opposed to studying a specific intervention. Although we provided feedback reports to ACO leaders and patient advisers, there was no standardized protocol in place at either ACO to ensure that leaders and care teams acted upon the feedback reports and recommendations. Studying the implementation of a common bundle of patient engagement strategies across practices might better elucidate the relationships of interest.

Conclusions

ACOs have largely not incentivized improved PROs among patients with chronic conditions, but the urgent need to reduce high‐cost utilization will most likely move PROs to the forefront of ACOs’ performance measurement and improvement initiatives. Initiatives that aim to incentivize performance on PROs should not expect large one‐year improvements, as ACO‐affiliated practices with substantial investment in adopting and implementing patient engagement strategies did not improve PROs over the study period. Taken together, our findings indicate that practices need resources, facilitation, and time to effectively implement patient engagement strategies. Recent evidence from the VA Healthcare System supports this conclusion.51 In the VA, updating data analytics, enhancing organization‐wide processes and procedures, managing staff time commitments, cultivating staff collaborations, and addressing patient care issues such as access, customer service, and patient education were found to be central to overcoming barriers to patient engagement. Like other disruptive changes to primary care practice, including implementing components of the patient‐centered medical home model,52 our findings highlight the importance of providing robust implementation support and the need to develop practice capabilities to implement patient engagement strategies to ensure that undertaking these major changes translates into benefits for patients, clinicians, and staff.

High relational coordination among team members may be foundational to chronic care management but is insufficient for improving patient activation and engagement. Improving PROs in the longer term may require stronger organizational support specifically for shared decision making, goal setting, and robust follow‐up to engage patients in their own care. Routine and structured dissemination of PRO scores for physical, emotional, and social health may enable care team members to see the value of collecting PROs. A longer timeline, however, is required for care teams to implement patient engagement strategies and to improve patient activation and PROs.