Conceptualizing success factors for patient engagement in patient medical homes: a cross-sectional survey

Abstract

Background:

Patient engagement is a priority for health care quality improvement and health system design, but many organizations struggle to engage patients meaningfully. We describe patient engagement activities and success factors that influence organizational decision-making in Ontario’s patient medical homes.

Methods:

From March to May 2018, we conducted an online survey focused on practice-level patient engagement that targeted primary care organization leaders at all Ontario family health teams, community health centres, nurse practitioner–led clinics and Aboriginal Health Access Centres. We asked questions from the Measuring Organizational Readiness for Engagement (MORE) and Public and Patient Engagement Evaluation Tool (PPEET) questionnaires. We used factor and mediation analysis to identify organizational conditions and activities that are associated with the outcomes of patient engagement, affecting board decisions, program-level decisions and the formation of collaborative partnerships.

Results:

We achieved a 53% response rate (n = 149/283); after removing missing data, our final sample size was 141 respondents. Most respondents perceived that their organization’s patient engagement activities and resources were insufficient. Processes that had a direct effect on outcomes (β = 0.7, p < 0.0001) included planning, training and supporting employees; identifying, recruiting and supporting relevant patients; and using leaders. Structures — including an organizational mission and vision for patient engagement, and policies, procedures, job positions, training programs and organizational culture that reflect that mission — indirectly affected outcomes, mediated by the aforementioned processes (β = 0.7, p < 0.0001).

Interpretation:

Based on the perceptions of primary care leaders, organizational structures and processes are related to successful patient engagement. Organizations that seek to improve patient engagement should assess their commitment and follow-through with associated resources and activities.

Patient engagement is a process of working “together to promote and support active patient and public involvement in health and health care” to strengthen the influence of patients on health care decisions at the individual and collective levels.^1,2 A growing body of evidence suggests that people who are involved in their health care experience improved satisfaction and better health outcomes; ³ patient involvement may also contribute to lower health care costs,³ highlighting the potential “triple aim” effect of engagement. Furthermore, patient engagement in priority settings for health care improvement has the potential to heighten focus on community health needs.⁴

Several initiatives across Canada have demonstrated not only the feasibility of including patient engagement in quality improvement and health system redesign, but also the contribution of patient engagement to successful program implementation and sustainability.^5–7 However, despite a growing body of research and health policy, the uptake of patient engagement activities by health care providers has been slow.⁸ Engagement at the collective level (in organizational design, governance and policy-making) remains a challenge in health care.² The reasons for this include difficulties in redesigning decision-making processes; recognizing and mitigating differentials in power and privilege; and reallocating time and other resources that may not have been built into the existing infrastructure and culture of health care.^2,4,6–11 Moreover, the field is only beginning to share lessons learned about the process of patient engagement.^12–15

Patient engagement and person-centred care are critical tenets of the patient medical home, a model of primary care practice that is being increasingly adopted.^16–18 Patient medical homes aim to offer accessible, comprehensive, continuous, coordinated care, where the patient’s values and preferences guide care practices.¹⁹ More specifically, in the patient medical home context, the pillar of patient- and family-partnered care focuses on achieving the patient’s own goals and responding to their preferences through shared decision-making; support for self-care; patient access to their own health information; convenient care options beyond the traditional office visit; including family caregivers as desired by the patient; and soliciting and using patient feedback to improve care at the practice level.^17,20,21

Despite substantial investments in primary care reform in Ontario over the past several decades, little is known about the extent to which patient engagement has been realized. Moreover, little has been published about how to enable patient engagement at the practice and organizational levels.²² In this study (as part of a broader program of work amplifying patient perspectives on quality improvement opportunities in primary mental health care and collaborative mental health care), we sought to understand the current extent of patient engagement in Ontario’s primary care organizations. We also sought to describe factors that may be associated with successful patient engagement in these organizations. We defined these factors as patient engagement activities that influenced decisions at the program or board level, or that influenced the organizational development of partnerships.

Methods

Study design and setting

Four models of team-based primary care have emerged in Ontario that exemplify the patient medical home concept: family health teams, community health centres, nurse practitioner–led clinics and Aboriginal Health Access Centres. Between March and May 2018, we conducted a survey of executive directors of these organizations to determine their perceptions of their organization’s willingness and ability to engage patients to inform practices and decision-making.

Participants and recruitment

Our recruitment included the executive directors of all 185 family health teams, 73 community health centres, 15 nurse practitioner–led clinics and 10 Aboriginal Health Access Centres in Ontario, using organization names and executive director contact information provided by research team members at the Association of Family Health Teams of Ontario (AFHTO) and the Alliance for Healthier Communities (the Alliance).

In March 2018, research team members at AFHTO and the Alliance sent a link for the online survey to potential participants by email (J.R., C.M.).

We used the Dillman method (modified for virtual communications; the original method was developed for mail communications) to advertise the survey and follow up with potential respondents to maximize recruitment (for the survey advertisement, see Appendix 1 available at www.cmajopen.ca/content/9/4/E1159/suppl/DC1).²³ One week after recruitment began (week 1), researchers sent a follow-up email to all potential respondents. At weeks 3 to 4, researchers telephoned those who had not yet responded. At week 7, researchers sent a final email invitation to nonrespondents. Recruitment closed at week 9.

Survey design and conceptual model

The aim of the survey was to measure the extent to which organizations were willing and able to foster patient engagement in health care, as defined by Coulter.¹

We developed a 49-item survey by selecting questions from the Measuring Organizational Readiness for Patient Engagement (MORE) and Public and Patient Engagement Evaluation Tool (PPEET) instruments that were directly relevant to understanding organizations’ patient engagement activities and their impact (N.S., A.I., V.T., J.R., C.M.).^24–26 We drew on empirical tools that have undergone rigorous processes for development and testing.^26–28 All questions (apart from demographic questions) were drawn from the MORE and PPEET instruments, and were chosen because they captured the 3 major domains of Donabedian’s conceptual framework for quality of health care,²⁹ in which organizational and health system conditions (structures) determine how care is delivered (processes), which in turn affects clinical outcomes, patient experience and health service utilization (outcomes). Questions from MORE and PPEET that were not directly related to these constructs were excluded from developed survey. See Figure 1 for an overview of survey content and Appendix 2, available at www.cmajopen.ca/content/9/4/E1159/suppl/DC1, for the entire survey. All answers were measured using a Likert scale.

Figure 1:

Survey content organized according to Donabedian’s conceptual framework.²⁹

The introductory page of the survey gave respondents information about the purpose of the study; how their data would be used and anonymized; the voluntary nature of participation; and the incentive to be entered in a lottery to win a $50 gift certificate. Respondents then confirmed their consent to participate and began the survey. Participants had the option to provide their name and contact information for the gift certificate lottery; this information was not linked to survey responses, was kept confidential among the research team involved in administering the gift certificate lottery (J.R., C.M., V.T., N.S.) and was discarded after data collection had been completed. We also collected characteristics of the participating organizations.

All questions were presented in an online questionnaire that consisted of 14 pages, each containing 1 to 6 items for response. All survey responses were automatically captured using survey software (SurveyMonkey.com) and exported for analysis. The research team tested the survey before distribution to ensure that it presented as intended, and that it was user-friendly (N.S., V.T.).

Outcomes

The outcomes of interest were patient engagement activities that influenced decisions at the board level; that influenced decisions at the program level; and that resulted in the development of collaborative partnerships. The structures of interest included supportive strategic documents, policies, job descriptions and organizational culture. The processes of interest included the identification, recruitment, training and support of relevant patients, and the allocation of time. We hypothesized that teams that had the structures of interest in place would be more likely to engage in the processes of interest, and thus would be more likely to report attaining the 3 outcomes of interest.

Statistical analysis

First, we assessed and managed the quality of our data (N.S., A.I., A.A.). As a quality-control measure and to ensure the reliability and validity of survey responses, we reduced our data set to questionnaires with greater than 85% completion. A midpoint histogram for survey response rate is provided in the appendices that shows a natural division between response rates above and below 85% (Appendix 3, available at www.cmajopen.ca/content/9/4/E1159/suppl/DC1). In addition, the accepted missing rate in published psychological and educational studies is commonly up to 20%.³⁰ Some records had a modest amount of missing data (< 15%), which we assumed to be missing at random, based on similarities in the descriptive data between records with and without missing data (Appendix 4, available at www.cmajopen.ca/content/9/4/E1159/suppl/DC1). We used single imputation for missing data, employing a proportional odds model for ordinal variables.^29,31

Next, we organized survey questions into categories according to whether the individual questions best represented structures, processes or outcomes, as per Donabedian’s conceptual model (Figure 1).²⁹ We then used confirmatory factor analysis to derive 3 latent composite scores (factors for structure, process and outcome).^32–34 All structure, process and outcome composite scores were standardized for a mean of zero and a standard deviation of 1.

Finally, we explored the relationships between structures, processes and outcomes. We investigated the correlation between these factors using a Pearson correlation matrix. We used a 3-dimensional scatter plot to confirm the linear relationship between the factors (Appendix 5, available at www.cmajopen.ca/content/9/4/E1159/suppl/DC1). Using mediation analysis, we assessed the direct, mediated (i.e., indirect) and total (i.e., direct + mediated) effects of structures and processes on outcomes.³⁵ We interpreted the direct effect as the main effect of structures and processes on outcomes, adjusting for each other. We interpreted the mediation effect of structures on outcomes through processes as a × b, where a was the main effect of structures on processes and b was the main effect of processes on outcomes. We interpreted the total effect as the sum of the direct and mediation effects.^36,37

We assessed the moderation (i.e., interaction) effect between structures and processes on outcomes using a general linear model. We inferred the statistical significance of all direct, mediation, total and interaction effect sizes using White–Huber standard errors.^38,39 We performed all statistical analyses using R version 3.5.2 (www.R-project.org), and we used the lavaan R package to fit the confirmatory factor analysis models.⁴⁰

Research team members with access to membership databases of the Alliance (J.R.) and AFHTO (C.M.) obtained available data from these 2 databases and provided summary statistics to the authors. We visually inspected these descriptive statistics to compare respondents with the broader field of Ontario patient medical homes; we did not conduct formal statistical tests of comparison.

Ethics approval

The study was approved by the St. Michael’s Hospital research ethics board (protocol #18–061).

Results

Of the 283 eligible organizations surveyed, 149 responded — equivalent to a 53% response rate. Of these, we analyzed the responses from 141 organizations, and excluded the remaining 8 responses because less than 85% of the survey had been completed. The characteristics of the respondent organizations are detailed in Table 1. Comparison with databases from the AFHTO and the Alliance suggested that these characteristics were consistent with those of the broader pool sampled with respect to urban versus rural location, but that participation was higher among community health centres than among family health teams.

Table 1:

Characteristics of respondent organizations (n = 141) and all Ontario patient medical homes^*

Characteristic	No. (%) of survey respondents	No. (%) of all Ontario patient medical homes*†
Organization type	n = 136	n = 283
Family health team	62 (45.6)	185 (65.4)
Community health centre	61 (44.9)	73 (25.8)
Nurse practitioner–led clinic	10 (7.4)	15 (5.3)
Aboriginal Health Access Centre	3 (2.2)	10 (3.6)
Organization affiliation	n = 136	n = 283
Academic and teaching centre	36 (26.5)	Unknown
Teaching centre only	50 (36.8)	Unknown
Nonteaching centre	50 (36.8)	Unknown
Organization location	n = 137	n = 272
Urban	77 (56.2)	171 (60.6)
Rural	51 (37.2)	101 (35.8)
Other	9 (6.6)	0 (0)
Other characteristics, median (IQR)
Years in operation	11 (17) [n = 129]	Unknown
Number of patients served	8000 (12 675) [n = 132]	Unknown
Number of primary care providers	10 (10) [n = 133]	Unknown
Number of interprofessional staff	15 (18) [n = 132]	Unknown
Ratio of patients to staff	333.3 (3448.9) [n = 129]	Unknown

Note: IQR = interquartile range.

^*Includes all Ontario family health teams, community health centres, nurse practitioner–led clinics and Aboriginal Health Access Centres.

^†Data from the databases of the Association of Family Health Teams of Ontario and the Alliance for Healthier Communities.

Patient engagement activities

Respondents agreed that their primary care organizations engaged in patient engagement activities, including the following: committing to patient engagement in key organizational documents; seeking patient input in quality improvement initiatives; having strategies to identify and recruit patients for engagement initiatives; ensuring patient input informed service planning and decision-making; and developing collaborative partnerships as a result of patient engagement initiatives (median response at least 4 out of 5, “agree”). However, respondents perceived that their primary care organizations had a fairly low ability to include patient engagement in all activities related to the design of health care services (median response equal to 2 out of 4, “somewhat able”). Overall, 65% of respondents perceived that their organization’s level of engagement activity was not enough (34% just right, 1% too much), and 75% of respondents perceived that their organization’s resources dedicated to engagement activities were not enough (25% just right, 0% too much).

Respondents varied most in whether they perceived that their organizations were committed to patient engagement in key organizational documents and job descriptions; allocated time and resources for patient engagement; sent reports on patient engagement to predetermined users; and evaluated patient engagement activities and acted on the results (interquartile range 2). See Table 2 for a summary of survey responses, as well as each question’s relative contribution to our conceptual model using latent factor analysis. The confirmatory factor analysis confirmed the structure of our conceptual model (p < 0.001 for all structure, process and outcome models; data not shown). All confirmatory factor analysis models supported the conceptual model, given corresponding survey responses for structure, process and outcomes.

Table 2:

Survey responses and item contributions to the factors of structures, processes and outcomes (n = 141)^*

Survey item	Survey question	Median (Q1, Q3)	Weight	No. (%)of missing responses
Factor: structures
Vision for patient engagement in key documents	Commitment to public and patient engagement values and principles is found in key organizational documents (e.g., mission and vision, strategy)	4 (3, 5)	0.572	0
Organizational structure	Commitment to public and patient engagement values and principles is demonstrated through the structure of the organization (e.g., dedicated patient engagement leadership positions, patient engagement in job descriptions)	3 (2, 4)	0.592	0
Policies and procedures	Ability of organization to include patient engagement in all areas of designing health care services (e.g., policies or processes, position descriptions or training programs)	2 (2, 3)	0.697	1 (0.7)
Organizational priority	Importance of patient engagement as an organizational priority	2 (2, 3)	0.897	0
Communication	Importance of communication about patient engagement	2 (2, 3)	0.959	0
Employee involvement	Importance of employee involvement in patient engagement strategies	3 (3, 4)	0.996	0
Employee attitudes	Importance of employee attitudes or beliefs about patient engagement	2 (2, 3)	0.946	0
Patient involvement	Importance of patient involvement in patient engagement strategies	3 (3, 4)	0.926	0
Evaluation of patient engagement	Importance of evaluation of patient engagement	3 (3, 4)	0.874	0
Factor: processes
Involve patients in developing shared vision	Ability of organization to involve patients in developing a shared organizational vision	3 (2, 3)	0.723	0
Seek input into quality improvement	Organization seeks public and patient input into quality improvement initiatives	4 (3, 4)	0.617	1 (0.7)
Address employee needs	Ability of organization to support employees in their efforts to promote patient engagement (e.g., addressing employee needs)	3 (3, 4)	0.745	0
Provide training	Openness of organization in providing training for health professionals in patient engagement (e.g., communication and shared decision-making skills)	3 (3, 4)	0.73	0
Identify, access and recruit patients	Organization has explicit strategies for identifying and recruiting relevant public and patient participants depending on the engagement activity	4 (3, 4)	0.705	0
Support patient engagement	Openness of organization in supporting patient engagement in the organization (e.g., access to patient representatives or a patient champion, recruiting patient representatives, patient training or coaching, money to pay patients for participation)	3 (2, 4)	0.794	0
Allocate time	Openness of organization in providing time to implement and monitor patient engagement (e.g., time to plan, longer consultations, reassessing targets)	3 (2, 4)	0.823	0
Evaluate patient engagement	Ability of organization to evaluate patient engagement in the organization	3 (2, 4)	0.846	0
Act on evaluation results	Ability of organization to act upon the results of this evaluation	3 (2, 4)	0.859	0
Send engagement reports to predetermined users	Public and patient engagement reports are sent to relevant predetermined users in the organization (e.g., program manager, senior management, board members)	3 (2, 4)	0.559	7 (5.0)
Leaders ensure patient input is used	Organizational leaders ensure that public and patient input is used in service planning and decision-making.	4 (3, 4)	0.743	0
Factor: outcomes
Impact at program level	Aware of public and patient engagement activities that have influenced relevant decisions at the program level	3 (3, 3)	0.876	5 (3.5)
Impact at board level	Aware of public and patient engagement activities that have influenced relevant board decisions	3 (2, 4)	0.852	12 (8.5)
Partnerships developed	As a result of our public and patient engagement work, we have developed collaborative relationships with our stakeholders (e.g., patients, community organizations)	4 (3, 4)	0.43	0

Note: Q = quartile.

^*See Appendix 2, available at www.cmajopen.ca/content/9/4/E1159/suppl/DC1 for the full survey, including Likert scale response options for each survey question.

Relationships between structures, processes and outcomes

The Pearson correlation coefficients for structures, processes and outcomes were positively correlated and statistically significant (Appendix 3). Structures and processes were highly correlated (r = 0.8, p < 0.0001, n = 141). The outcomes factor was moderately correlated with both structures (r = 0.5, p < 0.0001, n = 141) and processes (r = 0.6, p < 0.0001, n = 141).

The mediation and moderation regression analyses (Table 3) showed a direct effect of processes on outcomes (β = 0.7, p < 0.0001, n = 141) and a mediated effect of structures on outcomes through processes (β = 0.7, p < 0.0001, n = 141). Figure 2 displays the significant direct and mediated effects. Neither the direct effect of structures on outcomes nor the mediated effect of processes on outcomes was statistically significant. We observed no moderation effect between structures and processes.

Table 3:

Regression analyses showing the direct and mediation effects of structures and processes on outcomes (n = 141)

Effect	Factor	β (95% CI)
Direct effect	Structures	−0.040 (−0.335 to 0.254)
	Processes	0.665 (0.447 to 0.882)
Mediation effect	Structures	0.682 (0.0.437 to 0.928)
	Processes	−0.024 (−0.202 to 0.153)
Total effect (direct + mediation effect)	Structures	0.642 (0.438 to 0.846)
	Processes	0.640 (0.499 to 0.782)
Moderation effect	Structures × processes	0.014 (−0.257 to 0.284)

Note: CI = confidence interval.

Figure 2:

Relationships between structures, processes and outcomes. Note: The direct effect can be interpreted as the main effect of processes on outcomes, adjusted for structures. The mediation effect of structures on outcomes through processes is the indirect effect driven though the effect of structure on processes. The total effect is the sum of the direct and mediation effects.

Interpretation

In this study, we described patient engagement activities and their effects in highly organized primary care settings in Ontario, Canada. We also explored how organizational structures and processes related to the outcomes of having patient engagement activities influence programs and services, board decisions and organizational partnerships. We found that these primary care settings are conducting some patient engagement activities, and respondents reported that they wanted to do more.

Our analysis suggested a direct association between processes and outcomes, and that processes also mediated the relationship between structures and outcomes. This was congruent with Donabedian’s framework.²⁹ Some of the items in the processes factor that may contribute to the effect of patient engagement activities included planning (e.g., developing a shared organizational vision with patients, allocating adequate time); seeking input into quality improvement, training and supporting employees; engaging (e.g., identifying, recruiting and supporting relevant patients); using leaders to ensure that results are used; and reflecting and evaluating (e.g., evaluating patient engagement and acting on the results). Some of the items in the structures factor that may affect outcomes via processes included an organizational mission and vision for patient engagement, and policies, procedures, job positions, training programs and an organizational culture that reflected the mission.

Our findings resonated with other efforts to delineate such factors. Caplan and colleagues⁴¹ qualitatively explored primary care teams’ experience of system redesign at one academic health system and found that patient engagement was enabled by team prioritization of patient engagement, staff training and dedicated time for ongoing improvement in patient engagement efforts; barriers included organizational culture, leadership and resources. Herrin and colleagues⁴² surveyed patient engagement practices across hospitals in the United States and found wide variations in the extent of engagement; the most frequently cited barriers included competing organizational priorities, time and training. Fancott and colleagues⁴³ developed the concept of “engagement-capable environments” by observing the results of learning collaboratives facilitated by the Canadian Foundation for Healthcare Improvement; they posited that successful organizations have leadership support; mobilize resources; and prepare patients, families and staff.

Evaluating engagement efforts is key to learning and refining organizational approaches to engagement, as well as to improving care delivery and outcomes.⁴⁴ The use of structure, process and outcome measures embedded in our questionnaire was consistent with best practices in using a “family” of measures when conducting quality improvement initiatives.^29,45,46

By design, our study focused on the most organized (and arguably the most resourced) primary care settings in Ontario; future research should explore patient engagement activities and outcomes in other types of primary care settings. Along the same lines, further research into how organizational characteristics (e.g., size, years of operation) are associated with infrastructure and processes for patient engagement could inform health policy on models of primary care, but we did not explore this in our study. We acknowledge the trade-offs that organizational leaders must consider when embarking on patient engagement activities; numerous survey respondents described competing demands and resource constraints, not only to carry out engagement, but also to follow through on the results. There could be a complementary role for qualitative research to explore the nuances of how these decisions and activities play out in the daily work of primary care organizations and their leaders.

Finally, this study was part of a larger program of research that engaged patients in defining opportunities to improve primary care and the collaborative care of mental health issues, and this survey was inspired by our desire to understand primary care organizations’ reticence to engage in the research as knowledge users. Although patients were highly engaged as research partners and leaders in other aspects of the research program, they were not engaged in this survey. Further research would be helpful to understand patient perceptions of facilitators and barriers to their engagement.

Limitations

Several study limitations merit consideration. An overall response rate of 53% is acceptable and is in keeping with other surveys conducted in primary care organizations, particularly in an Ontario context⁴⁷ and using online surveys. ⁴⁸ However, it is possible that the sample of respondents who chose to participate was biased and, as suggested by Aerny-Perreten and colleagues,⁴⁸ may have underrepresented primary care representatives with the highest workloads and a minor interest in the survey topic. As well, 8 survey responses provided answers to less than 85% of the questions and were excluded from this study. Although our survey was directly targeted at executive directors of organizations, we did not collect information about who completed the survey.

Although the MORE instrument we used to create our survey was developed using a comprehensive Delphi process, ²⁴ further research is needed to determine the psychometric properties, validity, comprehensiveness and usability of the scale.^24,28 Our survey questions measured organizational “ability” and “openness” to carry out specific patient engagement activities, but we did not provide an operational definition of how able or how open beyond the Likert scale anchors, so the questions were subject to interpretation. Similarly, we did not provide respondents with a definition of “collaborative partnerships,” as long as these partnerships were developed as a result of patient engagement activities.

The survey data were ordinal, and the responses to the survey questions were highly correlated; these concerns were mitigated by our analytic approach testing a model of composite variables related to structures, processes, and outcomes. By conducting a confirmatory factor analysis, we were able to diminish concerns about multicollinearity. Given that our data were cross-sectional, our analysis described associations between variables and did not imply causal relationships. In other words, we did not “prove” that having certain structures or processes in place led to the outcomes we examined.

Conclusion

Our findings shed light on how organizational structures and processes could be related to successful patient engagement based on the perceptions of primary care leaders. Organizations may wish to conduct a self-assessment using the questions we drew from the MORE and PPEET tools to reflect on their commitment to patient engagement and their follow-through with associated resources and activities. Future research can further test these relationships for causality and distill key priorities for policy-makers and organizations, as well as explore the transferability of this model to other types of health care organizations beyond primary care.