Patient engagement study to identify and improve surgical experience

Abstract

Background

Patient engagement is the establishment of active partnerships between patients, families, and health professionals to improve healthcare delivery. The objective of this project was to conduct a series of patient engagement workshops to identify areas to improve the surgical experience and develop strategies to address areas identified as high priority.

Methods

Faculty surgeons and patients were invited to participate in three in-person meetings. Evaluation included identifying and developing strategies for three priority areas to improve the surgical experience and level of engagement achieved at each meeting.

Results

Sixteen faculty surgeons and 32 patients participated. Some 63 themes to improve the surgical experience were identified; the three highest-priority themes were physician communication, discharge process, and expectations at home after discharge. Individual improvement strategies for these three prioritized themes (12, 36 and 6 respectively) were used to develop a formal strategic plan, and included a physician communication survey, discharge process worksheet and video, and guideline regarding what to expect at home after discharge. Overall, the level of engagement achieved was considered high by over 85 per cent of the participants.

Conclusion

A high level of patient engagement was achieved. Priorities were identified with patients and surgeons to improve surgical experience, and strategies were developed to address these areas.

A series of patient engagement workshops to identify areas to improve the surgical experience were conducted. A high level of patient engagement was achieved and the output from these meetings led to a patient centred product that improved health outcomes. This project provides a straight forward example of how patient engagement can be operationalized at the institutional level.

key themes identified and strategies developed

Introduction

Patient engagement (PE) is the establishment of active partnerships and meaningful collaborations between patients, families, and health professionals to improve health and healthcare at various levels¹. A multidimensional framework purposed by Carman and colleagues² describes a continuum of engagement from consultation to partnership to shared leadership at various levels, including direct care, organizational design, and governance and policy-making. This continuum ranges between where patients are involved but have limited power or decision-making authority and where patients have shared power and decision-making authority. PE is considered a critical part of a continuously learning health system, and there is increasing recognition and acceptance that users of health services have the requisite expertise and experience to provide important contributions to enhancing the design and delivery of healthcare services^3–5. As a result, there have been increasing requests from hospitals, health institutions, and organizations to incorporate PE into both clinical and research activities.

Best Practice in Surgery is a quality improvement program that has been undertaken by the Department of Surgery at the University of Toronto. As part of this program, the Patient Experience Project was initiated with the overall goal of improving the surgical experience in the Department of Surgery at the University of Toronto⁶. The specific objective of this project was to conduct a series of PE workshops, in which surgeons and patients worked together to identify areas to improve the surgical experience, and develop strategies to address areas identified as high priority.

Methods

The overall plan for this project was to have a series of three 1-day, in-person meetings with surgeon and patient representatives. These three meetings were designed in advance with the project team and a professionally trained facilitator. The role of the facilitator was to assist in the planning of workshop activities that would facilitate meaningful collaborations with the surgeon and patient representatives. Before each meeting, the project team and facilitators finalized the objectives, agenda, and activities for each meeting. In addition to the professional facilitator, a professional transcriptionist documented the large group discussion at all three meetings.

Before the start of the project, the protocol was reviewed by the Research Ethics Board at the University of Toronto.

Participants

The Department of Surgery at University of Toronto includes 10 affiliated, academic hospitals in the Greater Toronto Area and has eight surgical divisions including: plastic surgery, general surgery, urology, neurosurgery, orthopaedic surgery, vascular surgery, cardiac surgery, and thoracic surgery. The department employs 260 full-time faculty members, and trains 200 residents and 175 fellows per year.

A faculty surgeon was appointed as the Patient Experience Project lead and, with the support of the Chairman of Surgery, invited 16 faculty surgeons to participate in the project, representing all eight surgical divisions and eight of the affiliated academic hospitals.

As this was an entirely new initiative for the department, each of the surgeon representatives was asked to select two of their own patients who had undergone any type of elective, inpatient surgery representative of their clinical practice and in the surgeon’s best judgement: were able to speak and understand English; would be able to voice their opinions; would be empathetic to other patients; and would be able to balance their own surgical experience with the overall goals of the project. The surgeon representative initially contacted their patients to introduce them to the project and then forwarded the patient’s contact information to the project coordinator, who provided more information about the study and obtained informed consent.

Event description

Meeting 1

The overall goals of meeting 1 were to: introduce the patient participants to the project and objectives; establish a safe and positive environment to facilitate mutual collaboration and partnership; and to identify strengths and opportunities to improve the surgical experience at the University of Toronto. It was decided a priori that a maximum of four surgeon representatives would participate in meeting 1 to ensure that a safe and positive environment was achieved. Furthermore, all of the meetings were organized so that the patients did not participate in any small group activities with their treating surgeon. At the start of meeting 1, the patient representatives were asked to participate in a ‘gold star and magic wand’ exercise, in which they were asked to provide one positive experience and one experience that they felt could be improved with a solution for fixing it. The patient representatives completed this exercise for each of the categories across the surgical experience including: diagnosis, surgical consultation and preadmission visit, surgery and hospital stay, discharge home and rehabilitation, and follow-up and recovery. Initially, the patient representatives worked together in small groups with one surgeon representative and discussed their experiences with each other. This was followed by a presentation of the collective experiences of the smaller group to the larger group. These individual experiences were then grouped into common themes and prioritized by an anonymous vote to identify the top 10 priority themes. For this voting exercise, all of the participants had a total of 10 votes and were allowed to place more than one vote on any item.

Meeting 2

The overall goals of meeting 2 were to: review the 10 priority themes and voting results from meeting 1; recommend three priority themes that should be considered the top priority; and develop strategies to address each of these three top priority themes. All patient and surgeon representatives were invited to participate in this meeting. All participants (including patients and surgeons) were instructed to review the list of 10 priority themes from meeting 1, and combine and/or modify any of these priority themes. Similar to meeting 1, both small and large group discussions were used to achieve consensus for the top three priority themes to improve the surgical experience. Participants were then asked to develop strategies to address each of these three top priority themes. They were encouraged to develop strategies that could be: implemented at a university level, had measurable outcomes to evaluate improvement, and were relatively inexpensive.

Meeting 3

The overall goal of meeting 3 was to review and finalize one strategic plan for each of the three top priority themes identified at meeting 2. Before meeting 3, the project team selected one common strategy for each of the three priority themes based on the output from meeting 2 and the preset criteria. The project team then developed a more detailed plan for each of these selected strategies to be used as the starting point for meeting 3.

At meeting 3, the participants were placed in three small groups (including 1–2 surgeon representatives) and each group was asked to evaluate one of the three strategic plans in terms of concept alignment, details of the plan, and implementation process. Concept alignment for each strategic plan was evaluated by having each small group rate the question ‘Overall, how does this project align with the issues addressed at meetings 1 and 2?’ on a 5-point Likert scale, with 1 representing no alignment and 5 representing perfect alignment. Each small group presented the results of their discussion to the large group for further feedback, and consensus was achieved for each strategic plan.

Post hoc data analyses and evaluation

Before the start of this project, the goal was to have three prioritized themes each with a respective, strategic plan to improve the surgical experience in this organization. It was also planned to have the participants complete an evaluation form at the end of each session. However, the authors did not have any formal conceptual framework or data analysis plan because this project was not considered to be a formal research study. However, during the first, in-person meeting it became evident that a more formal evaluation of the meeting output was necessary because of the depth and richness of the ideas and feedback from the patient representatives. Therefore, over the course of the project, a conceptual framework and data analyses were selected, and applied in an ad hoc fashion.

The conceptual framework selected for this project was developed by Deverka and colleagues⁷. One key aspect of this model is that it suggests that both the process of engagement and the output from the engagement process are necessary for formal evaluation of the engagement process. The process of engagement refers to the effectiveness or level of engagement achieved, and is important because the higher the level of engagement achieved, the higher the likelihood that the meeting output will subsequently lead to a positive impact on health processes and health outcomes. The project team adapted this conceptual framework for the purpose of the present project (Fig. 1).

Fig. 1.

Study flow diagram

Based on this conceptual framework, the level of engagement was evaluated using an approach described by Lavallee and co-workers⁸ based on the normative theory of public participation. This theory suggests that six meta-criteria, including respect, trust, legitimacy, fairness, competence, and accountability, must be present to establish effective engagement, defined as mutual understanding via free and uncoerced discussion among all parties⁷. To operationalize these meta-criteria, Lavallee et al. suggested using one question on the evaluation form to assess each criterion. For this project, the participants completed an evaluation form in real time after each meeting, and the level of engagement was assessed using these evaluations on an ad hoc basis.

To further analyse the meeting output, an analytical approach was used that incorporated some, but not all, principles of qualitative research methods as this project was not designed as a formal research study^9,10. Specifically, the project lead read and reviewed all of the transcripts and meeting output, and collapsed individual experiences into emergent categories that were then organized into common themes. The project team reviewed and discussed the major themes, and came to a final decision based on consensus.

Results

Meeting 1

Twenty-two patients and four surgeon representatives participated in meeting 1. The results from this meeting generated 235 individual experiences which were grouped into 63 common themes (Table 1). These 63 common themes were then prioritized by anonymous vote; the results for the top 10 priority themes are shown in Table 2.

Table 1.

Individual experiences and common themes to improve surgical care

	Individual experiences	Common themes
Diagnosis	60	13
Surgical consultation	49	15
Surgery	49	17
Discharge process	39	10
Recovery and follow-up	38	8
Total	235	63

Table 2.

Top 10 priority themes to improve surgical care

Priority	Description	Surgical category	No. of votes
1	Clear and detailed sheet with discharge plan	Discharge process	16
2	Follow-up call by surgical team member	Discharge process	14
3	Patient passport (improve communication between surgeon and patient)	Follow-up and recovery	13
4	Buddy system	Diagnosis	12
5	Electronic access to medical records	Diagnosis	12
6	Physician communication	Surgical consultation	12
7	Support from community care	Discharge process	12
8	Wellness centre	Follow-up and recovery	11
9	Shorter waiting times between appointments	Surgical consultation	11
10	Patient passport (improve communication and coordination of care between healthcare providers)	Follow-up and recovery	10

Meeting 2

Twenty-two patients and 16 surgeon representatives participated in meeting 2. At this meeting, the top 10 priority themes were reviewed, and physician communication, discharge process, and expectations at home after discharge were selected as the top three priority themes based on group discussion and consensus. Strategies to address each of these three priority themes were suggested and discussed. This activity yielded 12, 36 and 6 individual improvement strategies for physician communication, discharge process, and expectations at home after discharge respectively. These individual strategy ideas were grouped into common strategy ideas for each of the three priority themes (Table 3).

Table 3.

Suggested strategies to improve surgical care in priority areas

Priority area	Strategy
Physician communication	Improve interpersonal skills with communication courses
	Improve team communication with app that provides immediate updates from one team member to all team members
	Provide feedback to physicians about their communication skills
	Department of Surgery to implement strategies to incentivize good communication
Discharge process	Standardized booklet/instructions
	Discharge coordinator
	Discharge class (in hospital)
Expectations at home after discharge	Standardized booklet/instructions with guidelines about what to expect and normal ranges
	Follow-up telephone call from surgeon within 1 week of surgery
	24-h hotline for patients experiencing problems
	Drop-in clinic for patients experiencing problems

Meeting 3

Thirteen patients and 11 surgeon representatives participated in meeting 3. Based on the output from meeting 2, the project team selected one strategy for each of the top priority themes based on consensus and the predetermined criteria. For physician communication, this included a physician survey to assess current routines with respect to communication and professionalism; for the discharge process, this included a standardized discharge process with a discharge video and corresponding worksheet; and for expectations at home, this included patient guidelines to assist patients in knowing what is normal and what is not normal at home after surgery.

The overall concept alignment scores for the physician communication survey, discharge video and worksheet, and guideline regarding what to expect at home were 2, 4 and 4 of 5 respectively. Overall, the main reasons why the group felt that the physician communication survey did not align with previous discussions were that it would be difficult to implement and would have little impact on changing physician behaviour. The group suggested tool kits be provided to surgeons, trainees, and office staff, and would include items such as e-training modules, communication sessions, and increased recognition from the Department of Surgery for good communication, such as academic merit points and/or awards. Both the strategic plans for the discharge process and expectations at home were felt to be well aligned, and there were minor suggestions for improvement. Other major feedback from the group was to make sure that the authors did not ‘reinvent the wheel’ with any of these projects, and the need for strong champions to implement these initiatives was recognized. After discussion of each of the strategic plans, the group voted expectations at home after discharge as the most important priority item, and provided a clear message that patients needed individualized and timely advice regarding what was normal and not normal at home after surgery.

Meeting output and evaluation

The three priority areas identified were physician communication, discharge process, and expectations at home after discharge; and the respective strategic plans physician communication survey, discharge video and worksheet, and guideline regarding what to expect at home were developed.

Based on the evaluation forms, it was possible to evaluate all six of the meta-criteria to assess the level of engagement achieved. However, because this was done in an ad hoc manner, the authors were able to evaluate only four of the six meta-criteria at each of the three meetings (Table 4). Over 85 per cent of the meeting participants indicated that they strongly agreed or agreed with each of the statements corresponding to the six meta-criteria on the evaluation form. The highest scores were achieved for respect, fairness, and competence, which demonstrated that the participants were actively involved and felt that their individual ideas were heard, as well as the ideas of others. The scores were slightly lower for trust, legitimacy, and accountability, but overall they demonstrated that the participants felt they had control over the final decisions made at the meetings and that the group’s input would subsequently be used.

Table 4.

Evaluation of level of engagement

Meta-criteria	Description	Statements	Meeting 1 (n = 21)	Meeting 2 (n = 37)	Meeting 3 (n = 23)
Respect	Process is collaborative with an atmosphere conducive to open airing of diverse viewpoints	I felt the small group activities were well prepared and encouraged discussion and collaboration	21	36	23
	Two-way flow of information that facilitates mutual learning	I felt my ideas were heard and I was able to contribute	19	37	23
		I felt the ideas of others in the group were heard and that they were able to contribute	21	37	23
Trust	Stakeholders are confident that the project outcomes reflect the discussions and decisions reached through a deliberative process	I felt that the overall outcomes from the meeting reflect the discussions and decisions of the group	20	35	20
Legitimacy	A balanced representation of relevant stakeholders is achieved	I felt the ideas discussed and shared today would be representative of other patients who did not attend the meeting	19	–	–
	Conflicts and disagreements are well managed	Overall I felt that any conflict and disagreements were well managed by the facilitator	19	32	–
Fairness	Stakeholders understand the purpose and process of engagement	The goals and objectives of the meeting were clear	21	34	23
		The workshop was well organized	21	36	23
Competence	Stakeholders understand the information and are able to contribute to the discussions and decision-making	I felt comfortable sharing my ideas and experiences with the group	20	37	23
Accountability	Stakeholder input is incorporated into the final decision	I am confident that the group’s input will be used	19	34	–

Discussion

The results of this project identified physician communication, discharge process, and expectations at home after discharge as the three top priority themes or areas to improve the surgical experience at the University of Toronto. Of these, expectations at home after discharge was considered the top priority area. Strategies to address issues for each of these priority areas were developed in collaboration with surgeons and patients.

A number of lessons were learnt about the engagement process. First, there was no difficulty including patients with different diagnoses and undergoing different procedures at the meetings. The authors were initially concerned about inviting patients with benign and malignant diseases undergoing a wide range of procedures (such as hip replacement, colectomy, femoropopliteal bypass, breast reconstruction). However, the patients very quickly identified similarities in their surgical experience, and this shared experience of surgery created a safe and comfortable environment that led to meaningful and open discussions. Second, the patient participants found it particularly meaningful to interact with surgeon participants and not just other patients. Patient participants felt that their feedback was more meaningful because it was being heard directly by surgeons (frontline workers) who may be able to effect change. The surgeon participants also valued the patients’ opinions outside of the clinical encounter, and many indicated how they had made changes to their clinical practice as a result of participating in the meetings. Third, the quality of the meetings was increased greatly by the presence of a professional facilitator who was able to help plan activities to ensure that there was two-way engagement between the patients and surgeons, and also acted as a neutral third party to minimize the power differential between the surgeon and patient participants. Finally, the support of the Department of Surgery significantly increased the credibility and importance of the project for all participants. Based on their experience, the authors have provided a summary of some key items to consider when conducting patient engagement meetings (Table 5).

Table 5.

Key items to consider when conducting engagement meetings

	Description
Organizational and administrative Support	Organizational support (department and chairman) adds credibility to the project An administrative support person is essential to act as patient liaison
Appropriateness	Patient engagement needs to be considered early in the planning phase of a project Not all activities are suitable for patient engagement Inviting patients to activities that are not suitable or not designed to ensure two-way engagement is tokenistic
Representativeness	Ensuring representativeness is challenging Do your best to select patients who are representative of the entire population Understand and acknowledge limitations
Selection of engagement activities	Activities to promote two-way engagement between patients and physicians are necessary Having physicians involved in the engagement process is more meaningful to patients Having a professional facilitator is an effective strategy to neutralize the power differential between patients and physicians
Outcomes and evaluation	Formal engagement evaluation need to be planned Both outcome and process measures should be included Level of engagement is an relevant process measure that is relatively easy to evaluate
Dissemination of results	It is important to use the results of the meeting so that patients do not feel that their time as wasted
Sustainability	Consider how the Patient Advisory Committee will be maintained and how to keep momentum moving forward

As expectations at home after discharge was considered the top priority, the investigative team started to develop a discharge booklet, as was recommended. However, based on the discussion at the final meeting, it was also realized that the patients wanted to receive tailored information in real time. Therefore, the team developed and pilot tested an integrated discharge monitoring system to support patients at home after discharge from the hospital using a mobile app called Home to Stay¹¹. The results of pilot testing have shown high patient satisfaction, as well as a reduction in 30-day readmission rates. Therefore, it has been possible to demonstrate that the output from this project led to the development of a patient-centred product that has had a direct impact on patient care and outcomes. Although the results demonstrate the benefit of PE, the time lag between the final meeting and completion of the pilot testing for Home to Stay was approximately 2 years, and highlights one of the challenges in the evaluation of PE and its impact on direct patient care.

Despite the growing demand for PE, there is limited evidence in the literature to support its value¹². A recent systematic review¹³ of PE identified 20 957 studies; only 48 studies met the inclusion criteria, which included: engagement that involved partnering or engaging in co-design with patients; and reported outcomes of the PE itself or on the design, delivery or evaluation of health services. The majority of these studies were qualitative, and related to mental health and primary care. Similar to the present findings, the review found that use of trained facilitators, higher proportions of patients versus providers, and institutional commitment from leadership were important facilitators of PE. The types of outcome reported included enhanced care or service delivery (n = 35), development of specific policy or planning documents (n = 15), enhanced governance (n = 5), and education or tool development (n = 11). However, none of these studies showed that these specific outcomes directly improved health outcomes. The major conclusions from this review were that future research should incorporate longitudinal measures and approaches to explore the impact of PE.

One of the main limitations of this project was the ad hoc nature of the evaluation, reflecting the group’s inexperience with PE. However, major challenges were encountered in determining how to evaluate PE, even post hoc. The method purposed by Lavallee and colleagues⁸ was one of the only tools the group was able to identify to evaluate the level of engagement that was both straightforward to use and could be tailored specifically to the engagement activity. Another easy-to-use and accessible engagement evaluation tool is a 21-item questionnaire developed by Abelson et al.^14,15, which is based on the core principles of quality of engagement. A recent systematic review¹⁶ scrutinized 23 tools that have been used for evaluating PE and, based on these, proposed a taxonomy of 72 metrics to evaluate PE.

The other major limitation of this project was that the surgeon representatives hand-selected patients to participate, who represented a convenience sample. Although this convenience sample was relatively diverse with respect to age, sex, and ethnicity, overall this group was highly educated and included many working and retired professionals. Therefore, the selection process may have affected the representativeness of the group and the generalizability of the project. However, it is important to consider that, if a well educated group experienced issues with physician communication, discharge process, and what to expect at home after discharge, it is likely that these issues are considerably amplified in less educated and more vulnerable groups. It is also important to note that no standard technique has been endorsed as the best method for recruiting patients and significant variation exists. Investigators will need to weigh the potential for introducing bias by hand selecting patients against this same potential for self-selected participants as well as the time and cost implications of these approaches.