A Simulation-Based Workshop to Improve Residents' Collaborative Clinical Practice

ABSTRACT

Background

The Accreditation Council for Graduate Medical Education expects residents to attain competency in systems-based practice by advocating for quality patient care, working in interprofessional teams, and implementing system solutions to prevent errors. Diabetes in pregnancy was identified as an area for improvement through comprehensive interdisciplinary and interprofessional care.

Objective

An interdisciplinary and interprofessional workshop was created by 3 regional academic institutions to improve collaborative practice, clinical knowledge, and clinical judgment of residents.

Methods

A workshop consisting of 4 clinical simulation stations for ultrasound assessment, glycemic control, hyperglycemic emergencies, and macrosomia complications was designed to address gaps in quality of care. Workshop participants were residents from 6 programs and students in nursing, pharmacy, and sonography. Attitude and clinical knowledge were measured preworkshop and postworkshop, and at 3-month and 6- to 7-month follow-up.

Results

There were increases in average clinical knowledge scores across time points from residents: 56.4% preworkshop, 64.8% postworkshop, 66.0% at 3-month follow-up, and 68.1% at 6- to 7-month follow-up. Additionally, participants reported positive attitudes toward interprofessional education and indicated high overall satisfaction.

Conclusions

Residents demonstrated improved knowledge and attitudes toward interprofessional training after participating in a large-scale simulation workshop focused on the care of patients with diabetes in pregnancy.

What was known and gap

Residents must participate in quality improvement and performance improvement initiatives and train in interprofessional care. Gestational diabetes mellitus is a good area to improve through comprehensive interdisciplinary and interprofessional care.

What is new

An interdisciplinary and interprofessional simulation workshop on gestational diabetes mellitus to improve residents' collaborative practice, clinical knowledge, and clinical judgment.

Limitations

Data on clinical knowledge acquisition limited by high rate of attrition in participant responses during follow-up.

Bottom line

Residents demonstrated improved knowledge and attitudes toward interprofessional training and found the workshop acceptable.

Introduction

The Accreditation Council for Graduate Medical Education expects residents to participate in quality improvement (QI) and performance improvement (PI) initiatives and to train in interprofessional care.¹ The Triple Aim, proposed in 2008 as a direction for US health care delivery reform, unequivocally connects interprofessional health care teams to the provision of better health care services that will eventually lead to improved health outcomes.^2,3 Because of the educational interest and overlap of care associated with gestational diabetes mellitus (GDM), this was selected as a target for interprofessional care training.

Gestational diabetes mellitus is defined as glucose intolerance with onset or diagnosis during pregnancy.⁴ Gestational diabetes mellitus increases the risk of preeclampsia, stillbirth, large for gestational age infants, and shoulder dystocia.⁵ Data from the US Pregnancy Risk Assessment Monitoring System reported the prevalence of GDM as 12.1% and 6.8% in Hispanic and non-Hispanic whites, respectively.⁶ A local study found that GDM affected 8.6% of pregnancies in El Paso, Texas, which has an 80% Hispanic population.⁷

Given the significance of GDM and its elevated prevalence in El Paso, an interdisciplinary and interprofessional workshop was developed. The feasibility and acceptability of the workshop as well as interprofessional readiness and knowledge acquisition were evaluated across several specialties.

Methods

Faculty and educators from 3 regional academic institutions along the US-Mexico border who have special interests in interprofessional education (IPE) were invited to participate in the needs assessment and planning of an interprofessional workshop aimed to improve performance and quality of care in a clinical simulation environment. The team comprised 22 educators and 12 support staff (4 program coordinators and 8 health care simulation technology specialists to operate and support the training activities). The scenario design and diverse scheduling needs required planning 6 months in advance. Participants were from 6 residency programs: obstetrics and gynecology, emergency medicine, radiology, pediatrics, surgery, and family medicine. Students in nursing, pharmacy, and diagnostic medical sonography from 3 institutions also attended.

The online supplemental Figure illustrates the content and objectives of this 5-hour workshop, consisting of high-immersion clinical simulation stations and medical skills scenarios designed to address identified gaps in quality of care. Given the prevalence of GDM and the opportunities for multiple disciplines to influence care, clinical practice consensus analysis by educators identified areas where quality of care could be targeted and affected by this training. Learning objectives were discipline-specific, focusing on collaborative practice, improving attitudes toward interprofessional training, and enhancing clinical knowledge. The goal of this type of interprofessional training is to improve patient safety and optimize patient outcomes.

Figure.

Timing and Flow for a 5-Hour Workshop Day

Note: the simulations were divided into 4 stations, each with 3 substations. Each substation ran for 15 minutes and each 45-minute substation block had a 15-minute debrief before rotating to the next station. Further description of the content for each station appears in the online supplemental material.

The workshop was repeated in 3 half-day sessions to accommodate schedules. Piloting of scenarios also included an analysis of the simulation center layout to ensure a smooth flow of participants between sessions. Each of the 3 workshop days began with a prebrief orientation of the simulation space. Learner types were distributed across teams to ensure interprofessional collaboration. Additionally, participants completed the Readiness for Interprofessional Learning Scale (RIPLS), an instrument with validity evidence for measuring institutional interest in collaborative training.⁸ Learners were also asked to complete a specialty-specific, multiple-choice, 20-question knowledge assessment on GDM developed by faculty from participating residencies without further testing. Interprofessional teams were distributed across all 4 stations and rotated through the 3 substations in 1-hour blocks with 15 learners per station (5 in each substation); the workshop spanned 5 hours (Figure). Faculty facilitators observed and engaged learners at each substation.

Station 1 (ultrasound assessment) used ultrasound task trainers with real ultrasound machines for skill practice. Station 2 (glycemic control) used tabletop discussion, task trainers, and high-fidelity manikins for skill demonstration. Station 3 (hyperglycemic emergencies) utilized high-fidelity manikins for diabetic ketoacidosis management and precipitous delivery, and a standardized patient for evaluation and counseling for preterm contractions. Station 4 (macrosomia complications) used a high-fidelity manikin, tabletop discussion, and task trainer modeling to enhance learning. Manikins were controlled by a health care simulation technology specialist to display physiological parameters based on a written script. Learners were expected to obtain a history and perform a physical examination, demonstrate technical skills, administer medications, or perform interventions for each scenario.

Debriefing for the 4 stations included participants from the 3 substations. Because interprofessional training encourages different specialties to learn with, about, and from one another, a shared storytelling model of debriefing was incorporated. This model asks each participant to share a piece of the simulation scenario story from their perspective. These pieces combine to help paint a picture of the scenario and its relevance to each specialty.^9,10 The short session duration required direct performance feedback, while skills training sessions benefited from the gather, analyze, and summarize debrief technique.¹¹

The protocol was reviewed by the Texas Tech University Health Sciences Center El Paso Institutional Review Board, and it was deemed exempt from formal review.

Faculty members developed educational metrics, goals, and objectives for each specialty. Evidence-based practice protocols were available to participants. Participants' attitudes toward IPE were measured before and after completion of the workshop, and clinical knowledge was measured preworkshop and postworkshop and at 3-month and 6- to 7-month follow-up.

For nursing students and residents, paired t tests were used to analyze the preworkshop and immediate postworkshop scores (Table 1). The Wilcoxon signed rank test was used to compare preworkshop and immediate postworkshop scores of pharmacy and sonography students. For data on residents, a linear regression analysis was performed using generalized estimating equations. Data across all 4 knowledge assessment time points (preworkshop, postworkshop, 3-month follow-up, and 6- to 7-month follow-up) were used in this longitudinal analysis.

Table 1.

Clinical Knowledge Examination Scores: Average Pre– and Post–Interprofessional Education (IPE) Activity

Learner	Sample Size	Average Pre-IPE Score (SD)	Average Post-IPE Score (SD)	P Value
Nursing students	31	58.7 (10.6)	67.3 (10.4)	.0002a
Pharmacy students	7	63.6 (7.5)	75.7 (10.6)	.016b
Sonography students	4	51.3 (6.3)	61.3 (6.3)	.25b
Resident physicians	67	56.4 (20.2)	64.8 (16.0)	.0004a

^aP value is from a paired t test.

^bP value is from a Wilcoxon signed rank test.

Attitudes toward IPE were measured using RIPLS before and immediately following the workshop.^8,12 RIPLS is composed of 4 subscales: Teamwork and Collaboration (items 1 through 9), Negative Professional Identity (items 10 through 12), Positive Professional Identity (items 13 through 16), and Roles and Responsibility (items 17 through 19). Items 10 through 12 were reverse coded prior to calculating the Negative Professional Identity subscale score. Items 17 through 19 were also reverse coded, because participants were novice learners. Higher RIPLS scores indicate a more positive attitude toward IPE.^8,12 The average RIPLS subscale scores and standard deviations were calculated and analyzed using paired t tests. Statistical testing was performed using 2-tailed tests and an α of .05.

A 5-point Likert scale with responses ranging from strongly disagree to strongly agree was used to collect responses about workshop perceptions. A standard form used to assess campus simulation activities was anonymously administered following the workshop. Data were managed by REDCap and analyzed using SAS version 9.4 (SAS Institute Inc, Cary, NC). The 2 main outcomes were clinical knowledge and attitudes toward IPE.

Results

A total of 90 resident physicians attended the workshop as a required educational activity by their respective programs: emergency medicine (n = 36), family medicine (n = 19), obstetrics and gynecology (n = 19), pediatrics (n = 9), radiology (n = 6), and surgery (n = 1). A total of 109 participants of the 178 total learners voluntarily participated in the research component of this workshop by completing discipline-specific knowledge examinations. Of these, 67 (61%) were residents, 31 (28%) were nurses, 7 (6%) were pharmacy students, and 4 (4%) were sonography students. The knowledge examination scores were not linked to residents' specialty.

Table 1 reports the average pre-IPE and post-IPE clinical knowledge scores for each learner group. The average score increased 8.6% for nursing students pre-IPE and post-IPE training (P = .0002). Pharmacy students showed a 12.1% increase (P = .016). Resident physician responses showed an increase in clinical knowledge scores across all time points. The average correct score was 56.4% preworkshop, 64.8% postworkshop, 66.0% at 3-month follow-up, and 68.1% at 6- to 7-month follow-up. A repeated measures analysis found that this increase was statistically significant (P = .032).

Average RIPLS subscale scores are noted in Table 2. Increases were observed across 2 time points in the subscale scores except Roles and Responsibility.

Table 2.

Readiness for Interprofessional Learning Scale (RIPLS) Scores: Pre– and Post–Interprofessional Education (IPE) Activity

RIPLS Subscale	Sample Size	Average Pre-IPE Score (SD)	Average Post-IPE Score (SD)	P Value
Teamwork and Collaboration	94	39.2 (5.1)	40.7 (5.2)	< .0001
Negative Professional Identity	73	11.7 (2.5)	12.5 (2.6)	.013
Positive Professional Identity	71	16.2 (2.8)	17.5 (2.6)	< .0001
Roles and Responsibility	72	10.4 (2.3)	10.6 (2.5)	.31

Table 3 shows the results of the satisfaction survey. Of the respondents, 91% (90 of 99) agreed or strongly agreed that the activity was sufficiently realistic, and 89% (88 of 99) stated the activity allowed them to demonstrate and improve team communication.

Table 3.

Interprofessional Education Activity Satisfaction Survey

Question	Strongly Disagree, n (%)	Disagree	Neutral	Agree	Strongly Agree
Total combined averages (n = 99)	14 (14)			85 (86)
The activity demonstrated medical information that applies to my clinical practice.	12 (12)			87 (88)
The activity had appropriate realism for the content being taught.	9 (9)			90 (91)
The instructor organized the debriefing to allow a reflection of my performance and identified what I did well and poorly.	21 (21)			78 (79)
The instructor set the stage for an engaging learning experience.	11 (11)			88 (89)
This activity allowed me to demonstrate and improve my team communication.	11 (11)			88 (89)
This activity allowed me to demonstrate my ability to perform clinically.	17 (17)			82 (83)
This activity allowed me to demonstrate my fund of knowledge.	16 (16)			83 (84)
This activity taught me how to organize and prioritize patient care responsibility.	18 (18)			81 (82)
This activity taught me medical information that I can apply to my clinical practice.	12 (12)			87 (88)
This activity will help me provide improved patient care.	14 (14)			85 (86)

Discussion

Based on feedback, learners appreciated the opportunity to collaborate with other disciplines and residents demonstrated increased knowledge from preassessment to postassessment that was sustained across 3-month and 6- to 7-month follow-up.

Learners had more positive attitudes toward IPE after completion of the workshop as measured by 3 of the RIPLS subscales: Teamwork and Collaboration, Negative Professional Identity, and Positive Professional Identity. However, the study did not detect a statistically significant increase in the Roles and Responsibility subscale score. We noted statistically significant increases (pre versus post) in 3 RIPLS subscale scores; however, these increases were small. To our knowledge, there are no published reports on what constitutes a meaningful change in RIPLS scores, meriting further investigation.

Although this activity required a large amount of personnel, time, and resources (estimated cost of $58,160 for 3 days of faculty, staff, and simulation center time), 178 learners were trained to deliver collaborative clinical care (Table 4).

Table 4.

Estimated Expenses of Interprofessional Educational (IPE) Training

Expense	Units	Estimated $/Unit	Line Expenditure
Curriculum planning and design by 6 faculty members to create 12 cases	50 hr	100/hr	$5,000
6 core staff members trained to assist with this activity and help with logistical planning and meetings	20 hr × 6 staff = 120 hr	30/hr	$3,600
12 staff for each of the 3 days to implement the sessions	3 days × 7 hr/day × 12 staff  = 252 hr	30/hr	$7,560
16 faculty on each day to provide instruction	3 days × 5 hr/day × 16 faculty  = 240 hr	100/hr	$24,000
Resources: specialized space and equipment within the simulation center, manikin/task trainer use, and supplies for each of the 12 activities	3 days × 5 hr/day × 12 rooms  = 180 room hr	100/hr	$18,000
Overall cost estimate for this training activity			$58,160

Based on experience, simulated complex clinical scenarios enhanced learners' expertise and capacity for collaboration. Increased clinical knowledge was identified based on knowledge testing at 4 time points for resident participants. However, due to a significant decline in response rates, these data should be interpreted cautiously. For future studies, in-person requests rather than e-mail surveys may improve response rates. This attrition caused a significant limitation in long-term study interpretation. In addition, it is difficult to assess the effect of a single educational activity on the total knowledge relative to months of education and training opportunities.

Debriefing during the activity allowed for open team communication and sharing of each specialty's role in the delivery of health care for GDM. Relevant scenarios with evolving clinical situations reinforce the importance of “team competency,” which was viewed as the ability of learners to apply knowledge appropriately and efficiently by working with others to solve problems and successfully deliver services.^13–15 In addition to expressing an overall positive attitude toward interprofessional simulation training, learners reported the workshop was a realistic and engaging experience that challenged them to meaningfully integrate objectives with their clinical roles and team practice.

This experience enhanced partnerships between the local academic community and health care providers. Development of clinical simulation-based QI/PI curricula portend exciting research, scholarship, and professional development opportunities.^16–18

A number of challenges were encountered, including limited protected time for faculty, residents, clinical simulation educators, and staff. There were scheduling conflicts encountered when coordinating with multiple training programs. Despite these challenges, well-planned QI/PI activities using immersive simulation scenarios have the ability to enhance health system performance. The most significant limitation to this study was the high rate of attrition in participant responses during follow-up. Simulation was an effective format for this workshop, with the potential for reproducible expansion to other clinical departments, training programs, and institutions.

Conclusion

A regional interprofessional half-day simulation workshop on GDM in pregnancy, for residents from multiple specialties and students from other health professions, was highly acceptable to learners. Scores on knowledge tests as well as teamwork and collaboration tests improved in small, but significant, amounts for residents, student nurses, and student pharmacists. Residents, who were tested 6 to 7 months after the intervention, demonstrated a sustained increase in knowledge. Long-term planning before the sessions resulted in requiring only 3 half-day sessions to accommodate a large number of participants.