Evaluation of case based simulation teaching to improve the Family Medicine residents urgent care management skills at a teaching hospital

ABSTRACT

Context:

Simulation based teaching is effective strategy to allow trainees to acquire skills, develop clinical judgment and to become competent in order to manage the patients in a safe environment. The Case-based simulation improves the assessment and management skills of the trainees to prepare them as a competent physician.

Aim:

Evaluation of case based simulation teaching to improve the Family Medicine residents urgent care management skills at a teaching hospital.

Setting and Design:

An Interventional study (Pre and post design) was conducted on the residents of the Family Medicine department of the Aga Khan University hospital Karachi.

Methodology:

After getting their consent, pre intervention Objectively structured clinical examination (OSCE) was conducted at the Center for Innovation in medical education, AKUH. The scenarios were based on urgent care problems presenting in the Family Medicine setting. It was followed by the case based simulation teaching intervention by the facilitators and debriefing. The post intervention OSCE was conducted in order to assess the resident’s performance. Statistical Analysis: The data was analyzed in Stata version15 software in two stages; descriptive and inferential. In descriptive analysis frequency and proportion were calculated for categorical variables. Median and inter quartile range were reported for continuous variable. Paired T-tests were applied to compare the pre and post test results.

Results:

The resident’s scores significantly improved after case based simulation in majority of the post intervention OSCE stations proving the effectiveness of the intervention.

Conclusion:

Case based simulation is an effective teaching strategy for the learning process of the Family Medicine residents regarding the urgent care management skills. It is advised to use this strategy in the teaching and learning process of other Family medicine residency programs.

Introduction

Simulation is increasingly used in healthcare education to teach to acquire knowledge, and psycho-motor and affective skills to individuals and teams.[1] It is a generic term that refers to “an artificial representation of a real-world process to achieve educational goals through experiential learning.” Simulation-based medical education is defined as any educational activity that utilizes simulation tools to replicate clinical scenarios.[2] It creates the opportunity for practical and relevant presentations, accurate tests, more student interest, and can be easily modified according to individual skill needs.[3]

Case-based simulation is an educational activity that utilizes simulation aides to replicate clinical scenarios and helps in overcoming patient safety concerns. The trainees can practice on simulated patients, mannequins, and simulators. It also allows repetitive practice to reduce errors.[4] Simulation-based teaching offers a structured, learner-centered environment where students can practice their abilities without causing any harm to the patient.[5] A study conducted in Nepal on the effectiveness of in situ simulation in the emergency department to explore its impact on perception and learning experience among multidisciplinary healthcare professionals suggested improved self-assurance and teamwork after simulation.[6]

A similar study conducted on the effect of simulation-based emergency cardiac effect simulation on nursing students showed improvement in their self-efficacy and critical thinking. Another similar study showed that simulation-based training is an effective model for teaching residents’ best practices in family-centered rounds with a lasting impact on resident communication skills.[7,8]

It also improves the confidence and competence skills in managing the patients, especially for undergraduate as well as postgraduate trainees. A study conducted on the junior doctors showed that simulation-based teaching facilitated the transition from medical students to junior doctors.[3]

Family Medicine is the primary care medical specialty concerned with the provision of comprehensive healthcare to the individual and the family. The residency program is a four-year training program to prepare Family Physicians to provide broad-based healthcare at the Aga Khan University Hospital, Karachi, Pakistan. It had been observed through evaluations that although residents have enough exposure and training to achieve urgent care management skills since they have rotations in the various specialties, emergency care, and family medicine clinics, they were not competent to achieve those skills. This could be explained by the fact that although the residents have structured training programs in which they have rotations in various clinical specialties such as internal medicine, general surgery, pediatrics, obstetrics and gynecology, and various other specialties, they might not been able to come across those acute presentations when they were rotating in their perspective rotations or had limited exposure in that period. Moreover, they get the chance to manage patients who are already been stabilized in the emergency room. Hence, they do not get the exposure of the first-line emergency management which is already done before the shifting of patients. That is why they were not competent to achieve those skills.

The case-based simulation was found to be an effective strategy to enhance the teaching and learning process for the residents since it helps to explore the real-life problems in the simulated environment and enables them to develop a management plan. In simulation-based Objectively Structured Clinical Examination (OSCE), all candidates are presented with the same clinical tasks, under a variety of simulated conditions to be completed in the same timeframe and are scored using structured marking schemes. Feedback given after the OSCE allows the learners to understand exactly what went wrong and how they can improve.[9,10]

Therefore, a case-based simulation teaching strategy was selected to teach the residents to manage the urgent care problems seen in the Pakistani context. The assessments were performed by OSCEs.

Objective of the Study

To evaluate the effectiveness of case-based simulation teaching to improve the family medicine resident’s urgent care management skills as assessed by an OSCE at a teaching hospital.

Study Design

It was an interventional study (predesign and postdesign).[12]

Subjects and Methods

The study was conducted on the Family Medicine residents at the Center for Innovation in Medical Education (CIME) of The Aga Khan University Hospital, Karachi. CIME is the most advanced simulation-based healthcare teaching and learning institute in Pakistan. All the residents currently enrolled and present in the department were included in the study after obtaining their consent. Those who did not give consent were not included in the study, however, they were taught simulation-based teaching. The study duration was twelve months after the ERC approval (ERC# 31596-20399). It was a purposive sampling method since all residents who had given consent were included in the study. Expected Outcomes: Improvement in the (OSCE) scores of residents after intervention (case-based simulation teaching). Since the residents were from year 1 to year 4, the passing percentage was different according to their experience in the Family Medicine residency program. It was 50% for year 1, 60% for year 2, 70% for year 3, and 80% for year 4 residents. The table of specification (TOS) was formulated considering the most relevant urgent care problems seen in the Pakistani setting. Ten OSCE stations were prepared and reviewed by multiple faculty members for validity. The residents were assessed by multiple assessors to maintain reliability. The duration of each station was 10 min. The facilitators were briefed before the examinations regarding the rating scale and the OSCE stations. For each OSCE station, the candidate instructions, simulated patient instructions, and the marking grids were formulated. Scoring was done based on the rating scale as mentioned in the marking grid.[13]

Each OSCE station was divided into components of consultation skills, data gathering skills, examination skills, and management skills. Each skill was scored from excellent to poor on a global rating score approved by the Family Medicine department in consensus with the Department of Education and has been used since ten years.[11] The grades awarded for each skill and the conversion into number is as follows: Excellent = 4, Competent = 3, Unsatisfactory = 2, Poor = 1, Not done = 0 on a Likert scale. The assessors used the grades while marking for each skill. Later on, it was transformed into number form for the sake of analysis. Written consent was taken from all the participatory residents. The residents who had given consent participated in the preintervention and postintervention OSCEs. Pretest OSCE was conducted at the CIME following the coronavirus disease (COVID-19) standard operating procedures (SOPs). The simulated patients were trained by the facilitators.

Case-based simulation teaching was conducted on all the residents regardless of the fact they consented or not. It involved case-based simulation on different topics assessed in the pretest OSCE on the simulated patients. Different facilitators were involved in the teaching including the principal investigator. The presentations were prepared and reviewed by the content experts. The teaching duration was for three days including time for practice. Simulated patients, moulage, mannequins, and computer-operated mannequins were used for simulated teaching.

The intervention was conducted just after the pretest for two days. Debriefing sessions were conducted and feedback was provided to all the residents on their performances and reflective thinking was encouraged. The residents were also given some time to practice on the simulated patients before the postintervention OSCE. Immediately after the intervention, postintervention OSCE was conducted in a similar method as that of pre-OSCE.

The OSCE stations’ main themes were similar for preintervention and postintervention OSCEs. Seven out of ten assessors were the same in both the preintervention and postintervention OSCEs. The posttest OSCE was conducted in the same way as the pretest OSCE. Residents were given feedback regarding their performance. Each OSCE station was divided into components of consultation skills, data gathering skills, examination skills, and management skills, and checklists were created for each component. Each skill was scored on a global rating score approved by the Family Medicine in consensus with the Department of Education.[18,19]

Results and Analysis

The range of age of the research participants was from 26 to 36 years with a mean age of 30 years. The majority of the participants were females (85%). The clinical experience of the participants ranged from 2 to 9 years with a mean of 4.6 years. The years of graduation of the participants varied from two to nine years with a mean of six years [Table 1].

Table 1.

Details of the participants demographics

Participants	Demographics
Males n (%)	2 (14.29)
Females n (%)	12 (85.71)
Age: Mean years	30.28±2.89
Clinical experience years	4.64±2.01
Year of graduation range	2008-2019

The data was analyzed in Stata 15 version software and was analyzed in two stages of descriptive and inferential. In descriptive analysis, frequency and proportion were calculated for categorical variables. Median and interquartile ranges were reported for continuous variables. In addition, paired T-tests were applied to compare the pretest and posttest results. A P-value of <0.05 was considered significant. A total of 14 residents participated in the study. The range of age of the research participants was from 26 to 36 years with a mean age of 30 years. The majority of the participants were females (85%). The clinical experience of the participants ranged from 2 to 9 years with a mean of 4.6 years. The years of graduation of the participants varied from two to nine years with a mean of six years.

History taking, relevant physical examination, and the management of various urgent care skills were assessed.[14] Regarding the first clinical skill which was hypertensive urgency management skill, the two-sided (two-tailed) t-test P-value was 0.001 which indicated that knowledge uptake, relevant physical examination skills, and management skills regarding hypertensive urgency were significantly improved after the intervention [Table 2].

Table 2.

Statistical Analysis of clinical skills stations

Clinical Skills (History taking, physical examination and management)	Mean pre-test	95% Confidence Interval	Mean Post-test	95% Confidence Interval	t Statistics	P
Hypertensive urgency	2	1.4-2.5	3.1	2.8-3.5	-3.89	0.001
Hypoglycemia	2.2	1.6-2.8	3	3-3	-2.80	0.015
Pediatric Diarrhea	2.14	1.54-2.73	3.14	2.93-3.35	-3.17	0.007
Acute Asthma	1.85	1.30-2.40	2.92	2.77-3.08	-4.02	0.001
Acute Psychosis	2.28	1.66-2.90	3.14	2.93-3.35	-3.12	0.008
Backache	2.07	1.49-2.64	3.21	2.96-3.46	-3.88	0.001
Burn injury	1.85	1.30-2.40	3.57	3.27-3.86	-6.44	0.000
Fever (Dengue)	2.7	2.06-3.50	3.5	3.27-3.86	-2.34	0.035
Anaphylactic reaction	2.30	1.63-2.97	3.07	2.90-3.24	-2.132	0.054
Acute Abdomen	2.8	2.11-3.60	3.6	3.35-3.92	-2.064	0.059

The second clinical skill comprised of management of hypoglycemia. The results showed that hypoglycemia management skills were also significantly improved after simulation-based teaching as indicated by the t-test P-value of 0.001. The third clinical skill on the history taking, physical examination, and management of pediatric diarrhea showed significant improvement after the intervention as indicated by the P-value of 0.0072 [Table 2].

Fourth clinical skills on history taking, relevant physical examination, and management of acute exacerbation of asthma showed significant improvement after simulation-based teaching as indicated by the t-test P-value of 0.0015 [Table 2].

Fifth clinical skill on the relevant history taking, physical examination, and the management of acute psychosis also significantly improved after intervention as indicated by the t-test P-value of 0.0081 [Table 2].

Similarly, the sixth skill on the relevant history taking, physical examination, and management of backache was significantly improved by the simulation-based teaching as indicated by the t-test P-value of 0.001. A significant improvement in the relevant history taking, physical examination, and management of burn injury (seventh skill) was seen by the t-test P-value of 0.000 after the intervention. Eighth skill on the relevant history taking, physical examination, and management of fever was also significantly improved after intervention as shown by the t-test P-value of 0.035 [Table 2].

Ninth clinical skill on the relevant history taking, physical examination, and management of anaphylactic reaction was not improved after simulation-based teaching as shown by the t-test P-value of 0.054 [Table 2].

Tenth clinical skill on the relevant history taking, physical examination, and management of acute abdomen was also not improved after the simulation-based teaching as indicated by the t-test P-value of 0.059 [Table 2].

Pre-OSCE and post-OSCE reliabilities of the stations varied from 0.5 to 1, showing good reliability at the majority of the stations.

Discussion

The overall results of the study showed that the knowledge and management skills of the urgent care problems increased after the intervention; this is supported by the evidence in the literature that through case-based simulation, students can acquire new skills, improve their capacity for clinical reasoning, and prepare themselves to provide patient care in a secure setting.[15,16,17]

The results of the study showed that the resident’s performance was significantly improved in the management of skills such as burns (P-value: 0.000), acute asthma (P-value: 0.001), hypertensive urgency (P-value: 0.002), backache (P-value: 0.002), pediatric diarrhea (P-value: 0.007), acute psychosis (P-value: 0.008), Hypoglycemia (P-value: 0.015), and management of fever (P-value: 0.035) since the P-value is less than 0.05 which is statistically significant indicating the effectiveness of the case-based simulation intervention. This could be explained by the fact that the trainees were not exposed to those problems in their training and after the intervention they were competent in managing those problems.

Few of the case-based simulation stations did not show any improvement after the intervention. This can be explained by the fact that the residents in the first year of the rotation had surgical rotation and they had learned to manage the cases in that rotation. That is why they were competent in the pretest OSCE as well as in the posttest OSCE.

Similarly, the management skill of the anaphylactic reaction was not improved after the simulation-based intervention. This can be explained by the fact that the residents had an Internal Medicine rotation in the first year of the training and they had learned management skills in those rotations, so they performed well in the pretest OSCE as well as in the posttest OSCE. Another reason could be the small size of the participants.

Our study showed that the postintervention OSCE scores of the residents in the initial years of training (year one and year two) were significantly improved after case-based simulation. This has been supported by literature in which undergraduates’ and junior residents’ clinical and management skills improved after simulation-based teaching.[16]

The result of the study shows that perhaps adding case-based simulation along with other strategies can improve the knowledge and management skills of the Family Medicine trainees. This is supported by a similar study performed on undergraduate medical students in their Psychiatry rotation which showed improvement in their knowledge and confidence.[17]

The residents with greater experience (years 3 and 4) performed well in both preintervention and postintervention OSCE as evidenced by the global scoring in their marking grids. This can be explained by the fact that they had been trained in the past regarding those clinical problems and they were already competent in the management skills of those problems.

Conclusion

Based on the study findings, it is concluded that case-based simulation is an effective teaching strategy for the learning process of the Family Medicine residents regarding urgent care management skills as assessed by the OSCE.

Recommendations

1. The urgent care skills should be taught in the earlier years of the training so that residents are competent to manage the patients in their clinical rotations.

2. Results can be used as a model for other residency programs in the country to use case-based simulation strategies for the teaching and learning process of the residents.

3. The results of such studies can be disseminated to enhance awareness and to facilitate the information of such programs which can bring positive change in the healthcare system.

Ethical approval

Ethical review committee approval was taken. Participation was voluntary and informed consent was taken from all the participants. All the residents currently in the department had simulation-based teaching of urgent care skills (regardless of giving consent and participation in the pretests and posttests). Confidentiality of data was maintained, and coding was done. The scores of this study were not included in the academic performance assessment.

Limitations

1. Few facilitators in the postintervention OSCE were different from preintervention OSCE.

2. There was a gap of three days between intervention and post-OSCE due to the non-availability of venue on weekend days.

3. Few of the participants did not volunteer for preintervention OSCE but they attended postintervention, so their results were not included.

4. The sample size was small so the results cannot be generalized.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.