Abstract
Introduction
Pre-clerkship procedural skills training is not yet a standard across Canadian medical school curricula, resulting in limited exposure to procedures upon entering clerkship. While simulated skills training has been documented in the literature to improve performance in technical ability, anxiety and confidence have yet to be investigated despite their documented impact on performance and learning. This study therefore aims to evaluate the effect of pre-clerkship procedural skills training on medical student anxiety and confidence.
Methods:
A procedural skills training program was designed based on an evidence-based near-peer, flipped classroom model of education. Ninety-two second-year medical students volunteered for the study. Fifty-six were randomized to the training group, and 36 were randomized to the control group. Students in the training group attended seven procedural skills tutorials over seven months. The control group represented the average medical school student without standardized procedural training. Student anxiety and confidence were assessed at the beginning and end of the program using the State Trait Anxiety Inventory and Confidence Questionnaires.
Results:
Students who participated in the procedural skills program demonstrated greater reductions in their state anxiety and greater improvements in confidence compared to the control group.
Conclusion:
Longitudinal procedural skills training in the simulation setting has demonstrated improvements in anxiety and confidence among pre-clerkship medical students. These added benefits to training have the potential to ease medical students’ transition into clerkship, while also contributing to a safer and more effective clinical experience. Therefore, future integration of standardized pre-clerkship procedural skills training within medical school curricula should be considered.
Abstract
Introduction:
Les cursus de pré-externat des facultés de médecine canadiennes ne prévoient pas systématiquement de formation sur les habiletés techniques avant, si bien qu’à leur arrivée à l’externat, les étudiants ont une exposition limitée aux gestes techniques. Tandis que l’amélioration des aptitudes techniques par la simulation est bien documentée, il n’y a aucune étude sur l’effet de ces simulations sur l’anxiété et la confiance malgré leur impact avéré sur la performance et l’apprentissage. Cette étude vise donc à évaluer l’effet d’une formation sur les habiletés techniques au pré-externat sur l’anxiété et la confiance des étudiants en médecine.
Méthodes:
Une formation sur les habiletés techniques a été conçue sur la base d’un modèle d’enseignement fondé sur les données probantes, par les quasi-pairs suivant une approche de classe inversée. Quatre-vingt-douze étudiants en deuxième année de médecine ont participé à l’étude sur une base volontaire. Cinquante-six d’entre eux ont été répartis au hasard dans le groupe qui devait recevoir la formation, et 36 dans le groupe contrôle. Les étudiants du groupe qui a reçu la formation ont assisté à sept tutoriels étalés sur une période de sept mois. Le groupe témoin représentait l’étudiant en médecine moyen sans formation normalisée axée sur les habiletés techniques. L’anxiété et la confiance des étudiants ont été évaluées au début et à la fin du programme à l’aide du questionnaire State Trait Anxiety Inventory (questionnaire sur l’anxiété chronique et réactionnelle) et d’un questionnaire sur la confiance.
Résultats:
Chez les étudiants qui ont participé au programme de formation sur les habiletés techniques, la baisse de l’anxiété et l’amélioration de la confiance en soi ont été plus importantes que chez les étudiants du groupe contrôle.
Conclusion:
La formation longitudinale axée sur les habiletés techniques en contexte de simulation a eu des effets positifs en ce qui concerne l’anxiété et la confiance chez les étudiants en médecine au pré-externat. La formation offre l’avantage supplémentaire de faciliter la transition des étudiants en médecine vers l’externat, tout en contribuant à rendre l’expérience clinique plus sûre et plus efficace. Il serait donc intéressant pour les facultés de médecine d’intégrer dans le cursus une formation normalisée axée sur les habiletés techniques au pré-externat.
Introduction
The purpose of pre-clerkship is to prepare students to participate in direct patient care during clerkship.1 However, this transition often comes with substantial stressors.1–3 When asked directly, students transitioning to clerkship are most anxious about performing common procedural skills on patients.4–7 In fact, by the end of medical school, graduating students continue to perceive a gap between desired and actual procedural skill competency for basic skills.8
Not surprisingly, the resultant anxiety and lack of confidence has been associated with impaired learning and performance.9,10 Heightened anxiety has been shown to negatively affect learning and limit a student’s ability to apply knowledge.9,11 Moreover, according to White et. al, lack of self-confidence and the anxiety associated with fear of making mistakes leads to attentional narrowing resulting in less concentration on the task and the patient.10
In order to facilitate this transition into clerkship and optimize student learning and performance of procedural skills during clinical rotations, varying learning modalities are essential. Utilization of simulation training is an efficacious modality for procedural skills. There is evidence to suggest that simulation training enhances procedural skill performance compared to no training or standard clinical training.12 For novice learners developing the pieces of comprehensive procedural care, low-fidelity task trainers seem effective in this teaching. Moreover, longitudinal training and spaced repetition have shown to contribute to a high degree of procedural competence over time.13–16 A flipped classroom approach is also a beneficial addition to a training program as it significantly improves learner performance when compared to classroom-based teaching.17 Finally, procedural skill instruction by senior medical students, known as a near-peer model, is as effective for procedural skill acquisition as the instruction offered by staff physicians,13 while also developing the educational ability of student teachers.18
Despite these benefits, we did not find any standardized simulated pre-clerkship procedural skills training program available for medical students across Canada.4,5 In a previous study by our research team, we identified the top ten procedural skills medical students should know how to perform by graduation.19
The paucity of published material about the impact of procedural skills training on medical student anxiety and confidence elucidates a niche for further investigation.12 The goal of our study was to assess the impact on anxiety and confidence of a flipped classroom, near-peer procedural skills training program carried out in a simulated setting for pre-clerkship medical students.
Materials & methods
This program was reviewed by the Ottawa Hospital Research Institute Ethical Review Board (OHRI ERB) and was granted an REB exemption for curriculum development and investigation purposes.
Participants
All 160 students in second year were sent a recruitment email detailing the skills included in the training program, as well as the purpose of the research project to measure anxiety and confidence related to procedural skills. Students volunteered knowing they would later be randomized to either the training or control group. All students completed a State-Trait Anxiety Inventory (STAI)20 and a confidence questionnaire7 prior to randomization.
There were 92 applicants, with 56 applicants (the maximum that could be trained in the simulation facility at one time) randomly selected using a random number generator to form the training group. The remaining 36 students were assigned to the control group. The discrepancy in numbers between the program and control group were such that the maximum number of second-year students could be trained in procedural skills.
Program logistics
The training group was required to attend at least six of seven procedural skill training sessions, each one teaching a single new skill (refer to Table 1 for the program schedule). Training occurred in one-month intervals from September 2019 to March 2020 at the University of Ottawa Skills and Simulation Centre. The control group participants, who did not attend these formal sessions, represented the common medical student experience in Ottawa, without standard pre-clerkship procedural skills training in any of the skills taught. Of note, as part of the common medical student experience, university interest groups intermittently held informal skill training events for limited numbers of students. Participants were not excluded from the study if they had attended any of these informal clinical skills training events.
Table 1.
Procedural Skills Taught and Practiced in 2019-2020
| Skill Taught | Previous Skills Available for Practice | |
|---|---|---|
| Month 1 | Suturing | - |
| Month 2 | Airway Management | Suturing |
| Month 3 | IV Insertion | Airway Management Suturing |
| Month 4 | Local Anesthetic | IV Insertion Airway Management Suturing |
| Month 5 | Foley Catheterization | Local Anesthetic IV Insertion Airway Management |
| Month 6 | Phlebotomy | Foley Catheterization Local Anesthetic IV Insertion |
| Month 7 | Nasogastric tube insertion | Phlebotomy Foley Catheterization Local Anesthetic |
A near-peer, flipped classroom model of education was used as the foundation for the procedural skills program. By “near-peer” we mean skill instruction by clerkship students who previously completed the pilot program in the year prior. Their role included serving as tutors to the new second-year students by demonstrating procedural techniques and providing individual student feedback. Two tutors per skill session were recruited to maintain an average student to tutor ratio of 12:1. Clerkship student tutors were supervised by a resident proficient in those specific skills taught that session.
Prior to each session, students were required to read a skill-specific handbook, specially designed for the program, and edited by a physician proficient in that skill. Following reading of the handbook, students had to complete a formative online quiz.
At each training session, students were shown how to perform a single new skill by their clerkship student tutors. Tutors provided step-by-step instruction on simulation models, after which the students practiced the skill and were provided with individual feedback on their technique. The last hour of each session was reserved for practice of previously learned skills to consolidate this prior knowledge and further strengthen technique.
Data analysis
The primary outcomes of this study include pre- and post-program self-reported measures of anxiety and confidence related to procedural skills performance. Anxiety and confidence were assessed in both groups prior to the start and at the end of the program using the State-Trait Anxiety Inventory (STAI) questionnaire20 and a validated confidence questionnaire (Appendix A).21 The STAI questionnaire assessed overall anxiety regarding performance of procedures, while the confidence questionnaire assessed confidence as it pertained to each individual skill.
Data analysis was performed using SPSS Statistics Version 25.0.0.0.
A two tailed independent t-test was conducted on the difference in pre- and post-program state anxiety scores between groups to determine whether anxiety improved among trained students.
A two (group: intervention, control) by seven (individual skills) mixed measures ANOVA was conducted on the difference in pre- and post-program confidence scores. To further examine this interaction, Bonferroni-corrected pairwise comparisons were computed for each skill, comparing the difference in pre- and post-program confidence scores between the groups, thereby determining the subset of skills for which the trained students showed improved confidence. Levene’s Test was used to check the assumption of homogeneity of variance.
Results
Fifty-three students in the training program (95%) and 35 control group participants (97%) completed STAI and confidence questionnaires pre-randomization and post-program. One control group student and three training group students withdrew during the study. The control group was planned to have a consolidated training period at the end of the study (however, due to the COVID-19 pandemic, was not completed).
The t-test for state anxiety showed that the differences between the pre- and post- anxiety scores are statistically significant (p < 0.001, Cohen’s d = 0.90) between groups. Specifically, the procedural skills training group showed a reduction in state anxiety of -16.11 ± 4.010 which is statistically significantly different from the control group -3.87 ± 4.210 (Figure 1A).
Figure 1.
Change in anxiety and confidence pre- versus post-program between control and training groups. The training group took additional procedural skills training. The control group did not undergo such training, representing the average medical student. Anxiety scores were derived using the STAI questionnaire. Confidence scores were derived using a previously validated self-reported questionnaire.
The 2x7 ANOVA demonstrated that there was a significant interaction between group and skill confidence [F(2.910) = 0.009, p < 0.05]. Pairwise comparisons for the interaction using Bonferroni adjustments showed a significant difference between control and intervention groups across all skills (all pbonferroni < 0.001, all Cohen’s d > 0.97), except for nasogastric tube insertion which was not significantly different after accounting for multiple comparisons. Levene’s Test was significant, but the sample size was large enough (i.e., > 30) to render ANOVA robust to the violation of the assumption of homogeneity of variance. Mauchly’s Test of Sphericity was not significant, indicating similar variance across the repeated measures (0.673 with df=20 and p = 0.208).
More specifically, while confidence increased in both the training and control groups, there was a statistically significant difference in this increase (p < 0.001) between the two groups, with students in the training group having a higher increase in confidence in airway management [control 0.14 ± 0.241 vs training 1.35 ± 0.300], IV insertion [control 0.32 ± 0.320 vs training1.70 ± 0.316], local anesthetic [control 0.20 ± 0.276 vs training 1.86 ± 0.322], suturing [control 0.14 ± 0.270 vs training 1.14 ± 0.327], foley catheterization [control 0.11 ± 0.160 vs training 1.30 ± 0.329] and phlebotomy [control 0.00 ± 0.180 vs training 1.40 ± 0.356] (Figure 1B-H).
Discussion
This study evaluated the anxiety and confidence in students who participated in the program compared to untrained participants in the control group. Students who completed the program experienced a statistically significant reduction in state anxiety. Moreover, statistically significant improvements in student confidence were observed in the training group compared to the control across skills, except for nasogastric tube (NGT) insertion.
Although there was an improvement in confidence after NGT insertion training, the change was not significantly different from the control group. This is likely attributable to the fact that NGT insertion was the last session prior to the abrupt culmination of the program due to the COVID-19 pandemic. Students were therefore unable to get the same amount of practice as they did for those skills introduced earlier.
Despite no formal training in the control group, a slight increase in confidence across skills was observed. This may have been the result of medical student participation in informal student-led workshops that are occasionally offered throughout the year and have limited seating.
There are two key future goals that should be pursued to advance procedural skills training in medical education. Firstly, future research should attempt to objectively assess and then compare procedural skill competency before and after training in our program. Additionally, it would be useful to gather data from the current cohort of students at the end of medical school to assess if the effects of this program on anxiety and confidence were sustained or even accelerated.
There are notable limitations to our study. While efforts were made to select similar control and experimental groups through randomization, the educational experience and training background between the two groups may be varied. Although the results were significant, there were large standard errors, which may be attributed to the possible variability in training background among the students and the limited sample size. Ideally, we would have included all students in the medical student cohort; however, it was not feasible to incorporate this as mandatory training due to limited resources (teaching space, resident supervision, scheduling allotments) at the time.
Conclusion
There is limited longitudinal procedural skills training in Canadian medical school curricula. With a simulation-based near-peer training program, medical students not only practice technical procedural skills, but also appear to experience reduced anxiety and improved confidence when performing these skills that can ease student transition to clerkship and contribute to more effective student learning experiences. Therefore, future integration of standardized pre-clerkship procedural skills training within medical school curricula should be considered.
Acknowledgments
The authors would like to acknowledge the participants for their time and effort, as well as the undergraduate medical student tutors, the resident-physician tutors, and the staff at the University of Ottawa Skills and Simulation Centre without whom this study would not have been possible.
Appendix A.
Anxiety and confidence scores are given using the State Trait Anxiety Inventory. Experimental group refers to training program, while control group students did not undergo training.
Table 2.
Anxiety and confidence for pre- and post- training for 2019-2020
| Group | N (Pre-Training) | Mean (Pre- Training) | N (Post-Training) | Mean (Post- Training) | |
|---|---|---|---|---|---|
| How confident do you feel presently performing suturing on a patient in a clinical setting? | Control | 34 | 1.71 | 29 | 1.72 |
| Experimental | 50 | 1.84 | 50 | 3.00 | |
| How confident do you feel presently performing airway management on a patient in a clinical setting? | Control | 33 | 1.52 | 29 | 1.52 |
| Experimental | 48 | 1.40 | 49 | 2.80 | |
| How confident do you feel presently performing IV insertion on a patient in a clinical setting? | Control | 33 | 1.42 | 29 | 1.59 |
| Experimental | 46 | 1.24 | 49 | 3.04 | |
| How confident do you feel presently performing local anesthetic on a patient in a clinical setting? | Control | 30 | 1.40 | 29 | 1.52 |
| Experimental | 46 | 1.37 | 49 | 3.35 | |
| How confident do you feel presently performing phlebotomy on a patient in a clinical setting? | Control | 31 | 1.32 | 29 | 1.24 |
| Experimental | 46 | 1.35 | 49 | 2.82 | |
| How confident do you feel presently performing nasogastric tube insertion on a patient in a clinical setting? | Control | 32 | 1.06 | 29 | 1.21 |
| Experimental | 46 | 1.20 | 48 | 2.31 | |
| How confident do you feel presently performing foley catheter insertion on a patient in a clinical setting? | Control | 32 | 1.34 | 29 | 1.34 |
| Experimental | 46 | 1.35 | 48 | 2.67 | |
| Anxiety State | Control | 34 | 59.5294 | 30 | 56.6000 |
| Experimental | 51 | 57.5294 | 51 | 41.4118 | |
| Anxiety Trait | Control | 34 | 38.3529 | 30 | 39.0000 |
| Experimental | 51 | 39.1373 | 51 | 38.4902 | |
Conflicts of Interest
None
Funding
None
Authorship
Victoria Ivankovic, Maria Merlano, Vishesh Patel, Céline Sayed, and Hao Wang contributed equally to the work.
References
- 1.Houpy JC, Lee WW, Woodruff JN, Pincavage AT. Medical student resilience and stressful clinical events during clinical training. Med Educ Online. 2017;22(1). 10.1080/10872981.2017.1320187 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Johnson NR, Pelletier A, Chen X, Manning-Geist BL. Learning in a high-stress clinical environment: stressors associated with medical students’ clerkship training on labor and delivery. Teach Learn Med. 2019;31(4):385-392. 10.1080/10401334.2019.1575742 [DOI] [PubMed] [Google Scholar]
- 3.Lin C Der, Lin BYJ. Training demands on clerk burnout: Determining whether achievement goal motivation orientations matter. BMC Med Educ. 2016;16(1). 10.1186/s12909-016-0742-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.The AAMC Project on the Clinical Education of Medical Students – Clinical Skills . DC Assoc Am Med Coll. 2004. https://www2.kumc.edu/PDFATraining/Admin/documents/Med%20Ed%206.12.09%20The%20AAMC%20Project%20on%20the%20Clin%20Educ%20of%20Med%20Stud%20Clin%20Sk%20Edu.pdf
- 5.The Association of Faculties of Medicine of Canada . procedural skills. AFMC national clinical skills working group evidence-based clinical skills document. http://clinicalskills.machealth.ca/index.php/content/procedural
- 6.Moss F, McManus IC. The anxieties of new clinical students. Med Educ. 1992;26(1):17-20. 10.1111/j.1365-2923.1992.tb00116.x [DOI] [PubMed] [Google Scholar]
- 7.Stewart RA, Hauge LS, Stewart RD, Rosen RL, Charnot-Katsikas A, Prinz RA. A CRASH course in procedural skills improves medical students’ self-assessment of proficiency, confidence, and anxiety. Am J Surg. 2007;193(6):771-773. 10.1016/j.amjsurg.2007.01.019 [DOI] [PubMed] [Google Scholar]
- 8.Dehmer JJ, Amos KD, Farrell TM, Meyer AA, Newton WP, Meyers MO. Competence and Confidence With Basic Procedural Skills: The Experience and Opinions of Fourth-Year Medical Students at A Single Institution. Acad Med. 2013;88(5):682-687. 10.1097/ACM.0b013e31828b0007 [DOI] [PubMed] [Google Scholar]
- 9.Rhodes ML, Curran C. Use of the human patient simulator to teach clinical judgment skills in a baccalaureate nursing program. CIN-Comput Informatics Nurs. 2005;23(5):263-264. 10.1097/00024665-200509000-00010 [DOI] [PubMed] [Google Scholar]
- 10.White AH. Fourth-Year Nursing Students: An Interpretive Study. J Nurs Educ. 2003;42(3):113-120. [DOI] [PubMed] [Google Scholar]
- 11.Nielsen B, Harder N. Causes of student anxiety during simulation: What the literature says. Clin Simul Nurs. 2013;9(11):e507-e512. 10.1016/j.ecns.2013.03.003 [DOI] [Google Scholar]
- 12.Lynagh M, Burton R, Sanson-Fisher R. A systematic review of medical skills laboratory training: where to from here? Med Educ. 2007;41(9):879-887. 10.1111/j.1365-2923.2007.02821.x [DOI] [PubMed] [Google Scholar]
- 13.Phillips AW, Matthan J, Bookless LR, et al. Individualised expert feedback is not essential for improving basic clinical skills performance in novice learners: a randomized trial. J Surg Educ. 2017;74(4):612-620. 10.1016/j.jsurg.2016.12.003 [DOI] [PubMed] [Google Scholar]
- 14.Hu Y, Kim H, Mahmutovic A, Choi J, Le I, Rasmussen S. Verification of accurate technical insight: a prerequisite for self-directed surgical training. Adv Heal Sci Educ. 2014;20(1):181-191. 10.1007/s10459-014-9519-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Routt E, Mansouri Y, De Moll EH, Bernstein DM, Bernardo SG, Levitt J. Teaching the simple suture to medical students for long-term retention of skill. JAMA Dermatology. 2015;151(7):761-765. 10.1001/jamadermatol.2015.118 [DOI] [PubMed] [Google Scholar]
- 16.Bosse HM, Mohr J, Buss B, et al. The benefit of repetitive skills training and frequency of expert feedback in the early acquisition of procedural skills. BMC Med Educ. 2015;15(1):1-10. 10.1186/s12909-015-0286-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Hew KF, Lo CK. Flipped classroom improves student learning in health professions education: a meta-analysis. BMC Med Educ. 2018;18(38):1-12. 10.1186/s12909-018-1144-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Dandavino M, Snell L, Wiseman J. Why medical students should learn how to teach. Med Teach. 2007;29(6):558-565. 10.1080/01421590701477449 [DOI] [PubMed] [Google Scholar]
- 19.Battaglia F, Sayed C, Merlano M, et al. Identifying essential procedural skills in Canadian undergraduate medical education. Can Med Educ J. 2020. 10.36834/cmej.68494 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Metzger RL. A reliability and validity study of the state-trait anxiety inventory. J Clin Psychol. 1976;32(2):276-278. [DOI] [Google Scholar]
- 21.Stewart RA, Hauge LS, Stewart RD, Rosen RL, Charnot-Katsikas A, Prinz RA. A CRASH course in procedural skills improves medical students’ self-assessment of proficiency, confidence, and anxiety. Am J Surg. 2007;193(6):771-773. 10.1016/j.amjsurg.2007.01.019 [DOI] [PubMed] [Google Scholar]