Abstract
Background
In 2020, the global COVID-19 pandemic caused educational disruptions to many medical students nationally. Societal and hospital guidelines, including social distancing protocols, resulted in the cancellation or postponement of many elective procedures. A shortage in personal protective equipment also contributed to restrictions in clinical experiences for trainees. The purpose of this study was to determine resident-perceived preparedness in core clinical competencies and evaluate the disruptions to core clerkships.
Methods
A survey was developed to assess self-perceptions of clinical competencies and disruptions to core clerkship experiences. It was distributed to 63 incoming psychiatric residents who matched to training programs in the United States.
Results
The survey response rate was 97%. The majority of respondents achieved self-expected levels of proficiency in clinical skills. Deficits were greatest for pelvic/rectal exams and transitions of care. Most students did not experience disruptions to clerkships. Internal medicine, obstetrics, and gynecology clerkships reported the highest rates of virtual completion. Procedures with the lowest reported perceived preparation were arterial puncture, airway management, and IV placement, respectively.
Conclusion
Our survey results indicated that most learners did not perceive disruptions to their medical education and incoming psychiatry residents felt well-prepared to start residency. Some specific procedural skills appear to have been affected. Attempts to mitigate these specific inadequacies may help mitigate disruptions due to future events.
Keywords: psychiatry, COVID-19, survey, medical students, residents, health care, graduate medical education, internship and residency, pandemics, psychology
Background
Since January 2020, COVID-19 has spread into a global pandemic with over 767 million confirmed cases and over 6.9 million deaths.1 Despite social distancing strategies, disruptions occurred in the health care system, necessitating changes to medical education, including suspension of clinical rotations, transition to remote classrooms, and restriction of travel for electives.2 These changes altered the educational experience during a crucial phase in the development of clinical skills by medical school students. The effect of these changes on resident preparedness is poorly understood.
Early indicators suggested negative perceptions regarding the effect of the pandemic on medical training.3,4 The majority of students reported wanting to return to clinical rotations.5 The shift in online learning lead to disproportionate deficits in the perceptions of clinical skills relative to medical knowledge.6–8 Dental students reported similar disruptions in the acquisition of clinical knowledge; however, interventions ameliorated this effect.9 The available evidence suggests that the effect of the pandemic on medical training was most pronounced for clinical experiences. Much of the previous work has been conducted outside the United States or on other related populations and therefore has limited generalizability to medical students in the United States.
Perception of competence impacts both learners and the health care system. The appraisal of competence shapes behavior and influences reality.10 According to self-determination theory, an interplay exists between 3 fundamental psychological needs, which are competence, autonomy, and relatedness.11 Evidence is abundant that fostering competence improves well being and performance at both individual and organizational levels.12,13
While the quality of medical training during the pandemic is still unknown, student perceptions of competence inform the quality of medical education. Understanding any deficits is crucial not just in redressing the effects of the pandemic but also in guiding medical education during future events. The purpose of our study was to determine the perception of clinical preparedness among incoming psychiatry residents whose clinical education had been interrupted by the pandemic.
Methods
A cross-sectional survey, constructed by the authors with input from a research psychologist with expertise in survey design, was developed to determine self-perceptions of competency following disruptions to residents’ medical education due to the pandemic (Appendix 1). This novel survey had not been previously validated; however, it was constructed with iterative item generation and content validation from multiple authors with extensive graduate medical education experience until consensus on the final items was reached. Due to time constraints, regarding the pandemic, it was not piloted prior to administration. The cross-sectional survey asked incoming psychiatry residents to estimate their perceived clinical readiness on 13 core clinical skills and 6 core procedures. These clinical skills and procedures were drawn from the Association of American Medical Colleges’ Entrustable Professional Activities for Entering Residency and selected by the survey designers to focus on areas that may have been maximally impacted by the pandemic and have met optimal survey design principles (eg, designing positively worded questions and single non-compound questions of limited length, and placing demographic questions at the end of the survey to improve response rates). The survey was distributed in May 2021 by email to incoming residents who matched into HCA Healthcare psychiatry programs. All 15 psychiatry programs within this organization were involved in the survey, which was distributed to 63 eligible residents. Resident contact information was obtained through the program’s on boarding process and distributed through the organization’s electronic learning management system. The survey consisted of questions regarding self-assessed clinical skills as well as the perceived impact of COVID-related disruptions on clerkships and core procedures in alignment with the 2021 Transition in a Time of Disruption guidance document issued by the American Association of Osteopathic Medicine, Association of American Medical Colleges, Accreditation Council of Graduate Medical Education, and the Educational Commission for Foreign Medical Graduates Foundation for the Advancement of International Medical Education and Research.2 Questions regarding clinical skills consisted of responses ranging from “Below where I thought I would be” to “A strength of mine.” Questions regarding 11 core clerkships were answered with responses of “no disruption,” “partially virtual,” or “completely virtual.” Competency with 6 procedures was determined using responses of “no impact on ability to perform” or “impact on ability to observe or perform.” Statistics were conducted using the program SPSS 26 (IBM Corporation). The study was approved by the HCA Healthcare Institutional Review Board.
Results
Of the 63 surveys sent, 61 respondents replied, leading to a response rate of 97%. Respondent demographics were 30 (49%) male and 31 (51%) female (Table 1). Of the incoming residents, 29 (48%) had a doctorate of medicine, 31 (51%) had a doctorate of osteopathic medicine, and 1 (1%) had a bachelor of medicine, bachelor of surgery degree.
Table 1.
Respondent Demographics by Gender and Degree Type
| Responses (%); n = 61 | |||
|---|---|---|---|
| Gender | Male | Female | |
| 30 (49%) | 31 (51%) | ||
| Medical degree | Doctor of medicine | Doctor of osteopathic medicine | Bachelor of medicine, bachelor of surgery |
| 29 (48%) | 31 (51%) | 1 (1%) | |
The majority of learners did not experience disruptions to core clerkships (Table 2) or procedures (Table 3) and reported meeting expectations on core clinical skills. Responses to questions regarding clinical skills are summarized in Table 4. The majority of participants (80.3%–98.4%) reported being at the level they expected or better on all clinical skills (Table 4). Perceived deficits of core clinical skills were greatest in performing non-psychiatric procedures and placing consults to physicians in other specialties. Deficits in procedural skills were greatest for rectal and pelvic examinations in addition to transitions of care.
Table 2.
Perceived Impact of COVID-19 on Clerkships and Core Procedures
| Responses (%); (n = 61) | |||
|---|---|---|---|
| Clerkship | No disruption | Partially completed in the clinical environment | Completed virtually |
| Internal medicine, inpatient | 43 (70.5%) | 16 (26.2%) | 2 (3.3%) |
| Internal medicine, ambulatory | 48 (78.7%) | 10 (16.4%) | 3 (4.9%) |
| Intensive care unit | 44 (72.1%) | 5 (8.2%) | 12 (19.7%) |
| Emergency medicine | 42 (68.9%) | 5 (8.2%) | 14 (23.0%) |
| Family medicine continuity clinic | 48 (78.7%) | 10 (16.4%) | 3 (4.9%) |
| Pediatrics | 50 (82.0%) | 9 (14.8%) | 2 (3.3%) |
| General surgery | 50 (82.0%) | 5 (8.2%) | 6 (9.8%) |
| Anesthesiology | 43 (70.5%) | 4 (6.6%) | 14 (23.0%) |
| Radiology | 48 (78.7%) | 10 (16.4%) | 3 (4.9%) |
| Obstetrics and gynecology | 27 (44.3%) | 30 (49.2%) | 4 (6.6%) |
| Elective/selective experiences | 43 (70.5%) | 16 (26.2%) | 2 (3.3%) |
Table 3.
Perceived Impact of COVID-19 on Preparedness for Core Procedures
| Responses (%); (n = 61) | ||
|---|---|---|
| Procedure | No impact on ability to observe or perform | Impact on ability to observe or perform |
| Airway management | 45 (73.8%) | 16 (26.2%) |
| Basic cardiopulmonary resuscitation | 56 (91.8%) | 5 (8.2%) |
| Bag and mask ventilation | 50 (82.0%) | 11 (18.0%) |
| Venipuncture | 49 (80.3%) | 12 (19.7%) |
| Arterial puncture | 39 (63.9%) | 22 (36.1%) |
| Placement of an intravenous therapy | 46 (75.4%) | 15 (24.6%) |
| Not applicable | 29 (47.5%) | 32 (52.5%) |
Table 4.
Self-Assessed Level of Preparedness by Core Clinical Skill
| Responses (%); (n = 61) | ||||
|---|---|---|---|---|
| Clinical skills | Below where I thought I’d be | About where I thought I’d be | Above where I thought I’d be | A strength of mine |
| Taking a history from a patient that is appropriate for the clinical situation and the rotation | 1 (1.6%) | 27 (44.3%) | 22 (36.1%) | 11 (18.0%) |
| Performing a physical examination that is appropriate for the clinical situation and rotation | 2 (3.3%) | 40 (65.6%) | 16 (26.2%) | 3 (4.9%) |
| Pelvic examinations and rectal examinations as required for a PGY-1 in your specialty | 12 (19.7%) | 40 (65.6%) | 8 (13.1%) | 1 (1.6%) |
| Integrating the history and physical examination into an understanding of what the clinical problems seem to be | 2 (3.3%) | 37 (60.7%) | 17 (27.9%) | 5 (8.2%) |
| Deciding on the need for, and type of, additional diagnostic testing needed if any | 7 (11.5%) | 37 (60.7%) | 15 (24.6%) | 2 (3.3%) |
| Interpreting individual labs or imaging (eg, reading a chest x-ray) or interpreting a comprehensive metabolic panel | 8 (13.1%) | 38 (62.3%) | 12 (19.7%) | 3 (4.9%) |
| Interpreting the lab, imaging, history, and physical exam into a “post-test” probability of the likely clinical problems | 9 (14.8%) | 35 (57.4%) | 15 (24.6%) | 2 (3.3%) |
| Determining the treatments needed for a patient, including symptomatic medications, like pain medications | 7 (11.5%) | 40 (65.6%) | 8 (13.1%) | 6 (9.8%) |
| Performing procedures at the level I would expect a PGY-1 to perform | 19 (31.1%) | 33 (54.1%) | 9 (14.8%) | 0 |
| Calling consultations with other providers | 17 (27.9%) | 32 (52.5%) | 11 (18.0%) | 1 (1.6%) |
| Safely transitioning care at sign out or end of shift | 13 (21.3%) | 37 (60.7%) | 11 (18.0%) | 0 |
| Documenting appropriately in the electronic health record | 2 (3.3%) | 33 (54.1%) | 19 (31.1%) | 7 (11.5%) |
| Ability to look up and apply evidence-based recommendations and access point-of-care diagnostic aids | 1 (1.6%) | 38 (62.3%) | 14 (23.0%) | 8 (13.1%) |
Abbreviation: PGY = post graduate year
Disruptions to core clerkships are detailed in Table 2. Disruptions to obstetrics and gynecology clerkships were reported by 55.7% of respondents. Disruption of internal medicine clerkships was reported by 29.5% of respondents. Intensive care unit (ICU), emergency medicine (EM), and anesthesiology clerkships had the highest reported rates of remote learning at 19.7%, 23.0%, and 23.0%, respectively.
Perceived preparedness for core procedural skills are delineated in Table 3. Acquisition of the procedural skills arterial puncture, airway management, and IV placement were disrupted in 36.1%, 26.2%, and 24.6%, respectively. A majority of learners did not report a negative impact on preparedness for performing core procedures. Perceived disruption was greatest for arterial puncture and airway management.
Discussion
The purpose of this 2021 survey was to determine self-perceptions of competency following disruptions to the medical education of incoming psychiatric residents at 16 programs. It is clear the pandemic caused disruptions to health care and medical education that were experienced by providers, patients, and trainees.2 A prior survey of medical students found that 47% felt that the pandemic had negatively affected their education and only 42% reported feeling competent in clinical skills.14 The extent of the educational deficits due to the pandemic is not currently known; however, such deficits are likely to have affected medical knowledge in addition to self-perception of competency. The final year of medical school is a critical phase in education, in which students gain relevant hands-on experience in preparation for residency. Electives taken during the fourth year have classically provided students the opportunity to pursue interests relevant to their chosen career paths and to address gaps in clinical knowledge.
The majority of new residents we surveyed did not report significant disruption to their medical education throughout the pandemic, which is an encouraging result. The majority of trainees reported meeting or exceeding their expectations for the development of core clinical skills. Clerkship experiences and the development of procedural skills were not impacted for most trainees. A significant portion of ICU, EM, and anesthesiology clerkships were completed virtually, which is a plausible explanation for the relative deficiencies reported in airway management, arterial puncture, and IV placement. The results indicate that remote learning may have been detrimental to the development of procedural skills, although it did not appear to significantly impact other core skills. Most of the deficiencies may be of limited relevance to psychiatric practice but are necessary for requisite medicine rotations. Deficiencies of clinical skills reported by incoming psychiatric residents in the survey were consistent with those reported for a larger sample encompassing 10 different specialties.8 While less than ideal for many aspects of education, virtual learning has the potential advantage of bringing didactics to areas that otherwise could not access certain topics or educators. Due to the lack of pre-pandemic data, it is indeterminate to what extent deficiencies were present prior to the pandemic.
It is also difficult to compare the current results to pre-pandemic self-perceptions given the lack of pre-pandemic literature. Perceived deficiencies in proficiency in airway management predated the pandemic, but deficiencies in arterial and venipuncture appear to have been exacerbated. A survey of medical students conducted in 2001 demonstrated proficiency rates for arterial and venipuncture of 70% and 82%, respectively. However, only 36% were proficient in intubation.15 Over half of fourth year medical students surveyed in 2006 reported performing arterial and venipuncture 2 or more times but reported only moderate levels of self-confidence.16 However, a follow-up study found that close to a third of students who had not previously performed these procedures acquired these experiences during their fourth year.17
Incoming psychiatry residents typically spend a total of 6 months rotating across the medicine and neurology fields during the first postgraduate year. While on these services, psychiatry residents are expected to have the knowledge and skills of a first-year internal medicine or neurology resident. Significant disruptions to the internal medicine clerkships and those clerkships that teach procedural processes will directly impact the ability of the first-year residents to perform consultations and transitions of care at a satisfactory level. Understanding these deficits exist allows adaptation in the orientation process, including assessing an individual resident’s deficits and the training to improve them.
Study Limitations
Notably, both high response rate and sample size are identifiable strengths of this study. However, lack of pre-pandemic data and self-reported perceptions of confidence are relative areas of weakness. The generalizability to other specialties is limited, but the results are consistent with those available for a multi-specialty sample.8 Expectations of preparation may be unique among incoming psychiatric residents relative to other specialties. It is widely accepted that incoming residents are generally overwhelmed by the abundance of clinical skills to be mastered. It is impossible to draw conclusions regarding the effect of the pandemic without understanding the baseline level of self-confidence of residents. Additionally, this survey measured self-perceptions, which may not mirror true levels of preparedness, as it may be relative to peers who were also affected. The anonymous nature of the survey was communicated clearly, but potential bias in response should also be considered given the distribution of the survey by prospective employers.
Conclusion
Our study demonstrated that most incoming psychiatry residents did not report that COVID-19 affected the preparation for residency. Most did not experience disruptions to clinical clerkships due to the pandemic. A majority felt that they met self-perceived expectations in core clinical skills and did not experience disruptions in the ability to observe or perform procedures. The most commonly reported procedural deficiencies coincided with the clerkships with the highest reported rates of remote learning. The true impact of the pandemic on medical education is not known; however, the results here indicate that effects within this cohort may not have been as negative as feared. As interference with medical education may continue to occur throughout the world as a result of future pandemics, and other reasons, these data indicate that medical students adapted to limit negative effects on their education, which allows for some optimism in the face of future disruption.
Appendix 1. Survey Regarding Self-Perceived Competency Following the Pandemic
Based on your clinical experience while in medical school, please rank the following clinical skills (Abbreviation: PGY = post graduate year):
| Below where I thought I’d be | About where I thought I’d be | Above where I thought I’d be | A strength of mine | |
|---|---|---|---|---|
| Taking a history from a patient that is appropriate for the clinical situation and the rotation | ||||
| Performing a physical examination that is appropriate for the clinical situation and rotation | ||||
| Pelvic examinations and rectal examinations as required for a PGY-1 in your specialty | ||||
| Integrating the history and physical examination into an understanding of what the clinical problems seem to be | ||||
| Deciding on the need for, and type of, additional diagnostic testing | ||||
| Interpreting individual labs or imaging (eg, reading a chest x-ray) or interpreting a comprehensive metabolic panel | ||||
| Interpreting the lab, imaging, history, and physical exam into a “post-test” probability of the likely clinical problems | ||||
| Determining the treatments needed for a patient, including symptomatic medications, like pain medications | ||||
| Performing procedures at the level I would expect a PGY-1 to perform | ||||
| Calling consultations with other providers | ||||
| Safely transitioning care at sign out or end of shift | ||||
| Documenting appropriately in the electronic health record | ||||
| Ability to look up and apply evidence-based recommendations and access point-of-care diagnostic aids |
Other: ____________________________________________________________________________________________
___________________________________________________________________________________________________
___________________________________________________________________________________________________
Did the pandemic cause you to miss all or part of any of your clerkships or electives? Please indicate the appropriate level of impact in the following clinical rotations or settings. (Abbreviations: IM = internal medicine; ICU = intensive care unit; FM = family medicine; OBGYN = obstetrics and gynecology)
| No disruption | Partially completed in the clinical environment | Completed virtually | |
|---|---|---|---|
| IM inpatient | |||
| IM ambulatory | |||
| ICU | |||
| Emergency medicine | |||
| FM continuity clinic | |||
| Pediatrics | |||
| General surgery | |||
| Anesthesiology | |||
| Radiology | |||
| OBGYN | |||
| Elective/selective experiences |
Other: ____________________________________________________________________________________________
___________________________________________________________________________________________________
___________________________________________________________________________________________________
-
Did the pandemic impact your ability to observe or perform general procedures? If yes, which ones? (Check all that apply.)
Airway management
Basic cardiopulmonary resuscitation
Bag and mask ventilation
Venipuncture
Arterial puncture
Placement of an IV
Not applicable
Other procedures that are important to your matched specialty:
___________________________________________________________________________________________________
___________________________________________________________________________________________________
___________________________________________________________________________________________________
Funding Statement
This research was supported (in whole or in part) by HCA Healthcare and/or an HCA Healthcare-affiliated entity.
Footnotes
Conflicts of Interest: The authors declare they have no conflicts of interest.
Drs Anderson, Mahajan, and Sprenger are employees of TriStar Centennial Medical Center, a hospital affiliated with the journal’s publisher.
Drs Ayutyanont, Guldner, and Wells are employees of HCA Healthcare Graduate Medical Education, an organization affiliated with the journal’s publisher.
This research was supported (in whole or in part) by HCA Healthcare and/or an HCA Healthcare-affiliated entity. The views expressed in this publication represent those of the author(s) and do not necessarily represent the official views of HCA Healthcare or any of its affiliated entities.
References
- 1.Weekly epidemiological update on COVID-19-13 July 2023. World Health Organization; Jul 13, 2023. [Accessed May 13, 2024]. https://www.who.int/publications/m/item/weekly-epidemiological-update-on-covid-19---13-july-2023 . [Google Scholar]
- 2. Ioos D, Gallicchio V. Effects of COVID-19 on healthcare practice, medical education, and pre-medical educational experiences. J Public Health Epidemiol. 2020;12(3):186–192. doi: 10.5897/JPHE2020.1237. [DOI] [Google Scholar]
- 3. Wurth S, Sader J, Cerutti B, et al. Medical students’ perceptions and coping strategies during the first wave of the COVID-19 pandemic: studies, clinical implication, and professional identity. BMC Med Educ. 2021;21(1):620. doi: 10.1186/s12909-021-03053-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Sudi R, Chang WL, Arshad NH, Zainal Abidin SN, Suderman U, Woon LS. Perception of current educational environment, clinical competency, and depression among Malaysian medical students in clinical clerkship: a cross-sectional study. Int J Environ Res Public Health. 2022;19(23):16274. doi: 10.3390/ijerph192316274. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Harries AJ, Lee C, Jones L, et al. Effects of the COVID-19 pandemic on medical students: a multicenter quantitative study. BMC Med Educ. 2021;21(1):14. doi: 10.1186/s12909-020-02462-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Dergham P, Saudagar FNI, Jones-Nazar CC, et al. Medical students’ perceptions towards online teaching during the Covid-19 pandemic: a cross-sectional study from Saudi Arabia. Adv Med Educ Pract. 2023;14:407–419. doi: 10.2147/AMEP.S396912. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Bączek M, Zagańczyk-Bączek M, Szpringer M, Jaroszyński A, Wożakowska-Kapłon B. Students’ perception of online learning during the COVID-19 pandemic: a survey study of Polish medical students. Medicine (Baltimore) 2021;100(7):e24821. doi: 10.1097/MD.0000000000024821. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Guldner G, Wells J, Ayutyanont N, et al. COVID-19 related disruptions to medical education and perceived clinical capability of new resident physicians: a nationwide study of over 1200 first-year residents. Med Educ Online. 2023;28(1):2143307. doi: 10.1080/10872981.2022.2143307. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. D‘Emilio A, Sabounchi SS, Wu Y, et al. Comparison of clinical competency self-assessments among Advanced Education in General Dentistry (AEGD) residents before and during COVID-19 pandemic. J Dent Educ. 2022;86(7):774–780. doi: 10.1002/jdd.12898. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Gottlieb M, Chan TM, Zaver F, Ellaway R. Confidence-competence alignment and the role of self-confidence in medical education: a conceptual review. Med Educ. 2022;56(1):37–47. doi: 10.1111/medu.14592. [DOI] [PubMed] [Google Scholar]
- 11.Ryan RM, Deci EL. Self-Determination Theory: Basic Psychological Needs in Motivation, Development, and Wellness. New York, NY: Guildford Press; 2017. [Google Scholar]
- 12. Deci EL, Olafsen AH, Ryan RM. Self-determination theory in work organizations: the state of a science. Ann Rev Organ Psychol Organ Behav. 2017;4(1):19–43. doi: 10.1146/annurev-orgpsych-032516-113108. [DOI] [Google Scholar]
- 13. Sawatsky AP, O‘Brien BC, Hafferty FW. Autonomy and developing physicians: reimagining supervision using self-determination theory. Med Educ. 2022;56(1):56–63. doi: 10.1111/medu.14580. [DOI] [PubMed] [Google Scholar]
- 14. Devito JR, Cheung C, Stuparich M, Nahas S, Behbehani S. 106 medical student and faculty perceptions of the COVID-19 pandemic on medical education and personal well-being. Am J Obstet Gynecol. 2021;224(6):S806–S807. doi: 10.1016/j.ajog.2021.04.131. [DOI] [Google Scholar]
- 15. Sanders CW, Edwards JC, Burdenski TK. A survey of basic technical skills of medical students. Acad Med. 2004;79(9):873–875. doi: 10.1097/00001888-200409000-00013. [DOI] [PubMed] [Google Scholar]
- 16. Wu EH, Elnicki DM, Alper EJ, et al. Procedural and interpretive skills of medical students: experiences and attitudes of third-year students. Acad Med. 2006;81(10 Suppl):S48–S51. doi: 10.1097/00001888-200610001-00013. [DOI] [PubMed] [Google Scholar]
- 17. Wu EH, Elnicki DM, Alper EJ, et al. Procedural and interpretive skills of medical students: experiences and attitudes of fourth-year students. Acad Med. 2008;83(10 Suppl):S63–S67. doi: 10.1097/ACM.0b013e318183c5a7. [DOI] [PubMed] [Google Scholar]