COVID-19 related disruptions to medical education and perceived clinical capability of new resident physicians: a nationwide study of over 1200 first-year residents

ABSTRACT

The COVID-19 pandemic transformed the final year of undergraduate medical education for thousands of medical students across the globe. Out of concern for spreading SARS-CoV-2 and conserving personal protective equipment, many students experienced declines in bedside clinical exposures. The perceived competency of this class within the context of the pandemic is unclear. We designed and distributed a survey to measure the degree to which recent medical school graduates from the USA felt clinically prepared on 13 core clinical skills. Of the 1283 graduates who matched at HCA Healthcare facilities, 90% (1156) completed the survey. In this national survey, most participants felt they were competent in their clinical skills. However, approximately one out of four soon-to-be residents felt they were clinically below where they should be with regard to calling consultations, performing procedures, and performing pelvic and rectal exams. One in five felt they were below where they should be with regard to safely transitioning care. These perceived deficits in important skill sets suggest the need for evaluation and revised educational approaches in these areas, especially when traditional in-person practical skills teaching and practice are disrupted.

Introduction

The COVID-19 pandemic transformed the final year of medical education for approximately 20,900 graduating medical students in the USA during the academic year of 2020–2021 [1] and many more across the globe. The disruption to medical education was unprecedented [2] as many medical schools re-organized rotations to occur in an online format to protect students, conserve personal protective equipment, and limit the spread of the virus [3]. These changes likely provided modified forms of clinical education such as online discussions focused on medical decision-making and topics amenable to the distance learning format [4]. However, the loss of bedside patient exposure likely impacted some learners’ perceived readiness for residency, though the degree to which this occurred is unknown. The aim of our study was to assess the self-perceived clinical capabilities of a large cohort of recent graduates from medical schools in the USA and abroad who were preparing to begin residency training.

Materials and methods

HCA Healthcare is the largest hospital system in the USA and began sponsoring residency programs in 2016. At the time of this research, HCA Healthcare sponsored residency programs in 10 specialties that match newly graduated medical students directly into a first clinical year position immediately following medical school. These specialties include anesthesiology, emergency medicine, family medicine, general surgery, internal medicine, neurology, obstetrics and gynecology, pediatrics, psychiatry, and transitional year programs. All 1283 medical students who graduated in 2021 and matched into one of the above specialty residency programs sponsored by HCA Healthcare received an email invitation to participate in an online survey-based research study in May 2021. Two additional reminder emails were sent to those participants who had not yet completed the survey. Participants were notified that the survey results would be available to their program director to assist in determining if supplemental educational efforts were needed for the entering class of residents. Participation was voluntary and subjects could withdraw at any time. This cohort completed the majority of their third year prior to the pandemic and the entirety of their final year of medical school during the pandemic. The authors, who are educators, administrators, or research directors in GME programs, coupled with a research psychologist with expertise in survey design, developed a new (not previously validated) cross-sectional survey that asked incoming first-year residents to estimate their perceived clinical readiness on 13 core clinical skills. These clinical skills were determined by a review of the American Association of Medical College’s (AAMC) Entrustable Professional Activities for Entering Residency [5], modified to meet optimal survey design principles and to focus on areas that may have been maximally impacted by the COVID-19 pandemic. Table 1 shows the 13 core clinical skills. Response options for the clinical skills followed a modified Likert scale with options that included: ‘Below where I thought I would be’, ‘About where I thought I would be’, ‘Above where I thought I would be,’ and, ‘A strength of mine’. Additional scales, not reported here, were administered in this survey as part of a larger research project. Descriptive statistics, specifically proportions of the overall sample, were summarized using IBM SPSS Statistics for Windows, Version 26.0. Armonk, NY: IBM Corp. The study was approved by the HCA Healthcare Institutional Review Board (reference # 2021–480).

Table 1.

Self-assessed core clinical skills.

Clinical Skills	% Responses (n = 1156)
	1	2	3	4
Taking a history from a patient that is appropriate for the clinical situation and the rotation.	1.3	44.6	35.4	18.8
Performing a physical examination that is appropriate for the clinical situation and rotation	3.8	59.1	28.0	9.1
Pelvic examinations and rectal examinations as required for a PGY1 in your specialty	33.1	52.7	11.2	2.9
Integrating the history and physical examination into an understanding of what the clinical problems seem to be	2.1	55.4	33.2	9.3
Deciding on the need for, and type of, additional diagnostic testing needed if any	5.1	67.7	22.9	4.2
Interpreting individual labs or imaging (e.g., reading a chest x-ray) or interpreting a comprehensive metabolic panel	6.1	61.6	26.0	6.2
Interpreting the lab, imaging, history, and physical exam into a ‘post-test’ probability of the likely clinical problems	8.3	66.7	20.6	4.4
Determining the treatments needed for a patient, including symptomatic medications like pain medications	9.9	69.8	16.9	3.4
Performing procedures at the level I would expect a PGY1 to perform	25.8	54.0	15.1	5.1
Calling consultations with other providers	26.5	55.5	15.2	2.8
Safely transitioning care at sign out or end of shift	19.3	62.8	14.9	3.0
Documenting appropriately in the electronic health record	7.7	51.3	27.8	13.2
Ability to look up and apply evidence-based recommendations and access point-of-care diagnostic aids	2.6	56.7	28.7	12.0

1 Below where I thought I’d be; 2 About where I thought I’d be; 3 Above where I thought I’d be; 4 A strength of mine

The survey prompt for all items was: ‘Based on your clinical experience while in medical school, please rank the following clinical skills’

Results

The overall response rate was 90% (1156 out of 1283), representing 10 specialties at 52 HCA Healthcare affiliated hospitals across 14 states. Participants graduated from 173 different medical schools including 38% from allopathic schools located in the USA, 48% from osteopathic schools located in the USA, and 14% from schools outside of the USA. The degrees obtained included 51% M.D. and 49% D.O. Participants included 64% who identified as male and 36% as female. The results of the perceived capabilities in the 13 core clinical skills are summarized in Table 1.

For all clinical skills, the majority of participants recorded being either at or above their expected level of performance. Nineteen percent of participants felt that taking a history was a personal strength. However, significant proportions of residents reported being below where they should be with regard to the skills needed to call consultations (26.5%), perform pelvic and rectal examinations (33.1%), and perform procedures at the anticipated level for their matched specialty (25.8%). Safely transitioning care, which generally refers to the process of patient handoff between one outgoing provider and another provider who is either assuming care permanently or temporarily, was rated as below their anticipated level by 19.3% of participants.

Discussion

Our multi-specialty nationwide survey, with a high response rate, provides evidence of the self-perceived readiness of new residents during the COVID-19 educational disruptions. The impact of the pandemic on medical education was unparalleled and poorly understood[1]. While a few medical students were called to help supplement overwhelmed physician staff [6], most underwent reductions in clinical experiences to limit unnecessary exposures [2]. The last year of medical school normally allows a final opportunity to develop skills, attitudes, and behaviors necessary for the transition to residency. Students ordinarily experience an increase in clinical exposures, autonomy, and responsibility during their final year, all while embedded in healthcare teams that provide modeling of skills such as calling consultations and transitioning care. Understandably, greater clinical experience appears to be correlated with greater perceived confidence [7,8] and COVID-19 disrupted that process. Accordingly, in a joint publication, several organizations have suggested that residency programs assess their new residents for their clinical strengths and deficits as they begin their training [9]; a recommendation the current study supports.

In our study, a significant majority of newly graduated medical students felt as prepared as they expected to be with core clinical skills. This finding is consistent with the findings from the 2021 report from the American Association of Medical Colleges which found that 89% of graduates from the USA were satisfied with their medical education and 92% agreed that they had acquired the skills necessary to begin residency [10]. Smaller studies from the international community show a similar pattern of relatively high confidence in perceived skills [11,12]. This consistency is reassuring, given the potential impact of pandemic-related educational changes to undergraduate medical education. However, a minority of students do report perceived deficits.

We show that a non-trivial proportion of new residents note perceived deficits in the ability to call consultations, transition care, perform procedures, and perform rectal and pelvic examinations. Calling consultations and transitioning care requires interpersonal communication skills which, when done poorly, can result in significant patient safety risks [13]. The accrediting bodies for undergraduate medical education for many countries including the UK, Australia, Canada, and the USA specifically emphasize the need for competency in transitions of care [14–17]. Research done prior to the pandemic shows some deficits among graduating medical students in competence with transferring care [18–21]. This may be partially due to few opportunities to transfer care during undergraduate medical education. Comparing graduating medical students from 2019 with 2020 Brown and colleagues showed that the number of assessments for most of the skills actually increased in 2020, but this was not true of transitioning care [22]. The percentage of graduating students deemed ready for transitioning care without a supervising physician present was low in both the pre-pandemic year (2019) and post-pandemic (2020) at a combined 8.3% [22]. This suggests that the relative deficit in perceived competency in transitioning care found in the current study may represent a baseline deficit. Regardless, medical schools may wish to evaluate, and possibly enhance, their activities designed to teach transitions of care. Several educational programs exist to teach transitions of care and communication with consultants, many of which are consistent with distance-learning formats [23–30].

In comparing the number of evaluations conducted on medical students for 13 Entrustable Professional Activities, most increased between 2019 and 2020 with two exceptions: transitioning of care, as noted above, and performing procedures [22]. The current study found a similar pattern with transitions of care and performing procedures as exceptions to the general pattern of perceived competence. This suggests that the relative deficits in perceived competence with procedures may represent a baseline deficit and not necessarily an impact of the pandemic. Developing perceived confidence in the performance of procedures and rectal and pelvic examinations may require experience on some minimal number of live patients [7,8]. Deficits in exposures to these procedures may have existed even prior to the pandemic with a possible worsening during the educational changes prompted by COVID-19. For example, program directors rate the performance of procedures as the entrustable activity in which they are least confident among the 13 clinical skills typically assessed in medical students in a pre-pandemic study [31]. In the same study, the program directors also rated procedures as one of the least likely activities to be observed by supervisors during a medical student clerkship [31].

Although distinct from procedural skills, rectal and pelvic examinations typically also require supervised exposure to develop perceived competence. Pre-pandemic studies from several countries show that students typically perform few observed rectal exams and graduate with relatively low confidence in their abilities [32–36]. Similarly, studies on the quality of pelvic examination training in the pre-pandemic period suggest deficits in medical student experience and confidence with this skill [37–41]. Thus, the perceived deficits seen in the current study may reflect a combination of pre-pandemic deficits in training coupled with pandemic-related educational changes. A review of the core clinical skills that new residents reported as ‘below expectations’ suggests that they are skills that likely would be somewhat more difficult to develop via distance learning modalities. While many of the skills assessed could be obtained via small group discussions, simulated patient encounters, and online case discussions, others simply do not lend themselves easily to a distance learning environment. Taking a history and developing a plan for diagnosis and treatment likely was done repeatedly during online clerkships, leading to these students feeling comfortable with cognitive aspects of clinical skills [42]. Students also may feel that they developed basic physical examination skills during their third year of medical school which was less affected by COVID-19 in this cohort. Understandably, pelvic and rectal examinations, and procedures in general, are difficult to teach with confidence in an online environment. Similarly, efforts to teach the skills needed for confidence in calling consultations and transitions of care may have faltered in the absence of in-person clinical experience and modeling of these skills.

The COVID-19 pandemic may have impacted medical student education in other important ways as well. For example, as restrictions on medical student rotations were lifted the total number of patients in clinics and emergency departments remained below pre-pandemic levels in many locations potentially minimizing clinical learning opportunities [43,44]. Additionally, the emotional impact of the pandemic on clinical educators may have adversely impacted their motivation or ability to teach at an optimal level. With high levels of workplace burnout, depression, or both, educators may not create optimal work and learning environments [45].

Regardless of whether the perceived confidence in these skills was diminished by the COVID-19 pandemic, undergraduate medical education programs may wish to develop more robust educational approaches to teaching these skills prior to students transitioning into graduate medical education. In environments with limited in-person teaching, several options for distance learning formats to supplement clinical skills are available and described in the literature [46–48]. Additionally, graduate medical education programs should consider assessment of their incoming residents to determine individual skill sets and perceived competency as suggested by the AAMC, the American Association of Osteopathic Colleges, the Accreditation Council for Graduate Medical Education, and the Educational Commission for Foreign Medical Graduates [49].

Limitations

Our survey captured information only from applicants who matched into a residency program during the 2021 match cycle. Accordingly, we do not have information on those applicants who did not match into a residency program and that population may differ from the one described here. While self-perceived capability is an important outcome it differs from actual capability. Residents may over or under estimate their true capabilities. Furthermore, the ability to accurately estimate competence may also have been compromised by worsening workplace burnout, depression, or both brought on by the pandemic. We cannot determine whether the results noted here are related to COVID-19 or would have been similar had the study been done prior to COVID-19. Finally, social response bias, where residents answer questions with what they perceive as the most desirable answer, may have influenced our results.

Conclusions

Medical school graduates in 2021 preparing to begin graduate medical education in the USA generally felt clinically prepared despite their final year of medical school occurring during the COVID-19 pandemic. Relative deficits were noted in the perceived capability to call consultations, transition care to new providers, perform rectal and pelvic examinations, and perform procedures. Deficits in these areas may have been present in medical students even prior to COVID-19. Regardless, they represent possible opportunities for improvement in undergraduate medical education programs.

Funding Statement

The author(s) reported that there is no funding associated with the work featured in this article.

Disclosure statement

No potential conflict of interest was reported by the author(s).