Preparation for the United States Medical Licensing Examinations in the Face of COVID-19

Abstract

The COVID-19 pandemic has led to the canceling and rescheduling of the United States Medical Licensing Examination (USMLE) examinations due to the nationwide closure of the Prometric testing centers, which poses a significant challenge to medical students. The rescheduling of a high-stakes board examination leads to significant stress and potential burnout. Students may need guidance to decrease anxiety and reframe their study plan while maintaining their knowledge. Here, we combined board examination coaching tips with specific worked examples to describe how to prevent burnout, give recommendations for scheduling, and suggest practical approaches to USMLE and other high-stakes examinations.

Introduction

The COVID-19 pandemic has caused uncertainty and anxiety at all levels of education, and medical education is no exception. It has made a remarkable impact during a time of educational transition for medical students, especially for students who are preparing to take the United States Medical Licensing Examination (USMLE) licensing examinations.

Performance on USMLE step 1 and 2 examinations is a major consideration for residency programs in selecting medical school graduates [1–3]. Under normal circumstances, medical students who are facing the high-stakes USMLE examinations are challenged by the intensity and length of study. For example, in step 1 preparation, it was reported that the average studying time per day is over 11 h for 35 consecutive days [4]. During this period, students’ emotional, mental, and physical coping methods are continually challenged.

With the initial nationwide closure of Prometric centers, partial “soft” openings, and continued glitches of unpredictable cancelations of the test dates, students with pre-scheduled test dates in March and April had to reschedule to later dates with the hope of being able to sit for their respective exams. In addition, the discrepancy between the Prometric’s official statements and planned operating procedures (NBME.org) forced students to face waves of cancelations of their rescheduled dates in May and June. Some students with confirmed dates from the Prometric centers arrived to test and were met with cancelation. This prompted an unprecedented acknowledgment from the USMLE of Prometric’s “failure to fulfill its obligation to examinees.” As a result, students have to modify—and often extend—their study plans. This unexpected turn of events will likely increase their anxiety and risk of burnout, which can directly impact their performance [5].

Furthermore, with more than half of the medical student population already experiencing burnout, estimated by a major multi-institutional study pre-COVID-19, the negative impact of COVID-19 on student’s well-being and burnout cannot be ignored [6]. In addition, burnout has been associated with decreasing empathy [7]. Empathy holds one of the core values in physician-patient care. Chen and colleagues [8] characterized that there was a declining level of empathy among students as early as the third year of medical school. Therefore, it will require multi-faceted approaches to generate practical ways to alleviate stress. Specifically, during these delays and cancelations due to COVID-19, preventing burnout with modification or reframing of a study plan is of utmost importance.

Here, we describe the practical applications of the USMLE examination strategies, with a focus on study scheduling, burnout prevention, and approaches for the USMLE examination in this unprecedented time.

Practical Applications

The following two practical applications were developed by the corresponding author, YH, and based on the framework of cognitive load theory (CLT) and adult learning theory (ALT) in medical education [9–11]. These practical applications specifically focus on the CLT components of decreasing intrinsic and extraneous cognitive load, as well as self-directed learning in ALT. They were designed to promote a focused, intentional, and outcome-driven framework that can be applied to various learning situations.

Self-Assessment, Identification, and Patching

The self-assessment, identification, and patching (SIP) approach is a continuous cycle. Self-assessment can be accomplished using any learning resource, such as lecture notes, Anki cards, or question banks. It should facilitate the identification of specific areas needing improvement. For example, it may start broadly as “Let me see how I am doing in the Gastrointestinal (GI) system using a block of questions in GI.” Identification is achieved from data analysis of the performance on the pre-selected topic, e.g., performance on GI. This type of review may need further narrowing of the focus down to anatomy, biochemistry, and pathology within GI. With the specific question and the assessment data in hand, patching becomes more focused, personalized, and manageable.

Consequently, the selection of resources and study strategies to patch up these areas becomes more efficient. The second round of self-assessment (e.g., questions of anatomy, biochemistry, and pathology in GI) will demonstrate the effectiveness of the study strategies and selected resources. The method provides a personalized, logical, and focused approach that demands accountability from the student.

Goal, Means, and Outcome

The goal, means, and outcome (GMO) method is a simplified and effective tool meant to assist the learners in identifying and modifying achievable and measurable goals to reduce anxiety. It was created by the corresponding author (YH, manuscript in preparation). It aids in planning, outlining, and achieving an often intimidating task. When overwhelmed and stressed in preparation for USMLE examinations, students may set vague goals without any specific assessment to determine the outcome of their effort, such as “I need to improve Biochemistry” for step 1. When asked, “How do you know if you are successful,” some students cannot provide an answer. Therefore, we need to link the goals with means and outcomes. The goal in GMO can be either broad or focused but must be attainable with a clear and measurable outcome. Compare these two goals: “I need to put more effort into reviewing the GI system” versus “I need to put more effort into reviewing GI pathology.” The first goal is important, but too broad. Thus, time and effort may not be spent on the most-needed areas. The second goal is more defined and focused; it is likely the result of self-assessment and will lead to more success for the student. Means refers to the resources and strategies used to accomplish the goal. A well-defined goal will help drive resource selection. An individualized approach to match the students’ preferred learning style with the appropriate resource modality is critical. Using the example above, students can focus on the individualized GI pathology resources rather than resources dedicated to the entire GI system. Finally, thinking about outcome while deciding on the goals has two purposes. The first is to help define the goal and make it attainable. The second is to drive students to generate their next goal, as defined by the result of their self-assessment, perpetuating a focused cycle of addressing study needs.

Scheduling Suggestions

For students who were preparing for the USMLE examinations, the COVID-19 pandemic brought uncertainty to a process that, in normal circumstances, is already stressful. Scheduling—and by consequence, rescheduling—is one of a myriad of things contributing to the uncertainty that many students are experiencing. Studies have shown that stress negatively impacts cognition, including memory retrieval and updating [12, 13]. In a recent survey assessing 4th-year medical student perceptions to changes occurring to the USMLE examination as a result of COVID-19 at Wake Forest University School of Medicine, 94.8% (out of 144 students, 95 responded to the survey (66%)) reported some level of stress due to rescheduling their USMLE step 2 examination date. The stress students are experiencing is related to having to change their examination dates, while also having to alter their study plans, often having to extend them by weeks to months. These same feelings of anxiety can be expected for any other students who are having a delay any high-stakes testing dates and extend their study plans.

As the visual aid in Fig. 1 indicates, students’ study plans would typically follow the blue curve, with the intensity of studying increasing until a few days before the exam date. However, many students—especially those with test dates in late March and April—are now in limbo due to examination delays and rescheduled test dates. As a result, students may feel pressured to complete an extended period of high-intensity studying, which would likely contribute to increased student anxiety and increased levels of burnout and could negatively impact performance [5]. Instead, we suggest students adopt a study plan similar to the yellow line in Fig. 1, which takes into account whether the Prometric testing center closure lasts 30 days and its partial reopening. The study plan does this by implementing a plateau, or a period of consistent, daily studying at a lower intensity level. During this time, we recommend students to continue with the SIP method to maintain their current knowledge base and patch up weaker areas. Furthermore, students should consider to remain GMO-oriented to organize their days and facilitate intentional studying effectively.

Fig. 1.

A visual aid developed by the corresponding author for a recommended study plan based on the availability of the testing date and performance level. The level of intensity depicted on the graph represents an individual’s effort in preparation for USMLE examinations

Broadly, we suggest this study plan modification using the SIP and GMO approaches to prevent burnout, while simultaneously allowing students to maintain—and further develop—their knowledge base. Without this approach, students may struggle to find direction and lose motivation as their study period continues to extend (as demonstrated by the plateau in Fig. 1). Here, we provide examples of how to use the SIP and GMO methods to create a productive study schedule during this period. With SIP, we recommend that students take an initial self-assessment or use a recent assessment (S) to identify (I) areas that need to be patched (P). Then, we would advise students to use the GMO method to put the SIP plan in place. For example, if a student’s self-assessment identified cardiac physiology as an area of weakness, the goal would be to patch up this area. To achieve this, students would review appropriate resources (means) and gauge their effectiveness (outcome) with a set of cardiology- and physiology-specific questions selected from the relevant question bank. Overall, we believe that using these methods will provide direction and purpose, as well as allow students to create an effective study schedule during this challenging time.

Once the Prometric testing center test dates are confirmed, the intensity of studying should increase as students prepare for their new exam date. During this time, we recommend that students use assessment tests to determine their next steps. If students are content with their performances but acknowledge room for improvement, we recommend they take their respective step examination as soon as the testing center is available and their schedule allows it. However, if students are not content with their performance, we would advise them to consider delaying their test date while they continue to patch up specific focus areas.

Preventing Burnout and Reducing Anxiety

Burnout is broadly defined as a state of emotional exhaustion that can lead to a reduction in the perceived or actual personal accomplishment [6]. Medical students and residents, compared to the general population, have a higher rate of burnout rates [14]. In a recent meta-analysis, Frajerman and colleagues [15] showed that 50% of medical students—of the 17,431 students included across 24 studies—are experiencing burnout before reaching residency, with the most prevalent factor leading to burnout being emotional exhaustion. Burnout has been studied extensively, with data suggesting that it is associated with increased psychiatric illness and suicidal ideation both during and beyond medical school [7, 8]. In addition, the decline of empathy levels throughout the medical school is detrimental to high-quality patient care [8]. It is essential to develop tools and support for medical students.

With exam dates and study schedules uncertain, students are likely to experience emotional exhaustion leading to increased levels of burnout, which, in turn, would lead to deviation from organized study plans and less efficient studying [11]. To combat this and reduce anxiety, we suggest students apply the SIP and GMO methods to take control of their learning scope, resources, and daily scheduling. These methods prevent burnout by allowing students to see their progress—big or small—in a timely fashion, thus helping to improve work satisfaction. By giving students the ability to control how small or large their goals are, repeated use of the GMO method would be especially valuable for students struggling with motivation even to start studying. When students fail to use their performance data from self-assessment tools such as question banks or practice examinations to identify knowledge gaps, they are likely to create broad and unachievable goals. For example, they may plan to review all of the biochemistry without a measurable outcome. Over time, unfocused efforts may promote a vicious cycle of underperformance, misguided studying, limited improvement, and eventually, burnout. Developing concrete goals is the key to avoiding burnout during a time as dynamic as a global pandemic, especially while preparing for high-stakes examinations.

Effective use of SIP and GMO methods relies on adequate self-assessment and the development of clear, attainable goals and allows the student to select appropriate resources to patch knowledge deficits, which leads to targeted and more effective study efforts. Using this method will allow students to monitor their improvement, further motivating goal-oriented studying and decreasing the likelihood of burnout by breaking a student’s vicious cycle of unfocused studying. For example, if a student completed a block of 20 gastrointestinal-related questions (self-assessment) and realized they got several questions wrong regarding pancreatic enzymes (identification), they can use the resources that best suit their study strategies to address these areas of weakness (patching). By focusing solely on pancreatic enzymes and not digestion, topics become more manageable and achievable, and students remain more motivated to succeed through improving scores on question sets and other quantitative markers of improvement.

Approaches

In this section, we describe two approaches we would recommend during the extended study period. The first approach focuses on how to tackle weaker topics while maintaining stronger ones. The second approach discusses strategies based on the students’ perceived level of satisfactory performances on their self-assessment tests.

We applied cognitive load theory, which uses instructional techniques to decrease intrinsic and extraneous cognitive load to improve working memory, to medical education. We recommend simplifying the task by dividing it into two categories: a focused bucket and a maintenance bucket (Fig. 2) [9]. Guided by question bank performance, the lowest-performing subjects or systems should be placed in the focused bucket. To improve the knowledge in these weaker areas (goal), students should be intentional with resource selection (means) and, subsequently, should gauge the effectiveness of their efforts towards their intended goal (outcome). The maintenance bucket will contain the remaining subjects or systems that students have previously mastered. The guiding principle is to maintain current knowledge while continuing to self-assess, identify, and patch weaker areas. As a result, the topics in the focused bucket will improve and transition into the maintenance bucket, and the relatively weaker topics in the maintenance bucket will emerge and be moved into the focused bucket (Fig. 2). This approach provides a robust framework for students to reset their study plans.

Fig. 2.

The framework of continued study strategies in an extended studying period. The shaded parameter illustrates the recommended proportion of time each student should spend on the content within the respective buckets

The next approach uses the students’ self-assessment of their current performance and data analysis provided by the assessment test (e.g., Comprehensive Basic Science Self-Assessment (CBSSA) for USMLE step 1, Clinical Science Mastery Series, National Board of Medical Examiners subject examinations for step 2, or any recent assessment tests). Often, it can be challenging to identify areas for improvement when using highly integrated question sets. Here, we outline the following steps for this approach:

• First, students should consider tallying their missed questions into the appropriate cells in Table 1, general guidance for a more in-depth analysis of CBSSA results. This approach will assist students in identifying weaker areas.

• Second, students should consider dividing the missed questions into either content or test-taking errors. They could also divide missed questions into general principles versus system-specific errors.

Table 1.

A sample of a self-inventory table using the result of a Comprehensive Basic Science Self-Assessment for USMLE step 1

	General principles	Systems
Anatomy
Biochemistry
Immunology
Genetics
Microbiology
Pathology
Pharmacology
Physiology

Both steps aim to provide answers to individualized questions and help identify areas for improvement.

For students who scored lower than their desired outcome range on assessment tests, we recommend they spend more time on the content in the focused bucket during the plateau period in Fig. 1 of the proposed study modification. Students should consider blending both SIP and GMO methods in the process. For example, assessment tests (self-assessment) should reveal areas for improvement (identification), such as cardiac pharmacology versus just pharmacology. Identification of a specific goal will aid in resource selection (means) and guide the content reviewed, and such examples may include Anki decks or first aid. Once the student successfully patches the knowledge deficit as evidenced by continually achieving their target range (outcome), this content may transition from the focused bucket to the maintenance bucket (shown in Fig. 2), and overall scores should improve. When deficits have been sufficiently patched and moved to the maintenance bucket, the process can be repeated.

Without adopting this method may result in unachievable goals and ineffective study efforts. The goal of this method is to transfer the focused content that reaches the desirable performance into the maintenance bucket, without neglecting subjects mastered previously. Once the testing date is finalized, students should increase the intensity of their studying and focus more on overall improvement until decreasing a few days before the examination to avoid possible burnout.

For students who scored within their desired outcome range on assessment tests, maintenance is the objective (goal) of their study period. However, students may continue to utilize the strategy of focused versus maintenance buckets. In practice, students may do fewer blocks of questions (means) with all topics and systems selected (self-assessment), which allows students to identify relatively weaker areas to move into the focused bucket for further improvement (patching). Once the weaker topics are patched and transitioned back into the maintenance bucket, overall performance should improve (outcome). As a result, the SIP and GMO methods complement each other and can be applied at any level of performance.

Conclusion

During this extraordinary and trying time, we propose the integration of the GMO and SIP methods into three challenging situations that students may face: test scheduling, burnout prevention, and study approaches. These have been implemented in our USMLE step 1 and 2 coaching process at Wake Forest School of Medicine (manuscript in preparation). Here, these methods have been updated with practical approaches and student input to address the uncertainties that students may encounter in preparation for the USMLE examinations, the Comprehensive Osteopathic Medical Licensing Examination, or any other high-stakes examinations during the COVID-19 pandemic.

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they have no conflict of interest.