Exploring Preparation for the USMLE Step 2 Exams to Inform Best Practices

Abstract

Introduction

The United States Medical Licensing Examination (USMLE) Step 1 will transition to a pass-fail format in 2022. This is likely to result in an increased focus on Step 2 Clinical Knowledge (CK) scores. Thus, academic advisors must provide evidence-based guidance for preparing students. While prior research has examined the utility of academic indicators to predict student performance on the USMLE exams, no significant scholarly effort has described or evaluated students’ study approaches. The research study’s goal was to understand what strategies and resources students utilized when preparing for the Step 2 CK exam and investigate the relationship(s) between these approaches and performance.

Methods

Students at a single US medical school were surveyed about their Step 2 CK preparation. We analyzed self-reported exam preparation strategies and the use of specific resources to determine their relationship with Step 2 CK score.

Results

Student performance on Step 2 CK was correlated with performance on previous exams, including school-specific examinations, National Board of Medical Examiners clerkship shelf exams, and Step 1. Two study strategies were positively correlated with Step 2 CK score in preliminary analyses: completing more working practice questions, and the proportion of a question bank completed. In hierarchical regression, only completing more working questions remained predictive, after controlling for demographic variables and Step 1 performance.

Conclusions

Faculty and staff can optimize students’ Step 2 CK performance by encouraging them to work through case-based, clinically-focused questions. Further study is needed to describe optimal preparation strategies better.

Introduction

Academic advisors counsel medical students on study approaches and resources that promote student success on the United States Medical Licensing Examinations (USMLE) Step 2 Clinical Knowledge (CK) exam. Over time, the pressure on students to perform optimally on this exam has increased due to the importance of exam scores in the residency selection processes.^1,2 With the 2020 announcement that score reporting for Step 1 will move to pass/fail, students will likely experience increased pressure to achieve a competitive score on their Step 2 CK exam.^3–5 Moreover, because of the changes planned for scoring Step 1, students may not have learned and implemented effective exam preparation strategies previously. Thus, additional Step 2 CK preparation may be necessary.

While prior research has examined the utility of various academic indicators to predict student performance on the USMLE exams,^6,7 no significant scholarly effort has been conducted to investigate students’ Step 2 CK study approaches. This study aimed to describe what approaches and resources medical students utilized when preparing for the Step 2 CK exam and investigate the relationship(s) between various resources and strategies on Step 2 CK performance.

Methods

The humanist framework of self-directed learning theory guided this research.⁸ Students engaged in self-directed learning to diagnose their learning needs, formulate goals, identify resources, implement learning strategies, and evaluate outcomes.⁹ Two hundred-five students in the class of 2020 at a single public, allopathic US medical college were invited to participate in a survey focused on preparation for the Step 2 CK exam. Fifty-one students responded (25% response rate). The average USMLE Step scores for respondents was 232 on Step 1 and 248 on Step 2 CK. Survey invitations were sent on a rolling basis to students approximately 2 weeks after completing their exam. The survey included 102 multiple-choice and short-answer items.

Responses were linked to demographic and academic performance information using the college’s honest broker process to deidentify data before analysis. The study was exempted from review by the Michigan State University Institutional Review Board.

We analyzed Step 2 CK preparation strategies and the use of specific resources to determine their relationship with Step 2 CK scores. We used Pearson correlations for linear variables, and independent samples t tests for dichotomous variables. We analyzed results descriptively, using counts and proportions. In some cases, we combined variables. We initially intended to analyze the relationships between study approaches and passing or failing the Step 2 CK and CS exams, but the relatively small sample size, and the small number of failures, prohibited these analyses. We performed hierarchical multiple regression analysis to evaluate whether any specific study strategies and resources continued to be positively associated with Step 2 CK score, after controlling for Step 1 score, MCAT score percentile, and entry pathway into medical school.

Results

Students used a heterogenous mix of study strategies and resources (Tables 2, 3, and 4). In basic analyses, both the earliest school-administered Comprehensive Clinical Skills Exam (CCSE) scores and the most recent CCSE scores were positively correlated with Step 2 CK scores (Table 5). Each National Board of Medical Examiners’ clerkship exam score was also predictive of the Step 2 CK score. Additionally, practice test questions (using the average percentage of correct questions) and the predicted Step 2 CK score from the most recent practice test were correlated. Among preparation strategies and resources, only completing more practice questions and the proportion of a question bank completed were positively correlated with Step 2 CK score. Prioritizing specific content areas for study, using coaching, and using some specific resources (First Aid, Master the Boards, OnlineMedEd, and Step 2 Secrets) were negatively predictive of score (Table 4). None of the other study resources and strategies were associated with Step 2 CK score. The number of different study resources used was also not predictive.

Table 2.

Learning and Study Approaches and Strategies Used Reported by More Than 50% of Respondents

Learning and Study Strategies	Responses, n	%
Visited the USMLE Step 2 CS website content to learn about the exam	38	75
Studied topics by completing content-focused blocks or sets of practice questions	38	75
Practiced writing patient notes prior to Step 2 CS	36	71
Spent more time “working” questions than reading, rereading, or watching videos	36	71
Reviewed the educational content or feedback on missed (incorrect questions)	35	69
Studied topics by completing random or mixed blocks or sets of practice questions	35	69
Visited the USMLE Step 2 CK website content to learn about the exam	32	63
Incorporated time for mental, physical, spiritual health, family, etc. in my study schedule	32	63
Role played or simulated Step 2 CS cases as practice	29	57
Utilized the results from practice tests to alter my study plan	27	53
Used video- or animation-based review materials	26	51

Abbreviations: USMLE, United States Medical Licensing Examination; CS, Clinical Skills; CK, Clinical Knowledge.

Table 3.

Review Resource Utilization and Perceived Importance

Resource Used	Total Count	%	Average Rating*
First Aid for Step 2 CS	45	88	4.64
OnlineMedEd	43	84	3.26
Study Notes	40	78	3.00
Anki (premade deck)	38	75	3.16
Step Up to Medicine	38	75	2.63
First Aid for Step 2 CK	38	75	2.50
MTB for Step 2 CK	37	73	2.05
Sketchy Medical	36	71	2.50
First Aid for Specific Specialties	35	69	2.00
Case Files	33	65	1.64
USMLE Step 2 Secrets	31	61	1.42
Master the Boards for Step 3	30	59	1.30
Blue Prints	30	59	1.27
Boards and Wards	30	59	1.23
Crush Step 2	30	59	1.23

Abbreviations: CS, Clinical Skills; CK, Clinical Knowledge; MTB, Master the Boards; USMLE, United States Medical Licensing Examination.

^*Rating Scale: 5=Extremely important, 4=Moderately important, 3=neutral, 2=slightly important, 1=not at all.

Table 4.

Question Bank Utilization and Perceived Importance

Question Bank (Qbank)	Count	%	Rating*
UWorld Step 2 CK Qbank	48	100	4.94
USMLE Step 2 CK Practice Questions (from website)	20	42	3.80
UWorld Step 2 CS	8	17	3.75
Amboss Step 2 CK Qbank	7	15	3.14
Board Vitals Step 2 Qbank	3	6	2.00
UWorld for Biostats Qbank	2	4	5.00
Kaplan Step 2 CK Qbank	0	0	N/A

Abbreviations: USMLE, United States Medical Licensing Examination; CK, Clinical Knowledge; CS, Clinical Skills.

^*Rating Scale: 5=Extremely important, 4=Moderately important, 3=neutral, 2=slightly important, 1=not at all

Table 5.

Significant Correlations to Step 2 CK Score

Category	Preparation or Resource	Correlation to Step 2 CK Score (Pearson)	n
Academic performance (Actual scores confirmed by an IRB approved honest broker)	USMLE Step 1 Score	.844**	51
	Pediatrics	.811**	44
	Surgery	.797**	41
	Internal medicine	.780**	45
	Ob-Gyn	.756**	41
	Psychiatry	.684**	42
	Comprehensive Clinical Skills Exam (first)	.784**	49
	Comprehensive Clinical Skills Exam (second)	.817**	46
Practice tests (Self-reported scores)	UWorld Self Assessment-1	.764**	34
	UWorld Self Assessment-2	.895**	32
	CCSSA 6	.460*	29
	CCSSA 7	.676**	29
	CCSSA 8	.642**	24
Prematriculation	MCAT Score	.374*	51
	Entry pathway	−.290*	51
Question bank	Average correct questions in question bank before Step 2 CK	.550**	48
	Percent of question bank completed before Step 2 CK	.290*	48
Resources	Step 2 Secrets	−.380*	31
	Online Medical Education	−.438**	43
	Master the Boards	−.367*	37
	First Aid for the USMLE Step 2 CK	−.335*	38

Abbreviations: CK, Clinical Knowledge; IRB, institutional review board; USMLE, United States Medical Licensing Examination; CCSA, Comprehensive Clinical Science Self-Assessment; MCAT, Medical College Admission Test.

^*Correlation is significant at the .05 level (2-tailed).

^**Correlation is significant at the .01 level (2-tailed). Negative correlations are shaded above.

Demographic variables (gender, race/ethnicity, underrepresented in medicine status, rural origin, and age) were not predictive of CK score (Table 1). Medical school entry pathway and MCAT score percentile were associated with Step 2 CK score (correlation coefficients 0.290 and 0.374, respectively; P=.020 and .007, respectively). In hierarchical regression, only completing more questions remained predictive of Step 2 CK score, after controlling for entry pathway, MCAT percentile, and Step 1. However, the impact was small.

Table 1.

Demographic Variables

Variable	N	%
Gender
Female	33	65
Male	18	35
Race and Ethnicity
White	34	68
Asian	5	10
Black or African American	5	10
Other	3	6
Multiple race/ethnicity	3	6
Underrepresented in Medicine
No	45	88
Yes	6	12
Rural or Metro Origin
Metro	45	88
Rural	6	12
Age (in Years)
20 – 24	34	67
25 – 29	14	27
30 +	3	6
Alternative Medical School Entry Pathway
No	45	88
Yes	6	12

Conclusions

Although students used a heterogenous mix of study strategies and resources, few were positively correlated with examination performance. Notably, students performed better if they focused on working through case-based, clinically-focused questions. By incorporating the regular review of case-based questions, students engaged in a form of practice testing, which has been found to enhance learning and long-term retention.¹⁰ Studying based on focused content areas did not lead to improved performance. This may be because students who study broad content areas are missing core foundational knowledge due to ineffective self-monitoring and the inability to identify and implement effective learning strategies.⁹

Results confirmed that student performance on clerkship exams was associated with performance on the Step 2 CK exam. Thus, students can reliably use these assessments as indicators of expected performance on Step 2 CK. This affirms results of a previous study with similar findings. ⁷ Researchers were unable to identify significant positive associations using specific resources (eg, texts, videos, study aids, question banks, practice tests) and performance on the USMLE Step 2 CK exam.

The results of this analysis are limited by the small sample size and evaluation of a single class at a single allopathic US institution. Further research into students’ Step 2 CK study habits is needed to identify and develop evidence-based best practices. Based on results of this study, students should be encouraged to work through case-based, clinically-focused questions as they prepare. Additional research focused on students’ ability to self-direct learning could also inform optimal preparation approaches to Step exams and appropriate educational interventions.