Observations of Score Changes Between USMLE Step 1 and Step 2 Among Students of Different Demographic Groups in a Longitudinal Clinical Curriculum

Abstract

BACKGROUND

The transition of the United States Medical Licensing Examination Step 1 to a pass/fail scoring system is reshaping its role in medical students’ residency placements. This compels institutions to rethink Step 2 preparation strategies, raising concerns about a clerkship's impact on various student groups. Traditionally, medical schools followed the traditional block rotation model for clerkships, which limits longitudinal learning, and many schools are switching to longitudinal integrated clerkships and longitudinal interleaved clerkships (LInCs). The growth in longitudinal popularity sparks concern for the success of diverse medical student groups as there is minimal research regarding LInC students’ USMLE performance. Our study aims to identify which student groups at Kirk Kerkorian School of Medicine at the University of Nevada, Las Vegas (KSOM) saw the greatest improvement in their USMLE Step scores after completing the LInC clerkship model.

METHOD

Utilizing institutional data from KSOM, 145 students from 3 KSOM cohorts’ Step 1 and Step 2 3-digit scores and their self-identified demographic information prior to the change in Step 1 grading were categorized by admissions and initial performance factors. Binary groups were created for each variable. Descriptive statistics and t-tests (including Levene’s test) gauged score change significance (P < .05) within these groups. Changes were assessed by subtracting Step 1 from Step 2 scores, identifying groups showing significant score improvements after completing the LInC clerkship.

RESULTS

Analysis revealed significant score improvements between Step 1 and Step 2 for the following groups: females, students with low socioeconomic status, and students who originally received lower Step 1 scores.

CONCLUSION

This study underscores the significance of gender, socioeconomic status, and prior exam performance in clerkship model development given the changes to Step 1 scoring. Further research should discern whether the observed score changes are attributed to the LInC model or its associated testing model.

Background

One of the most reliable predictors of medical students’ residency placement is their performance on the United States Medical Licensing Examination (USMLE) Step 1 and Step 2 CK. ¹ These exams are usually taken at the end of the secondary year and in the fourth year of medical school, respectively. ¹ Step 1 and Step 2 are used to evaluate a medical student's proficiency in applying knowledge and concepts to deliver patient care that is both safe and effective. ¹ However, as of 2022, the USMLE Step 1 exam score reporting shifted from numerical scoring to pass/fail. ² Consequently, medical schools may find it necessary to reassess their existing clerkship and Step 2 preparation models. ³ This newfound emphasis on Step 2, coupled with the evolving transition to longitudinal clerkship models, could potentially have a disproportionate impact on specific student demographic groups at the school level and nationwide.

Students classified as disadvantaged, including those from lower socioeconomic backgrounds or underrepresented groups in medicine, face significant hurdles when seeking admission to medical school. This is in contrast to their peers who may have parents with higher education levels or fewer financial obstacles, for example. ⁴ These barriers can significantly hinder the likelihood of these students applying to and successfully enrolling in medical school and play a prominent role in explaining the range of disparities observed in application and matriculation rates. ⁴ As they enter the field of medicine, perpetuating these challenges through potentially antiquated program designs would not only be an extension of these obstacles but also a great disservice to its students. Therefore, it is of the utmost importance to thoroughly assess current medical education program designs and the achievements of the students who undergo these rigorous processes.

Historically, medical schools have used the traditional block rotation (TBR) model for third-year medical clerkships. Through this model, students participate in a multiweek block focused on the practices of one specialty, such as surgery or internal medicine, and complete a written exam prior to moving onto the next multiweek rotation in a different specialty. ⁵ Despite being the traditional model, it does come with its limitations, such as a lack of continuity of patient care and difficulty building relationships with patients and physicians leading to the adoption of a longitudinal integrated clerkship (LIC) model by many schools. ⁶ The LIC was first used by the University of Minnesota in 1971. ⁵ This model of clerkship typically requires students to rotate through various specialties daily and forces students to focus on longitudinal learning and developing a more well-rounded foundation of clinical care. This transition to LICs was a way of challenging the traditional approach to clinical education of sequential, time-limited rotations through specialty departments. ⁷ The LIC model does have some variability but typically share 3 similarities: (1) medical students engage in the care of patients over an extended period of time; (2) medical students foster ongoing professional relationships with the clinicians who are also overseeing the patients; and (3) these immersive experiences allow the medical students to fulfill a large amount of their core clinical competencies across many disciplines at the same time. ⁷ The variabilities that do occur between schools that elect to use the LIC models ultimately depend on the schools’ preferences and needs. Research shows that longitudinal clerkships promote students’ educational and professional development by allowing for a deeper understanding of diseases and the development of critical clinical skills.^7–10 Those students who participated in LICs also felt more prepared to care for patients, understood how social aspects influence patients, and how to approach ethical decisions in a more confident manner. ¹⁰

The University of Colorado School of Medicine (CUSOM) culminated their own LIC program in 2014, called the Denver Health LIC (DH-LIC). ¹¹ Eight to ten students from the CUSOM program participated in a LIC-based curriculum which differed from the rest of the cohorts’ curriculum model. ¹¹ Those who participated in the DH-LIC program outperformed their peers and the national mean on Step 2 and National Board of Medical Examiners (NBME) clinical science subject exams. ¹¹ The DH-LIC model was so successful that the CUSOM program only offers the LIC clerkship model now. Similarly, from 1998 to 2004 at the University of North Dakota, students who participated in their nontraditional longitudinal clerkship model outperformed their traditional clerkship counterparts on clinical proficiency ratings. ¹² From 2009 to 2017, the University of Hawaii also compared their own LIC students versus the traditional track students and found that those in the LIC program gained more clinical experience during their obstetrics and gynecological clerkships. ¹³ From 2004 to 2007, the Harvard Medical School-Cambridge Integrated Clerkship also found the students who participated performed better on their Objective Standardized Clinical Examination (OSCE) compared to the traditional track students. ¹⁴ The overall success of LIC programs has caused many medical schools to adopt the LIC model, growing from 29 programs in 2010-2011 to 44 in 2015-2016 based on the Association of American Medical Colleges (AAMC) most recent data. ¹⁵

As medical schools continue to transition to longitudinal clerkship models, some programs are unable to adapt or implement LICs as they have previously been designed. The newly established Kirk Kerkorian School of Medicine at the University of Nevada, Las Vegas (KSOM) is one of these institutions that wanted to ensure the LIC model met student needs with the resources available in their program. To ensure these needs were met, KSOM created their own variation, the longitudinal interleaved clerkship (LInC). The LInC model involves rotating through specialties with multiple 2-week periods, rather than individual days as is done in most LICs. ¹⁶ Students take all Clinical Subject Exams once halfway through the year and a second time upon completion of the academic year. ¹⁶ KSOM has successfully graduated 3 classes of students who participated in the LInC schedule.

The inevitable route of medical programs seems to be leaning toward a transition to LIC-based models. However, there is very little research done that delves into how LInC or LIC models affect student groups of various demographics, especially in the context of their performance on the USMLE Step 1 and Step 2 CK despite evidence that highlights demographic disparities between Step 1 and Step 2 exam scores. A notable study from 2019 shed light on these disparities, revealing that certain underrepresented groups, including English as a second language test takers and nonwhite students, exhibited lower performance across Steps 1, 2, and 3 compared to their English-proficient and self-identified white counterparts. ¹⁷ This finding echoes an earlier 1994 study, which established that white students consistently achieved notably higher scores on Step 1 in comparison to their peers from other racial backgrounds. ¹⁸ Another article reported that both perception of and actual economic status affected students’ performance on the USMLE exams, noting that low socioeconomic status (SES) students and those that perceived themselves as being of a low SES performed worse than their peers. ¹⁹ Furthermore, the previously mentioned 2019 study also established a link between Medical College Admissions Test (MCAT) scores and subsequent Step exam performance, underscoring the predictive value of MCAT scores for discerning Step score discrepancies among underrepresented minority students. ¹⁷ Multiple studies have identified performance variations between genders. These studies indicate that, on average, male students achieved higher scores on Step 1 than their female counterparts, while females, on average, surpassed their male peers on Step 2.^17,20 However, one study outlined that there was no difference in performance between LIC or TBR clerkship, indicating that neither the LIC nor TBR clerkships affected this pattern and both yielded similar results among both genders. ²¹

Given that these patterns are important predictors for success on Step exams, the lack of research on how LIC and LInC models affect these vulnerable demographics highlights the need for further research. Such exploration may better inform medical schools’ clerkship model decisions and aid incoming students in selecting the best medical school for their needs. This study aims to identify which student demographic groups at KSOM experienced the most significant improvement between USMLE Step 1 and Step 2 following their participation in the LInC clerkship model and testing schedule. By addressing this gap in research, these findings seek to inform a reevaluation of program models to better promote student success.

Methods

Data were drawn from KSOM institutional databases in accordance with an approved IRB protocol. Data examined in the study included Step 1 and Step 2 3-digit scores along with readily available admission demographic data of student subjects. Only Step 1 data prior to the change to pass/fail were included in the study. All students enrolled in the classes of 2021, 2022, and 2023, a total of 145 students, at KSOM were included in the analysis. The class of 2025 and beyond was excluded as these students received the pass/fail Step 1 exam and cannot be quantified.

Each of the 145 students was categorized based on their admission demographic survey information and initial exam performance, including underrepresentation in medicine (URM), gender, self-report disadvantage (SRD), first generation (FirstGen), low socioeconomic (low SES), low MCAT, and low Step 1 performance. These demographic variables were utilized in this research presentation because they are consistently collected by the institution prior to students’ matriculation and were readily available for analysis. They encompass a wide array of diverse characteristics, from gender to current representation in the medical field. Consequently, it was determined that additional variables were unnecessary for this research, as incorporating them could introduce confounding factors, particularly given the potential for retroactive recall bias in their collection as well as requiring an alternative survey modality with varying amounts of time between matriculation of the classes, the initial survey, and when this project was conducted.

Each variable was divided into binary groups based on its definition. URM, SRD, and FirstGen were defined with a simple no or yes according to admission demographic data surveys. Low SES is defined as EO1-EO2 status per the AAMC definition and low MCAT is defined as scores of 505 or lower and both were classified as either no or yes. Low Step 1 performance is defined as a score below the institution's mean score and was also grouped as no or yes, no indicating students who scored above the institution's mean and yes for those who scored below. Gender was grouped based on self-identified males or females according to KSOM admission demographic data surveys. Comparisons were made between the binary categories within each variable to determine which groups of students experienced the most change between their Step 1 and Step 2 scores. Change was calculated by subtracting Step 1 scores from Step 2 scores. Additionally, a chi-squared test was performed to evaluate trends between males and females within each variable. The reporting of this study conforms to the STROBE statement for cross-sectional studies (STROBE cross-sectional checklist). ²²

Statistical analysis

To analyze the data, the mean Step 1 and Step 2 were calculated into 3-digit scores for each group within each of the 7 variables. To determine which students within each variable experienced a significant change in scores between Step 1 and Step 2, the difference between Step 1 scores and Step 2 scores in each binary group for each variable was calculated. Descriptive statistics and t-tests (with Levene's test) between the 2 differences were performed to compare the average change between groups within a given variable. A p-value of 0.05 was used to determine statistical significance, enabling us to identify which groups experienced a significant increase in scores following completion of the LInC clerkship and testing model. Levene's test for all variables indicated no significant difference of variance between groups, so t-tests were run with equal variances assumed. The same process was performed to evaluate the change in students’ percentile ranks between Step 1 and Step 2.

Results

Because gender is known to be an important predictor of Step 1 and Step 2 scores, crosstabs with chi-square significance tests were run on the other independent variables. Table 1 illustrates these relationships between gender and the other independent variables. The only variable of significance was low MCAT, where males were overrepresented in low MCAT compared to females.

Table 1.

Chi-square distribution by gender of KSOM students in the classes of 2021-2023.

	FEMALE N, %		MALE N, %		CHI-SQUARE	SIG
Not URM	60	84.5	58	78.4	0.898	0.343
URM	11	15.5	16	21.6
Not SRD	58	81.7	58	78.4	0.248	0.618
SRD	13	18.3	16	21.6
Not FirstGen	58	81.7	52	70.3	2.581	0.108
FirstGen	13	18.3	22	29.7
Not low Step 1	34	47.9	43	58.1	1.520	0.218
Low step 1	37	52.1	31	41.9
Not low SES	45	63.4	46	62.2	0.023	0.879
Low SES	26	36.6	28	37.8
Not low MCAT	61	85.9	53	71.6	4.404	0.036
Low MCAT	10	14.1	21	28.4

Table 2 outlines the variables evaluated for this study: URM, gender, SRD, FirstGen, low SES, low MCAT, and low Step 1. The subsequent columns show the dispersion of KSOM students within the binary groups of each variable. The Step 1 and Step 2 columns include the mean Step 1 or Step 2 score for each group. Score change is the difference between the mean Step 1 and mean Step 2 score for each group. Of note, all of these changes were positive, indicating each group's mean Step score increased. As previously mentioned, t-tests were used to evaluate if there was a significant difference in the improvement between Step 1 and Step 2 between each group within their respective variables. The results of these statistical analyses can be found in the last column of Table 2. A 2-sided P-value less than .05 indicates a statistically significant difference in the increase of score between Step 1 and Step 2 between the binary group of each of the 7 variables evaluated.

Table 2.

Step 1 to Step 2 change means and significance.

		N	STEP 1	STEP 2	SCORE CHANGE	T-TEST	TWO-SIDED P
URM	No	118	232.2797	247.8559	15.5763	0.725	.47
	Yes	27	223.8519	241.1852	17.3333
Gender	Female	71	228.8732	248.2254	19.3521	−3.743	<.001
	Male	74	232.4730	245.0676	12.5946
SRD	No	116	232.5431	248.1552	15.6121	0.617	.538
	Yes	29	223.3793	240.4483	17.0690
FirstGen	No	110	231.3091	247.0818	15.7727	0.245	.807
	Yes	35	228.8286	245.1429	16.3143
Low SES (EO1-EO2)	No	91	231.6703	246.1429	14.4725	1.991	.048
	Yes	54	229.0926	247.4074	18.3148
Low MCAT (≤505)	No	114	232.6930	248.0965	15.4035	1.017	.311
	Yes	31	223.4194	241.1613	17.7419
Low Step 1	No	77	242.3506	253.9481	11.5974	5.300	<.001
	Yes	68	217.5294	238.3088	20.7794

Abbreviations: URM, underrepresented in medicine; SRD, self-reported disadvantage; FirstGen, first generation; SES, socioeconomic status.

Table 3 illustrates the percentiles and changes between Step 1 and Step 2. Each variable is divided into those below and above the schools’ Step 1 mean. The results of the t-test and each associated significant level are present in the last column of the table.

Table 3.

Step 1 to Step 2 percentile change and significance.

		N	PERCENTILE MEAN	STANDARD DEVIATION	STANDARD ERROR MEAN	T-TEST	TWO-SIDED P
Step 1 percentile	Step 1 below mean	68	22.6324	13.2720	1.6095	−20.331	<.001
	Step 1 above mean	77	72.4935	15.9172	1.8139
Step 2 percentile	Step 1 below mean	68	31.4559	22.7815	2.7627	−8.392	<.001
	Step 1 above mean	77	64.5455	24.4710	2.7887
Percentile change	Step 1 below mean	68	8.8088	19.9209	2.4158	4.787	<.001
	Step 1 above mean	77	−7.9740	22.0293	2.5105

Figure 1 simplifies the findings of Table 2 to show which groups experienced a statistically significant difference in their Step score change. The highlighted variables indicate which variables were significant. The color red is used to denote which group saw a statistically significant greater change in their Step 1 to Step 2 scores in reference to their comparison group, represented with the color blue.

Figure 1.

Average score changes by category.

Our findings demonstrate that females, low SES students, and students who initially received low Step 1 scores experienced a statistically significant increase between their Step 1 and Step 2 scores compared to their male, not low SES, and not low Step 1 comparison groups.

Discussion

As outlined in Table 2, the findings of this study suggest that the students who identified as females, had a low SES, or had a low score on Step 1 experienced a statistically significant score increase between their Step 1 and Step 2 scores, compared to their pair counterparts after the completion of the LInC clerkship. Figure 1 simplifies the information found in Table 2 and shows that females improved by 19.35 points, those with low SES improved by 18.31 points, and those with a low Step 1 improved by 20.78 points. All 3 of these improvements were statistically significant, as outlined by significance levels of Table 2.

Our study, like those before, supports the pattern of score disparities identified between males and females.^17,20 The average Step 1 score for females was lower than the male average score, but the average Step 2 score for females was higher than the male average score. The study done by Latessa et al ²¹ found that when comparing gender, they did not have significant differences in performance after completion of an LIC or TBR clerkship. This contrasts with our findings from students who completed the LInC model of clerkship. The difference between gender performance in KSOM LInC program as noted in our study could have implications for future programs looking to support more incoming female students.

In this study, students with low SES saw a significant increase in scores between Step 1 and Step 2. This is consistent with the findings of Jerant et al ¹⁹ and may be due to the structure of the LInC clerkship and its focus on longitudinal learning. It should be noted that there was also no statistically significant difference between URM and SRD students’ score improvement compared to their peers upon completion of the LInC model.

Interestingly, previous studies have shown that MCAT scores and Step 1 scores are both positively associated with Step 2 scores.^17,23 Our sample demographics in Table 1 reveal that just over 21% of students, both male and female, had MCAT scores at or below 505. Within this same sample group, the number of Step 1 scores that fell below the institution's mean was nearly 47%. Since both of these 2 variables are known to correlate to Step 2 performance, it is worth discussing the rise in low Step 1 scores compared to the number of low MCAT scores prior to matriculation. Future research should be performed to evaluate this trend and identify risk factors attributing to the increase in lower standardized exam scores. Furthermore, a cross tabulation of Step 1 and Step 2 scores relative to their school mean revealed that 29% of students who initially scored below the Step 1 school mean eventually scored above the Step 2 school mean. This change prompted the creation of Table 3, which indicates the change of percentile rank within the Step 1 variable. This table shows that the improvement between Step 1 and Step 2 was not only based on raw scores but also percentile rank. Students who initially scored below the mean on Step 1 showed a significant improvement in their mean percentile rank from 22.63 to 31.56. This change in percentile rank supports the hypothesis that this improvement in the schools’ mean scores is more than simple regression to the mean.

Programs that have implemented longitudinal clerkship models, such as those in Colorado, North Dakota, and Hawaii, have reported significant benefits for students. Students in these programs not only outperformed their peers from traditional block models on Step 2 and NBME clinical science subject exams ¹¹ but also achieved higher clinical proficiency ratings ¹² and gained more clinical experience. ¹³ Additionally, those who participated in the longitudinal model demonstrated superior performance on their OSCEs. ¹⁴ This model of education was a single intervention that each student experienced, but education and training varies for individual students. Given the nature of the current study, it was not possible to make comparisons with other curricular models and further study will be needed to determine if these findings exist within those models. While this study cannot directly compare results to TBRs, it suggests that students—particularly females, those from low socioeconomic backgrounds, and those with initially low Step 1 scores—may significantly benefit from a longitudinal clerkship model like LInC. Future research should prioritize qualitative studies that gather subjective input from participants, utilizing methods such as focus groups or interviews. This approach would provide valuable personal testimonies related to the clerkship model, enhancing our understanding of the experiences of individuals in this study and similar contexts.

Limitations

First, KSOM did not have a traditional block clerkship to serve as a control group, making it unreasonable to attribute the observed results solely to the longitudinal clerkship model. The multifactorial influences on students’ education between Step 1 and Step 2 cannot be accurately measured, and it would be misleading to assume that any single factor significantly contributed to the improvements noted. Instead, this study can only report the findings observed among students who participated in the longitudinal clerkship. Second, this research was confined to a single institution and involved a limited sample size, with only 3 cohorts at KSOM eligible for inclusion due to changes in Step 1 grading. Additionally, various confounding factors may have influenced the students’ scores between Step 1 and Step 2, yet this study focused on just 7 variables across these cohorts.

Conclusion

This study identified which medical student groups at KSOM saw a greater increase between their Step 1 and Step 2 scores after compiling the LInC clerkship model. The results demonstrated that females, students with a low SES, and students with a low Step 1 scores saw a statistically significant increase between their Step 1 and Step 2 scores compared to males, students not of low SES, and students who did not receive a low Step 1 score.

Given the small sample size of this study and its limitations to its external validity, future research should investigate if the change in scores that was seen in students belonging to these 3 variables is due to the LInC clinical rotation model, the testing schedule, or if this trend is due to other multifactorial variables. This research along with our conclusions could have important implications for medical schools as they adjust their curriculum to accommodate the transition to pass/fail scoring of Step 1 and prepare their students for Step 2 and residency applications.

Data Availability

Raw data can be made available upon reasonable request to the correspondence author.