From prediction to advising: a multi model approach using NBME subject examinations to inform step 2 CK performance

Abstract

Background

Following the transition of the USMLE Step 1 to pass/fail scoring, Step 2 Clinical Knowledge (CK) has become a primary metric in residency screening. This study evaluates multiple approaches to identify performance patterns that may support individualised longitudinal advising and targeted preparation aligned with students’ educational and specialty goals.

Methods

De-identified first-attempt scores from 819 medical students who completed Step 2 CK between May 2021 and April 2026 were analysed. Three models were applied: linear regression, a conditional inference tree, and weighted logistic regression to predict at-risk status. Outcomes included continuous Step 2 CK scores and an at-risk classification defined as scoring at least 1 standard deviation below the national mean for the corresponding passing standard. Model outputs were synthesised to identify performance patterns relevant to advising.

Results

First-attempt pass rates ranged from 95% to 98% across testing years. 184 students (22.4%) met the at-risk criteria. Paediatrics, Internal Medicine, and Neurology showed relatively strong associations with Step 2CK performance. Linear regression explained 71.8% of the variance in Step 2 CK scores. The conditional inference tree produced 17 terminal nodes and yielded a mean root mean squared error of 10.63. Weighted logistic regression demonstrated good discrimination for at-risk classification (AUC = 0.915).

Conclusions

Combining these modelling approaches offers a multidimensional view of student performance that may support targeted preparation for Step 2 CK. By integrating prediction, risk stratification, and performance interpretation, we developed the TRACE Framework to translate institutional assessment data into actionable insights for individualised residency advising.

Introduction

The transition of the United States Medical Licensing Examination (USMLE) Step 1 to pass/fail scoring in 2022 reshaped altered the residency selection landscape [1–3]. In the 2024 National Resident Matching Programme (NRMP) Programme Director Survey, programme directors reported using medical licensure examination results in interview selection, and Step 2 Clinical Knowledge (CK) has become an increasingly important numerical metric in applicant review and ranking [3]. Consequently, identifying factors associated with Step 2 CK performance has become a central priority for medical schools seeking to strengthen academic advising and support student match success.

Prior literature characterises Step 2 CK achievement as a multifactorial outcome associated with pre-matriculation metrics such as Medical College Admission Test scores and undergraduate grade point average, as well as prior standardised examination performance and clinical clerkship evaluations [4–10]. The studies show that Step 2 CK is further associated with downstream measures of resident performance, including in-training examination outcomes and board certification-related measures, supporting its continued salience in residency selection and postgraduate prediction.

Building on these associations, predictive modelling studies have demonstrated that longitudinal academic data collected during medical school can be used to estimate Step 2 CK performance, with model performance improving as additional clerkship data become available [11–13]. However, much of the existing work emphasises prediction accuracy rather than interpretability and practical use at the level of advising. For faculty and advisors, the key challenge is not simply whether Step 2 CK can be predicted, but how to translate available academic data into timely, actionable guidance for individual learners. The proposed approach aims to bridge this gap by evaluating longitudinal data across clinical rotations to inform targeted interventions before the final exam. Accordingly, we developed and evaluated complementary modelling approaches using NBME clinical subject examination performance. Specifically, we sought to (1) quantify the association between individual clerkship examination performance and Step 2 CK outcomes, (2) generate interpretable estimates of expected Step 2 CK performance using available academic data, and (3) identify patterns of performance that may inform academic advising regarding targeted preparation for Step 2 CK aligned with students’ educational or specialty goals. By evaluating performance across sequential clinical rotations, institutions can deploy targeted academic interventions before students sit for the examination.

Methodology

Ethics

This study was reviewed and approved by the University of Arkansas for Medical Sciences Institutional Review Board: IRB number 239432. All procedures involving human participants were conducted in accordance with the ethical standards of the institutional review board and with the principles of the Declaration of Helsinki. The requirement for informed consent was waived by the institutional review board because the study involved retrospective analysis of de-identified data and posed minimal risk to participants.

Study design and study population

This study used retrospective, de-identified academic performance data obtained from five consecutive cohorts of medical students at a single public medical school who completed the USMLE Step 2 CK examination between May 1, 2021, and April 30, 2026. This period coincides with the end of third-year clerkships at our institution, when students begin preparing for Step 2 CK. Because students may take Step 2 CK on individualised timelines due to dual-degree programmes, research years, or other academic tracks, the study population was not stratified by graduating class. Instead, students were grouped according to the Step 2 CK passing standard in effect when they first took the examination, as both the passing standard and the national statistics used to define the at-risk threshold depend on test date. Examination periods were pooled to maximise sample size and model stability, consistent with the study's aim of prediction rather than characterisation of trends over time.

The analytic sample was restricted to first-attempt NBME clinical subject examination scores and first-attempt Step 2 CK scores. Repeat attempts for either NBME subject examinations or Step 2 CK were excluded to maintain consistency between predictor and outcome measures. Preclinical academic metrics, including MCAT scores, undergraduate GPA, preclinical course performance, and NBME Comprehensive Basic Science Examination scores, were excluded because they were not temporally aligned with the clinical clerkship sequence and were outside the focus of this clerkship-based analysis.

Step 2 CK scoring context

The USMLE Step 2 CK passing standard in effect at the time of examination (209, 214, or 218) is presented alongside contemporaneous national performance statistics in Table 1.

Table 1.

National Step 2 CK performance statistics and definition of “at-risk” threshold by examination period.

Examination period a	Step 2 CK passing standard b	n	National mean (SD)	At-risk threshold c
2021–2022	209	158	247 (15)	≤232
2022–2023	209/214	165 (43/122)	248 (15)	≤233
2023–2024	214	168	249 (15)	≤234
2024–2025	214	170	250 (15)	≤235
2025–2026 d	214/218	158 (55/103)	250 (15)	≤235

^a Examination periods reflect May 1 - April 30 cycles, corresponding to the period during which most third-year medical students complete Step 2 CK before entering their fourth year.

^b The passing standard increased from 209 to 214 on July 1, 2022, and from 214 to 218 on July 1, 2025. Students were assigned to the passing standard in effect on their test date; for periods spanning a change, n is shown as (earlier/later standard). Overall: 209, n = 201; 214, n = 515; 218, n = 103.

^c At-risk defined as a Step 2 CK score at or below 1 SD below the national mean (mean − 1 SD) for the examination period in which the student tested.

^d Includes students tested through September 2025. National statistics for 2025–2026 were not published at the time of submission; values from 2024–2025 were used.

Predictor and outcome variables

The primary predictors were first-attempt NBME clinical subject examination scores from core clerkships. These were standardised subject examinations administered during clinical training and were analysed as raw scores. The primary outcome was the USMLE Step 2 CK score, treated as a continuous variable. A secondary binary outcome, at-risk status, was defined as a Step 2 CK score at least one standard deviation below the national mean for the corresponding academic year (Table 1).

Statistical analyses

All analyses were conducted in R version 4.5.2. Statistical significance was defined as p < 0.05. Analyses included association modelling using linear and nonlinear approaches to characterise relationships between NBME subject examination scores and Step 2 CK performance, and classification modelling to identify students meeting the predefined at-risk threshold. Descriptive statistics were calculated for Step 2 CK and NBME clinical subject examination scores across risk groups. One-way analysis of variance was used for exploratory comparisons of mean scores across groups.

To examine the association between NBME clinical subject examination performance and Step 2 CK scores, we fit a multiple linear regression model using all available NBME subject examination scores as predictors. Regression coefficients represent the expected change in Step 2 CK score associated with a one-point increase in each subject examination score. Model fit was assessed using R² and adjusted R², and 95% confidence intervals are reported.

To explore nonlinear relationships and subgroup patterns, we fit a conditional inference tree using raw scores from all seven NBME clinical subject examinations and passing-standard era as predictors of Step 2 CK performance. Tree growth was constrained using a minimum split p-value of 0.05, maximum depth of 5, and minimum terminal node size of 35. Model performance was evaluated using 10-fold cross-validation, and results are reported as terminal node distributions and prediction error.

Finally, to classify students as at risk for lower Step 2 CK performance, we fit a weighted logistic regression model to predict at-risk status. Because the outcome was imbalanced, with approximately 22% of students classified as at risk, inverse weighting was applied to address class imbalance. Model performance was evaluated using the area under the receiver operating characteristic curve (AUC). An optimal probability threshold was selected using Youden’s index, and classification performance was reported as accuracy, sensitivity, specificity, positive predictive value, and negative predictive value.

Results

The cohort included 819 students. Of these, 201 students were tested under a Step 2 CK passing standard of 209, 515 under a standard of 214, and 103 under a standard of 218. First-attempt Step 2 CK pass rates ranged from 95% to 98% across passing standards. Using the predefined risk classification, 184 students (22.4%) were classified as at risk. This included 28 students who failed Step 2 CK on the first attempt and 156 students who passed but scored at least 1 standard deviation below the national mean for their respective passing standard. The remaining 635 students (77.5%) were classified as not at risk.

Table 2 presents Step 2 CK and NBME clinical subject examination scores stratified by risk group. Mean Step 2 CK scores differed across groups, with the highest scores observed in the not-at-risk group (250.4 ± 10.8), intermediate scores in the at-risk group who passed but were at least 1 standard deviation below the national mean (225.8 ± 5.9), and the lowest scores among students who failed on the first attempt (205.0 ± 8.3). Similar patterns were observed across all NBME clinical subject examinations, with all comparisons showing statistically significant differences (all p < 0.0001).

Table 2.

USMLE Step 2 CK and NBME subject examination scores by risk group.

Measure	At risk: first attempt failure (N = 28)	At risk: passed but ≤1 SD below national mean (N = 156)	Not at risk (N = 635)	P value
USMLE Step 2 CK	205 (8.3)	225.8 (5.9)	250.4 (10.8)	<0.0001
Family Medicine	64.8 (6.3)	70.3 (6.4)	76.2 (7)	<0.0001
Internal Medicine	62.1 (5.3)	66.1 (6.6)	75.9 (7.5)	<0.0001
Neurology	65.6 (9.2)	71.8 (6.6)	81.1 (7.1)	<0.0001
Obstetrics and Gynaecologist	65.8 (7)	72.1 (6.8)	80.3 (7)	<0.0001
Paediatrics	65.3 (5.7)	70.6 (6.2)	80.1 (6.7)	<0.0001
Psychiatry	76 (7.7)	79.8 (6.1)	86.3 (5.2)	<0.0001
Surgery	63.2 (6.5)	66.6 (7)	75.7 (7.6)	<0.0001

Note: Values are mean (SD). At risk was defined as first-attempt Step 2 CK failure or a first-attempt Step 2 CK score at least 1 standard deviation below the national mean for the corresponding academic year. P values were derived from one-way analysis of variance.

Model 1: linear regression

Linear regression was used to estimate associations between NBME clinical subject examination scores and Step 2 CK performance. All seven NBME subject examinations were statistically significant predictors of Step 2 CK scores (Table 3).

Table 3.

Linear regression coefficients for predicting Step 2 CK performance.

Predictor	Coefficient (β)	95% CI	P-value
Family Medicine	0.17	0.07–0.26	<.001
Internal Medicine	0.32	0.22–0.43	<.001
Neurology	0.31	0.20–0.42	<.001
Obstetrics and Gynaecologist	0.32	0.21–0.44	<.001
Paediatrics	0.46	0.34–0.58	<.001
Psychiatry	0.24	0.11–0.37	<.001
Surgery	0.23	0.12–0.32	<.001
Passing standard (214)	1.37	−0.01–2.77	.050
Passing standard (218)	0.84	−1.17–2.86	.326

Note: Values are regression coefficients with 95% confidence intervals. Passing standard 209 was the reference category.

The model explained 71.8% of the variance in Step 2 CK scores (R² = 0.718, adjusted R² = 0.715). The largest coefficients were observed for Paediatrics (β = 0.46), Obstetrics and Gynaecologist (β = 0.322), and Internal Medicine (β = 0.32). Coefficients for all NBME subject examinations were positive. Indicators for national passing standard were small and not statistically significant at the 0.05 level, except for the 214 standard, which was of borderline significance.

Model 2: conditional inference tree

The conditional inference tree produced 17 terminal nodes based on sequential splits of NBME clinical subject examination scores (Figure 1). Paediatrics was the first and strongest split (≤78 vs >78), followed by major splits involving Neurology and Internal Medicine, with additional terminal node structure contributed by Surgery, Obstetrics and Gynaecologist, and Psychiatry. Ten-fold cross-validation yielded a mean root mean squared error of 10.63 (SD 0.81).

Figure 1.

Conditional inference tree predicting Step 2 CK scores.

Model 3: logistic regression

The weighted logistic regression model demonstrated good discrimination for identifying students meeting the predefined at-risk classification (AUC = 0.915). Using the optimal probability threshold derived from Youden’s index, the model achieved an accuracy of 0.83 (95% CI, 0.80–0.85), with sensitivity of 0.88, specificity of 0.81, positive predictive value of 0.57, and negative predictive value of 0.96. Higher NBME clinical subject examination scores were associated with lower odds of being classified as at risk (Table 4).

Table 4.

Logistic regression predicting at‑risk status.

	OR	95% CI	p - value
Family Medicine	0.98	0.96–1.01	0.128
Internal Medicine	0.92	0.90–0.94	<.0001
Neurology	0.94	0.91–0.96	<.0001
Obstetrics and Gynaecologist	0.97	0.95–1.00	0.042
Paediatrics	0.91	0.88–0.93	<.0001
Psychiatry	0.95	0.92–0.97	<.001
Surgery	0.95	0.92–0.97	<.0001
Passing standard 214	0.93	0.66–1.31	0.682
Passing standard 218	0.97	0.60–1.56	0.896

Note: Values are odds ratios with 95% confidence intervals. Passing standard 209 was the reference category.

Discussion

This study evaluated the relationship between NBME clinical subject examination performance and USMLE Step 2 CK outcomes using three complementary analytic approaches. Across models, NBME subject examination scores were consistently associated with Step 2 CK performance and with classification into predefined risk groups. Although Paediatrics, Internal Medicine, and Neurology showed relatively strong associations in this dataset, the magnitude and ranking of subject examination contributions varied by model. These findings suggest that clerkship-level assessment data contain clinically and educationally meaningful information about later Step 2 CK performance.

Previous studies have shown that Step 2 CK performance can be predicted using a broad range of academic measures. Jacobparayil et al summarised evidence supporting associations with premedical academic metrics, standardised examinations, clerkship performance, and other learner characteristics [8]. Similarly, Kaur et al incorporated MCAT scores, CBSE performance, preclinical examinations, and NBME subject examinations into a comprehensive multivariable prediction model [13]. Furthermore, contemporary multi-cohort modelling frameworks [12] and longitudinal predictive systems have heavily relied on accumulating multiple [11], varied historical datasets over time to refine their predictions. In contrast, our goal was not to maximise predictive performance by combining all available historical academic variables. Instead, we examined whether routinely collected NBME clinical subject examination data alone could provide informative and interpretable estimates to support longitudinal academic advising during the clinical curriculum.

Our findings are broadly consistent with prior work demonstrating that NBME clinical subject examination performance is associated with subsequent Step 2 CK outcomes, although the clerkships contributing most strongly to prediction differed across studies. Kaur et al identified Surgery, Neurology, and Psychiatry as the strongest predictors in a multivariable model that also included MCAT scores, CBSE performance, and preclinical assessments [13]. In our study, Paediatrics and Internal Medicine emerged as particularly informative across analytic approaches, with Neurology also showing consistent association. These differences may reflect institutional variation in curriculum, assessment practices, and model specification, suggesting that the relative contribution of individual clerkships may not be uniform across medical schools.

Bird et al showed that prediction of Step 2 CK improves as students advance through training and additional assessment data become available [11]. Our findings complement that work, as well as the predictive modelling framework outlined by Huynh et al, by emphasising the educational use of accumulating clerkship data rather than prediction alone [12]. Specifically, our results support the use of sequential NBME subject examination performance as part of longitudinal advising conversations that evolve during the clinical curriculum. More broadly, the study reinforces that NBME subject examination performance can provide meaningful context for subsequent Step 2 CK outcomes when interpreted within the setting in which the scores were obtained.

From an applied perspective, these findings support structured academic advising by mapping sequential clerkship performance to Step 2 CK distributions, enabling individualised coaching over a one-size-fits-all approach. This approach advances precision medical education by translating routine, data-driven patterns into actionable, personalised student guidance [14–16].

In this context, conditional inference trees may be useful for describing how specific patterns of subject examination performance correspond to historical Step 2 CK outcome ranges within the institutional dataset. This methodology supports a broader discussion of performance relative to prior cohorts rather than generating isolated, individualised predictions. Such a framework aligns directly with the core tenets of precision medical education, which advocates for using institutional data to create adaptive, highly personalised learning pathways. However, the interpretation of these conditional outputs must remain cautious, context-dependent, and heavily mediated by skilled advisors. While data summaries are valuable for structured student reflection, prior work has highlighted the potential for raw predictive performance feedback to increase learner stress and negatively influence self-perceptions of ability [17]. Therefore, to safely translate this data into actionable guidance, advisors should act as coaches who use these risk profiles not to label students, but to co-create tailored, supportive action plans as learners navigate stressful clinical transitions. The models are intended to function as adjunct tools that support, not replace, holistic academic advising.

Operationalizing the models: the TRACE framework

To translate these findings into educational practice, we propose the TRACE Framework, a structured, data-informed workflow for integrating predictive modelling into longitudinal academic advising (Figure 2). TRACE conceptualises predictive modelling as one component of an iterative advising process rather than as a stand-alone prediction tool.

Figure 2.

The TRACE Framework: An iterative workflow for integrating NBME subject examination performance into longitudinal Step 2 CK academic advising.

The framework begins with Tracking student performance across NBME clinical subject examinations as new assessment data become available. At our institution, these data are incorporated into an existing student progress dashboard, enabling advisors and academic leadership to review updated performance information in near real time [18]. Performance trends are then Recognised by comparing individual trajectories with historical institutional cohorts and identifying students whose developing patterns may warrant additional academic support.

The next step is Advising and Contextualising model outputs within each student’s educational goals. After several clerkships have been completed, model estimates can be used to generate expected ranges of Step 2 CK performance based on historical cohort data. These estimates provide an opportunity for early, low-stakes advising discussions focused on remediating specific clerkship-specific knowledge gaps and tailoring study strategies for the remainder of the clinical curriculum. Because evidence-based study resource selection, such as optimised question bank usage, spaced repetition, and formal time management, is heavily correlated with superior Step 2 CK outcomes [19], advisors can use these predictive ranges to steer students away from passive review methods and toward high-yield, structured preparation programmes. As additional clerkship examination data accumulate, projected Step 2 CK performance becomes increasingly informative, allowing advisors to refine recommendations using progressively more complete academic profiles. For students pursuing highly competitive specialties, projected Step 2 CK performance may also be considered alongside specialty-specific outcome data, such as Charting Outcomes in the Match [20], to inform discussions about competitiveness, examination preparation, and application strategy.

Finally, TRACE incorporates Evaluation through ongoing review of student progress and model performance. Individual advising recommendations can be revisited as new clerkship assessment data become available, while model performance can be periodically recalibrated using data from subsequent student cohorts. In this way, TRACE functions as an iterative advising process that combines longitudinal assessment data with predictive modelling to support timely, individualised educational guidance. This approach allows predictive relationships to evolve with changes in learner populations, assessment practices, and national examination standards while integrating institutional assessment data into ongoing academic advising.

Limitations and future directions

This study has several limitations. First, the models were developed using retrospective data from a single institution and may not generalise directly to medical schools with different curricula, assessment practices, or student populations. Although the specific coefficients should be interpreted within the context of our institution, the analytic approach and TRACE advising framework are intended to be adaptable to institution-specific historical data.

Second, predictor variables were intentionally limited to NBME clinical subject examination performance. Although prior studies have incorporated earlier academic measures such as MCAT scores, preclinical performance, and the NBME Comprehensive Basic Science Examination, we focused on clerkship examinations because they assess clinical knowledge most closely aligned with Step 2 CK and are available during the period when academic advising may still meaningfully influence preparation.

Third, this study did not account for the sequence in which students completed their core clerkships. Students at our institution are randomly assigned to clerkship sequences, which reduces the likelihood that sequence is systematically related to student performance. Additionally, all students complete every clerkship and its subject examination before sitting for Step 2 CK. The models were intended to support advising across the natural variation in students' educational paths rather than to be sequence-specific. Prior research has likewise reported that clerkship sequence does not significantly influence performance on NBME clinical subject examinations or Step 2 CK [6]. Nonetheless, institutions with different clerkship structures may wish to examine sequence effects in their own data.

Finally, the study period included cohorts affected by the COVID-19 pandemic and the emergence of AI-based study tools, both of which may influence Step 2 CK performance. Although passing-standard era did not contribute to prediction in the conditional inference tree, suggesting that the relationships examined were stable across the study period, institutions replicating this approach should consider whether their historical data adequately represent typical student performance and should periodically re-evaluate model performance as testing conditions change.

Future research should validate these models across multiple institutions with differing curricula and clerkship structures. Prospective studies should evaluate implementation of these models within academic advising programmes and determine whether integrating predictive estimates into longitudinal advising improves educational outcomes, including Step 2 CK performance, specialty competitiveness, and match outcomes. Additional work should examine how students and advisors interpret predictive information, including its effects on self-regulated learning, advising practices, and learner well-being. Because predictive estimates may influence student confidence and anxiety, faculty development will be important to ensure that model outputs are communicated appropriately as probabilistic estimates within a holistic advising process rather than as deterministic predictions.

Conclusion

Existing prediction models primarily estimate Step 2 CK performance; this study extends that work by demonstrating how predictive modelling can be incorporated into structured academic advising. NBME clinical subject examination performance showed consistent associations with Step 2 CK outcomes across complementary analytic approaches, providing interpretable information that may support earlier identification of students who could benefit from targeted academic support. Rather than functioning as stand-alone prediction tools, these models are intended to complement holistic advising by contextualising student performance within historical institutional experience. The TRACE framework offers one approach for translating predictive analytics into longitudinal, data-informed advising that supports individualised educational planning throughout the clinical curriculum.

Supplementary Material

Supplementary Appendix.docx

Supplementary Appendix.docx

Disclosure statement

No potential conflict of interest was reported by the author(s).

Funding

None.

Data availability statement

Data is not available due to institutional restrictions. However, each medical school has access to their own similar data that can be used to replicate the study.

Ethical approval

The study was approved as exempt by the University of Arkansas for Medical Sciences Institutional Review Board: IRB number 239432.

Disclaimers

None.

Previous presentations

Spollen JJ. Robinson S, Boateng B, Clemmons, KR, Sward L, Latch R, Graham J. Predicting Student Performance on Licensing Exams Using Conditional Inference Trees. Poster presentation at the Group on Regional Medical Campus Conference (GRMC), Kansas City, MO. June 2-4, 2025.

Spollen JJ, Boateng B. Predicting Student Performance on Licensing Exams Using Conditional Inference Trees. Oral presentation at the Southern Group on Educational Affairs (SGEA), April 28-30, 2025, Miami, FL.

Supplementary material

Supplemental data for this article can be accessed at https://doi.org/10.1080/10872981.2026.2741461.