Abstract
Purpose
To describe educational outcomes for a national cohort of U.S. medical students who initially failed Step 1 of the United States Medical Licensing Examination
Method
The authors analyzed de-identified, individualized records for the 1993–2000 cohort of U.S. medical school matriculants who both initially failed Step l and were no longer in medical school as of March 2, 2009, using multivariable logistic regression to identify factors associated with attempting, and initially passing, Step 2 Clinical Knowledge (CK).
Results
Of 6,594 students who failed Step l, 5,985 (90.8%) took Step 2CK. Women and Asian/Pacific Islander students were more likely to attempt Step 2CK; more recent matriculants and students with lower failing Step 1 scores were less likely. Of the 5,985 students who attempted Step 2CK, 4,168 (69.6%) initially passed. Women, students with higher Medical College Admission Test scores, and more recent matriculants were more likely to pass Step 2CK; Asian/Pacific Islander students, underrepresented minority students, older students, and students with lower failing Step 1 scores were less likely. Ninety percent of students in the study sample (5,952/6,594) ultimately graduated from medical school, including 99.5% (4,148/4,168) of those who initially passed, 96.7% (1,757/1,817) of those who initially failed, and 7.7% (47/609) of those who never attempted Step 2CK.
Conclusions
The authors identified variables associated with educational outcomes among students who failed Step l. These findings can inform medical schools’ efforts to develop tailored interventions to maximize the likelihood that students will take Step 2CK and pass it on the first attempt.
Passing scores on all steps of the United States Medical Licensing Examination (USMLE) sequence are among the requirements for MD-degree graduates for permanent medical licensure in every state in the United States.1 Step l in the USMLE sequence is designed to assess the examinee’s understanding and application of the basic sciences relevant to medical practice2 and is typically taken by students in U.S. medical schools after they have completed the preclinical curriculum but before they have entered third-year clinical clerkships. Most examinees pass Step l on the first attempt. However, as reported annually from 1998–2010 by the National Board of Medical Examiners (NBME), the first-attempt failure rate among examinees at MD-degree granting medical schools has ranged from a low of 5% in 20073 to a high of 9% in 2001.4
Medical school policies regarding Step l requirements for student advancement and graduation have changed substantially since the introduction of the USMLE sequence. The percentage of medical schools that require a passing score on Step l for advancement and/or graduation has steadily increased from 63.5% (80/126) in 1992-1993,5 when the USMLE sequence was introduced, to 90.0% (117/130) in 2009-2010.6 Thus, increasingly greater numbers of students who initially failed Step 1 are subject to policies that stipulate a passing score must be achieved before they can advance along the medical education continuum.
An understanding of the variables associated with attempting, and with passing, Step 2 Clinical Knowledge (CK) on the first attempt among students who initially failed Step l can inform medical schools’ efforts both to assess the risk for experiencing subsequent difficulties for the students who initially fail Step l and to provide support for those students who are at risk for experiencing continued difficulties with the USMLE sequence. Thus, we examined these educational outcomes (attempting Step 2CK and passing it on the first attempt) for students who were enrolled in U.S. medical schools accredited by the Liaison Committee on Medical Education (LCME) and who initially failed Step l of the USMLE sequence.
Because Step 2CK is typically taken only after students have advanced through the third-year clinical clerkships, we hypothesized that, among students who initially failed Step l, a more recent matriculation year would be associated with a lower likelihood of attempting Step 2CK. Because Step 2CK first-attempt 3-digit scores correlate with Step 1 first-attempt 3-digit scores,7-9 we also hypothesized that, among those students who initially failed Step l and attempted Step 2CK, lower Step 1 failing scores would be associated with a lower likelihood of passing Step 2CK on the first attempt. We report here the results of a retrospective, national cohort study that tested these two hypotheses.
Method
Our study dataset included individualized, de-identified records for all matriculants in academic years 1993-1994 through 2000-2001 to LCME accredited U.S. medical schools. As medical school completion and attrition rates continue to accrue more than 8 years after matriculation,10 we selected 2000 as the last matriculation year of our study to allow sufficient time for students in our study sample to arrive at their final status (graduation or dismissal/withdrawal) as of March 2, 2009, which was the last date of follow-up for all matriculants in the study sample. This lengthy follow-up period post matriculation may be particularly important for students experiencing academic difficulties (e.g., our cohort of examinees who failed Step l on the first attempt), because dismissal/withdrawal due to academic reasons continues for many years after matriculation.10 Our dataset included information from the Student Record System (SRS) of the Association of American Medical Colleges (AAMC), responses to items on the AAMC Matriculating Student Questionnaire (MSQ), Medical College Admission Test (MCAT) scores, and, for Step 1 and Step 2CK, both first-attempt pass/fail results and 3-digit scores.
SRS variables included matriculation and last-status dates; sex (female vs. male) and self-identified race/ethnicity, as selected by matriculants from the list of options on the American Medical College Application Service Questionnaire (Asian/Pacific Islander; underrepresented minorities in medicine [URM], including black, Hispanic, and American Indian/Alaska Native; or white). We included sex and race/ethnicity in our study because a previous study reported that these variables were associated with first-attempt Step 1 and Step 2CK passing rates.7
MSQ variables included:
age at matriculation (>24 years vs. ≤ 24 years),
premedical debt (no debt, $100 to $4,999; $5,000 to $9,999; $10,000 to $14,999; $15,000 to $19,999; $20,000 to $24,999; $25,000 to $29,999; $30,000 to $49,999; $50,000 to $74,999; $75,000 to $99,999; and ≥ $100,000), and
medical-school ownership (private vs. public).
The AAMC provided students’ MCAT results for their most recent attempt. We computed a composite MCAT score for each student as the sum of the verbal reasoning, physical science, and biological science scores. The AAMC also provided students’ first-attempt Step l and Step 2CK results, released with permission from the NBME. We created a 3-category variable for failing Step 1 scores (lowest tertile [97-165], middle tertile [166-173], or highest tertile [174-183]). We also created a 3-category variable for Step 2CK scores (pass, fail, or score N/A [not available]; score N/A applied to those students who never attempted Step 2CK). We linked records using a unique, AAMC-generated identification number and merged them into a single file for analysis. The Institutional Review Board at Washington University School of Medicine approved this study.
We used chi-square tests to measure associations among categorical variables and analysis of variance to describe differences in continuous variables between groups. We report here adjusted odds ratios (ORs) and 95% confidence intervals (CIs) from two multivariate logistic regression models. The first model identified independent predictors of attempting Step 2CK. In other words, we compared all students with Step 2CK scores, both passing and failing, with students who did not have Step 2CK scores (score N/A). The second model, which included only students who took Step 2CK, identified independent predictors of passing Step 2CK on the first attempt compared with failing on the first attempt. We used SPSS version 17.0.3 (SPSS, Inc., Chicago, IL, 2009) to perform all tests. We considered two-sided P values < .05 to be significant.
Results
Of the 129,867 students who matriculated to LCME-accredited U.S. medical schools in academic years 1993-1994 through 2000-2001, 126,611 (97.5%) students took Step l, and 7,889 (6.2%) of these students initially failed. Of these 7,889 students, 1 subsequently died during medical school and another had his or her degree revoked, 61 were still enrolled in medical school as of March 2, 2009, and 1,232 did not have complete data for all the variables included in our regression models. Thus, our final sample of 6,594 students included 83.6% of the 7,889 students in our dataset who initially failed Step l. The mean (standard deviation [SD]) Step l score of 167.7 (10.4) for the 6,594 students included in our final study sample was higher than the mean (SD) Step l score of 166.2 (11.8) for the 1,295 students who failed Step l but were not included in our final study sample (P <.001) (data not shown).
Included in Table 1 are descriptive statistics for all 6,594 students in our study sample, including the 5,985 (90.8%) students who had attempted Step 2CK and the 609 (9.2%) students who had not attempted Step 2CK. We found significant differences between these two groups in sex, race/ethnicity, age at matriculation, composite MCAT score, and Step l score. Mean (SD) Step l scores among all 6,594 students in our sample increased over time from 164.2 (10.5) for matriculants in 1993-1994 to 170.3 (9.7) for matriculants in 2000-2001 (P <.001) (data not shown).
Table 1.
Characteristics of U.S. Medical School Matriculants, 1993-1994 through 2000-2001, Who Initially Failed Step 1 of the United States Medical Licensing Examination*
| Characteristic | Failed Step 1 No. (% of 6,594) |
Attempted Step 2CK No. (% of 5,985) |
Didn’t attempt Step 2CK No. (% of 609) |
P value |
|---|---|---|---|---|
| Sex | .041 | |||
| Male | 2,936 (44.5) | 2,641 (44.1) | 295 (48.4) | |
| Female | 3,658 (55.5) | 3,344 (55.9) | 314 (51.6) | |
| Race/ethnicity | .024 | |||
| White | 2,444 (37.1) | 2,226 (37.2) | 218 (35.8) | |
| Asian/Pacific Islander | 952 (14.4) | 883 (14.8) | 69 (11.3) | |
| URM | 3,198 (48.5) | 2,876 (48.1) | 322 (52.9) | |
| Age at matriculation | .014 | |||
| ≤24 years | 4,439 (67.3) | 4,056 (67.8) | 383 (62.9) | |
| >24 years | 2,155 (32.7) | 1,929 (32.2) | 226 (37.1) | |
| Premedical debt ($) | .570 | |||
| No debt | 3,514 (53.3) | 3,203 (53.5) | 311 (51.1) | |
| 100-4,999 | 440 (6.7) | 395 (6.6) | 45 (7.4) | |
| 5,000-9,999 | 612 (9.3) | 545 (9.1) | 67 (11.0) | |
| 10,000-14,999 | 635 (9.6) | 578 (9.7) | 57 (9.4) | |
| 15,000-19,999 | 492 (7.5) | 453 (7.6) | 39 (6.4) | |
| 20,000-24,999 | 341 (5.2) | 309 (5.2) | 32 (5.3) | |
| 25,000-29,999 | 157 (2.4) | 143 (2.4) | 14 (2.3) | |
| 30,000-49,999 | 268 (4.1) | 243 (4.1) | 25 (4.1) | |
| 50,000-74,999 | 101 (1.5) | 86 (1.4) | 15 (2.5) | |
| 75,000-99,999 | 20 (0.3) | 18 (0.3) | 2 (0.3) | |
| ≥100,000 | 14 (0.2) | 12 (0.2) | 2 (0.3) | |
| Matriculation year | .247 | |||
| 1993-1994 | 797 (12.1) | 736 (12.3) | 61 (10.0) | |
| 1994-1995 | 822 (12.5) | 757 (12.6) | 65 (10.7) | |
| 1995-1996 | 654 (9.9) | 590 (9.9) | 64 (10.5) | |
| 1996-1997 | 653 (9.9) | 579 (9.7) | 74 (12.2) | |
| 1997-1998 | 829 (12.6) | 746 (12.5) | 83 (13.6) | |
| 1998-1999 | 888 (13.5) | 800 (13.4) | 88 (14.4) | |
| 1999-2000 | 1,123 (17.0) | 1,023 (17.1) | 100 (16.4) | |
| 2000-2001 | 828 (12.6) | 754 (12.6) | 74 (12.2) | |
|
Medical school
ownership |
.629 | |||
| Private | 2,289 (34.7) | 2,083 (34.8) | 206 (33.8) | |
| Public | 4,305 (65.3) | 3,902 (65.2) | 403 (66.2) | |
|
Step 1 tertile
(score range) |
<.001 | |||
| Highest (174-183) | 2,240 (34.0) | 2,150 (35.9) | 90 (14.8) | |
| Middle (166-173) | 2,249 (34.1) | 2,117 (35.4) | 132 (21.7) | |
| Lowest (97-165) | 2,105 (31.9) | 1,718 (28.7) | 387 (63.5) | |
|
Status as of March 2,
2009 |
<.001 | |||
| Graduated | 5,952 (90.3) | 5,905 (98.7) | 47 (7.7) | |
| Withdrew/dismissed | 642 (9.7) | 80 (1.3) | 562 (92.3) | |
| Characteristic |
Failed Step 1
Mean (SD) |
Attempted
Step 2CK Mean (SD) |
Didn’t attempt
Step 2CK Mean (SD) |
P value |
| Composite MCAT score | 24.0 (4.7) | 24.0 (4.7) | 23.6 (5.1) | .032 |
| 3-digit Step 1 score | 167.7 (10.4) | 168.6 (9.3) | 158.5 (15.0) | <.001 |
CK indicates Clinical Knowledge; MCAT, Medical College Admission Test; SD, standard deviation; URM, underrepresented minorities (i.e., minorities that are underrepresented in medicine relative to their proportion in the general population, including black, Hispanic, and American Indian/Alaska Native)
Included in Table 2 are descriptive statistics, grouped by Step 2CK results, for the 5,985 students who attempted Step 2CK, including 4,168 (69.6%) students who passed Step 2CK on the first attempt and 1,817 (30.4%) students who failed. We found significant differences between these two groups in sex, race/ethnicity, age at matriculation, matriculation year, medical school ownership, MCAT score, and Step l score. Mean (SD) Step l scores for the 5,985 students who took Step 2CK increased over time from 165.0 (9.4) for matriculants in 1993-1994 to 171.3 (8.4) for matriculants in 2000-2001 (P <.001) (data not shown). In addition, mean (SD) Step 2CK scores for these 5,985 students increased over time from 173.6 (18.8) for matriculants in 1993-1994 to 191.7 (18.8) for matriculants in 2000-2001 (P < .001) (data not shown).
Table 2.
Characteristics of U.S. Medical School Matriculants, 1993-1994 through 2000-2001, Who Initially Failed Step 1 of the United States Medical Licensing Examination and Attempted Step 2CK*
| Characteristic | Attempted Step 2CK No. (% of 5,985) |
Passed Step 2CK† No. (% of 4,168) |
Failed Step 2CK† No. (% of 1,817) |
P Value |
|---|---|---|---|---|
| Sex | <.001 | |||
| Male | 2,641 (44.1) | 1,663 (39.9) | 978 (53.8) | |
| Female | 3,344 (55.9) | 2,505 (60.1) | 839 (46.2) | |
| Race/ethnicity | <.001 | |||
| White | 2,226 (37.2) | 1,753 (42.1) | 473 (26.0) | |
| Asian/Pacific Islander | 883 (14.8) | 628 (15.1) | 255 (14.0) | |
| URM | 2,876 (48.1) | 1,787 (42.9) | 1,089 (59.9) | |
| Age at matriculation | <.001 | |||
| ≤24 years | 4,056 (67.8) | 2,946 (70.7) | 1,110 (61.1) | |
| >24 years | 1,929 (32.2) | 1,222 (29.3) | 707 (38.9) | |
| Premedical debt($) | .074 | |||
| No debt | 3,203 (53.5) | 2,273 (54.5) | 930 (51.2) | |
| 100-4,999 | 395 (6.6) | 250 (6.0) | 145 (8.0) | |
| 5,000-9,999 | 545 (9.1) | 371 (8.9) | 174 (9.6) | |
| 10,000-14,999 | 578 (9.7) | 396 (9.5) | 182 (10.0) | |
| 15,000-19,999 | 453 (7.6) | 321 (7.7) | 132 (7.3) | |
| 20,000-24,999 | 309 (5.2) | 220 (5.3) | 89 (4.9) | |
| 25,000-29,999 | 143 (2.4) | 96 (2.3) | 47 (2.6) | |
| 30,000-49,999 | 243 (4.1) | 170 (4.1) | 73 (4.0) | |
| 50,000-74,999 | 86 (1.4) | 54 (1.3) | 32 (1.8) | |
| 75,000-99,999 | 18 (0.3) | 11 (0.3) | 7 (0.4) | |
| ≥100,000 | 12 (0.2) | 6 (0.1) | 6 (0.3) | |
| Matriculation year | <.001 | |||
| 1993-1994 | 736 (12.3) | 454 (10.9) | 282 (15.5) | |
| 1994-1995 | 757 (12.6) | 459 (11.0) | 298 (16.4) | |
| 1995-1996 | 590 (9.9) | 337 (8.1) | 253 (13.9) | |
| 1996-1997 | 579 (9.7) | 391 (9.4) | 188 (10.3) | |
| 1997-1998 | 746 (12.5) | 555 (13.3) | 191 (10.5) | |
| 1998-1999 | 800 (13.4) | 630 (15.1) | 170 (9.4) | |
| 1999-2000 | 1,023 (17.1) | 796 (19.1) | 227 (12.5) | |
| 2000-2001 | 754 (12.6) | 546 (13.1) | 208 (11.4) | |
|
Medical school
ownership |
<.001 | |||
| Private | 2,083 (34.8) | 1,377 (33.0) | 706 (38.9) | |
| Public | 3,902 (65.2) | 2,791 (67.0) | 1,111 (61.1) | |
|
Step 1 tertile
(score range) |
<.001 | |||
| Highest (174-183) | 2,150 (35.9) | 1,727 (41.4) | 423 (23.3) | |
| Middle (166-173) | 2,117 (35.4) | 1,494 (35.8) | 623 (34.3) | |
| Lowest (97-165) | 1,718 (28.7) | 947 (22.7) | 771 (42.4) | |
|
Status as of March 2,
2009 |
<.001 | |||
| Graduated | 5,905 (98.7) | 4,148 (99.5) | 1,757 (96.7) | |
| Withdrew/dismissed | 80 (1.3) | 20 (0.5) | 60 (3.3) | |
| Characteristic |
Attempted
Step 2 CK Mean (SD) |
Passed Step 2CK
Mean (SD) |
Failed Step
2CK Mean (SD) |
P value |
| Composite MCAT score | 24.0 (4.7) | 24.6 (4.5) | 22.8 (4.8) | <.001 |
| 3-digit Step 1 score | 168.6 (9.3) | 170.2 (8.0) | 165.1 (10.9) | <.001 |
CK indicates Clinical Knowledge; MCAT, Medical College Admission Test; SD, standard deviation; URM, underrepresented minorities (i.e., minorities that are underrepresented in medicine relative to their proportion in the general population, including black, Hispanic, and American Indian/Alaska Native)
Students were grouped by the results of their first attempt to pass Step 2CK.
As also shown in Tables 1 and 2, most students (5,952 of 6,594 [90.3%]) in our study sample ultimately graduated from medical school. These graduates included 4,148 (99.5%) of the 4,168 students who initially passed Step 2CK and 1,757 (96.7%) of the 1,817 students who initially failed Step 2CK but only 47 (7.7%) of the 609 students who never attempted Step 2CK. Table 3 shows the results of two multivariable logistic regression models that identified variables independently associated with each of attempting (vs. not attempting) Step 2CK and of passing (vs. failing) Step 2CK on the initial attempt. Among all 6,594 students in our study sample, women and Asian/Pacific Islander students were significantly more likely to have attempted Step 2CK; students who had matriculated in more recent years and students who had Step l failing scores in the lowest and middle tertiles were significantly less likely to have attempted Step 2CK. Among the 5,985 students in our study sample who had attempted Step 2CK, women, students with higher MCAT scores, and students who had matriculated in more recent years were significantly more likely to have initially passed Step 2CK; students who were of Asian/Pacific Islander or URM race/ethnicity, were older, had Step l failing scores in the lowest and middle tertiles, and had attended private medical schools were significantly less likely to have initially passed Step 2CK. The Hosmer and Lemeshow test indicated that each of the two models fit the data (each P > .05).
Table 3.
Results of Two Multivariable Logistic Regression Models to Identify Independent Predictors of Attempting Step 2CK and Passing Step 2CK Among U.S. Medical School Matriculants, 1993-1994 through 2000-2001, Who Initially Failed Step 1 of the United States Medical Licensing Examination*
| Characteristic | Attempted vs. not attempted Step 2CK Adj. OR (95% CI) |
Passed vs. failed Step 2CK Adj. OR (95% CI) |
|---|---|---|
| Sex | ||
| Male (reference) | 1.00) | 1.00 |
| Female | 1.20 (1.00-1.43)¶ | 2.18 (1.93-2.46)** |
| Race/ethnicity | ||
| White (reference) | 1.00 | 1.00 |
| Asian/Pacific Islander | 1.34 (1.00-1.80)¶ | 0.61 (0.51-0.74)** |
| URM | 1.01 (0.80-1.27) | 0.59 (0.51-0.70)** |
| Age at matriculation | ||
| ≤24 years (reference) | 1.00 | 1.00 |
| >24 years | 0.86 (0.72-1.03) | 0.63 (0.56-0.72)** |
| Premedical debt† | 1.00 (0.96-1.03) | 0.99 (0.96 – 1.02) |
| Matriculation year‡ | 0.93 (0.90 – 0.97)** | 1.08 (1.05-1.10)** |
| Composite MCAT score§ | 0.99 (0.97-1.01) | 1.05 (1.03-1.07)** |
| Step 1 tertile (score range) | ||
| Highest (174-183) (reference) | 1.00 | 1.00 |
| Middle (166-173) | 0.61 (0.46-0.81)** | 0.69 (0.59-0.80)** |
| Lowest (97-165) | 0.16 (0.13-0.21)** | 0.38 (0.33-0.44)** |
| Medical school ownership | ||
| Public (reference) | 1.00 | 1.00 |
| Private | 1.08 (0.90-1.30) | 0.83 (0.73-0.94)** |
Adj. OR indicates adjusted odds ratio; CI, confidence interval; CK, Clinical Knowledge; MCAT, Medical College Admission Test; URM, underrepresented minorities (i.e., minorities that are underrepresented in medicine relative to their proportion in the general population, including black, Hispanic, and American Indian/Alaska Native)
Adj. OR > 1.00 indicates a higher likelihood and Adj. OR < 1.00 indicates a lower likelihood with increasing pre-medical debt
Adj. OR > 1.00 indicates a higher likelihood and Adj. OR < 1.00 indicates a lower likelihood with a more recent matriculation year
Adj. OR > 1.00 indicates a higher likelihood and Adj. OR < 1.00 indicates a lower likelihood with increasing MCAT score
P < .05
P ≤ .002
Discussion
Our findings regarding educational outcomes for students with first-attempt Step 1 failing scores should be considered in the context of previous studies of examinees’ Step 1 and Step 2CK performance, evolving medical school policies for advancement/graduation, and the implications of these failing scores for students who ultimately graduate from medical school and seek graduate medical education (GME) positions.
The finding that older students (>24 years) who initially failed Step 1 were less likely to initially pass Step 2CK in our national sample is aligned with a previous single-institutional report, published in 2009, of a significant, inverse relationship between age and Step 2 performance among all examinees, regardless of Step l score.11 The finding that students with higher MCAT scores were more likely to pass Step 2CK on the first attempt extends previous observations of significant positive correlations between MCAT scores and 3-digit Step 2CK scores.8,9 In addition, our finding that students with failing Step l scores in the lowest and middle tertiles were less likely to pass Step 2CK on the first attempt both extends the results of other studies that have reported positive correlations between 3-digit Step l and Step 2CK scores7,9 and suggests that the lower that students’ first-attempt failing Step 1 scores are, rather than merely having a failing score, the more academic support that those students may need to advance and pass Step 2CK on the first attempt. The issue of support for students who initially fail Step l is likely to be of growing interest for faculty and administrators at U.S. medical schools because of increases in the first-attempt Step 1 failure rate in the past 4 years, from 5% in 20073 to 8% in 2010,12 as well as increases in the total number of Step l examinees in the past 4 years.3,12 The number of Step 1 examinees may be expected to grow further in coming years, reflecting rising enrollment at U.S. LCME-accredited medical schools.13 Comprehensive medical school remediation and other academic support programs for the increasing number of students who initially fail Step l, especially those students with very low failing scores who are at a particularly high risk of failing Step 2CK, might minimize the likelihood of repeated licensure examination failures.14
Although most U.S. medical school matriculants in the academic years from 1993 through 2000 were men and of white race/ethnicity,15 most students in our study sample of students who initially failed Step 1 were women and of non-white race/ethnicity. On an annual basis, of all U.S. medical school matriculants, women constituted from 42.0% (in 1993-1994) to 45.8% (in 2000-2001), and non-white students from 31.8% (in 1993-1994) to 33.9% (in 2000-2001);15 whereas, women constituted 55.5% of students in our sample of matriculants who initially failed Step l, and non-white students constituted 62.9%, of students in our sample of matriculants who initially failed Step l. The relative over-representation of women and non-white students in our sample of students is consistent with previously-reported differences in first-attempt Step l passing rates on the basis of sex and race/ethnicity.7 Among a national cohort of nearly 15,000 students enrolled in U.S. LCME-accredited medical schools, 89.1% passed Step l on the first attempt, including 91.1% of men and 85.9% of women. First-attempt Step l passing rates also differed by race/ethnicity; 93.4% of white, 86.8% of Asian,77.5% of Hispanic, and 58.2% of African American students in this national cohort passed Step l on the first attempt.7 Nevertheless, among students in our sample who initially failed Step l, women were more likely than men to take and to pass Step 2CK. In the national cohort study of U.S. medical students by Case and colleagues, 92.9% of women and 92.3% of men passed Step 2CK on the first attempt.7 Other investigators have reported that women performed better than men on clinical skills’ assessments16-18 and also obtained higher numeric scores on Step 2CK.19 Thus, among students who initially fail Step l and advance to the clinical years, women’s performance in medical school might generally be expected to improve during the clinical years.
The finding that each of URM and Asian/Pacific Islander race/ethnicity was independently associated with a lower likelihood of passing Step 2CK similarly extends a previous observation by Case and colleagues that the percentages of Asian (87.6%), Hispanic (86.9%), and African American (71.5%) students who passed Step 2CK on the first attempt were lower than the percentage of white students (96.3%).7 Our observation that URM and Asian/Pacific Islander students were less likely to pass Step 2CK in a model that controlled for Step l scores, premedical debt, and MCAT scores should be of concern to U.S. medical schools in light of ongoing efforts to promote greater racial and ethnic diversity in the emerging physician workforce.20 Further research is needed to identify other variables, amenable to intervention, that might disproportionately and negatively impact the subsequent Step 2CK success of non-white students who experience difficulties with Step l.14
The finding that a more recent matriculation year and Step l scores in the lowest and middle tertiles were associated with a lower likelihood of taking Step 2CK might be explained by changes in school policies regarding advancement/graduation and increases in Step l passing score cut-offs. Our results suggest that, as the number of schools mandating a passing score on Step l for advancement/graduation increases and as the minimum Step l passing score increases (from 176 in 199421 to 188 in 201022), more students who initially fail Step l may not advance sufficiently to take Step 2CK but rather will be dismissed or will withdraw from medical school. Our observation that there was a small percentage of students in our study sample who never took Step 2CK but graduated is consistent both with the wide range in medical school policies regarding passage of Step l and Step 2CK to advance and with the continual evolution of these policies since the introduction of the USMLE sequence. In the academic year 2009-2010, 117 of 130 U.S. LCME-accredited medical schools required a passing Step l score, 11 schools required students only to take (but not pass) Step l, and two schools had no Step l requirements for advancement/graduation; similarly, 98 of 130 schools required a passing Step 2CK score, 30 schools required students only to take (but not pass) Step 2CK, and two schools had no Step 2CK requirements for advancement/graduation.6 It was beyond the scope of our study to examine whether there was a relationship between school policy regarding passing Step l for advancement/promotion and the likelihood of a student attempting, or passing, Step 2CK. However, such studies in the future could further inform the academic medicine community’s understanding of the variables associated with educational outcomes among students at risk for experiencing difficulties with the USMLE sequence.
Our observation that a more recent matriculation year was associated with a greater likelihood of passing Step 2CK might seem counterintuitive, particularly since the minimum Step 2CK passing score cut-off, in 1994, of 16721 has also been raised several times, most recently in 2010, to the current cut-off of 189.22 However, our findings are consistent with the hypothesis that the poorest-performing students of those who failed Step 1 become less likely to advance sufficiently to attempt Step 2CK, so the subset of Step l examinees who failed yet still took Step 2CK had obtained increasingly higher first-attempt Step l scores.
Students at private medical schools who initially failed Step 1 were neither more nor less likely to attempt Step 2CK than their counterparts at public medical schools, but they were less likely to pass Step 2CK on their first attempt. We speculate that this observation might reflect school specific differences in the extent and nature of interventions to support students who experienced difficulties with Step 1.5 Medical schools have not uniformly provided remediation for students who failed Step l. According to the LCME Annual Medical School Questionnaire responses from the 124-125 medical schools in the U.S. that completed the questionnaire in academic years 1993-1994 through 2001-2002, only 60-68 schools annually reported that they provided remediation for students who failed Step l.23 Robust support programs for at-risk students might be particularly well developed at public schools with a strong commitment to recruiting and graduating state residents.8 Future research to identify model Step l remediation programs that best promote student success on subsequent Step l, and initial Step 2CK, attempts could inform the efforts of all medical schools to provide effective support for their students who experience particular difficulties with the USMLE sequence.
Finally, as most students in our study sample ultimately graduated, including those who failed both Step 1 and Step 2CK on the first attempt, the implications of first-attempt Step 1 failing scores for graduates should be considered. First-attempt failing Step l scores may adversely affect the efforts of graduating students to secure GME positions in programs accredited by the Accreditation Council for Graduate Medical Education (ACGME). Many program directors use Step l results to select applicants to interview and to place on their rank lists for GME positions; most of these program directors seldom or never consider applicants who failed Step l or Step 2CK on the first attempt.24 With increasing competition for positions in ACGME-accredited residency training programs, un-matched students who have experienced difficulties with the USMLE sequence during medical school also may face growing challenges in securing unfilled GME positions.25 Thus, students who fail Step l, even if they subsequently pass Step 2CK on the first attempt and graduate from medical school, remain at risk for experiencing difficulties in gaining entry into GME programs, particularly in their preferred specialties and programs. The risk for experiencing such difficulties is likely even greater for those students who also failed Step 2CK on the first attempt.
A strength of our study was that we included a national cohort of students who never took Step 2CK and/or did not graduate. Including these students provided a more complete picture of the educational outcomes for all students who failed Step 1 than have previous studies, which included only those students who also took Step 2CK7 or graduated from medical school.26A limitation of our study was that we did not include information about school-specific requirements regarding passing scores on Step l for advancement/graduation. When we conducted this study, the NBME had not limited the number of times that an examinee could attempt Step l. However, some schools that require a passing score on Step l for advancement/graduation may have had policies in place regarding the number of attempts allowed to pass Step l. For example, failure to pass Step l in three attempts is grounds for dismissal at some schools.27,28 Effective January 1, 2012, however, the USMLE Program placed a six-attempt limit to pass each Step or Step component of the USMLE sequence for all examinees.29 We also lacked information about school-specific differences in academic support programs that might have contributed to observed differences in passing Step 2CK on the first attempt.5,14,23 Finally, as the mean Step l score of matriculants excluded from our final study sample was significantly lower than the mean Step l score of matriculants included, outcomes among all U.S. medical school matriculants who fail Step l may be somewhat less favorable than the outcomes that we observed among matriculants in our study sample. Despite these limitations, our results may inform medical schools’ efforts both to better assess the risk of experiencing subsequent difficulties for the growing numbers of students who initially fail Step l and to develop interventions tailored to meet the educational needs of these students to maximize the likelihood that they will take Step 2CK and pass it on the first attempt.
Acknowledgements
The authors thank Dr. Paul Jolly, Dr. Gwen Garrison, and Dr. David Matthew at the Association of American Medical Colleges, Washington, D.C., for their support of this research through the provision of data and assistance with coding. The authors also thank Dr. Robert M. Galbraith at the National Board of Medical Examiners, Philadelphia, Pennsylvania, for assistance with United States Medical Licensing Examination Step l and Step 2CK data.
Funding/Support: The National Institute of General Medical Sciences provided funding for this study (R01 GM085350-03). The National Institute of General Medical Sciences was not involved in the design and conduct of the study; nor the collection, management, analysis, or interpretation of the data; nor the preparation, review, or approval of the final report.
Footnotes
Ethical Approval: The Institutional Review Board at Washington University School of Medicine approved this study as non-human-subjects research.
Publisher's Disclaimer: Disclaimer: The conclusions that the authors express in this report are not necessarily those of the Association of American Medical Colleges, the National Board of Medical Examiners, the National Institutes of Health, or the respective staff members of these organizations.
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Contributor Information
Dorothy A. Andriole, Washington University School of Medicine, Saint Louis, Missouri..
Donna B. Jeffe, Health Behavior, Communication and Outreach Core, Alvin J. Siteman Cancer Center, Barnes-Jewish Hospital and Washington University School of Medicine, Saint Louis, Missouri..
References
- 1.Federation of State Medical Boards [Accessed December 8, 2011];State-specific requirements for initial medical licensure. Updated July 2010. Available at: http://www.fsmb.org/usmle_eliinitial.html.
- 2.United States Medical Licensing Examination [Accessed December 8, 2011];Step l. Available at: http://www.usmle.org/step-1/
- 3.United States Medical Licensing Examination [Accessed December 8, 2011];2007 performance data. Available at: http://www.usmle.org/performance-data/default.aspx#2007_step-1.
- 4.United States Medical Licensing Examination [Accessed December 8, 2011];2001 performance data. Available at: http://www.usmle.org/performance-data/default.aspx#2001_step-1.
- 5.Jonas HS, Etzel SI, Barzansky B. Educational programs in US medical schools. JAMA. 1993;270:1061–1068. [PubMed] [Google Scholar]
- 6.Barzansky B, Etzel SI. Educational programs in US medical schools 2009–2010. JAMA. 2010;304:1247–1254. doi: 10.1001/jama.2010.1154. [DOI] [PubMed] [Google Scholar]
- 7.Case SM, Swanson DB, Ripkey DR, Bowles LT, Melnick DE. Performance of the class of 1994 in the new era of USMLE. Acad Med. 1996;71:S91–S93. doi: 10.1097/00001888-199610000-00055. [DOI] [PubMed] [Google Scholar]
- 8.Julian ER. Validity of the medical college admission test for predicting medical school performance. Acad Med. 2005;80:910–917. doi: 10.1097/00001888-200510000-00010. [DOI] [PubMed] [Google Scholar]
- 9.Donnon T, Paolucci EO, Violato C. The predictive validity of the MCAT for medical school performance and medical board licensing examinations: A meta-analysis of the published research. Acad Med. 2007;82:100–106. doi: 10.1097/01.ACM.0000249878.25186.b7. [DOI] [PubMed] [Google Scholar]
- 10.Garrison G, Mikesell C, Matthew D. Analysis in brief: Medical school graduation and attrition rates. Association of American Medical Colleges; [Accessed December 8, 2011]. 2007. Available at: https://www.aamc.org/download/102346/data/aibvol7no2.pdf. [Google Scholar]
- 11.Kleshinski J, Khuder SA, Shapiro JI, Gold JP. Impact of preadmission variables on USMLE step 1 and step 2 performance. Adv Health Sci Educ Theory Pract. 2009;14:69–78. doi: 10.1007/s10459-007-9087-x. [DOI] [PubMed] [Google Scholar]
- 12.United States Medical Licensing Examination [Accessed December 22, 2011];2010 performance data. Available at: http://www.usmle.org/performance-data/default.aspx#2010_step-1.
- 13.Schulman S, Salsberg E. Analysis in brief: Projections of future medical school enrollment. Association of American Medical Colleges; [Accessed December 8, 2011]. 2009. Available at: https://www.aamc.org/download/102372/data/aibvol9no3.pdf. [Google Scholar]
- 14.Laatsch L. Evaluation and treatment of students with difficulties passing the step examinations. Acad Med. 2009;84:677–683. doi: 10.1097/ACM.0b013e31819faae1. [DOI] [PubMed] [Google Scholar]
- 15.Association of American Medical Colleges . AAMC data book: Medical schools and teaching hospitals by the numbers. Association of American Medical Colleges; Washington, DC: 2006. [Google Scholar]
- 16.Haist SA, Witzke DB, Quinlivan S, Murphy-Spencer A, Wilson JF. Clinical skills as demonstrated by a comprehensive clinical performance examination: Who performs better--men or women? Adv Health Sci Educ Theory Pract. 2003;8:189–199. doi: 10.1023/a:1026072102739. [DOI] [PubMed] [Google Scholar]
- 17.Van Zanten M. Correlates of performance of the ECFMG clinical skills assessment: Influences of candidate characteristics on performance. Acad Med. 2003;78:S72–S74. doi: 10.1097/00001888-200310001-00023. [DOI] [PubMed] [Google Scholar]
- 18.Swygert KA, Muller ES, Colette LS, Swanson DB. The relationship between USMLE Step 2 CS patient note ratings and time spent on the note: Do examinees who spend more time write better notes? Acad Med. 2010;85:S89–S92. doi: 10.1097/ACM.0b013e3181ed39d0. [DOI] [PubMed] [Google Scholar]
- 19.Cuddy MM, Swanson DB, Dillon GF, Holtman MC, Clauser BE. A multilevel analysis of the relationships between selected examinee characteristics and United States Medical Licensing Examination Step 2: Revisiting old findings and asking new questions. Acad Med. 2006;81:S103–S107. doi: 10.1097/00001888-200610001-00026. [DOI] [PubMed] [Google Scholar]
- 20.Cohen JJ, Steinecke A. Building a diverse physician workforce. JAMA. 2006;296:1135–1137. doi: 10.1001/jama.296.9.1135. [DOI] [PubMed] [Google Scholar]
- 21.United States Medical Licensing Examination [Accessed December 22, 2011];1994 performance data. Available at: http://www.usmle.org/performance-data/default.aspx#1994_step-1.
- 22.United States Medical Licensing Examination [Accessed December 22, 2011];Scores and transcripts. Available at: http://www.usmle.org/transcripts/
- 23.Etzel S, Research Associate. American Medical Association Personal written communication dated September 22, 2011.
- 24.Raether J. Results of the 2010 NRMP program director survey. National Resident Matching Program; Washington, DC: [Accessed December 8, 2011]. 2010. Available at: http://www.nrmp.org/data/programresultsbyspecialty2010v3.pdf. [Google Scholar]
- 25.Sondheimer HM. Graduating US medical students who do not obtain a PGY-1 training position. JAMA. 2010;304:1168–1169. doi: 10.1001/jama.2010.1316. [DOI] [PubMed] [Google Scholar]
- 26.Biskobing DM, Lawson SR, Messmer JM, Hoban JD. Study of selected outcomes of medical students who fail USMLE Step 1. Med Educ Online. 2006;11:11. doi: 10.3402/meo.v11i.4589. [DOI] [PubMed] [Google Scholar]
- 27.University of Alabama School of Medicine [Accessed December 8, 2011];Undergraduate medical education. USMLE steps 1 and 2 and the clinical skills examination. Available at: http://main.uab.edu/uasom/2/show.asp?durki=21318.
- 28.University of Missouri-Kansas City School of Medicine [Accessed December 8, 2011];Council on evaluation policy manual & guidelines. 2011 June 1; Available at: http://www.med.umkc.edu/councils/coe_documents/COE_Policy_Manual.pdf.
- 29.United States Medical Licensing Examination Bulletin [Accessed December 22, 2011];Eligibility. Available at: http://www.usmle.org/bulletin/eligibility/#retakes.