Factors associated with performance in an internal medicine clerkship

Abstract

The purpose of this retrospective study was to examine the relationship between demographic and educational variables and student performance on an internal medicine (IM) clerkship in order to target areas for educational intervention and potential early remediation. This study examined data associated with third-year medical student performance (N = 505) during the IM clerkship at Baylor Scott & White, Temple/Texas A&M Health Science Center College of Medicine from 2005 to 2011. Multiple regression analysis (N = 341) showed that a model containing the following variables was significantly associated with scores on the National Board of Medical Examiners (NBME) subject exam, accounting for 46.5% of the variance: Objective Structured Clinical Exam (OSCE), Medical College Admissions Test (MCAT), US Medical Licensing Exam (USMLE) Step 1, second-year grade point average (GPA), and clinical evaluation. A model containing USMLE Step 1, clinical evaluation, and NBME was significantly associated with OSCE score, accounting for 30% of the variance. Additionally, a model containing age, MCAT score, undergraduate GPA, NBME subject exam score, and OSCE was significantly associated with clinical evaluation score, accounting for 22% of score variance. Age accounted for the most unique variance in clinical evaluation score. Gender and IM interest group were not significantly associated with any outcome variable. In conclusion, in contrast to previous studies in the field, we did not find a significant association between undergraduate GPA and NBME score. Our findings supply further evidence that the OSCE, typically believed to be a clinical performance exam, actually assesses a broader set of domains. Interest group membership did not confer any academic benefit to medical students in IM clerkships in our study.

The association between medical student academic variables and academic performance on clerkships has been examined in previous studies (1–4). Studies have also examined nonacademic variables, including demographic and internal variables (e.g., intrinsic motivation), as contributors to clerkship performance (5–10). The purpose of this study was to examine both types of variables in relation to internal medicine (IM) clerkship performance, with a goal of continuous program improvement. This appears to be the first time interest group membership was examined in a study of academic performance on the IM clerkship.

METHODS

This single-site, retrospective study examined academic and nonacademic data associated with student performance on the IM clerkship at Baylor Scott & White, Temple/Texas A&M Health Science Center College of Medicine (BSW-Temple/A&M) from 2005 to 2011. As the IM clerkship was not offered at all campuses during the study's time frame, we utilized data from the Temple campus only to ensure an adequate sample size and identical educational experiences. At the time of the study, the 12-week IM clerkship at BSW-Temple/A&M included 8 weeks of inpatient and 4 weeks of outpatient experiences. The IM clerkship grade on all campus sites comprises the following: Objective Structured Clinical Exam (OSCE) score (20%), National Board of Medical Examiners (NBME) subject exam score (30%), and clinical evaluation score (50%).

The IM interest group at BSW-Temple/A&M is open to students interested in IM. The group, with approximately 20 members per year, provides peer support and access to physician mentors and sponsors activities focused on careers in IM. Meeting topics include academic expectations during the third year, preparation for residency, career options, work-life balance issues, selection of faculty advisors, and residency interviews. Most members enter careers in IM.

The study was approved by the institutional review board (IRB) at Texas A&M University. Scott & White Healthcare's IRB granted oversight to the Texas A&M IRB.

Data were collected from educational records of 505 third-year medical students who participated in the IM clerkship at BSW-Temple/A&M between June 2005 and June 2011. Data sources included academic and demographic data from the IM Clerkship Office in Temple, Texas, and Student Affairs Office and Registrar's Office from Texas A&M. Formal data collection did not end until summer 2012.

The following data were collected for each subject: name, age (in years), gender, undergraduate major, IM interest group membership, undergraduate GPA, overall IM clerkship grade, OSCE scores, clinical evaluation scores from attendings, NBME subject exam scores (raw and converted), MCAT score, USMLE Step 1 and 2 scores, and GPA for the first, second, and third years of medical school. Names were initially linked to demographic and academic data, but all data utilized for analysis and reporting were deidentified. Data diagnostics were run to ensure that normality assumptions were met. NBME converted scores (grades) were utilized, as both raw and converted scores had similar ranges and standard deviations and met normality assumptions. Descriptive statistics (SPSS Version 16.0) were used to characterize the sample. Pearson product-moment correlation coefficients were generated to examine relationships among variables. Multiple regression analyses were run to address research questions. Squared partial correlations allowed us to examine the unique variance contributed by predictor variables once shared variance was removed (11). The significance threshold was set at P < 0.05. Missing values were not replaced prior to analyses.

RESULTS

Student ages ranged from 24 to 57 (M = 29.75; SD = 4.09), and females (N = 251) comprised 49.7% of the sample. Of the 505 students, 108 (21.4%) were members of the IM interest group. Most students in the sample (387, 76.6%) were science majors as undergraduates. Means and ranges for the academic variables are listed in the Table.

Table.

Academic variable means and ranges

Variable	N	Mean ± SD	Range
MCAT score	432	28.08 ± 3.44	15–39
USMLE Step 1	420	217.96 ± 19.10	101–262
NBME IMED	504	85.64 ± 6.73	61–100
Undergraduate GPA	502	3.68 ± 0.26	2.35–4.00
Second-year GPA	486	3.12 ± 0.59	1.21–4.00

GPA indicates grade-point average; IMED, internal medicine subject exam; MCAT, Medical College Admissions Test; NBME, National Board of Medical Examiners; USMLE, US Medical Licensing Exam.

A multiple regression analysis (N = 341) was run to determine the strength of relationships between predictor variables and NBME as the outcome variable. As first- and third-year GPA were found to be highly correlated with second-year GPA, they could not be entered into the regression model as predictor variables due to multicollinearity issues (12). A model containing the following predictors was significantly associated with NBME subject exam score: OSCE score, MCAT, USMLE Step 1, second-year GPA, and clinical evaluation score. The model provided a good fit to the data—F (9, 331) = 33.891, P < 0.001, R² = 0.480, adjusted R² = 0.465—accounting for 46.5% of the variance in NBME score. Gender, age, interest group membership, and undergraduate GPA were not significantly associated with NBME score. USMLE Step 1 accounted for the most unique variance, followed by OSCE score, based upon squared partial correlations.

A multiple regression analysis (N = 341) was run to determine the strength of relationships between predictor variables and OSCE score as the outcome variable. A model containing the following predictors was significantly associated with OSCE score: USMLE Step 1, clinical evaluation, and NBME score. The model was found to provide a good fit to the data—F (9, 331) = 17.183, P < 0.001, R² = 0.318, adjusted R² = 0.300—accounting for 30% of the variance in OSCE score. Gender, age, IM interest group membership, MCAT score, second-year GPA, and undergraduate GPA were not significantly associated with OSCE score. NBME score accounted for the most unique variance based upon squared partial correlations.

A multiple regression analysis (N = 341) was run to determine the strength of relationships between predictor variables and clinical evaluation score. A model containing the following predictors was significantly associated with clinical evaluation score: age, MCAT score, NBME score, undergraduate GPA, and OSCE score. The model provided a good fit to the data—F (9, 331) = 11.647, P < 0.001, R² = 0.241, adjusted R² = 0.220—accounting for 22% of the variance in clinical evaluation score. Gender, IM interest group, USMLE Step 1, and second-year GPA were not significantly associated with clinical evaluation score in this model. Age accounted for more unique variance in clinical evaluation score than other predictors based upon squared partial correlations. A low to moderate positive correlation between age and clinical evaluation score was found (r = 0.203; P < 0.001).

DISCUSSION

This study found that clinical evaluation and OSCE scores were significantly and positively associated with NBME subject exam scores, supplying further evidence that the OSCE, considered to be a clinical exam, assesses a broader range of clinical performance domains, including some that underlie performance on the NBME exam (12). USMLE Step 1, MCAT, and second-year GPA were also significantly associated with NBME subject exam scores (1, 2, 13). In contrast to previous research, we did not find a significant association between USMLE Step 1 and clinical evaluation score (14) or between undergraduate GPA and NBME score. Additionally, as USMLE Step 1 was significantly associated with NBME score in the regression analysis, an association between USMLE Step 1 and clinical evaluation score may have been expected, but was not found. In examining variables associated with OSCE performance, we found that NBME score accounted for more unique variance than other variables. The OSCE in the IM clerkship utilizes standardized patients for a majority of the stations. We assumed that the clinical evaluation score would therefore account for more unique variance than the NBME score.

Our study showed that age was significantly and positively associated with clinical evaluation score. This finding contrasts with other studies, wherein older age was associated with lower clerkship grades and risk for academic difficulties (5, 7). Gender was not significantly associated with academic variables in this study, which also contrasts with prior research (4, 5, 10, 15, 16). No association was found between interest group membership and academic performance, which is a new finding. We hypothesized there would be a positive association between interest in IM, a proxy for intrinsic motivation, and clerkship academic performance, based upon other findings in the literature (4, 17–19).

This was a single-site study, and findings may not generalize to other academic settings. As with other retrospective studies, the findings can speak to associations among variables, but not causality. We were limited to the data already existing within students' academic records. The retrospective nature of this study prevented us from determining whether age represented a set of attributes not measured, such as better study skills among older students. Strengths of the study included adequate sample size, variables examined, and analysis methods. It was beyond the scope of this study to examine the influence of clerkship curricular requirements, such as patient loads, on academic performance.