Do Medical Students in Their Fifth Year of Undergraduate Training Differ in Their Suitability to Become a “Good Doctor” Depending on Their Admission Criteria? A Pilot Study

Abstract

Background

In Germany, the pre-university grade point average (pu-GPA) has to be the main criterion for medical school applicant selection. This is also mandatory in the university-specific selection procedures (Auswahlverfahren der Hochschulen [AdH]). The admission framework has now been reworked following a judgement by the German Federal Constitutional Court. From 2020, more students will be admitted based solely on the pu-GPA and at least two selection criteria independent of the pu-GPA have to be considered in the AdH. However, the question whether an AdH (the core of the AdH at Lübeck Medical School [LMS], Germany, is a 30-mins panel interview led by two faculty members and one student) leads to better doctors as compared to pu-GPA-based selection, remains unanswered.

Objective

To compare students selected based either on their pu-GPA alone (“pu-GPA-students”) or based on the result of the AdH at LMS (“AdH-students”) regarding their suitability to become a good doctor.

Design

We conducted a cross-sectional observational pilot study at LMS. Students were judged regarding their overall suitability to become a good doctor by their supervising general practitioners after a two-week internship in their last year of theoretical medical education. The scores were matched to the selection procedure and compared between the pu-GPA-students and AdH-students.

Results

In all, 79% of the AdH-students were rated as “absolutely suitable” for the medical profession, as compared to 42% of the pu-GPA-students (p = 0.01, odds ratio 5.17, 95% confidence interval = 1.41, 18.99). We did not find any association between gender or age and the suitability rating.

Conclusion

Despite the small sample size, our results indicate that it could be favourable to select medical students not only based on their pu-GPA but also using additional selection criteria.

Introduction

In Germany, there are five applicants per study place at medical schools.¹ At the moment, the statutory framework for medical school admission in Germany is in a transitional phase. To date, the pre-university grade point average (pu-GPA) is the main selection criterion, as 20% of the study places are allocated to the applications via the pu-GPA.² Universities also allocate 60% of their study places via university-specific selection procedures (Auswahlverfahren der Hochschulen [AdH]). However, even with the AdH, the pu-GPA has to be the decisive selection criterion.² The remaining 20% of the study places are mainly allocated to applicants on a waiting list (current waiting time: 7 years). This framework has recently been judged as unconstitutional by the German Federal Constitutional Court.³ The restructured legal requirements for the selection process include a pu-GPA rate of 30%, no waiting time quota and the consideration of at least two selection criteria, which are independent of the pu-GPA, in the AdH.

However, selecting those applicants most likely to become good students and, more importantly, good doctors is very difficult; there neither a generally accepted definition of a “good doctor”⁴ nor does a set of characteristics exist that allow for the prediction of who will succeed in medical education. Several studies have shown that, for academic performance (eg, knowledge test scores, grades, and study progress), especially in the preclinical stage of medical school, the pu-GPA is the best predictor.^5,6 However, the evidence base for the predictive validity of certain applicant characteristics for developing into a good doctor is small, and methodological issues make conclusive studies difficult. Considering the immense effort for the AdH [the core of the AdH at Lübeck Medical School (LMS), Germany, is a 30-mins panel interview led by two faculty members and one student] and the necessity to select those who are most likely to become good doctors, the question arises whether students selected by this kind of procedure are more likely to become good doctors, as compared to students selected solely on the basis of their pu-GPA.

There is some evidence that selection procedures not solely based on cognitive criteria are efficient in identifying applicants with desirable skills. For instance, knowledge test scores, study progress and professionalism scores during the pre-clinical education of medical students admitted to one Dutch university were found to be better in those students selected via a multifaceted selection procedure as compared to those selected via a lottery.⁷ In a study at another Dutch university, those students admitted via a selection procedure aiming at non-cognitive characteristics had a higher mean grade for the practical clinical course in year 3 than those admitted via a cognitive selection.⁸ In a third study, students selected via an elaborate, multistage selection procedure outperformed students who did not pass this selection procedure but were admitted to medical school through a lottery regarding a number of outcomes (eg, cognitive and [inter-]personal skills) during a three-year programme.⁹ However, the selection procedures in these studies were heterogeneous and not readily comparable to the German framework. Additionally, none of the cited studies compared students in different admission quotas after the third year of undergraduate training.

Therefore, the aim of our study was to compare students selected based either on their pu-GPA alone (“pu-GPA-students”) or based on the result of the AdH of LMS (“AdH-students”) regarding their suitability to become a good doctor (rated in the last year of theoretical studies).

Materials and Methods

We conducted a cross-sectional observational pilot study including medical students in their last year of theoretical medical education at one public German medical school (LMS). Students, who had given written consent to participate in the first session of the general practice course after receiving information on the study, were rated by the supervising general practitioner (GP) at the end of a two-week internship in a general practice regarding their overall suitability to become a good doctor; a 4-point Likert scale from “not at all suitable” to “absolutely suitable” was employed for the rating system. There were no exclusion criteria. This question is part of a questionnaire developed by Knorr et al¹⁰ containing 11 items on different aspects, such as communicational and interpersonal skills.

The ratings and selection approach were matched by a data custodian and subsequently anonymized.

Data analyses were conducted in IBM SPSS Statistics for Windows Version 22.0 (IBM Inc., Armonk, New York, United States). We used χ²-tests with an alpha of 0.05. After dichotomising the suitability rating into “absolutely suitable” and “less than absolutely suitable”, the odds ratio (OR) was calculated as a measure for the effect size.

Results

Seventy-three ratings by GPs could be matched to the selection approach (N = 61 AdH-students quota, N = 12 pu-GPA-students). The gender distribution in the two groups did not differ significantly (AdH-students: 66% female; pu-GPA-students: 67% female; χ² [1, N = 73] = 0.01, p = 1.0). The mean age of the AdH-students was 24.1 years (Standard deviation [SD]: 1.51) as compared to 23.6 years for the pu-GPA-students (SD: 1.24).

The majority (73%) of the students were rated as “absolutely suitable” by their supervising GP. Forty-eight (79%) of the AdH-students were rated as “absolutely suitable” for the medical profession, as compared to five (42%) of the pu-GPA-students (χ² [1, N = 73] = 6.91, p = 0.01, OR 5.17, 95% confidence interval = 1.41, 18.99). We did not find any statistically significant association between gender or age and the suitability rating.

Discussion

In our cross-sectional observational pilot study at one German medical school, medical students who had been admitted through the AdH, including panel interviews, were rated significantly better by supervising GPs regarding their suitability to become a good doctor.

Our results are in line with the results of earlier studies on the predictive validity of selection procedures employing non-academic criteria. The studies conducted at different Dutch universities cited above showed consistently, despite heterogeneous selection procedures, that the consideration of non-academic criteria leads to better “patient-relevant” outcomes, such as interpersonal skills.^6–9 Knorr et al (2018) examined the suitability of medical students for the medical profession in order to validate their selection procedure based on multiple mini-interviews (MMIs).¹⁰ In their study, supervising GPs were asked to rate medical students after a one-week internship in their practices, following the first year of medical education. The MMI score showed a significant correlation with the suitability (assessed by the same question we used) rating.

Due to the small sample size, limiting the possibility of more in-depth analyses, our results have to be interpreted with caution. However, the difference in competence ratings between the students selected via the different approaches is anything but negligible. Another limitation of our study is its single-centred nature. Due to the unique nature of the AdH at LMS, the results cannot be generalised to other medical schools. The global, single-item suitability rating is prone to the subjective views of the raters. However, to our knowledge, this is the first study showing differences between students selected for medical school by different criteria in the later years of medical education, just before their final undergraduate year. We chose the fifth year of undergraduate training for the assessment, because at LMS, students attend a two-week internship in a general practice in this year. The rating by GPs, who come to know the students for these 2 weeks during their consultation hours/home visits and teach them in a 1:1 setting, should, therefore, be comparatively valid, as no other supervisor spends this amount of time with single students. Thus, our results could be a signal and motivation for other universities to employ similar studies for the evaluation of their often resource-intensive AdHs.

Conclusions

Our results indicate that it could be worth the effort to select medical students not only based on their pu-GPA but also using certain criteria in a well-designed selection process. The probative value of our results is clearly limited by the small sample size. Hence, we will resume collecting data in a broader context. Further research (preferably by means of prospective, longitudinal studies of adequate size) on suitable selection criteria for medical education is needed.

Funding Statement

We acknowledge financial support by Land Schleswig-Holstein within the funding programme Open Access Publikationsfonds. We received no further funding.

Abbreviations

AdH, Auswahlverfahren der Hochschulen (university-specific selection procedures); GP, General Practitioner; LMS, Lübeck Medical School; MMI, Multiple Mini-Interview; OR, Odds Ratio; pu-GPA, Pre-university Grade Point Average; SD, Standard Deviation.

Ethics Approval

The study protocol was approved by the Ethics Committee of the University of Lübeck (file reference 16-143). All participants provided written informed consent.

Data Sharing Statement

The dataset generated during this study is not publicly available due to ethico-legal reasons (lack of consent of the study participants) but will be available from the corresponding author upon reasonable request to all interested researchers.

Disclosure

The authors declare no competing interests.