The Allopathic Program Director's Dilemma

Between 1985 and 2006, the number of osteopathic physicians training in Accreditation Council for Graduate Medical Education (ACGME) allopathic residency programs increased 419%,¹ and by 2016, that number is expected to increase an additional 62%.¹ In contrast, the growth in allopathic medical school output is projected to increase by only 21%.^1,2 Unfortunately, allopathic residency directors face difficulties when directly comparing osteopathic and allopathic applicants. These difficulties are exacerbated by differences in curricular emphasis, licensing concerns, and lack of experience with specific osteopathic medical schools because 82% of schools have been founded since 1970 and 32% since 2000.^3,4

Given the anticipated increases in medical school graduates without a significant increase in residency positions, the competition for residencies likely will increase substantially. It is highly probable that graduates of US medical schools will be “dead-ended,” having obtained doctoral degrees but being prevented from practice because they have not completed a residency. In this new competitive environment, the role of objective metrics used in resident selection will assume far greater importance.

The Step 1 score on the United States Medical Licensing Examination (USMLE) is rated as the second most important factor in resident selection.⁵ In a head-to-head comparison of osteopathic and allopathic applicants to an allopathic surgery residency, Schlitzkus et al⁶ found essentially no differences in applicant characteristics, with the exception of higher reported USMLE Step 1 scores in the osteopathic group. A significant criticism of the Schlitzkus study, however, was the possibility of self-selection bias, with students reporting only favorable USMLE scores. An additional bias may also have been that better osteopathic students elected to take that more difficult examination, which was not required of them for graduation. The Comprehensive Osteopathic Medical Licensing Examination of the United States Level 1 (COMLEX–USA) is similar to USMLE Step 1 and is required of all osteopathic graduates. By developing a method to correlate COMLEX Level 1 to USMLE Step 1, program directors of both osteopathic and allopathic residencies would have the ability to evaluate the full applicant pool on at least 1 metric.

In an article in this issue of the Journal of Graduate Medical Education, Lee et al⁷ have attempted to determine the reliability and validity of 2 conversion formulas using COMLEX Level 1 scores to predict USMLE Step 1 scores. The authors have correctly identified that investigations by Slocum and Louder,⁸ of 155 subjects, and Sarko et al,⁹ with only 90 subjects, were underpowered to draw any firm conclusions. By comparing the 2 formulas using 1016 students who took both COMLEX Level 1 and USMLE Step 1, the current study is appropriately powered. The actual impact on resident selection, however, may be less significant than the statistical analysis implies. Although the actual scores on USMLE were significantly higher from a statistical standpoint, the 7.8-point difference for the Sarko formula is likely of little practical concern. In contrast, the 14.2-point difference using the Slocum formula may have some effect on resident selection.

Although residency directors and faculty may have the desire to “compare apples to apples” by translating the unfamiliar COMLEX score into a more familiar USMLE score, the actual utility and appropriateness of doing so is open to discussion. There may be an evolving trend that more osteopathic students will take the USMLE. Whereas Schlitzkus et al⁶ found that 54% of osteopathic applicants reported USMLE scores, Lee and his associates⁷ found that during the study period, 70% of osteopathic students and 100% of the students in the class of 2015 took the USMLE. In contrast to USMLE scores, which are difficult to interpret¹⁰ and for which there is no published mean, COMLEX scores are easy to interpret. For Levels 1 and 2 of the COMLEX, a minimum score of 400 has been set for passing, with a mean score of 550.¹¹ Additionally, the National Board of Osteopathic Medical Examination website provides a formula to convert COMLEX scores into a percentile at the time the examination results are released.¹² Finally, although both examinations have similar formats, the COMLEX contains questions on manipulative treatments and core osteopathic principles. In light of those findings, significant caution should be exercised when interpreting the results of the article by Lee et al.⁷

In February 2014, the American Osteopathic Association announced that the ACGME and the American Association of Colleges of Osteopathic Medicine would pursue a single, united accreditation system for graduate medical education programs in the United States. That agreement may result in uniformity of testing for osteopathic students, which may render conversions between USMLE and COMLEX obsolete.^13,14

In closing, Lee and colleagues⁷ should be congratulated for a clean study design using the largest number of subjects in the literature. Between 2002 and 2012, the number of osteopathic graduates applying to allopathic residencies has nearly doubled.¹⁵ That trend is expected to continue, along with the rapid expansion of osteopathic medical schools, and the work by Lee et al⁷ is important and timely.