To the Editor: Reporting Limitations Misses the Mark to Reshape the Residency “Research Arms Race”

We read with great interest the article by Elliott and Carmody wherein they posit a potential solution of limiting the number of research experiences residency applicants may report to 3 to 5.¹ This solution is predicated on an expanding “publish or perish” phenomenon affecting residency applicants, reporting limits in the 2023-2024 application cycle, as well as what Elliott and Carmody describe as an increasing publication rate of rarely cited, lower-level evidence designs. We agree that the current “publish or perish” model is a detriment to the publication of high-quality biomedical literature and understand that “quantity over quality” promotes research waste, diluting impactful science. We write this to provide additional context regarding the perpetuation of poorly conducted research, the position of residency applicants within the scientific enterprise, and additional pitfalls of the proposed solution.

Poorly conducted research is continuously perpetuated by subpar research training, flawed methodological approaches, inadequate peer review, as well as the “publish or perish” model.^2,3 At the same time, the competitiveness of residency selection forces applicants to seek out research opportunities from primary investigators confined in and influenced by a flawed research enterprise. In this scenario, poorly designed research is continuously conducted and published by residency applicants, perpetuating the “research arms race.” Unfortunately, Elliott and Carmody fail to provide this context for the causal factors that perpetuate poor quality research while placing undue focus on the perceived abilities of physician and residency applicant researchers.

Given the state of biomedical research quality, a publication limit would have little to no substantive effect on research quality but may hinder meaningful research experiences. It is foreseeable that applicants will place effort to simply be published and “check the box” rather than engage in activities to improve their research acumen. Further, we believe applicants would likely limit their research experiences and engage in unethical authorships of more publishable study designs, promoting a new scientometric standard not necessarily improving research quality. In this way, limiting publication reporting may increase poorly conducted systematic reviews and meta-analyses—designs requiring extensive training and knowledge of synthesis methodology, methodological considerations of primary studies, and statistical competence required to be conducted robustly. This is concerning as these Level I evidence designs are frequently used to bolster best practice recommendations, guide health policy, and change clinical practice.⁴ Due to this potential risk, we would consider the status quo of the continued publication of rarely cited, lower-level evidence to be more beneficial to evidence-based medicine and patient care than the risky scenario we describe here.

Ultimately, we wish Elliott and Carmody would have conducted an in-depth analysis with greater context before putting forth their solution. Their article, in our opinion, constitutes a form of “fire fighting” in place of more meaningful solutions such as engaged mentorship and improved evidence-based education. Further exploration of problems and solutions can be found in a joint opinion published in BMJ Evidence-Based Medicine by selected awardees of the Douglas Altman Scholarship consisting of physicians, statisticians, doctoral students, and even a medical student.⁵