Authors’ Response to Recent Letters

To the Editor:

Thank you for the opportunity to review and respond to the two letters 1, 2 being published in this issue of Family Medicine that are in response to our article, “The Predictive Value of the Residency AOBFP In-Service Exam, Produced and Administered by ACOFP,” 3 which was published in the September 2022 issue of this journal. Given the similar themes presented in both letters, we will respond to both here.

First, thank you for the positive and constructive feedback. We also agree that it is important to keep this conversation going in the spirit of collectively supporting family medicine as a specialty and our current and future residents. Thank you for making the important clarification about American Board of Family Medicine, not the American Academy of Family Physicians, developing the In-Training Examination (ITE). To clarify the correlations: Table 1 includes the correlations among in-service examinations. The biserial correlations are reported in the results and are slightly different. So, both are correct as reported.

We also appreciate the suggestion about having the same pool of item writers for both the in-service examination (ISE) produced and administered by the American College of Osteopathic Family Physicians (ACOFP) and the American Osteopathic Board of Family Physicians (AOBFP) exam. While we understand and appreciate where this idea is coming from, we believe a case can be made that this would run counter to our belief that high-stakes exam questions shouldn’t be identical to those used in the formative assessment. Doing so could run the risk of adversely affecting the integrity of the resulting board certification.

Although the ISE is developed by a separate group of family medicine subject matter experts, this formative assessment is constructed to largely mirror the AOBFP test blueprint and ensures that residents are receiving an education that prepares them to pass the AOBFP certifying examination and for independent osteopathic family medicine practice. While the correlation between performance on the ISE and certifying examinations is the focus of this paper, our ultimate goal for the ISE is to provide residents and program directors with meaningful insight into resident growth and identify practice knowledge gaps to optimize residency training programs. For this reason, we have occasionally deviated from the AOBFP blueprint to strategically emphasize valuable material, primarily to improve osteopathic content offered.

The ISE has always included osteopathic content, historically 10 items per exam. ACOFP strategically expanded this content area to 30 questions in 2016. In 2018, we offered the expanded blueprint for osteopathic items with creation of the CORTEx examination, which has been embedded into the ISE for the last 3 years and has maintained the high-quality annual reliability of the exam. The CORTEx exam includes osteopathic items assessing the art and practice of osteopathic medicine across a broad range of disciplines, which complements the broad scope of family medicine practice. The CORTEx examination has high reliability, which is competitive for other residency training examinations, despite our multispecialty subject matter experts crafting the exam entirely from newly written items since the inception of CORTEx in 2017.

For the comment referring to eligibility changes in AOBFP, there appears to be some confusion. For AOBFP’s Early Entry Initial Certification, applicants are required to have completed two osteopathic in-service exams produced and administered by ACOFP. There is no ISE or ITE requirement to be eligible for the standard AOBFP exam for AOA board certification in family medicine. CORTEx, however, was never considered a qualifying exam to sit for the AOBFP. This assessment, however, does help residency programs maintain their Osteopathic Recognition status. It is also a great way to offer focused osteopathic formative assessment for programs only offering the ITE exam.

On behalf of the authors, thank you for the opportunity to receive and respond to the feedback collected in response to our recently published article.