“Make it Count Twice”—Studying Curricular Innovations

In this special education edition of the Journal of General Internal Medicine, seven manuscripts illustrate the range of methods that can be used to study, report, and discuss curricular innovations. Young medical educators are taught to “make it count twice”; publishing scholarly work about curricular innovation is an example of doing just that. Scholarly dissemination of creative and novel methods to teach the art and science of doctoring is essential for the continuous improvement of medical education in an ever-changing environment. Scholarship related to curriculum development helps to build an evidence base for innovations that can, in turn, be broadly implemented. Further, medical educators benefit by developing a body of scholarship that can help them to establish a professional identity and propel their own advancement.

Curricular manuscripts must be sufficiently detailed to serve as blueprints for implementation in other educational settings and to provide direction for further scholarly investigation. Publishing high-quality scholarly innovation requires careful planning, thoughtful implementation, and, perhaps most importantly, robust program evaluation. Curricular scholarship should be systematically planned, starting with clear goals and objectives and attention to factors that may affect successful implementation. Kern’s stepwise approach to curriculum development provides a roadmap for thoughtfully developing new curricula. [6]

Appropriate program evaluation methods must be carefully chosen during the curriculum planning phase to allow for an accurate assessment of the effectiveness and practicality of the curriculum. Kirkpatrick levels of outcomes are important to consider when designing a program evaluation. Higher level outcomes like transfer (change in learner behavior after an intervention) or results (actual change in patient outcomes) are more impactful than lower level outcomes like reaction (learners enjoyed the intervention) or learning (learners demonstrated knowledge improvement).[1] It should be noted that manuscripts that include higher level outcomes tend to be more favorably viewed by editorial teams. The seven curricular manuscripts in this special education issue of JGIM vary considerably in their content, approaches to implementation, and program evaluation but all are excellent examples of “making it count twice.”

The first curriculum manuscript, a perspective entitled “Medical Training in Home Care Medicine: The Time is Now,” describes the importance of educating learners about Home Care Medicine and Hospital at Home models, particularly during the COVID-19 pandemic. The authors highlight the growing population of medically and socially complex homebound individuals for whom existing models of healthcare delivery fall short. The authors point out that most training programs inadequately prepare future practitioners to function safely and effectively within these models of care. They issue a call to include clinical training experiences in patient homes. Perspective pieces, such as this one, can effectively raise awareness and equip educators with information and arguments to leverage resources for curricular efforts and further scholarly work. [11]

“Skin cancer education interventions for primary care providers: A scoping review” serves as a high-quality example of a scoping review. An ideal curriculum starts with an exploration of the literature to develop a needs assessment. A scoping review provides a way to “make it count twice” by applying an intentional and rigorous approach to these initial steps of curriculum development. A scoping review may be less familiar to many medical educators and can serve as a powerful way to methodically explore a topic. Arksey and Munn describe scoping reviews as exploratory and descriptive of the nature, extent, and boundaries of a topic; methodologically rigorous, transparent, and highly structured; and unique to systematic reviews, typically excluding quality appraisals and synthetic statistical analyses. [2, 3] The scoping review included in this issue describes the elements associated with more successful interventions for educating primary care practitioners on skin cancer and provides a convincing needs assessment for future curriculum development. [9]

Program evaluation is often neglected in the curriculum development process thereby limiting the generalizability and critical appraisal of interventions. Two manuscripts in this issue are particularly excellent examples of program evaluation. In their evaluation of a clinical curriculum, Rusiecki and colleagues describe an intervention whose aim was to improve the care of female patients in a residents’ clinic. Because they were clear in their curricular objectives, they identified measures of improved care and conducted an appropriate program evaluation to demonstrate the effectiveness of the curriculum in reaching their objective. By reviewing referral patterns of residents through chart audit, the authors demonstrate a change in practice patterns associated with their curriculum. [12] The second example of a higher level curricular evaluation, “Development of a Point of Care Ultrasound Track for Internal Medicine Residents,” describes the development and outcomes of a point of care ultrasound training track for internal medicine residents. Residents were more comfortable with and used ultrasound more frequently after participating in the curriculum again demonstrating a change in practice patterns. Demonstration of high level outcomes allows educators to make a convincing case for further resources or expanded implementation of their curricular efforts. [13]

A randomized, controlled trial (RCT), entitled “Anthropomorphic Character Animations versus Digital Chalk-Talks in a Resident Diabetes Pharmacotherapy Curriculum: A Randomized Controlled Trial,” provides an outstanding example of this difficult educational research methodology. RCTs are unusual in medical education because of the difficulty in preventing cross-contamination for single-site learners, or in controlling for multiple variables in a multi-site study. The authors investigated the acceptability of two different methods of virtual education in a resident diabetes clinic. Although they found little difference in educational attainment between the two methods, learners’ perceived experience was enhanced with the animated series of videos. The authors note that the choice of virtual learning should incorporate several environmental factors, such as learners, resources at the site, and context of the learning. [10]

The qualitative manuscript entitled “Learning Outcomes from an Academic Internal Medicine Morbidity and Mortality Conference” explores what participants learn and implement after participating in an institutional morbidity and mortality (MM&I) program. The authors appropriately chose to use qualitative methods to provide depth and understanding that they might not have achieved if they used quantitative methods. [5] The authors used content analysis to identify and refine themes described by participants in post-session questionnaires. These authors effectively used qualitative analysis to gain insight into the experiences of participants in this institution’s MM&I conference. [8]

“A Mixed-Methods Program Evaluation of a Self-Directed Learning Panel Management Curriculum in an Internal Medicine Residency Clinic” is an excellent mixed-methods study that studied the outcomes of a panel management curriculum initiative built on a framework of self-directed learning (SDL). Mixed methods studies use both quantitative and qualitative methods to triangulate, complement, expand, or otherwise augment one or the other modes of inquiry. [4] The authors qualitatively identified themes from both written narratives and transcripts of facilitated small group interviews. They also used quantitative methods to define the change in delivery of certain health maintenance interventions over time. The authors report that their curriculum did not seem to significantly change health maintenance metrics and their lessons learned from the curriculum. They also noted learners specifically valued self-directed goals, protected time, mentorship for panel management, and meaningful, relevant metrics that are specific, accurate, and timely. [7]

Taken together, these seven manuscripts provide excellent examples of “making it count twice” by creating scholarship from curriculum development. Medical educators are frequently asked to improve or create new curricula; converting these often high effort endeavors into scholarship is both possible and highly advised. When incorporated at the outset, effective scholarly assessment of the process and outcomes of curricular change is far more manageable. Designing curricular innovations with publication in mind clarifies the interventions and the outcomes. This clarity allows for the proactive selection of the most appropriate and achievable study design, leading to planned actions to support the quality of scholarship. Last minute attempts to insert scholarly questions are likely to be more time consuming and less productive, often leading to regret about lack of forethought and missed opportunities. The authors of these seven curriculum manuscripts demonstrate that high-quality curricular scholarship can take multiple forms, depending on the circumstances, including descriptive/perspective pieces, quantitative and qualitative studies (sometimes with randomized and controlled design), mixed methods, and reviews. Whatever the design, an eye for more impactful curricular outcomes, such as changed behaviors and the outcomes of those behaviors, is ideal. Ultimately, high-quality curricular scholarship is vital to the medical education community, individual medical educators, and the future of our learners.

Declarations

Conflict of Interest

None of the authors have reported a conflict of interest.