Abstract
Background:
Physicians are charged with implementing evidence-based medicine, yet few are trained in the science of Dissemination and Implementation (D&I). In view of the potential of evidence-based training in D&I to help close the gap between research and practice, the goal of this review is to examine the importance of D&I training in medical education, describe challenges to implementing such training, and provide strategies and resources for building D&I capacity.
Methods:
We conducted (1) a systematic review to identify US-based D&I training efforts and (2) a critical review of additional literature to inform our evaluation of the challenges and opportunities of integrating D&I training in medical education.
Results:
Out of 269 unique articles reviewed, 11 described US-based D&I training. Although vibrant and diverse training opportunities exist, their capacity is limited, and they are not designed to meet physicians’ needs. Synthesis of relevant literature using a critical review approach identified challenges inherent to changing medical education, as well as challenges related to D&I science. Finally, selected strategies and resources are available for facilitating incorporation of D&I training into medical education and overcoming existing challenges.
Conclusions:
Integrating D&I training in the medical education curriculum, and particularly in residency and fellowship training, holds promise for bridging the chasm between scientific discoveries and improved patient care and outcomes. However, unique challenges should be addressed, including the need for greater evidence.
Keywords: Dissemination and implementation, implementation science, knowledge transfer, medical education, systematic review and translational science
Introduction
One of the key tasks facing medical education consists of training physicians who can bridge the translational gap between research and practice. With estimates that 1 out of 3 patients receive care that does not comply with current scientific evidence,1 the provision of evidence-based care is a key challenge.2–4 Medical educators are also grappling with this gap,4–7 and calls for reforms of medical education are at least a century old.7 For instance, following decades-long efforts to teach patient safety and quality improvements and identification of best practices,8 dissemination of such programs is lacking.9 Addressing translational gaps involves multifaceted, complex processes that consider the context as well as the systemic nature of adoption of innovations. Dissemination and implementation (D&I) science is tasked with identifying effective ways to reduce translational gaps between research and practice, often referred to as the “valley of death.”10 According to the National Institutes of Health (NIH), dissemination refers to the purposive distribution of health information and evidence-based interventions, whereas implementation science refers to the study of how to integrate research findings into evidence-based policy and practice.11 As a new science, however, consistency of concepts and their definitions remains a challenge, and knowledge translation, knowledge transfer, and diffusion, as well as similar constructs are related concepts that often include overlapping definitions.12,13
The NIH,14 Institute of Medicine (IOM),15 Veterans Administration, Agency for Healthcare Research and Quality,16 AcademyHealth, American Board of Internal Medicine, and Patient-Centered Outcomes Research Institute17 have all declared advancing D&I efforts to reduce the translational gap to be a priority area, which creates critical pressure on the next generation of clinicians and medical educators. Numerous authors emphasized the importance of building stakeholders’ capacity for D&I of interventions in medical education,18–23 health care,24 and community settings.25 In view of physicians’ key role as change agents and leaders in the health care system, including medical education,26,27 researchers and practitioners raised concerns about lack of training in the science of how to lead such changes18,28 and a growing number of medical education scholars underscored the importance of training physicians in D&I.16,29–32
Despite the recognized need to build physicians’ capacity in D&I, integration of D&I training into medical education and the specific needs and challenges of such integration have not been previously examined. Therefore, in this review, we aim to identify and describe specific D&I training opportunities and to critically examine literature on D&I training and medical education to identify challenges associated with possible integration of D&I training into medical education, as well as available resources. We acknowledge important D&I training programs in Canada.33,34 However, in view of the importance of the health care context and particular requirements of medical education in the United States, we focus this analysis on training opportunities available in the United States. Specifically, the objectives of this critical narrative review are 3-fold: (1) to examine the importance of D&I training in medicine and medical education nationally, (2) to describe challenges to implementing such training, and (3) to provide strategies and resources for building D&I capacity in medical education.
Materials and Methods
In view of our focus on integration of D&I training in medical education curriculum, we selected a critical review approach.35 This approach aims to document a comprehensive search of the literature and to provide a critical evaluation of its content. Effectiveness of critical reviews is measured in the degree to which they present, analyze, and synthesize materials from diverse sources. This method provides an opportunity to assess the current situation based on a previous body of work and to propose a new path based on synthesis of different schools of thought.35
First, we used systematic literature review processes36 to identify all articles reporting on specific D&I training programs in the United States. We searched multiple databases (Medline, PubMed, CINAHL Complete, Web of Science, and EBSCOhost) in December 2016. Search terms included “dissemination and implementation training,” “D&I Training,” and “implementation training.” Two reviewers independently reviewed the titles and abstracts of the retrieved articles.
Inclusion criteria: articles reporting on specific training programs within the United States. Articles were excluded if they were not written in English or did not report on specific US-based D&I training opportunities. Following a critical review approach to literature synthesis,37,38 we also examined reference lists of widely cited papers and review articles. The identified articles were shared with the research team to ensure that no articles were overlooked (see Figure 1 for details on this process). Consistent with systematic review guidelines, 2 authors then independently evaluated the identified articles for strength of evidence regarding the evaluation and effectiveness of the D&I training.
Although the above systematic search strategies provide vigor, we also aimed at enhancing the scope of this review. In view of our overall goal of conducting a critical review that synthesizes different research disciplines and approaches, we also did a targeted literature search35 to locate articles that identified general factors associated with medical education curriculum changes. In contrast to systematic literature review, this search strategy is consistent with the goal of critical reviews “to collect, integrate and interpret results from the most compelling studies that satisfy the search terms and strategy. The search and written presentation need not be exhaustive.”37,39 Therefore, we judged the relevance and rigor of available research studies in relation to our overall focus, with the goal of summarizing findings from different studies qualitatively to inform our understanding of integrating D&I training in medical education. We therefore used 2 different search strategies to explore medical education as a context for D&I training that can explain challenges and opportunities to adopt D&I training.39
Results and Discussion
Opportunities for D&I trainings and outcomes
As illustrated in Figure 1, the systematic search yielded 415 articles, with 147 duplicates and 193 that on review of title did not meet eligibility criteria. We reviewed 75 abstracts, of which 10 articles described specific D&I training programs taking place in the United States.40–49 An additional article was identified by one of the authors,50 for a total of 11. Based on these articles and additional sources, we identified diverse D&I training formats, including webinars, conferences, training institutes, certificate programs, graduate courses and programs, internships, and fellowships. Selected programs and institutes are shown in Table 1.
Table 1.
Program name | Goal(s) | Trainees | Length | Components | Outcomes | Funding source(s) | Citation |
---|---|---|---|---|---|---|---|
Ongoing programs | |||||||
Colorado Research in Implementation Science Program (CRISP) D&I workshop | To bring D&I training to local investigators | Investigators affiliated with CRISP and Colorado Clinical and Translational Sciences Institute (CCTSI), most with doctoral level training and limited D&I experience; University faculty (73%), research staff (8%), graduate students/fellows (15%), practicing health care providers (15%), other (13%); Trainee degrees included 36% MD, 38% PhD/DrPH, 35% MPH/MSPH/MS, 23% other |
1.5-day workshop | 1. Day 1: Introduction to D&I, strategies, evaluation and measurement, RE-AIM framework 2. Day 2: In-depth feedback on D&I proposals/projects, submission of short concept paper on D&I research, practice, or project management, group discussion |
1. High overall satisfaction ratings 2. Increase in self-reported knowledge 3. Suggested areas of improvement included continued seminars and feedback 4. One-third of 6-month postworkshop evaluations reported: (a) A D&I scientific paper (b) A D&I grant proposal (c) New D&I collaborations |
AHRQ, NIH/ NCATS Colorado CTSI; VA | Morrato et al44 (www.CRISPebooks.org) |
Implementation Research Institute (IRI; http://iristl.org/) | 1. To train implementation researchers 2. Production of scholarly implementation products (papers, books, and curriculum models) |
10-11 PhD or MD research fellows per year, ranging from 1-year postterminal degree to full professor Fellow disciplines for 2010-2012 included psychology (61%), social work (13%), medicine (10%), psychiatry (6%), epidemiology (3%), and anthropology (6%) |
Two annual week-long trainings with ongoing mentoring | 1 Annual 5-day institute 2. Site visits 3. Mentoring monthly/bimonthly 4. Pilot research 5. Attendance at implementation science conference 6. Evaluations by fellows and faculty |
1. Overall program satisfaction very high 2. Fellows from first 3 cohorts (n = 31) have received 52 funded awards 3. Average 7.64 publications submitted and/or published per fellow |
NIMH (R25); VA | Stamatakis et al,49 Proctor et al47 |
Mentored Training for Dissemination and Implementation Research in Cancer (MT-DIRC; http://mtdirc.org/) | 1. To focus on training for D&I research in cancer prevention and control 2. To develop and refine a set of core D&I competencies 3. To develop and refine a model curriculum specific to D&I research on cancer disparities |
Postdoctoral fellows in the early stages of their careers, as well as mid-career scholars with an interest in D&I in the field of cancer | Two annual week-long trainings | 1. One-week on-site training at Washington University in St. Louis 2. Monthly, long-distance, mentoring to help craft a competitive research proposal 3. Networking with other fellows and faculty to produce scholarly products 4. Pilot project funding |
Several publications, presentations, and funded grant proposals, and the development and refinement of core competencies Social network analysis of patterns of mentoring and collaboration among IRI participants, modeling of the relationship between mentoring and subsequent collaboration |
NCI (R25) | Padek et al,46 Luke et al50 |
Prevention and Control of Cancer: Post-Doctoral Training in Implementation Science (PRACCTIS; http://www.umassmed.edu/pracctis) | US citizens or permanent residents with recent doctoral degrees (MD, DO, PhD, etc) from a range of disciplines | 2- or 3-year full-time postdoctoral program | 1. Take specialized courses in implementation science 2. Complete optional Master of Science in Clinical Investigation 3. Receive team mentorship from faculty and site partners 4. Join research teams, gather data, publish manuscripts, 5. Participate in mock NIH grant review sessions and draw on career planning and placement resources |
Not reported | National Cancer Institute | ||
UCSF Program in Implementation Science (https://accelerate.ucsf.edu/training/ids) | 1. To design and implement effective interventions 2. To design comprehensive evaluations of interventions 3. To develop better grant proposals |
Broad range of professionals engaged in D&I | A concentration track in a 2-year full-time master’s degree in clinical research or a 1-year part-time certificate program (http://ticr.ucsf.edu/courses/implementation_research.html) | 1. Translating Evidence into Practice Theory and Design 2. Translating Practice into Evidence: Community-Engaged Research 3. Translating Evidence Into Practice: Individual-Centered Implementation Strategies 4. Program Evaluation in Clinical and Public Health Settings 5. Translating Evidence Into Practice: System-Centered Implementation Strategies 6. Translating Evidence Into Policy: Framing Research to Influence Policy |
Not reported | UCSF | Gonzales et al42 |
Training in Dissemination and Implementation Research in Health 2016 (TIDIRH 2016; https://www.scgcorp.com/tidirh2016/index.html) | To encourage institutional interest in D&I by having participants share training methods with their home institutions | Junior or senior investigators with doctoral degree who have not previously received funding for D&I-related research Researchers in fields of medicine, behavioral medicine, nursing, medical anthropology, health economics, public health, or health policy |
Three-month online course and a 2-day in-person workshop | 1. Theory, implementation, and evaluation 2. Creating partnerships and multidisciplinary research teams 3. Research methods, design, and analysis 4. Clinical, community, and policy interventions |
Not reported | NIH (OBSSR, NCI, NIDDK, NHLBI); VA | |
Recent programs | |||||||
11 different programs in North America | Synthesis of multiple D&I research training programs | NA | Not reported | Review of existing programs | Lessons learned in mental health practice | NIH (NCI, NIDDK, NCATS) | Chambers et al41 |
Developing a field-based approach to D&I research training | To address key gaps and opportunities in D&I research training | NIH staff and 10 researchers | Not reported | NIH meeting addressing the overarching issue of developing D&I research training |
1. Expert opinion on training 2. Identified resources and gaps |
NIH | Proctor et al51 |
Implementation Research Workshop | To advance implementation research in respiratory, sleep, and critical care medicine | Researchers from diverse disciplines (n = 24) 15 MD (61%) |
One day | Expert opinion on and experiences with barriers and facilitators to implementation | Recommendations for implementation were provided, but no assessment of the workshop | American Thoracic Society; NHLBI | Bender et al40 |
Training in Dissemination and Implementation Research in Health (TIDIRH) | 1. To increase submission rate and quality of D&I grant applications and publications 2. To encourage institutional interest in D&I by having participants share training methods with their home institutions |
35 investigators (new and experienced) with no prior major research funding Trainee disciplines include psychology (34%), medicine (17%), epidemiology (14%), health behavior/education (11%), other (24%) | Five-day training | 1. Introduction, overview, design 2. Design and measurement 3. Intervention and methods 4. Scale-up 5. Evidence |
1. Very high ratings of agenda relevance, appropriateness of teaching strategies, and confidence in applying skills 2. At 6-month follow-up, 72% of trainees had initiated a new D&I grant application 3. 28% of trainees received funding for a new D&I grant 4. 97% of trainees reported using TIDIRH skills/knowledge to influence thinking of colleagues 5. 33% participated in post-TIDIRH online networking platform; 67% participated in post-TIDIRH conference call |
NIH (OBSSR, NCI, NIMH); VA | Meissner et al43 |
UAB School of Public Health D&I in Health Course | To complement and expand on existing D&I training programs | Current graduate public health students and academic researchers (24 students and 19 faculty total); faculty associated with School of Medicine (44%), School of Nursing (11%), College of Arts/Sciences (5%), School of Dentistry (5%), School of Health Professions (5%), School of Public Health (5%), and other (5%); Student degrees included bachelor’s (78%), master’s (18%), and medical degree (9%) and were associated with departments of health behavior (91%) and health care organization/policy (4%) |
Biweekly 75-minute seminar over Fall semester (2012-2013) | 1. Didactic lectures 2. Viewing/listening to online presentations 3. Reviewing pertinent resources 4. Collaborative learning project with mixed student/faculty teams |
1. High overall satisfaction ratings 2. Suggested areas of improvement from students included logistics and delaying start of collaborative project to give time to learn general concepts; faculty recommended clearer expectations for collaborative project and opportunity to attend lectures |
UAB | Norton45 |
Abbreviations: AHRQ, Agency for Healthcare Research and Quality; CIHR, Canadian Institutes of Health Research; EBP, evidence-based practice; KT, knowledge translation; NCATS, National Center for Advancing Translational Sciences; NHLBI, National Heart, Lung, and Blood Institute; NIDDK, National Institute of Diabetes and Digestive and Kidney Diseases; NIH, National Institutes of Health; NIMH, National Institute of Mental Health; OBSSR, Office of Behavioral and Social Sciences Research; UAB, University of Alabama at Birmingham; UCSF, University of California San Francisco; VA, Department of Veterans Affairs.
The 11 identified articles reported on 6 different D&I training programs, including the American Thoracic Society and National Heart, Lung, and Blood Institute Implementation Research (ATS-NHLBI IR) workshop that centered on implementation research in respiratory, sleep, and critical care medicine,40 3 NIH-funded D&I training institutes,41,43,46-50 2 university-based training opportunities,42,44 and a D&I in health course.45 Table 1 summarizes information about each of these training opportunities, including trainees, goals, assessment, and outcomes. Although most of the programs listed physicians among participants in the training,43-45,48 the authors did not report on any attempts to align curriculum or evaluation criteria to these trainees or to the mission of medical education.
Publications about these programs have focused on training needs, competencies, and frameworks.42,46,48 Trainees’ perceptions of these programs have generally been positive.41,44,46,48 Important factors in training satisfaction included the expertise of the faculty and trainees, faculty flexibility in adjusting content to meet trainee needs, highlighting concrete D&I examples,42 learning about the development of practice linkages,45,49 and enjoyment of collaborative learning projects.45 Faculty have reported challenges in deciding on the curriculum. These challenges related to striking a balance of didactics, focusing on structure versus interactivity and flexibility, and meeting the needs of trainees from different fields, institutions, and at various levels of career development.43
Unfortunately, the evidence regarding the effectiveness of these programs is limited. Only 5 of the 11 articles reported outcomes assessment.43,44,48,50 These outcomes included positive trainee perceptions of the programs43,49,48-50 and in some cases objective outcomes such as numbers of publications or grants awarded43,47 and networks formed.50 Timing of follow-up assessment differed, with 2 programs assessing outcomes at 6 months after the programs occurred43,44 and others looking cumulatively over several years and thus apparently at different time points after the training depending on the cohort.47,49 An additional limitation of the articles related to their strength of evidence. The studies reported were descriptive with no comparison groups. Findings of the 5 articles reporting on outcomes are therefore consistent with the fourth level of 5 acceptable levels assessing scientific evidence.52 Finally, although some outcomes reported were impressive, their relevance to medical education is not always clear. For example, 70% of grant proposals submitted by Implementation Research Institute trainees were funded.47 This outcome might be highly relevant to those engaged in research careers but the mission of medical education might require different measures of effectiveness.5
In summary, our analysis of the available literature on D&I US-based trainings reveals that the vibrant and diverse training opportunities described above provide exciting options for individuals interested in D&I, including physicians, yet the capacity of current training programs and their evidence base have not kept up with the growing demand for D&I workforce education and development.43 Consequently, the development and implementation of rigorous, sustainable training has been recognized as one of the major challenges facing the field of D&I.41,48 Despite the important role of physicians as change agents in the health system and congruence between the mission of medical education and D&I efforts, few if any, opportunities have been designed specifically for physicians. Moreover, scholars have not explored the importance and feasibility of consistently integrating D&I training into specific medical education training and the potential factors that should be addressed to facilitate such integration. To address this gap, we examine these factors in the following section.
Barriers and facilitators to integration of D&I training into medical education
Medical education and practice patterns are complex and constantly evolving in response to scientific discoveries, technological advancement, social trends, and policy changes. The dynamic nature of practicing medicine poses challenges to medical education, including medical school curriculum, residency training, and fellowships.53 The diverse, systemic challenges facing training in different medical education contexts are well-documented.25,54–56 More than a dozen factors have been shown to be consistently associated with such changes.55 These factors relate to organizational culture, communicative factors such as internal networking, and factors within the external environment55 such as financial pressures.56 Integration of D&I into medical education programs necessitates addressing the above organizational factors and the pressures on medical curriculum, including competing agendas in an environment of limited time, financial resources, and faculty capacity.56
An additional challenge that should be addressed in transforming physicians’ behavior relates to considering not only the formal curriculum that resides in current medical school educational content but also the “hidden” curriculum, which relates to a less obvious, but more influential set of behaviors that should be recognized in attempts to change provider practices.54 Such changes are relevant on the continuum of medical education, including graduate medical training (ie, residency), just prior to setting providers free into the delivery system where they will face a host of organizational, provider and patient factors that may influence their behavior. The importance of graduate medical education is further underscored by recent scrutiny of the effectiveness of continuing medical education (CME). Although well-designed CME has been demonstrated to improve physician performance and patient outcomes,57 CME is often ineffective in changing medical practices.58
In addition to addressing challenges inherent to changing medical education as described above, integration of D&I training in medical education should address specific challenges inherent to the field of D&I. Such challenges include the difficulty in generalizing across delivery system contexts, defining and maintaining intervention fidelity, the extent to which adaptation of an intervention’s components influences effectiveness, as well as challenges related to funding availability and the timing of funding cycles.59 In addition, D&I is a transdisciplinary field, and its science and practice involve multiple and complex theories and models.60 Although this complexity increases the difficulty of implementing such training, it also increases its importance.61 In the following section, we discuss opportunities for overcoming the challenges in integrating D&I training in medical education.
Strategies and resources to support improved implementation of D&I training into medical education
The prior sections provided examples of training programs that could inform D&I training within medical education and challenges to integrating D&I training into medical education that could be addressed. In this section, we present an overview of selected strategies and resources for facilitating incorporation of D&I training into academic medicine. We also highlight ways in which such strategies and resources can begin to address some of the inherent challenges in conducting D&I research and practice.
Understanding variability in contextual factors influencing medical curriculum change
Dissemination and Implementation calls attention to the important influences of various contextual factors, including differences in clinical settings, patient populations, and policies. All of these contextual factors are important to consider when trying to generalize study findings or implement evidence-based practices. Similarly, many factors are relevant to encouraging curriculum change and spurring innovative D&I training at the institutional level. Early on in the process, it is important to establish an organization’s level of readiness to adopt a curriculum change. In contrast to the recognition of the importance of identifying effective learning assessment strategies,62 organizational factors, including organizational readiness in medical education, are largely understudied.63 The Medical School’s Organizational Readiness for Curriculum Change is a validated questionnaire that provides a structured way to assess readiness.64 Furthermore, the need for change should be recognized among multiple levels and types of stakeholders and not be dictated by administration. Using a student-centered curriculum review team could be a strategy to solicit and apply student feedback into curriculum design in academic medicine.65 As described above, tailoring strategies to specific institutional contexts can be a challenge in conducting D&I research and practice. Institutionally supported mentorship programs could serve as a time-efficient strategy that is tailored to mentee needs66 either as an addition to D&I training or as a stand-alone D&I training program.67
Opportunities in graduate medical education for formalized D&I research and practice
Graduate medical education is uniquely positioned to adopt D&I training, as it provides “formal intersection of medical education and care delivery” according to the Association of American Medical Colleges (AAMC). The Accreditation Council for Graduate Medical Education (ACGME) employs best practices, research, and advancements across the continuum of medical education with a specific focus on 6 core competencies for residency and fellowship training including patient care, medical knowledge, interpersonal and communication skills, professionalism, practice-based learning and improvement, and systems-based practice.26 The latter is clarified as “Residents must demonstrate an awareness of and responsiveness to the larger context and system of health care, as well as the ability to call effectively on other resources in the system to provide optimal health care.” This core competency is very much aligned with opportunities for formalized D&I training during an influential growth period for early-career physicians. Moreover, residents and fellows are recommended to “participate in identifying system errors and implementing potential systems solutions” according to the ACGME’s common program requirements.68 However, challenges associated with the complexity of such efforts and the lack of D&I training may often lead to suboptimal efforts and lack of effective system solutions. Such potentially missed opportunities are evident in ACGME’s core competencies of systems-based practice and practice-based learning and improvement. Although residents are required to demonstrate patient safety and quality improvement skill, site visits indicated that many graduate medical education clinical learning environments “do not provide the necessary systems-based practice context for residents’ clinical experience.”69 The authors also expressed a concern about what they described as a potentially “lost opportunity to create a cadre of young physicians equipped to lead sustainable systems-based improvement in clinical care” (p. 991). Harnessing the enthusiasm of trainees and their fresh take on challenging delivery system dilemmas through formal D&I didactic and core competencies such as practice-based learning curriculum could be transformational for the next generation of practicing physicians and promote physician-scientists capable of not only advancing D&I science but also affecting population health through evidence-based implementation practices.
Organizational-level changes to promote the integration of D&I curricula
Internal D&I mentorship programs and incentivizing participation in national training programs (see Table 1) have the potential to advance D&I training in medical education. Such changes would require institutional-level changes, consistent with the recognized need to reorganize structural aspects of medical schools to promote health care innovation.70 Strategies for providing training and support to encourage budding physician-scientists interested in health systems innovation can be applied to spur demand from trainees to pursue areas of D&I science.71 We propose that these strategies include creation of career pathways and additional promotion criteria for those focused on D&I science that could be analogous to basic science which has a long tradition of integration into medical school core curriculum and a track record for promoting the physician-scientist model. Developing academic capacity in D&I could have tremendous spillover effects into the broader mission of the medical education system to improve population health. Moreover, as a way to limit the effects of departmental silos, a structure for blending (or “interweaving”) faculty from across departments into multiple curriculum committees can support a shared school mission72 including support for the multidisciplinary field of D&I science. Ranking and evaluating medical schools using metrics that value D&I research and practice outcomes (including metrics related to practice improvement and reduction in medical errors) has the potential to enhance institutional engagement and commitment to integrate D&I training into the core curriculum for medical students, residency training, and CME.
Advancing D&I research and practice as part of medical education can support the IOM vision of developing a “learning health care system,” designed to initiate and use the best evidence for the collaborative health care choices of each patient and provider by integrating the process of discovery as part of patient care.73 To encourage the transition toward this vision, D&I training would provide additional opportunities to support and increase the utility of the practice components of a medical school, including university hospitals and affiliated clinics as essential components in a learning health care system. To harness the benefits of a learning health care system, integrating D&I research and practice training into rotations, internships, residencies, and fellowships as a core component appears warranted. Fashioning clinician training as a D&I enterprise focused on iterative improvement of practice, implementation of evidence-based interventions, sustainability of high-quality care, and improved understanding of the interventions once they have been implemented can provide a unique way to advance the learning health care system and ultimately patient care.
Conclusions
The chasm between biomedical discoveries and improved patient care has been deemed the “valley of death”10 as evidence-based practices and guidelines are not well adhered to. The literature that we have reviewed documented the state of the art in D&I science training. Despite the known challenges to changing the curriculum of medical education, existing training opportunities are not designed to meet the needs of medical education and are not ready to be disseminated and upscaled. Therefore, greater evidence is needed before such integration is viable. Based on this literature, we have provided suggestions for and examples of D&I training that could be incorporated into medical education. More rigorous research, including well-designed, targeted training efforts, is needed to successfully integrate D&I training best practices in medical education.
Acknowledgments
The authors would like to thank Sayyed Fawad Ali Shah for his help in the systematic search of articles included in this review, and Kristen Sorice for her assistance in creating the initial version of the table.
Footnotes
Funding:The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Support for this work came from National Cancer Institute at the National Institutes of Health, through the project entitled Mentored Training for Dissemination and Implementation Research in Cancer Program (MT-DIRC) (5R25CA171994-02).
Declaration Of Conflicting Interests:The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Author Contributions: TG led in conceiving the paper, wrote the first draft of the paper, conducted the analysis, and edited the final version.
CJH helped conceive the paper, created the table, contributed to writing, reviewed the paper, and edited the final version.
DC helped conceive the paper, contributed to analysis, writing, reviewed the paper, and edited the final version.
LMQ helped conceive the paper, contributed to analysis, writing, and reviewed the paper.
DAC helped conceive the paper, contributed to writing, reviewed the paper, and edited the final version.
TAS contributed to writing and reviewed the paper.
RCB helped conceive the paper, contributed to writing, reviewed the paper, and edited the final version.
References
- 1. Grimshaw JM, Eccles MP. Is evidence-based implementation of evidence-based care possible? Med J Australia. 2004;180:S50–S51. [DOI] [PubMed] [Google Scholar]
- 2. Bero LA, Grilli R, Grimshaw JM, Harvey E, Oxman AD, Thomson MA. Closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings. Br Med J. 1998;317:465–469. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Adm Policy Ment Health. 2009;36:24–34. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Simpson D, Sullivan GM. Knowledge translation for education journals in the digital age. J Grad Med Educ. 2015;7:315–317. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Tractenberg RE, Gordon M. Supporting evidence-informed teaching in biomedical and health professions education through knowledge translation: an interdisciplinary literature review. Teach Learn Med. 2017:1–12. [DOI] [PubMed] [Google Scholar]
- 6. Smith S, Kind T, Beck G, et al. Further dissemination of medical education projects after presentation at a pediatric national meeting (1998-2008). Teach Learn Med. 2014;26:3–8. [DOI] [PubMed] [Google Scholar]
- 7. O’Brien BC, Irby DM. Enacting the Carnegie foundation call for reform of medical school and residency. Teach Learn Med. 2013;25:S1–S8. [DOI] [PubMed] [Google Scholar]
- 8. Wong B, Hollenberg E, Etchells E, Kuper A, Levinson W, Shojania K. The emergence of quality improvement and patient safety training in postgraduate medical education: an updated systematic review of QI and patient safety curricula, 2012. https://static1.squarespace.com/static/564b7112e4b0c2d8793bda33/t/56af9b97d210b8652093e8f2/1454349207356/Wong+Updated+Syst+Review+AmJMedQ.pdf.
- 9. Wong BM. Reporting on patient safety and quality improvement education: designing projects for optimal dissemination. J Grad Med Educ. 2015;7:513–516. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Butler D. Translational research: crossing the valley of death. Nature. 2008;453:840–842. [DOI] [PubMed] [Google Scholar]
- 11. (NIH) NIoH. PAR-16-238 Funding opporunity: dissemination and Implementation Research in Health (R01). [Google Scholar]
- 12. Rabin BA, Brownson RC. Developing the terminology for dissemination and implementation research. In: Brownson RC, Colditz GS, Proctor EK, eds. Dissemination and Implementation Research in Health: Translating Science to Practice. Oxford, UK: Oxford University Press; 2012:23–51. [Google Scholar]
- 13. Graham ID, Logan J, Harrison MB, et al. Lost in knowledge translation: time for a map? J Contin Educ Health Prof. 2006;26:13–24. [DOI] [PubMed] [Google Scholar]
- 14. Glasgow RE, Chambers DA, Cynkin L. News from the NIH: highlights in implementation science from the National Cancer Institute and the National Institute of mental health. Trans Behavior Med. 2013;3:335–337. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Zerhouni E. Medicine. The NIH roadmap. Science. 2003;302:63–72. [DOI] [PubMed] [Google Scholar]
- 16. Bernstein SL, Stoney CM, Rothman RE. Dissemination and implementation research in emergency medicine. Acad Emer Med. 2015;22:229–236. [DOI] [PubMed] [Google Scholar]
- 17. Esposito D, Heeringa J, Bradley K, Croake S, Kimmey L. PCORI Dissemination and Implementation Framework. Washington, DC: Patient-Centered Outcomes Research Institute; 2015. [Google Scholar]
- 18. Cilliers FJ, Tekian A. Effective faculty development in an institutional context: designing for transfer. J Grad Med Educ. 2016;8:145–149. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Sullivan GM. Deconstructing quality in education research. J Grad Med Educ. 2011;3:121–124. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Kozakowski SM, Eiff MP, Green LA, et al. Five key leadership actions needed to redesign family medicine residencies. J Grad Med Educ. 2015;7:187–191. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Fordis M, King JE, de Nigris FB, et al. Dissemination of evidence from systematic reviews through academic CME providers: a feasibility study. J Cont Educ Health Prof. 2016;36:104–112. [DOI] [PubMed] [Google Scholar]
- 22. Carney PA, Crites GE, Miller KH, et al. Building and executing a research agenda toward conducting implementation science in medical education. Med Educ Online. 2016;21:32405. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. McGaghie WC, Barsuk JH, Cohen ER, Kristopaitis T, Wayne DB. Dissemination of an innovative mastery learning curriculum grounded in implementation science principles: a case study. Acad Med. 2015;90:1487–1494. [DOI] [PubMed] [Google Scholar]
- 24. Katzman JG, Galloway K, Olivas C, et al. Expanding health care access through education: dissemination and implementation of the ECHO model. Milit Med. 2016;181:227–235. [DOI] [PubMed] [Google Scholar]
- 25. Mendel P, Meredith LS, Schoenbaum M, Sherbourne CD, Wells KB. Interventions in organizational and community context: a framework for building evidence on dissemination and implementation in health services research. Adm Policy Ment Health. 2008;35:21–37. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Shamseer L, Weeks L, Turner L, Straus S, Grimshaw J, Moher D. Identifying barriers to uptake and implementation of the CONSORT statement. Paper presented at: The Seventh International Congress on Peer Review and Biomedical Publication; September 8-10, 2014; Chicago, IL. [Google Scholar]
- 27. Brownson RC, Diem G, Grabauskas V, et al. Training practitioners in evidence-based chronic disease prevention for global health. Promot Educ. 2007;14:159–163. [PubMed] [Google Scholar]
- 28. Waserman JE, Hategan A, Saperson K, McConnell M, Harms S. Scholarship in residency: invoking change in the era of competency-based medical education. J Grad Med Educ. 2015;7:493–494. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Santen SA, Deiorio NM, Gruppen LD. Medical education research in the context of translational science. Acad Emer Med. 2012;19:1323–1327. [DOI] [PubMed] [Google Scholar]
- 30. Thomas DC, Berry A, Djuricich AM, et al. What is implementation science and what forces are driving a change in medical education? Am J Med Qual. 2016;32:438–444. [DOI] [PubMed] [Google Scholar]
- 31. Price DW, Wagner DP, Krane NK, et al. What are the implications of implementation science for medical education? [published online ahead of print April 23, 2015]. Med Educ Online. doi: 10.3402/meo.v20.27003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. McGaghie WC. Medical education research as translational science. Sci Trans Med. 2010;2:19cm8. [DOI] [PubMed] [Google Scholar]
- 33. Kho ME, Estey EA, DeForge RT, Mak L, Bell BL. Riding the knowledge translation roundabout: lessons learned from the Canadian Institutes of Health Research Summer Institute in knowledge translation. Implement Sci. 2009;4:33. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34. Straus S.E., et al. , Core competencies in the science and practice of knowledge translation: description of a Canadian strategic training initiative. Implementation Science, 2011. 6(1): p. 127. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35. Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Informat Libraries J. 2009;26:91–108. [DOI] [PubMed] [Google Scholar]
- 36. Methods Guide for Effectiveness and Comparative Effectiveness Reviews (AHRQ Publication No. 10(14)-EHC063-EF). Rockville, MD: Agency for Healthcare Research and Quality; 2014. http://www.effectivehealthcare.ahrq.gov. [PubMed] [Google Scholar]
- 37. Eva KW. On the limits of systematicity. Med Educ. 2008;42:852–853. [DOI] [PubMed] [Google Scholar]
- 38. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M; Medical Research Council Guidance. Developing and evaluating complex interventions: the new medical research council guidance. BMJ. 2008;337:a1655. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39. McGaghie WC, Issenberg SB, Barsuk JH, Wayne DB. A critical review of simulation-based mastery learning with translational outcomes. Med Educ. 2014;48:375–385. [DOI] [PubMed] [Google Scholar]
- 40. Bender BG, Krishnan JA, Chambers DA, et al. American thoracic society and national heart, lung, and blood institute implementation research workshop report. Ann Am Thorac Soc 2015;12(12):S213–S221. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41. Chambers DA, Proctor EK, Brownson RC, Straus SE. Mapping training needs for dissemination and implementation research: lessons from a synthesis of existing D&I research training programs. Trans Behavior Med. 2016;7:593–601. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42. Gonzales R, Handley MA, Ackerman S, et al. framework for training health professionals in implementation and dissemination science. Acad Med 2012;87(3):271–278. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43. Meissner HI, Glasgow RE, Vinson CA, et al. The U.S. training institute for dissemination and implementation research in health. Implement Sci. 2013;8:12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44. Morrato EH, Rabin B, Proctor J, et al. Bringing it home: expanding the local reach of dissemination and implementation training via a university-based workshop. Implement Sci 2015;10:1–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45. Norton WE. Advancing the science and practice of dissemination and implementation in health: a novel course for public health students and academic researchers. Public Health Rep (Washington, DC: 1974). 2013;129(6):536–542. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46. Padek M, Colditz G, Dobbins M, et al. Developing educational competencies for dissemination and implementation research training programs: an exploratory analysis using card sorts. Implement Sci. 2015;10:114. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47. Proctor EK, Landsverk J, Baumann AA, et al. The implementation research institute: training mental health implementation researchers in the United States. Implement Sci. 2013;8:105. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48. Proctor EK, Chambers DA. Training in dissemination and implementation research: a field-wide perspective. Trans Behavior Med. 2016:1–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49. Stamatakis KA, Norton WE, Stirman SW, Melvin C, Brownson RC. Developing the next generation of dissemination and implementation researchers: insights from initial trainees. Implement Sci. 2013;8:29. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50. Luke DA, Baumann AA, Carothers BJ, Landsverk J, Proctor EK. Forging a link between mentoring and collaboration: a new training model for implementation science. Implement Sci. 2016;11:137. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51. Proctor E, Carpenter C, Brown CH, et al. Advancing the science of dissemination and implementation: three “6th NIH meetings” on training, measures, and methods. Implement Sci. 2015;10(1):A13. [Google Scholar]
- 52. Green S, Higgins JPT, Alderson P, Clarke M, Mulrow CD, Oxman AD. What is a systematic review? In: Higgins JPT, Green S. eds. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0. Updated March, 2011. Available at: www.cochrane-handbook.org.
- 53. Sklar DP. What happens after medical school? current controversies about licensure, maintenance of certification, and continuing professional development. Acad Med. 2016;91:1–3. [DOI] [PubMed] [Google Scholar]
- 54. Hafferty FW, Franks R. The hidden curriculum, ethics teaching, and the structure of medical education. Acad Med. 1994;69:861–871. [DOI] [PubMed] [Google Scholar]
- 55. Bland CJ, Starnaman S, Wersal L, Moorhead-Rosenberg L, Zonia S, Henry R. Curricular change in medical schools: how to succeed. Acad Med. 2000;75:575–594. [DOI] [PubMed] [Google Scholar]
- 56. Jones R, Higgs R, de Angelis C, Prideaux D. Changing face of medical curricula. Lancet. 2001;357:699–703. [DOI] [PubMed] [Google Scholar]
- 57. Cervero RM. Changing continuing medical education. JAMA. 2015;314:1072–1072. [DOI] [PubMed] [Google Scholar]
- 58. Nissen SE. Reforming the continuing medical education system. JAMA. 2015;313:1813–1814. [DOI] [PubMed] [Google Scholar]
- 59. Colditz GA, Emmons KM, Vishwanath K, Kerner JF. Translating science to practice: community and academic perspectives. J Public Health Manag Prac. 2008;14:144–149. [DOI] [PubMed] [Google Scholar]
- 60. Tabak RG, Khoong EC, Chambers DA, Brownson RC. Bridging research and practice: models for dissemination and implementation research. Am J Prev Med. 2012;43:337–350. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61. Rogers EM. Diffusion of Innovations. New York, NY: Simon & Schuster; 2010. [Google Scholar]
- 62. Epstein RM. Assessment in medical education. N Engl J Med. 2007;356:387–396. [DOI] [PubMed] [Google Scholar]
- 63. Steinert Y, Mann K, Anderson B, et al. A systematic review of faculty development initiatives designed to enhance teaching effectiveness: a 10-year update: BEME guide no. 40. Med Teach. 2016;38:769–786. [DOI] [PubMed] [Google Scholar]
- 64. Jippes M, Driessen EW, Broers NJ, Majoor GD, Gijselaers WH, van der Vleuten CP. A medical school’s organizational readiness for curriculum change (MORC): development and validation of a questionnaire. Acad Med. 2013;88:1346–1356. [DOI] [PubMed] [Google Scholar]
- 65. Hsih KW, Iscoe MS, Lupton JR, et al. The student curriculum review team: how we catalyze curricular changes through a student-centered approach. Med Teach. 2015;37:1008–1012. [DOI] [PubMed] [Google Scholar]
- 66. Gagliardi AR, Perrier L, Webster F, et al. Exploring mentorship as a strategy to build capacity for knowledge translation research and practice: protocol for a qualitative study. Implement Sci. 2009;4:55. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 67. Gagliardi AR, Webster F, Straus SE. Designing a knowledge translation mentorship program to support the implementation of evidence-based innovations. BMC Health Serv Res. 2015;15:1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 68. Education ACfGM. Common program requirements. 2016. https://www.acgme.org/Portals/0/PFAssets/ReviewandComment/CPR_SectionVI_ImpactStatement.pdf. Accessed June 4, 2016.
- 69. Nasca TJ, Weiss KB, Bagian JP. Improving clinical learning environments for tomorrow’s physicians. N Engl J Med. 2014;370:991–993. [DOI] [PubMed] [Google Scholar]
- 70. Goldstein MJ, Lunn MR, Peng L. What makes a top research medical school? a call for a new model to evaluate academic physicians and medical school performance. Acad Med. 2015;90:603–608. [DOI] [PubMed] [Google Scholar]
- 71. Ellner AL, Stout S, Sullivan EE, Griffiths EP, Mountjoy A, Phillips RS. Health systems innovation at academic health centers: leading in a new era of health care delivery. Acad Med. 2015;90:872–880. [DOI] [PubMed] [Google Scholar]
- 72. Bridges S, Yiu CKY, Botelho MG. Design Considerations for an Integrated, Problem-Based Curriculum. Med Sci Edu. 2016;26(3):365–373. [Google Scholar]
- 73. Aisner D, Olsen L, McGinnis JM. The Learning Healthcare System: Workshop Summary (IOM Roundtable on Evidence-Based Medicine). Washington, DC: National Academies Press; 2007. [PubMed] [Google Scholar]