Mapping training needs for dissemination and implementation research: lessons from a synthesis of existing D&I research training programs

Abstract

With recent growth in the field of dissemination and implementation (D&I) research, multiple training programs have been developed to build capacity, including summer training institutes, graduate courses, degree programs, workshops, and conferences. While opportunities for D&I research training have expanded, course organizers acknowledge that available slots are insufficient to meet demand within the scientific and practitioner community. In addition, individual programs have struggled to best fit various needs of trainees, sometimes splitting coursework between specific D&I content and more introductory grant writing material. This article, stemming from a 2013 NIH workshop, reviews experiences across multiple training programs to align training needs, career stage and role, and availability of programs. We briefly review D&I needs and opportunities by career stage and role, discuss variations among existing training programs in format, mentoring relationships, and other characteristics, identify challenges of mapping needs of trainees to programs, and present recommendations for future D&I research training.

INTRODUCTION

The gap between scientific discovery and practice has long been a limitation to fully realizing the benefit of biomedical research on health care and population health [1]. In the USA, study of this area has been commonly referred to as dissemination and implementation (D&I) research [2] or implementation science [3]. Other countries use other terms but in this paper, we use the term “D&I research” for consistency, except in referencing specific programs that incorporate alternative terms.

The past decade has seen multiple efforts to build capacity, advance theoretical frameworks, methods and measures, and conduct a range of observational and experimental studies to identify effective strategies for promoting adoption, implementation, and sustainability of evidence-based practices. Various funding agencies like the National Institutes of Health (NIH), the Agency for Healthcare Research and Quality (AHRQ), and the Veterans Administration (VA) have called for D&I research through funding opportunities. Foundations have supported initiatives in this area, and support internationally for funding implementation research has been a priority in the UK (MRC and NIHR), Canada (CIHR), Australia (MRC), and New Zealand (HRC) amongst others.

From 2007 to 2012, the NIH, along with the VA, supported five large meetings on the science of D&I. Following the 2012 meeting, an NIH working group organized smaller working meetings to generate recommendations for future D&I research in three key areas—training, measurement, and standardized reporting, and research design. The training meeting, which the first two authors organized, recognized the growth of individual D&I research training programs and the need for a cohesive plan for expanding future training capacity.

Training opportunities for D&I research have recently emerged, including immersive training institutes, 1-day workshops, academic graduate programs, individual academic courses, webinar series, and career development awards (Table 1). Popular among these have been the NIH/VA-organized Training Institute for Dissemination and Implementation Research in Health (TIDIRH) [4] the Washington University-St. Louis-hosted Implementation Research Institute [5] and Mentored Training for Dissemination and Implementation Research in Cancer [6], the KT Canada Summer Training Institute [7], Prevention and Control of Cancer Post-Doctoral Training in Implementation Science (PRACCTIS) [8], and multiple technical assistance workshops held at the annual NIH/VA D&I annual research conferences [9]. In planning the training meeting in September 2013, we became aware of an additional set of programs. This paper compiles information shared by representatives of these training programs and those who have participated in other types of NIH research training programs (e.g. institutional training grants, career development awards). The set of programs discussed and participants invited to the meeting were identified through web and literature searches, snowball sampling using faculty of the above programs and review of the Implementation Network listserv content, which publicizes D&I research training opportunities to over 2000 subscribers every month [10].

Table 1.

A survey of selected D&I research training courses, as discussed at the 2013 NIH workshop

Course title	Length of course; frequency held	Target audience	Number of slots per cohort	T Theory/frameworks	O Research designs	P Measurement	I Comparative effectiveness research	C Economics/cost	S Reviews of individual study ideas	What is D&I research	Comments
TIDIRH	1 week; annually	All researchers new to D&I, any content area	40–45	YES; overview	YES; overview	YES; measurement and evaluation lecture and exercises	YES	YES	YES, small group discussions daily	YES, as part of a pre-training webinar format
IRI	1 week for 2 years; annually	Early-to-mid career researchers new to D&I, mental health focus	10–12	YES	YES	YES	Yes in relation to D&I research	Yes	YES, one-on-one mentoring and small-group discussions	Yes as brief overview
MT-DIRC	1 week for 2 years; annually	Early-to-mid career researchers new to D&I, cancer focus	14	YES	YES	YES	No	YES	YES, one-on-one mentoring and small-group discussions	YES
KT Canada Summer training Institute	3 days; annually	Early career researchers and graduate students/fellows focused on D&I	36	YES	Yes	Yes	Yes	Yes	Yes, one-on-one mentoring and small group discussions	YES
UW PhD	Multi-year program, generally 3–5 years.	Graduate students	3–5	YES; didactic and dissertation phases	YES; heavy focus throughout didactic and dissertation phases	YES; ~70% overlap in coursework with metrics area of emphasis	YES	YES; at a minimum covered through coursework	YES; mentoring and cohort review of disseration projects	YES	Total program time is generally 3–5 years.
UCSF	Master’s Track in Implementation Science (ImS) within a 2 year masters program in clinical research Certificate Program; Two-Week Summer ImS Institutes (new) at UCSF and previously in Croatia and Uganda	Early-to-mid career researchers new to D&I as well as graduate and medical students; early transitioning faculty meeting NIH disparities criteria	No limits for masters or certificate; 10-12/cohort for summer institute programs	YES	YES	YES	YES	YES	YES, masters committee and WIP options; 1:1 and team mentors for summer ImS institutes	YES	We also have developed competencies for the masters students and are working on those for the other formats. We have one on-line class and are working on more
UICC	8 h webinars; once	Researchers new to D&I
NCI IS advanced topics webinar	10 hour-long webinars per year	D&I researchers of all levels/ experiences	No limit on participation; webinars are archived for future retrieval.	YES; webinars feature individual frameworks
KT Canada Seminar series	12 one-hour-long webinars per year	D&I researchers of all levels/ experiences	Unlimited – an average of about 70 participants	YES	YES	YES	Yes	Yes	YES	Yes
KT Canada Twitter journal club	6 one-hour-long webinars per year		Unlimited – an average of about 25 participants	YES	Yes	Yes	Yes	Yes	n/r	Yes
UW IS for family planning, reproductive health & HIV/STI prevention and control	10 days (80 h); annually	Researchers, policymakers, practitioners from low and middle-income countries	30	Yes	Yes; emphasis of short course is development of a study protocol	Yes; overview	Yes	Yes; overview	Yes; mentoring and cohort review of developed protocols	Yes	2nd year of short course in 2015, with the USAID-supported EVIDENCE project

The meeting co-chairs, themselves core faculty of multiple D&I research training courses, invited 14 individuals primarily involved in D&I research training, as faculty, course directors and trainees, identified through the searches referred to above. In addition, 6 participants also were selected to represent other types of training programs (e.g. Institutional training awards, individual career development awards, post-doctoral fellowships, dissertation grants). Finally, participants also included leadership from the Veterans Administrations’ Quality Enhancement Research Initiative (QUERI), which has long provided training, research infrastructure, and study funding to D&I scientists within VA health systems. The participants engaged in preliminary work (summarized in Tables 1 and 2) to share information about the programs they were involved in, and the discussions informed the organization of this paper and the identification of both challenges and recommendations (Table 3).

Table 2.

D&I research training needs across key content areas by stage of researcher and other key stakeholders

	T Theory/frameworks	O Research designs	P Measurement	I Comparative effectiveness research	C Economics/cost	S Reviews of individual study ideas	What is D&I research
Graduate student	YES	YES	YES	YES	YES	YES	YES
Post-doctorate trainee	YES	YES	YES	YES	YES	YES	YES
Junior investigator	YES	YES	YES	YES	YES	YES	YES
Mid Career New to D&I/Within D&I	YES/focus on novel Frameworks	YES/focus on novel designs	YES/focus on new measures	Only if CER is specific to D&I	YES/focus on costs of implementation	YES/YES	YES/NO
Senior researcher New to D&I/Within D&I	YES/focus on novel frameworks	YES/focus on novel designs	YES/focus on new measures	Only if CER is specific to D&I	YES/focus on costs of implementation	YES/YES	YES/NO
Implementation practitioner	YES	NO	YES, for evaluation	NO	YES, for evaluation	NO	If new to role
Service provider	YES, targeted to specific context	NO	YES, for evaluation	NO	YES/for evaluation	NO	YES
Policymaker	YES, if targeted to specific context	NO	YES, for evaluation	NO	YES/for evaluation	NO	YES

Table 3.

Challenges and recommendation to improve the fit of D&I training programs to needs

Challenges of mapping training needs to programs	Recommendations
Balancing between core components and specialization	1. Tracks within programs could allow for specialization for health topic, context 2. Specialists could adapt core materials for given audience
Rapid pace of field development	1. Clearinghouse could host new materials as created. 2. Online virtual community could permit wiki-like updating of course materials
Follow-up/Sustainability of training outcomes	1. Linkage of course to ongoing activities (e.g. NCI Advanced Webinars; CIPRS/VA webinars) 2. Training “Graduate” Forums at large D&I conferences 3. Online networking via linkedin/facebook/researchgate
Increasing supply to meet demand	1. Train the trainer model could foster satellite trainings at “graduate” institutions 2. MOOCs/Online courses could expand reach, increase efficiency of training
Sequencing of training vs. complementarity	1. Training programs should specify inclusion/exclusion criteria in relation to one another 2. A field-wide strategy should articulate the complementarity of different training programs (see Proctor et al. paper, in preparation)

The format of the meeting focused on three key questions: What is a field-based vision for training investigators at multiple career stages to gain expertise in D&I research; what tools and resources could we pull together to support this vision; and what gaps need to be filled to better train the range of future D&I Scientists? The agenda included presentations of existing D&I research training programs, discussion of other training mechanisms that could be used for D&I research training, and then focused on challenges and resources to support current and future D&I research training programs.

Due to time and space constraints, the meeting was not able to ensure that representation across all D&I research training programs was saturated, but the selection of participants did cover both the publications of D&I research training programs (as of 2013) and the programs and training activities identified through the search. This paper builds from the meeting discussions and preliminary work to map D&I research training efforts to the heterogeneity of current and future trainees.

To categorize the different training opportunities currently offered for D&I research, we used a conceptual frame (Fig. 1) that identified the target audience seeking training (Who?), the degree of intensity of the training (How?) and the desired goal for which D&I training would be used (Why?). Mapping the needs of trainees to the available programs requires attention to these three dimensions as well as a fourth—the content of the training to be provided within each program. The organization of the paper further elaborates on this conceptual frame by first focusing on the different career stages of the trainees and their needs (the Who), drawing lessons across the programs on the differing formats and goals (the How and Why), and concluding with recommendations that address all three.

Fig. 1.

Multiple axes identifying who attends D&I training programs, how the programs are formatted, and why the training is desired

One challenge raised in the workshop is to ensure that training in D&I research is both foundational and responsive to the advances in the field. In some areas of science, a core curriculum is widely agreed upon. As D&I research has matured, training programs have identified common elements of coursework (Table 1). Existing programs have balanced that with the leading edge of D&I research. For example, the National Cancer Institute (NCI) has established an Advanced Topics in Implementation Science Webinar [11] to present newer research topics such as improving transparency and reporting for D&I. In addition, some topics have emerged from policy makers and funders’ identification of key issues [12]. From these activities, de-implementation and the science of stakeholder engagement have risen as priorities for D&I research training.

MULTIPLE CAREER STAGES FOR D&I RESEARCH

Course faculty has found heterogeneity in the backgrounds and career stages of those interested in receiving training. In TIDIRH and IRI, for example, trainees have ranged from post-doctoral level to full professor, and fields of study have included a range of biomedical, behavioral and social sciences. D&I training programs are challenged to meet the needs of all their attendees. While there are benefits to the heterogeneity of a training cohort, programs risk providing content that trainees are already familiar with or starting beyond the expertise of some of the trainees. Irrespective of career stage, different trainees have varied experience with the D&I research contexts (e.g., health systems, community settings, policy environments). In Table 2, we map trainee needs by level of expertise, and below we briefly spotlight existing programs and training opportunities that may serve as an exemplar to provide resources to meet those needs.

Early career (graduate students to junior investigators)

Increasingly, graduate students are identifying D&I research as a career track. Training for this audience must supplement D&I content with more fundamental coursework and training experience, including research methods, grantwriting, project management, and pilot investigations. Several academic centers now offer D&I courses [13], and the University of Washington and several universities in Canada have PhD tracks for D&I research either within an existing PhD program (e.g. University of Washington’s Global Health program (http://globalhealth.washington.edu/education-training/phd-gh)) or as a standalone program (e.g. knowledge translation PhDs in Canada).

The post-doctoral and junior investigator stage offers a similar chance to gain valuable experience in D&I research as part of a project team or teams conducting studies within the field. The major emphasis of training at this stage is in achieving a base level of understanding of theories, methods and key questions, and beginning independent pilot studies. Increasingly, post-doctoral and junior investigators are a significant proportion of the participating group of summer training institute trainees [4, 5], and many have transitioned to principal investigator on D&I studies [14]. Junior investigators in D&I also use the career development award for both training and pilot research activities. The coursework includes training in research design and measurement, theories, and a multi-level understanding of the contexts in which D&I studies will be set.

Experienced investigators (mid-level to senior researcher)

For years, the prototypical D&I researcher began her research career in a clinical or practice research field [15], achieved independence, and recognized that research impact on practice was less successful than expected. Many training programs (TIDIRH, IRI, MT-DIRC, VA) support transition to a D&I research career, offering D&I theory, methods, and design content, and focusing explicitly on the trainee’s research project. Seminar series and workshops at annual D&I meetings provide further opportunities for experienced investigators to get targeted training and technical assistance. In addition, D&I workshops and pre-conference sessions at a number of national meetings (e.g., American Thoracic Society, American College of Epidemiology) tie an investigator’s research area to D&I.

The experienced D&I researcher can also benefit from new training as the field matures. A D&I research grantee from 10 years ago may not be well positioned given the growth of the field. Webinars, targeted pre-conference methods workshops, and online communities of practice [16] can provide additional training and resources to this cohort. The NCI Advanced Topics in Implementation Science Webinar series [11], for example, enables emergent content areas to be presented to all comers, and all webinars are archived and posted on the NCI website.

D&I RESEARCH TRAINING FOR OTHER KEY PARTICIPANTS

D&I research training may go beyond the investigator to those in the trenches. The implementation practitioner’s exposure to D&I research may increase the success of implementation efforts. Training for this stakeholder group centers on the evidence base for implementation strategies, understanding of context, and tools to evaluate impact of both the implementation strategy and clinical practice being implemented. One example of practitioner training is the NCI-sponsored Research to Reality [17] webinar series, connecting researchers and practitioners around cancer control topics. In addition, several academic institutions have recently introduced courses to train implementation practitioners. Washington University in St. Louis, for example, introduced a three credit course in Spring semester 2015.

Service providers also benefit from understanding D&I research. Many states and health care organizations, for example, have mandated delivery of evidence-based practices and the traditional method of continuing medical education [18] is ineffective in supporting implementation. D&I research training can help providers to identify barriers and facilitators to change within local settings, identify evidence-based implementation strategies, and understand how to adapt practices to fit the context. Recently, the Centers for Disease Control and Prevention (CDC) and NCI-supported Cancer Prevention and Control Research Network posted an online training resource for putting public health evidence into action [19], which walks providers through D&I processes.

Finally, policymakers are frequently in the position of making decisions about using health-related evidence in both “Big P” (e.g., a state legislature) and “small p” (e.g., a private business) policy settings. Implementation of many practices and programs may require trade-offs between the research and policy processes [20]. Policymakers may benefit from D&I research training to understand potential impacts and trade-offs of decision options, and in assessing the outcome of a policy initiative.

It is worth noting that in many cases, the key participants discussed above may blend their “practitioner” roles with that of scholarship. The “scientist-practitioner” hybrid role may be particularly advantageous in D&I research where understanding of the local context will enable generation of practice-relevant questions, and the ability to immediately apply lessons from science can have a significant impact on ongoing practice improvement.

D&I RESEARCH TRAINING FOR TEAMS

The “Science of Team Science” has generated new interest in the design of optimal research teams for the high-priority research investigations of the future [21]. D&I research is a multi-disciplinary field, drawing from the health and behavioral sciences, among others. In addition, D&I research frequently requires partnerships with other stakeholders in order to produce the most meaningful studies and results [22].

Today, trans-disciplinary research teams have become the norm rather than the exception [21]. An evaluation of NIH-funded D&I research [23] shows that the majority of awarded projects are conducted by teams with multiple disciplines represented. D&I research training can be administered to teams. A recent Washington University-St. Louis “boot camp” to train D&I researchers required participation by at least two members of the research team. The day’s agenda included slots preserved for the team to meet and discuss the advice received from various research consultants.

Opportunities to train teams of researchers, practitioners, and policymakers are starting to become available, primarily in the international space. KT Canada, for example, supports researchers, policymakers, and practitioners to jointly learn about D&I research while advancing a collaborative project. Supporting the Promotion of Activated Research and Knowledge (SPARK) Training Program, a mental health training effort in Canada, similarly brought together blended teams that combined didactic sessions with team brainstorming on project development.

VARIATIONS WITHIN EXISTING TRAINING PROGRAMS

Training formats

Across the existing programs, a range of formats have been used, including immersive in-person courses of anywhere between 3 days and a week in length, in several cases (e.g. MT-DIRC, IRI) offered as a 2-year course. Individual university courses have been administered, and in several cases, curricula for an entire degree program or certificate. Recipients of NIH or VA career development awards in D&I research have been supported for didactic learning, mentorship, and pilot research studies. Some programs have taken advantage of long-distance technologies (e.g. Advanced Topics Implementation Science (IS) Webinars, Union for International Cancer Control (UICC)) and sessions have been archived and made available for asynchronous training (e.g. TIDIRH). While the benefits of online training permit greater accessibility regardless of location, these programs may not provide the depth of interactions of immersive training programs. Further exploration of the costs and benefits of specific technologies for D&I training may be warranted. In addition, D&I research findings have generally shown the benefit of interactive, pragmatic styles of learning [24]. Training programs should move away from primarily didactic styles in favor of discussion formats and problem-solving or practical activities in courses.

Mentoring/coaching relationships

The connection between mentors and trainees is central to many of the D&I training programs. Multiple programs (e.g., IRI, MT-DIRC) assign a mentor who works with the trainee over the entire course. TIDIRH, in contrast, assigns a number of trainees to specific mentors for the duration of the 1-week course, but that relationship rarely endures beyond the course. For early-stage investigators longer-term methods of mentorship may be required. Technologies can enable longer term mentorship to create a path to research independence. Some [25] have drawn distinctions between the mentor, whose role may include a more holistic approach to support of career development, and a coach who would focus on the content of D&I research. The latter may be useful for trainees at more senior career levels, where D&I content is the clear need, while the mentor may be most useful for people requiring broader support.

CHALLENGES OF MAPPING NEEDS OF TRAINEES

D&I research is broad in its orientation, and requires expertise across different domains (theory, context, clinical topic, methods, measurement, analysis). It is thus challenging to develop a program that effectively conveys a bolus of information while still meeting the needs of individuals with very different starting points and trajectories. Table 3 summarizes a few specific challenges felt by the organizers of existing D&I training programs, several of which are described below.

Balance between core competency and specialization

One major challenge has been whether training should be open to D&I investigators studying a range of health-related topics and settings or whether trainees should be more homogeneous in their interests. TIDIRH and KT Canada, for example, are broad in their acceptance of health content areas, provided the D&I research focus. Other programs, like IRI and MT-DIRC, specify a particular health area and concentrate on developing cohorts of investigators who are specifically oriented toward one set of diseases. For the former, the challenge becomes how much one is able to dig deep into the particularities of the given settings and health issues while maintaining a one-size-fits-all approach; for the latter, the challenge may be in how one addresses trainees whose interests lie on the margin of the health focus (e.g., comorbidities, cross-cutting topics, context-specific research).

Rapid pace of field development

Another challenge experienced across the multiple programs is keeping up with the rapid pace of change in the field. A number of the programs were developed with some core resources available in D&I research, frequently some key papers, a funding announcement, and existing presentations that were pulled together into a curriculum. Since many of these programs began, the amount of work in the field has grown, and new methods and measures are making their way into use. Determining how frequently to refresh the material and methods of training is a challenge, particularly where faculty have other responsibilities that limit time for revising the course.

Follow-up/sustainability of training

Several of the training courses have made explicit the need to follow-up with trainees following the course. IRI and MT-DIRC have used national D&I research meetings as times to bring their alumni together, and the nature of the 2-year course, with a D&I research site visit in between the 2 years, allows for more sustained connection with the program. KT Canada has supported an online community for past and current trainees to support ongoing learning, and NCI’s Advanced Topics IS Webinar series is intended to follow up on the week-long training to delve deeper into D&I research topics. All programs, though, may be limited in their ability to follow-up and customize additional training to the emerging needs of their participants.

Limitations in local capacity

One major challenge in meeting the needs of D&I research trainees is limited support and local capacity to engage in D&I work. The applications for a number of programs show variability in how local institutions can actively support trainees and provide resources to further D&I work. Low-resource settings in the USA and abroad may have disadvantages in D&I support, and even where resources may be plentiful, there may be limited support and capacity in D&I research. Virtual learning collaboratives, like NCI’s Research to Reality may help to alleviate this challenge, by bringing trainees and mentors together, but limited local resources remain an impediment to supporting D&I research training.

Sequencing of training vs. complementarity

The training needs of the D&I research community are variable, as discussed in the previous section, and as a field, we have been trying to field a set of training programs and resources that map onto this variability. Indeed, inherent in several training programs (e.g., IRI, TIDIRH, MT-DIRC) is the notion that not only are trainees learning from these programs, the faculty are as well. It is thus a challenge to figure out whether there is a particular sequence through which a D&I research trainee should follow. Should an early stage investigator seek a mentored training institute before or after undertaking a career development award? Should a mid-level investigator focus more on targeted D&I research topics rather than embark upon a more formalized approach? Does a senior-level investigator need a more immersive D&I experience or can self-directed learning be sufficient to create expertise? Multiple programs have struggled with identification of the “sweet spot” of trainee development that is most appropriate for a specific program. The field as a whole should consider to what degree training programs may lead one to another, and whether more specialized training that caters to basic, intermediate, or advanced grounding in D&I research (and research as a whole) is needed.

RECOMMENDATIONS FOR THE D&I RESEARCH TRAINEE OF THE FUTURE

The experiences of current and past training programs have suggested some clear opportunities to leverage the work within the field. The following recommendations came both from the discussions and materials generated at the September 2013 meeting (summarized in the tables), and from the experiences of the authors in leading many of the D&I research programs included in this article.

Creating a clearinghouse of materials

As a plethora of materials have been developed and used in specific courses, the D&I research community may benefit from those materials being made available centrally via a clearinghouse of materials. Capturing the available slides, worksheets, papers, and exercises in a clearinghouse may lower the barrier to entry for new training efforts, and broaden the base for which local institutions and organizations can provide their own D&I training to all stakeholders. For example, the resources of the TIDIRH training program are available to anyone who is interested. In the 5 years of the program, all slide presentations have been posted online, and videos of many sessions exist. Each includes key readings and take home messages. The IRI and MT-DIRC courses have included a “resource of the day,” measurement tool, web portal, information about an upcoming conference, or a publication relevant to trainees and faculty, but no single site has pulled everything together. Given that many more prospective trainees exist than available training slots, a centralized resource of training materials could be crucial.

Leveraging ongoing initiatives for trainee investigations

IRI, among others, has incorporated D&I research site visits as a core part of their curricula. In career development awards, trainees sometimes are able to augment an ongoing investigation to conduct pilot D&I work. For example, intervention studies that primarily seek to determine the efficacy or effectiveness of a particular treatment or prevention program may provide an opportunity for a trainee to gather information on implementation outcomes. Other platforms (e.g., HMORN, PCORnet, others) may be able to house small studies that can give trainees in vivo D&I research experiences. Cataloguing these different platforms and identifying the types of trainee-led studies that might fit the platform could be of great benefit in catapulting investigators toward independence.

Creating networking opportunities across programs/field

There are currently limited ways in which trainees from across different programs can network. A number of large conferences (e.g., Annual D&I Science meetings, Society for Implementation Research Collaboration (SIRC), KT Canada, AcademyHealth Annual Research Meeting, Global Implementation Conference) may provide some networking opportunities, but there are few collaborative networks available for trainees to convene. Great value may be produced by launching a virtual collaborative space for newly trained D&I researchers. Such a platform could extend upon the value of existing webinars, workshops, and conferences to provide peer support, consultation, collaborative space, and learning opportunities.

Expanding opportunities/training in partnered research/team science

As previously stated, D&I research has clearly staked its claim as a team sport. As such, the field needs more training opportunities to support the development of research partnerships, both within the academic field, and across multiple stakeholder groups, including practitioners, policymakers, and patients. Current offerings for training research teams are promising, but team science training is the exception rather than the norm. The next generation of D&I training may need to include far more content on the development and functioning of research teams, skills in transdisciplinary research, the science of research-practice partnerships, and a range of models for supporting partnered research.

Refining what to teach

Competency-based education is rapidly becoming a norm in all levels of education in the USA, [26–28]. Formally, a competency is defined as a cluster of related knowledge, attitudes, and skills that affects the major part of one’s job and can be measured against well-accepted standards and improved through training [29]. Competency sets are used both to guide credentialing processes, and for this program, curriculum development [30, 31]. There are initial steps at developing and refining D&I research competencies [32]. As the field of D&I research grows, competencies will need to adapt to the field.

Supporting ongoing evaluation of D&I training initiatives

Many of the training programs referred to in this paper have undergone some type of evaluation (4,5,7, 13), which has helped to reinforce the impact of the training for the D&I community. However, given the ongoing evolution of the field, the importance of continual assessment and updating is essential to ensure the relevance and long-term contribution of training programs. The ability to assess the impact of particular topics, training format, technical assistance on specific study ideas, and other facets of many of the training programs should be incorporated into the ongoing delivery of the training and curriculum planning. Ideally, data from evaluation of any D&I program can be leveraged by other programs, and may helpfully be included in the type of training clearinghouse introduced above. While we reference articles that have presented evaluation data from some of the programs, an in-depth analysis of the quality of these programs was beyond the scope of this article, but would greatly aid planning of current and future D&I research programs.

CONCLUSION

This article has captured the experiences of multiple D&I research training programs as a way to reflect on the current offerings of training, identify challenges in meeting the needs of trainees, and offering recommendations of better alignment between D&I training needs of researchers and other key stakeholders and the available programs. While much progress has been made in a relatively short time period, the field still requires synergistic action to ensure that the field’s future capacity to perform the needed research studies is robust and of sufficient size. Consideration of the existing opportunities, variability in course design, remaining challenges, and potential future directions will put the D&I research community in a stronger position to train future scientists and push the field forward.

AHRQ Agency for Healthcare Research and Quality, CDC Centers for Disease Control and Prevention, CIHR Canadian Institutes of Health Research, CIPRS Center for Implementation Practice and Research Support, D&I Dissemination and implementation research, FDA U.S. Food and Drug Administration, HMORN HMO Research Network, HRC Health Research Council (New Zealand), IRI Implementation Research Institute, IS Implementation Science, KT Knowledge Translation, MRC Medical Research Council (UK, Australia), MT-DIRC Mentored Training for Dissemination and Implementation Research in Cancer, NCI National Cancer Institute, NIH National Institutes of Health, NIHR National Institute for Health Research (UK), PCORnet Patient Centered Outcomes Research network, SPARK Supporting the Promotion of Activated Research and Knowledge (SPARK) Training Program, SIRC Society for Implementation Research Collaboration, TIDIRH Training Institute for Dissemination and Implementation Research in Health, UCSF University of California, San Francisco, UICC Union for International Cancer Control, VA U.S. Department of Veterans Affairs