Advancing mechanisms of implementation to accelerate sustainable evidence-based practice integration: protocol for generating a research agenda

Abstract

Introduction

Mechanisms explain how implementation strategies work. Implementation research requires careful operationalisation and empirical study of the causal pathway(s) by which strategies effect change, and factors that may amplify or weaken their effects. Understanding mechanisms is critically important to replicate findings, learn from negative studies or adapt an implementation strategy developed in one setting to another. Without understanding implementation mechanisms, it is difficult to design strategies to produce expected effects across contexts, which may have disproportionate effects on settings in which priority populations receive care. This manuscript outlines the protocol for an Agency for Healthcare Research and Quality-funded initiative to: (1) establish priorities for an agenda to guide research on implementation mechanisms in health and public health, and (2) disseminate the agenda to research, policy and practice audiences.

Methods and analysis

A network of scientific experts will convene in ‘Deep Dive’ meetings across 3 years. A research agenda will be generated through analysis and synthesis of information from six sources: (1) systematic reviews, (2) network members’ approaches to studying mechanisms, (3) new proposals presented in implementation proposal feedback sessions, (4) working group sessions conducted in a leading implementation research training institute, (5) breakout sessions at the Society for Implementation Research Collaboration’s (SIRC) 2019 conference and (6) SIRC conference abstracts. Two members will extract mechanism-relevant text segments from each data source and a third member will generate statements as an input for concept mapping. Concept mapping will generate unique clusters of challenges, and the network will engage in a nominal group process to identify priorities for the research agenda.

Ethics and dissemination

This initiative will yield an actionable research agenda to guide research to identify and test mechanisms of change for implementation strategies. The agenda will be disseminated via multiple channels to solicit feedback and promote rigorous research on implementation mechanisms.

Strengths and limitations of this study.

• This study will synthesise multiple data sources to uncover key challenges to studying implementation mechanisms.

• This study will yield a research agenda outlining challenges, priorities and activities that will advance the study of implementation mechanisms.

• This study will disseminate a mechanisms-focused research agenda for implementation science and invite international feedback.

• The generation of this research agenda is largely informed by stakeholders from the USA, potentially limiting its relevance internationally; however, the network has been expanded to obtain global perspectives.

• Given the focus on advancing research methods, stakeholder engagement in this effort focuses primarily on researchers, limiting opportunities for patients and policy makers to inform the research agenda.

Mechanisms and what we know about them in implementation science

Mechanisms are broadly defined as processes that are responsible for change.¹ Defining, testing and establishing mechanisms is increasingly a priority across fields of study where biological, psychological or social intervention or behaviour change is the focus.^{2 3} In the context of implementation science, mechanisms explain how or why implementation strategies exert their effects on outcomes.⁴ Implementation strategies are defined as methods used to facilitate the adoption, implementation, sustainment or scale-up of evidence-based practices (EBPs).^{5 6} While over 70 implementation strategies have been identified and defined for use in healthcare settings,^{7 8} we know little about the mechanisms through which these strategies generate change.^{9 10} A recent systematic review of implementation mechanisms in health identified 46 studies assessing mechanisms.⁹ The majority of studies were not designed to formally establish implementation mechanisms per recommended criteria.^{1 11} In an earlier systematic review of nine randomised implementation trials, Williams¹⁰ observed similar methodological deficiencies, and no trials supported a hypothesised mediator. Both reviews identified numerous challenges impeding the study of mechanisms in implementation science, including conceptual (eg, lack of harmony in defining constructs and their roles), theoretical (ie, few theories exist), methodological (eg, poor quality measures, design challenges) and practical (eg, difficult to power studies for mediation analyses) challenges. More work is needed to identify barriers to identifying and testing mechanisms in implementation science and to propose concrete, actionable steps to overcome these barriers.

The impact of limited implementation mechanisms research on achieving health equity

Insufficient understanding of implementation mechanisms stymies the field. First, it hinders efforts to systematically design and tailor implementation strategies to meet the needs of heterogeneous contexts and stakeholders. Second, it limits our ability to learn from negative studies and to replicate positive findings. Third, it prevents the successful adaptation of an implementation strategy developed in one setting to another. These difficulties weaken implementation effectiveness and efficiency and can exacerbate health disparities when implementation efforts are poorly adapted for safety net and other settings where underserved populations are found.¹² Eliminating health disparities may require careful design and tailoring of implementation strategies to the unique circumstances and needs of specific priority populations and the contexts in which they receive care. For example, a clinic may begin to offer a new EBP with fidelity, but social factors (eg, access to transportation, childcare) might need to be addressed through provider-focused and patient-focused implementation strategies (eg, via telehealth) to improve ‘reach’.¹³ Contexts serving priority populations might not respond to certain implementation strategies, such as external consultation or be able to afford robust strategies, such as practice facilitation. A fundamental understanding of implementation mechanisms will enable implementers to design and, as necessary, tailor implementation strategies to locally relevant determinants (ie, factors that enable or obstruct changes in professional behaviours or healthcare delivery processes¹⁴) while retaining their core function.^{15 16}

Developing and disseminating a research agenda: conference series aims

Given ongoing challenges associated with delivering and tailoring implementation strategies previously found to be effective,^17–20 research reporting guidelines emphasise the need to justify the selection of specific implementation strategies.^{5 21 22} This requires identifying the barrier(s) to be addressed and the selected strategy’s mechanism of action. Table 1 provides examples of how strategies may work to address determinants and influence specific implementation outcomes. Identifying the mechanisms through which implementation strategies exert their effects, rather than focusing solely on whether or not they are effective, is an important step forward for the field of implementation science.^{6 23}

Table 1.

Examples of links between determinants, implementation strategies, mechanisms and implementation outcomes

Determinant	Implementation strategy	Mechanism	Implementation outcome
Provider knowledge deficit	Education (provision of information)	Awareness-building, knowledge-acquisition	Feasibility, acceptability, appropriateness, adoption
Provider skill deficit	Training (teaching and practice with corrective feedback)	Skill acquisition, refinement, mastery	Fidelity to EBP
Provider views EBP unfavourably	Audit and feedback provision of descriptive social norms indicating peer use of EBP	Social pressure/norms	Adoption
Turnover	Train-the-trainer	Real-time training and consultation	Sustainability
Competing clinical demands	Leadership training	Growing leadership support/perseverance	Adoption, sustainability

EBP, evidence-based practices.

Accordingly, we are undertaking an initiative to advance the study of mechanisms of implementation strategies in healthcare and public health. We will conduct a 3-year meeting series with between-meeting activities designed to achieve two aims: (1) establish priorities to guide a research agenda on implementation mechanisms, and (2) actively disseminate the research agenda to research, policy and practice audiences. This protocol outlines our plans to engage a network of experts to produce detailed guidance and identify research approaches and tools needed to study mechanisms of implementation strategies and whether they vary with respect to target problems, EBPs, priority populations and contexts. Ultimately, the research and practice efforts guided by this agenda may be more actionable, practical and equitable given their clear focus on mechanisms.

Methods

A variety of methods will be used to identify challenges, priorities and activities related to the study of implementation mechanisms, drawing on data from numerous sources. The methods include qualitative content analysis,²⁴ which will generate the inputs for concept mapping,²⁵ followed by nominal group technique²⁶ to prioritise the research agenda. Table 2 provides an overview of the steps of developing the research agenda, which are described in more detail below.

Table 2.

Overview of steps in developing the research agenda

Step	Focus	Brief description
1	Partnership	Develop strategic partnership with SIRC
2	Network recruitment	Recruit members of the Mechanisms Network of Expertise (MNoE)
3	Identifying challenges	Identify challenges and opportunities related to the study of mechanisms in implementation science across six different data sources
4	Synthesising and prioritising challenges via content analysis and concept mapping	Engage MNoE members to generate conceptually distinct clusters of challenges related to studying mechanisms, and generate ratings of the criticality and pervasiveness of challenges (at both the item and cluster level)
5	Identifying research priorities via nominal group technique	Employ the nominal group technique with MNoE members to identify priorities and specific activities for advancing research on implementation mechanisms
6	Dissemination	Disseminate the research agenda via the SIRC website, conference presentations, open access publications and briefs and videos
7	Discovery and refinement	Refine the agenda over time and encourage original research to advance understanding of how and why implementation strategies work

SIRC, Society for Implementation Research Collaboration.

Step 1: Partnership with the Society for Implementation Research Collaboration (SIRC)

In 2017, SIRC hosted its fourth biennial conference with the following theme: ‘Implementation Mechanisms: What Makes Implementation Work and Why?’.²⁷ This conference was inspired by the realisation that the field of implementation science was in a position to pivot from characterising implementation efforts (eg, qualitative exploration of barriers and facilitators) and evaluating the general effectiveness of strategies (ie, empirical treatment approach) to examining the causal mechanisms by which strategies achieve their effects. Given SIRC’s commitment to this work and its engaged global membership,²⁸ we partnered with them to pursue funding from the Agency for Healthcare Research and Quality (AHRQ) to shape the developing agenda. In addition to leveraging several SIRC activities as inputs for our research agenda, we will conclude our meeting series in collaboration with SIRC in 2022, where we will disseminate the agenda and invite feedback from attendees. A summary of our proposed work can be found on the SIRC website.²⁹

Step 2: Mechanisms Network of Expertise

In preparing this proposal, we convened 26 implementation scientists from the USA to form a Mechanisms Network of Expertise (MNoE). We have expanded the MNoE to over 40 individuals to include international experts and early career researchers, and plan to continue to do so as this work progresses. This network approach allows us to leverage the collective wisdom of key stakeholders who have been contributing to the study of implementation mechanisms and to engage in a collaborative immersion across 3 years to achieve our aims. By convening the MNoE regularly, roughly quarterly in virtual meetings and annually for Deep Dive retreats, we will be able to develop a shared understanding, build momentum and generate products for the larger field to apply and refine. We intend for the MNoE to represent expertise in studying implementation mechanisms and to display diversity across several dimensions (eg, geography, gender, race/ethnicity, work with priority populations, settings in which they worked). In the initial phases, the MNoE will be organised into four workgroups that will focus initial network activities: (1) measurement; (2) design and analysis; (3) causal theory and context and (4) strategy, mechanism and outcome linkages.²⁹ However, we plan to reorganise the workgroups into the conceptually distinct areas of the research agenda that emerge from the concept mapping work described below.

Patient and public involvement

This effort is primarily focused on challenges and opportunities related to research methods; thus, the primary stakeholder involvement in this effort is from healthcare researchers and there is no explicit patient or public involvement. Many members of the MNoE have experience not only as researchers, but as clinicians and patients. Moreover, there will be opportunities for individuals with a diverse array of experiences to provide input to the research agenda (eg, the SIRC breakout sessions) and respond to initial iterations (eg, see steps 6 and 7 below).

Step 3: Identifying challenges across six data sources

In collaboration with the MNoE, we will synthesise information from six sources to inform the research agenda. This broad sourcing will ensure that the agenda is grounded in the empirical literature, informed by SIRC’s global membership and curated by our MNoE. Table 3 provides an overview of the six sources and offers examples of the types of research agenda items we might obtain from each. We believe the selected sources allow for a systematic and relatively comprehensive approach to generating a research agenda; however, we note several limitations in the discussion.

Table 3.

Summary of data sources informing concept mapping and nominal group technique

Input	Data represented	Global	Examples of expected data in form of challenge statements
Systematic reviews	Peer-reviewed literature	Y	Lack of shared terminology (linguistic inconsistencies) and definitions (lack of conceptual clarity) Number of determinants make it unrealistic to isolate impact Majority of studies focus on intrapersonal mechanisms, and few studies examined multilevel relationships
Matrix mapping	Expert input from investigators’ recent and ongoing studies	N	Several investigators use ‘conduct educational meetings’ across numerous studies and upwards of 20 different mechanisms are being studied The majority of studies do not indicate a theory to guide their evaluation Very few investigators explore system-level mechanisms There is insufficient attention to several priority populations (eg, minoritised populations, low-income families)
Implementation Development Workshop	Expert input from investigators proposing new studies	Y	Very few proposals study implementation mechanisms because their budget is not equipped to power for these analyses Investigators do not choose strategies based on their putative mechanisms of action An equity lens is rarely integrated
Implementation Research Institute	Expert input from researchers	N	Available measures might drive conceptualisation of what is important It is difficult to decide how often we need to measure each putative mechanism to measure the change trajectory or sequence
SIRC breakout sessions	Novice to expert input including practitioners, policy makers, purveyors, students and researchers	Y	Unevenness in availability of measures across constructs Often context is invoked retrospectively and is insufficiently measured Longitudinal, iterative nature of mechanisms evaluations make it difficult to measure and analyse
SIRC abstracts	Novice to expert input including practitioners, policy makers, purveyors, students and researchers	Y	Despite ubiquitous nature of training and post-training consultation, few studies evaluate their mechanisms of action Within a complex social–ecological system, there are multiple mechanisms by which an intervention could have its effect on the distal implementation outcome Descriptions of implementation strategies are too general and do not include full and consistent descriptions of their active ingredients

‘Global’ refers to including global participants or data.

SIRC, Society for Implementation Research Collaboration.

Systematic reviews

Our work will be informed by two systematic reviews of mechanisms in implementation. The only mechanisms-focused systematic review published prior to our grant funding focused on multilevel mechanisms of implementation strategies in the context of randomised-controlled trials in mental health.¹⁰ That review included nine randomised trials that formally tested mediators linking an implementation strategy to an implementation or clinical outcome. Mediation analysis is one quantitative approach to studying mechanisms in which the total effect of an implementation strategy on an outcome is divided into an indirect effect that occurs through a proposed mediating variable and a direct effect that is not explained by the mediating variable.³⁰ None of the trials provided sufficient evidence to support a plausible mediator primarily because there was no evidence that the implementation strategies changed the targeted mediating variables. This review also identified several issues in mediation studies of implementation strategies, including study designs that were not conducive to detecting a relationship between the implementation strategy and candidate mediator, insufficient measurement and underuse or misuse of theory to link implementation strategies to outcomes. We will draw on the findings of this review, including Williams’¹⁰ suggestions for future research when populating our research agenda. For instance, Williams¹⁰ suggested investigating theory-informed constructs as potential mediators, improving theoretical links between implementation strategies and hypothesised mediators, enhancing study designs and analytical methods to detect and analyse mediators in multilevel contexts, and further study of implementation strategy change processes to better target candidate mechanisms.

A second systematic review conducted by several members of the MNoE built on Williams’¹⁰ work by expanding the scope to include implementation studies across health, a wide array of study designs and the investigation of moderators.⁹ This review included 46 studies (including the nine from Williams’¹⁰ review), and applied seven criteria for establishing a mechanism:^{1 11} (i) strong associations between the implementation strategy and mechanism and between the mechanism and outcome, (ii) specificity (demonstration that one construct and not others influencing change), (iii) consistency (replication across studies), (iv) experimental manipulation of the strategy or proposed mechanism, (v) timeline (change in mechanisms precede change in outcomes), (vi) gradient (dose–response relationship between level of mechanism and level of outcome) and (vii) plausibility/coherence (process-outcome relationship is reasonable or supported by other research). This review found only one study that met six criteria;³¹ the vast majority of included studies (n=38) met three or fewer. Fewer than half of studies included an implementation strategy as the independent variable of interest. As with Williams,¹⁰ this review identified underuse of theory and overuse of study designs and analytical methods that do not afford the opportunity to establish mechanisms. We will draw on the findings and suggestions from this review when generating the research agenda.

Matrix mapping of ongoing research

To understand how members of the MNoE are studying mechanisms and to identify gaps in this area of inquiry, we will synthesise the data obtained from~30 researchers in an online Matrix Mapping exercise. Matrix mapping allows members to report on recently completed and in-progress projects that include the study of mechanisms (eg, mediation studies and other studies of implementation strategies), and for us to map, or identify, saturated versus thin areas of the matrix. To this end, the matrix is organised by the following: priority populations (coded using categories identified by AHRQ),³² target problem (disease, disorder, symptom or risk factor), EBP intended to improve target problem, context/setting in which the EBP was delivered, guiding theory or framework, implementation strategies, mechanism(s) under investigation (or hypothesised mechanisms), implementation outcome(s), study status (planned, in progress or complete) and references for the project. The matrix will be coded using the Expert Recommendations for Implementing Change implementation strategies compilation^{7 8} and the Implementation Outcomes Framework.³³ We will calculate frequencies and proportions of responses occurring in each category to determine areas where there is representation (eg, commonly used implementation strategies) and areas for which there are gaps (eg, understudied implementation outcomes, dearth of interventions to improve the health of certain target populations) to inform the research agenda.

Implementation Development Workshop (IDW)

The IDW is a half-day, preconference event that is regularly part of the biennial SIRC conference, in which SIRC Network of Expertise members (ie, practitioners, students, new and established investigators) are invited to present ‘works in development’ to receive expert feedback from colleagues.³¹ Based on the Behavioural Research in Diabetes Group Exchange model used by the Psycho-Social Aspects of Diabetes research group,³⁴ the format includes 10–20 min for each presenter to orally (without technology) describe their project/proposal and 20–40 min for feedback coordinated by a facilitator. A note-taker records notes of the feedback so the presenter can be involved in the discussion. In 2019, 54 researchers and/or practitioners participated. Qualitative content analysis²⁴ of the notes reflecting each proposal discussion will serve as a third source for the research agenda to capture challenges and possibilities in the study of implementation mechanisms among a unique group of experts beyond the MNoE. That is, we will learn how mechanisms are being considered (if at all) in project proposals and uncover the challenges researchers face when considering mechanism evaluation.

Implementation Research Institute

The Implementation Research Institute³⁵ is a 2-year interdisciplinary training programme funded by the US National Institutes of Health that focuses on developing and nurturing a network of scholars who focus on researching the implementation of effective practices within the field of behavioural health.^{36 37} Every summer features a week-long training institute in which fellows convene with core and expert faculty to engage with emerging issues in implementation science. Working sessions to advance the field are often conducted and one focused specifically on challenges to studying implementation mechanisms will serve as our fourth data source. As a precursor to the exercise, a general overview of implementation strategy research was provided as well as specific content related to the concept of mechanisms (eg, What is a mechanism? What do we know about mechanisms in implementation science?). Faculty (n =~10) and fellows (n =~20) were then split into three groups: (1) theories, frameworks and context, (2) design and analysis and (3) measurement. Each group considered two questions related to their focus area: (1) What are the major challenges to advancing our understanding of mechanisms? and (2) What do you view as promising paths forward to advance our understanding of mechanisms? The groups were given 30 min to discuss and asked to assign one or more note-takers to document ideas. Following this exercise, each group reported their results to the larger group. We will use qualitative content analysis²⁴ to code the notes for unique challenges and opportunities to inform the research agenda.

SIRC breakout sessions

All SIRC 2019 conference attendees were invited to participate in two breakout sessions facilitated by investigative team members. These sessions were advertised as opportunities for the MNoE to learn from the experiences of global researchers, policy makers, practitioners and intermediaries. To participate, attendees agreed to complete 1 hour of preparatory work, including viewing a recorded webinar and reviewing an overview handout. These requirements were set to ensure all attendees would enter the breakout with the same foundational knowledge (eg, What is an implementation mechanism?) and to encourage active participation by breakout session attendees. Participants had the option of attending one or both breakout sessions (over 90 attendees attended each session), and they were informed that their input would be used to ultimately shape the research agenda.

The first breakout session focused on challenges of using theory to understand context and to inform the study of implementation strategies, mechanisms and outcome linkages. It began with an introduction and discussion of the current landscape of implementation mechanisms and then moved to a conversation about key challenges and gaps associated with studying implementation mechanisms through the lens of the four workgroups. Attendees then split into four groups: (1) causal theory and context; (2) connections from implementation strategies to mechanisms and outcomes; (3) design and analysis and (4) measurement. Attendees were instructed to select the group that best fit their expertise or interests, and the majority of the session was spent in facilitated discussion. At the end of the breakout session, one member of each group presented a summary of their discussion to the larger group. Notetakers were embedded throughout to document and permit subsequent generation of challenges and opportunities relevant to the research agenda.

The second breakout session began with brief ‘stimulus talks’ from the MNoE workgroup leads and a case study. The 3 min stimulus talks and 13 min case study focused on thorny issues related to the study of implementation mechanisms. Following these talks, as with the first breakout session, attendees separated into four groups: (1) causal theory and context, (2) connections from implementation strategies to mechanisms and outcomes, (3) design and analysis and (4) measurement. Participants spent most of the time discussing the case study through the lens of their workgroup’s topic area, coming together at the end of the session for summary discussion. Small group discussion was designed to elevate additional issues not raised in the presentations, and to use the case study to address the issues raised by the presenters through the lens of their workgroup. At the end of the breakout session, one member of each of the four workgroups presented a summary of their discussion to the larger group. Note-takers were embedded throughout to allow for data extraction of qualitative data that could inform the research agenda. We will use qualitative content analysis²⁴ to mine the notes of these two breakout sessions to inform the research agenda.

SIRC abstracts

Our final source will be the 2019 presentations. The purpose of this input is to understand the extent to which implementation mechanisms were represented in the SIRC 2019 programme,²⁸ uncover gaps in the study of implementation mechanisms and reveal exemplar studies that may not be yet captured in the published literature. All 205 abstracts from the 2019 SIRC conference programme will be qualitatively coded for implementation mechanism-related content. If authors make mention of mechanisms (or mediators and moderators) in their abstract, the coder will record the mechanisms reported and the measurement method if available. Abstracts will also be coded for setting and project design; priority populations;³² guiding theories/frameworks;³⁸ implementation strategies;^{7 8} service outcomes, implementation outcomes and clinical/client outcomes³³ and barriers to implementation.³⁹ Abstract information will be entered into REDCap, and three coders will independently extract data from a random sample of abstracts with 5% coded by all three members of the team to assess for inter-rater reliability. The results of the abstract coding will directly inform the research agenda.

We will conduct qualitative content analysis²⁴ to code each of the six sources described above across two categories: (1) challenges in conducting mechanistic implementation science and (2) priorities for improving the study of implementation mechanisms. Data analysis will occur in three phases: immersion, reduction and interpretation. During the immersion phase, study team members will obtain a sense of ‘the whole’ before rearranging it into smaller segments.²⁴ This will be accomplished by reviewing each of the six inputs described above to gain a better sense of these data. The reduction phase will involve condensing data into text segments by coding each of the six data sources. To increase reliability and reduce bias,^{40 41} two researchers will use an electronic spreadsheet to independently extract text segments that represent challenges or priorities related to studying implementation mechanisms. These coders will regularly debrief during project meetings to ensure a common frame of reference for each input and to discuss and resolve discrepancies. A third researcher will work with BJP and CL to examine all extracted text segments to eliminate redundancies and ensure that each segment is worded clearly and concisely. Rather than identifying themes across text segments, each text segment describing challenges related to researching implementation mechanisms will be used in the concept mapping²⁵ study described below. This will generate conceptually distinct clusters of challenges that will need to be addressed to advance the field’s understanding of implementation mechanisms, and it will constitute the interpretation phase of the qualitative content analysis approach. Although coders will also extract possibilities or opportunities for the field, these will be tabled for consideration until the Deep Dive retreat when the results of the concept mapping are turned into an actionable research agenda.

The members of the MNoE will be engaged in a concept mapping study to further develop and refine the emerging research agenda to advance the study of mechanisms in implementation. Concept mapping is a structured process designed to organise concepts into conceptually distinct categories and to generate ratings of specified dimensions such as importance and feasibility. As noted by Powell et al⁴² concept mapping is particularly useful for structuring the ideas of diverse groups of stakeholders and has been leveraged by implementation researchers to identify and prioritise implementation barriers and facilitators,^{43 44} implementation strategies,⁸ training needs,⁴⁵ dimensions of pragmatic measures⁴² and an agenda for studying sustainability of EBPs.⁴⁶ Concept mapping is a mixed-methods, multistep approach that typically involves the identification of specific statements through brainstorming or other sources, statement analysis and synthesis, statement rating, unstructured sorting of statements, multidimensional scaling and hierarchical cluster analysis and the generation of interpretable maps and data displays. Concept mapping is described in more detail by Trochim and Kane²⁵ and Kane and Trochim.⁴⁷

The Concept mapping process will be completed via the Group Wisdom software platform.⁴⁸ Statement identification, analysis and synthesis will occur via the consolidation of the six research agenda sources as described above. Statements describing challenges to advancing research on implementation mechanisms will be entered into the Group Wisdom software platform. To ensure feasibility and to remain consistent with recommendations for the overall number of statements for concept mapping studies,^{49 50} we will keep the number of statements to approximately 100 statements or less. Members of the MNoE will then be invited to engage in the concept mapping study via the Group Wisdom platform.⁴⁸ MNoE members will be asked to complete a brief demographic survey and then be presented with the list of statements to engage in rating and sorting tasks (in any order they so choose). They will be asked to rate each statement on a 5-point Likert scale across two dimensions, including criticality (ie, how important the challenge is to advancing the science and achieving health equity) and pervasiveness of the challenges. They will also be asked to sort each of the statements into piles or categories that make sense to them and will be given the opportunity to label each of the categories they create.

We will utilise multidimensional scaling to generate a point map depicting each of the statements and relationships between them based on a summed square similarity matrix.⁴⁷ Statements that are sorted together more frequently will be placed closer together on the map. Model fit will be assessed using the stress value, an indication of goodness of fit between the point map and the total similarity matrix.^{47 50} Hierarchical cluster analysis will be used to partition the point map into non-overlapping clusters.⁴⁷ We will convene the MNoE to consider a range of cluster solutions produced by the analysis to determine the number of clusters that best represents the core domains of our emerging research agenda. The Group Wisdom software platform⁴⁸ will aid in the labelling process by suggesting potential cluster labels based on participant responses; however, the MNoE will engage in discussion to determine the names for each cluster in our final cluster map. Descriptive statistics of MNoE members’ ratings of criticality and pervasiveness will be presented at the statement and cluster levels.

Step 5: Identifying research priorities via nominal group technique

The Deep Dive retreat will be focused on engaging the MNoE in small workgroups organised by the conceptually distinct clusters that emerged from the concept mapping. The workgroups will consider the discrete challenges in their cluster(s), examine how they are related and reflect on their degree of criticality and pervasiveness. They will then engage in a facilitated nominal group process²⁶ to brainstorm broad priorities and specific activities for addressing each challenge. Nominal group process is a structured method of small-group discussion to reach consensus in problem-solving, idea generation and priority setting.²⁶ Nominal group process brings structure to what is often described in vague terms in published reports as an ‘iterative process’ for developing insights and exhausting the expertise of a group. Once all challenges have been subjected to the nominal group process, the resulting opportunities and activities will be captured in a structured research agenda. The research agenda will be comprised of challenges, priorities and specific activities that will be organised according to the clusters identified through the concept mapping study.

Steps 6 & 7: Dissemination, discovery & refinement

We will disseminate the research agenda via multiple channels to maximise reach to research, practice and policy communities. This includes (but is not limited to) conference presentations, peer-reviewed publications, webinars, policy briefs and short video clips. For the research community, we will present our findings at multiple national and international conferences (eg, SIRC 2022, AcademyHealth/NIH Conference on the Science of Dissemination and Implementation, etc) and webinars (eg, the US Department of Veterans Affairs Health Services Research and Development Cyberseminar series). We will also publish a special issue or collection of peer-reviewed articles, which will include commentaries and critiques from internationally renowned implementation scientists. For policy and practice audiences, we will develop policy briefs and short video clips for dissemination via the SIRC website, relevant listservs and social media channels. Both the MNoE and SIRC 2022 attendees will have opportunities to inform the dissemination of the agenda to diverse research, policy and practice communities internationally. These dissemination activities and the pursuit of the priorities outlined in the research agenda will inevitably lead to ongoing refinement as our understanding of implementation mechanisms deepens.

Discussion

Understanding how and why implementation strategies achieve their effects will enable stakeholders to deploy these strategies with more precision while retaining their core functions, develop multifaceted and multilevel implementation strategies that combine component strategies in a synergistic manner,⁵¹ systematically match strategies to high-priority determinants and tailor strategies to diverse and under-resourced contexts. This is particularly important as a means of addressing health disparities by ensuring that implementation efforts address social, political and environmental factors that contribute to inequitable outcomes for underserved populations.⁵² However, there are significant challenges to advancing the study of implementation mechanisms that cut across conceptual, methodological, analytical and practical spaces. By partnering with SIRC, convening an MNoE and hosting Deep Dive retreats, we will clarify these challenges and develop a research agenda that pairs challenges with priorities and activities that will advance the field.

We acknowledge several limitations to our planned approach. First, although we recently expanded our MNoE to include international members, we are primarily based in the USA, which may limit the relevance of our research agenda internationally where researchers and implementers may face different challenges or generate unique opportunities. We will continue to seek opportunities to obtain feedback from international experts in implementation research, practice and policy by expanding our MNoE and by seeking international dissemination outlets that will afford the opportunity to receive feedback on the developing research agenda. Second, given the focus on advancing research methods, stakeholder engagement in this effort focuses primarily on researchers. Unfortunately, the limited opportunities for patients and the general public to inform the research agenda may have negative unintended consequences. To fill the potential gaps in this protocol, we will draw on findings from a related, recently funded Centre grant from the US National Institute of Mental Health (P50MH126219) where we have robust stakeholder participation (eg, youth and family advocates, providers, organisational leaders and policy makers). Finally, our sources are not exhaustive but are intended to capture diverse and expert perspectives, which may result in missed challenges and approaches that are important but not included in the six sources described above.

The development of this research agenda represents a tremendous opportunity to initiate international discourse about how we can develop more precise, scalable and effective implementation strategies capable of improving the quality of health and social care delivery. We welcome input from the practice, policy and research communities on the development and dissemination of this research agenda.

Ethics statements

Patient consent for publication

Not required.