Building capacity from within: qualitative evaluation of a training program aimed at upskilling healthcare workers in delivering an evidence-based implementation approach

Abstract

Translating evidence into complex health systems is an ongoing challenge. Building the capacity of healthcare workers in behavioral and implementation science methods may facilitate the use of evidence-based implementation approaches, leading to sustainable and effective translation. The aim was to describe the development, contents and evaluation of a training workshop aimed at upskilling hospital-embedded staff to deliver an evidence-based implementation approach. The Hide and Seek Project (HaSP) is a cluster randomized controlled trial testing two implementation approaches for improving hereditary cancer referral at eight Australian hospitals. Healthcare workers were recruited as “Implementation Leads” and trained via a one-day workshop—TRAining in evideNce-baSed ImpLementATion for hEalth (TRANSLATE). The purpose of TRANSLATE was to upskill Implementation Leads in the delivery of HaSP, as well as implementation science methods more broadly. Implementation Leads participated in semi-structured evaluation interviews, which were analyzed using inductive thematic analysis. Nine Implementation Leads from various professional backgrounds completed the training. Four key themes were identified: (i) training day reactions, (ii) learning, (iii) implementation barriers and facilitators, and (iv) building health system capacity for implementation. Participants reported high levels of satisfaction, and anticipated that the knowledge and skills may be useful in the future. We describe a novel training program focused on the delivery of evidence-based implementation within health systems. Guided by insights from this study, methods to deliver the training on a larger scale and across different contexts are being explored. The prolonged impact of TRANSLATE will be further evaluated at trial completion.

Trial registration: ANZCTR, ACTRN12618001072202. Registered on June 27, 2018

Implications.

Practice: Upskilling healthcare workers from within the system may enhance implementation effectiveness and fidelity, while building longer-term capacity for evidence-based approaches to quality improvement.

Policy: Partnerships between researchers and healthcare policymakers should work toward building a system-wide implementation training and support framework targeted to the specific needs of healthcare workers and organizations.

Research: Further research is needed to formally describe and evaluate existing and emerging evidence-based implementation training programs and models of system-led implementation to identify opportunities for refinement and potential upscaling.

Background

Despite a growing body of research dedicated to improving the uptake of evidence-based clinical practices, introducing and maintaining change within complex health systems remains an ongoing challenge [1]. While theories and frameworks can help to optimize intervention design and implementation [1, 2], hospital-based implementation trials are often led by external researchers who, despite expertise in implementation science, lack crucial understanding of the inner workings of the system they are attempting to change (e.g., policies, procedures, networks, politics). Such approaches may impact engagement among frontline clinical staff, and potentially result in less successful implementation [3, 4].

One strategy to address this issue is to identify healthcare workers (e.g., patient facing healthcare professionals, administrators, clinical educators, clinical researchers) from within the organization to lead the implementation effort internally. Quality improvement processes are often routinely at play within hospital systems, and while hospital staff are well-placed to apply their tacit and experiential knowledge to address the clinical problem they are attempting to solve, capacity constraints (e.g., lack of staff time and resources) can limit the effectiveness of these efforts [5, 6]. Implementation science frameworks and tools offer the ability to overcome these contextual and capacity constraints and enable faster, more effective and more sustainable improvement [7]. In addition to addressing organizational and contextual barriers, a number of these frameworks and tools also draw on behavioral and social science theories that may be more effective in eliciting change at the individual level [7, 8]. However, these frameworks are currently underutilized in the hospital quality improvement literature [9, 10].

Partnerships with behavioral science and implementation researchers to train and support staff from within the health system to lead evidence-based implementation approaches may be a more sustainable approach for effective translation [11], and evidence suggests that this approach can improve the fidelity of implementation program delivery [12]. These partnerships enable the complementary combination of contemporary evidence-based approaches to implementation and behavior change (including the explicit application of theory) with the tacit and contextual knowledge of end-user hospital staff [13]. Furthermore, knowledge and skills can be applied to address other problems beyond the scope of the intended program, building capacity within the system for ongoing change and quality improvement. However, there is limited practical guidance on how researchers should conduct this training, or on ways to provide ongoing support to the frontline healthcare professionals implementing the program.

Recognizing the value of staff capacity-building, a number of initiatives have focused on training healthcare workers in quality improvement methods, such as Lean Six Sigma and Systems Redesign [14–16]. These training initiatives have seen the successful application of quality improvement methods within local hospital environments and, on a larger scale, enterprise-wide programs [14, 17, 18]. Although the goals of quality improvement and implementation science both seek to improve patient outcomes, there are a number of key differences in the methods used between the two fields [19]. For example, while quality improvement methods often use rapid cycles of change to test what strategies might work best, implementation science methods emphasize the application of evidence and theory to guide the process [20].

In recent years, training programs have been developed to upskill researchers (including clinician researchers) and academics in evidence-based implementation methods and the application of theory [21–24]. For example, The Training Institute for Dissemination and Implementation Research in Health (TIDIRH) course builds capacity among researchers, healthcare professionals and health managers internationally to apply evidence-based implementation methods in the design of improvement efforts [24]. However, relatively few studies describe programs targeted to healthcare workers to address the additional need to train and support those in the healthcare system who will be tasked with delivering these programs in practice [12, 24, 25]. Delivering an evidence-based implementation approach may require healthcare workers to tackle unfamiliar tasks and apply new skillsets. Indeed, several studies have identified that healthcare workers experience barriers when attempting to apply implementation science methods without specific training (e.g., limited accessibility and understanding of theory, practical challenges applying implementation frameworks) [3]. There have been calls among healthcare workers for targeted training efforts to ensure that implementation programs are well-delivered in practice [4]. Training methods focused on the application of implementation science may be particularly valuable—and indeed required—as workforces (including hospitals) increasingly seek to employ implementation science methods for improvement.

The specific needs of healthcare workers tasked with delivering an implementation approach are currently understudied. In order for researchers and academics to work toward a recommended set of learning outcomes and competencies to guide the design of implementation training programs targeted to healthcare workers, in-depth evaluations of emerging programs are needed. The Kirkpatrick framework for evaluating educational outcomes emphasizes four key levels for evaluation: reaction (participants’ initial response to training materials, instructor, and delivery format), learning (improvements in participant knowledge and skills), behavior (the extent to which training influenced participant behavior in practice), and results (the organizational impact of the training) [26]. Applying such frameworks in the evaluation of training programs can be useful in determining program strengths and weaknesses, and modifying content and delivery modes to enhance effectiveness [26, 27].

The Hide and Seek Project (HaSP) is a cluster randomized controlled trial (RCT) comparing two implementation approaches aimed at improving detection of Lynch syndrome (a hereditary cancer predisposition) across eight large Australian hospital networks [28]. At each site, a healthcare worker has been sought from within the hospital and trained as an “Implementation Lead” to oversee HaSP program delivery in a part-time role. Implementation Leads are working in partnership with the HaSP research team (with theoretical and implementation expertise), who provide training and ongoing support as the trial progresses.

Here, we describe the development, contents, and evaluation of a 1-day training workshop—TRAining in evideNce-baSed ImpLementATion for hEalth (TRANSLATE)—aimed at upskilling healthcare workers to deliver an evidence-based implementation approach. The most novel aspect of this training is the focus on implementation science methods and application of theory in a way that is specifically targeted to healthcare workers, and enhancing skills and strategies needed to support the delivery of these methods in real-world settings. We also describe a partnership model of ongoing support, through which healthcare workers and implementation researchers can effectively collaborate to produce positive, and more sustainable change within the healthcare system.

METHODS

Implementation lead recruitment

Implementation Leads were primarily recruited via liaison with site Principal Investigators and other hospital staff stakeholders. For each hospital site, the research team sought to employ an existing member of staff who was involved in the Lynch syndrome referral pathway (e.g., nurses, genetic counselors), for an additional 0.2 full-time equivalent (i.e., 1 day per week). As there were eight hospital sites, eight Implementation Leads were recruited. Site Principal Investigators provided information internally to potentially eligible staff members via known professional networks. Those who were interested were put in contact with the research team who then provided additional information and facilitated formal recruitment processes. Where eligible staff members could not be identified, recruitment was advertised according to hospital network requirements. Although the primary aim was to recruit healthcare workers with a working knowledge of the LS referral pathway, other hospital staff with relevant experience were also considered (e.g., clinical researchers, medical educators).

Following successful recruitment, Implementation Leads were invited to participate in a one-day face-to-face TRANSLATE workshop held at Cancer Council NSW. Implementation Leads were exposed to one of two training programs, depending on their trial arm allocation (see Appendix 1).

Overview of HaSP

HaSP is a cluster RCT testing two structured implementation approaches, differentiated only by the explicit use of theory, for improving Lynch syndrome related molecular tumor testing and risk-appropriate referral practices for colorectal cancer (CRC) patients in eight large Australian hospital networks (clustered by state). The trial is currently underway, and a detailed rationale and protocol is available elsewhere [28].

Implementation Leads are overseeing the following phases over a 2-year period: (i) baseline audits of Lynch syndrome referrals among CRC patients, (ii) formation of multidisciplinary “Implementation Teams,” (iii) identification of target behaviors to achieve practice change, (iv) identification and confirmation of barriers to change, (v) generation of intervention strategies, (vi) support of staff to implement interventions, and (vii) evaluation of intervention effectiveness using audit and process evaluation data to assess practice and culture change. The theoretical and non-theoretical components of each trial arm are distinguished in phases 4–5, and involve the use of theory for barrier identification (Theoretical Domains Framework; TDF; [2]) and intervention design (Behavior Change Techniques; BCTs; [29]). Clinical data will be extracted pre- and post-implementation, the primary outcome measure being the proportion of patients with risk-appropriate completion of the Lynch syndrome tumor testing and referral pathway.

Development of the TRANSLATE workshop

TRANSLATE workshop materials and accompanying resources were developed by the research team, with expertise in implementation science and behavioral science (N. Taylor, L. Wolfenden, D. Debono), teaching (D. Debono, A. Morrow), genetics and genomics (J. Steinberg, K.M. Tucker, A. Morrow), and research and project management (E. Hogden, P. Chan, G. Tiernan). Two team members had similar clinical backgrounds to the Implementation Leads involved in this study and provided input from a stakeholder perspective (D. Debono—nursing; A. Morrow—genetic counseling). Lay-person feedback was also sought from staff external to the project without any experience or prior training in implementation science or behavior change methods.

The key training objectives were to provide the Implementation Lead with an in-depth understanding about the HaSP study and their role (including the rationale for using an implementation science and/or behavior change approach), as well as practical strategies and applied skills to promote implementation success (e.g., running focus-groups, process-mapping hospital systems, maintaining stakeholder engagement). Some materials were adapted from the original Achieving Behavior Change for Patient Safety materials designed by research team member N. Taylor and colleagues, also with significant clinician stakeholder input [30]. The workshop was designed to suit an interactive, small-group format. Lecture-style presentations were combined with small group discussions and activities. Training materials consisted of PowerPoint slides, a comprehensive HaSP study toolkit (electronic and hard copies), and workshop handouts for completion of activities. Efforts were made to accommodate diverse learning styles, for example by including: different content delivery modalities and media types (e.g., videos, PowerPoint slides, supplementary written materials); dialectic and questioning activities (e.g., hypothetical scenarios to stimulate critical thinking); and opportunities for personal and group reflection (e.g., group discussion, Project Logs) [31].

TRANSLATE workshop content

Training material contents were tailored according to trial arm allocation. Implementation Leads trained in the theory-based approach were exposed to additional content related to theoretical frameworks and behavior change techniques (specifically the TDF and BCT Taxonomy). Appendix 1 provides a summary of the training material content delivered to both trial arms. The workshop was designed to be transferrable to other clinical contexts beyond Lynch syndrome (only the background and introduction component of the workshop was specific to Lynch syndrome and the HaSP trial).

Ongoing support

Ongoing training and support are provided to the Implementation Leads individually via scheduled teleconferences conducted prior to each HaSP phase (see Appendix 2 for an example pre-phase teleconference agenda). In line with recommendations by Dolcini et al. [12], the teleconferences aim to supplement the workshop training by revisiting phase-specific tasks in detail, reviewing Toolkit instructions and relevant materials, and providing tailored, site-specific guidance. Teleconferences also provide opportunities for Implementation Leads to practice new skills (e.g., explaining audit data and process maps) and obtain feedback from the research team.

Additional support is also provided via a centralized HaSP research team email address, and regularly updated “Frequently Asked Questions” available online to all Implementation Leads. As challenges arise, ad-hoc teleconferences are scheduled between Implementation Leads and the HaSP research team.

Evaluation interviews

Following the training day, Implementation Leads were invited via email to participate in a semi-structured telephone interview. Open-ended questions were designed to gain detailed feedback about the workshop, and any perceived challenges they anticipated encountering throughout their role (see Appendix 3 for interview guide). Briefly, the interview guide was structured to include basic demographic information, general impressions of the training package (e.g., length of time, resources provided, delivery format), understanding of implementation science (and whether/how understanding changed as a result of the workshop), anticipated challenges and suggestions areas for improvement. For participants in the theory-arm only, additional questions explored their understanding of, and opinions about, a theoretical behavior change approach (specifically the TDF).

Although efforts were made to conduct interviews within a week of the training, due to ethics and governance delays some Implementation Leads had already commenced early study phase activities (in this case, reporting challenges experienced, rather than anticipated). Interviews were conducted individually by a member of the HaSP research team (PC) who was not involved in delivery of the TRANSLATE workshop. On average, interviews were 31 min in duration (range = 16–44 min, median = 32 min). Written and verbal consent was obtained from participants, and all interviews were audio-recorded and transcribed verbatim for qualitative analysis.

Analysis

Interview transcripts were de-identified and coded via the qualitative analysis software NVivo Version 12 (QSR International, Victoria, Australia). An inductive thematic analysis approach was used, guided by Braun and Clarke [32]. The first four transcripts were read and line-by-line coded by two researchers (A. Morrow and P. Chan), who then met to compare codes and develop an initial coding framework for subsequent transcripts. All remaining transcripts were then analyzed separately by A. Morrow and P. Chan, who met regularly throughout the transcript analysis process to review and refine the coding tree. Any discrepancies in coding were discussed with a third reviewer (N. Taylor). A. Morrow reviewed the codes and grouped them into themes and subthemes. Themes and subthemes were refined based on discussion among members of the analysis team (A. Morrow, P. Chan, N. Taylor, E. Hogden).

RESULTS

Participant demographics

Four separate TRANSLATE workshops took place over a 12-month period. In total, nine Implementation Leads completed the training workshop (one Implementation Lead moved to a new role and an additional Implementation Lead was identified). While the majority (7/9) took part in small-group workshops, two Implementation Leads were recruited at a later stage and therefore received individual training. Seven of nine Implementation Leads were able to be identified and recruited internally, while the remaining two were recruited externally due to bureaucratic hurdles at hospitals. All nine Implementation Leads participated in post-workshop telephone interviews. Implementation Leads came from a variety of professional health related backgrounds and levels of experience (Table 1).

Table 1.

Participant characteristics (n = 9)

Variable	N (%)
Gender
Female	8 (89)
Male	1 (11)
Professional background
Nursing	2 (22)
Genetic counseling	3 (33)
Clinical research	3 (33)
Medical education	1 (11)
Years in current role (i.e., primary role outside HaSP trial)
<2	6 (67)
2–5	0 (0)
5–10	2 (22)
10+	1 (11)
Trial assignment
Theory-based	5 (56)
Non-theory based	4 (44)
Recruitment
Internally driven	7 (78)
Externally driven	2 (22)

Qualitative analysis

Qualitative analysis of transcripts identified four themes and 13 subthemes, which are summarized below in Table 2. Themes were largely consistent between participants in both trial arms (with the exception of responses to the additional theory-based questions explored only among participants in the theory-based trial arm). Given that the majority of TRANSLATE workshop contents were common across both trial arms, results have been grouped and differences noted throughout. Representative quotes are provided in the text below, with additional quotes provided in Appendix 4.

Table 2.

Themes and categories

Theme	Subthemes
I.Training day feedback	1. Overall satisfaction
	2. Workshop format and content
	3. Workshop duration
	4. Timing of the workshop in relation to study rollout 5. Ongoing researcher support
II.Learning	6. Implementation knowledge 7. Behavior change theory (theory-based trial arm only) 8. Skills
III.Implementation barriers and facilitators	9. Staff engagement 10. Hospital culture 11. Practical challenges 12. Implementation Lead embeddedness within the system
IV.Building health system capacity for implementation	13. Value of the Implementation Lead role 14. Transferability of implementation knowledge and skills to other settings

Themes are derived from the Kirkpatrick framework.

Theme I: Training day feedback

Overall satisfaction

All participants (9/9) expressed overall satisfaction with the workshop. Participants felt that the training improved their understanding of the HaSP trial, as well as their role within it. As a result of the training, Implementation Leads expressed greater confidence in their ability to oversee the HaSP implementation approach at their site.

“I felt that this is a really good training package. I thought there wasn’t any wasted time. I felt that it was really valuable’. (Participant 6)

“The training package was very professionally delivered, but at a friendly level where questions were encouraged and anything that I asked at the time was really nicely explained and it gave me high confidence in that. What was being presented was really well thought-out.” (Participant 8)

Workshop format and content

The majority of participants (8/9) expressed overall satisfaction with the variety of content and resources, and their level of detail. Five participants also commented that the small-group, interactive workshop format contributed to a supportive and productive learning environment.

“I was probably most impressed by the fact of how comprehensive it was and I feel like that deep level of detail was really important to the success of it.” (Participant 3)

Given the long duration of the study, participants valued having a comprehensive Toolkit to refer to as they move through each phase of the HaSP trial. Participants also expressed that it helped to consolidate their learning from the workshop.

“The implementation training day toolkit mirrors my perception of the presentation that was given. It’s really well set out, quite comprehensive, but easy to refer to. So I know exactly where I’m going when I have to undertake each step and understanding why each part is important and how it all is put together in the grand scheme of the project over the few years that it’s going to be happening’. (Participant 5)

Duration

Although the majority of participants (7/9) felt that a 1-day workshop was an appropriate length of time, one participant suggested reducing the content, while another suggested splitting the delivery of the workshop across 2 days.

It didn’t actually seem like a really long day, but I know that we covered a lot of ground. I don’t think it would need to be any longer and I don’t really see reason for it to be shorter because I think we had a lot of really good discussion’. (Participant 5)

“Perhaps just splitting it over two days, perhaps one day really just focused on a lot of the background, having that opportunity to really get a grasp of why we are there. Then perhaps once that’s sunk in, the next day really just drilling down into as an Implementation Lead, what your roles and responsibilities would be.” (Participant 4)

Timing of the workshop in relation to study rollout

For a number of sites, planned study rollout dates were impacted by governance and contractual delays. As such, some Implementation Leads experienced a time-lag between completion of training and commencement of study activities.

“There was an element of the fact that it felt somewhat abstract because the job hadn’t actually started’. (Participant 3)

Furthermore, the HaSP trial takes place over the course of a two-year period. Most Implementation Leads (5/9) found it conceptually challenging having detailed planning discussions about the later study phases. Two participants suggested follow-up “refresher” training sessions be delivered during the study period.

“Perhaps because of the length of the project, I feel as though follow-up trainings or – I mean it was nice to have a whole overview of it on that day, but obviously, once I get to even trying to remember what the phases four and five and six – I mean that seems like in the far future and perhaps when I get to that phase, that training will have been over a year before.” (Participant 3)

Ongoing implementation support

Participants felt well-supported by the HaSP research team, and particularly valued the individual pre-phase teleconferences. This provided a sense of reassurance, particularly as they applied new skills and approached tasks that were unfamiliar.

“I felt very confident that with each phase, there was just going to be so much support and guidance that it really took away any of that anxiety I guess for some of those bigger tasks. It was definitely reassuring to know that there was a team of people there who were going to guide me with each step.” (Participant 4)

It was also reported that the ongoing support would enhance the fidelity of the planned implementation approach, as the research team could work with the Implementation Lead and ensure that tasks are completed as intended.

I’ve always felt like everybody was really accessible and really supportive and also helpful in reining me in, because like I said, you get your teeth a little bit into it, you get really excited about moving forward and sometimes there are things that just aren’t ready yet. (Participant 1)

Theme II: Learning

Implementation knowledge

Participants felt that they gained an appropriate level of implementation knowledge as a result of the training. All participants were able to accurately articulate a basic definition of implementation science, and to describe the key steps involved in each of the seven HaSP trial phases.

“It’s certainly given me a lot more knowledge about implementation science than I had to prior to coming to being involved with this study. I think it’s given me a better understanding of how interventions can be made more effective and the different ways that we can target interventions to specific behaviours and the more pivotal behaviours to achieve the outcome that we want.’ (Participant 7)

Behavior change theory (theory-based trial group only)

Some participants in the theory-based trial arm (2/5) experienced challenges grasping theoretical concepts of behavior change, though still expressed overall improvements in understanding and their ability to apply these concepts in practice.

“I suppose the framework that will be used – the theoretical domains framework – was all quite a new concept to me, but I found it very interesting and on review, I was able then to understand it a little bit better and apply that and understand that in the context of the study. But that was probably the most challenging part for me to get my head around.” (Participant 8)

“I found a lot of the theory, the behavioural theory, very new to me and I did get a bit lost. It got a bit heavy at some points. But then I think having those examples that we used to draw on did help for a few things to click, but it was a lot to take in some of that background theory.” (Participant 4)

All five theory-based participants expressed the value of a theory-informed approach, and believed it to be a useful approach for addressing suboptimal LS referral practices.

“It just seems so much more approachable rather than something that’s vague and uninformed. It gives people opportunity to really identify true barriers rather than perceived ones from general knowledge.” (Participant 8)

Skills

As a result of the training, participants gained new skills and felt better prepared to take on new or unfamiliar tasks. Many participants (6/9) expressed increased confidence in their ability to fulfil the Implementation Lead role.

“My confidence in my ability to apply those skills, and to gain new skills, and perform the role has definitely grown as a result of the training session. It sort of broke everything down in a way that was approachable and I think something that is very achievable for me.” (Participant 7)

Theme III: Implementation barriers and facilitators

Staff engagement

Five participants across both trial arms identified staff engagement as a potential challenge, particularly in the context of working with time-poor health workers. Some also cited potential difficulties maintaining engagement levels over the long course of the study.

“I think the biggest barriers are gonna be working with some of the staff here at the hospital because I think the levelling of engagement is gonna kind of run the whole gamut of how much people want to invest both their time and interest in this project.” (Participant 3)

Hospital culture

A number of participants (3/9) reported that a hospital culture that was open to change and supportive of research would be key to the success of the HaSP trial. Given that most Implementation Leads were employed from within the study sites, many already had a sense of their hospital’s culture and receptiveness to improvement efforts.

“I think our culture here in terms of generating ideas, particularly the colorectal department is quite good. Research is really well-embedded in everything that we do and I believe all our colorectal surgeons are involved in research, our nurses, and everyone from interns to staff specialists are quite involved. So I think they would be quite open to implementing new ideas.” (Participant 5)

In contrast, a hospital culture resistant to change was seen as a potential barrier to effective implementation, particularly when effort was required among staff stakeholders.

“Inertia is what I think of a lot. Any change takes energy and that’s why so many people keep doing things the way they’ve always done it, cause they’ve always done it that way for the last 20 years. You need to put something into it to generate change, even if the outcome might be more productive or more straightforward or simpler than what you have been doing. The actual change itself is the hard part.’ (Participant 1)

Practical challenges

Implementation Leads also cited practical and logistical challenges as a potential barrier to staff involvement in HaSP trial activities. At each site, input is sought from multiple staff groups involved in the Lynch syndrome referral pathway, including (but not limited to) surgeons, pathologists, nurses, oncologists, and genetic counselors. Implementation Leads are required to coordinate a series of meetings and focus-groups, aiming for representation from each of these staff groups. The majority of participants (7/9) identified challenges associated with coordinating a large, multidisciplinary team of time-poor healthcare workers.

“I don’t think approaching or having access to the people will be an issue. I think they’re very accessible, easy to speak to, but I think perhaps getting people in the same place to talk about things for an hour or two is probably quite challenging’. (Participant 5)

Implementation Lead embeddedness within the system

One externally recruited Implementation Lead noted the challenges of attempting to implement change as an “outsider.” To overcome this issue, this Implementation Lead made additional efforts to gain credibility by integrating themselves within the existing colorectal team.

“I’m somebody coming in from outside and that can sometimes get people backs off, especially if there were major changes or significant things that needed to happen, as well as getting the buy-in from people to give you some of their time - even though we’re not asking for a lot’. (Participant 1)

“I turn up to the MDT [multidisciplinary team meeting] every week to make sure that people see my face. I sit at the table. I make sure that I’m in contact with the team on a really regular basis…and so they see me as someone who is part of the team and is involved and they are willing to give me some airtime’. (Participant 1)

In contrast, participants who were recruited from within the system expressed an advantage in having known networks and credibility among other staff, which they leveraged to promote engagement with HaSP trial activities.

“I’m fortunate that I work in the colorectal team. The surgeons are very familiar with me, so that won’t be an issue. I think being known is useful.’ (Participant 6)

Theme IV: Building health system capacity for implementation

Dedicated Implementation Lead role

Having an Implementation Lead employed at each hospital site to oversee the HaSP trial was deemed crucial to the feasibility of the project. This was seen to alleviate workload burden among other hospital staff, and ensure momentum is maintained throughout the duration of the study.

“I said, “My anticipation is that I’ll be the person doing the leg work and basically I’ll be asking you for your help along the way, but I won’t be expecting you to do much in the line of legwork apart from what your specialty area is” and thus far, those people all seem to love to be part of it.’ (Participant 6)

“There’s always a bit of a hot-potato effect with clinical teams when it’s gonna take someone a bit of extra time or a bit of extra effort. As much as someone might want something to be different or can see a better way of doing something - whose responsibility is it to make it happen? So I think that having someone dedicated to doing the implementation is really helpful.’ (Participant 1)

Transferability of implementation knowledge and skills to other settings

The majority of participants (7/9) articulated that the training would influence the ways in which they would approach future hospital-based improvement efforts. Participants expressed that they were likely to consider using the evidence-based implementation strategies they learned during the training to address other clinical problems, deeming these strategies to be more effective than traditional “top-down” approaches.

I think having that evidence base behind it and sort of understanding whether it’s a big project or a little change, actually planning it out properly instead of just sort of springing it on people. So I think if, for example, we were implementing some completely new research system or something like that, bringing in from the top to the bottom the consultants and the staff, especially right down to interns, who might be the ones who are actually filling out forms, anything that’s new, just sort of getting the people who are going to be really involved with it, to have their say. (Participant 5)

“I think it’s just given me more of a context and an understanding to be able to approach new problems in my workplace in the future.” (Participant 7)

Participants also felt that the implementation skills and knowledge gained through their involvement in the HaSP trial were applicable to other settings beyond Lynch syndrome and hereditary cancer.

“It gives me a foundational understanding of the processes that can be followed to reach an effective strategy. And so that can be applied across multiple settings.” (Participant 2)

Discussion

To our knowledge, TRANSLATE is the first training program focused on preparing and providing ongoing support to healthcare workers from within the system to deliver evidence-based implementation science methods, including the application of behavior change theory. Results from the post-training evaluation provide early evidence that the TRANSLATE workshop is of benefit to healthcare workers delivering an implementation trial. Post-training evaluation interviews allowed us to explore the first two levels of the Kirkpatrick framework: reaction and learning [26]. Participants reacted positively overall to the content and delivery format, while highlighting a number of areas for potential improvement. Participants valued the small-group, interactive learning environment, and felt that the content and supplementary resources were relevant and informative.

Results of the evaluation also suggest that key learning outcomes were achieved. Following the workshop, participants reported increased knowledge and skills (and greater confidence as a result) related to evidence-based implementation approaches. “Refresher” training sessions were suggested to maintain this knowledge throughout the course of the HaSP study. While supplementary training contents will be incorporated into the HaSP pre-phase teleconferences, options for a second training workshop (or splitting training contents into two staggered workshops) will be considered for future training efforts. Acknowledging the extensively reported time delays associated with hospital ethics and governance [33, 34], expediting or streamlining these processes to prevent delays in study progression may lessen the need for follow-up training sessions.

Some participants, however, found the theory-based training content more difficult to grasp. This is in line with other studies suggesting that the application of implementation frameworks and/or behavior change theory can at first appear an abstract and daunting task, within and outside the healthcare setting [1, 11]. Although the TDF is designed to provide a behavior change framework accessible to non-psychologists [2], some participants in this study struggled with these concepts, despite focused training. This finding supports the need for (a) partnerships with health psychologists and/or behavior change experts (where possible) to provide ongoing theoretical expertise and support [1], and (b) further efforts to make these approaches more accessible. The theory-based components of the TRANSLATE workshop will be reviewed and refined based on Implementation Lead feedback. End-user stakeholder input will be sought to inform these refinements and ensure content is clear, understandable and directly applicable in the clinical setting. Given that participants in this study valued applied clinical examples, additional scenarios may be provided to supplement the theory-based content. New or revised materials will undergo focused piloting with stakeholders prior to the delivery of future training workshops.

The impact of the TRANSLATE training workshop is likely to be enhanced when coupled with ongoing support from the implementation research team. Participants in this study valued opportunities to practice new skills gained through the training (e.g., process mapping, focus group facilitation, applying behavior change theory), and navigate any barriers or challenges encountered. Ongoing support from the research team is also likely to enhance fidelity to the prescribed implementation approach. Healthcare workers in this study acknowledged a desire to immediately search for solutions to the problem, overlooking crucial steps in defining target behaviors, determining barriers, and designing targeted intervention strategies. In light of efforts of training programs (e.g., TIDIRH) to enhance the use of evidence-based implementation methodologies, further training and support is needed to ensure that these methodologies are delivered as prescribed [24]. While adaptations may be necessary at times (particularly in the context of complex and dynamic hospital systems), these require careful and pragmatic balancing between fidelity and flexibility [35]. The provision of ongoing support by the HaSP research team facilitates this balancing act, allowing researchers and healthcare workers to discuss challenges as they arise and develop appropriate solutions. Implementation fidelity is being assessed as the trial progresses via mixed-methods process evaluation [36].

Employing and training healthcare workers to lead an implementation effort from within the system may be a more effective and sustainable approach for translation. In this context, the role of the Implementation Lead represents the construct of an implementation “champion”—that is, an individual dedicated to driving an implementation effort within an organization [37]. The ability to leverage the Implementation Leads’ knowledge of the system—including policies, procedures, networks, and politics—was seen as a factor that could contribute to the success of the HaSP study at their site. While champions are often expected to incorporate these activities into their existing clinical roles, the Implementation Leads in this study were specifically employed to oversee the implementation efforts. Having a dedicated, employed staff member leading the implementation approach was seen to alleviate potential workload burden for other staff, making them more willing to engage in the improvement effort. These findings are in line with a recent systematic review by Miech et al., which concluded that implementation champions are crucial for achieving implementation success in health systems [38]. Providing training, ongoing support, and adequate resourcing may provide additional benefits. Implementation Leads were unable to be recruited from within the hospital system at two sites due to hiring restrictions. The two externally recruited Implementation Leads experienced initial struggles navigating the system and gaining credibility. Similar findings have been previously reported, whereby university-based researchers attempting to drive a hospital implementation project from the outside experienced a number of challenges leading to stalled and, in some cases, unsuccessful implementation [3, 39, 40]. These findings further justify efforts to build partnerships between researchers and stakeholders within the system to maximize opportunities for successful implementation [41]. The impact of the Implementation Lead champions on the success of the HaSP approach will be further explored via post-implementation qualitative stakeholder interviews, and will be interpreted alongside trial outcomes at each site [36].

The Kirkpatrick framework also emphasizes the need for evaluation at the level of behavior and results [26]. Exploration of these levels were beyond the scope of this study due to the timing of the interviews, however an in-depth process evaluation is currently underway and will be conducted alongside all phases of the HaSP trial [36], allowing assessment of the impact of training on behavior and results. Kirkpatrick notes that the transfer of knowledge into behavior is dependent (among other factors) on the person working in the right climate [26]. Participants noted a number of barriers and facilitators (some of which were anticipated, others experienced in early HaSP trial phases) that could impact their ability to perform their role as an Implementation Lead, as well as the overall success of the HaSP trial (e.g., staff engagement, practical challenges, hospital culture). Participants also anticipated that the knowledge and skills gained through the training could be applicable to other contexts, therefore building longer-term capacity for implementation within the health system. The extent to which these observations evolved over the 18-month course of the trial will be explored in more detail via post-implementation process evaluation stakeholder interviews [36].

The contextual, structural and system barriers to implementation reported by participants in this study are unlikely to be overcome by training initiatives alone. The success of these initiatives are likely to be enhanced when coupled with broader approaches within the hospital system that build more durable infrastructures to support implementation and improvement efforts [6]. Examples include robust data measurement systems (enabling audit and feedback), staff incentives, sustained leadership, involvement of senior management, human resources practices, and building overall cultural receptivity to change [6]. Some of these approaches have been incorporated into the HaSP trial (e.g., audit and feedback, senior management involvement) to overcome system barriers and build capacity for implementation and change.

In the context of this particular study, one of the key aims of the TRANSLATE workshop was to upskill healthcare workers in the delivery of HaSP. However, the workshop was designed to be transferrable to other clinical contexts more broadly. Only the background and introduction component of the workshop was specific to Lynch syndrome and the HaSP trial—all other content and resources could be directly applied to address other clinical problems in the health system. Opportunities for adapting and upscaling the delivery of TRANSLATE are currently being explored. For example, while TRANSLATE was delivered face-to-face in this study, online modes of delivery (e.g., pre-recorded videos coupled with online interactive activities, self-evaluation and discussions forums) may facilitate scale-up efforts. Such adaptations will require further evaluation to ensure key learning outcomes are maintained.

This study is not without limitations. While we obtained a complete response rate, the overall sample size was small (reflective of the HaSP site numbers). Should the training be delivered on a larger scale, further evaluations will be conducted to enhance the validity and generalizability of our findings. Our evaluation did not include pre-post measures to assess changes in knowledge resulting from the workshop. Quantitative knowledge measures should be included in future evaluations to better assess whether learning outcomes were achieved. Furthermore, despite attempts to limit potential biases (e.g., interviews conducted by a member of staff who was not involved in training delivery, honesty and critical feedback encouraged in order to improve the training program) there is still the potential that Implementation Leads were more likely to provide positive feedback about the training given the ongoing involvement with the HaSP research team.

Nonetheless, we have described an implementation training workshop and model for ongoing support which, in the early phases of evaluation, has been received positively by healthcare workers. While there are a number of training programs focused on the design of evidence-based implementation methods, to our knowledge TRANSLATE is the first formally evaluated training program focused on preparing and providing ongoing support to healthcare workers for the application of these methods in the delivery of an evidence-based implementation approach. Suggestions for improvement will be taken into account in the delivery, and possible upscaling, of future training. Based on our findings, we propose the following recommendations:

1. Training efforts to enhance adoption of evidence-based implementation methods should be accompanied by efforts to enhance delivery of these methods among healthcare workers in practice.

2. Where possible, ongoing support should be provided by researchers to ensure application of theoretical frameworks and overall fidelity to the intended implementation program.

3. New and existing training programs should be evaluated so that researchers can partner with healthcare agencies to work toward building a system-wide training and support framework targeted to the specific needs of healthcare workers and organizations.

Conclusions

We have demonstrated early satisfaction among healthcare professionals who participated in a novel training program focused on the delivery of an evidence-based health system implementation approach. Participants anticipated that the knowledge and skills gained through the TRANSLATE workshop would be useful for their future work and could potentially be applied to other clinical settings. The real-world impact of the TRANSLATE workshop will be explored at completion of the HaSP implementation trial through in-depth stakeholder interviews.

Appendix 1 | TRANSLATE workshop contents

Session	Content	BC	IS
Background and rationale	Overview of Lynch syndrome	X	X
	Current literature on Lynch syndrome referral practices and barriers	X	X
	Aims of the Hide and Seek Project	X	X
	Introduction to implementation science	X	X
	Introduction to behavior change theory and Theoretical Domains Framework (TDF)	X
	Activity 1: In pairs, participants pick a behavior (e.g., healthy eating, exercise) and discuss barriers to performing the behavior	X
	Activity 2: In pairs, participants asked to consider own clinical practice and list five barriers to changing health professionals’ practice	X	X
Case study	Changing clinical practice: example case of nasogastric tubes	X	X
	Top-down vs. bottom-up approaches to implementation	X	X
	Description of intervention strategies to improve nasogastric tube placement practices and evidence of practice change	X	X
	Application of the TDF and Behavior Change Techniques (BCTs) in design of intervention strategies to improve nasogastric tube placement practices	X
Implementation Lead role	Overview of the Implementation Lead role, key tasks, and deliverables	X	X
	Strategies to generate and maintain stakeholder engagement	X	X
	Strategies to promote collaborative learning and inclusivity when conducting HaSP team meetings and focus-groups	X	X
Forming an Implementation Team (HaSP Phase 2)	Rationale, key steps, and recruitment strategies to form a multidisciplinary Implementation Team	X	X
Identifying target behaviors for change (HaSP Phase 3)	Explanation and rationale for identifying target behaviors for clinical practice change (with examples)	X	X
	Explanation of process mapping and rationale for use in identifying target behaviors	X	X
	Use and interpretation of process map and audit data to identify target behaviors, with hypothetical worked example	X	X
	Approaches for selecting from multiple candidate target behaviors (e.g., feasibility rating scales)	X	X
	Activity 1: participants work through a mixed list of goals and target behaviors and are asked to differentiate the two	X	X
	Activity 2: a member of the research team demonstrates the facilitator role in a group-based process mapping role play activity	X	X
	Activity 3: participants review example process map with audit data and use the information to identify potential target behaviors	X	X
Identifying barriers to change (HaSP Phase 4)	Rationale for barrier identification	X	X
	Use of TDF to identify barriers	X
	Strategies for facilitating focus-groups	X	X
	Facilitating a focus-group guided by the TDF	X
	Activity 1: participants match barrier questionnaire items to TDF domains	X
	Activity 2: participants watch a focus-group video and identify facilitator “do’s’ and “don’t’s”	X	X
Generating intervention strategies (HaSP Phase 5)	Rationale for designing targeted intervention strategies based on site-specific barriers	X	X
	Introduction to BCTs and rationale for theory-based approaches to intervention design	X
	Selecting BCTs to address TDF barriers, operationalizing BCTs as intervention strategies (plus worked clinical scenario)	X
	Designing intervention strategies to address intuitively identified barriers		X
	Selecting intervention strategies based on feasibility and impact ratings
	Activity 1: participants match a list of example intervention strategies to a list of BCTs	X
Supporting staff to implement interventions (HaSP Phase 6)	The importance of systematically planning intervention implementation, including stakeholder support and engagement	X	X
Evaluating interventions and assessing practice and culture change (HaSP Phase 7)	Rationale for audit and feedback approach	X	X
	Instruction on data extraction and linkage procedures	X	X
Process evaluation and record keeping	Process evaluation methods and their importance for understanding implementation outcomes
	Recommendations for record-keeping and handover

Appendix 2. Example telecon agenda

Phase 3 teleconference 1 agenda: Hospital X

Attendees: Cancer Council NSW Research Team, Implementation Lead

Time: DD Month YYYY 00:00 – 00:00 am/pm AEST

Dial in details: To join the teleconference, please dial XXXX XXX XXX and enter Conference ID

Attachments

Implementation Science Approach Toolkit (pp. XX-XX), Hospital X Process mapping meeting guide (Phase 3 Meeting 1), Project Log (template)

Meeting 1 pack (hardcopy documents to be provided by mail):

• -Large Post It notes
• -Whiteboard marker
• -Laminated process mapping cards and bluetack

Items

1. Review Phase 2

   1. Can you tell us about any deviations from the approach suggested in the Toolkit?

   2. Are you aware that anyone has changed their practice as a result of participating in a study activity?

2. Preparing for the meeting (Toolkit v1.3 pp.24–25)

   1. Implementation Team members recruited?

   2. Meeting time arranged—date?

   3. If you get stuck trying to line up calendars

      1. Try to ensure a range of professions/seniority for the meeting

      2. Consult with hard-to-catch clinicians before or after the meeting

   4. Room set up—whiteboard available?

   5. Review Phase 3 Resource B

      1. Process mapping symbols that will be used for this project

      2. Simplified Process map (yours will be more complex!)

3. Running the meeting (Toolkit pp. XX-XX)

   1. Hospital X Process mapping meeting guide

      1. Do you feel confident to use this meeting guide?

      2. Confident to keep the meetings collaborative and inclusive?

      3. Don’t forget to turn on the audio recorder!

         1. Teasing out decision points—use the “5 Whys” for step that do not always happen

            1. a process step that does not always happen suggests that there is a decision point beforehand—this is a red flag for a potential target behavior (which will be explored in Meeting 2)—try to dig down into these

   2. Process mapping cards

      1. CRC resection

      2. IHC test on tumor

      3. Abnormal MLH1 (with or without abnormal PMS2) in tumor

      4. Abnormal PMS2 (with normal MLH1) in tumor

      5. Abnormal MSH2 (with or without abnormal MSH6) in tumor

      6. Abnormal MSH6 (with normal MSH2) in tumor

      7. BRAF test on tumor

      8.  BRAF V600E mutation in tumor

      9. Referred to FCC

      10. Not referred to FCC

      11. (site-specific process mapping cards)

   3. Use Post Its for all other steps in the process and edit cards above as required

   4. Take a photo of the map at the end of the meeting, and create hand drawn sketch while it is still fresh in your mind

   5. Take notes about your perceptions or for explanatory notes to the process map. As you are recording the audio, notes should not be to be extensive

4. After the meeting

   1. We will review the audio, notes, sketch, and convert this to a computer drawn process map

      1. Expect to hear from us in the two weeks following the meeting to check and ask for clarifications

      2. Once designed, we will send the map to you. Please review this with some or all of the Implementation Team—

         1. Send us any final changes/feedback

5. Phase 3 Teleconference 2 and Meeting 2 dates

   1. Teleconference 2 approx. 6 weeks after we receive the data from Meeting 1

   2. Meeting 2 held following Teleconference 2 (e.g., 7 weeks after Meeting 1)

6. Participant Information Sheets, Consent Forms, revocation of Consent

   1. If you need more (send in Phase 2), please let us know

   2. Ensure all meeting attended have signed a consent form before recording the meeting

7. Review Project Log

   1. Word document template provided

   2. Please record staff roles/job titles rather than names (e.g., Data Manager, CNC)

8. Contact details if you need help

   1. HaSP email address

9. Actions

10. Any questions?

Appendix 3. Interview guide

Semi-structured Interview Guides: Implementation Lead experiences of training in an implementation approach aimed at improving referral of colorectal cancer patients with a high risk of Lynch syndrome

Introduction and informed consent: 10 min

Introduce topic: Improving detection of colorectal cancer patients with a high risk of Lynch Syndrome

Define problem: Suboptimal FCC referral rates for colorectal cancer patients with a high risk of Lynch syndrome, missed opportunities for early cancer detection/prevention amongst at-risk relatives

State purpose of interview (e.g., process evaluation conducted alongside implementation of an intervention to improve LS detection)

Will be recording the discussion which will be transcribed by a third-party transcription service

Voluntary participation and right to withdraw without giving a reason

Info about practitioner

Request brief introduction, including role and experience of practitioner

Can you give me an overview about your current role in the hospital and any relevant prior experience?

How long have you been based at <hospital name>?

Can you talk me through your understanding of the Lynch syndrome identification and referral process at your site?

How confident are you that 100% of patients at high risk of Lynch syndrome are being referred to genetic services?

Have these referral rates ever been formally assessed? If so, can you explain how this was done?

Can you talk me through the reasons why your hospital initially decided to take part in the study?

Experiences of the implementation training package

• What were your general impressions of the training package?

• Can you explain your understanding of what implementation research is?

• Prior to the training day, can you explain what your understanding was of the purpose of the HaSP trial?

• Did that understanding change at all as a result of the training day?

  • How so?

• How easy/difficult was it to engage with the resources provided?

• Can you comment on the length of time through which the training was delivered?

• How confident do you feel in your ability to perform the roles of the implementation lead (as described in the training package)?

• How well do you feel the training package was delivered by the researcher(s)? Did you feel as though your questions were satisfactorily answered?

• Were there any particular areas that you found difficult to grasp?

• Are there any particular aspects of the approach that you think might already be occurring intuitively in the clinical setting?

• Can you think of any barriers that you might encounter performing the role of an implementation lead?

• Having received the training, to what extent would it change the way you approach other implementation problems in the future?

• Can you briefly talk me through how you will go about each step? (interviewer to provide name of each step to prompt)

(Theory-arm only):

• The TDFI approach is based in behavior change theory. Can you comment on what you have learnt about behavior change from your experience of the training package?

• Do you think the behavior change approach makes sense for a project like this?

• Can you comment on the process mapping exercise? How useful do you think it will be?

• Do you have any suggestions about how the training can be improved?

• Do you have any other comments you would like to make about your overall experience of training in the implementation approach?

For sites that have withdrawn from the trial

• Can you talk me through the key factors underlying your hospitals decision to withdraw from the study?

• Do you have any suggestions for how we might be able to overcome these barriers for future studies to improve Lynch syndrome detection?

• Can you elaborate on how you and your colleagues thought through the decision to withdraw from the trial in relation to the potential value it could have for your patients?

Appendix 4 | Representative quotes

Theme	Subtheme	Representative quotes
Training day feedback	Overall satisfaction	“It was really helpful to actually sit down and get a really good idea of exactly what I was meant to be doing because it was a bit nebulous before then, and so I’ve had this really vague idea of where we were headed but have an idea of the more of the detail about the implementation science and what that was going to look like and what some of the evidence behind it was, was really interesting.” Participant 1 (theory group)
		“My first impression was the comprehensiveness and thoroughness of the training that it seemed to me really well thought out and particularly with the documentation that was provided with it, which I guess also then gave me the impression that it would be the kind of position and the role would be well-supported. So that was – I guess kind of gave me quite a lot of confidence on the day.” Participant 3 (theory group)
		“I just thought it was highly organised and very coordinated. I guess I came into the day, it was all very quick me being appointed this role so I didn’t have much background, so I came in a bit blind, not knowing too much about things. But I mean overall it was such a thorough day and they’re very interactive, so it was a really good day.” Participant 4 (theory group)
		“The way it was structured, looking at the background work that was done, I suppose almost like a ‘needs assessment’ for the project, if you will, and sort of getting a good understanding of implementation research. So that was something that was quite new to me, but I found it really well-explained and I really understood how it related to the current project.” Participant 5 (non-theory group)
		“I felt that this is a really good training package. I thought there wasn’t any wasted time. I felt that it was a really valuable.” Participant 6 (non-theory group)
		“It felt like the training package was very professionally delivered, but at a friendly level where questions were encouraged and anything that I asked at the time was really nicely explained and it gave me high confidence in that. What was being presented was really well-thought-out.” Participant 8 (theory group)
		“I found the training package really fantastic, to be quite truthful. I know we had to fill out a feedback form at the end and I struggled to think of things that I didn’t find satisfactory or anything like that. It was very thorough. It was very well set-out and very logical.” Participant 7 (theory group)
	Workshop format and content	“I thought there was a good variety of resources. It was really nice to be able to sit down face-to-face and go through the training day. The written information was good but it’s nice to have a bit of both and some of the realistic examples that I gave of how it’s been used in the past made it really, a lot easier to understand how actually is applied, so I found that quite useful. So combination of reading and presentation and visual and actually interacting with people on the day was really helpful as well.” Participant 1 (theory group)
		“There was a lot of time and opportunity for people who were attending to ask questions, to make comments, which I thought was great, because as I said, I don’t have a lot of experience in this area. And so, it sort of wasn’t like sitting in a lecture. It really was sort of a roundtable discussion and I think that really helped get the message across.” Participant 5 (non-theory group)
		“I was probably most impressed by the fact of how comprehensive it was and I feel like that deep level of detail was really actually – was really important to the success of it.” Participant 3 (theory group)
		“There was a lot of content that was delivered and I felt that it was delivered at a pace that was easy to absorb. It wasn’t overwhelming by any means, but it was still engaging.” Participant 7 (theory group)
		“The implementation training day toolkit, that sort of mirrors my perception of the presentation that was given. It’s really well set out, quite comprehensive, but easy to refer to. So I know exactly where I’m going when I have to undertake each step and understanding why each part is important and how it all is put together in the grand scheme of the project over the few years that it’s going to be happening. So, overall, I thought it was really great, easy to reference, and I feel it’s a resource that I’ll obviously be using and referring to a lot as I implement the project.” Participant 5 (non-theory group)
		“To spend a little bit more time on, those theoretical domains and how they’re targeted and how the – and I don’t know if this was because of the blinded nature of the study – but how the intervention approaches could be mapped to the theoretical domains framework. I felt that my understanding of that was still a little bit lacking” Participant 8 (theory group)
	Workshop duration	“It seemed like the right amount of time to get through the information. There were good number of breaks in it, but it was a full day with a lot of other things going on, but after the actual training itself, it didn’t seem too long or too short.” Participant 1 (theory group)
		“It seemed adequate for the amount of information that needed to be covered.” Participant 2 (non-theory group)
		“It was a full day or close enough to a full day’s worth of training in Sydney and that seemed like the appropriate amount of time. I mean, there was enough time to kind of do some activities, to talk about what the role would be and the position and do training around what would be required… I thought the length of time was pretty spot-on.” Participant 3 (theory group)
		“Perhaps just splitting it over two days, perhaps one day and really just focused on a lot of the background, just having that opportunity to really get a grasp of why are there and then perhaps once that’s dunked in the next day, really just drilling down into as an implementation, what your roles and responsibilities would be.” Participant 4 (theory group)
		“There were regular breaks, which I thought was really good and we could get to know everyone involved in the project. We finished sort of about maybe 3:30, four o’clock, and so it didn’t actually seem like a really long day, but I know that we covered a lot of ground. So I don’t think it would need to be any longer and I don’t really see reason for it to be shorter because I think we had a lot of really good discussion and within the implementation leads and also with the research team.” Participant 5 (non-theory group)
		“I think that length of time is possibly just about right. You wouldn’t want it any longer because it does start to become a long day, but at the same time, I believe the length of time that was utilised is right for the amount of overview for the project that was needed to begin with.” Participant 8 (theory group)
		“ I think it was a suitable amount of time. There was a lot of content that was delivered and I felt that it was delivered at a pace that was easy to absorb. It wasn’t overwhelming by any means, but it was still engaging. So, I found that a day was a suitable amount of time.” Participant 7 (theory group)
		“Ot’s really hard when you’re delivering a session and you’re pulling people from all over – Australia-wide and I think that’s really hard because from a resource point of view, you’ve got to do what you’ve got to do in the short amount of time and people being time-poor with work. I think probably you need to cut the content down a bit. I think you can deliver it over a day.” Participant 9 (non-theory group)
	Timing of the workshop in relation to study rollout	“It’s almost more of a timeframe issue in that it’s really useful to have a general idea early on and you get yourself all geared up and exited but then there’s often a long timeframe before you actually have to apply any of that knowledge…but I think we’ve been set up well to have something really useful to go back to.” Participant 1 (theory group)
		“Perhaps because of the length of the project, running over potentially two, two and a half years, I feel as though follow-up trainings or – I mean it was nice to have a whole overview of it on that day, but obviously, once I get to even trying to remember what the phases four and five and six – I mean that seems like in the far future and perhaps when I get to that phase, that training will have been over a year before. So, I wonder if perhaps some of the training, not to necessarily change that first day of training, but to think about how that might be refreshed or updated along the way.” Participant 3 (theory group)
		“I think it’s always quite difficult to think of things in the future. It’s kind of an abstract sort of thing, but I think that’s only perhaps my inexperience or unfamiliarity with implementing the project at this stage. So I think once we get to it, once we’ve implemented all the prior phases, I think it will be a lot clearer as to how to do that later on…I feel kind of prepared and know that at the moment, I’m quite confident with implementing say, phase one and phase two, and then even if I’m not that confident, I perhaps don’t have the brain space for say, three to seven at the moment, I know that when I get to those points, there’ll be an opportunity for me – I won’t just have to rush into them.” Participant 5 (non-theory group)
		“I guess the only thing that could be slightly overwhelming, but at the same time, I think there’s no way – ifs or buts about it – you have to present a good overview of the entire project and I think that the subsequent assurance that there’s gonna be individual teleconferences for each phase as it proceeds in order to clarify and ask questions, because some things you just don’t know ahead of time, I think everything there felt very reassuring. Even if I’m not so crystal clear on certain aspects of it, I know that I will be as I go along.” Participant 8 (theory group)
		“We’ve got to a spot where we’re coming up to a year down the track, and it’s just so far behind us now. When you’re ready to up and go and get the ball rolling, I think you have to have all your dots in line and this needs to come at a time when we’re like a month’s out, “Okay, in four weeks, we’re ready to go. Everything is in place. We’re ready to go. This is it.” Participant 9 (non-theory group)
	Ongoing researcher support	“I think you’ve got an amazing and very supporting team at Cancer Council NSW and I never felt like I was asking stupid question. I’ve always felt like everybody was really accessible and really supportive.” Participant 1 (theory group)
		“I remember on the day feeling a little bit nervous about, things like leading a focus group of a room full of high level professionals. I thought that would be a bit daunting but I think what I walked away with from the day is that I felt very confident that with each phase, there was just going to be so much support and guidance that it really took away any of that anxiety I guess for some of those bigger tasks. It was definitely reassuring to know that there was a team of people there who were going to guide me with each step.” Participant 4 (theory group)
		“I know that I can also ask the research team at any time. I can email them or speak to them and get their advice and just clarify things as well… if I do have any problems, I know who I can contact and I think that’s probably the most important thing – is knowing that if I do have problems, I know where I can go.” Participant 5 (non-theory group)
		“Before I started with the training package, I was a little bit uncertain as to my exact role and how I’ll perform it, but with the training package, it’s given me, I guess, a high confidence in performing the role because I know that there is a lot of support in doing so and that it’s not – I don’t feel alone, that I’ve actually got people to ask questions of and clarify and brainstorm with, if needed” Participant 7 (theory group)
		“ The most intimidating part for me – is this idea of building a team of doctors from different disciplines within the hospital. But that being said, it was still – I felt like there was very kind of – a plan in place, a stepwise kind of fashion. You’re gonna send out these emails, you’re gonna do this, and you’re gonna meet with them. And so, while it was maybe a little difficult to imagine exactly how that was gonna pan out in the real world, I still felt like there was a lot of support.” Participant 3 (theory group)
Learning	Implementation knowledge	“My understanding before and after the session just grew exponentially. So I gained a lot out of it and having the documents to take away with me as well allowed me to reflect and to sort of cement that understanding that I gained.” Participant 7 (theory group)
		“A better insight into exactly what the project was, what the goals of the project were, and I think perhaps my role in the project and even maybe more so in a sense that I was in my role as implementation lead was sort of part of the project itself, which I probably didn’t have a really good sense of that before. So that was – I felt like I definitely thought differently of it before and after.” Participant 3 (theory group)
		“Having that foundation from the training day, it’s gonna mean that it’s familiar when I go back to it rather than trying to figure it out myself from a book” Participant 1 (theory group)
		“Coming up on 12th of July (training day) was total immersion for me. And I really enjoyed reading more about it and that sort of thing. And I suppose since the 12th of July, I’ve been doing planning in my own brain about things I would like to do carrying the implementation forward.” Participant 6 (non-theory group)
		“It’s certainly given me a lot more knowledge about implementation science than I had to prior to coming to being involved with this study. I think it’s given me a better understanding of how interventions can be made more effective and the different ways that we can target interventions to specific behaviours and the more pivotal behaviours to achieve the outcome that we want.” Participant 7 (theory group)
		“I suppose the framework that will be used – the theoretical domains framework – was all quite a new concept to me, but I found it very interesting and on review, I was able then to understand it a little bit better and apply that and understand that in the context of the study. But that was probably the most challenging part for me to get my head around.” Participant 2 (non-theory group)
		“I have a better understanding of implementation science, behavioural change, and the way to approach that process.” Participant 3 (theory group)
	Skills	“Having done the training session with the Cancer Council NSW, my confidence in my ability to apply those skills, and to gain new skills, and perform the role has definitely grown as a result of the training session. It sort of broke everything down in a way that was approachable and I think something that is very achievable for me.” Participant 7 (theory group)
		“I had a lot of those skills to back me up already, but it was really nice to see them I guess organised into the coherence structure that I could actually refer to and apply and it would be useful to be able to refer back to it as I go through the project as well. It will be my first time using something structured rather than whinging it with my own ideas as to how we might make things happen.” Participant 1 (theory group)
		“The process mapping was quite useful. That was the main thing that I picked up but obviously all the information presented was relevant and useful to the study and training.” Participant 2 (non-theory group)
		“Having that interactive discussion about thinking of some of the other possibilities, it was really useful just to see how that can be done and how you can work with the wider group to achieve it.” Participant 4 (theory group)
		“There is always an amount of apprehension when you’re doing something that you haven’t done before, but given the training, I feel reasonably prepared.” Participant 3 (theory group)
Implementation barriers and facilitators	Staff engagement	“I think the biggest barriers are gonna be working with some of the staff here at the hospital because I think the levelling of engagement is gonna kind of run the whole gamut of how much people want to invest both their time and interest in this project.” Participant 3 (theory group)
		“It takes so long between getting things like getting governance and then getting to the point of having enough data to be able to run the implementation meetings, and so it’s just really stretched out. So keeping that network alive and keeping people engaged without overwhelming them is really important.” Participant 1 (theory group)
		“I think probably within our hospital, the main barriers will be I think getting a good large enough group involved and getting people on board and getting them enthusiastic.” Participant 7 (theory group)
	Hospital culture	“I think our culture here in terms of generating ideas, particularly the colorectal department is quite good. Research is really well-embedded in everything that we do and I believe all our colorectal surgeons are involved in research, our nurses, and everyone from interns to staff specialists are quite involved. So I think they would be quite open to implementing new ideas.” Participant 5 (non-theory group)
		“Inertia is what I think of a lot. Any change takes energy and that’s why so many people keep doing things the way they’ve always done It, cause they’ve always done it that way for the last 20 years. You need to put something into it to generate change, even if the outcome might be more productive or more straightforward or simpler than what you have been doing. The actual change itself is the hard part.” Participant 1 (theory group)
		“I think they’ll respond very well. We have, at least within the Department of General and Colorectal Surgery, a very research involved group, so I think that the response will be largely positive and people will be more than happy to facilitate this study.” Participant 2 (non-theory group)
	Practical challenges	“I don’t think approaching or having access to the people will be an issue. I think they’re very accessible, easy to speak to, but I think perhaps getting people in the same place to talk about things for an hour or two is probably quite challenging.” Participant 5 (non-theory group)
		“I think one of the bigger barriers might be really trying to pull that multidisciplinary team together…Everyone is busy and it’s really just trying to time that approach, that I can bring everyone together and everyone are really keen to be there and be involved.” Participant 4 (theory group)
		“Everybody believes they’re so busy. These days, everybody believes that they don’t have time” Participant 6 (non-theory group)
		“I think time will be a fairly large factor for our clinicians in particular. But I think that we will find ways to overcome that.” Participant 7 (theory group)
		“Getting everyone together and I think it will be everyone’s problem. Everyone is a bit time-poor and every consultants and specialists are more important. So getting everyone together at the same time, it’s probably going to be a little bit harder.” Participant 9 (non-theory group)
	Implementation Lead embeddedness within the system	“I’m somebody coming in from outside and that can sometimes get people backs off, especially if there were major changes or significant things that needed to happen, as well as getting the buy-in from people to give you some of their time - even though we’re not asking for a lot.” Participant 1 (theory group)
		“I turn up to the MDT [multidisciplinary team meeting] every week to make sure that people see my face. I sit at the table. I make sure that I’m in contact with the team on a really regular basis…and so they see me as someone who is part of the team and is involved and they are willing to give me some airtime.” Participant 1 (theory group)
		“I’m fortunate that I work in the colorectal team. The surgeons are very familiar with me, so that won’t be an issue. I think being known is useful.” Participant 6 (non-theory group)
		“I’m not quite as familiar with all the clinicians, or the surgeons, or the admin staff, or nursing staff that are directly part of the colorectal surgical pathway or even their referral pathway.” Participant 8 (theory group)
Building health system capacity f or implementation	Value of the Implementation Lead role	“I said, “My anticipation is that I’ll be the person doing the leg work and basically I’ll be asking you for your help along the way, but I won’t be expecting you to do much in the line of legwork apart from what your specialty area is” and thus far, those people all seem to love to be part of it.” Participant 6 (non-theory group)
		“There’s always a bit of a hot-potato effect with clinical teams when it’s gonna take someone a bit of extra time or a bit of extra effort. As much as someone might want something to be different or can see a better way of doing something - whose responsibility is it to make it happen? So I think that having someone dedicated to doing the implementation is really helpful.” Participant 1 (theory group)
	Transferability of implementation knowledge and skills to other settings	“I think having that evidence base behind it and sort of understanding whether it’s a big project or a little change, actually planning it out properly instead of just sort of springing it on people. So I think if, for example, we were implementing some completely new research system or something like that, bringing in from the top to the bottom the consultants and the staff, especially right down to interns, who might be the ones who are actually filling out forms, anything that’s new, just sort of getting the people who are going to be really involved with it, to have their say.” Participant 5 (non-theory group)
		“I think it’s just given me more of a context and an understanding to be able to approach new problems in my workplace in the future.” Participant 7 (theory group)
		“It gives me a foundational understanding of the processes that can be followed to reach an effective strategy. And so that can be applied across multiple settings.” Participant 2 (non-theory group)
		“I think this is going to be beneficial long term. Just to have that appreciation of that overarching approach, even just mapping processes and just those sort of skills are just going to be utilised I guess throughout any role from here, so I think it’s gonna be really valuable.” Participant 4 (theory group)
		“I think having the training, I guess, have opened my eyes a little bit to the approach to implementing anything in the hospital. So it’s not simply putting people in a group together and deciding what’s the best course to changing or implementing a change in the hospital – it actually needs to be more thoroughly investigated as to what is actually occurring and then actually looking at what everyone’s skills and maybe what’s preventing any change from occurring. And I guess the training has helped open my eyes to that. It’s not a simple process necessarily if you want it to be successful, that a lot more thought and discussions and actually, I guess, mapping out what behaviours need to be changed and what barriers may exist to that change and a lot more thought needs to go in – towards them in order to successfully implement any changes in the future.” Participant 8 (theory group)
		“It’s kind of bolstered my ideas, systems and processes of my design” Participant 6 (non-theory group)

Compliance with Ethical Standards

Conflicts of Interest: All authors declare that they have no conflicts of interest.

Human Rights: All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Ethical Approval: Ethical approval has been granted for this study by the Royal Prince Alfred Hospital Human Research Ethics Committee (reference HREC/17/RPAH/542).

Informed Consent: Informed consent was obtained from all individual participants included in the study.

Welfare of Animals: This article does not contain any studies with animals performed by any of the authors.

Authors’ Contributions: AM led the development of the manuscript, contributed to the development of training materials, and led the workshop evaluation study design and analysis of interview transcripts. N. Taylor, E. Hogden, J. Steinberg, P. Chan, G. Tiernan, L. Wolfenden, and D. Debono all contributed to the development of training materials, with NT (alongside colleagues from the UK) having led the development of the original Achieving Behavior Change for Patient Safety materials (Yorkshire Quality and Safety Research Group, Bradford Institute for Health Research) from which some HaSP training materials have been adapted. PC conducted post-workshop interviews, and contributed to qualitative analysis. All authors read, revised, and approved the final manuscript.

Transparency Statement

The lead author (the manuscript’s guarantor) affirms that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.