Quality improvement lessons learned from National Implementation of the “Patient Safety Events in Community Care: Reporting, Investigation, and Improvement Guidebook”

Abstract

Objective

To evaluate nationwide implementation of a Guidebook designed to standardize safety practices across VA‐delivered and VA‐purchased care (i.e., Community Care) and identify lessons learned and strategies to improve them.

Data Sources and Study Setting

Qualitative data collected from key informants at 18 geographically diverse VA facilities across 17 Veterans Integrated Services Networks (VISNs).

Study Design

We conducted semi‐structured interviews from 2019 to 2022 with VISN Patient Safety Officers (PSOs) and VA facility patient safety and quality managers (PSMs and QMs) and VA Facility Community Care (CC) staff to assess lessons learned by examining organizational contextual factors affecting Guidebook implementation based on the Consolidated Framework for Implementation Research (CFIR).

Data Collection/Extraction Methods

Interviews were conducted virtually with 45 facility staff and 10 VISN PSOs. Using directed content analysis, we identified CFIR factors affecting implementation. These factors were mapped to the Expert Recommendations for Implementing Change (ERIC) strategy compilation to identify lessons learned that could be useful to our operational partners in improving implementation processes. We met frequently with our partners to discuss findings and plan next steps.

Principal Findings

Six CFIR constructs were identified as both facilitators and barriers to Guidebook implementation: (1) planning for implementation; (2) engaging key knowledge holders; (3) available resources; (4) networks and communications; (5) culture; and (6) external policies. The two CFIR constructs that were only barriers included: (1) cosmopolitanism and (2) executing implementation.

Conclusions

Our findings suggest several important lessons: (1) engage all collaborators involved in implementation; (2) ensure end‐users have opportunities to provide feedback; (3) describe collaborators' purpose and roles/responsibilities clearly at the start; (4) communicate information widely and repeatedly; and (5) identify how multiple high priorities can be synergistic. This evaluation will help our partners and key VA leadership to determine next steps and future strategies for improving Guidebook implementation through collaboration with VA staff.

What is known on this topic

• Little is known about whether implementation of the “Patient Safety Events in Community Care: Reporting, Investigation, and Improvement Guidebook” was effective in standardizing safety practices across VA‐delivered and VA‐purchased care settings (i.e., Community Care).

What this study adds

• This project identified several important lessons learned and strategies to improve standardization of patient safety practices/processes between care provided within VA and care provided in community settings.

• Developing collaborator relationships and supporting ongoing training and education are key strategies learned from this study that may be applicable to other implementation initiatives.

1. INTRODUCTION

The Veterans Access, Choice and Accountability Act of 2014 (“Choice Act”) and the subsequent Veterans Health Administration (VA) Maintaining Internal Systems and Strengthening Integrated Outside Networks (MISSION) Act of 2018 were passed to ensure that Veterans have timely access to high‐quality care through increased use of services purchased by the VA in the community. ¹ , ² , ³ Passage of the Choice and MISSION acts precipitated a major transformation in the way VA delivers care, changing VA's role from that of a provider to that of both a provider and purchaser of care. Although the majority of care for Veterans is still delivered in VA facilities, utilization of VA‐purchased Community Care (CC) in non‐VA facilities has increased rapidly over time. More than 30% of the care provided to Veterans is delivered through VA‐purchased care in the community. ⁴ In 2023, approximately $28 billion dollars was spent on VA‐purchased care. ⁵

At present, there is limited knowledge about the quality of care Veterans receive in the community compared with the quality of care available in the VA. The few studies available in the literature in this area, however, suggest that overall, VA provides better quality of care than that purchased in the community, with respect to surgical care and other care specialties. ⁴ , ⁶ , ⁷ , ⁸ , ⁹ , ¹⁰ , ¹¹ , ¹² , ¹³ Even less is known about the safety of care in the community, and how it compares with that in the VA. As more Veterans increase their use of CC, the consequences resulting from care fragmentation are also likely to grow, ¹⁴ , ¹⁵ , ¹⁶ potentially leading to worse health outcomes, including increases in numbers of patient safety events. ¹⁷ , ¹⁸

The VA Office of Integrated Veteran Care (formerly VA Office of Community Care) leadership established a Quality and Patient Safety workgroup which identified several major areas as “safety gaps” associated with the reporting and investigation of CC patient safety events and improvement of care processes that led to such events. ¹⁹ , ²⁰ A key recommendation of the workgroup called for Office of Integrated Veteran Care to implement standardized, evidence‐based processes that would decrease barriers to improving quality and safety, increase communication between VA and CC providers, and delineate clear roles and responsibilities for VA staff, including those VA employees who are liaisons with CC providers (“VA Facility CC staff”). ²¹ Given recognition of VA's strong safety culture and system for tracking and reporting safety events through the Joint Patient Safety Reporting system, ²¹ , ²² , ²³ The Office of Integrated Veteran Care sought guidance from VA National Center for Patient Safety leadership on how to most effectively align existing VA safety policies and processes across the two care settings (VA and CC).

Subsequently, this collaboration with National Center for Patient Safety became the impetus for developing the “VA Patient Safety Events in Community Care: Reporting, Investigation, and Improvement Guidebook” (“Guidebook”), with the primary goal to address CC safety events that involved community providers who were part of a VA Community Care Network (CCN). ²⁴ The Guidebook is now considered the main strategy and blueprint for standardizing implementation of safety processes across the spectrum of inpatient and outpatient VA‐delivered and VA‐purchased care. The Guidebook is based on existing VA patient safety reporting and investigation processes for VA‐internal care (VHA Directive 1050.01‐VHA Quality and Patient Safety Programs) and is regularly reviewed, updated, and disseminated to VA facility and VA regional networks (i.e., Veterans Integrated Service Networks [VISNs]) staff by our operational partners (Office of Integrated Veteran Care and National Center for Patient Safety). Additionally, National Center for Patient Safety shares patient safety event data with Office of Integrated Veteran Care on a quarterly basis. The Guidebook provides expectations to VA facility directors and staff about standardized patient safety event reporting, tracking, and robust monitoring of safety events at the facility level in both VA‐delivered and VA‐purchased care settings. It also allows staff to take corrective actions to mitigate significant system vulnerabilities. Further, the Guidebook contains the specific steps necessary for VA Facility staff and CC staff to integrate National Center for Patient Safety's existing safety processes into their regular workflows and explicitly describes the roles, responsibilities, and required tasks of VA and VA Facility CC staff in order to facilitate better communication and information sharing between staff across settings. ²⁴ For example, VA Patient Safety Managers (PSMs), VA Facility CC staff, and Quality and Safety representatives from the CCN Third Party Administrators (i.e., companies contracted by VA to create a regional network of community care providers) are expected to work together to review all reported CC events, examine local‐level trends in CC patient safety events reported in Joint Patient Safety Reporting system, and collaborate on actions to improve safety investigations. ¹⁰

In 2019, the Quality Enhancement Research Initiative (QUERI) funded the “VA Implementation of Patient Safety Practices in Community Care” Partnered‐Evaluation Initiative project, in collaboration with Office of Integrated Veteran Care and National Center for Patient Safety to evaluate the implementation of the Guidebook throughout VA care settings and provide findings to more rapidly translate research into practice. The goals of this paper were to (1) assess the organizational barriers and facilitators related to Guidebook implementation using the Consolidated Framework for Implementation Research (CFIR) framework, a commonly used framework to assess contextual determinants; and (2) identify lessons learned during Guidebook implementation and develop strategies for improving approaches for future Guidebook implementation.

2. METHODS

2.1. Project design and setting

This project was a component of a larger observational mixed‐methods evaluation to examine the impact of the Guidebook on VA safety reporting practices and occurrence of patient safety events. The VA Boston Healthcare System's Institutional Review Board and Research and Development Committee determined that the work conducted under this Partnered‐Evaluation Initiative was quality improvement and therefore exempt from review by the local Institutional Review Board.

2.2. Site selection and sample

Sites were selected in collaboration with our operational partners as well as recommendations from Patient Safety Officers (PSOs) from 17 of the VA Veterans Integrated Services Networks (VISNs) (i.e., regional systems of care). Site selection criteria included geographical diversity and size (i.e., number of inpatient beds). Once sites were selected, our operational partners sent out a jointly supported communication to all PSOs to inform them that our evaluation of the Guidebook was about to launch. VISN Directors disseminated this information to their facilities and staff. Our team then followed up with the VA facilities that were selected and began recruitment of participants. Our goal was to interview key informants (i.e., PSMs, Quality Managers [QMs], and VA Facility CC staff) at 18 VA facilities, as well as the PSOs in the nine VISNs where the 18 facilities were located. At the VA facility level, PSMs serve as primary point of contact for patient safety event‐related actions; and QMs support the ongoing assessment and improvement of healthcare outcomes and delivery processes. VA Facility CC staff, who directly support care coordination for Veteran community care, directly assist with the coordination of care for Veterans receiving VA‐purchased care. The VISN‐level PSOs serve as primary point of contact for patient safety event‐related processes and procedures within the VISN. ²⁴ , ²⁵

2.3. Data collection

2.3.1. Semi‐structured interview guide

We developed a semi‐structured interview guide (Table 1) with questions that focused on eliciting information on the implementation of the Guidebook safety processes (i.e., those processes outlined in the Guidebook related to safety reporting, investigation, and improvement). We added two other questions on organizational contextual factors from the CFIR to the interview guide, because they were also likely to affect implementation of Guidebook safety processes. These two CFIR constructs, “networks and communications,” and “culture,” highlighted key factors identified in the literature as important to patient safety initiatives. ²⁶ , ²⁷ For example, to obtain information about networks and communications, staff were queried: “Can you describe your working relationships with your colleagues who are responsible for implementing the CC safety processes?” To capture any additional information, we included an open‐ended question broadly asking if there were any other factors that affected Guidebook implementation.

TABLE 1.

Interview guide questions.

Guidebook safety processes	Question
Reporting	Walk me through an example on your experience related to the reporting process that is outlined in the Guidebook. What did you find challenging about this experience? Can you give some examples of what went well? In what ways could this process be improved?
Investigation	Walk me through an example on your experience related to the investigation process that is outlined in the Guidebook. What did you find challenging about this experience? Can you give some examples of what went well? In what ways could this process be improved?
Improvement	Walk me through an example on your experiences related to the improvement process that is outlined in the Guidebook. What kinds of improvement activities have been initiated (if any)? Can you give some examples of what went well? In what ways could this process be improved?

Consolidated framework for implementation research (CFIR) constructs	Question
Networks and communications	Can you describe your working relationships with your colleagues who are responsible for implementing the Community Care safety processes?
Culture	How do you think your organization's culture has affected the implementation of the Community Care safety processes outlined in the Guidebook?
Any other	Are there any other factors (i.e., facilitators or barriers) that have affected Guidebook implementation that we haven't specifically discussed?

2.3.2. Interviews

From February 2020 through January 2022, we conducted virtual 45‐minute, semi‐structured interviews using the Microsoft Teams platform. This included 21 individual and 11 group interviews consisting of 2–3 VA facility‐level participants. Group interviews occurred if participants said they preferred to be interviewed with colleagues from the same department. When conducting group interviews, we prompted each participant on their perceptions (e.g., “Would you like to share your thoughts?”); this method created an opportunity for all individuals to provide feedback. VISN‐level PSOs were interviewed individually. Interviews were digitally audio‐recorded with participants' agreement and professionally transcribed.

2.4. Data analysis

Verbatim transcripts served as the primary source for data analysis. Two of the project team (JS and MHS) used a directed content analysis and deductive approach guided by the CFIR constructs to identify emergent organizational factors affecting Guidebook implementation. ²⁶ , ²⁸ Using an iterative process, they coded one transcript separately and then jointly identified CFIR constructs affecting Guidebook implementation (i.e., facilitators and barriers) reaching consensus on the codebook definitions and coding excerpts. Thereafter, they double‐coded all transcripts using NVivo (version 12) qualitative software and met regularly to discuss and reach consensus on any coding differences.

Coding reports were generated for each of the constructs in the data analysis. Project authors used a matrix to consolidate and organize the data per construct, site, and participant. ²⁹ This matrix was then reviewed to compare and contrast evidence across sites to determine the factors that emerged as facilitators and barriers to Guidebook implementation. ³⁰ A summary report describing the findings was produced through this process.

Once the CFIR facilitators and barriers were identified, JS and MHS mapped the CFIR constructs to the Expert Recommendations for Implementing Change (ERIC) implementation strategies. The ERIC strategies are used frequently in implementation studies to facilitate successful implementation through a list of recommended activities that have been shown to improve the implementation of a program. ³¹ Implementation strategies enabled our partners to develop approaches that could leverage facilitators as well as address barriers to improve Guidebook implementation.

3. RESULTS

3.1. Site characteristics and participants

Table 2 presents the characteristics of the sites recruited for the interviews and the number of participants interviewed at each VA site. Geographic distribution of the 18 sites varied, with the fewest number of sites from the Northeast region and the greatest number of sites from the Midwest and West regions. Size of the 18 facilities also varied, ranging from 36 to 678 inpatient beds.

TABLE 2.

Site characteristics and number of participants interviewed at each site.

Site	Geographic region*	Number of inpatient beds (size)	Patient safety manager	Quality manager	Community care staff	Total
1	Northeast	51	1	0	2	3
2	Northeast	471	0	1	0	1
3	South	271	1	0	3	4
4	South	159	1	0	2	3
5	South	678	3	0	2	5
6	South	370	2	1	0	3
7	Midwest	250	1	0	2	3
8	Midwest	500	1	1	2	3
9	Midwest	431	2	1	0	3
10	Midwest	135	1	0	0	1
11	West	83	0	1	2	3
12	West	127	0	1	0	1
13	West	306	1	0	2	3
14	West	36	1	0	0	1
15	West	311	2	0	0	2
16	West	295	1	0	0	1
17	Midwest	110	1	0	1	2
18	Midwest	83	1	1	1	3
Total			19	7	19	45

Note: Geographic region was determined from using: * https://www.census.gov/programs‐surveys/economic‐census/guidance‐geographies/levels.html#par_textimage_34.

We interviewed a total of 45 key informants: 19 PSMs, 7 QMs, and 19 VA Facility CC staff at 18 VA facilities. Additionally, we interviewed 10 VISN‐level PSOs from 10 of the 17 VISNs who responded to our queries about participating in the interviews.

3.2. CFIR facilitators and barriers

Table 3 describes CFIR construct definitions, key themes based on participants' perceptions, and exemplar quotes. ¹⁴ Key CFIR constructs included six facilitators and eight barriers that affected Guidebook implementation; all except two of the CFIR constructs acted as both facilitators and barriers. The six CFIR constructs that were both facilitators and barriers to Guidebook implementation included: (1) planning for implementation; (2) engaging key knowledge holders; (3) available resources; (4) networks and communication; (5) culture; and (6) external policies. The two CFIR constructs that were only barriers included: (1) cosmopolitanism and (2) executing implementation.

TABLE 3.

Key themes by consolidated framework for implementation research (CFIR) constructs and exemplar quotes.

CFIR construct	CFIR construct definitions	Key themes exemplar quotes
Planning for implementation	The degree to which a scheme or method of behavior and tasks for implementing an intervention are developed in advance, and the quality of those schemes or methods	Collective implementation helped to provide a network of shared experiences. “My VISN Patient Safety is a great resource and reference and I never hesitate to call. On a VISN level, she knows the different other facilities that are out there that may be having similar process problems. And she's very quick to share what those are and we discuss them and just share insights.” Patient Safety Manager “We have a monthly Patient Safety Mangers call with the VISN [to discuss and get feedback] and so 1 month of each quarter we spend talking about Community Care events…We've talked about the Community Care but I think really going through [the implementation] and especially now that we have a little more interaction with the Guidebook it's helpful to know what people find the most useful and what people find maybe less confusing.” Patient Safety Manager Not including key collaborators in implementation planning hindered buy‐in and building of relationships. “I have a lot of familiarity with this and speaking freely I believe that the initial version was created in haste, quickly trying to evolve. It's going to be difficult to get the buy‐in of the Community Care, the non‐VA folks on board with this.” Patient Safety Manager “We need to have the ability to ask questions. We can read a manual but there needs to be interactions to make things more real with examples and walking through processes and responsibilities. I believe people want to do the right thing but we need to set them up with the right tools and knowledge…Front line users involved in process are much more likely to embrace this. With fundamental change management, it would have been much easier to implement and get buy‐in for the Guidebook if frontline users had been involved.” Patient Safety Manager “What could have been improved is just how [the Guidebook] was all vetted and communicated. [The Guidebook] kind of automatically showed up 1 day and there didn't seem to be a lot of dialogue….vetting [the Guidebook] and ensuring all people impacted have an opportunity to reply and be part of the changes. I think when changes are made, it would be important to provide that education out to the field.” VISN Patient Safety Officer
Engaging key knowledge holders	Involving appropriate individuals in the implementation and use of the intervention through various strategies such as social marketing, education, role modeling, training, and other similar activities	Using strategies, such as trainings, to engage staff helped collaborators to gain knowledge and set expectations. “We had a conversation with our [VA Facility] community care staff last year when we did the education about ensuring that they were reporting these incidences as they were made aware of them. We gave them specific education about how to access, locate, and use the Joint Patient Safety Reporting system and then encouraged them to report and gave them some ideas of different things that they should be reporting to give them a foundation. Of course, told them to report anything and we will make the determination on patient safety if something isn't appropriate or should be put into a different system or a different place. We didn't want them to have to try to critically think through that and make that a barrier to reporting. We'd rather get everything into the patient safety office and determine if there's something that should go elsewhere.” Patient Safety Manager “[Someone] from the VISN from Office of Community Care did a nice rollout of how you report events and why and where it goes [at a patient safety training] and that was the most helpful I've seen in terms of trying to make that relationship between [Patient] Safety and [VA Facility] CC [staff].” VISN Patient Safety Officer “We've improved education. We've been teaching with the community care nurse educators to new Patient Safety staff, so that's one thing that has changed, an improvement that's been made…It was in response to expressed concerns from the field about feeling uncertain about how to follow up on events. We've been doing that this past year… [The training is about] Community Care Patient Safety Event Reporting so it's a training about the Guidebook and we co‐teach it to both the community care staff and the Patient Safety staff….the community care staff have regular calls and we join in their calls and when they ask, one of the Patient Safety Officers will join the call and co‐teach with them and then we use it for training of new patient safety staff as well. So it's a learning module we've put together to help people understand how to report…One of the Nurse Educators from community care [put the training together].” VISN Patient Safety Officer Collaborating together provided for opportunities to offer feedback on the Guidebook [processes] and build relationships. “[E]ncourage making those relationships with [VA Facility CC] and getting together to have them explain their processes and how they do their consults and then just making that relationship so that you have that go‐to person…I really count on my [VA Facility] CC partners to know their processes and give me that information so if they need help with leadership involvement, I can steer them that way. I think [VA Facility] CC and Patient Safety need to develop those relationships because they're invaluable. Sometimes it may be awkward at first, especially in a larger facility, you may not know who the people are that you need to reach out to. So just making sure that the Patient Safety Managers are aware of who their [VA Facility CC] people are.” Patient Safety Manager “Anything new that is being implemented I think is important to get feedback from staff. Because I can tell you we learn a lot as we roll things out and we can actually give feedback that probably is useful to make it more efficient…I think would make things a lot better and more transparent when it comes to what we need to do as [VA Facility CC staff] on our end, make sure that we're referring things to Veterans that are in the network and are safe to be referred to.” VA Facility CC Staff “When the first Guidebook came out, one of the initial versions, we sat down with Patient Safety and talked through the processes, identified stakeholders in the event there was an incident reported that we needed to investigate. We had an idea of who we would need to involve and how that process would look. We talked through that with the Interdisciplinary Team. I think that made it very beneficial. There was National training and we followed up locally and had just a local discussion about [Guidebook processes].” VA Facility CC Staff Not enough initial cross‐collaborator engagement might have led to insufficient communication about the implementation and confusion about roles/responsibilities and process ownership. “I think there is a disconnect between the information I get and the information they get. I don't know what they get and I'm not really sure I get all that they get. I'm not briefed [whether] [VA Facility] CC [staff] has been trained…I think it would be more helpful if the quality improvement chiefs know what [VA Facility] CC [staff] is getting. What's coming, how things are changing, the impacts or are [they] invited in the same kind of call briefings [together]. That way if your VISN Quality Managers get [the Guidebook revisions], they'll include us. That way we know the expectation of this—one, two, three were given to you.” Quality Manager
Available resources	The level of resources dedicated for implementation and on‐going operations, including money, training, education, physical space, and time	Clear workflow and training helped to clarify expectations and roles/responsibilities and increase knowledge. “I do believe [the revised Guidebook] has more clarity. [I] truly appreciate the algorithms [in the Guidebook]…[A]s a patient safety manager world, there's nothing like checklists or algorithms to clearly define what the next steps are. And I think it's truly good to show that a lot of people have not left this on the backburner and they are still trying to improve this entire process.” Patient Safety Manager “The Guidebook is pretty clear…it really flows nicely. [The flow diagram] actually thinks like a patient safety person thinks. I appreciate that because we all like those little diagrams that tell us where to go next and where the divisions are because it's easier to find where the breakdown is when you map it out that way. I love those flow diagrams.” Patient Safety Manager “About once a year our Patient Safety Manger comes to a staff meeting and explains that process; whatever service we feel the need, reiterate that. So she'll do an overview of the process that we agreed to essentially that is the pertinent. The first contact is the entry person—opening that Joint Patient Safety Reporting and entering it.” VA Facility CC Staff Having enough training and person‐power continued to be challenging. “We've handed it off to Patient Safety and that's a lot because remember they're one little person. Even when they have a few people, still there's a Patient Safety Manager who is one little person and they're often kind of at a loss because we can't make a facility who is not the VA do an Root Cause Analsis or the conduct that we think that they should do….I would say in all instances the patient safety crew is staffed for the facility where they work and now you're adding to them responsib[ility] for everything that happens or is an unexpected or untoward event that's in the community realm.” VISN Patient Safety Officer “We had our patient safety manager go out into the Community Care office and offer training and tell them what they needed to do so we were hoping we would get more of those trainings…[P]eople are really [looking] for guidance on what can be reported, what should be reported, what happens to [a report] at the end of the day. When we first gave a report to the [Community Care] council, they were saying, nope this is not what happened, this is not [what] was supposed to be done and someone put it in. Obviously, that's when you need to have a better relationship with your patient safety manager…[We] do tell people about the Guidebook but we're still trying to figure out how to bridge this better.” VA Facility CC Staff
Networks and communications	The nature and quality of social networks and quality of formal and informal communications within an organization	Working together through councils/committees helped to improve collaboration, communication, and coordination. “I will say that probably one thing that has helped us a lot and the Guidebook does talk about it, we do have a Community Care Quality Oversight Committee that meets on a monthly basis. The [Committee] has Patient Safety; it has Community Care; it actually has the Chief of Staff and it has the Patient Safety Representatives from our Third Party Administrator, and our head of pharmacy as well. That is an excellent meeting we have on a monthly basis where we can talk about problems that we're seeing and how to fix them before they become a serious problem.” Patient Safety Manager “At all of our facilities, our patient safety managers participate in their Community Care meetings…There is communication [through meetings] should an event occur; there is an awareness and what I have seen across [VA facilities in] our VISN is a lot of good collaborations—having that patient safety manager as part of their [Community Care] committee meeting has been very good.” VISN Patient Safety Officer Regular meetings provided collaborators with an opportunity to discuss and offer ongoing feedback about Guidebook processes. “Once we had our [VA Facility] Community Care in place, they are the experts. I work with the Nurse Supervisor over in the [VA Facility] Community Care and if I ever have any question, I lean on her. I do not understand the intricacies of their processes or who pays for what and the whole TPAs and some of the non‐TPAs and why we go here and why we go there. She is just my lean‐on person to share that information with me so that I can bring that information to the patient safety meeting on Fridays. And if I feel like I'm not able to speak to it because it dives deep, then I bring those experts to the meeting.” Patient Safety Manager “We are a small facility and I have a very good rapport with the [VA Facility] community care folks…[If] something bad happened or anything I would get a phone call first before the Joint Patient Safety Reporting would be entered and then we initiate that conversation, then they put the event report in; wheels are already rolling prior to the Joint Patient Safety Reporting getting put in, we are already taking action immediately. We do have a Community Care Oversight Council that meets quarterly and I do have, as the Patient Safety Manger, a standing agenda placeholder there. I talk about the number of community care events that are reported on a quarterly basis and what we've done and how we've addressed each issue.” Patient Safety Manager “Patient Safety staff know they need to go these [Community Care] council meetings at the facilities. They have to be part of it, and they need to bring to the table what's been put in Joint Patient Safety Reporting to that committee so the [Committee] can comment on it. If we're wrong about is being reported, we need to change the categories and that would be helpful rather than having reports that really don't even belong in Community Care. That's what we've been working on.” VA Facility CC Staff Relationships between key collaborators were still under‐developed and hindered the comfort‐level needed to work closely together on Guidebook processes. “I've had several conversations with our Patient Safety Manager and our [VA facility Community Care] Specialist. We would sit down and we would go through [the Guidebook processes]. We talked about it and identified issues…the assessment process and issues has been back and forth several times.” Quality Manager “We talk about community care every single month at our VISN Patient Safety subcommittee meetings. I've developed a checklist for them–all Patient Safety Managers—to use when a serious event is reported in Joint Patient Safety Reporting just so we all remain standard. I've also tried my best to work with our VISN Community Care Office…I said, “Look, what you people don't understand is we are at the mercy of the reporter here. You can't expect [Patient Safety] to have all knowledge when it's not reported. And so anything that is reported that is a serious event, we are—each Patient Safety Manager—is placing it on the [VA facility] community care site to help them understand which events have come in so they know things that are being reported that I want to ensure that each Patient Safety Manager does review and appropriately determine where it needs to go, who needs to follow up on it and such. So yeah, I developed a short little cognitive checklist for everyone to follow to make sure we, as a VISN, across all the facilities are managing those exactly the same. And I do my best to do it according to what the guidebook says outside of the unrealistic expectations.” VISN Patient Safety Officer
Culture	Norms, values, and basic assumptions of a given organization	Psychological safety helped with the culture of reporting Community Care safety events. “Another very good improvement is [that] the services lines are talking to one another, and that's what's important. They have that availability to feel psychologically safe to report and talk.” VISN Patient Safety Officer “We're trying to do some education on that, but I do think people are more willing to report. I know definitely they feel comfortable calling [us] all the time to tell us different things or come over and look at this and should they put this in or any of that kind of stuff.” Patient Safety Manager “I got a great relationship with Community Care. They know me very, very well. I walk in the door and people are very open to talk to me about problems and issues that they are seeing. We have a great reporting culture. It's nonpunitive; everybody understands that we're trying to help and because I've been very active within Community Care doing investigations and projects to improve processes that involve the staff there in those projects and investigations, they are very receptive.” Patient Safety Manager “Our culture has existed for a long time so even when we went to Community Care saying hey, we want everybody trained on this, there was no resistance at all…They are very, very much fans of the Patient Safety and of being involved with us helping them out and them helping us out…I think [the Guidebook] might have been more information than they need but there was no resistance about it…They were [receptive]…[The Guidebook] had more information than they needed, but it certainly gave them a good foundation of information.” Patient Safety Manager “It was a bit of a struggle for us at first to get [VA facility CC staff] to think about what might be a report for Community Care because we haven't previously done that. So it was just giving them a sense of what kind of things might they see that they would report. We did a roll out session of people to do face‐to‐face meetings with a lot of the community care staff in different departments that would handle things in the community, including like the [VA facility] business office as well as [VA facility] Community Care [staff] so that we just had more people receiving the training on how to enter a report, what kinds of things might be required and that was very helpful. We started to see more reports. Our reporting rate was very low, maybe two to three a quarter for Community Care events and we know that there is more and more going on out there. We doubled it and then maybe tripled it. We're still not getting a lot but I feel like at this point they are better quality…[VA Facility CC staff] do really embrace just culture.” Patient Safety Manager Shifting focus from reporting internal VA safety events to reporting CC events was challenging. “I think that a lot of the education when this Guidebook came out the emphasis was to educate the [VA] Community Care staff for reporting which was very valuable to do but I also believe that [VA] services that are heavily involved with Community Care consults those ordering providers also need to be educated on how to put in incidents reports on these because they are usually the first people that are finding out about the problem…I know as a service [patient safety] tried to ensure that a lot of the providers get trained on the process, but out of all our reporting group, physicians are probably the smallest group for reporting [community care] incidents.” Patient Safety Manager
External policies	A broad construct that includes external strategies to spread interventions, including policy and regulations (governmental or other central entity), external mandates, recommendations and guidelines, pay‐for‐performance, collaboratives, and public or benchmark reporting	Implementation of High Reliability Organization helped to provide movement toward a more welcoming culture of reporting. “I think with us being a pilot site for High Reliability Organization it is helping us. They're doing a lot of training on just culture and reporting and how we should respond to these reports. Again, we have had a pretty high reporting rate for our Joint Patient Safety Reporting [events]…We have seen an increase in reporting since we've been on this High Reliability Organization journey and so I think the culture is pretty good overall.” Patient Safety Manager “[High Reliability Organization] has been rolled out at our facility. There has been baseline training for managers, supervisors, and frontline staff and each service or each department has embraced it to different levels. I think speaking for the Clinical Community Care Department the nurses and the clinical staff are very comfortable bringing [safety] concerns forward and working through those…They validated feeling comfortable reporting and participating in any of the investigations on the incident reports.” VA Facility CC Staff Uncertainty about whether broad systemic communication across VA occurred. “[W]e're assuming Community Care distributed [the Guidebook] because it wouldn't be [Patient Safety's] place to distribute it. That's another gap or barrier that you don't know who really is well versed in it or even has read it.” VISN Patient Safety Officer
Cosmopolitanism	The degree to which an organization is networked with other external organizations	Not enough clarity on how to conduct investigations with community care networks (CCN) Third Party Administrators hindered meeting timelines. “[CCN Third Party Administrators] don't really share the notes. They usually do a verbal communication back to the [VA facility] coordinator or social worker what the outcome was, but we don't see a diagram of what the root cause was, what the breakdown was, their action plans, all of that. We just get kind of a summary of the outcome. We don't really have any way of following up and ensuring that if they put an action plan in place that those action plans actually took place…it's definitely hard because they don't have any obligation to participate and tell us even though in a way, they kind of do…[I]t's a little bit challenging and definitely their standards are different than our VA standards.” Quality Manager “I think the frustration is that when talking to the [CCN Third Party Administrator] and pushing forward concerns, we do have a process to send them concerns and requests for RCAs. However, their timeline of response is vastly different than the VA's. With us, when we have a catastrophic event, it requires a root cause analysis and we have 45 days. Our Third Party Administrator for our region, I think they either have a 90 to 180 days depending on the event classification. So it's just vastly different and it causes for a lot of anxiety because the Patient Safety Managers and the VISNs over here saying okay, this is something that needed a Root Cause Analysis, we need to know what is going on and they have practically, what is that like almost a half a year to respond? …And the thought is if this is a serious enough event, action should be taken immediately. We shouldn't be waiting because within that 45 days, it could very easily happen again.” VISN Patient Safety Officer Needing more information on the CCN Third Party Administrators' roles and responsibilities with the processes in the Guidebook hindered implementation. “When we do send information to CCN Third Party Administrators for an investigation, it's vague if they got [the information] or not. When [community] providers call us, we direct them back to [CCN Third Party Administrators]. They call us back asking tons of questions because apparently when they go to [CCN Third Party Administrators], [CCN Third Party Administrators] does not know what [community providers] are talking about. That's what we were told…[W]e would like to know what their review process looks like…disclose a report to us at the facility level of how many providers they reviewed, the timeliness of the review, the thoroughness of the review—some sort of transparency so that we understand what's going on. We are the frontline taking care of patients so our confidence in these providers needs to be there. Leaving us out of the loop, I don't know if that's an effective approach.” VA Facility CC Staff “I think the biggest issue with the Guidebooks is, we don't really seem like we're getting the juice for the squeeze with the whole process. We're really not getting any useful information back from the [CCN Third Party Administrator] so it seems to be processes that's creating work but not really adding value. So, like putting all the burden on patient safety to report the quality or patient safety issues to the Third Party Administrator but then they tell us point blank that they're not obligated to return any information to us. They may just say that they handed it off to whatever the community provider was.” VISN Patient Safety Officer
Executing	Carrying out or accomplishing the implementation according to plan	Need for more clarity on Guidebook content especially at the initial rollout phase so that collaborators understand their roles and responsibilities. “We initially started doing that. We invited [CCN Third Party Administrator representative] to be on the call but it was not the best use of their time or our time because we really weren't clear why they were on the call with us. They weren't able to answer our questions right at that time. It was a little bit awkward I would say. And so it was really just to kind of introduce them to all these Patient Safety Managers across the VISN but I think they were kind of confused like we just have questions, like okay, we requested a Root Cause Analysis how are we going to get feedback, how long is it going to take for us to get feedback. That's all we need to know–was it done and they were not able to answer that question. I think that was unfortunate that our Quality person at the [CCN Third Party Administrator] really didn't have the answers to that.” Patient Safety Manager “I think there's a challenge working through [the Guidebook processes] the first time, trying to identify who the specific point of contact [is] for the different roles…I think that was our biggest obstacle…knowing who to work with the [CCN Third Party Administrator], how to document incident information to them. Our Patient Safety Officer played a big role in facilitating that piece.” Patient Safety Manager “I think that [the Guidebook] can be a little bit confusing about who's doing what and how that process would really roll out. It's been in the works a long time but it still not really clear [the] processes within the Guidebook when we get into what should be, what can be reported to [CCN Third Party Administrator] what that pathway would look like, it's certainly not there.” VISN Patient Safety Officer “I think we need to have a better understanding what Patient Safety's role is in the community and it looks good on paper but in the actual carrying out the processes we need to have a better understanding and some guidance and education on that. We don't know enough about their internal process so it is very hard for us to dig and ask questions. The way I'm seeing it right now is we are all trying to figure it out, how it's going to work. It's a challenge every day.” Patient Safety Manager “I don't think [Guidebook process ownership] really has been defined. I think whoever gets the email, then distributes it to one or the other and basically that's that…Like if it came up at National Center for Patient Safety and it came from the patient safety office, then I would then send that or we would then send that to the community care office and kind of vice versa… [R]eally outlining a little bit more specifically ownership of various parts, I think that's something people have always struggled with—who does own this and whose responsibility is what.” Patient Safety Manager “When we had our initial investigation for this [event] and this one of course involved the [CCN Third Party Administrator]. So we were thinking that they were part of the investigation as well and [they would] reach out to that [CCN] provider and make their own investigation. What we're learning in the process is that they were saying that it's not the responsibility. They're saying that it's not their responsibility; it's us, the VA, would do the investigation, but according to this Guidebook, there is [a] pretty clear outline here saying the [CCN Third Party Administrator] would have to have their own investigation; so far nothing is shared about it.” VA Facility Community Care Staff Not enough information about CCN Third Party Administrator representatives hindered implementation. “One challenge that we have is how do we get in contact with our [CCN Third Party Administrator], who is our [CCN Third Party Administrator] person…if we can figure it out [before] something serious did happen, it would be nice to know…how do we work together and how do we get that mandated. Is it optional…who is doing that? Some real clear guidance on who that is? …There's no point of contact [at CCN Third Party Administrator], other than this is like a conglomerate…I wouldn't know who is the right person [to contact]. I wouldn't even know where to start.” Patient Safety Manager “Some of the individuals mentioned in the Guidebook like the [CCN Third Party Administrator], I don't even know who those individuals are. I don't even know how to contact them if I needed to.” Patient Safety Manager “If we identify something else in the community, we need to know that [CCN Third Party Administrator] liaison because that would be the frontline [CCN Third Party Administrator representative] do[ing] the ground work with that contracted service because we don't know them and we as the individuals don't have the authority specifically. Contract is different. They're a network. I think we will have to figure out what that really looks like.” Quality Manager “I recognize that these [CCN Third Party Administrators] they're encompassed in this guidebook and that's something that wasn't communicated very well. I think just better communication of the Guidebook and its expectations and I honestly think more [CCN Third Party Administrator] points of contact for each individual [VA] facility because I think what is happening is, the email gets sent to a big pool to where anyone can reach out. I think it would be a little more beneficial if a [VA] facility…had a point of contact for their [CCN Third Party Administrator], and it was John Smith. John Smith may cover a few [VA] facilities, but [these VA facilities] have a direct person from [CCN Third Party Administrator] rather than just sending it into kind of the oblivion…Because they would be able to say we got these concerns; these three concerns are being sent to that specific [VA] facility. We are waiting to hear from them and then we will review—just to get that kind of update [from a CCN Third Party Administrator representative].” VISN Patient Safety Officer

Abbreviations: CC, community care; CCN, community care network; CFIR, consolidated framework of implementation research; VA, U.S. Department of Veterans Affairs; VISN, Veterans Integrated Services Networks.

3.2.1. Planning for implementation

Facilitators

Guidebook implementation occurred more smoothly when all sites within a VISN had been tasked to implement the Guidebook collectively. This helped to standardize the implementation process and provided PSMs with a network of colleagues to engage with and answer questions about experience with similar hurdles and share lessons learned.

Barriers

Guidebook implementation was hindered when VA patient safety and Facility CC staff and CCN Third Party Administrators were not included in implementation planning. For example, initially, patient safety and VA Facility CC staff had challenges in jointly deciding how to implement formal safety event reporting, data review, and investigation processes within their facility's CC department as well as with CCN Third Party Administrators. Uncertainty surfaced about whether broader systemic communication to different departments or services within the VA as well as to the CCN providers and Third Party Administrators about the Guidebook had occurred as effectively as it could have, particularly during the initial planning and early stages of Guidebook roll‐out.

3.2.2. Engaging key collaborators

Facilitators

Patient safety developed and led event reporting training sessions about the Guidebook which helped inform VA Facility CC staff about CC safety events and expectations. In addition, VISN CC nurse educators and PSOs conducted training sessions with VA Facility CC staff and PSMs on CC safety event reporting processes; this enabled patient safety and CC to be involved in jointly educating and setting expectations about Guidebook processes. Collaborators were also able to provide feedback on and participate in revising later versions of the Guidebook.

Barriers

At the beginning of Guidebook roll‐out, additional efforts were needed to engage all appropriate collaborators whose roles and responsibilities were outlined in the Guidebook about safety, quality, and CC events. Not enough initial cross‐collaborator engagement may have resulted in insufficient communication about Guidebook implementation or confusion about process ownership for CC safety event reporting and investigation.

3.2.3. Available resources

Facilitators

Involvement of key collaborators in providing feedback on revisions to the Guidebook improved workflow of processes, with person‐power and time more appropriately outlined from prior versions. Similar to facilitators for “engaging key collaborators,” trainings helped to increase knowledge and more accurate reporting of CC safety events.

Barriers

While initial education was helpful, there were barriers regarding sustained staff learning and understanding and getting new staff members trained. In addition, although improvements had been made since initial Guidebook roll‐out, having enough person‐power and time continued to be a challenge; this became a heightened concern when staff turnover occurred or when new hires needed to get up to speed on Guidebook processes as well as their other VA duties.

3.2.4. Networks and communications

Facilitators

Strong coordination, collaboration, and communication between patient safety and VA Facility CC staff were key organizational factors that facilitated Guidebook implementation; networks and communications improved as roll‐out progressed over time. For example, stronger networks and communications developed when VA PSMs and VA Facility CC staff regularly worked together through CC councils/committees and patient safety meetings where they could discuss CC safety events and concerns; trends and types of CC safety events that should be reported; and ways to follow‐up on Guidebook processes. These meetings facilitated opportunities to provide feedback when reviewing CC events and addressing questions or next steps.

Barriers

Although networks and communications improved as Guidebook implementation progressed, there were ongoing challenges in coordination, collaboration, and communication between key collaborators. Whether or not CC safety events should be discussed and who should be present at these CC council/committee meetings remained unclear. Relationships between key collaborators were still under‐developed and hindered the comfort‐level needed to work closely together on Guidebook processes.

3.2.5. Culture

Facilitators

Key factors that facilitated the reporting of CC events across VA facility staff were having an established safety event reporting culture; having organizational support and encouragement; making staff feel psychologically safe and comfortable.

Barriers

The extent to which there was organizational support and encouragement of CC safety event reporting varied. The shift from focusing solely on internal VA events to a paradigm where VA staff must also report CC safety events involved a tremendous change in reporting culture. Safety practices and processes were also affected, involving parties outside of VA‐internal staff (e.g., CCN providers). These changes created challenges to existing culture.

3.2.6. External policies

Facilitators

The nationwide roll‐out to create a High Reliability Organization within VA was a top priority initiative that dovetailed with Guidebook implementation, ³² since one of the three pillars of a High Reliability Organization is the presence of a strong patient safety culture. Perceptions of safety culture that is open and welcoming were integral to increase the reporting of safety events that occurred in both VA‐delivered and VA‐purchased care settings. Engagement in implementing High Reliability Organization initiatives helped to improve reporting in general by increasing feelings of comfort (e.g., psychological safety).

Barriers

There was uncertainty about whether broad systemic communication about the Guidebook had occurred as effectively as it could have, particularly at the early stages of Guidebook roll‐out. For example, similar to barriers to “planning for implementation,” participants were unsure whether different departments, services, or leadership within the VA or CCN Third Party Administrators were aware of the Guidebook and its proposed processes, which could then affect broader buy‐in and partnerships.

3.2.7. Cosmopolitanism

Barriers

Engaging with CCN Third Party Administrators was the most frequently mentioned barrier to Guidebook implementation, even though there were some working relationships with community partners that improved over time. As a result, there were challenges in obtaining an adequate amount of information for VA PSMs to conduct their investigation of CC safety events and in meeting event investigation timelines. In addition, being unaware of community partners' roles and responsibilities led to confusion about what information and what actions partners needed to accomplish when a CC safety event occurred.

3.2.8. Executing implementation

Barriers

Carrying out or accomplishing Guidebook implementation, especially at the start of national roll‐out, was hindered by several challenges (as noted in other CFIR constructs). More clarity was needed on the content contained in the Guidebook, such as who was required to be involved in the Guidebook processes, roles/responsibilities, and expectations, especially given the complexities of working with multiple collaborators within and outside of the VA.

Unlike staffing changes within a VA facility which could be navigated with more ease, changes in CCN Third Party Administrator representatives delayed the building of relationships needed to share information. More communication and information on which CCN Third Party Administrator representatives to contact about CC safety events could have helped with executing Guidebook implementation.

3.3. Strategies to improve the guidebook implementation approach

We also identified implementation strategies that could be used by our partners to inform future Guidebook implementation efforts. Table 4 presents the ERIC implementation strategies and their definitions mapped to each of their associated CFIR constructs. We categorized ERIC implementation strategies based on groupings used by Waltz et al. ³³ : (1) Develop collaborator interrelationships (nine ERIC strategies); (2) Train and educate collaborators (six ERIC strategies); (3) Provide interactive assistance (two ERIC strategies); and (4) Support clinicians (1 ERIC strategy).

TABLE 4.

Implementation strategy identification by consolidated framework of implementation research (CFIR) constructs.

ERIC implementation strategy	Strategy definition 31	CFIR construct
		P	En	NC	Co	AR	Ex	Cu	EP
Develop collaborator interrelationships
Identify and prepare champion	Identify and prepare individuals who dedicate themselves to supporting, marketing, and driving through an implementation, overcoming indifference or resistance that the intervention may provoke in an organization	X					X	X
Develop implementation glossary	Develop and distribute a list of terms describing the innovation, implementation, and collaborators in the organizational change	X
Conduct local consensus discussions	Include local providers and other stakeholders in discussions that address whether the chosen problem is important and whether the clinical innovation to address it is appropriate	X
Inform opinion leaders	Inform providers identified by colleagues as opinion leaders or “educationally influential” about the clinical innovation in the hopes that they will influence colleagues to adopt it		X
Organize implementation team meetings	Develop and support teams of clinicians who are implementing the innovation and give them protected time to reflect on the implementation effort, share lessons learned, and support one another's learning			X
Build a coalition	Recruit and cultivate relationships with partners in the implementation effort			X			X		X
Obtain formal commitments	Obtain written commitments from key partners that state what they will do to implement the innovation			X			X		X
Capture and share local knowledge	Capture local knowledge from implementation sites on how implementers and clinicians made something work in their setting and then share it with other sites			X
Involve executive boards	Involve existing governing structures (e.g., boards of directors, medical staff boards of governance) in the implementation effort, including the review of data on implementation processes				X		X	X
Train and educate collaborators
Provide ongoing consultation	Provide ongoing consultation with one or more experts in the strategies used to support implementing the innovation	X					X
Create a learning collaborative	Facilitate the formation of groups of providers or provider organizations and foster a collaborative learning environment to improve implementation of the clinical innovation			X	X
Conduct educational meeting	Hold meetings targeted towards different stakeholder groups (e.g., providers, administrators, other organizational stakeholders, and community, patient/consumer, and family stakeholders) to teach them about the clinical innovation			X	X		X		X
Develop educational materials	Develop and format manuals, toolkits, and other supporting materials in ways that make it easier for stakeholders to learn about the innovation and for clinicians to learn how to deliver the clinical innovation						X
Distribute educational materials	Distribute educational materials (including guidelines, manuals, and toolkits) in person, by mail, and/or electronically						X
Conduct ongoing training	Plan for and conduct training in the clinical innovation in an ongoing way						X
Provide interactive assistance
Provide technical assistance	Develop and use a system to deliver technical assistance focused on implementation issues using local personnel	X		X			X
Centralize technical assistance	Develop and use a centralized system to deliver technical assistance focused on implementation issues			X
Support clinicians
Develop resource sharing agreements	Develop partnerships with organizations that have resources needed to implement the innovation				X	X

Abbreviations: AR, available resources; CFIR, consolidated framework of implementation research; Co, cosmopolitan; Cu, culture; En, engaging key collaborators; EP, external policies; ERIC, expert recommendations for implementing change; EX, executing; NC, networks and communications; P, planning.

Across the CFIR constructs, two strategy groupings emerged most often. These included: Develop collaborator interrelationships and Train and educate collaborators. Within the Develop collaborator interrelationships grouping, the discrete ERIC implementation strategies represented most frequently were (1) Identify and prepare champions; (2) Build a coalition; (3) Obtain formal commitments; and (4) Involve executive boards. For the Train and educate collaborators grouping, three discrete ERIC implementation strategies were identified: (1) Provide ongoing consultation; (2) Conduct educational meetings; and (3) Create a learning collaborative.

4. DISCUSSION

This paper presents the organizational factors and associated ERIC implementation strategies found in our evaluation of the Guidebook implementation. Based on these findings, we identified several lessons learned across CFIR constructs: (1) engage all collaborators involved in implementation; (2) ensure end‐users have opportunities to provide feedback; (3) describe collaborators' purpose and roles/responsibilities clearly at the start; (4) communicate information widely and repeatedly; and 5) identify how multiple high priorities can be synergistic.

4.1. Lesson 1: Engage all collaborators involved in implementation

Engagement with all collaborators involved in the implementation emerged as being one of the most important lessons learned during all phases of implementation. This lesson builds on the literature regarding the importance of engagement and involvement of collaborators across multiple organizational levels when implementing evidence‐based practices. ³⁴ , ³⁵ , ³⁶ Research has shown collaborator involvement is also closely aligned with Learning Health System approaches which emphasize continuous learning and improvement efforts during program implementation to achieve more efficient healthcare delivery. ³⁷ For example, providing VISN‐level PSOs the opportunity to be engaged in reviewing the Guidebook prior to implementation efforts could have mitigated concerns that previous versions of the Guidebook did not involve them as subject matter experts.

4.2. Lesson 2: Ensure end‐users have opportunities to provide feedback

Ensuring end‐users have opportunities to provide feedback emerged as one of the next most important lessons learned. In our work, participants echoed the desire to have all parties responsible for the Guidebook safety processes engaged in implementation planning and providing feedback on the Guidebook as early as possible. This might have helped to identify potential risks and implementation issues earlier on and improve relationships and communication within and across organizational entities—key factors in the literature that can be addressed to engage all collaborators from the start on implementation (e.g., planning phase). ³⁸ Research indicates that multilevel engagement may result in better implementation and teamwork by including diverse perspectives and providing opportunities for feedback. ³⁹ , ⁴⁰

4.3. Lesson 3: Describe collaborators' purpose and roles/responsibilities clearly at the start

We found evidence describing the importance of defining collaborators' purpose and roles/responsibilities clearly throughout the planning and implementation phases. While roles and responsibilities in the Guidebook became clearer as implementation progressed, individuals were unsure of who should be doing what in terms of specific safety processes. The literature suggests that delineating and clarifying all collaborators' roles and responsibilities from the start of a roll‐out would have helped individuals understand their purpose and what was expected of them. It also could have helped promote accountability, streamline workflows, prevent duplicate efforts, and reduce conflicts over role confusion. ⁴¹ , ⁴² Clearly defined roles also enhance communication by providing a common understanding of whom to contact for specific issues or information. When people understand their roles and how they should contribute to the program's success, they are more likely to be motivated and engaged in their work. ⁴³

4.4. Lesson 4: Communicate information widely and repeatedly

Another lesson learned related to widespread and repeated communication about Guidebook implementation. Sharing consistent information across multiple levels of an organization can reinforce desired behaviors, such as patient safety and VA Facility CC staff working more closely together, and can also motivate staff to seize opportunities to improve CC processes related to Guidebook implementation. ⁴⁴ Communicating regularly with staff involved in Guidebook implementation would have allowed for messaging about any new updates or changes occurring during the roll‐out; this would have assisted staff in building capabilities for implementing Guidebook processes. ⁴⁵

4.5. Lesson 5: Identify how multiple high priorities can be synergistic

Identifying ways that multiple priorities can be synergetic to meet the goals of each VA facility was the final lesson learned. Literature indicates that recognizing multiple competing priorities and finding ways to synergize across them can ultimately lead to greater success and positive outcomes. ⁴⁶ , ⁴⁷ , ⁴⁸ , ⁴⁹ Research studies have found that when competing priorities are identified and understood, resources can be effectively allocated as well as adequately funded and supported. ⁵⁰ In addition, synergizing priorities helps align program objectives with broader organizational goals to ensure that the program contributes to overarching missions and strategies, such as VA's High Reliability Organization national roll‐out. Balancing and harmonizing competing priorities also streamline decision‐making processes, reduce conflicts, and enhance the program's agility. ⁵¹ , ⁵² , ⁵³

The identification of implementation strategies is key to tailoring and improving implementation approaches based on CFIR facilitators and barriers. ⁵⁴ We identified several ERIC implementation strategy groupings: “Develop collaborator interrelationships,” “Train and educate collaborators,” “Provide interactive assistance,” and “Support clinicians.” These strategy groupings can guide collaborators with tailoring implementation approaches as they continue to build on the lessons learned and strengthen Guidebook implementation. Developing strong relationships with collaborators, both internal and external parties, can foster trust, teamwork, and support for the program. ²⁷ , ⁵⁵ We found that when collaborators had stronger relationships, Guidebook implementation occurred more smoothly, with staff more actively engaged in working together toward common goals. This mostly related to internal VA relationships, as opposed to external VA and CC relationships where relationships were still underdeveloped. Properly training and educating collaborators about the program's objectives, processes, and expected outcomes enhanced their understanding of the Guidebook. When collaborators are knowledgeable, they are better equipped to contribute effectively to program implementation and make informed decisions. ⁵⁶ Offering ongoing assistance and support in a responsive and interactive manner can address challenges and issues as they arise during implementation. This can prevent roadblocks and setbacks, ensuring that the program stays on track and adapts to changing circumstances and ensuring that collaborators feel supported, which are crucial factors for successful program implementation. ⁵⁷ Support can be shown through providing resources, tools, and guidance to ensure the delivery of high‐quality care or services that is aligned with the program's objectives. Application of these strategies creates an environment of collaboration, competence, and adaptability which is essential for effective program implementation. ⁵⁸ , ⁵⁹ In addition, utilizing these strategies can help identify and resolve issues promptly, align collaborator efforts with program goals, and enhance efforts of implementation success.

This project has both strengths and limitations. First, it provides a detailed analysis of Guidebook implementation offering insights into barriers, facilitators, and implementation strategies. Second, it offers clear recommendations for improving Guidebook implementation in the future. However, there are limitations. We relied on self‐reported information from facility‐ and VISN‐level staff which could be subject to social desirability (i.e., responding favorably about implementation efforts). Additionally, achieving consensus on coding was only based on one interview. However, the team members conducting the analysis were very familiar with each other's coding styles, particularly related to coding constructs from the CFIR framework, and have worked together on many projects using this same framework. Thus, they did not require coding additional interviews to achieve consensus before moving forward. Nevertheless, they did meet regularly to discuss any coding issues or concerns they had during team meetings.

5. CONCLUSIONS AND IMPLICATIONS

This QUERI Partnered‐Evaluation Initiative was conducted in a collaboration between researchers and two VA operational partners (National Center for Patient Safety and Office of Integrated Veteran Care). It offered a timely opportunity to evaluate the national implementation of the Patient Safety Guidebook and recommend useful implementation strategies to enhance uptake and spread of the Guidebook safety processes. This helped inform our VA operational partners about organizational contextual factors affecting implementation and strategies that could be used to strengthen implementation. Based on these findings, our partners are well‐informed and ready to bring these recommendations to each of their own organizations' leadership and staff. National Center for Patient Safety and Office of Integrated Veteran Care have recently begun to have collaborative meetings to help clarify Guidebook roles and revise the Guidebook more generally in order to improve Guidebook implementation and further educate VA staff through additional training opportunities and collaborative events.

Given the increase in Veterans' use of CC purchased by VA, effective implementation of the Guidebook and its associated safety event processes remains critical to improve safety and importantly, to standardize safety event reporting across VA care settings. Through ongoing collaboration with our partners, we will continue to further refine implementation strategies to increase effective implementation of the Guidebook. The five key lessons learned from our project can help not only with ongoing improvements in Guidebook implementation, but they can also provide guidance as sites move toward sustainment of the safety practices outlined in the Guidebook. Finally, our results may provide lessons learned to VA's adoption of Learning Health System tenants. These principles, consistent with the lessons learned in our project, include the need for continued collaboration in executing initiatives; development of continuous process improvement initiatives to learn and iteratively improve implementation efforts. ⁶⁰

Though our specific findings are limited to the VA, the general lessons learned may be of interest to other healthcare organizations that contract care out to other provider organizations or entities.

FUNDING INFORMATION

This evaluation was funded by VA QUERI PEC‐18‐204. Dr. Rosen is also supported by an HSR&D Senior Research Career Scientist Award (RCS 97–401).

Sullivan JL, Shin MH, Chan J, et al. Quality improvement lessons learned from National Implementation of the “Patient Safety Events in Community Care: Reporting, Investigation, and Improvement Guidebook”. Health Serv Res. 2024;59(Suppl. 2):e14317. doi: 10.1111/1475-6773.14317