From implementation to sustainment: A large-scale adverse event disclosure support program generated through embedded research in the Veterans Health Administration

Abstract

In 2008, the Veterans Health Administration published a groundbreaking policy on disclosing large-scale adverse events to patients in order to promote transparent communication in cases where harm may not be obvious or even certain. Without embedded research, the evidence on whether or not implementation of this policy was generating more harm than good among Veteran patients was unknown. Through an embedded research-operations partnership, we conducted four research projects that led to the development of an evidence-based large-scale disclosure toolkit and disclosure support program, and its implementation across VA healthcare. Guided by the Consolidated Framework for Implementation Research, we identified specific activities corresponding to planning, engaging, executing, reflecting and evaluating phases in the process of implementation. These activities included planning with operational leaders to establish a shared research agenda; engaging with stakeholders to discuss early results, establishing buy-in of our efforts and receiving feedback; joining existing operational teams to execute the toolkit implementation; partnering with clinical operations to evaluate the toolkit during real-time disclosures; and redesigning the toolkit to meet stakeholders’ needs. Critical lessons learned for implementation success included a need for stakeholder collaboration and engagement, an organizational culture involving a strong belief in evidence, a willingness to embed researchers in clinical operation activities, allowing for testing and evaluation of innovative practices, and researchers open to constructive feedback. At the conclusion of the research, VA operations worked with the researchers to continue to support efforts to spread, scale-up and sustain toolkit use across the VA healthcare system, with the final goal to establish long-term sustainability.

1. Background and problem

Large-scale adverse events are unanticipated incidents that occurred during the process of patients receiving health care, which either led to multiple patients’ injury or increased their risk of injury, yet were not recognized by the health care system at the time of the incident.¹ Examples of large scale adverse events include equipment disinfection lapses (e.g, endoscopes, dental equipment), unsafe injection practices (e. g., reuse of single patient syringes) and events related to provider behavior, such as knowingly or unknowingly practicing unsafe medicine.² Disclosure of large-scale adverse events is a formal process by which health care system officials assist with coordinating the notification to multiple patients that they may have been affected by a system issue during their care.³ Large-scale adverse events are unique in that many patients are potentially exposed but few are truly at risk of injury or illness.⁴ In the VA, improperly reprocessed (e.g., improperly disinfected according to manufacturer’s and VA guidelines) endoscopes required thousands of patients to be notified of potential disclosure, although no known patients acquired a bloodborne pathogen as a result of the reprocessing breakdown.⁴ These large-scale adverse events present challenges for health care systems and public health officials alike, because of the absences of known bloodborne disease transmission.^5,6 From 2009 to 2011, lawsuits were filed following VA large-scale disclosures, indicating the distress that many patients felt following these disclosure of events related to improperly reprocessed endoscopes in several VA medical centers, and the potential of contracting bloodborne pathogens such as HIV or hepatitis C following this exposure.⁷

The Veterans Health Administration (VHA) policy on disclosing adverse events, Directive 1004.08, “Disclosure of Adverse Events to Patients”, calls for the “unwavering ethical obligation to disclose to patients harmful adverse events that have been sustained in the course of their Department of Veterans Affairs (VA) care, including cases where the harm may not be obvious, or where there is a potential for harm to occur in the future” (page 4).³ As one of the early leaders in this field, the VA helped to promote transparent disclosure policies across other healthcare organizations and regulatory bodies, including for clinical and institutional adverse events, in addition to large-scale adverse events.^8,9 Although scholars and ethicists have called for full, honest, and transparent disclosure,^10,11 and patient-centered care advocates have insisted that all aspects of healthcare be shared with patients,¹² little evidence existed to indicate how best to implement large-scale disclosures related to adverse events where there was a high degree of uncertainty regarding impact on Veterans, their families or the VA health care system.

2. Organizational context to this background and problem

To address this concern, the VHA Principal Deputy Undersecretary for Health (PDUSH) requested research support from investigators in the Health Services Research and Development (HSR&D) service. HSR&D consists of academic researchers embedded within the VA healthcare system who are able to collaborate with clinical and operational teams to identify, design and conduct research studies and share findings which respond to the needs of VA leaders.¹³ HSR&D requested proposals to select an embedded research team to move forward with this research. Our team, the Study of the Communication of Large-Scale Adverse Events (SCALE) team, was funded to conduct four projects within one study. During our initial two-year funding period, we identified past communication problems through media analyses and called for more timely disclosures and a more streamlined communication approval process¹⁴; we developed an evidence-base for the impact of disclosure on Veterans’ health and healthcare perceptions and health-seeking behavior, using mixed methods, and identified how Veterans and family members want to be told the truth about their care, even if the outcomes are uncertain^2,15,16; and we developed and tested strategies for communicating with Veterans and external stakeholders regarding these events, using specific and direct language to describe the event, and encouraging follow-up care and testing for bloodborne infections.¹⁷ We received a further year of funding to pilot test an evidence-based toolkit for disclosing large-scale adverse events, consisting of action plans, tracking sheets, training materials, templates, checklists, and communication scripts developed from our research.¹⁸

The objective of this paper is to describe the process our embedded research-operations team identified to first create and then implement our evidence-based solution, the Large-Scale Disclosure Toolkit, when disclosures were warranted, guided by an implementation science framework. This process consisted of 1) collaborating with VHA operational leaders to plan a shared research agenda, 2) engaging with operational stakeholders to discuss early results and establish buy-in of our efforts and to receive feedback, 3) using evidence to create and implement the toolkit for use by clinical and operational leaders, 4) partnering with the operations-led Clinical Episode Review Team (CERT) to evaluate the toolkit during two real-time disclosures, and 5) continuous service as permanent members of the CERT.³ We then present an overview of the ongoing Disclosure Support Program which has developed as an extension to the toolkit, based on stakeholder feedback. We conclude with lessons learned from this partnership for both the VA and other healthcare systems.

3. Solutions to the problem of disclosing large-scale adverse events

Implementation of healthcare delivery innovations is often complex because diverse individuals from across an organization and among different managerial levels must be involved in all aspects of the process.¹⁹ The Consolidated Framework for Implementation Research (CFIR)²⁰ provides a framework and identifies steps needed to overcome this complexity and ensure successful change. CFIR outlines four steps for implementation: planning, engaging, executing, and reflecting and evaluating. Planning refers to the degree to which tasks for implementing an intervention are developed in advance and the quality of those methods. Engaging involves working with appropriate individuals in the implementation through combined strategies of education, training, and other similar activities. Executing is defined as carrying out the implementation according to plan. Reflecting and evaluating comprises data-driven feedback about the progress and quality of implementation accompanied with regular personal and team debriefing about progress and experience. Below, we describe each of our solution activities as it relates to one of these four CFIR implementation process steps.

3.1. Research-operations collaboration (planning)

The research team principal investigator and a key staff member in the Office of the PDUSH established a communication channel in order to set up telephone and in-person meetings as needed throughout the study period. Our research team met with the PDUSH, the directors of the offices of Risk Management, Ethics Policy, Public Health, HSR&D, and the Deputy Undersecretary for Health for Policy and Services, to discuss their priorities for the research, to learn about our proposed methods and analyses and to finalize our research questions. At this meeting, researchers learned of VA leaders’ preferences for involving Veterans’ family members in our qualitative interviews, recognizing the role that spouses often play in determining whether or not care is received. The research team then updated our study protocol to include outreach to Veterans’ spouses and partners, inviting them to participate in 30–45 minute interviews at nine sites where interviews were conducted, either in-person or by telephone. In a follow-up research-operations meeting, when results from our first study examining media reports to VA large-scale disclosures were presented, operations leaders asked the research team to include non-VA sites who reported similar large-scale disclosures in our media analysis, to examine whether or not these media reactions were the same across VA and non-VA large-scale adverse events. The specific reason for this request was so that VA leaders would be able to provide these data to the House and Senate Veterans Affairs Committees when asked to comment on large-scale disclosures.

3.2. Stakeholder engagement (engaging)

Through our ongoing communication channel established in the planning stage, the research-operations team agreed that quarterly research briefs provided by the research team to the Office of the PDUSH and the HSR&D Service would allow for ongoing engagement of the operations team in all aspects of the research process. Twelve briefs over the three-year project period were developed. Each one-to-two page research brief consisted of a four-line overview section, followed by more details about each of the four studies underway or completed. Occasionally, these would lead to telephone briefings and in-person presentations. During the funded research period, the PI made six inperson presentations on interim and final study results at VA head-quarters to a wide-ranging group of operational leaders. This allowed those in clinical operations to ask questions, provide comments, and suggest other directions for research, prior to the research being completed or presentations taking place at national conferences or published in peer-reviewed journals.

Our embedded research team also benefited from a highly engaged Stakeholder Advisory Board (SAB). This 9-member board, consisting of representatives from the U.S. Centers for Disease Control and Prevention, the U.S. Food and Drug Administration, Veterans Health Administration (not involved as operational partners) and a Veterans Service Organization who have a professional interest in large-scale adverse event disclosure and improving Veterans’ care, met virtually with the PI and project manager twice per year. Each session was geared towards the research team requesting input on upcoming research strategies or areas in which potential problems may occur. SAB members provided feedback, drawing on experiences in their agencies or evidence from other areas of the literature. Meetings were one-hour long, and an agenda and questions to consider were distributed two weeks prior to each meeting.

As a result of these engagement processes, members of the research-operations and SAB teams were invited to present panels at annual research conferences sponsored by VA, to highlight this research-operations partnership, giving equal attention to both arms of this embedded team, presenting results and discussing next steps.^21,22

3.3. An evidence-based large-scale disclosure toolkit (planning)

Our toolkit was designed to assist leaders and frontline providers at VA facilities with the implementation of our evidence-based disclosure strategies identified through our research.¹⁸ Current toolkit development guidance calls for the inclusion of educational material such as research summaries and supporting evidence for healthcare interventions; information on achieving change in organizations such as action plan checklists; templates or actual material to raise awareness of the activities needed; detailed materials for training staff to facilitate staff education; and other tools useful for staff such as a list of frequently asked questions, worksheets, or example forms.²³ As our SCALE research projects were guided by the Crisis and Emergency Risk Communication (CERC) model, developed by the Centers for Disease Control and Prevention, to assist health communicators, emergency responders, and leaders of organizations to communicate effectively during emergencies²⁴ Our toolkit disclosure activities correspond to the five stages of the CERC model: pre-crisis, initial event, maintenance, resolution and evaluation. We considered using two other crisis communication models to guide our work: 1) the Situational Crisis Communication Theory, which examines the consistency and distinctiveness of an event,²⁵ and 2) the Stealing Thunder model, which involves a timing strategy and a proactive approach to admitting weaknesses before another organization communicates these weaknesses.²⁶ However, the CERC model was chosen instead, because it combines effective crisis and effective risk communication principles into one model that views communication as a series of five developmental stages. General communication goals reflecting these five CERC stages, and specific responsibilities for each disclosure team are presented in Fig. 1. Table 1 provides a de-identified example of how Fig. 1 was operationalized in one disclosure event related to the improper sterilization of medical equipment. The complete Large-Scale Disclosure Toolkit, with examples of disclosure team members needed for each of the five CERC stages, along with checklists, templates, and scripts required for these teams’ communication activities, is presented in the supplementary material.

Fig. 1.

Toolkit is comprised of five sections based on CERC stages with action items for five teams.

Table 1.

Example of disclosure support program activities along CERC stage.

CERC Stage	Description of VA Large-Scale Disclosure Activities
Initial Event Stage	Following determination at a Clinical Episode Review Team (CERT) meeting that a facility would be required to disclose a large-scale adverse event, the HSR&D research team was asked to follow up with the facility to provide communications planning and support. On a follow-up phone call with facility leadership and the research team, edits were made to a draft script for communication with patients. The facility originally developed the script, and the research team, along with the VA Ethics Office and CERT leadership, provided edits on this call, as well as on an additional follow-up call and via email. Three days later, the research team met virtually via videoconferencing with the facility to present the toolkit process, including the description of the five teams needed to implement the disclosure activities, and support that could be provided by the research team. The research team provided a disclosure tracking spreadsheet for local use and discussed use of apology in communication in response to facility leadership questions. After a week, the research team and facility leadership met again by phone to discuss the planning for disclosure and follow-up with patients. The research team provided some insights and updates about the communications plan approval process from VHA Communications. Following this call, the facility made disclosures and conducted the necessary follow-up care with patients to whom disclosures were made.
Maintenance Stage	The research team conducted email follow-up with the facility team to check in on number of disclosures made, efforts to reach unresponsive patients, responses from patients, and any follow-up support needed.
Resolution Stage	Two months after disclosures began, the research team again followed up to debrief as part of the resolution stage. The research team asked about what went well with the disclosure process, and what could be improved for future communications.
Evaluation Stage	Two members of the research team, not involved in the disclosure training process, scheduled time with the facility participants in the disclosure process to conduct telephone interviews to evaluate the disclosure support program. Feedback from those interviews and from the resolution stage discussions were integrated into the toolkit stages, and in the pre-crisis planning for the facility and the research team.

CERC: Crisis and Emergency Risk Communication.

3.4. Clinical Episode Review Team participation (executing)

Recognizing that sharing of research findings and the toolkit with operational partners was not enough to lead to system-wide change, the VHA Undersecretary of Health invited our team to join the Clinical Episode Review Team (CERT), in order to work closely with VA program offices, medical facilities and Veterans Integrated Service Networks (VISNs) when disclosures were warranted. The CERT meets every other week to discuss Issue Briefs submitted by VISN leaders which identify events with a potential for large-scale disclosure, to determine next steps in the investigation process and whether or not large-scale disclosure (as opposed to institutional or clinical disclosures) is warranted. If CERT, along with its Subject Matter Experts, such as those from radiology, pathology, dentistry, and so forth, determine that large-scale disclosure is needed, the HSR&D team is then put into a rapid response mode, working closely with national and regional communication officers, medical center and VISN leadership, and public affairs staff to provide training on the disclosure identification, tracking, and communication processes detailed in our toolkit, beginning with the Initial Event stage activities (Fig. 1).

3.5. Toolkit evaluation (Reflecting and evaluating)

With CERT support, we implemented and evaluated the roll-out of our Large-Scale Disclosure Toolkit, which involved the primary implementation strategy of external facilitation–defined as a process of interactive problem solving and support that occurs in a context of a recognized need for improvement and a supportive interpersonal relationship²⁷–by the research team at two facilities where we initially tested this toolkit implementation during real-time disclosures. The PI and project manager contacted the Chief of Staff at the medical center where the disclosure was to be made, to describe the toolkit disclosure process, and to help the leadership team identify which employees would be part of each of the five disclosure teams. We then conducted videoconference trainings on each toolkit stage and the specific activities involved with these employees, who were often primary care providers, primary care nurses, social workers, public affairs officers, and medical center service line directors or hospital leaders, and as needed, conducted follow-up trainings with these same team members. Disclosure templates and scripts were tailored for each site’s specific large-scale adverse event and disclosure team, working with VHA Public Communications and the public affairs staff at the specific facility or region. Two members of the research team who were uninvolved with the implementation evaluated the utility of the toolkit and training. These two researchers conducted semi-structured telephone interviews with eight key stakeholders involved in these trainings and disclosures at the facility, VISN and national operations level. Interview data were coded in NVivo 10 ²⁸ using an a priori approach²⁹ based on CFIR and an emergent thematic analysis³⁰ to identify factors related to stakeholder perceptions of the toolkit, disclosure support services, and facilitation as implementation strategy.²⁷

Qualitative analyses revealed themes mapped to CFIR domains of the Characteristics of the Intervention (the disclosure policy and use of the disclosure toolkit), Characteristics of the Individuals (those involved in the disclosure process), the Inner Setting of an organization (the specific VA facility, as well as VA Central Office), and the Implementation Process. Participants had few, if any, standardized resources to guide disclosure so the toolkit provided a blueprint that was more advantageous to use. The design of the toolkit was perceived as accessible as well, and this increased its use. Additionally, the embedded research team provided support services directly to sites and this support was praised for flexibility, responsiveness and content expertise. A public affairs officer noted:

The toolkit, presentation and support offered made all of us think about what we had already done, what we needed to do and what we planned to do. It made us sure that we had covered things. The Chief of Staff really liked the support and personalized attention offered by [the team], we all did too. It was all useful and not one piece of the toolkit or presentation was more important than any other but the whole process helped us gain confidence that what we were doing was right. It brought uniformity and it made us think about things we had not thought of before. I really learned that it is important for the person who does the disclosure to have some sort of relationship with the Veteran.

One VA leader stated that in large-scale disclosure there were “many beliefs that were hotly contested by people in the position of making the disclosure” and that “research made an invaluable contribution” to improving the knowledge of those in charge of making disclosures and also helping change people’s belief systems. Another leader stated:

[The team] brought a perspective gained through research. They really knew through research that Veterans like to know more, not less, in a disclosure. It was their research data and experience that made them helpful. They had the ability to point to specific research to back up what they were saying. They were the experts.

Facilitation by the embedded research team was found to be practical (the communication management tracking sheet the team created and disseminated), reassuring (knowledge that other facilities faced similar situations) and objective (research team facilitators were content experts). For example, a clinic leader, who was leading disclosure at one facility, said:

We developed a succinct plan [for the disclosure] and the presentation [of the toolkit] showed us how to go about it. [The research team] modeled how to do things and helped me organize and prepare carefully … I really liked the quotes from other employees [that appear in the toolkit] who had to deliver disclosure notices. Others have felt the same way I did. It is important to have a standardized way of doing things in a process that is confusing and chaotic.

Despite some successes, full implementation and use of the toolkit was impeded through the complexity and slow pace of the large-scale disclosure decision processes. Further, facilities often operate in a crisis mode during the disclosure notification process, such that sorting through a 65-page comprehensive toolkit posed a challenge without telephone, videoconference or email guidance. One VA leader stated that the research team’s ability to personalize the toolkit through their training was a great strength. Importantly, this leader stated: “The toolkit with the support services is not a rigid product. The VA has enough toolkits and dashboards, tons in fact, but what people do not have is help. And help is what we need”.

Another participant, a VISN Continuous Readiness Officer, also emphasized help and training:

“The phone training was very good and I really like that the toolkit has benchmarks. [The team’s] whole approach does not involve shaming and focuses on giving objective information about how to make decisions to disclose and how to do the disclosure. I really like that they focus on the process, not the person. By focusing on the process, not the people, this creates an open environment that makes people willing to answer questions and makes the atmosphere comfortable and supportive for people.

For full dissemination and sustainment, the toolkit and facilitated support services need to be readily available. One operational leader believed that “no facility did or would use the toolkit in its entirety” and that “people would take bits and pieces of it and find things they liked and it was these things that they would use”. This leader’s overall belief was that “a redesign of the toolkit [is needed], to make it easier to use in bits and pieces, something shorter for busy leaders who might only look at it when there is a fire”.

3.6. Disclosure support program (Reflecting and evaluating)

Reflecting on our evaluation interviews and specifically the above operational leaders’ feedback, we created a web-version of our toolkit, termed the Disclosure Support Program (Fig. 2). The Disclosure Support Program was available through VA Pulse, a VA-specific social media platform. Any VA employee was able to access any part of the disclosure toolkit at any time. Requests for training were also made via this page. The toolkit and program were publicly launched in 2018 during a research-operations workshop describing how CERT operates as an integrated healthcare system response effort for identifying and disclosing large-scale adverse events.²²

Fig. 2.

Web-based disclosure support program.

3.7. HSR&D continuous service to CERT (executing)

In October 2018, the VHA updated VHA Directive 1004.08 on disclosure of adverse events, which included a new CERT standard operating procedure (SOP). The HSR&D embedded research role in this SOP involves assisting VA facilities and VISNs with all aspects of the disclosure process. HSR&D also tracks the types of potential large-scale adverse events that are brought before CERT. The Directive allows for one of the following CERT recommendations: 1) disclosure is not required and case is closed; 2) convene subject matter expert review panel to conduct further fact-finding; 3) convene clinical review board and/or 4) proceed with large-scale disclosure. Between January 2016 and October 2019, CERT, comprised of 20 core VA program office members, reviewed 188 events that had the potential for causing harm to multiple patients. Four main types of large-scale disclosure issues were reviewed: equipment reprocessing lapses, delay of care, provider/employee behavior, and technology/software issues. Of these events, many resulted in lookback investigations and large-scale disclosures, involving over 10,000 patients in the aggregate. When disclosure is recommended, the HSR&D research team works directly with the facility, VISN and VHA Communications and Public Health to use the toolkit during real-time disclosures.

4. Unresolved question: operational sustainment

As mentioned by stakeholders, a toolkit is not enough for crisis communication situations; tangible help is required. Although HSR&D funding for the SCALE project ended in 2015, two members of the team continue serving on CERT. As grant-funded researchers, their time is supported financially through a Memorandum of Understanding with VHA Clinical Operations, renewed each year. However, as leadership changes occur and research priorities shift, training and use of the disclosure toolkit needs to become integrated within the overall CERT, and ideally not as an activity that belongs only to research. CERT program office members are now often the first to point out the need for Veterans and family members to learn about all aspects of a large-scale adverse event; VHA Communications builds on previously developed toolkit communication templates and scripts in working with each new facility; and the process of communication seems to instinctively follow toolkit procedures. Currently, the Veterans Health Administration is reorganizing clinical operations, research and policy offices under a new office of “Clinical Services”, overseen by the Assistant Undersecretary for Health. As part of this reorganization, the Office of Healthcare Transformation is working with all VA medical centers and regional office to standardize the CERT reporting, computerized monitoring and surveillance of large-scale adverse events, and to develop protocols for dissemination of the disclosure toolkit to the field. Thus, the path to sustained support of VA facilities during disclosure seems achievable.

5. Lessons for the field: VA and other health care systems

Our work is complementary to the communication-and-resolution programs (CRPs) being implemented in non-VA healthcare systems across the country.³¹ In these CRPs, health systems and liability insurers encourage the disclosure of unanticipated care outcomes to affected patients and proactively seek resolutions, including offering an apology, an explanation, and, where appropriate, reimbursement or compensation. Critical implementation strategies for CRPs involve organizational readiness for this change, clinical leader and patient engagement to learn from each perspective, investing in tools and resources to support communication and resolution, and monitoring and tracking progress of how these resources are used. Similar to the Disclosure Support Program discussed here, implementation of these strategies requires ongoing partnerships between healthcare operational leaders and researchers. Evaluations of the CRPs have identified key implementation insights for hospital leaders, including the presence of a strong institutional champion, investing in building and marketing the program to skeptical clinicians, and making it clear that the results of such transformative change will take time.³² Similar to our VA Large-Scale Disclosure Toolkit, the Agency for Healthcare Research and Quality’s Communication and Optimal Resolution (CANDOR) toolkit offers methods and tools for hospital leaders and physicians to use to respond immediately when patients are harmed by the medical care they receive.³³ And further similar to the VA Disclosure Support Program, the Collaborative for Accountability and Improvement, a non-profit organization consisting of a network of the healthcare leaders, attorneys, insurers, patient advocates, and researchers who pioneered the earliest CRPs in the United States, was created to help non-VA medical centers implement the CANDOR toolkit for themselves.³⁴

Guided by an implementation science framework, our embedded research team was able to determine what works in disclosure of VA large-scale adverse events, for whom, and in what contexts.¹⁷ Critical lessons learned for our embedded research-operations partnership included a need for stakeholder collaboration and engagement, an organizational culture involving a strong belief in evidence, a willingness to embed researchers in clinical operation activities, allowing for testing and evaluation of innovative practices, and researchers open to constructive feedback. A research-operations partnership involving planning, engaging, executing, reflecting on and evaluating these disclosure research activities, together, led to a VA sanctioned role for research in clinical operations. Ongoing sustainment of the disclosure toolkit requires a multi-stakeholder leadership commitment and ownership of the disclosure support process.