Implementation of New Clinical Programs in the VHA Healthcare System: The Importance of Early Collaboration Between Clinical Leadership and Research

ABSTRACT

Collaboration between policy, research, and clinical partners is crucial to achieving proven quality care. The Veterans Health Administration has expended great efforts towards fostering such collaborations. Through this, we have learned that an ideal collaboration involves partnership from the very beginning of a new clinical program, so that the program is designed in a way that ensures quality, validity, and puts into place the infrastructure necessary for a reliable evaluation. This paper will give an example of one such project, the Lung Cancer Screening Demonstration Project (LCSDP). We will outline the ways that clinical, policy, and research partners collaborated in design, planning, and implementation in order to create a sustainable model that could be rigorously evaluated for efficacy and fidelity. We will describe the use of the Donabedian quality matrix to determine the necessary characteristics of a quality program and the importance of the linkage with engineering, information technology, and clinical paradigms to connect the development of an on-the-ground clinical program with the evaluation goal of a learning healthcare organization. While the LCSDP is the example given here, these partnerships and suggestions are salient to any healthcare organization seeking to implement new scientifically proven care in a useful and reliable way.

INTRODUCTION

Collaboration between research and healthcare delivery systems is crucial to achieving the goal of proven quality care. Since the 1960s, large integrated delivery systems such as the Veterans Health Administration (VHA) and Kaiser Permanente have engaged in cutting-edge health services research designed to directly apply to and improve care.^1,2 Over the last two decades, both integrated delivery systems and academic health systems have connected research centers to health delivery systems.^1,3,4 Such collaborations can be especially valuable, as health systems implement new clinical programs to ensure that they will be able to answer the critical questions about how effective the programs are, how to maximize safety and value, and how to make sure they are implemented reliably and efficiently. However, the goal of tightly integrated partnerships among researchers, health system managers, and policy makers has not always been fully realized, reducing the impact of health services research (HSR) on the health system’s performance.⁵ HSR plays multiple roles within a learning healthcare system. One of the most common roles for HSR is to examine areas of variation in quality or outcomes. Similarly, HSR can test interventions to improve quality, safety, and the value of healthcare. An equally important but somewhat less studied role for HSR is how to implement a completely new technology or program that has been developed and proven effective in research studies but never applied widely throughout the healthcare system.^6,7 It is much more common to let new interventions diffuse gradually and to slowly accumulate lessons from adopters in an unsystematic fashion. The purpose of this paper is to describe how the VHA brought together key stakeholders in policy, health care delivery, and research to develop a much more systematic and comprehensive plan to collaboratively create and evaluate a new clinical program around lung cancer screening (LCS).

INTEGRATING AND APPLYING RESEARCH RESULTS THAT MAY IMPACT A LARGE HEALTH SYSTEM

In 2011, the National Lung Screening Trial (NLST) published results indicating that among patients at high risk for lung cancer based on smoking history and age, there was a 20 % mortality benefit for patients with annual LCS with low-dose computed tomography (LDCT) for three years compared to those screened with chest radiography.^8,9 As a result, both the American Cancer Society and US Preventive Services Task Force (USPSTF) issued recommendations for annual lung cancer screening.^10,11 However, potential harms of LCS for patients and consequences for healthcare systems are as yet unknown.^12,13 Despite recommendations for LCS by the USPSTF,¹¹ many others raised important questions about potential harms of screening and unknown downstream consequences of large-scale screening efforts.^14–17 There is a lack of evidence on how to establish a comprehensive screening and follow-up LCS program that would replicate the effectiveness and safety of NLST.

Nearly 20 % of the Veteran population uses tobacco,¹⁸ and lung cancer is the second most common cancer in the VA, with over 8,000 new cases annually.¹⁹ VHA leaders recognized that the mortality reduction from lung cancer with LCS represented a potentially significant benefit for VHA patients. Given the potential impact for the VHA, the VHA National Center for Health Promotion and Disease Prevention (NCP) was tasked by senior clinical and policy leadership to develop an LCS demonstration project to assess the impact of an LCS program on VHA system resources and quality of patient care.

In partnership with other VHA stakeholders in key clinical and policy areas, NCP developed a Lung Cancer Screening Demonstration Project (LCSDP), which is being conducted at eight selected VA medical centers. These sites were chosen on the basis of their interest and ability to actively participate in the LCSDP. A steering committee was formed, comprised of leaders within primary care, pulmonology, radiology, oncology, tobacco cessation, and HSR, among others. The steering committee guided design of the project and individual members took the lead on specific responsibilities related to their areas of expertise (e.g., Radiology developed a radiology dictation guide for systematic reporting of LDCT findings). NCP reached out to HSR to lead the evaluation of the project, and to the Veterans Engineering Research Center (VERC) to develop electronic clinical tools needed for patient care related to LCS. Figure 1 provides a time line of key events in program development and implementation. Rather than research urging health systems to implement care based on research findings, we describe a partnership between clinical, policy, and research leadership for the design, planning, and implementation of LCS, with the goal of helping inform potential national implementation throughout the healthcare system.

Figure 1.

Timeline of development of LCSDP.

PROGRAM AND EVALUATION PLAN DEVELOPMENT

The first step was to solidify the project’s objectives. HSR used the Donabedian Quality Matrix to define key components of a quality LCS clinical program (Table 1).^20,21 Broadly, this included clinical outcomes for patients, policy outcomes for the healthcare system, and the structural outcomes surrounding the process of implementation (Table 2). This is a hybrid approach that focuses on both intervention and implementation effectiveness.⁷ Whereas a research study designed to establish efficacy can rely on a network of study investigators and coordinators to ensure appropriate recruitment, screening, follow-up and data collection, a real-world implementation evaluation requires those functions be integrated into the routine delivery of care.

Table 1.

Donabedian Quality Matrix for Lung Cancer Screening Demonstration Project

	Access	Technical Management	Management of Interpersonal Process	Continuity
Structure	• PACT team member designated to coordinate LCS process. • Adequate radiology staff and CT machines/time to perform screening and required follow-up at a given center. • Resources to treat new cancer diagnoses.	• System for capturing appropriate patients. • Systematic method to measure the number of procedures performed as a result of LCS. • Established protocols for performance and reporting of CT scans.	• Providers/facilities have decision-support materials needed to implement LCS. • Providers have sufficient information/ understanding of background literature to have full discussion with patient regarding risks/benefits of LDCT screening. • Providers have adequate time to address lung cancer screening with eligible patients, without affecting other primary care responsibilities and medical center responsibilities	• Standard operating procedures developed for handoff of patients between specialties along the lung cancer screening and follow-up process. • Report templates and/or progress notes available for each step of the screening and follow-up process.
Process	• Equitable distribution of screening resources that reflect the targeted patient population. • Patients who do not qualify for LDCT screening by NLST criterion are not getting screening unnecessarily. • LDCT will occur within reasonable period from being ordered. • Diagnostic CT for positive screens will be done within reasonable period from being ordered. • Treat new cancer cases (at medical center or referral medical center) in a timely manner [based on diagnosis to first treatment]. • Nodules found on LCS followed up in an appropriate manner per Fleishner guidelines. • CT procedures for other indications will be done within same time frame as prior to initiation of LC screening project.	• Facilities participate in lung cancer project activities. • Lung cancer screening reminders are fully satisfied/completed for all patients for which they are activated. • No significant deviation from NLST treatment protocol. Specific measures of guideline concordance to be developed in consultation with appropriate work groups. • Ability to estimate the time needed to complete radiology procedures involve in the LCS program. Time intervals include: • Time to perform a CT on a patient, both room time and technician time. • The time it takes to read and report a CT scan. • Number of fluoroscopic and CT guided biopsies and times to perform those procedures. • Algorithm developed to link LCS LDCTs to national databases. • Ability to estimate the number and types of additional imaging.	• Patients have the information necessary to make an informed decision regarding screening. • Patients with triggered reminder have discussion with PCP about screening benefits and risks recorded as part of the lung cancer screening reminder. • Patients are informed of screening results in a timely manner.	• Patients requiring further imaging/biopsies will receive such care in a timely manner. • Newly identified lung cancers are treated in a timely manner (i.e., no more slowly non-screened lung cancer patients). • Report templates and/or progress notes available for each step of the screening and follow-up process utilized by facilities. • Barriers and facilitators to ongoing implementation of lung cancer screening are identified. • Plan established to continue screening program if it is determined to be successful and feasible.
Outcome	• Screened patients diagnosed at an earlier stage than eligible patients at the same medical center and similar patients at other medical centers. • Budget impact—cost for the number of LDCT tests, based on various levels of uptake. • Budget impact—costs of biopsies, bronchoscopies, surgical procedures, etc. • Budget impact—cost (or savings) resulting from changes in patient utilization. • Patients requesting access to Stop Smoking clinic/ resources are provided these in a timely manner • Patients requesting medications to assist in quitting smoking are provided these. • No decrease in number of patients desiring to quit smoking.	• Predicted decrease in lung cancer mortality from lung cancer based on differences in stage and performance status at the time of diagnosis among screened and matched unscreened individuals. • False positive rate similar to clinical trial patients. • Minimal inter-reader discrepancy and minimal missing data requiring imaging study to be re-read. • Report templates and/or progress notes available for each step of the screening and follow-up process utilized by facilities have required information.	• Patients do not experience undue anxiety related to being screened or having a positive screen. • Patients indicate understanding of the potential benefits and risks of lung cancer screening.	• Patients screened in year 1 and appropriate for continued screening will be screened in years 2 and 3. • Patients adhere to recommended follow-up from + LDCT or continue annual LDCT if initial LDCT is negative. • Facilities have developed systems needed to continue to program if found to be effective. • Minimal adverse events related to screening and biopsies (especially in false positive LDCTs).

The layout of this Donabedian quality matrix is based on Donabedian, A. (1980). Explorations in Quality Assessment and Monitoring. Volume 1. The Definition of Quality and Approached to its Assessment. Ann Arbor, MI: Health Administration Press and information presented in Health Policy and Administration 263, Quality and Utilization Management, at the University of North Carolina at Chapel Hill, Fall 1997, Susan I. DesHarnais, Ph.D., instructor.

Table 2.

Outcomes To Be Measured

Clinical Outcomes	Structural Outcomes	Policy Outcomes
Demographics and smoking history of patients screened for eligibility for LCS	Capacity of healthcare facilities to identify potentially appropriate patients	Standard operating procedures for follow-up of patients with suspicious findings on LCS
Demographics and smoking history of patients completing LCS	Capacity of healthcare facilities to provide screening	Budget impact of LCSDP
Nodule follow-up intervals based on nodule size and type	Consistency of radiology reading of LDCTs and recommended follow-up	Recommendations for staff equipment, and processes needed to fully implement LCS across VHA
False-positive rate of LCS	Number and type of additional imaging studies and invasive procedures as a result of LCSDP
Adverse event rate for invasive procedures related to LCS	Organizational readiness to adopt LCSDP.
Number and stage of newly diagnosed lung cancers	Number and demographics of patients requesting assistance with stopping smoking with LCSDP cohort
Number and demographics of patients returning annually for LCS
Number and type of incidental findings requiring follow-up

Data collection needed to be incorporated into the routine care process rather than treated as an add-on research function. As an example, NCP partnered with the VERC to develop clinical reminders that identified appropriate patients, based on age, smoking history, and other clinical information that might affect a decision about screening, such as limited life expectancy. Note templates were designed to allow local sites to input screening results and follow-up plans. Data elements created through the use of note templates and clinical reminders automatically fed into a nodule tracking database used for daily clinical operations, and into the evaluation database for use in ongoing assessments of the program.

PRE-IMPLEMENTATION COLLABORATION

NCP, HSR, and VERC agreed a priori on essential components of the program to be implemented uniformly across the sites. These components were discussed with the demonstration sites so that they could create processes that worked for them individually, while still maintaining fidelity with the key elements needed for successful implementation and evaluation. Implementation of the LCSDP is a formative process with ongoing communication between NCP, HSR and the sites to better understand the alterations necessary to accommodate differences in clinical environments and available resources (e.g., radiology capacity, division of labor among medical providers.) Understanding and accommodating this variation in implementation was crucial for developing a collaborative spirit, while still understanding how variation might affect data collection and analysis.

IMPLEMENTATION OF THE PROGRAM AND EVALUATION

Implementation was carried out in a phased approach at each site, with early adopter primary care providers participating in the program ahead of others. These early adopters were selected on the basis of their willingness to be “pioneers” for the project. Subsequently, additional providers have been added gradually without regard to their specific interest in screening. This approach was used so as not to overwhelm clinical areas with a large number of patients to be identified and screened at once. It also allows for the development of a comparison cohort of providers not yet participating in the screening process. As this process continues, HSR has been conducting a mixed-methods evaluation that includes quantitative and qualitative data. Quantitative data regarding number and characteristics of those screened and outcomes of screening are pulled from national databases. Surveys of site implementation teams and providers are ongoing to evaluate perceptions of their organization’s readiness to change,^22,23 and their knowledge of and acceptance of recent LCS recommendations based on the Weiner Organizational Theory of Implementation Effectiveness.²⁴ Qualitative interviews have been used to assess the degree of implementation readiness and potential barriers and facilitators of program implementation. Additional surveys and qualitative interviews will be done as the program progresses. The data generated from the quantitative evaluation of LCS processes in conjunction with qualitative data will refine our understanding of the implementation process over time.

As sites expand LCS, the evaluation team will provide regular reports to NCP about the impact of screening (e.g., number and characteristics of those screened, benefits and harms to patients, impact on workflow) at each of the sites. NCP will be monitoring accrual at each site, and reporting to the individual sites and to VHA clinical and policy leadership.

OPPORTUNITIES AND CHALLENGES OF THE PARTNERSHIP

There are multiple benefits to the healthcare system and ultimately to patients from partnerships between research, policy, and clinical care. Standardized protocols and ongoing monitoring of data collection and care delivery, coupled with rigorous evaluation and analysis can help define and measure quality of care and care delivery. Working in tandem with healthcare deliverers allows for the rapid, seamless integration of findings and implementation of best processes and practices into the healthcare system, allowing patients to benefit from the findings in a timely manner.

There are challenges to these partnerships as well. Researchers are accustomed to standardized protocols to ensure data integrity, promote patient safety, and enhance efficiency. Those who deliver health care are faced with multiple and often competing demands in a changing healthcare environment with changing priorities and resources. While the goal of each is to improve patient care, the different perspectives, cultures, and demands may create tensions that can threaten the partnership if not addressed. Information may need to be collected in non-traditional ways to accommodate variability among clinical settings. Providers may need to alter methods for patient identification and enrollment in the program due to staffing needs and space constraints.

Defining what questions can and cannot be answered by this project, and which questions are most important to the healthcare system, also requires collaborative discussion. The purpose of a hybrid design using real-world implementation in selected centers is not to establish the true effectiveness of LCS in the new population.⁷ Nonetheless, this project will analyze intermediate outcomes data such as uptake of screening, accuracy and yield of screening, stage distribution of cancers detected, and surgical outcomes, which can all give insights into whether our results mirror the conditions under which screening is likely to have a significant health benefit. Similarly, our project can measure harms of screening accurately and project these costs for screened populations. Flexibility among partners, creative problem solving and compromise are critical components of a successful partnership. Engagement of all stakeholders in the partnership is also crucial to effective program implementation and evaluation.

CONCLUSION

Providing high-quality healthcare to our nation’s Veterans is at the heart of the VHA mission. To achieve that goal, it is imperative that clinical and research partners work together closely and collaboratively, both early and often. Often, clinical policy decisions need to be resolved in the face of incomplete or imperfect evidence, but clinical–research partnerships can help generate better evidence and more effective practice as new programs are implemented. It is only through partnership that we can jointly implement care, measure its impact, and ensure its quality. This is the goal of the Lung Cancer Screening Demonstration Project: to implement a new program based on recent evidence and guidelines, and evaluate that program in real time. We are on the cusp of seeing whether LCS becomes the standard of care, and the lessons learned here can be applied to other healthcare systems seeking to implement LCS. Partnerships between clinical leadership and research can be effective. The shared goal of improving patient care, combined with a spirit of collaboration and an ability to compromise, are essential to their success. LCS is another in a long series of partnerships in the VHA that have been created to improve patient care.^25–28 The LCSDP is an example of a partnership that can be used by the VHA as a model to implement new clinical programs. Lessons learned from the VHA can serve as an example to other healthcare systems as they develop partnerships to implement new clinical programs.