Quality and Value of Health Care in the Veterans Health Administration: A Qualitative Study

Abstract

Background

The attitudes of Department of Veterans Affairs (VA) cardiovascular clinicians toward the VA's quality‐of‐care processes, clinical outcomes measures, and healthcare value are not well understood.

Methods and Results

Semistructured telephone interviews were conducted with cardiovascular healthcare providers (n=31) at VA hospitals that were previously identified as high or low performers in terms of healthcare value. The interviews focused on VA providers’ experiences with measures of processes, outcomes, and value (ie, costs relative to outcomes) of cardiovascular care. Most providers were aware of process‐of‐care measurements, received regular feedback generated from those data, and used that feedback to change their practices. Fewer respondents reported clinical outcomes measures influencing their practice, and virtually no participants used value data to inform their practice, although several described administrative barriers limiting high‐cost care. Providers also expressed general enthusiasm for the VA's quality measurement/improvement efforts, with relatively few criticisms about the workload or opportunity costs inherent in clinical performance data collection. There were no material differences in the responses of employees of low‐performing versus high‐performing VA medical centers.

Conclusions

Regardless of their medical center's healthcare value performance, most VA cardiovascular providers used feedback from process‐of‐care data to inform their practice. However, clinical outcomes data were used more rarely, and value‐of‐care data were almost never used. The limited use of outcomes data to inform healthcare practice raises concern that healthcare outcomes may have insufficient influence, whereas the lack of value data influencing cardiovascular care practices may perpetuate inefficiencies in resource use.

Short abstract

See Editorial Heidenreich et al

Clinical Perspective

What Is New?

• Department of Veterans Affairs clinicians have extensive familiarity with process‐of‐care quality data, but they are limited in both their awareness and use of clinical outcomes and value‐of‐care data.

What Are the Clinical Implications?

• Outcomes and value are increasingly viewed as important to healthcare stakeholders, including patients, providers, and payers.

• This study identifies a dearth of outcomes and value data informing current clinical practice within the Department of Veterans Affairs, and it identifies an opportunity for outcomes and value information to be more frequently measured and more commonly used in routine clinical care settings.

Introduction

The US Department of Veterans Affairs (VA) healthcare system has been intently focused on improving quality of care for the past 25 years.1 Additionally, there is an increasing imperative for the VA to provide health care that is not only high in quality, but also high in value (ie, the ratio of good clinical outcomes produced per healthcare dollar spent),2 particularly in an era in which the VA is increasingly engaging private sector providers to deliver care.3 Healthcare quality is classically defined using Donabedian's framework of structure, process, and outcomes.4 Although many of the VA's quality efforts have been designed to improve the processes of care throughout its national system, there nevertheless is evidence that healthcare outcomes vary widely across the VA,5 potentially revealing a disconnect between the processes of care on which the VA has focused and the patient‐centered healthcare outcomes that are the ultimate goal. Furthermore, costs of care vary widely across the VA system,6 suggesting underlying variation in the delivery of high‐value care.

Despite the VA's focus on quality, relatively little is known about how the VA's front‐line clinicians regularly engage with the concepts of healthcare outcomes and value in the conduct of their clinical work. There are substantial opportunities and incentives within the VA for outcomes and value measurements to be influential in VA health care. For example, the VA's national databases include a remarkably robust mortality database,7 hypothetically presenting the opportunity to inform VA clinicians of the clinical outcomes of the care they provide. Furthermore, the VA's fixed‐budget financing provides a strong incentive to ensure that limited resources are used efficiently.8 However, the VA's historical focus on process measures of quality, including incentive programs for both VA clinicians and administrators that reward high performance on process‐of‐care benchmarks,9 may “crowd out” clinical outcomes and heathcare value as influential factors on the routine delivery of VA clinical care.10

Therefore, the goal for this research was to qualitatively assess VA clinicians’ familiarity with and attitudes toward the VA's efforts to measure and improve quality‐of‐care processes, clinical outcomes, and healthcare value at their medical centers.

Methods

Transparency and Openness Promotion

Because of the sensitive nature of the data collected for this study, requests to access a de‐identified version of the data by qualified researchers trained in human subject confidentiality protocols may be directed to the Corporal Michael J. Crescenz VA Medical Center at peter.groeneveld@va.gov. Any release of data would require previous approval by the Corporal Michael J. Crescenz VA Medical Center's Research and Development Committee.

Study Overview

We conducted a qualitative assessment of use and perceptions of cardiovascular healthcare processes, outcomes, and value, using semistructured interviews of cardiovascular healthcare providers (n=31) in the VA. The design and reporting of our study followed published guidelines for reporting qualitative research.11 Participants were recruited from VA facilities identified in a previous analysis as either high‐performing (n=10) or low‐performing (n=10) medical centers in terms of cardiovascular outcomes and costs. Details are published elsewhere12; in brief, these centers were identified by ranking VA medical centers (VAMCs) according to their cardiovascular mortality per healthcare dollar spent, using the VA's national healthcare data, expenditure data, and mortality data from 2010 to 2014. The facilities in the top 10 and bottom 10 of this rank list were included in the current study. The 20 facilities from which participants were recruited comprised a broadly based national sample of VA facilities.

Investigator Team

The multidisciplinary research team members were of varied backgrounds (medicine, psychology, public health, and health services research), and the team had extensive experience in qualitative research methods. A highly experienced qualitative researcher (K.L.R., a PhD‐trained medical sociologist), not previously known to the participants, conducted all telephone interviews between October 2017 and July 2018. Duration of each interview ranged from 12 to 48 minutes (mean=23.5). Interviews varied in duration because of the open‐ended nature of the questions, with some participants being more succinct in their responses and/or having more‐limited insight into the domains of interest. Verbal informed consent was obtained from each participant before beginning each interview.

Study Design

We selected individual semistructured interviews as our research modality because our goal was to gather in‐depth insight into each provider's experience of cardiovascular care at his or her particular VA facility13 and to compare these insights across high‐ and low‐performing centers. We applied Donabedian's structure‐process‐outcomes quality framework4 to the development of the interview guide, and we utilized a content analysis approach to its design. Our interview questions focused sequentially on the domains of processes, outcomes, and value, with the goal of gaining insight into the healthcare providers’ experiences with these 3 domains at their VA facility, as well as understanding each facility's organizational structure and policies relating to the collection and dissemination of data measuring components of these 3 domains.

Process measures, borrowed from the Donabedian framework, were defined (and explained to the interview respondents) as measures of whether the “right action was being done at the right time in the right patient.” Outcomes were defined as the end‐result experiences of health care directly affecting a patient's duration of life, quality of life, and/or lived experience. Last, value was defined as the ratio of favorable healthcare outcomes per healthcare dollar spent, in accord with recent definitions of “value” in the medical literature.2, 14 While not all respondents specifically used the term “value” to describe the economic aspects of their care, we attributed respondents’ explicit or implicit considerations of the cost of a healthcare service relative to the benefit it provided as an expression of the value concept. We were interested in learning each facility's approach to the processes of cardiovascular care, the outcomes experienced by their patients, and whether value impacted providers’ clinical decision making.

Participant Selection

Potential participants who fit eligibility criteria (healthcare providers involved in cardiovascular care at the selected VA facilities) were identified using VA intranet searches of facilities’ clinical staff listings, as well as “snowball sampling” whereby participants referred colleagues at their VA facility who might be interested in the study and could provide insight. In particular, we asked participants to provide names of additional individuals who would be able to provide knowledgeable insight on cardiovascular care quality, outcomes, and value at their medical centers. E‐mail addresses were found using the VA's Global Address List. Eligible providers were initially contacted individually by e‐mail by the study's research coordinator (L.W.), with 1 follow‐up email sent by the study's Principal Investigator (P.W.G.).

Statistical Analysis

All interviews were audio recorded and transcribed/verified verbatim, with redactions to anonymize identifying information (eg, names and worksites). After 4 interviews were conducted, the study team developed a comprehensive codebook that was both informed by the interview guide and relevant literature, as well as through line‐by‐line reading of these first transcripts. Each code was given an explicit definition to ensure uniform meaning and guidelines for the 2 coders. Responses were coded based on our definitions of each domain (ie, process, outcomes, and value), even if the response did not coincide with the domain addressed in the question (eg, a question regarding value measurement eliciting a response about outcomes, etc).

The first 4 transcripts were co‐coded together by the primary and secondary coder (K.L.R. and K.L.H.) using the qualitative analysis software NVivo 11 Pro for Windows (QSR International, Doncaster, Australia) to allow for familiarization with the codebook and its application. Then, the primary coder coded the remaining 28 transcripts, with the secondary coder coding every third transcript. In total, 9 transcripts were independently co‐coded by 2 members of the research team to ensure coding consistency. Inter‐rater reliability was assessed by calculating Cohen's kappa statistic for each of our codes for the 9 co‐coded transcripts. Any coding discrepancies between coders were eventually resolved by negotiated consensus15 whereby the 2 coders discussed their coding logic and reached an agreement on the appropriate code. This consensus process was adjudicated by the Principal Investigator as necessary. An audit trail (ie, a detailed, comprehensive accounting of all data collection processes, analytical decisions, and research activities) was completed. To identify emergent themes, transcripts were reviewed by the coders and interviewer, and related codes were combined into categories. These categories were further aggregated into the 5 themes described below. The titles and content of the themes were selected by group discussions involving all members of the research team.

The study was approved by the institutional review boards at the Michael J. Crescenz (Philadelphia) VA Medical Center and the VA Pittsburgh Healthcare System.

Results

Five hundred seventy‐six potential participants were contacted, 36 providers expressed interest in participating, and a total of 31 cardiovascular care providers were ultimately interviewed. Fifteen providers refused participation (citing lack of time or lack of current involvement in VA cardiovascular care), and 525 did not respond. Respondents represented 11 VAMCs, including respondents from 5 high‐performing centers in terms of cardiovascular value and 6 low‐performing centers (Table 1). The mean of the inter‐rater correlation coefficients (kappa) values for the 9 interviews that were independently coded by 2 coders was 0.95, with 91% of the text coded with kappas exceeding 0.8, and the mean intercoder agreement was 99.7%, indicating very high inter‐rater reliability.16

Table 1.

Characteristics of Participants in Semistructured Interviews on Processes, Outcomes, and Value of VA Cardiovascular Care

Characteristic	All Respondents (N=31)	High‐Performing Hospital (N=22)a	Low‐Performing Hospital (N=9)a	P Value
Unique VA facilities	11	5	6	···
ICU on‐site	11	5 (100)b	6 (100)	1.00
Total bed days per 1000 Veterans, median [IQR]	28 541 [18 225, 37 404]	28 541 [24 595, 28 762]	26 832 [15 884, 40 764]	0.19
Professional role
Physician	21 (68)	16 (76)	5 (24)	0.42
Nurse/nurse practitioner	10 (32)	6 (60)	4 (40)
Leadership positionc	11 (35)	8 (73)	3 (27)	0.61
Years worked in VA, median [IQR]	10 [5, 17]	13 [8, 19]	5 [3, 6]	0.01
Cardiovascular specialistd	14 (45)	11 (79)	3 (21)	0.33
VA facility complexitye
IA‐IC	28 (90)	22 (79)	6 (21)	0.02
II to III	3 (10)	0 (0)	3 (100)
US census region
Northeast	0 (0)	0 (0)	0 (0)	<0.001
Midwest	20 (65)	20 (100)	0 (0)
South	7 (23)	0 (0)	7 (100)
West	4 (13)	2 (50)	2 (50)

ICU indicates intensive care unit; IQR, interquartile range; VA, Department of Veterans Affairs.

^aVA hospitals’ cardiovascular value performance (high=top 10, low=bottom 10), based on a previous analysis of VA cardiovascular outcomes and cost data from 2010 to 2014.12

^bNumbers are N (%), unless otherwise specified. Percentages may not sum to 100% because of rounding.

^cRespondent reported that they held a clinical leadership position at their VA hospital.

^dRespondent self‐identified as a cardiologist or nurse/nurse practitioner specializing in cardiovascular care.

^eVA hospitals are classified by institutional “complexity” from highest (1A) to lowest (3) based on each hospital's breadth of services, volume of care, and technical capacity.

Comparing Responses From High‐ and Low‐Performing VAMCs

In comparing the responses between VA hospitals that were high and low performers in terms of cardiovascular value of care, we did not observe any material differences in the content and themes of respondents’ interviews (Table 2). We therefore proceeded with the analysis by identifying “global” themes that were mentioned by respondents in both high‐ and low‐performing VAMCs.

Table 2.

Example Participant Quotes From High‐ and Low‐Performing Sites, Organized by Theme

	High‐Performing Site Quote	Low‐Performing Site Quote
Data collection	“Yeah, I mean obviously measure compliance with various standard medications depending on what the patients’ cardiovascular disease is. You know, aspirin compliance, use of beta blocker, use of a statin, and you know those types of measures and the use of platelet inhibitors, inpatients who have had interventional procedures done.” “I do know that they – the quality measured, for example, in the cardiac cauterization lab. They use the CART‐CL database to track outcomes and to track quality and to track metrics around patients and their care. And that's obviously based at a national level but I assume that we internally also track our patients and track our clinical outcomes and things such as complications. For example, all patients are contacted after – within three to five days of returning home to follow‐up, to see if they've had any complications, any issues, just to reach out to them.”	“We measure our interventions as far as code response, resuscitation response, and rapid responses.” “For patients who are positive for cardiovascular disease, there is a reminder about cardiovascular disease and asks if the patients are on aspirin, a beta blocker, an ACE inhibitor, have they had an echo, what's the ejection fraction, and I think there's something else on that reminder that I'm not remembering cause it's not in front of me.” “We do it [collect data] very closely at our readmission rate and medication reconciliation, and if the patient understood their discharge instructions, particularly if they are readmitted within 30 days.”
Feedback	“I guess the obvious [example] which is they provide individual data to us so we get some feedback on a regular basis as to how we're complying with the measures that are being looked at and reviewed so we can try to improve on what the issues are. You know, compliance with medications or follow‐up with specialists, et cetera, so.” “We have SAIL's meeting every month. We have heart failure meetings every week. We have with administration the cardiology and business meeting which is every month, but then we also have the ACSC meeting, which is every two weeks, so pretty much, I pretty much have at least one consult a week. Sorry, one meeting a week.” “If, for example, the SAILs data were to show that we are below standards for those quality measures, we would be notified immediately, and I say heart failure, because that is the one we are most actively working on.”	“Yeah, I mean there's the whole quality management division, and then they have, you know, meetings on the SAIL data and all of the quality data I think monthly. And so, different people present at those meetings. Usually for the cardiac stuff, historically it would be the chief of cardiology along with the chief of medicine presenting the data.” “We do have a weekly Cath Conference where we discuss patients as a group in terms of our recommendations for what surgery they may need or say our TAVR patients with our, usually two cardiothoracic surgeons, multiple cardiology faculty or fellows. We present the case and then we develop our recommendations based on that Cath Conference and then as part of this we have adverse outcomes. There are M&M rounds so we hear about the complications from either the surgeries that were done here that we referred the patient out for.”
Data driving decision making	“We went out and looked for best practices, called a bunch of people around the country, figured out what was working and what wasn't. And then one of the things that was really working well was a clinical dashboard that had been developed at the [name of city], and so we disseminated that nationally. So we got it in [name of city] about a year ago and then we, between, mostly cardiology has been using that to try to manage patients as a population…” “I know we – for example in the trenches our residents will – many of them get involved in quality improvements projects, and they use that to generate ideas about how we can improve consistency with offering patients various services.”	“Well, we have a working group for our heart failure patients because it's a priority to improve our SAIL measures and so we have a high homeless population burden and so we've been trying to figure out what are sort of big gaps in care and how we can better communicate or identify high risk patients and get them plugged in with appropriate services.” “It's more aggregated [data]. It's kind of more, ‘Here you go. And this is kind of where we are as a facility to this is what we need to do. And now we're putting in this reminder. Now we need to do this because these are what our scores were from…’”
Special consideration for high‐cost care	“Well, I think part of the beauty of the VA is that it's all sort of wrapped up. I mean, you know, we make decisions that I think are based on effectiveness and you know, risk benefit value. But I don't think that‐I mean I don't stop to think about cost. So and I think that's part of the beauty of treatment at the VA is that I am making those decisions based on what I think the patient needs.” “I refer a lot of patients outside to non‐VA care in particular for services that we don't provide, particularly cardiac rehabilitation for some of our Veterans. They don't live anywhere nearby so they live very far away. We end up often doing non‐VA care referrals for them, which is, in cardiac rehabilitation, is not particularly expensive treatment but that being said, we have a lot of eligible patients. Just the pure volume of patients that we refer, it does start to add up. And I don't know, I haven't gotten any push back, I'll say, from anyone, whether that's administration or clinically, saying, “Oh, it's just too much,” but I've brought it up or I've thought about myself at least internally.”	“I do think that there is consideration from the pharmacy about some of these more expensive medicines and as far as these studies, cardiology really likes to look at the patient and decide if that study is necessary. Where they take cost into consideration, I don't necessarily have seen a lot of costly cardiac studies, I mean, just your standard stress test and things like that.” “Cost effectiveness? We always look at, with our system, we kind of always have the sickest of the sick, and some of the oldest of the old, so really looking more at a cost effectiveness versus life expectancy versus outcomes. If it's really ‐ test is really gonna change what I do for this patient, improving their life, you know, their outcomes or help outcomes, their life expectancy. I think it's probably more what we weigh versus actually the cost. You know, I don't know any of my providers that haven't ordered something just ‘cause it's too expensive.”
Endorsement of the VA's quality improvement system	“I believe that they have that ability to—if there's something that the patient needs that they're not able to provide here or it's an emergent situation, they have access to those other facilities and can get the patient the care that they need. I think, in my experience, that our section chief here does really go above and beyond in terms of connecting with patients, making sure people are taken care of well. I think all of our providers in cardiology are very dedicated to what they do.” “Yeah I mean I think one of the biggest opportunities we have being a large VA facility is that so many of our‐ you know all of our services, the grand majority of our services are in house so the opportunity for things like care coordination, warm handoffs, you know, being able to pick up a phone and say that, “You know, I think this patient needs follow‐up care sooner.” Or anything like that I think is a real‐ great benefit to us relative to other, other VAs that need to outsource a lot of that.” “I think that some—having worked at cardiology in other facilities in this area, in the private facilities and working here at this particular point, with the care that's given and provided, and though I can't attest to cost and quality measures—only because I don't have access to that—just in terms of outcomes that I see, it meets and exceeds what goes on in our—in the facilities, other facilities, the private ones.”	“I don't see people sitting around content to just do the barest minimum on their job. They're hungry, they want to get better. They wanna be on the cutting edge. They wanna be doing these things. One of the frustrations I've heard expressed at many levels, including our chief of anesthesiology is that one of the things the VA has sacrificed in the last 10, 20 years is being that cutting edge place where we have the latest equipment. We weren't doing reckless experimentation, but we were right along with the best academic centers in terms of our writing, what modern medicine has to offer. And the mission of the VA has grown. The responsibilities of the VA system wide has mushroomed.” “I'm impressed with the whole CART‐CL database that basically, we are a system that already that sort of nationally polices itself, it has its own, basically M&M system built out where if there is a complication, it's reviewed nationally. I mean and the outcomes I think they reported them in some recent journal articles for cath labs at the VA are very good if not better than, you know other cath labs in the community. So I think at least cath lab which I know the most we are doing – I know we're involved in that and for most of the outcomes we look better than the local hospitals.”

ACE indicates angiotensin‐converting‐enzyme; ACSC, ambulatory care sensitive condition; CART‐CL, Cardiovascular Assessment, Reporting and Tracking System for Cath Labs; M&M, mortality and morbidity; SAIL, Strategic Analytics for Improvement and Learning; TAVR, transcatheter aortic valve replacement; VA, Department of Veterans Affairs.

Key Themes Extracted From Participants’ Responses

Responses were organized into 5 themes described below: (1) data collection, (2) feedback, (3) data driving decision making, (4) special consideration for high‐cost care, and (5) endorsement of the VA's quality improvement system.

Data Collection

The vast majority of participants mentioned that their facilities collected various types of data on cardiovascular care processes. Among the process‐of‐care data mentioned by respondents were data collected for the Healthcare Effectiveness Data and Information Set (HEDIS), the VA's Strategic Analytics for Improvement and Learning (SAIL) system, the VA's CART‐CL (Cardiovascular Assessment, Reporting and Tracking System for Cath Labs) registry, and the VA's External Peer Review Program (EPRP), as well as national registries sponsored by the American Heart Association/Get With the Guidelines, the American College of Cardiology/National Cardiovascular Data Registry, and the Heart Failure Society of America.

Respondents also reported routine, but less abundant, measurement of clinical outcomes data. These data included individual case reports that described the occurrence of and circumstances surrounding adverse events, along with aggregated data reported to national registries. The most commonly reported outcomes mentioned by respondents were 30‐day hospital readmission rates. Additional outcome measures included 30‐day postprocedure mortality statistics and the incidence of adverse clinical events following cardiovascular procedures (ie, CART‐CL).

At the local level, information regarding the outcomes of individual cases were reportedly discussed routinely at clinical conferences:

“[We have a] weekly cath[eterization] conference where…we present the case and then we develop our recommendations based on that cath conference and then, as part of this, we have adverse outcomes.”

Participants stated that among the cost and cost‐related measures routinely collected by their facilities were rates of costly diagnostic testing and out‐of‐system referral rates. Some mentioned that selected cost considerations were mandated nationally (ie, to determine the medical center's receipt of Veterans Equitable Resource Allocation dollars)17 and purchases of high‐cost equipment were scrutinized closely.

Feedback

Two‐thirds of participants reported that process data were regularly shared with providers. One participant stated, “We have SAILs meetings every month” to receive feedback based on SAIL data. Other measures, such as CART‐CL, HEDIS, and EPRP data, were routinely reviewed at many VAMCs. Roughly one‐third of participants were aware of who was responsible for reviewing, analyzing, and disseminating their facility's outcomes and cost data. Half of the participants stated that they did not receive any feedback on costs of care, and they were not aware whether their facility measured the cost of care in relation to processes and outcomes.

Several participants were directly involved in data collection and dissemination, whereas some mentioned various other VA staff members were responsible for collecting and sharing data. Specific roles mentioned as involved in the process of cardiovascular data collection and feedback were a designated cardiovascular clinic manager, the cardiology section chief, the chief of medicine, and the associate chief of staff for quality. Some participants stated that the responsibility for collecting and sharing data fell on various groups rather than any 1 individual, including quality management divisions, a medical informatics team, as well as:

“Nursing staff, respiratory staff, it's a multidisciplinary approach. There's a team that is taking care of the problem, including the main hospital, primary care setting comes in the first level, then the main hospital, then all the leadership at the VISN [VA Integrated Service Network] level.”

One participant acknowledged that while multiple parties may be involved in reviewing the data, the primary collectors and disseminators of data were the chiefs of clinical services:

“Most of it all goes through the chief of cardiology…at least initially, as it goes up the chain of course…all the way up to administration… If there were going to be any complications or readmissions we would know it, I'm going to say, next morning. Every morning our director has a morning report, and that's one of the items that's on the morning report. Our chief of surgery goes over those things. The chief of medicine goes over those items. Our chief of infectious disease goes over those items. So, every little blip is monitored one at a time.”

The frequency of data dissemination varied based on the type of data being shared: regular yearly meetings (ie, medical center data), quarterly (eg, quality improvement meetings), monthly (eg, staff or section meetings), or weekly (eg, “morbidity and mortality” meetings).

Data Driving Decision Making

Many participants indicated that the positive and negative feedback they received encouraged them to either continue current practices, or highlighted areas in need of change. Positive feedback encouraged them to continue providing high‐level care:

“It's the confidence we have that by sticking to evidence‐based medicine, we are providing the most value‐conscious and compassionate care.”

Negative feedback gave them the opportunity to evaluate current practices, in relation to other facilities, and determine barriers hindering their care processes and outcomes to create action plans for improvement. One participant noted:

“[We] come up with decision trees also to help clinicians, guide clinicians with patients in different sub‐specialties.”

Another participant stated:

“We're looking behind the scenes at, you know, best practices and, you know, what we can do to improve patient…the delivery of care.”

Some participants reported that their facilities created task forces and initiatives to implement change and monitor improvement. One facility utilized a “Lean Six Sigma” team to evaluate and improve outcomes:

“[It was an] initiative specifically looking, you know, at admissions for ambulatory sensitive conditions where there is a smaller, more focused group that is looking at those particular utilizer readmissions and actually focusing to see if they can come up as you guys are looking at best practices to keep those individuals out of the hospital.”

Special Consideration for High‐Cost Care

To enhance the value of care provided, some respondents noted that their facilities followed up with patients after hospital discharge to increase medication adherence, with the goal of preventing the high cost of hospital readmission. While most providers did not routinely consider the value of their care, some providers reported that they did consider cost in their decision making, particularly for higher‐cost treatments (eg, transcatheter aortic valve replacements, cardiac resynchronization therapy defibrillators).

Barriers to the delivery of high‐value care reported by participants included outsourced care and the use of high‐cost medications. Several participants reported that when care was outsourced to non‐VA facilities through the Veterans Choice Program (ie, VA's program allowing veterans in particular circumstances to obtain healthcare services outside the VA system), it often resulted in higher costs. Several reasons were cited for this outsourcing, such as: no ability to perform cardiac surgery at their facility, limited staffing hours on nights and weekends, and geographical constraints (particularly in rural areas).

Another challenge to providing high‐value care noted by participants was the use of high‐cost medications. They noted that some patients were sometimes prescribed high‐cost medications from both VA and non‐VA providers. Several respondents mentioned that VA restricts high‐cost medication usage through local Pharmacy and Therapeutics committees, which use the VA's Pharmacy Benefits Management guidelines and issue strict protocols to evaluate the need for high‐cost medications on a case‐by‐case basis. However, participants were also emphatic that costs were secondary to perceived clinical necessity. One participant noted that, although VA providers were cognizant of costs, “we think about what the patient needs first.” Another provider stated that when there was adequate clinical evidence supporting the use of a particular medication, it was usually provided by the VA, regardless of cost:

“[T]here are some extremely expensive drugs which are restricted – not available to us non‐specialists. There's one that's called Ranolazine. I'm not sure how much of a morbidity or mortality impact it has been shown to have, but it is a drug that treats refractory chest pain in people who have refractory angina, cardiac chest pain. So, that drug is a non‐formulated drug. It needs to be approved by a cardiologist so I, of course, anyone who I feel may benefit from that I can easily get them to a cardiologist who will approve it if it's appropriate. And I can't think of a situation where it hasn't been approved in a reasonable situation.”

Endorsement of the VA's Quality Improvement System

Many participants praised the VA quality improvement system as a whole and found certain national programs, such as national data repositories, particularly helpful. Additionally, most viewed their VA facility highly in comparison to other hospitals (both VA and non‐VA facilities). They identified many care processes in which their VA facility excelled (eg, advanced cardiac imaging, cardiac magnetic resonance imaging, cardiac computed tomography, and smoking cessation), as well as exemplar teamwork among physicians and staff contributing to positive outcomes. One participant praised their VA facility's protocol for disseminating objective feedback on the care provided, citing it as evidence of their facility's outstanding communication and teamwork. Many providers cited objective feedback (eg, “based on the data,” “gold star center,” and “won awards”) as proof of their facility's high quality and positive outcome rates. Some compared their VA facilities with non‐VA hospitals as evidence of their positive performance:

“[F]or most of the outcomes, we look better than the local [non‐VA] hospitals. So even though some of our CL data were not as good as other VA's, we're still better than our local hospital. So, I think that's a good thing for people to compare to.”

Some participants from smaller facilities noted that they often relied on other local hospitals to help fill certain gaps in care, while still praising their ability to provide quality care synergistically with outside providers:

“Our patients, I'd say at least 60‐70% of them are rural or semi‐rural or, you know, they pass seven or eight hospitals and medical centers before they get to the main VA clinic along the way. So, we do have a very good program where we work with the community cardiologists as well.”

Several participants opined that their VA hospital's patient population was “more challenging” than those of other VA hospitals, citing higher rates of cardiovascular risk factors (eg, smoking) in their hospital's population. However, none of these respondents indicated that these challenges were a defensible reason why their VA medical center could not achieve optimal health outcomes and provide high‐value care.

Discussion

This qualitative analysis of VA providers’ perspectives on the process, outcomes, and value of VA health care revealed that many providers are observing the collection of measurement data, are receiving feedback generated from those data, and are (either personally and/or collectively) using that feedback to change the way health care is delivered. However, the vast majority of this activity is focused on process measures of healthcare delivery. Much less emphasis is placed on using outcomes data to change practice, and data on healthcare value have an extremely limited role in influencing VA cardiovascular practitioners outside of administrative barriers to high‐cost utilization (Figure).

Figure 1.

Visual model of respondent attitudes toward processes, outcomes, and value improvement. The left portion of the figure indicates strong engagement and endorsement of the measure‐learn‐improve cycle in the domain of healthcare processes. The center portion indicates respondents’ reduced engagement with the same cycle in the domain of clinical outcomes. The left portion indicates respondents’ limited engagement with the measure‐learn‐improve cycle in the domain of value.

We found a marked level of endorsement and enthusiasm of the quality measurement process, with relatively few criticisms about the workload or opportunity costs inherent in devoting substantial effort to quality measurement. Whereas there have been recent reports in the literature regarding overmeasurement of quality and the potential “crowd‐out” effect of additional quality measures,18 we detected few sentiments among our respondents that were critical of the VA's quality measurement process.

Many respondents tended to conflate process and outcome measures of quality, despite the interview guide being designed to make the distinction clear. Frequently, when participants were asked whether their VA institutions measured outcomes, they reverted to a recitation of process measures or mentioned composite measures (eg, SAIL scores) that include outcomes as 1 of several components producing a single score. Although it has been argued that processes are the most appropriate domain of quality on which healthcare providers should focus, a lack of clarity in how those processes connect to observable patient‐centered outcomes raises the risk of a quality‐of‐care “disconnect” where VA's quality‐of‐care efforts do not improve patients’ well‐being, longevity, or both.

Most of our participants confined their comments on the value of health care to administrative processes governing the use of high‐cost drugs, devices, and services. Whereas many participants mentioned data on 30‐day readmissions when asked about patient outcomes, it was more common that readmissions were mentioned in the context of hospital performance measures rather than because of the cost‐saving aspect of reducing readmissions. That the VA routinely scrutinizes and often limits the use of selected high‐cost tests and treatments was reported by several respondents and seemed to be a familiar concept to many. However, there was little, if any, description of individual providers or provider groups routinely receiving feedback on the value of care provided. Given that healthcare providers are the dominant driver of healthcare expenditures,19 this finding illuminates a potential area for improvement in the VA's attempts to have data influence not only the quality of a provider's care, but also the efficiency of care.

It is likely that the more‐limited mentions of outcomes and value measures by respondents is, in part, a reflection of the lack of available measures in these domains, rather than a deliberate choice of VA employees to ignore outcomes and value, or a deliberate strategy of VA managers to emphasize process measures more heavily. This “global” phenomenon of measurement deficits would explain why responses from clinicians at VAMCs that had been previously identified as low performers in terms of value did not differ materially from responses from clinicians at high‐performing VAMCs. Indeed, data collection and feedback on cardiovascular value were extremely rare in all the hospitals we surveyed; thus, it was unlikely that providers’ perceptions of the importance of healthcare value were influenced by knowledge of their own institution's cardiovascular value performance.

Comparisons With Previous Studies

Our findings revealed widespread adoption of the quality improvement “measure‐learn‐improve” triad by VA providers, in stark contrast to a 2005 report by Audet et al of a national survey of US physicians that revealed very limited engagement in quality improvement activities.20 More recently, Curry et al found that hospitals with low death rates from myocardial infarction had a consistent culture of “problem solving and organizational learning”21 similar to the findings from our study. However, those researchers’ findings of a substantial difference in culture between hospitals that were high‐ and low‐quality performers differed from our study, where no material difference was observed. This was possibly a result of VA's high level of system integration that inherently reduces organizational variation across VA hospitals.

Limitations

There were several limitations to this investigation. Telephone interviews do not capture facial expressions, and therefore it is possible our interviewer may have missed nuances in participant responses that would have been evident in face‐to‐face interviews and that may have resulted in a different line of questioning. Our respondents were volunteer participants who, while varied in their geographical location, roles, and experience within the VA system, may not have been representative of the broader population of VA cardiovascular providers. In addition, given that our investigator team was sponsored by the VA, and this was known by our respondents, it is possible (despite assurances of confidentiality) that responses were biased to be less critical of current VA processes, leadership, etc, than what might have been obtained by an independent study team. Given that the focus of our interviews were VA healthcare providers and not administrators (eg, hospital directors), it is possible that the concept of value was underemphasized by our respondents because the VA may institutionally designate value as primarily in the domain of its administrators rather than its clinicians. If so, it is possible that the limited mention of healthcare value by clinicians is a purposeful result of the VA's organizational structure. Approximately one‐third of our respondents reported leadership positions in the VA, and this may have biased our findings toward a more‐favorable report of the VA's activities, although we observed no material difference in responses between leaders and nonleaders. Additionally, our relatively small sample size (n=31), particularly from low‐performing medical centers (n=9), limits our ability to interpret distinctions between participants from high‐ and low‐performing centers. Though there are many similarities between the VA healthcare system and external facilities, our focus on VAMCs limits the generalizability of the findings to facilities outside of the VA system. Finally, we had no respondents from VA medical centers located in the northeast US Census region, in part because only n=2 hospitals from our initial subset of n=20 hospitals were located in the Northeast, and no invitees from these 2 hospitals agreed to participate. Hence, our results may not generalize to VAMCs in the Northeast.

Summary

Cardiovascular providers in the VA identified and endorsed an institutional quality‐of‐care culture of data collection, feedback, and behavior change, predominantly involving process‐of‐care measures. This “measure‐learn‐improve” cycle was less prominent for healthcare outcomes and it is not operative in the domain of healthcare value.

Sources of Funding

This research was sponsored by a grant (IIR 14‐077) from the Veterans Affairs Health Services Research and Development Service. Dr Groeneveld had full access to all of the data in the study and takes responsibility for the integrity of the data analyses. The opinions expressed in this article are the authors’ own and do not represent the official views of the US Department of Veterans Affairs.

Disclosures

None.

(J Am Heart Assoc. 2019;8:e011672 DOI: 10.1161/JAHA.118.011672.)