Perceptions of Public and Nonpublic Reporting of Interventional Cardiology Outcomes and Its Impact on Practice: Insights From the Veterans Affairs Clinical Assessment, Reporting, and Tracking Program

Abstract

Background

Physicians have expressed significant mistrust with public reporting of interventional cardiology outcomes. Similar data are not available on alternative reporting structures, including nonpublic quality improvement programs with internally distributed measures of interventional quality. We thus sought to evaluate the perceptions of public and nonpublic reporting of interventional cardiology outcomes and its impact on clinical practice.

Methods and Results

A standardized survey was distributed to 218 interventional cardiologists in the Veterans Affairs Healthcare System, with responses received from 62 (28%). The majority of respondents (90%) expressed some or a great deal of trust in the analytic methods used to generate reports in a nonpublic quality improvement system within Veterans Affairs, while a minority (35%) expressed similar trust in the analytic methods in a public reporting system that operates outside Veterans Affairs (P<0.001). Similarly, a minority of respondents (44%) felt that in‐hospital and 30‐day mortality accurately reflected interventional quality in a nonpublic quality improvement system, though a smaller proportion of survey participants (15%) felt that the same outcome reflected procedural quality in public reporting systems (P<0.001). Despite these sentiments, the majority of operators did not feel pressured to avoid (82% and 75%; P=0.383) or perform (72% and 63%; P=0.096) high‐risk procedures within or outside Veterans Affairs.

Conclusions

Interventional cardiologists express greater trust in analytic methods and clinical outcomes reported in a nonpublic quality improvement program than external public reporting environments. The majority of physicians did not feel pressured to avoid or perform high‐risk procedures, which may improve access to interventional care among high‐risk patients.

Collection and reporting of interventional cardiology outcomes have been widely adopted. Several states (Massachusetts, New York, and Washington) provide this information to the public, with hopes of improving transparency and clinical outcomes. Public reporting has also become a strategic objective of many professional societies, including the American College of Cardiology, which publicly lists cardiovascular performance metrics for many hospitals throughout the United States.1 Initial investigations suggested that public reporting of clinical outcomes after percutaneous coronary intervention was associated with a reduction in periprocedural mortality.2 However, public reporting of interventional outcomes has also been associated with increasing risk aversion and worse clinical outcomes among patients with myocardial infarctions as a whole, when accounting for patients declined for angiography and intervention.3, 4 Interventional cardiologists have thus remained skeptical of the benefits of public reporting and have expressed significant mistrust with the system currently in place.5, 6

Public reporting of interventional outcomes does not occur in the largest integrated healthcare system in the United States, the Veterans Affairs (VA) Healthcare System. Rather, a national quality and safety oversight organization (Clinical Assessment, Reporting, and Tracking Program) internally distributes process metrics and clinical outcomes benchmarked against national medians with hopes of improving cardiovascular care through intramural processes. Quantitative reports are supplemented with qualitative peer reviews for all periprocedural major adverse events that occur at a VA hospital, as well as the same review for ad hoc cases as requested by facilities. Physician attitudes toward this nonpublic quality improvement (QI) system remain unknown, and any potential impact on risk aversion within a closed healthcare system has not been evaluated. Many physicians working within this system also operate in affiliated academic medical centers that may be subject to public reporting from state agencies or through professional society organizations like the American College of Cardiology, offering an opportunity for a direct comparison of the attitudes and impacts of public and nonpublic reporting systems.

Accordingly, the present analysis sought to evaluate the perception of a nonpublic system of interventional cardiology QI among clinicians and its impact on clinical practice within an integrated healthcare system. Further investigations compared these attitudes and practice patterns among operators who also performed procedures in other reporting environments.

Methods

The data that support the findings of this study are available from the corresponding author on reasonable request, though it will be subject to the stringent data privacy rules of the VA Healthcare System and the United States government.

Survey

A 99‐question survey was derived from previous publications, with additions and modifications to make it more applicable to the integrated VA Healthcare System.6 The survey questions and definitions are available in Data S1. This survey was distributed to the study population for completion via electronic mailing using the Research Electronic Data Capture system.7 The anonymous survey remained open for a 4‐week period, with weekly reminders before the survey was closed (March 15, 2019). This analysis was approved by the Colorado Multiple Institution Review Board that includes the Rocky Mountain Regional VA Medical Center, with a waiver of informed consent.

Population

The VA Clinical Assessment, Reporting and Tracking Program is a national quality and safety program for invasive cardiac procedures performed by cardiologists throughout the VA Healthcare System. The contact information for all practicing interventional cardiologists within this integrated healthcare system is maintained by the Clinical Assessment, Reporting, and Tracking Program, as part of its mission to monitor and enhance the quality and safety of invasive cardiac procedures. These contacts were then cross‐referenced with a global directory of active employees within the VA, to ensure that only currently practicing attending interventionalists were included in the study population. Of note, these active physicians could practice either solely in the VA or also at other nonfederal institutions.

Analysis

Survey participant and site characteristics were presented as median (interquartile range) for continuous variables and number (percentage) for categorical variables. Comparisons of paired ordinal responses were performed with the Stuart‐Maxwell test, an extension of McNemar's 2×2 chi‐squared test of paired data to k ordinal response categories. All statistical analyses were performed using R: A Language and Environment for Statistical Computing (version 3.5.2). A P<0.05 was considered statistically significant.

Results

Population

The survey was distributed to 218 active interventional cardiologists within the VA Healthcare System, and 62 (28%) provided a response. As shown in Table, the majority of survey respondents were male (74%) with a single female respondent (2%) and several others (24%) who did not respond to this question, all of whom reported a median duration of practice of 11.5 (interquartile interval, 6–23) years. A significant percentage of interventionalists surveyed (45%) also practice at non‐VA facilities, for a combined median of 4 (interquartile interval, 2.5–4.5) total procedural days a week resulting in a median of 145 (interquartile interval, 115–170) total interventions in the past year. A majority of interventional cardiologists who responded to the survey worked in VA facilities with cardiothoracic surgery on site (68%) and the availability of all‐hours interventional services (61%) with mechanical circulatory alternatives to intra‐aortic balloon pumps (74%). The majority of respondents also reported having weekly multidisciplinary meetings with cardiologists and cardiothoracic surgeons (71%). Training programs for both general cardiology (90%) and interventional cardiology (68%) were common at respondent facilities.

Table 1.

Characteristics of Responding Interventional Cardiologists

	Participants (n=62)
Sex
Male	46 (74)
Female	1 (2)
Did not answer	15 (24)
Interventional cardiology fellowship
Completed	40 (65)
Not completed	7 (11)
Practice years	11.5 (6–23)
Practice at non‐VA facility	28 (45)
Total procedural days (per wk)	4.0 (2.5–4.5)
VA procedural days (per wk)	2.5 (2.0–3.5)
Non‐VA procedural days (per wk)	1.0 (0.5–1.0)
Total procedural volume (PCI/y)	105 (75–150)
VA procedural volume (PCI/y)	80 (57–100)
Non‐VA procedural volume (PCI/y)	50 (30–75)

All entries are number (percentage) or median (Q1–Q3). Non‐VA procedural days and volumes are restricted only to respondents who practiced at a non‐VA facility. PCI indicates percutaneous coronary intervention; VA, Veterans Affairs.

Perceptions of Nonpublic QI and Public Reporting

Figure 1 summarizes the perceptions of the analytic methods and clinical outcomes described in varying reporting environments. As shown, a majority of survey participants (90%) expressed some or a great deal of trust in the analytic methods used to generate reports in the VA QI system. A smaller proportion of survey participants (35%) had some or a great deal of trust in the same analytic methods used in public reporting systems outside the VA (P<0.001). A minority of respondents (44%) felt that in‐hospital and 30‐day mortality accurately reflected interventional quality in a nonpublic QI system, though an even smaller proportion of survey participants (15%) felt that the same outcome reflected procedural quality in public reporting systems (P<0.001). Based on these perceptions, the majority of respondents (87%) did not believe that the reports produced by the VA QI system should be disseminated to the public.

Figure 1.

Trust in analytic methods and clinical outcomes. Among operators that responded to both questions, a majority of respondents (90%) expressed some or a great deal of trust in the risk‐adjustment methodologies and reports produced in a nonpublic system, with a smaller proportion (35%) suggesting the same of reports released in a public environment (P<0.001, A). Similarly, a plurality (44%) of respondents agreed that in‐hospital and 30‐day mortality reflected interventional quality in a nonpublic environment, while a significantly smaller proportion (15%) agreed that it represented interventional quality in a public reporting environment (P<0.001, B). CART indicates Clinical Assessment, Reporting, and Tracking.

Impact of Nonpublic QI and Public Reporting

Further questions sought to clarify how perceptions of varying reporting systems might impact clinical practice among those who practiced in VA and non‐VA environments. As shown in Figure 2, similar proportions of respondents worried that a potential complication would sometimes or often impact their VA or non‐VA site (51% versus 48%; P=0.250) or personal reputation (46% versus 54%; P=0.262). The overwhelming majority of respondents indicated that they had never or rarely been pressured to avoid a high‐risk intervention at the VA (82%) or at an affiliate non‐VA site (75%; P=0.383). Similarly, a majority of operators indicated that they had never or rarely been pressured to perform a high‐risk intervention within (72%) or outside the VA (63%; P=0.096).

Figure 2.

Clinical impact of reporting environments. Among operators who answered both questions, a similar proportion of individuals worried that a potential complication would sometimes or often impact their facility (51% vs. 48%; P=0.250) or personal reputation (46% vs. 54%; P=0.262) at a VA or non‐VA site (A and B). The majority of respondents indicated that they had never or rarely been pressured to avoid (82% vs. 75%; P=0.383) or perform (72% vs. 63%; P=0.096) a high‐risk intervention at their VA or non‐VA site (C and D). PCI indicates percutaneous coronary intervention; VA, Veterans Affairs.

Discussion

The present study evaluated the attitudes toward procedural quality reporting and its impact on clinical practice among interventional cardiologists in an integrated healthcare system. As the data demonstrate, a majority of survey participants conveyed a great deal of trust in the analytic methods and clinical outcomes reported in the VA QI system. In contrast, a minority of operators expressed the same level of trust in the analytic methods and clinical outcomes in non‐VA practice. Despite these perceptions, similar proportions of respondents had concerns about a complication affecting their facility or personal reputation inside and outside the VA. Similarly, the majority of operators rarely felt pressured to avoid or perform high‐risk interventions regardless of their practice site. These data provide important insights into the attitudes regarding various QI environments and their impact on clinical practice.

Interventional cardiologists have expressed significant mistrust in the reporting systems currently in place. Previous surveys have demonstrated that a small minority of physicians (9%) believe that publicly reported outcomes accurately reflect interventional quality.5 Similarly, the majority of respondents to this survey did not believe that these outcomes were useful in selecting a treatment facility or in improving the overall quality of care. The negative opinion of public reporting may arise from mistrust in the analytic methods used and the clinical outcomes reported. Prior data have suggested that a large proportion of interventional cardiologists do not trust the risk‐adjustment models used in several public reporting systems, likely because of the possibility of significant residual confounding.8 The presentation of short‐term mortality as a surrogate for interventional quality has also been questioned, with increasing data suggesting that death is more likely to reflect the underlying acuity of the patients treated rather than a complication of the index procedure.9 The present analysis provides additional data that confirm the mistrust in public reporting, even among physicians who may primarily practice outside that environment. More importantly, however, it demonstrates enhanced trust in reports generated in a nonpublic QI system. The vast majority of interventional cardiologists surveyed (90%) had some degree of trust in the adjustment methods used to generate internal reports in this environment, perhaps related to the increased transparency of the process whereby a variety of metrics are presented to the site rather than just risk‐adjusted mortality. The VA nonpublic reporting environment produces less external scrutiny, such that the clinical outcomes conveyed may be accepted in the context of QI rather than in a punitive context. Additionally, the reports generated in the nonpublic environment may be more explicit about the risk‐adjustment methods used to produce the findings. Regardless, future iterations of procedural quality reporting may benefit from additional input from proceduralists, given their strong impressions of the current state.

The divergent levels of trust in the various reporting systems may be associated with differences in clinical practice. Previous research has demonstrated that 79% of physicians believe that public reporting of interventional outcomes may dissuade operators from performing an appropriate and indicated percutaneous coronary intervention.5, 10 In fact, 66% of interventional cardiologists surveyed in a public reporting environment admitted to avoiding an indicated procedure because the outcome was publicly reported.6 This risk aversion can lead to a significant decrease in access to invasive cardiovascular procedures and worsen clinical outcomes for patients with myocardial infarction as a whole.3 The present analysis does not mimic these findings, however, as a similar proportion of interventionalists raise concerns about a potential complication impacting their facility or personal reputation both within and outside the VA. Further, the majority of respondents never or rarely felt pressured to avoid or perform a high‐risk procedure because of potential complications at either site. These data are reassuring and suggest that the mistrust identified in the nonpublic QI system and public reporting systems have not led to the same levels of risk aversion by practitioners within the VA. Perhaps operators primarily practicing within this integrated healthcare system have fewer concerns about the public report of their performance at affiliated institutions. Regardless, these data still provide important insights into the ideal mechanism to measure and report interventional quality.

A variety of interventions can be employed to improve interventional quality, including the collection and reporting of clinical data. Our findings should promote development and testing of QI programs that do not rely on public reporting and are more accepted by practicing clinicians. Peer review may be more effective in generating meaningful feedback of QI, especially in the evaluation of procedural complications.11 QI systems could also move beyond mortality to consider alternative metrics that better reflect interventional quality.12 Process measures that apply to the entire population of patients with coronary artery disease, including discharge medications, would be a potential alternative that highlights the care of an entire population with a given diagnosis and assesses the impact of risk aversion in procedural case selection.13, 14 Procedural process metrics could also be considered, focusing on radial access or intravascular imaging, as both of these interventions are associated with improved outcomes.15 Reporting this relevant data in a nonpublic manner could improve trust in the reporting of interventional outcomes, possibly resulting in a reduction in risk aversion and improved interventional quality.

Limitations

The present project should be interpreted in the context of several limitations. The overall number of participants in this survey was low, though the response rate of 28% is similar to prior analyses.6 We attempted to enrich the response rate by weekly reminders and ensuring that the target population consisted of active practitioners. Second, the low response rate raises the possibility of bias in response, wherein respondents would be more likely to express extremes of positions. Third, the majority of respondents practice in academic environments such that the findings may not be representative of community practices without teaching programs. Fourth, operators practicing in the VA Healthcare System face different legal sequelae from complications than those in the community, which may impact their attitudes toward risk. Fifth, the anonymous responses make it impossible to fully characterize the QI environment for practitioners outside the VA, including whether they participate in public reporting from governmental agencies or professional societies such as the American College of Cardiology. Finally, the current analysis did not address patient attitudes regarding public and nonpublic reporting of interventional outcomes. Prior publications have demonstrated positive patient attitudes toward public reporting, which is contrary to operators’ opinion.5 Further studies could address the opinions of patients in a nonpublic reporting system.

Conclusions

In conclusion, interventional cardiologists express greater trust in the analytic methods and clinical outcomes reported in nonpublic QI environments. These findings suggest an opportunity to focus percutaneous coronary intervention quality interventions on internal reporting of trusted measures that promote practice change.

Disclosures

Dr Waldo receives unrelated investigator‐initiated research support to the Denver Research Institute from Abiomed, Cardiovascular Systems Incorporated, and Merck Pharmaceuticals. He serves as the national director and Drs Doll and Valle as the assistant directors for the Clinical Assessment, Reporting, and Tracking Program, the internal reporting agency for invasive cardiac procedures within the VA Healthcare System. The remaining authors have no disclosures to report.

Supporting information

Data S1. Survey interventional reporting.

(J Am Heart Assoc. 2019;8:e014212 DOI: 10.1161/JAHA.119.014212.)