Skip to main content
NCBI home page
CJC Open logoLink to CJC Open
. 2021 Oct 23;4(3):271–281. doi: 10.1016/j.cjco.2021.10.006

Development and Evaluation of an Audit and Feedback Process for Prevention of Acute Kidney Injury During Coronary Angiography and Intervention

Bryan Ma a, Peter Faris b, Bryan J Har c,d, Ben Tyrrell e, Eleanor Benterud a, John A Spertus f,g, Neesh Pannu e, Braden J Manns a,b,d,h, Michelle M Graham e,i, Matthew T James a,b,d,h,∗
PMCID: PMC8978052  PMID: 35386131

Abstract

Background

Contrast-associated acute kidney injury (CA-AKI) is a potentially preventable complication of coronary angiography and intervention. Relatively little research has been done to determine how knowledge on CA-AKI prevention can be translated into clinical practice.

Methods

We developed, implemented, and surveyed end-users about the usability, acceptability, and utility of an audit and feedback process for CA-AKI prevention in Alberta, Canada. The audit and feedback reported on amount of radiocontrast dye used, hemodynamic optimization of intravenous fluids, and CA-AKI incidence for each cardiologist practicing coronary angiography or percutaneous coronary intervention, compared with peers at their site and across the province. Reports were developed through an iterative process involving interventional cardiologists throughout the design process and usability testing.

Results

Cardiologists participating in usability testing indicated a preference for the visual displays of data and summarizing indicators on the front page, and endorsed the value of peer-to-peer comparisons of performance measures. Of 31 eligible cardiologists from across Alberta, 17 responded to a survey evaluating the audit and feedback process. Fifteen respondents (88.2%) agreed that the data presented in the audit and feedback report were understandable; 17 respondents (100%) agreed or strongly agreed that the presentation of the report helped them better understand their performance compared with that of their peers; and 14 (82.4%) agreed that the audit and feedback process helped them identify ways to reduce the risk of AKI for their patients.

Conclusions

Conducting an audit and providing feedback was an understandable and acceptable intervention to help cardiologists identify ways to improve prevention of CA-AKI during coronary angiography or intervention.

Contrast-associated acute kidney injury (CA-AKI) is a preventable complication of coronary angiography and percutaneous coronary intervention (PCI),1, 2, 3, 4 that has increased in incidence in recent years, is associated with increased hospital length of stay, readmission, and adverse cardiovascular and kidney outcomes, and increases costs of care by US $7000-$9000 per patient.5, 6, 7, 8 Although evidence indicates that minimizing the volume of iodinated radiocontrast media9, 10, 11, 12, 13 and optimizing delivery of intravenous (IV) hydration14, 15, 16 can prevent CA-AKI in patients at risk,17, 18, 19 these strategies are used inconsistently in current clinical care.1,20,21 Relatively little research has been done to determine how to better integrate these CA-AKI prevention strategies into clinical practice.

An audit and feedback process in the healthcare arena involves measurement of clinical performance, with results fed back to the individuals who provide clinical care. An audit and feedback process can identify and highlight differences between actual and desired performance, inspire participants’ self-reflection on the appropriateness of their practices, and motivate behavioural changes to improve clinical practice and patient outcomes.22,23 Barriers to understanding how to best use an audit and feedback process include gaps in reporting the design of audit and feedback interventions and limited research dedicated to improving audit and feedback reporting effectiveness.24,25 Nonetheless, several generalizable principles have been associated with an increased likelihood of an audit and feedback process successfully changing physician behaviour,24,26,27which is influenced by the content presented in these reports,22,28, 29, 30, 31, 32 the manner in which that content is presented,33, 34, 35, 36 and interactions between the participant and the feedback provider.37, 38, 39, 40

Here, we describe the development and end-user evaluation of the usability, acceptability, and utility of an audit and feedback process for CA-AKI prevention in coronary angiography and intervention, informed by theory on best practices for audit and feedback effectiveness.

Methods

Setting

We implemented an audit and feedback process for CA-AKI prevention for all cardiologists performing coronary angiography or PCI in Alberta, Canada between November 2017 and November 2019, as part of a pragmatic clinical trial that introduced a multifaceted intervention comprised of provider education, computerized clinical decision support, and an audit and feedback process for CA-AKI prevention. The design of the clinical trial within which the audit and feedback process was implemented has been described in detail elsewhere; the trial included a study population of patients who were not on dialysis and were undergoing coronary angiography or PCI (excluding emergency primary PCI for ST-elevation myocardial infarction), with a predicted risk of AKI > 5%, compared∖an intervention that included point-of-care computerized decision support and auditing and feedback regarding individualized targets for safe radiocontrast media volume use and hemodynamically guided IV fluid administration based on left-ventricular end diastolic pressure41 vs usual care, and evaluated the primary outcome of AKI. We conceived audit and feedback reporting to be part of the intervention, to give participating physicians data on their actual performance, compared with the desired performance on these process-of-care measures, for which guidance was provided by the computerized decision-support tool.22

Selection of measures for audit and feedback reports

We consulted with 5 members of the trial steering committee who had clinical expertise in cardiology and nephrology, to identify the outcome measures for the audit and feedback process. We selected 2 process-of care-outcomes that were under the control of physicians performing coronary angiography or intervention and were modifiable. The first was the volume of iodinated radiocontrast media used for each patient’s procedure, accompanied by comparison with targets for the desired contrast volume to achieve a low risk of CA-AKI (Table 1), as previously described.41, 42, 43 The second was the amount of IV crystalloid fluid provided to each patient in the peri-procedural period, accompanied by comparison with the desired hemodynamically guided IV fluid volume for prevention of CA-AKI based on left-ventricular end-diastolic pressure measurement in patients without active (within the last 2 weeks) heart failure or severe aortic valvular heart disease.44 We selected one clinical outcome, the incidence of CA-AKI, based on evidence that lowering radiocontrast media volume and optimizing IV fluid administration can reduce the incidence of CA-AKI.10,45

Table 1.

Individualized strategies to support safe radiocontrast media use and hemodynamically guided intravenous (IV) fluid administration

Safe radiocontrast media use

  • Individualized safe contrast targets:

  • •

    Calculated using a multivariable model developed from the National Cardiovascular Data Registry AKI risk model to estimate a limit for the volume of contrast material needed to reduce the relative risk of CA-AKI by 15% for each patient

  • •

    Provided by ePRISM tool (Terumo Health Outcomes, Somerset, NJ), implemented in APPROACH software

  • •

    Applied when a patient’s absolute risk of CA-AKI > 5%

  • Strategies to reduce contrast volume:

  • •

    Avoid left ventriculograms

  • •

    Use small syringe catheter

  • •

    Avoid puff injections

  • •

    Use contrast injector

  • •

    Consider staging procedures

Hemodynamically guided IV fluid administration

  • Strategies to personalize IV fluids:

  • •

    Start normal saline @ 3 ml/kg for 1 h prior to procedure

  • •

    Adjust IV rate according to LVEDP-based recommendation during procedure

  • •

    Order LVEDP-based IV @ recommended rate for 4 h post-procedure
LVEDP (mm Hg)–guided IV fluid (ml/kg/h) protocol:
  • •

    5 for LVEDP < 13

  • •

    3 for LVEDP 13–18

  • •

    1.5 for LVEDP > 18

For patients weighing > 100 kg, the bolus and infusion rate are limited to those calculated for patients weighing 100 kg.
Open in a new tab

APPROACH, Alberta Provincial Project for Outcome Assessment in Coronary Heart Disease; CA-AKI, contrast-associated acute kidney injury; LVEDP, left-ventricular end-diastolic pressure.

Development of CA-AKI audit and feedback reports

We developed the audit and feedback report through an iterative, user-centred design and development process that included the following participants: a physician leader from each of the 3 hospital sites involved in the trial (site champions representing the end-users of the report); a data scientist/biostatistician (providing expertise in developing audit and feedback reports and familiarity with the required provincial data sources); a research project coordinator (to collate and summarize feedback); and the trial principal investigators (to ensure the audit and feedback process was integrated with other components of the trial). We identified the sources of data for the process and outcome measures from the Alberta Provincial Project for Outcome Assessment in Coronary Heart Disease (APPROACH) registry, the provincial Alberta Health Services laboratory test database, and hospital electronic medical record systems. We used data from these sources preceding the start of the trial to develop drafts of the audit and feedback reports, including varying formats for the display of data and inclusion of explanatory text. We developed successive drafts of the audit and feedback report, reviewed them with members of this group, modified them based on discussion and feedback, and reviewed them again within this group until consensus was achieved on an initial design for the reports. Final reports were automatically generated using RStudio Desktop (RStudio, Boston MA). We incorporated evidence-based principles for optimizing how audit information and feedback were presented within the reports, including the following: (i) presentation of individual data and appropriate comparators; (ii) selection of an appropriate volume of information; (iii) identification of appropriate goals; (iv) use of visual displays and summary messages; (v) provision of appropriate action plans; and (vi) determination of the frequency and timing of reports, which are described in further detail in Supplemental Appendix S1.

Usability testing of CA-AKI audit and feedback reports

Formative usability testing of the audit and feedback reports was undertaken based on a human-factors approach designed to evaluate comprehension of information presented in the report and assess performance metrics for successful completion of comprehension tasks.46 Interventional cardiologists who were not involved in the development of the audit and feedback report were recruited from the participating cardiac catheterization units and provided with a common sample audit and feedback report. Participants completed the usability remotely, via a video-conferencing call with a moderator who asked a series of questions to assess comprehension of the report. The moderator asked each participant to complete 13 tasks and determined their success in correctly interpreting information presented across the entirety of the audit and feedback report. Testing was followed by a semistructured interview regarding their experience using the report (Supplemental Appendix S2). Quantitative analysis of usability was performed by calculating the proportion of users who were able to successfully complete each comprehension task. Qualitative analysis of participant responses to tasks, in-test behaviours observed by test moderators, and responses to interview questions was used to identify themes related to the usability of various aspects of the report.

Delivery of the audit and feedback process

We designed a process for delivery of the audit information and feedback to practicing physicians, both in writing and verbally.23,26 Audit and feedback reports were provided directly via e-mail, in portable document format, and in print to cardiologists and were reviewed in person with each participant by the site-lead interventional cardiologist for each cardiac catheterization unit. Site-leads were provided with guidance on how to provide audit information and feedback, informed by knowledge on best practices for providing audit information and feedback and tailored to address areas in which usability testing had identified lower levels of comprehension of the reports. Individual or group meetings were held, depending on recipient availability, to ensure that recipients had dedicated time to read the reports, discuss results, ask questions, and engage in a dialogue about the results and plans to facilitate improvement. The audit and feedback program was accredited as a Section 3 (Performance Assessment) Continuing Medical Education Activity of the Royal College of Physicians and Surgeons of Canada. We encouraged the following principles for delivering audit information and feedback in these sessions: (i) constructive feedback through social interaction; (ii) credibility of information and prevention of defensive reactions to feedback; and (iii) recommendations and actionable messages consistent with goals, which are further explained in the Supplemental Appendix S1.

Cardiologists’ evaluation of acceptability and utility of CA-AKI audit and feedback process

A summative evaluation of the audit and feedback process for CA-AKI prevention was conducted by a survey distributed to all recipients following the receipt of their first audit and feedback report summarizing their own data and performance. This survey assessed end-user ratings of audit and feedback process content, presentation, delivery, and motivation to change their behaviour, using a 5-point Likert scale. To reinforce motivation for behaviour change, short-answer questions were included that encouraged participants to reflect on their practices and describe ways that the audit and feedback process helped them identify how they could change their practice to improve CA-AKI prevention.

Informed consent was obtained from all cardiologists participating in the study. Ethics approval was obtained from the health research ethics boards of the University of Alberta and the University of Calgary.

Results

Audit and feedback report

The final report consisted of a summary page followed by the body of the report for each cardiologist (Supplemental Appendix S3). The summary page included bar graphs of a cardiologist’s data from each quarter for each of the process and outcome measures, color-coded performance indicators (better, worse, or within control limits) for each performance measure based on the most recent quarter’s data, and a recommendations section (Fig. 1). The body of the report included 9 pages, containing the following: detailed background information (page 1); summary of results table (page 2); patient characteristics (page 3); and written and graphical presentation of data summarizing the process and outcomes measures (pages 4-9). For all measures, data were presented for the receiving cardiologist, in comparison to those of other cardiologists from their site and across the province. Examples of figures used within the report to display radiocontrast dye use and hemodynamic optimization of IV fluids are shown in Figure 2.

Figure 1.

Figure 1

Open in a new tab

Summary page of the audit and feedback report for contrast-associated acute kidney injury prevention in coronary angiography and intervention. AKI, acute kidney injury; APPROACH, Alberta Provincial Project for Outcome Assessment in Coronary Heart Disease; IV, intravenous; LV, left ventricular; Q, quarter; UAH, University of Alberta Hospital.

Figure 2.

Figure 2

Open in a new tab

Example of figures for reporting (A) physician radiocontrast volume use and (B) hemodynamically optimized intravenous (IV) use for contrast-associated acute kidney injury prevention in coronary angiography and intervention. (A) Funnel plot shows percentage of procedures in which desired radiocontrast volume was exceeded for each physician in the province. (B) Bar graph shows percentage of procedures in which left-ventricular end-diastolic pressure (LVEDP) was measured and used to hemodynamically optimize IV fluid (IVF) according to LVEDP for the physician receiving the report (My Patients) and other physicians from the same site (My Site). APPROACH, Alberta Provincial Project for Outcome Assessment in Coronary Heart Disease; CI-AKI, contrast-induced acute kidney injury; PCI, percutaneous coronary intervention.

Usability of CA-AKI audit and feedback report

Six cardiologists completed usability and comprehension testing; 5 were male, and all were between the ages of 45 and 54 years (Table 2). Participants had from 8 to 18 years of experience performing cardiac catheterization. Usability testing of the summary page showed that 3 participants (50%) correctly interpreted performance for all 3 performance measurements, based on both the graphs and performance-indicator symbols, and 3 (50%) were partially successful, correctly interpreting performance measurements on the graphs but not checking the performance indicators (Table 3). Usability testing of the body of the report revealed that all 6 participants (100%) were successful on 6 tasks related to interpreting the numbers of patients treated (1 task), risk factor characteristics of patients (1 task), performance related to safe radiocontrast dye use (2 tasks), and optimal IV fluid use (2 tasks). One participant (17%) failure was observed for 4 tasks related to interpreting safe radiocontrast dye use (1 task), optimal IV fluid use (1 task), and AKI incidence (2 tasks). Two participant (33%) failures were observed for 2 tasks related to interpreting baseline characteristics of patients and identifying the recommendations provided in the report.

Table 2.

Characteristics of cardiologists participating in usability testing of audit and feedback reports for contrast-associated acute kidney injury prevention in coronary angiography and intervention

Characteristic Result
Age, y
 < 45 1 (16.7)
 45–54 5 (83.3)
 ≥ 55 0
Sex
 Female 1 (16.7)
 Male 5 (83.3)
Years in clinical practice 17.3 (±6.3)
Years performing coronary angiography or PCI 11.8 (±3.8)
Open in a new tab

Values are n (%) or mean (± standard deviation).

PCI, percutaneous coronary intervention.

Table 3.

Audit and feedback report usability tasks, operational definitions, and results for contrast-associated acute kidney injury prevention in coronary angiography and intervention

Task number Task description Operational definitions
 Result
Success Partial success Failure
1 Describe the performance of this physician for the three indicators found on the summary page. The participant correctly identifies if the report shows better or worse performance than top performers for % exceeding safe contrast, for % below optimal IV fluid volume, and AKI incidence. The participant examines the graphs until they come to the correct conclusion for all three measures. However, they do not check the performance indicators on the right. Participant does not use performance indicators or information on graphs to arrive at correct answers. Success: 3 (50%)
Partial success: 3 (50%)
Failure: 0 (0%)
2 Does the report show a lower or higher incidence of patients where the physician exceeded safe contrast levels in Q3 compared to other physicians at their site and across the province? Correctly identifies a lower incidence of patients exceeding safe contrast levels in Q3 compared to other physicians at their site and across the province. Participant answers incorrectly or is unable to come to the correct conclusions. Success: 6 (100%)
Failure: 0 (0%)
3 In Q3, did the physician have a lower or higher incidence of not meeting optimal IV fluid volume, compared to the target? Correctly identifies a higher incidence of not meeting the optimal IV fluid volume than the target. Participant answers incorrectly or is unable to come to the correct conclusions. Success: 5 (83%)
Failure: 1 (17%)
4 Can you identify any information in the report that provides the physician with actions to improve results? Identifies recommendations Participant answers incorrectly or is unable to come to the correct conclusions. Success: 4 (67%)
Failure 2 (33%)
5 Did this physician have a lower or higher mean AKI risk for their patients compared to other physicians in the province? Correctly identifies a higher mean risk of AKI for this physician’s patients. Participant answers incorrectly or is unable to come to the correct conclusions. Success: 6 (100%)
Failure 0 (0%)
6 Which AKI risk variable was most prevalent in this physician’s patients, and how does it compare with the physician’s site averages? Correctly identifies the NSTEMI indication was most prevalent (62%), with the site average being lower (57%). Participant answers incorrectly or is unable to come to the correct conclusions. Success: 4 (67%)
Failure: 2 (33%)
7 How many patients did this physician treat during the reporting period? Correctly identifies 41 patients, including 14 at increased risk of AKI Participant answers incorrectly or is unable to come to the correct conclusions. Success: 6 (100%)
Failure: 0 (0%)
8 By what percent, approximately, did this physician exceed the safe contrast levels? Correctly answers 20%–30% Participant answers incorrectly or is unable to come to the correct conclusions. Success: 6 (100%)
Failure: 0 (0%)
9 Of all the physicians performing catheterizations in the province during this reporting period, how many had an AKI incidence that was higher than the 99% control limit? Correctly answers 3 physicians Participant answers incorrectly or is unable to come to the correct conclusions. Success: 5 (83%)
Failure: 1 (17%)
10 What does the graph on Contrast Dye Volume tell you about this physician’s performance versus other physicians in the province? Correctly identifies that it shows that the contrast dye amounts used by this physician are lower than other physicians in the province. Participant answers incorrectly or is unable to come to the correct conclusions. Success: 5 (83%)
Failure: 1 (17%)
11 What percent, approximately, of patients with LVEDP measured had the optimal IVF ordered by this physician? Correctly answers “approximately 70%” Participant answers incorrectly or is unable to come to the correct conclusions. Success: 6 (100%)
Failure: 0 (0%)
12 What percentage of patients for this physician received optimal IVF when their dye volume was 30–100 cc? Correctly answers “73% of patients” Participant answers incorrectly or is unable to come to the correct conclusions. Success: 6 (100%)
Failure: 0 (0%)
13 Is the AKI incidence of patients for this physician within or outside of the expected range of values due to chance? Correctly answers “within” Participant answers incorrectly or is unable to come to the correct conclusions. Success: 5 (83%)
Failure: 1 (17%)
Open in a new tab

AKI, acute kidney injury; IV(F), intravenous (fluid); LVEDP, left-ventricular end-diastolic pressure; NSTEMI, non-ST-elevation myocardial infarction; Q3, quarter 3.

Qualitative evaluation based on comments of physicians while reviewing reports and from semistructured interviews with physicians identified 3 themes related to the audit and feedback reports. These themes, and exemplar quotes, are provided in Supplemental Table S1. The themes are as follows:

  • 1.

    The summary page at the start of the report was well received. Participants felt that the summary page provided a valuable and quick summary of performance. Some participants stated that if their performance was well rated on this page, they would not be inclined to read their personal report further. The performance indicators were well received as a means to quickly communicate main findings; however, users tended to focus on the graphs rather than the performance indicators when reviewing this page.

  • 2.

    Influence of peer-to-peer comparisons: Several participants mentioned that peer-to-peer competition would be a driving force in leading them to improve their performance, more so than seeking to meet the indicated targets.

  • 3.

    Preference for visual data over text or tables: All participants preferred the graphs over tables and described the graphs as being more valuable in helping them understand the data rapidly, without requiring in-depth study. However, not all users were confident in their interpretation of the box plots, and many described the bar graphs as being a more difficult means by which to obtain information. Several participants felt that the funnel plots were the most effective way to show the physician data and compare them with others.

Cardiologists’ evaluation of the audit and feedback process

From among 31 eligible cardiologists practicing coronary angiography or intervention in Alberta at the time of the study, 17 (54.8%) provided responses to the survey on evaluation of the audit and feedback process. The characteristics of these physicians are shown in Table 4. Fifteen respondents (88.2%) agreed or strongly agreed that the data presented in the audit and feedback report were understandable; 17 respondents (100%) agreed or strongly agreed that the visual presentation of the report helped them understand their performance compared to that of their peers; and 14 respondents (82.4%) agreed or strongly agreed that discussion of the audit and feedback process with their site lead adequately addressed questions or concerns about their results (Fig. 3).

Table 4.

Characteristics of cardiologists receiving audit and feedback for contrast-associated acute kidney injury prevention in coronary angiography and intervention

Characteristics Results
Age, y
 < 45 2 (11.8)
 45–54 9 (52.9)
 ≥ 55 6 (35.3)
Sex
 Female 3 (17.6)
 Male 14 (82.4)
Years in clinical practice 19.6 (± 8.1)
Years performing coronary angiography or PCI 16.0 (± 7.1)
Number of coronary angiographies performed annually 488 (± 243)
Number of percutaneous interventions performed annually 262 (± 136)
Open in a new tab

Values are n (%) or mean (± standard deviation).

PCI, percutaneous coronary intervention.

Figure 3.

Figure 3

Open in a new tab

Responses of cardiologists to survey about the audit and feedback process for contrast-associated acute kidney injury prevention in coronary angiography and intervention. A&F, audit and feedback; AKI, acute kidney injury.

In response to questions about the specific process of care and outcome measures included in the audit and feedback, 16 respondents (94.1%) agreed or strongly agreed that the audit and feedback process helped them understand their performance on each measure compared to that of their peers. Fourteen respondents (82.4%) agreed or strongly agreed that the audit and feedback process helped them identify ways they could reduce the risk of AKI for their patients, and the same number agreed or strongly agreed that they were satisfied with the overall audit and feedback process. Open-ended responses on ways participants felt they could change their practice to improve their performance included 5 responses related to opportunities to use more procedural tactics to minimize the volume of radiocontrast dye used, and 2 responses identifying opportunity to improve their use of hemodynamic-guided IV fluid administration.

Discussion

In this study, we applied evidence-based principles to develop, implement, and evaluate an audit and feedback reporting process as part of a multifaceted intervention for CA-AKI prevention in coronary angiography and intervention. Audit and feedback reports were developed through a user-centred process involving cardiologists who perform these procedures, and they were designed to provide cardiologists with information on their performance on processes-of-care measures, including the volume of radiocontrast dye used, hemodynamic optimization of IV fluid administration, and AKI incidence, in relation to peer performance and targets for the same. Evaluation of the audit and feedback process suggested that cardiologists found the information understandable and the process acceptable with potential to support participant self-reflection and desired behaviour changes to improve uptake of steps for CA-AKI prevention.

To the best of our knowledge, this study is the first to describe the use of an audit and feedback process as a strategy to encourage uptake of approaches for preventing CA-AKI. Numerous prior studies have described the development of audit and feedback interventions to improve the delivery of many other aspects of clinical care.22,47, 48, 49, 50, 51, 52, 53, 54, 55 Similar to our work, these studies have typically described the design of an audit and feedback process based on its content and presentation, and the manner in which audit information and feedback are reported to end-users. Brehaut et al. have described several under-utilized principles associated with the effectiveness of an audit and feedback process, which address the nature of the desired action(s), the nature of the data available for feedback, the display of feedback, and how the feedback intervention is delivered.24 We applied many of these principles to our audit and feedback process for CA-AKI; a summary of these principles and how we applied them is provided in Table 5. Our evaluation suggests that the resulting audit and feedback reports included appropriate content, were understandable, supported users in making comparisons with their peers, and prompted users to consider behaviour changes consistent with the recommendations provided by the decision-support tool and consistent with the study goals, to improve prevention of CA-AKI.

Table 5.

Evidence-based principles and their application for effective audit and feedback reporting for contrast associated-acute kidney injury (CA-AKI) prevention in coronary angiography and intervention

Principles Description and rationale References Application
Recommend actions that are consistent with established goals and priorities Goals help direct attention and efforts. Goals should be explicit, specific, time-bound, recipient-defined, and somewhat challenging. 10,38,44 Provides tailored recommendations based on the processes of care to reduce CA-AKI
Recommend actions that can improve and are under the recipient’s control Encourages participant willingness to act 38 Relates to procedural behavior and intravenous fluid orders
Recommend specific actions and set benchmarks/targets Strengthens behavioural changes 43 Provides tailored recommendations based on the processes of care to reduce CA-AKI
Provide multiple instances of feedback Develops an iterative feedback loop where participants self-evaluate their progress 38 Quarterly reporting
Provide feedback as soon as possible and at a frequency informed by the number of new patient cases Too frequent leads to alert fatigue/discounting of feedback, too infrequent leads to discounting the feedback as no longer relevant 47,48,52 Quarterly reporting
Provide individual rather than general data Group-level data may be discounted, whereas personal reports increase a sense of responsibility. 27,32,46 Data are physician-specific
Choose comparators that reinforce desired behaviour change Comparators can help inform goals. Should be simple/clear and minimize possible mixed messages 41 Comparisons to peers at the same hospital and to peers across the province
Closely link the visual display and summary message Visual representations of summary messages should be consistent and should be linked clearly. 50 Plots are consistent with tables and written notes
Minimize extraneous cognitive load for feedback participants Graphical displays should be simple and consistent with the message being conveyed. Text and organization should minimize effort required to understand. 31 Clear graphical displays with legends and concise, written explanations
Provide short, actionable messages followed by optional detail Contains details for those interested and a main message for those who won’t dive into all of the details, to maximize retention 45 Recommendations, self-reflection
Address credibility of the information Deliver through a supervisor or colleague, characterize data source and quality, mitigate potential conflicts of interest, clarify strengths and weaknesses of feedback to increase perceived reliability 48,56,57 Provided by site-leads, data drawn from reliable data registries
Prevent defensive reactions to feedback Actively guiding reaction toward self-reflection and extending areas of previous success may be more effective 50,51,53,54,55 Questions to guide self-reflection and behaviour improvement
Construct feedback through social interaction Recipients should work to engage actively with the feedback and the provider 35,36 Discussion between site-lead and cardiologist
Open in a new tab

The development and evaluation of this audit and feedback intervention have limitations. First, cardiologists indicated that the comparison of their data to that of their peers would be more influential than comparison to targets on the audit and feedback reports, which suggests that the reports may be more effective at motivating improved performance of physicians whose practice metrics fall outside the control limits based on the performance of their peers, but the reports may not motivate average or better performers to improve further if they are already outperforming many of their colleagues.56,57 Further, if the performance of all physicians is less than ideal, the group as a whole may not be motivated to move toward appropriate targets. However, we included targets in the reports based on the best-performing cardiologists at each site, and included discussion of the targets, recommendations, and action plans during audit and feedback delivery, with site-leads to reinforce the importance of continuous improvements and sustained behavioural change for all participants. Second, not all participating cardiologists completed our survey, and it is possible that those who did were more engaged in the audit and feedback process and had more positive feedback than those who did not. Third, although the evaluation reported here supports the usability and acceptability of the audit and feedback process, these findings are a necessary but insufficient step toward achieving effective and meaningful behaviour change of clinicians. Further research is needed to determine whether an audit and feedback process improves processes of care for CA-AKI prevention and leads to a reduction in CA-AKI incidence following coronary angiography or intervention.

A notable aspect of our audit and feedback process is its implementation in conjunction with education and point-of-care clinical decision support. Combining these interventions helped to support the psychological model of control theory, defined by continuous feedback to assess and respond to discrepancies from a desired state.58 Integrating the audit and feedback process with computerized decision support allowed us to reinforce targets and recommendations for radiocontrast dye use and IV fluid administration not just at the time of audit and feedback report delivery, but also immediately prior to and during procedures when decision-making is occurring in real time. However, whether this is a more effective way to improve patient care has yet to be determined, as the mechanisms behind audit- and feedback-driven behavioural change remain unclear. Atlhough numerous theories and hypotheses have been proposed, reliable empirical studies comparing various audit and feedback strategies are lacking. Thus, further research that directly compares the effectiveness of different audit and feedback tactics, in isolation as well as in combination with other knowledge-translation strategies, is required to better understand ways to optimize feedback delivery and clinician behaviour change.

Conclusion

In this study, we incorporated previously identified evidence-based principles in an iterative process to develop and evaluate delivery of an audit and feedback for prevention of CA-AKI in coronary angiography and intervention. Usability testing and end-user feedback from cardiologists suggested overall acceptability of and satisfaction with the audit and feedback process, which is a critical step prior to broad implementation and evaluation of an audit and feedback process to increase CA-AKI prevention within the healthcare setting.

Acknowledgements

The Alberta Provincial Project for Outcomes Assessment in Coronary Heart Disease (APPROACH) project has been made possible by support from Alberta Health Services(Cardiac Sciences), Canada, the Libin Cardiovascular Institute of Alberta, and Mazankowski Alberta Heart Institute.

Funding Sources

This work was supported by a Partnership for Research in the Health System award from Alberta Innovates, and a Canadian Institutes for Health Research (CIHR) Foundation Grant. M.T.J. was supported by a CIHR New Investigator Award and a CIHR Foundation Grant.

Disclosures

M.T.J. was the principal investigator of an investigator-initiated research grant (outside the submitted work) funded by Amgen Canada. The other authors have no conflicts of interest to disclose.

Footnotes

Ethics Statement: Informed consent was obtained from all cardiologists participating in the study. Ethics approval was obtained from the health research ethics boards of the University of Alberta and the University of Calgary.

See page 280 for disclosure information.

To access the supplementary material accompanying this article, visit CJC Open at https://www.cjcopen.ca/ and at https://doi.org/10.1016/j.cjco.2021.10.006.

Supplementary Material

Supplementary Appendix S1
mmc1.docx (53.7KB, docx)
Supplementary Appendix S2
mmc2.docx (20.7KB, docx)
Supplementary Table S1
mmc3.docx (20.2KB, docx)
Supplementary Appendix S3
mmc4.pdf (81.4KB, pdf)

References

  • 1.Amin A.P., Salisbury A.C., McCullough P.A., et al. Trends in the incidence of acute kidney injury in patients hospitalized with acute myocardial infarction. Arch Intern Med. 2012;172:246–253. doi: 10.1001/archinternmed.2011.1202. [DOI] [PubMed] [Google Scholar]
  • 2.Weisbord S.D., Chen H., Stone R.A., et al. Associations of increases in serum creatinine with mortality and length of hospital stay after coronary angiography. J Am Soc Nephrol. 2006;17:2871–2877. doi: 10.1681/ASN.2006030301. [DOI] [PubMed] [Google Scholar]
  • 3.Chertow G.M., Burdick E., Honour M., Bonventre J.V., Bates D.W. Acute kidney injury, mortality, length of stay, and costs in hospitalized patients. J Am Soc Nephrol. 2005;16:3365–3370. doi: 10.1681/ASN.2004090740. [DOI] [PubMed] [Google Scholar]
  • 4.Azzalini L., Candilio L., McCullough P.A., Colombo A. Current risk of contrast-induced acute kidney injury after coronary angiography and intervention: a reappraisal of the literature. Can J Cardiol. 2017;33:1225–1228. doi: 10.1016/j.cjca.2017.07.482. [DOI] [PubMed] [Google Scholar]
  • 5.James M.T., Ghali W.A., Knudtson M.L., et al. Associations between acute kidney injury and cardiovascular and renal outcomes after coronary angiography. Circulation. 2011;123:409–416. doi: 10.1161/CIRCULATIONAHA.110.970160. [DOI] [PubMed] [Google Scholar]
  • 6.James M.T., Ghali W.A., Tonelli M., et al. Acute kidney injury following coronary angiography is associated with a long-term decline in kidney function. Kidney Int. 2010;78:803–809. doi: 10.1038/ki.2010.258. [DOI] [PubMed] [Google Scholar]
  • 7.Azzalini L., Spagnoli V., Ly H.Q. Contrast-induced nephropathy: from pathophysiology to preventive strategies. Can J Cardiol. 2016;32:247–255. doi: 10.1016/j.cjca.2015.05.013. [DOI] [PubMed] [Google Scholar]
  • 8.Prasad A., Rosenthal N.A., Kartashov A., Knish K., Dreyfus J. Contemporary trend of acute kidney injury incidence and incremental costs among US patients undergoing percutaneous coronary procedures. Catheter Cardiovasc Interv. 2020;96:1184–1197. doi: 10.1002/ccd.28824. [DOI] [PubMed] [Google Scholar]
  • 9.Freeman R.V., O'Donnell M., Share D., et al. Nephropathy requiring dialysis after percutaneous coronary intervention and the critical role of an adjusted contrast dose. Am J Cardiol. 2002;90:1068–1073. doi: 10.1016/s0002-9149(02)02771-6. [DOI] [PubMed] [Google Scholar]
  • 10.Minsinger K.D., Kassis H.M., Block C.A., Sidhu M., Brown J.R. Meta-analysis of the effect of automated contrast injection devices versus manual injection and contrast volume on risk of contrast-induced nephropathy. Am J Cardiol. 2014;113:49–53. doi: 10.1016/j.amjcard.2013.08.040. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Stacul F., van der Molen A.J., Reimer P., et al. Contrast induced nephropathy: updated ESUR Contrast Media Safety Committee guidelines. Eur Radiol. 2011;21:2527–2541. doi: 10.1007/s00330-011-2225-0. [DOI] [PubMed] [Google Scholar]
  • 12.Nayak K.R., Mehta H.S., Price M.J., et al. A novel technique for ultra-low contrast administration during angiography or intervention. Catheter Cardiovasc Interv. 2010;75:1076–1083. doi: 10.1002/ccd.22414. [DOI] [PubMed] [Google Scholar]
  • 13.Anne G., Gruberg L., Huber A., et al. Traditional versus automated injection contrast system in diagnostic and percutaneous coronary interventional procedures: comparison of the contrast volume delivered. J Invasive Cardiol. 2004;16:360–362. [PubMed] [Google Scholar]
  • 14.Barrett B.J., Parfrey P.S. Clinical practice. Preventing nephropathy induced by contrast medium. N Engl J Med. 2006;354:379–386. doi: 10.1056/NEJMcp050801. [DOI] [PubMed] [Google Scholar]
  • 15.Pannu N., Wiebe N., Tonelli M. Alberta Kidney Disease Network. Prophylaxis strategies for contrast-induced nephropathy. JAMA. 2006;295:2765–2779. doi: 10.1001/jama.295.23.2765. [DOI] [PubMed] [Google Scholar]
  • 16.Wichmann J.L., Katzberg R.W., Litwin S.E., et al. Contrast-induced nephropathy. Circulation. 2015;132:1931–1936. doi: 10.1161/CIRCULATIONAHA.115.014672. [DOI] [PubMed] [Google Scholar]
  • 17.Silver S.A., Shah P.M., Chertow G.M., et al. Risk prediction models for contrast induced nephropathy: systematic review. BMJ. 2015;351:h4395. doi: 10.1136/bmj.h4395. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Kooiman J., Gurm H.S. Predicting contrast-induced renal complications in the catheterization laboratory. Interv Cardiol Clin. 2014;3:369–377. doi: 10.1016/j.iccl.2014.03.005. [DOI] [PubMed] [Google Scholar]
  • 19.Tsai T.T., Patel U.D., Chang T.I., et al. Validated contemporary risk model of acute kidney injury in patients undergoing percutaneous coronary interventions: insights from the National Cardiovascular Data Registry Cath-PCI Registry. J Am Heart Assoc. 2014;3 doi: 10.1161/JAHA.114.001380. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Weisbord S.D. AKI and medical care after coronary angiography: renalism revisited. Clin J Am Soc Nephrol. 2014;9:1823–1825. doi: 10.2215/CJN.09430914. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Weisbord S.D., Mor M.K., Kim S., et al. Factors associated with the use of preventive care for contrast-induced acute kidney injury. J Gen Intern Med. 2009;24:289–298. doi: 10.1007/s11606-008-0894-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Ivers N., Jamtvedt G., Flottorp S., et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012:CD000259. doi: 10.1002/14651858.CD000259.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Straus S.E., Tetroe J., Graham I.D. Wiley; Oxford: 2013. Knowledge Translation in Health Care: Moving from Evidence to Practice. [Google Scholar]
  • 24.Brehaut J.C., Colquhoun H.L., Eva K.W., et al. Practice feedback interventions: 15 suggestions for optimizing effectiveness. Ann Intern Med. 2016;164:435–441. doi: 10.7326/M15-2248. [DOI] [PubMed] [Google Scholar]
  • 25.Foy R., Eccles M.P., Jamtvedt G., et al. What do we know about how to do audit and feedback? Pitfalls in applying evidence from a systematic review. BMC Health Serv Res. 2005;5:50. doi: 10.1186/1472-6963-5-50. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Jamtvedt G., Young J.M., Kristoffersen D.T., Thomson O'Brien M.A., Oxman A.D. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2003;(3):CD000259. doi: 10.1002/14651858.CD000259. [DOI] [PubMed] [Google Scholar]
  • 27.Veloski J., Boex J.R., Grasberger M.J., Evans A., Wolfson D.B. Systematic review of the literature on assessment, feedback and physicians' clinical performance: BEME Guide No. 7. Med Teach. 2006;28:117–128. doi: 10.1080/01421590600622665. [DOI] [PubMed] [Google Scholar]
  • 28.Mitchell E., Sullivan F., Grimshaw J.M., Donnan P.T., Watt G. Improving management of hypertension in general practice: a randomised controlled trial of feedback derived from electronic patient data. Br J Gen Pract. 2005;55:94–101. [PMC free article] [PubMed] [Google Scholar]
  • 29.Locke E.A., Latham G.P. Building a practically useful theory of goal setting and task motivation. A 35-year odyssey. Am Psychol. 2002;57:705–717. doi: 10.1037//0003-066x.57.9.705. [DOI] [PubMed] [Google Scholar]
  • 30.Locke E.A., Latham G.P. Prentice-Hall; Englewood Cliffs, NJ: 1990. A Theory of Goal Setting & Task Performance. [Google Scholar]
  • 31.Stull M.K. Performance feedback: a question of source credibility. J Nurs Adm. 1986;16:17–39. [PubMed] [Google Scholar]
  • 32.Lam C.F., DeRue D.S., Karam E.P., Hollenbeck J.R. The impact of feedback frequency on learning and task performance: challenging the "more is better” assumption. Organ Behav Hum Dec. 2011;116:217–228. [Google Scholar]
  • 33.Opiyo N., Shepperd S., Musila N., et al. Comparison of alternative evidence summary and presentation formats in clinical guideline development: a mixed-method study. PLoS One. 2013;8 doi: 10.1371/journal.pone.0055067. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Lavis J.N., Wilson M.G., Grimshaw J.M., et al. Supporting the use of health technology assessments in policy making about health systems. Int J Technol Assess. 2010;26:405–414. doi: 10.1017/S026646231000108X. [DOI] [PubMed] [Google Scholar]
  • 35.Archerkath J., Johnson D.W., Johnson R.T. Individual versus group feedback in cooperative groups. J Soc Psychol. 1994;134:681–694. [Google Scholar]
  • 36.Hibbard J.H., Peters E., Slovic P., Finucane M.L., Tusler M. Making health care quality reports easier to use. Jt Comm J Qual Improv. 2001;27:591–604. doi: 10.1016/s1070-3241(01)27051-5. [DOI] [PubMed] [Google Scholar]
  • 37.Sargeant J., Lockyer J., Mann K., et al. Facilitated reflective performance feedback: developing an evidence- and theory-based model that builds relationship, explores reactions and content, and coaches for performance change (R2C2) Acad Med. 2015;90:1698–1706. doi: 10.1097/ACM.0000000000000809. [DOI] [PubMed] [Google Scholar]
  • 38.Reiter H.I., Rosenfeld J., Nandagopal K., Eva K. Do clinical clerks provide candidates with adequate formative assessment during objective structured clinical examinations? Adv Health Sci Educ. 2004;9:189–199. doi: 10.1023/B:AHSE.0000038172.97337.d5. [DOI] [PubMed] [Google Scholar]
  • 39.Carless D., Salter D., Yang M., Lam J. Developing sustainable feedback practices. Stud High Educ. 2011;36:395–407. [Google Scholar]
  • 40.Overeem K., Driessen E.W., Arah O.A., et al. Peer mentoring in doctor performance assessment: strategies, obstacles and benefits. Med Educ. 2010;44:140–147. doi: 10.1111/j.1365-2923.2009.03580.x. [DOI] [PubMed] [Google Scholar]
  • 41.James M.T., Har B.J., Tyrrell B.D., et al. Clinical decision support to reduce contrast-induced kidney injury during cardiac catheterization: design of a randomized stepped-wedge trial. Can J Cardiol. 2019;35:1124–1133. doi: 10.1016/j.cjca.2019.06.002. [DOI] [PubMed] [Google Scholar]
  • 42.Amin A.P., Crimmins-Reda P., Miller S., et al. Reducing acute kidney injury and costs of percutaneous coronary intervention by patient-centered, evidence-based contrast use. Circ Cardiovasc Qual Outcomes. 2019;12 doi: 10.1161/CIRCOUTCOMES.118.004961. [DOI] [PubMed] [Google Scholar]
  • 43.Malik A.O., Amin A., Kennedy K., et al. Patient-centered contrast thresholds to reduce acute kidney injury in high-risk patients undergoing percutaneous coronary intervention. Am Heart J. 2021;234:51–59. doi: 10.1016/j.ahj.2020.12.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Brar S.S., Aharonian V., Mansukhani P., et al. Haemodynamic-guided fluid administration for the prevention of contrast-induced acute kidney injury: the POSEIDON randomised controlled trial. Lancet. 2014;383:1814–1823. doi: 10.1016/S0140-6736(14)60689-9. [DOI] [PubMed] [Google Scholar]
  • 45.Mehran R., Dangas G.D., Weisbord S.D. Contrast-associated acute kidney injury. N Engl J Med. 2019;380:2146–2155. doi: 10.1056/NEJMra1805256. [DOI] [PubMed] [Google Scholar]
  • 46.University of Calgary. W21C-Human Factors. https://www.w21c.org/human-factors-and-simulation/ Available at:
  • 47.Hartley S., Foy R., Walwyn R.E.A., et al. The evaluation of enhanced feedback interventions to reduce unnecessary blood transfusions (AFFINITIE): protocol for two linked cluster randomised factorial controlled trials. Implement Sci. 2017;12:84. doi: 10.1186/s13012-017-0614-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Eccles M., Steen N., Grimshaw J., et al. Effect of audit and feedback, and reminder messages on primary-care radiology referrals: a randomised trial. Lancet. 2001;357:1406–1409. doi: 10.1016/S0140-6736(00)04564-5. [DOI] [PubMed] [Google Scholar]
  • 49.Thomas R.E., Croal B.L., Ramsay C., Eccles M., Grimshaw J. Effect of enhanced feedback and brief educational reminder messages on laboratory test requesting in primary care: a cluster randomised trial. Lancet. 2006;367:1990–1996. doi: 10.1016/S0140-6736(06)68888-0. [DOI] [PubMed] [Google Scholar]
  • 50.Kempegowda P., Coombs B., Nightingale P., et al. Regular and frequent feedback of specific clinical criteria delivers a sustained improvement in the management of diabetic ketoacidosis. Clin Med (Lond) 2017;17:389–394. doi: 10.7861/clinmedicine.17-5-389. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Elouafkaoui P., Young L., Newlands R., et al. An audit and feedback intervention for reducing antibiotic prescribing in general dental practice: the RAPiD cluster randomised controlled trial. PLoS Med. 2016;13 doi: 10.1371/journal.pmed.1002115. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Dunn S., Sprague A.E., Grimshaw J.M., et al. A mixed methods evaluation of the maternal-newborn dashboard in Ontario: dashboard attributes, contextual factors, and facilitators and barriers to use: a study protocol. Implement Sci. 2016;11:59. doi: 10.1186/s13012-016-0427-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Wald H.L., Kramer A.M. Feasibility of audit and feedback to reduce postoperative urinary catheter duration. J Hosp Med. 2011;6:183–189. doi: 10.1002/jhm.846. [DOI] [PubMed] [Google Scholar]
  • 54.Roberts E., Dawoud D.M., Hughes D.A., Cefai C. Evaluation of a consultant audit and feedback programme to improve the quality of antimicrobial prescribing in acute medical admissions. Int J Pharm Pract. 2015;23:333–339. doi: 10.1111/ijpp.12173. [DOI] [PubMed] [Google Scholar]
  • 55.Tinmouth J., Patel J., Hilsden R.J., Ivers N., Llovet D. Audit and feedback interventions to improve endoscopist performance: principles and effectiveness. Best Pract Res Clin Gastroenterol. 2016;30:473–485. doi: 10.1016/j.bpg.2016.04.002. [DOI] [PubMed] [Google Scholar]
  • 56.Brown J.R., McCullough P.A., Splaine M.E., et al. How do centres begin the process to prevent contrast-induced acute kidney injury: a report from a new regional collaborative. BMJ Qual Saf. 2012;21:54–62. doi: 10.1136/bmjqs-2011-000041. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57.Brown J.R., Solomon R.J., Sarnak M.J., et al. Reducing contrast-induced acute kidney injury using a regional multicenter quality improvement intervention. Circ Cardiovasc Qual Outcomes. 2014;7:693–700. doi: 10.1161/CIRCOUTCOMES.114.000903. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 58.Carver C.S., Scheier M.F. Control theory: a useful conceptual framework for personality-social, clinical, and health psychology. Psychol Bull. 1982;92:111–135. [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Appendix S1
mmc1.docx (53.7KB, docx)
Supplementary Appendix S2
mmc2.docx (20.7KB, docx)
Supplementary Table S1
mmc3.docx (20.2KB, docx)
Supplementary Appendix S3
mmc4.pdf (81.4KB, pdf)

Articles from CJC Open are provided here courtesy of Elsevier

RESOURCES