Skip to main content

Some NLM-NCBI services and products are experiencing heavy traffic, which may affect performance and availability. We apologize for the inconvenience and appreciate your patience. For assistance, please contact our Help Desk at info@ncbi.nlm.nih.gov.

HHS Author Manuscripts logoLink to HHS Author Manuscripts
. Author manuscript; available in PMC: 2020 Nov 1.
Published in final edited form as: Int J Med Inform. 2019 Sep 4;131:103956. doi: 10.1016/j.ijmedinf.2019.103956

Interrupting providers with clinical decision support to improve care for heart failure

Saul Blecker 1,2,3, Jonathan S Austrian 2,4, Leora Horwitz 1,2,3, Gilad Kuperman 5, Donna Shelley 1,2, Meg Ferrauiola 4, Stuart D Katz 2
PMCID: PMC6994190  NIHMSID: NIHMS1065540  PMID: 31525580

Abstract

Background:

Evidence-based therapy for heart failure remains underutilized at hospital discharge, particularly for patients with heart failure who are hospitalized for another cause. We developed clinical decision support (CDS) to recommend an angiotensin converting enzyme (ACE) inhibitor during hospitalization to promote its continuation at discharge. The CDS was designed to be implemented in both interruptive and non-interruptive versions.

Objectives:

To compare the effectiveness and implementation of interruptive and non-interruptive versions of a CDS to improve care for heart failure.

Methods:

Hospitalizations of patients with reduced ejection fraction were pseudo-randomized to deliver interruptive or non-interruptive CDS alerts to providers based on even or odd medical record number. We compared discharge utilization of an ACE inhibitor or angiotensin receptor blocker (ARB) for these two implementation approaches. We also assessed adoption and implementation fidelity of the CDS.

Results:

Of 958 hospitalizations, interruptive alert hospitalizations had higher rates of discharge utilization of ACE inhibitors or ARBs than non-interruptive alert hospitalizations (79.6% vs. 74.2%, p=0.05). Utilization was higher for interruptive alert versus non-interruptive alert hospitalizations which were principally for causes other than heart failure (79.8% vs. 73.4%; p=0.05) but not among hospitalizations with a principal heart failure diagnosis (85.9% vs.81.7%; p=0.49). As compared to non-interruptive hospitalizations, interruptive alert hospitalizations were more likely to have had: an alert with any response (40.6% vs. 13.1%, p<0.001), contraindications reported (33.1% vs 11.3%, p<0.001), and an ACE inhibitor ordered within twelve hours of the alert (17.6% vs 10.3%, p<0.01). The response rate for the interruptive alert was 1.7%, and a median (25th, 75th percentile) of 14 (5,32) alerts were triggered per hospitalization.

Conclusions:

A CDS implemented as an interruptive alert was associated with improved quality of care for heart failure. Whether the potential benefits of CDS in improving cardiovascular care were worth the high burden of interruptive alerts deserves further consideration.

ClinicalTrials.gov Identifier:

Keywords: clinical decision support, comparative effectiveness, heart failure


Patients with heart failure are hospitalized 1 million times annually for acute heart failure and 3 million times for other causes.1 Regardless of the reason for hospitalization, heart failure has been associated with poor outcomes, including a greater than 20% rate of mortality in the year following discharge.2 Guideline recommended therapies such as angiotensin converting enzyme (ACE) inhibitors are associated with improved outcomes in heart failure3,4 and, as a result, quality improvement initiatives have targeted improving use of such therapies for patients with acute decompensated heart failure.57 Although such initiatives have improved care for patients with heart failure, there are still gaps in care, particularly for heart failure patients hospitalized for other causes.1

One potential solution to improving care-- and ultimately outcomes-- for hospitalized patients with heart failure is computer based clinical decision support (CDS). CDS has been shown to increase uptake of cardiovascular care metrics in the inpatient setting.8,9 However, CDS has not been studied in the larger population of heart failure patients hospitalized for any cause, who are typically overlooked for heart failure quality improvement efforts.57,10 Furthermore, although CDS has been shown to improve compliance with guideline recommended care, the effectiveness of CDS has been limited by inadequate implementation including lack of user-centered design and frequent workflow interruptions.1113

Implementing CDS as a non-interruptive alert has appeal, as non-interruptive CDS may reduce cognitive burden and workflow impediments compared to interruptive alerts such as pop-up windows.14,15 However, non-interruptive alerts may be easier to ignore and, as a result, may be less effective.15 Studies examining the effect of non-interruptive alerts on outcomes and utilization are limited with mixed results1618 and we are unaware of any evaluations of the provider checklist tool functionality as an intervention format.

We developed an inpatient CDS that recommends prescription of an ACE inhibitor for hospitalized patients with heart failure with reduced ejection fraction, regardless of reason for admission.19,20 We tested two approaches for implementation of the CDS tool: an interruptive alert, in which a pop-up displayed the recommendations to providers, and a non-interruptive alert, in which the CDS could be accessed anytime in the patient chart. Both versions of the CDS were triggered during hospitalization with the goal of improving percent of patients discharged on this evidence-based therapy at discharge, a commonly used process measure.6 The purpose of this study was to evaluate the comparative effectiveness of a CDS implemented as an interruptive alert and the same CDS implemented as a non-interruptive alert.

Methods

We performed a study of patients with heart failure hospitalized at Tisch Hospital of the NYU Langone Health System, an urban academic medical center. The study included two patient cohorts. The primary analytic cohort consisted of patients appropriate for therapy with an ACE inhibitor at discharge and included patients discharged from the hospital between introduction of the CDS on April 17, 2017 and March 31, 2018. Inclusion criteria were hospitalizations for at least 24 hours, an ejection fraction≤40%, and the following labs or vital signs at time of discharge: potassium<5.2mEq/L, estimated glomerular filtration rate (eGFR) ≥30mL/min/1.73m2, and systolic blood pressure≥90mmHg.4 Exclusion criteria included: allergy to an ACE inhibitor, patients on the obstetrical service, and patients who died during hospitalization or were discharge to hospice care. In secondary analysis, we studied trends in the pre-intervention and post-intervention period; for these analyses, we used the same inclusion and exclusion criteria for patients who were discharged between March 1, 2016 and March 31, 2018. The second cohort was used for assessment of CDS implementation and included all hospitalizations that received an alert, for patients discharged between introduction of the CDS tool and March 31, 2018.

CDS Intervention

We developed CDS, i.e., clinical logic and data presentation formats, to suggest providers prescribe an ACE inhibitor for appropriate inpatients with heart failure, as defined by a reduced ejection fraction.4 The logic was triggered for hospitalized patients with an ejection fraction≤40% who were not on an ACE inhibitor, angiotensin receptor blocker (ARB), or ARB-neprilysin inhibitor, were not pregnant, and had a blood pressure>90 mmHg, eGFR>30 mL/min/1.73m2, and potassium<5.2 mEq/L; all data came from the electronic health record (EHR). The CDS indicated that the patient was not on an ACE inhibitor and that this therapy is recommended for patients with heart failure. Recent trends in the patient’s blood pressure, potassium, creatinine, eGFR, and ejection fraction were also displayed. Providers were given the option to order a low dose ACE inhibitor, order a contraindication, or dismiss the alert (Figure 1). The order for the ACE inhibitor included the option for a concurrent order for a basic metabolic panel in 48 hours. If an order was chosen, it was added to the sidebar order shopping cart to be signed. The CDS did not require a response to continue another task, i.e. it could be dismissed immediately. The CDS could also be dismissed when a provider clicked to “reassess patient,” which would silence the CDS for six hours for that provider. The CDS tool was built using standard functionality within the EHR (Epic, Epic Systems, Verona, WI). To improve success of the CDS, we employed a user-centered approach in development, including formative assessment with end-user providers19 and usability testing.20

Figure 1.

Figure 1.

Screenshots of clinical decision support (CDS) tool. (A) Interruptive version of CDS; (B) location of non-interruptive version of CDS, highlighted with red box; clicking the link in the non-interruptive alert would take the user to a screen similar to the interruptive alert.

We developed two approaches for implementation of the CDS tool: an interruptive version and a non-interruptive version of the alert. For patients assigned to the interruptive CDS tool, providers received a pop-up alert whenever entering into the order activity; the timing of the alert was chosen to target providers who write orders for the patient and to fit within the workflow of placing orders.19 The alert was triggered anytime a provider entered into the order activity, so long as inclusion criteria were met, no orders from a prior CDS had been placed, and the CDS had not been locked out by that provider in the prior six hours. The non-interruptive alert was available to providers through notification in the daily provider checklist that displays on the side of the patient chart as well as adjacent to the provider’s patient list (Figure 1). When a provider selected the non-interruptive alert checklist, the same CDS would be displayed as in the interruptive alert. Therefore, the only difference between the interruptive and non-interruptive alerts was in how they were presented to the provider; the alerts were otherwise the same in content and view.

Treatment Assignment

We used pseudo-randomization to assign the approach for implementation based on last digit of medical record number (MRN). Hospitalizations of greater than 24 hours with an even MRN were eligible for the interruptive alert, while hospitalizations with an odd MRN were eligible for a non-interruptive alert. We used an intention-to-treat approach, such that we included all patients who were appropriate for therapy in the cohort, regardless of whether an alert was actually triggered. For clarity, we referred to hospitalizations with an even MRN, i.e. those who were eligible for the interruptive alert, as “interruptive alert hospitalizations” and hospitalizations with an odd MRN as “non-interruptive alert hospitalizations.” We applied this terminology to the pre-intervention period as well.

Outcomes

The primary outcome was the relative effectiveness of the two versions of the CDS tool, measured as the percent of patients who were discharged on an ACE inhibitor or ARB.

Our implementation outcomes were informed by Proctor’s taxonomy.21 We measured adoption as the percent of hospitalizations with at least one alert that included an action to an alert that was not a dismissal, i.e. at least one alert led to the initiation of an order for an ACE inhibitor or a contraindication reported (Appendix Table). We measured fidelity as the percent of hospitalizations with at least one alert that led to signing of any order, led to signing of any of the individual orders, and led to an order for an ACE inhibitor within twelve hours of the alert being triggered. We also evaluated adoption and fidelity at the level of the individual alert for the interruptive alert. We estimated positive predictive value (PPV) as the number of hospitalizations in the primary analytic cohort, i.e. those eligible for the alert, that had CDS triggered over the total number of hospitalizations that received CDS. Sensitivity was measured as the number of hospitalizations in the primary analytic cohort that had the CDS triggered divided by these hospitalizations plus the number of hospitalizations in the primary analytic cohort that were not discharged on an ACE inhibitor or ARB and did not received an alert.

Subgroups

We assessed outcomes for hospitalizations with a principal discharge diagnosis of heart failure and hospitalizations with only a secondary discharge diagnosis of heart failure. Heart failure diagnosis was defined by standard International Classification of Disease (ICD) discharge diagnosis codes.1

Statistical Analysis

We compared ACE inhibitor or ARB utilization at discharge for interruptive and non-interruptive hospitalizations using chi-squared tests. We also used chi-squared tests to compare utilization for subgroups. To determine the comparative effectiveness of the interruptive and non-interruptive alerts after adjusting for potential confounders, we developed a logistic regression model in which the dependent variable was discharge utilization of an ACE inhibitor or ARB and the primary independent variable was an indicator for method of CDS implementation, i.e. interruptive or non-interruptive. We first developed an unadjusted model and then added the following covariates: age, sex, race, ethnicity, Medicaid insurance, and the presence of a principal or secondary heart failure diagnosis.

In secondary analysis, we compared ACE inhibitor or ARB utilization in the pre-CDS period and the post-CDS periods for all eligible patients using chi-squared tests. We also used chi-squared tests to compare pre- and post-utilization for various subgroups: those eligible for the interruptive alert, those eligible for the non-interruptive alert, those hospitalized with a principal diagnosis of heart failure, those hospitalized with a secondary diagnosis of heart failure, and the intersections of these subgroups.

We evaluated implementation outcomes by calculating percentages observed. We compared these outcomes between the interruptive and non-interruptive alerts using chi-squared tests.

We estimated the tradeoff of benefit of quality and burden of the interruptive alert using a metric we termed “number of triggers needed to change quality.” To calculate this, we divided the number of additional patients who were discharged on an ACE inhibitor or ARB with the interruptive alert versus non-interruptive alert by total number of times the alert was triggered. The number of the additional patients was calculated as the difference in rate of prescription for ACE inhibitors or ARBs between the two groups times the number of patients in the interruptive group. We also performed these estimates for patients with a secondary diagnosis of heart failure.

Results

Of 958 hospitalizations of patients included in the primary analytic cohort, 465 were in the interruptive alert group and 493 were in the non-interruptive alert group. A principal discharge diagnosis of heart failure was present in 15.6% of hospitalizations and another 73.0% of hospitalizations had a secondary discharge diagnosis of heart failure (Table 1).

Table 1.

Baseline characteristics of patients with heart failure (HF), based on eligibility for interruptive or non-interruptive alerts.

Full Cohort Interruptive Alert Group Non-Interruptive Alert Group
Characteristics n=958 n=465 n=493
Age, mean (SD) 70.6 (14.1) 69.2 (13.9) 71.9 (14.2)
Female 32.1 30.8 33.3
Black race 14.4 13.1 15.6
Hispanic ethnicity 1.9 1.1 2.6
Medicaid insurance 3.8 5.2 2.4
Principal HF diagnosis 15.6 16.8 14.4
Secondary HF diagnosis 73.0 71.2 74.7
No HF discharge diagnosis 11.5 12.0 11.0

In percentage unless otherwise shown.

Following implementation of CDS, interruptive alert hospitalizations had higher rates of discharge utilization of ACE inhibitors or ARBs than non-interruptive alert hospitalizations, although the difference in rates did not quite reach statistical significance (79.6% vs. 74.2%, p=0.05; Figure 2). As compared to non-interruptive alert hospitalizations, interruptive alert hospitalizations had an unadjusted odds ratio of 1.35 (95% CI 1.00–1.83) and an adjusted odds ratio of 1.31 (95% CI 0.96–1.79) for discharge utilization of ACE inhibitors or ARBs. Among the subgroup of patients with a secondary diagnosis of heart failure, utilization of ACE inhibitors or ARBs was also higher for interruptive alert versus non-interruptive alert hospitalizations (79.8% vs. 73.4%; p=0.05). We found no difference in utilization between the two alerts for the subgroup of patients with a principal discharge diagnosis of heart failure (85.9% vs. 81.7%; p=0.49; Figure 2).

Figure 2.

Figure 2.

Compliance with angiotensin receptor enzyme (ACE) inhibitor or angiotensin receptor blocker (ARB) for interruptive versus non-interruptive clinical decision support (CDS) alert hospitalizations in the overall cohort and subgroups with principal and secondary heart failure (HF) discharge diagnoses.

We included a total of 1,849 hospitalizations in the pre-CDS versus post-CDS analysis, with 891 hospitalizations in the pre-CDS period and 958 in the post-CDS period. Among all hospitalizations, the rate of ACE inhibitor or ARB utilization at discharge was 74.6% at baseline and did not improve with introduction of the CDS (p=0.27; Table 2). However, among hospitalizations which would be eligible for the interruptive alert, i.e. those appropriate for therapy and with an even MRN, ACE inhibitor or ARB use increased from 73.6% in the pre-CDS period to 79.6% in the post-CDS period (p=0.04). This improvement with the interruptive alert was primarily observed among the subgroup of hospitalizations with a secondary diagnosis of heart failure, for whom compliance with an ACE inhibitor or ARB at discharge increased from 71.1% to 79.8% (p=0.01; Table 2).

Table 2.

Use of ACE inhibitor or ARB on discharge among eligible patients with heart failure, before and after introduction of the clinical decision support (CDS) intervention.

Pre-CDS (%) Post-CDS (%) p-value
Overall 74.6 76.8 0.27
 With Principal HF Diagnosis 85.3 83.9 0.71
 With Secondary HF Diagnosis 72.0 76.4 0.08
Interruptive Alert Group 73.6 79.6 0.04
 With Principal HF Diagnosis 82.8 85.9 0.58
 With Secondary HF Diagnosis 71.1 79.8 0.01
Non-Interruptive Alert Group 75.6 74.2 0.63
 With Principal HF Diagnosis 87.8 81.7 0.27
 With Secondary HF Diagnosis 73.0 73.4 0.91

With regards to CDS implementation outcomes, there were 822 hospitalizations that had the CDS triggered at least once. Adoption was higher for interruptive alert hospitalizations: 40.6% of these hospitalizations had at least one alert that was not dismissed as compared to 13.1% of non-interruptive alert hospitalizations (p<0.001; Table 3). Hospitalizations receiving the interruptive alert were more likely to have had: an ACE inhibitor ordered through the CDS (3.6% vs. 0.9%, p<0.01), an ACE inhibitor ordered within 12 hours of CDS being triggered (17.6% vs. 10.3%, p<0.01), and a contraindication reported through the alert (33.1% vs. 11.3%, p<0.001). The sensitivity of the CDS was 93% and the PPV was 70% and similar between the two implementation groups (Table 3).

Table 3.

Implementation characteristics of clinical decision support (CDS) tool, by type of alert

All alerts Interruptive alerts Non-interruptive alerts p-value
Number of hospitalizations that received the CDS 822 387 435
Hospitalizations with any response to CDS (%) 26.0 40.6 13.1 <0.001
Hospitalizations with any order placed through CDS (%) 23.5 36.2 12.2 <0.001
Hospitalizations with ACE inhibitor ordered through CDS (%) 2.2 3.6 0.9 0.008
Hospitalizations with contraindication reported through CDS (%) 21.5 33.1 11.3 <0.001
Hospitalizations with basic metabolic panel ordered through CDS (%) 1.3 2.3 0.5 0.02
Hospitalizations with ACE inhibitor placed within 12 hours of CDS (%) 13.8 17.6 10.3 0.003
Positive predictive value of the alert 0.70 0.69 0.70 0.67
Sensitivity of the alert 0.93  0.91  0.94  0.31

A total of 10,034 interruptive alerts were triggered among the 387 hospitalizations that received this CDS implementation strategy. This resulted in a mean (standard deviation) of 25.9 (32.9) interruptive alerts having been triggered per hospitalization (Table 4). Only 170 of the 10,034 interruptive alerts were not dismissed, for a dismissal rate of 98.3%. Of these 170 alerts, 143 (84.2%) resulted in a completed order being placed through the CDS tool. The majority of orders (n=128) were for a reported contraindication to therapy. Fifteen alert-based orders, representing 0.1% of all alerts, were for a provider ordering an ACE inhibitor; nine of these orders (60%) were accompanied by a signed order for a basic metabolic panel (Table 3). In total, 76 orders for Lisinopril were placed within 12 hours of an interruptive alert being triggered, for an ordering rate of 0.8%.

Table 4.

Frequency and provider responses to interruptive version of CDS tool.

Alerts
Total number of alerts (N) 10,034
Median (25th, 75th percentile) alerts per hospitalization 14 (5,32)
Mean (SD) alerts per hospitalization 25.9 (32.9)
Alert responses (N)
 Any response to alert 170
 Order placed within an alert 143
 Contraindication reported 128
 ACE inhibitor ordered 15
 Basic Metabolic Panel ordered 9

Based on a 5.3% absolute increase in ACE inhibitor or ARB discharge prescriptions in the interruptive versus non-interruptive groups, we estimated that an additional 24.8 individuals were discharged on an ACE inhibitor or ARB in the interruptive group because of the alert. Given that the alert was 10,034 times, the number of triggers needed to change quality was 405 (95% CI 203–2,157,849); in other words, we estimate that the interruptive alert had to be fired 405 times for a new ACE inhibitor or ARB to be prescribed. Among the subgroup of patients discharged with a secondary diagnosis of heart failure, the number of triggers needed to changed quality was 366 (95% CI 185–17,553).

Discussion

We found that a CDS tool improved evidence-based care for heart failure when implemented as an interruptive alert as compared to a non-interruptive alert. Specifically, the interruptive alert was associated with a 5% absolute increase in discharge utilization of an ACE inhibitor or ARB. As compared to the non-interruptive alert, the interruptive alert also led to higher rates of ordering an ACE inhibitor during hospitalization and improved documentation of contraindications to this evidence-based therapy.

We observed improvements in utilization of ACE inhibitors or ARBs at discharge primarily among hospitalizations with a secondary diagnosis of heart failure. Similar to prior work, we found that utilization of these evidence-based therapies was higher among patients discharged with a principal versus secondary diagnosis of heart failure.10 As the intervention did not significantly improve care for patients with a principal diagnosis of heart failure, it did have a small effect in decreasing the quality gap between patients discharged with a principal versus secondary diagnosis of heart failure. Our ability to improve care for patients with a principal diagnosis of heart failure was likely limited for two reasons. First, another, more resource intensive hospital intervention was in place: a heart failure nurse reviewed the care delivered for every patient hospitalized principally for heart failure and made recommendations to start ACE inhibitor or ARB therapy as appropriate. Second, given the high baseline rates of utilization of ACE inhibitors or ARBs among patients with a principal diagnosis of heart failure, there was likely a ceiling effect on use of these evidence based therapies as compared to patients with a secondary diagnosis of heart failure.

The benefits of the CDS implemented as an interruptive alert must be considered in the context of its high alert burden: the interruptive alert was triggered an average of over 25 times per hospitalization with a response rate of less than 2%. Additionally, the number of triggers needed to change quality, an estimate we developed to be comparable to the number needed to treat (NNT) in clinical trials, was 405, i.e. the alert needed to be triggered 405 times in order for one additional patient to be discharge on an ACE inhibitor or ARB. In the context of widespread use of EHRs,22 CDS is often cited as a target to improve guideline directed cardiovascular care.4,23 EHR-based CDS has appeal as it is scalable within, if not across, institutions24 and relatively inexpensive to implement. However, CDS does result in important non-monetary costs, including increasing provider time and cognitive burden.25 In the current study, we had a small success with a well-designed interruptive CDS, but also felt that 405 alerts per quality outcome were too high for an additional ACE inhibitor prescription in the inpatient setting. As a result, we have decided to stop the CDS from triggering in the EHR.

A number of systematic reviews have examined factors associated with improved effectiveness from CDS, although results have been somewhat mixed.2628 For instances most closely related to factors in our study, reviews led by both Van de Velde and Kawamoto suggest that automatic provision of CDS improved CDS effectiveness, although a review by Roshanov and colleagues found no such association; conversely, Roshanov found an association between required documentation related to CDS dismissal and CDS effectiveness while Van de Velde did not.2628 Nonetheless, both versions of our CDS would be classified as automatic provision with no required documentation for dismissal. These reviews did not examine the comparative effectiveness of interruptive and non-interruptive alerts, features that have become increasingly available in contemporary EHRs. Indeed, one recent systematic review concluded that more studies were needed to determine what implementation factors lead to improved CDS effectiveness.28 Our study is unique in that it compares CDS approaches relevant for current practice and estimates the benefits relative to the burden of the intervention.

The study should be interpreted in the context of its limitations. First, the CDS tool was implemented at a single hospital system and our results may not be generalizable. Second, our definition of heart failure for inclusion was not based on discharge diagnosis, resulting in some hospitalizations that did not carry a heart failure diagnosis. Nonetheless, our definition of heart failure was based on reduced ejection, which is consistent with guidelines and indications for ACE inhibitor or ARB therapy.4 Third, the alert could be triggered at any time during hospitalization, while our outcome measurement was based on discharge utilization. We did this as discharge utilization is a standard quality metric6 and there is evidence that initiating medications in hospital has long term benefits for outpatient management of heart failure.19,29,30 However, this resulted in our PPV being lower than expected due to time-varying factors for determination of appropriateness for ACE inhibitor therapy. For instance, we found one example in which the CDS appropriately fired once but at discharge an ACE inhibitor was not indicated due to an increased potassium level. Fourth, we randomized at the patient-level, which could lead to bias from having the same provider care for patients in both treatment groups. However, patient-level randomization was necessary. We were concerned that provider-level randomization in a hospital setting would lead to crossover contamination as multiple providers care for the same patient and there is significant cross-coverage and handoffs.31 Patient-level randomization should have a conservative bias, which is minimized by the fact that providers typically respond to recommendations when presented by a CDS at the point of care but not for future patients when no CDS is present.32 Furthermore, we did not cluster by provider in our analysis because of concerns of multiple providers and attending handoffs.

In conclusion, implementing a CDS tool designed to have high usability and specificity as an interruptive alert increased discharge utilization of ACE inhibitor therapy for patients with heart failure. As compared to implementing the CDS as a non-interruptive alert, the interruptive alert also led to better documentation of contraindications to evidence based therapy. However, these benefits occurred at the tradeoff of high burden of the alerts. Although CDS shows promise for care improvement and scalability, its perils suggest that novel additional or supplementary approaches are also needed for broad implementation of guideline based therapy.

Supplementary Material

Appendix Table

Funding:

This study was supported by the Agency for Healthcare Research and Quality (AHRQ) grant K08HS23683.

References

  • 1.Blecker S, Paul M, Taksler G, Ogedegbe G, Katz S. Heart failure-associated hospitalizations in the United States. J Am Coll Cardiol. 2013;61(12):1259–1267. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Loehr LR, Rosamond WD, Chang PP, Folsom AR, Chambless LE. Heart failure incidence and survival (from the Atherosclerosis Risk in Communities study). The American journal of cardiology. 2008;101(7):1016–1022. [DOI] [PubMed] [Google Scholar]
  • 3.Garg R, Yusuf S. Overview of randomized trials of angiotensin-converting enzyme inhibitors on mortality and morbidity in patients with heart failure. Collaborative Group on ACE Inhibitor Trials. JAMA. 1995;273(18):1450–1456. [PubMed] [Google Scholar]
  • 4.Yancy CW, Jessup M, Bozkurt B, et al. 2013 ACCF/AHA Guideline for the Management of Heart Failure: A Report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol. 2013;62(16):e147–239. [DOI] [PubMed] [Google Scholar]
  • 5.Adams KF Jr., Fonarow GC, Emerman CL, et al. Characteristics and outcomes of patients hospitalized for heart failure in the United States: rationale, design, and preliminary observations from the first 100,000 cases in the Acute Decompensated Heart Failure National Registry (ADHERE). American heart journal. 2005;149(2):209–216. [DOI] [PubMed] [Google Scholar]
  • 6.Bonow RO, Ganiats TG, Beam CT, et al. ACCF/AHA/AMA-PCPI 2011 Performance Measures for Adults With Heart Failure: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Performance Measures and the American Medical Association-Physician Consortium for Performance Improvement. Circulation. 2012;125(19):2382–2401. [DOI] [PubMed] [Google Scholar]
  • 7.Fonarow GC, Gheorghiade M, Abraham WT. Importance of in-hospital initiation of evidence-based medical therapies for heart failure-a review. The American journal of cardiology. 2004;94(9):1155–1160. [DOI] [PubMed] [Google Scholar]
  • 8.Qian Q, Manning DM, Ou N, et al. ACEi/ARB for systolic heart failure: closing the quality gap with a sustainable intervention at an academic medical center. Journal of hospital medicine : an official publication of the Society of Hospital Medicine. 2011;6(3):156–160. [DOI] [PubMed] [Google Scholar]
  • 9.Riggio JM, Sorokin R, Moxey ED, Mather P, Gould S, Kane GC. Effectiveness of a clinical-decision-support system in improving compliance with cardiac-care quality measures and supporting resident training. Academic medicine : journal of the Association of American Medical Colleges. 2009;84(12):1719–1726. [DOI] [PubMed] [Google Scholar]
  • 10.Blecker S, Agarwal SK, Chang PP, et al. Quality of care for heart failure patients hospitalized for any cause. J Am Coll Cardiol. 2014;63(2):123–130. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Backman R, Bayliss S, Moore D, Litchfield I. Clinical reminder alert fatigue in healthcare: a systematic literature review protocol using qualitative evidence. Syst Rev. 2017;6(1):255. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Nanji KC, Seger DL, Slight SP, et al. Medication-related clinical decision support alert overrides in inpatients. Journal of the American Medical Informatics Association : JAMIA. 2017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Payne TH, Hines LE, Chan RC, et al. Recommendations to improve the usability of drug-drug interaction clinical decision support alerts. Journal of the American Medical Informatics Association : JAMIA. 2015;22(6):1243–1250. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Horsky J, Phansalkar S, Desai A, Bell D, Middleton B. Design of decision support interventions for medication prescribing. Int J Med Inform. 2013;82(6):492–503. [DOI] [PubMed] [Google Scholar]
  • 15.Phansalkar S, van der Sijs H, Tucker AD, et al. Drug-drug interactions that should be non-interruptive in order to reduce alert fatigue in electronic health records. Journal of the American Medical Informatics Association : JAMIA. 2013;20(3):489–493. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Scheepers-Hoeks AM, Grouls RJ, Neef C, Ackerman EW, Korsten EH. Physicians’ responses to clinical decision support on an intensive care unit--comparison of four different alerting methods. Artif Intell Med. 2013;59(1):33–38. [DOI] [PubMed] [Google Scholar]
  • 17.Strom BL, Schinnar R, Bilker W, Hennessy S, Leonard CE, Pifer E. Randomized clinical trial of a customized electronic alert requiring an affirmative response compared to a control group receiving a commercial passive CPOE alert: NSAID--warfarin co-prescribing as a test case. Journal of the American Medical Informatics Association : JAMIA. 2010;17(4):411–415. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.van Wyk JT, van Wijk MA, Sturkenboom MC, Mosseveld M, Moorman PW, van der Lei J. Electronic alerts versus on-demand decision support to improve dyslipidemia treatment: a cluster randomized controlled trial. Circulation. 2008;117(3):371–378. [DOI] [PubMed] [Google Scholar]
  • 19.Blecker S, Meisel T, Dickson VV, Shelley D, Horwitz LI. “We’re Almost Guests in Their Clinical Care”: Inpatient Provider Attitudes Toward Chronic Disease Management. Journal of hospital medicine : an official publication of the Society of Hospital Medicine. 2017;12(3):162–167. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Blecker S, Pandya RK, Stork S, Mann DM, Austrian JS. “The only advantage is it forces you to click ‘dismiss’”: Usability testing for interruptive versus non-interruptive clinical decision support. Journal of General Internal Medicine. 2018; S83. [Google Scholar]
  • 21.Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Administration and policy in mental health. 2011;38(2):65–76. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Adler-Milstein J, Holmgren AJ, Kralovec P, Worzala C, Searcy T, Patel V. Electronic health record adoption in US hospitals: the emergence of a digital “advanced use” divide. Journal of the American Medical Informatics Association : JAMIA. 2017;24(6):1142–1148. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Lobelo F, Rohm Young D, Sallis R, et al. Routine Assessment and Promotion of Physical Activity in Healthcare Settings: A Scientific Statement From the American Heart Association. Circulation. 2018;137(18):e495–e522. [DOI] [PubMed] [Google Scholar]
  • 24.Kawamoto K, Del Fiol G, Lobach DF, Jenders RA. Standards for scalable clinical decision support: need, current and emerging standards, gaps, and proposal for progress. Open Med Inform J. 2010;4:235–244. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Ancker JS, Edwards A, Nosal S, et al. Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system. BMC medical informatics and decision making. 2017;17(1):36. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005;330(7494):765. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Roshanov PS, Fernandes N, Wilczynski JM, et al. Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials. BMJ. 2013;346:f657. [DOI] [PubMed] [Google Scholar]
  • 28.Van de Velde S, Heselmans A, Delvaux N, et al. A systematic review of trials evaluating success factors of interventions with computerised clinical decision support. Implementation science : IS. 2018;13(1):114. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Fonarow GC. Role of in-hospital initiation of carvedilol to improve treatment rates and clinical outcomes. The American journal of cardiology. 2004;93(9A):77B–81B. [DOI] [PubMed] [Google Scholar]
  • 30.Hamaguchi S, Kinugawa S, Tsuchihashi-Makaya M, et al. Spironolactone use at discharge was associated with improved survival in hospitalized patients with systolic heart failure. American heart journal. 2010;160(6):1156–1162. [DOI] [PubMed] [Google Scholar]
  • 31.Berner ES. Clinical decision support systems. Springer; 2007. [Google Scholar]
  • 32.Shelton JB, Ochotorena L, Bennett C, et al. Reducing PSA-Based Prostate Cancer Screening in Men Aged 75 Years and Older with the Use of Highly Specific Computerized Clinical Decision Support. J Gen Intern Med. 2015;30(8):1133–1139. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Appendix Table

RESOURCES