Deimplementation of Routine Chest X-rays in Adult Intensive Care Units

Abstract

BACKGROUND

Choosing Wisely^® is a national initiative to deimplement or reduce low-value care. However, there is limited evidence on the effectiveness of strategies to influence ordering patterns.

OBJECTIVE

We aimed to describe the effectiveness of an intervention to reduce daily chest X-ray (CXR) ordering in two intensive care units (ICUs) and evaluate deimplementation strategies.

DESIGN

We conducted a prospective, nonrandomized study with control data from a historical period. Qualitative evaluation was guided by the Consolidated Framework for Implementation Research.

SETTING

The study was performed in the medical intensive care unit (MICU) and cardiovascular intensive care unit (CVICU) of an academic medical center in the United States from October 2015 to June 2016.

PARTICIPANTS

The initiative included the staff of the MICU and CVICU (physicians, surgeons, nurse practitioners, fellows, residents, medical students, and X-ray technologists).

INTERVENTION COMPONENTS

We utilized provider education, peer champions, and weekly data feedback of CXR ordering rates.

MEASUREMENTS

We analyzed the CXR ordering rates and factors facilitating or inhibiting deimplementation.

RESULTS

Segmented linear time-series analysis suggested a small but statistically significant decrease in CXR ordering rates in the CVICU (P < .001) but not in the MICU. Facilitators of deimplementation, which were more prominent in the CVICU, included engagement of peer champions, stable staffing, and regular data feedback. Barriers included the need to establish goal CXR ordering rates, insufficient intervention visibility, and waning investment among medical residents in the MICU due to frequent rotation and competing priorities.

CONCLUSIONS

Intervention modestly reduced CXRs ordered in one of two ICUs evaluated. Understanding why adoption differed between the two units may inform future interventions to deimplement low-value diagnostic tests.

Despite increased awareness of Choosing Wisely (CW)^® recommendations to reduce low-value care,1 there is limited published data about strategies to implement these guidelines or evidence that they have influenced ordering patterns or reduced healthcare spending.2-6 Implementation science seeks to accelerate the translation of evidence-based interventions into clinical practice and the deimplementation of low-value care.7-9 Based on established principles of implementation science, we used a prospective, nonrandomized study design to assess a CW intervention to reduce chest X-ray (CXR) ordering in adult intensive care units (ICUs).10

In ICUs, CXR ordering strategies may be routine (daily) or on-demand (with clinical indication). The former strategy’s principal advantage is the potential to detect life-threatening situations that may otherwise escape diagnosis.11 Disadvantages include cost, radiation exposure, patient inconvenience, false-positive workups, and low diagnostic and therapeutic value.12,13 On-demand strategies may safely reduce CXR ordering by 32% to 45%.11-17 Based on this evidence, the Critical Care Societies Collaborative and the American College of Radiology have recommended on-demand CXR ordering.18,19 Here, we describe the effectiveness of an intervention to reduce CXR ordering in two ICUs while evaluating the deimplementation strategies using a validated framework.

METHODS

Setting and Design

Vanderbilt University Medical Center (VUMC) is an academic referral center in Nashville, Tennessee. The cardiovascular ICU (CVICU) has 27 beds and the medical ICU (MICU) has 34 beds. Acute care nurse practitioners (ACNPs) and two critical care physicians staff the CVICU; cardiology fellows, anesthesia critical care fellows, and transplant and cardiac surgeons are also active in patient care. The MICU is staffed by two critical care physicians who supervise one team of ACNPs and two teams of medical residents who rotate through the unit every two weeks. Each MICU team is assigned a fellow in pulmonary and critical care.

We conducted a prospective, nonrandomized study in these units from October 2015 to June 2016. The VUMC Institutional Review Board approved the intervention as a quality improvement (QI) activity, waiving the requirement for informed consent.

Intervention

Following the top CW recommendation of the Critical Care Societies Collaborative—“Don’t order diagnostic tests at regular intervals (such as every day), but rather in response to specific clinical questions.”19—the VUMC resident-led CW Steering Committee designed a multifaceted approach to reduce ordering of routine CXRs in ICUs. The intervention included a didactic session on CW and proper CXR ordering practices, peer champions, data audits, and feedback to providers through weekly e-mails (see Supplemental Materials, 1 – Resident Presentation and 2 – CXR Flyer). 20

In September 2015, CVICU and MICU teams received a didactic session highlighting CW, current CXR ordering rates, and the plan for reducing CXR ordering. On October 5, 2015, teams began receiving weekly e-mails with ordering rates defined as CXRs ordered per patient per day and a brief rationale for reducing unnecessary CXRs. To encourage friendly competition, we provided weekly rates to the MICU teams, allowing for transparent benchmarking against one another. A similar competition strategy was not used in the CVICU due to the lack of multiple teams.

In the CVICU, two ACNPs volunteered as peer champions. These champions coordinated data feedback and advocated for the intervention among their colleagues. In the MICU, three internal medicine residents volunteered as peer champions and fulfilled similar roles.

To facilitate deimplementation, we conducted two Plan-Do-Study-Act (PDSA) cycles, the first from November to mid-December 2015 and the second from mid-December 2015 to mid-January 2016. During these cycles, we tailored our deimplementation strategy based on barriers identified by the peer champions and ICU leaders (described in the Qualitative Results section). Peer champions and the CW Steering Committee generated potential solutions by conversing with stakeholders and using the Expert Recommendations for Implementing Change (ERIC).20 Interventions included disseminating promotional flyers, holding meetings with stakeholders, and providing monthly CXR ordering rates. After the PDSA cycles, we continued reexamining the deimplementation efforts by reviewing ordering rates and soliciting feedback from ICU leaders and peer champions. However, no significant changes to the intervention were made during this time.

Quantitative Evaluation

We extracted data from VUMC’s Enterprise Data Warehouse during the intervention period (October 5, 2015 to May 24, 2016) and a historical control period (October 1, 2014 to October 4, 2015). Within each ICU, descriptive statistics were used to compare patient cohorts in the baseline and intervention periods by age, sex, and race.

The primary outcome was CXRs ordered per patient per day by hospital unit (CVICU or MICU). The baseline period included all data between October 1, 2014 and September 15, 2015. To account for priming of providers from didactic education, we allowed a washout period from September 16, 2015 to October 4, 2015. As a preliminary analysis, we compared CXR rates in the baseline and intervention periods using Wilcoxon rank-sum tests. We then conducted interrupted time-series analyses with segmented linear regression to assess differences in linear trends in CXR rates over the two periods. To account for different staffing models in the MICU, we stratified the impact of the intervention by team—medical resident (physician) or ACNP. R version 3.4.0 was used for statistical analysis.21

Qualitative Evaluation

Our qualitative evaluation consisted of embedded observation and semistructured interviews with stakeholders. The qualitative portion was guided by the Consolidated Framework for Implementation Research (CFIR), a widely used framework for design and evaluation of improvement initiatives that helped us to determine major facilitators and barriers to implementation.22,23

Embedded Observation

From November 2015 to January 2016, we observed morning rounds in the CVICU and MICU one to two times weekly to understand factors facilitating and inhibiting uptake of the intervention. Observations were recorded and organized using a CFIR-based template and directed toward understanding interactions among team members (eg, the decision-making process hierarchy), team workflows and decision-making processes, process of ordering CXRs, and providers’ knowledge and perceptions of the CXR intervention (see Supplemental Material, 3 – CFIR Table).22,23 After rounds, ICU team members were invited to share suggestions for improving the intervention. All observations occurred during and shortly following morning rounds when the vast majority of routine CXRs are ordered; we did not evaluate night or evening workflows. In the spirit of continuous improvement, we evaluated data in real-time.

Semistructured Interviews

Based on the direct observations, we developed semistructured interview questions to further evaluate provider perspectives (eg, “Do you believe ICU patients need a daily CXR?”) and constructs aligning with CFIR (eg, “intervention source— internally vs externally developed;” see Supplemental Material, 4 – Interview Questions).

Stakeholders from both ICUs were recruited through e-mail and in-person requests to participate in semistructured interviews. In the CVICU, we interviewed critical care physicians, anesthesia critical care fellows, and ACNPs. In the MICU, we interviewed medical students, interns, residents, critical care fellows, attending intensivist physicians, and ACNPs. We also interviewed X-ray technologists who routinely perform portable films in the units.

RESULTS

Quantitative Results

We analyzed CXR ordering data from a period of 86 weeks, comprising 50 weeks of baseline data, three weeks of washout period, and 33 weeks following the introduction of the intervention. In both ICUs, patient characteristics were similar in the baseline and intervention periods (Table 1).

TABLE 1.

Patient Characteristics During the Baseline and Intervention Periods in the CVICU and MICU from October 1, 2014 to May 24, 2016

	CVICU			MICU
	Baseline n = 1,180	Intervention n = 847	P Value	Baseline n = 2,378	Intervention n = 1,524	P Value
Age in years (median)	63.0	62.0	.9171	59.0	58.0	.0681
Days in ICU (median)	3.0	3.0	.0831	2.0	2.0	.1711
Sex: male	66.2%	64.6%	.4532	52.4%	53.1%	.6942
Race			.0012			.3712
White	82.9%	84.1%		76.7%	76.2%
Black	10.2%	6.0%		16.6%	15.8%
Other	1.3%	2.4%		1.6%	2.2%
Unknown	5.7%	7.6%		5.0%	5.8%
In-hospital mortality	6.6%	4.4%	.0312	19.4%	20.9%	.2732

Note: Tests used include the 1Wilcoxon rank-sum test and 2Pearson chi-square test

Abbreviations: CVICU, cardiovascular intensive care unit; MICU, medical intensive care unit.

Cardiovascular Intensive Care Unit

The median baseline CXR ordering rate in the CVICU was 1.16 CXRs per patient per day, with interquartile range (IQR) 1.06-1.28. During the intervention period, the rate dropped to 1.07 (IQR 0.94-1.21; P < .001; Table 2). The time-series analysis suggested an essentially flat trend during the baseline period, followed by a small but significant drop in ordering rates during the intervention period (P < .001; Table 3 and Figure 1). Ordering rates appeared to increase slightly over the course of the intervention period, but this slight upward trend was not significantly different from the flat trend seen during the baseline period.

TABLE 2.

Average Number of Chest X-rays per Patient per Day During the Baseline and Intervention Periods

	Baseline				Intervention				P Value
	25th Percentile	Median	75th Percentile	Mean ± SD	25th Percentile	Median	75th Percentile	Mean ± SD
CVICU	1.06	1.16	1.28	1.17 ± 0.18	0.94	1.07	1.21	1.08 ± 0.22	<.001
MICU: MD Care Team	0.48	0.60	0.73	0.61 ± 0.18	0.50	0.62	0.74	0.63 ± 0.19	.353
MICU: ACNP Care Team	0.21	0.39	0.57	0.41 ± 0.29	0.20	0.38	0.56	0.39 ± 0.26	.572

Test used: Wilcoxon rank-sum test.

Abbreviations: ACNP, acute care nurse practitioner; CVICU, cardiovascular intensive care unit; MICU, medical intensive care unit; MD, medical doctor; SD Standard Deviation.

TABLE 3.

Results of Segmented Linear Regression Analyzing the Impact of Intervention on Chest X-ray Ordering Rates

	CVICU			MICU: MD Team			MICU: ACNP Team
	Est.	95% CI	P Value	Est.	95% CI	P Value	Est.	95% CI	P Value
Intercept in baseline period	1.17	1.13, 1.20	<.001	0.65	0.61, 0.68	<.001	0.46	0.40, 0.52	<.001
Baseline trenda	0.00	0.00, 0.00	.653	0.00	0.00, 0.00	.011	0.00	0.00, 0.00	.022
Level change associated with intervention period	–0.13	–0.19, –0.06	.001	0.07	0.02, 0.12	.010	0.08	0.00, 0.15	.055
Trend change associated with intervention period	0.00	0.00, 0.00	.479	0.00	0.00, 0.00	.557	0.00	0.00, 0.00	.491

^aChange in rate of x-rays per patient per day.

Abbreviations: ACNP, acute care nurse practitioner; CVICU, cardiovascular intensive care unit; MICU, medical intensive care unit; MD, medical doctor.

FIG 1.

Model-based estimates with 95% confidence intervals: chest x-rays per patient per day in the cardiovascular intensive care unit from October 2014 to May 2016.

Medical Intensive Care Unit

For both physician and ACPN teams, the median baseline CXR ordering rates in the MICU were much lower than the baseline rate in the CVICU (Table 2). For the MICU physician care team, the baseline CXR ordering rate was 0.60 CXRs per patient per day (IQR 0.48-0.73). For the ACNP team, the median rate was 0.39 CXRs per patient per day (IQR 0.21-0.57). Both rates stayed approximately the same during the intervention period (Table 2). The time-series analysis suggested a statistically significant but very slight downward trend in CXR ordering rates during the baseline period, in the physician (P = .011) and ACNP (P = .022) teams (Table 3, Figure 2). Under this model, a small increase in CXR ordering initially occurred during the intervention period for both physician and ACNP teams (P = .010 and P = .055, respectively), after which the rates declined slightly. Trends in ordering rates during the intervention period were not significantly different from the slight downward trends seen during the baseline period.

FIG 2.

Model-based estimates with 95% confidence intervals: chest x-rays per patient per day in the medical intensive care unit from October 2014 to May 2016.

Qualitative Results

We identified 25 of 39 CFIR constructs as relevant to the initiative (see Supplemental Materials, 3 – CFIR Table.) We determined the major facilitators of deimplementation to be peer champion discussions about CXR ordering on rounds and weekly data feedback, particularly if accompanied by in-person follow-up.

Major differences between the units pertained to the “inner setting” domain. Compared with the CVICU, which is staffed by a stable group of ACNPs, two of the three MICU teams are staffed by resident physicians who rotate on and off service. CVICU providers and ACNPs in the MICU reported significant investment in the CXR and other QI interventions. Conversely, resident physicians, who complete two- to four-week MICU rotations, reported less investment as well as greater fatigue and competing priorities. Some MICU residents began ignoring weekly feedback, citing “e-mail fatigue” and the lack of in-person follow-up or didactic sessions associated with the reports.

We also noted differences in CXR ordering rationales and decisions between the units. Generally, residents in the MICU and ACNPs in the CVICU made decisions to order CXRs. However, decisions were influenced by the expectations of attending physicians. While CVICU providers tended to order CXRs reflexively as part of morning labs, MICU providers—in particular, ACNPs who had been trained on indications for proper CXR ordering—generally ordered CXRs for specific indications (eg, worsening respiratory status). Of note, MICU ACNPs reported the use of bedside ultrasound as an alternate imaging modality and a reason for their higher threshold to order CXRs.

Deimplementation barriers in both units included the need to identify goal CXR ordering rates and the intervention’s limited visibility. To address these barriers, we conducted PDSA cycles and used the CFIR and ERIC to generate potential solutions.24 We established a goal of a 20% absolute reduction in the CVICU, added monthly CXR rates to weekly e-mail reports to better account for variations in patient populations and ordering practices, and circulated materials to promote on-demand CXR ordering. Promotional materials contained guidelines on CXR ordering and five “Frequently Held Misconceptions” about ordering practices with succinct, evidence-based explanations (see Supplemental Material, 2 – CXR Flyer).

Approximately four months after the start of intervention, some CVICU physicians became concerned that on-demand CXR ordering might be inappropriate for high-risk surgical patients, including those who are undergoing or have undergone heart transplants, lung transplants, or left-ventricular assist device placement. This concern arose following two adverse outcomes, which were not attributed to the CXR initiative, but which heightened concerns about patient safety. A rise in CXR ordering then occurred, and CVICU providers requested that we perform an analysis of these high-risk groups. While segmented linear regression in this subgroup suggested that average daily CXR ordering rates did decrease among the high-risk group at the start of the intervention period (P = .001), the difference between the rates in the two periods was not significant using the Wilcoxon rank-sum test. Exclusion of these patients from the main analysis did not alter the interpretation of the findings reported above for the CVICU.

DISCUSSION

A deimplementation intervention using provider education, peer champions, and data feedback was associated with fewer CXRs in the CVICU (P < .001) but not in the MICU. The CFIR-guided qualitative analysis was valuable for evaluating our deimplementation strategy and for identifying differences between the two ICUs.

Relatively few studies have demonstrated effective interventions that address CW recommendations.25-28 However, three population-level analyses of insurance claims show mixed results.3,4,29 Experts have thus proposed using implementation science to improve uptake of CW recommendations.2,3,7,8 Our study demonstrates the effectiveness of this approach. As expected, providers largely endorsed an on-demand CXR ordering strategy. Using the CFIR, however, we discovered barriers (eg, concern that data feedback did not reflect variations in patients’ needs). Using methods from implementation science allowed us to diagnose and tailor our approaches.

Our qualitative evaluation suggested that the intervention was ineffective mostly due to CFIR’s “inner setting” constructs, including resident fatigue, competing priorities, and decreased investment in QI projects because of the rotating nature of providers in training. Baseline CXR ordering rates in the MICU were also considerably lower than in the CVICU. We observed that CVICU providers ordered many CXRs following the placement of lines or tubes and that ACNPs in the MICU had received education on appropriate CXR ordering practices and had access to an alternate imaging modality in ultrasound. These factors may partially explain the difference in baseline rates.

As noted in a study of cardiac stress testing guidelines, the existence of high-value care recommendations does not mean overuse.30 Indeed, the lack of significant CXR over-ordering in the MICU highlights the importance of baseline measurement and partnering with information technology departments to create the best possible data feedback systems.30-32 Our ex perience shows that these systems should provide sufficient pre-implementation data (ideally >1 year), such that teams selecting QI projects can ensure that a project is a good use of institutional resources and change capital.

To inform future work, we informally assessed program costs and savings. We estimate the initiative cost $1,600, including $1,000 for curriculum development and teaching time, $300 for educational materials, and $300 for CXR tracking dashboard development. Hospital charges and reimbursements for CXR vary widely.33 We calculated savings using a range of rates, from a conservative $23 (the Medicare reimbursement rate for single-view CXR, CPT code 71010, global fee) to $50 (an approximate blended reimbursement rate across payers).34,35 In the CVICU, we estimate that 51 CXRs were avoided each month, saving $1,173-$2,550 per month or $9,384-$20,400 over eight months of follow-up. Annualizing these figures, we estimate net savings of $12,476-$29,000 in the first year in a 27-bed ICU. Costs to continue the program include education of new employees, booster training, and dashboard maintenance for an estimated annual cost of $1,000. It is difficult to estimate effectiveness over time, but if we conservatively assume that 30 CXRs were avoided each month, then the projected savings would be $8,280-$18,000 per year or an annual net savings of $7,280-$17,000 in the ICU. Although these amounts are modest, providing trainees with experiential learning opportunities in high-value care is valuable in its own right, meets curricular goals, may result in spill-over effects to other diagnostic and therapeutic decisions, and may influence long-term practice patterns. Institutional decisions to pursue projects such as this should take into account these potential benefits.

This evaluation is not without limitations. First, the study was conducted in a single tertiary-care hospital, potentially limiting its generalizability.36 Second, the study design lacked a concurrent control group, and observed outcomes may have been influenced by broader CXR utilization trends, increased awareness of low-value care generally or from previous CW projects at VUMC, seasonal effects, or the Hawthorne effect. Third, the study outcome was all CXRs ordered, rather than CXRs that were unnecessary or not clinically indicated. We chose all CXRs because it was more pragmatic, did not require clinical case review, and could be incorporated promptly into dashboards, enabling timely performance feedback. Other performance measures have taken a similar tack (eg, tracking all-cause re-admissions rather than preventable readmissions). Given this approach, we did not track clinical indications for CXRs (eg, central line placement). Fourth, although we compared resident and APRN orders, we did not collect data on other provider characteristics such as years in/out of training or board certification status. These considerations should be addressed in future research.

Finally, the increase in CVICU CXR ordering at the end of the intervention period, which occurred following two adverse events, raises concerns about sustainability. While unrelated to CXR orders, the events resulted in increased ordering of diagnostic tests and showed the difficulty of deimplementation in ICUs. Indeed, some CVICU providers argued that on-demand CXR ordering represented minimal potential cost savings and had not been studied among heart and lung transplant patients. Subsequently, Tonna et al. have shown that on-demand CXR ordering can be safely implemented among such patients.37 Also similar to our study, Tonna et al. observed an initial decrease in CXR ordering, followed by a gradual increase toward baseline ordering rates. These findings highlight the need for sustained awareness and interventions and for the careful selection of high-value projects.

In conclusion, our study shows that a deimplementation intervention based on CW recommendations can reduce CXR ordering and that ongoing evaluation of contextual factors provides insights for both real-time modifications of current interventions and the design of future interventions. We found that messaging about reducing unnecessary tests works well when discussions are framed at the unit level but may be counterproductive if used to question individual ordering decisions.38 Additional lessons learned include the value of participation on rounds to build trust among stakeholders, the utility of monthly rather than weekly statistics for feedback, stakeholder input and peer champions, and differences in approach with physician and ACNP audiences.