Skip to main content
PMC home page
JAMA Network logoLink to JAMA Network
. 2019 Mar 25;179(5):648–657. doi: 10.1001/jamainternmed.2018.8358

Evaluation of an Intervention to Reduce Low-Value Preoperative Care for Patients Undergoing Cataract Surgery at a Safety-Net Health System

John N Mafi 1,2,✉, Patricia Godoy-Travieso 3, Eric Wei 3, Malvin Anders 4, Rodolfo Amaya 5, Carmen A Carrillo 6, Jesse L Berry 4,7, Laura Sarff 3, Lauren Daskivich 8, Sitaram Vangala 1, Joseph Ladapo 1, Emmett Keeler 2, Cheryl L Damberg 2, Catherine Sarkisian 6,9
PMCID: PMC6503569  PMID: 30907922

This study assesses the utilization and costs of a multipronged quality improvement initiative to reduce low-value preoperative utilization for patients undergoing cataract surgery at 2 safety-net health systems in California.

Key Points

Question

Can a multipronged quality improvement initiative reduce low-value preoperative care for patients undergoing cataract surgery and save costs at a large safety-net health system?

Findings

In this study at 2 academic safety-net medical centers in California, the quality improvement initiative was associated with reduced preoperative testing compared with the control health system. Also, 3-year projections estimated a modest amount of cost savings associated with the initiative; simulating fee-for-service health system perspective estimated losses and a societal perspective estimated savings.

Meaning

These findings suggest that reducing low-value care is associated with cost savings for financially capitated health systems and society but also with losses for fee-for-service health systems, highlighting a potential barrier to eliminating low-value care.

Abstract

Importance

Preoperative testing for cataract surgery epitomizes low-value care and still occurs frequently, even at one of the nation’s largest safety-net health systems.

Objective

To evaluate a multipronged intervention to reduce low-value preoperative care for patients undergoing cataract surgery and analyze costs from various fiscal perspectives.

Design, Setting, and Participants

This study took place at 2 academic safety-net medical centers, Los Angeles County and University of Southern California (LAC-USC) (intervention, n = 469) and Harbor–UCLA (University of California, Los Angeles) (control, n = 585), from April 13, 2015, through April 12, 2016, with 12 additional months (April 13, 2016, through April 13, 2017) to assess sustainability (intervention, n = 1002; control, n = 511). To compare pre- and postintervention vs control group utilization and cost changes, logistic regression assessing time-by-group interactions was used.

Interventions

Using plan-do-study-act cycles, a quality improvement nurse reviewed medical records and engaged the anesthesiology and ophthalmology chiefs with data on overuse; all 3 educated staff and trainees on reducing routine preoperative care.

Main Outcomes and Measures

Percentage of patients undergoing cataract surgery with preoperative medical visits, chest x-rays, laboratory tests, and electrocardiograms. Costs were estimated from LAC-USC's financially capitated perspective, and costs were simulated from fee-for-service (FFS) health system and societal perspectives.

Results

Of 1054 patients, 546 (51.8%) were female (mean [SD] age, 60.6 [11.1] years). Preoperative visits decreased from 93% to 24% in the intervention group and increased from 89% to 91% in the control group (between-group difference, −71%; 95% CI, –80% to –62%). Chest x-rays decreased from 90% to 24% in the intervention group and increased from 75% to 83% in the control group (between-group difference, −75%; 95% CI, –86% to –65%). Laboratory tests decreased from 92% to 37% in the intervention group and decreased from 98% to 97% in the control group (between-group difference, −56%; 95% CI, –64% to –48%). Electrocardiograms decreased from 95% to 29% in the intervention group and increased from 86% to 94% in the control group (between-group difference, −74%; 95% CI, –83% to −65%). During 12-month follow-up, visits increased in the intervention group to 67%, but chest x-rays (12%), laboratory tests (28%), and electrocardiograms (11%) remained low (P < .001 for all time-group interactions in both periods). At LAC-USC, losses of $42 241 in year 1 were attributable to intervention costs, and 3-year projections estimated $67 241 in savings. In a simulation of a FFS health system at 3 years, $88 151 in losses were estimated, and for societal 3-year perspectives, $217 322 in savings were estimated.

Conclusions and Relevance

This intervention was associated with sustained reductions in low-value preoperative testing among patients undergoing cataract surgery and modest cost savings for the health system. The findings suggest that reducing low-value care may be associated with cost savings for financially capitated health systems and society but also with losses for FFS health systems, highlighting a potential barrier to eliminating low-value care.

Introduction

Of the $3.3 trillion spent annually on US health care, approximately 10% to 30% of spending consists of low-value care, which is patient care that provides no net benefit in specific clinical scenarios.1,2,3,4 As early as the 1960s, US policy makers have been trying to curb unnecessary national health care spending5; however, few initiatives have yielded substantial or durable savings. Health care costs continue to rise, and the epidemic of low-value care remains pervasive.6,7,8,9,10,11,12,13,14,15,16

Among low-risk surgical procedures, preoperative testing for cataract surgery is the quintessential example of low-value care. Large randomized clinical trials have demonstrated that routine preoperative testing offers no net clinical benefit.17,18,19 However, despite widespread knowledge of the futility of preoperative testing for decades, most Medicare beneficiaries continue to receive routine preoperative testing for cataract surgery.20 Previous initiatives to reduce low-value testing for cataract surgery have yielded, at best, modest results.21 Because health systems typically receive more revenue when they deliver more care, policy experts often implicate perverse financial incentives as the primary driver of unnecessary care.22 However, even at capitated health systems, such as the Los Angeles County Department of Health Services (LAC-DHS), one of the largest safety-net health systems in the United States, more than 90% of patients continue to receive low-value preoperative testing for cataract surgery. Such examples suggest that factors beyond finances contribute to the persistence of low-value preoperative testing, such as lack of knowledge, habit, and non–evidence-based protocols.23,24,25,26,27,28 Given the multitude of factors associated with low-value care delivery, addressing this problem will require multipronged solutions. A meta-analysis of interventions to reduce low-value care found that multicomponent initiatives are more likely to succeed than are single component efforts.29

We evaluated a multicomponent quality improvement (QI) initiative to reduce low-value preoperative care for patients undergoing cataract surgery at LAC-DHS. To understand the financial consequences on the health system associated with reductions in the use of low-value care, we performed a cost analysis.30,31 We also extrapolated the findings of our analysis to estimate the costs from the perspective of a fee-for-service health system and society.

Methods

Study Design and Setting

In this study, we compared preoperative cataract surgery utilization at 2 major academic safety-net medical centers: Los Angeles County and University of Southern California (LAC-USC) Medical Center (intervention site) vs Harbor–UCLA (University of California, Los Angeles) Medical Center (control site) before and after the intervention. Although LAC-USC and Harbor-UCLA are both part of the larger LAC-DHS health system, they function as separate, independent medical centers with their own leadership and clinical structures. The study population and analysis protocol for this study was preregistered on ClinicalTrials.gov (NCT03253874). The UCLA and USC Health Sciences institutional review boards approved our evaluation of this intervention. Because this was an observational study of a health system quality improvement initiative, the UCLA and USC institutional review boards waived participant informed consent.

All of the practitioners in this study were salaried employees of LAC-DHS, and it is standard policy that all preoperative evaluations take place at a LAC-DHS site. Preoperative care was team based and similar at both sites. A licensed vocational nurse at LAC-USC and a nurse practitioner at Harbor-UCLA scheduled and performed most preoperative evaluations on behalf of the ophthalmology department. Ophthalmology residents at both sites would also perform some preoperative evaluations, but the residents ordered all preoperative tests for patients; the resident and/or licensed vocational nurse at LAC-USC and resident and/or nurse practitioner at Harbor-UCLA would follow up the test results. If patients had an active medical problem or if test results were abnormal, the ophthalmology resident would refer the patient for appropriate evaluation or follow-up as needed. In parallel, surgical coordinators consisting of a nurse practitioner and physician assistant at LAC-USC and 2 nurse practitioners at Harbor-UCLA would also follow up test results. If test results were normal, the surgical coordinators at both sites would proceed to schedule the cataract surgery.

Participants and Data

The study participants included all adult patients (≥18 years) undergoing cataract surgery at both sites between April 13, 2015, and April 12, 2016 (6 months before and after the intervention). We also evaluated an additional 12 months of data from April 13, 2016, through April 13, 2017, to assess whether the intervention had a sustained outcome. We obtained study data from the LAC-DHS electronic health record (EHR). Patients were identified using primary Current Procedural Terminology codes (66982, 66983, and 66984) based on previous research20 and those commonly used at LAC-USC and Harbor-UCLA for cataract surgery. In May 2015, LAC-USC underwent an EHR upgrade (3 months before the rollout of the intervention); as a result, 3 months of data within the 6-month preintervention period are missing. To evaluate the consequences of this missing data and to determine whether utilization trends were stable at both medical centers in the months preceding the EHR rollout, we performed a sensitivity analysis including an additional 3 months of utilization data using the previous EHR.

Intervention

The intervention that we evaluated consisted of 2 phases. The first phase was an implementation phase led by the QI, ophthalmology, and anesthesiology departments at LAC-USC without direct input from the evaluation team (P.G.-T., L.S., E.W., M.A., J.L.B., R.A., and L.D.). In the second phase, a team from UCLA and RAND Corporation led the evaluation independently from the implementation team (J.N.M., C.A.C., S.V., J.L., E.K., C.L.D., and C.S.) to maximize objectivity.

Figure 1 gives details on the intervention (further details on the intervention are given in the eAppendix in the Supplement). In the context of (1) the director of LAC-DHS emphasizing value improvement, (2) learning about the Choosing Wisely campaign, (3) hearing about the UCLA/LAC-DHS team’s pending American Board of Internal Medicine Foundation Choosing Wisely grant application, and (4) noting that the high volume of cataract surgeries had an inefficient preoperative process that could be streamlined to liberate resources, the chief quality officer of LAC-USC (L.S.) and the QI nurse (P.G.) decided to focus on low-value preoperative testing before cataract surgery. Before the intervention, the QI nurse from LAC-USC who led the intervention participated in a 10-month improvement advisor professional development course from the Institute for Healthcare Improvement.32 Applying QI principles and methods from the course (eg, data gathering, frontline clinician education and engagement, and plan-do-study-act cycles), the QI nurse (1) reviewed medical records of patients undergoing cataract surgery and extracted data about rates of low-value preoperative testing, (2) presented overuse data to anesthesiology and ophthalmology clinical chiefs to garner their support, (3) educated residents (with help from the clinical chiefs) to stop the protocol of routinely ordering tests, and (4) instructed the licensed vocational nurse to stop routinely scheduling preoperative medical visits. Next, a resident champion and the QI team emailed new preoperative guidelines to physicians, trainees, and staff that recommended eliminating routine preoperative visits and testing for cataract surgery by October 13, 2015. The anesthesiology and ophthalmology chiefs continued to reinforce the initiative with faculty, trainees, and staff. The QI nurse spent approximately 20% of her time leading the initiative from March 1, 2015, through March 1, 2016. Of importance, the QI team encouraged clinicians to use clinical judgment and continue to order tests when indicated (eg, blood glucose testing for a patient with diabetes or potassium testing for a patient with chronic kidney disease; both tests were also available to clinicians at the point of care on the day of surgery).

Figure 1. Multicomponent Intervention Timeline.

Figure 1.

Open in a new tab

Further details on the intervention are given in the eAppendix in the Supplement. LVN indicates licensed vocational nurse; PDSA, plan-do-study-act; QI, quality improvement.

Primary Measures of Low-Value Care

Our primary outcome measures were the percentage of patients undergoing cataract surgery with preoperative medical visits, chest-x-rays, laboratory tests (complete blood cell count, comprehensive metabolic panel, and hemoglobin A1C), and electrocardiograms within 80 days of surgery. Preoperative was defined as 80 days before surgery based on manual medical record review of the maximum time between relevant preoperative visits or testing and the procedure. We also performed sensitivity analyses using 30 and 60 days before surgery to define the preoperative period. Although many patients had 2 separate eye surgeries on different dates within the study window, we only included the first surgery for each patient. We used Current Procedural Terminology codes to find these surgeries, and a coauthor (C.A.C.) verified whether the main measures were ordered and delivered using manual medical record review. Many options exist for attributing testing to preoperative care using administrative data, such as the novel use of biometry codes33; however, because biometry was rarely coded at LAC-DHS, we used manual medical record review to determine whether the test was associated with the preoperative care.

Secondary measures included serious 30-day postoperative adverse events (eg, myocardial infarction, stroke, or hypoglycemia) as defined by previous research.18 Although we originally listed wait time between cataract diagnosis and surgery as a secondary measure in the study protocol, we did not complete this analysis because of unforeseen technical challenges in data collection. Because the intervention occurred during an upgrade to a new EHR, calculating accurate surgical wait times was impossible, because almost all initial cataract diagnoses artificially (and incorrectly) were labeled to have occurred on the day of or after the EHR rollout (May 29, 2015).

Finally, because trade-offs and unintended consequences are often underreported in QI research,23,31,34 we assessed additional secondary measures not listed in our protocol. First, because reducing low-value care may unintentionally increase staff utilization and raise costs (because of considerable intervention costs), we monitored clinical staffing and performed a cost analysis. Second, we evaluated the frequency of routine preoperative eye examinations as a control utilization metric. Because these preoperative examinations are almost always medically appropriate, reducing such examinations would be an unintended and harmful consequence. Third, we measured no-shows and cancellations of cataract surgeries, which may inadvertently increase when reducing routine preoperative medical visits (surgeries are often scheduled during these visits). For this latter measure, however, we only had data available at LAC-USC; therefore, we used no-shows and cancellations for both noncataract ophthalmologic surgeries and all other nonophthalmologic surgeries at LAC-USC for the 2 control metrics.

Statistical Analysis

We compared the percentage of patients scheduled for cataract surgery who received preoperative care at LAC-USC Medical Center (intervention group) with that of those at Harbor-UCLA Medical Center (control group) before and after the intervention. For assessing utilization differences, we estimated logistic regression models adjusted for patient age, sex, race/ethnicity, Charlson comorbidity index,35,36 time, and group, with the independent variable being the intervention group (LAC-USC) and the dependent variable being the primary measures (preoperative utilization). We assessed time-by-group interactions for each outcome measure. We considered 2-tailed P < .05 to be significant, and we considered odds ratios to be significant if the 95% CI for the odds ratio excluded one.

To evaluate costs, we performed a cost analysis from the perspective of LAC-USC Medical Center, which receives an annual capitated budget from the California Health and Human Services Agency. We calculated costs by subtracting savings from investments. Specifically, we included 1-time investment costs of LAC-USC paying for the QI nurse to attend the 10-month Institute for Healthcare Improvement course ($16 200 for tuition) plus 30% of her time in the course ($46 612.63 including publicly available salary and benefits37 for 10 months). We also added the costs of the same QI nurse spending 20% of her time after the course to work on the project for 1 year ($38 307.95 including salary and benefits). We then subtracted estimated savings that would be generated from reduced use of services (using the difference in utilization between LAC-USC and Harbor-UCLA before and after the intervention) by applying publicly available Medi-Cal (California Medicaid) prices for each service (eTable 1 in the Supplement gives these payment rate estimates).38 Although the physicians and clinicians in our study did not receive a direct economic benefit from the savings, any cost savings would largely benefit LAC-USC as a whole by liberating resources for use in other areas of need.

To consider the cost outcomes under different payment models, we also estimated costs from 2 other (hypothetical) economic perspectives: (1) a health system under a fee-for-service payment model and (2) US society overall (including patients, payers, and tax payers). Although our cost analysis primarily focused on observed costs associated with changes in our primary end points, we used publicly available Medicare reimbursement prices39 and different assumptions to extrapolate the cost savings or losses of physician time and effort based on these different stakeholder perspectives. For example, reducing utilization at a fee-for-service health system would lead to revenue losses instead of savings, using the assumption that the health system would earn on average approximately 15% profit margin on health services delivered.40,41 In addition, for both simulations, we used a national mean salary estimate for the QI nurse.42 For the societal cost analysis, we used full Medicare national facility rates and calculated patients’ productivity losses by counting 4 hours (time spent performing a test or visit instead of working), assuming at most 1 service per patient and earnings of $18 per hour based on US Bureau of Labor Statistics national median wages during the study period.43 Finally, we performed a 3-year projection of costs assuming that utilization trends remained similar after the first year ended. On the basis of standard cost-effectiveness analysis guidelines, we assumed that costs would be discounted by 5% per year (2% for inflation and 3% in interest rate returns).44 All analyses were completed using SAS, version 9.4 (SAS Institute Inc).

Results

During the main study 6-month preintervention and 6-month postintervention periods, 469 intervention and 585 control patients underwent cataract surgery (with an additional 1002 intervention and 511 control patients at each site in the 12-month follow-up period). The mean (SD) age was 60.6 (11.1) years (60.4 [10.7] years for Harbor-UCLA and 61.0 [11.7] years for LAC-USC), and 546 (51.8%) were female. Patients at the 2 sites had some demographic and clinical differences (Table 1). The LAC-USC group had fewer female patients (59 of 133 [44.4%] vs 163 of 288 [56.6%], P = .02), more Hispanic patients (107 of 133 [80.5%] vs 186 of 288 [64.6%], P < .001), and patients with fewer comorbidities (mean Charlson comorbidity index, 0.5 vs 1.3; P < .001) compared with at the Harbor-UCLA group at baseline.

Table 1. Patient Characteristics.

Characteristic Medical Center Site P Value
LAC-USC (Intervention) Harbor-UCLA (Control)
Before (n = 133) After (n = 336) Follow-up (n = 1002) P Value Before (n = 288) After (n = 297) Follow-up (n = 511) P Value Time-by-Group Interaction Baseline Comparison
Age at surgery, y
Mean (SD) 61.7 (11.5) 60.7 (11.7) 62.0 (11.1) .18 60.9 (10.4) 59.9 (10.9) 60.6 (10.8) .47 .80 .49
Median (IQR) [range] 63 (55-71) [31-86] 62 (53-68) [25-94] 62 (56-68) [19-91] 61 (55-67) [26-88] 60 (53-66) [25-89] 61 (54-67) [21-101]
Sex, No. (%)
Male 74 (56) 171 (51) 435 (43) .02 125 (43) 138 (46) 236 (46) .70 .03 .02
Female 59 (44) 165 (49) 567 (57) 163 (57) 159 (54) 275 (54)
Race/ethnicity, No. (%)
Hispanic 107 (80) 270 (80) 715 (71) <.001 186 (65) 208 (70) 290 (57) <.001 .73 <.001
Non-Hispanic 24 (18) 56 (17) 174 (17) 101 (35) 89 (30) 169 (33)
Unknown 2 (2) 10 (3) 113 (11) 1 (<1) 0 52 (10)
Mean (SD) Charlson comorbidity index 0.5 (1.3) 1.1 (1.4) 1.6 (1.7) <.001 1.3 (1.5) 1.4 (1.4) 1.5 (1.6) .02 <.001 <.001
Open in a new tab

Abbreviations: IQR, interquartile range.

Visit and Testing Utilization

The baseline percentage of preoperative utilization was similar at both sites. Preoperative utilization declined more for intervention than for control patients (Figure 2). Preoperative visits decreased from 93% to 24% in the intervention group and increased from 89% to 91% in the control group (between group difference, −71%; 95% CI, −80% to −62%). Chest x-rays decreased from 90% to 24% in the intervention group and increased from 75% to 83% in the control group (between group difference, −75%; 95% CI, −86% to −65%). Laboratory testing decreased from 92% to 37% in the intervention group and decreased from 98% to 97% in the control group (between group difference, −56%; 95% CI, −64% to −48%). Utilization of ECGs decreased from 95% to 29% in the intervention group and increased from 86% to 94% in the control group (between group difference, −74%; 95% CI, −83% to −65%). All time-by-group interactions had P < .001. Table 2 gives multivariable adjusted odds ratios of these interactions. On entering baseline utilization trends into the analysis from the previous EHR, we confirmed stable trends and similar results (eFigures 1-4 in the Supplement). Surgical no-shows or cancellations increased at the intervention site in the 6 months after the intervention; however, the interaction tests between time and group (cataract surgery vs noncataract ophthalmologic surgery and cataract surgery vs all other nonophthalmologic surgeries) yielded odds ratios for surgical no-shows (1.98 [95% CI, 0.34-11.37]) or cancellations (2.55 [95% CI, 0.67-9.65]) that were nonsignificant (eTable 2 in the Supplement).

Figure 2. Unadjusted Percentage of Patients Receiving Preoperative Care.

Figure 2.

Open in a new tab

LAC-USC indicates Los Angeles County and University of Southern California.

Table 2. Multivariable Adjusted Logistic Regression Analysis of Time-by-Group Interactionsa.

Preoperative Care Adjusted Odds Ratio (95% CI)
After vs Before Intervention vs Control Group Multivariable Follow-up vs Before Intervention vs Control Group Multivariable
Medical visits 0.02 (0.01 to 0.05) 0.03 (0.01 to 0.09)
Chest x-rays 0.02 (0.01 to 0.04) 0.01 (<0.01 to 0.01)
Laboratory testing 0.04 (0.01 to 0.15) 0.02 (0.01 to 0.06)
Electrocardiograms 0.01 (<0.01 to 0.02) <0.01 (<0.01 to 0.01)
Open in a new tab
a

Multivariable logistic regression models adjusted for patient age at time of surgery, sex, race/ethnicity, and Charlson comorbidity index, and interacted time by group for each outcome.

During the 12 months of additional follow-up, the QI team reestablished preoperative medical visits (67%); however, low-value preoperative chest x-rays (12%), laboratory tests (28%), and electrocardiograms (11%) remained low. All time-by-group interactions had P < .001, including for preoperative medical visits.

Sensitivity analyses using 30-day and 60-day preoperative periods showed similar findings to our main results. Routine eye examinations remained unchanged (eFigure 5 in the Supplement), and serious 30-day adverse events remained rare at both sites (eTable 3 in the Supplement).

Cost Analysis and Staff Utilization

Table 3 gives details on the cost analysis. After accounting for the cost of the QI intervention, the intervention was associated with a loss of $42 241 for LAC-USC in year 1; however, 3-year projections revealed that the intervention would be associated with net savings to the health system of $67 241 at LAC-USC. The intervention also was associated with an LAC-USC ophthalmology clinic licensed vocational nurse reducing her workload by 100% in the first 6 months (at which time she began to pursue other clinical work) and 70% in the 12 months of follow-up (including after preoperative medical visits were reinstated). Preoperative workload for other clinicians did not substantively change.

Table 3. Cost Analysis of Investments and Estimated Savings From 3 Fiscal Perspectivesa.

Investments and Savings LAC-USC (Actual) Fee-for-Service Health System (Simulation) Societal (Simulation)b
Price, $ Changes in Quantityc Costs, $d Price, $ Changes in Quantityc Costs, $d Price, $ Changes in Quantityc Costs, $d
Training costs of QI, 10 mo before year 1
IHI quality advisor course 16 200 1 (16 200) 16 200 1 (16 200) 16 200 1 (16 200)
30% FTE of QI nursee 46 613 1 (46 613) 25 560 1 (25 560) 25 560 1 (25 560)
Year 1 intervention costs
Registered nurse quality officer, 20% FTE 38 308 1 (38 308) 20 448 1 (20 448) 20 448 1 (20 448)
Preoperative savings
Medical visits 18.10 −710 12 842 7.75 −710 (2750) 25.84 −710 18 334
Chest x-rays 25.98 −754 19 586 8.51 −754 (3206) 28.35 −754 21 372
Laboratory testing 9.28 −557 5171 4.31 −557 (1916) 14.37 −557 8007
Electrocardiograms 28.70 −742 21 282 5.17 −742 (1201) 17.23 −742 12 776
Total estimated savings NA NA 58 880 NA NA (9073) NA NA 60 490
Estimated avoided lost work because of visits and testing NA NA NA NA NA NA 72.00 518 37 268
Total observed costs through end of year 1 NA NA (42 241) NA NA (71 281) NA NA 35 550
Total estimated projected 3-y costs (estimated) NA NA 67 241 NA NA (88 151) NA NA 217 322
Open in a new tab

Abbreviations: EHR, electronic health record; FTE, full-time employee; IHI, Institute for Healthcare Improvement; LAC-USC, Los Angeles County and University of Southern California; QI, quality improvement.

a

Costs = price × changes in quantity. Cost analysis was performed from 3 fiscal perspectives: (1) LAC-USC (capitated safety-net health system), (2) hypothetical fee-for-service private health system, and (3) societal (US society including patients, payers, and tax payers).

b

Three-year projections were extrapolated from 1 year before and after intervention data, not including the uptick in visits during the follow-up period. Three-year projections also assumed that costs would be discounted by 5% per year (2% for inflation and 3% in interest rate returns) based on standard cost-effectiveness analysis guidelines.44

c

The 2 sites had slightly different denominators, and 1 year of continuous data was not available because of the EHR upgrade. The denominator (n = 1002 patients) was used based on the extra 1 year of follow-up data at LAC-USC. Accordingly, changes in the percentage of testing at LAC-USC vs Harbor-UCLA before and after the intervention are given in Figure 1. We calculated changes in quantity at LAC-USC vs Harbor-UCLA before and after the intervention as follows: –71% × 1002 = –710 for preoperative medical visits; –75% × 1002 = –754 for preoperative chest x-rays; –74% × 1002 patients = –742 for preoperative electrocardiograms; and –55 × 1002 patients = –557 for preoperative laboratory testing.

d

Losses are in parentheses.

e

The FTE calculation for QI nurse included actual salary plus benefits to reflect how much the health system actually paid. The simulations used an FTE estimate based on the national average annual salary for a QI nurse ($78 645 plus 30% for benefits) to maximize generalizability.

When extrapolating the estimated reductions in utilization to understand the cost outcomes under a fee-for-service payment model, the intervention would be associated with losses of $71 281 in year 1 and $88 151 for 3 years from the start of the intervention. From a societal perspective, the intervention would be associated with an estimated $35 550 in savings in year 1 and an estimated $217 322 in savings for 3 years.

Discussion

This multidisciplinary QI initiative was associated with a marked and sustained decline in low-value preoperative testing for patients undergoing cataract surgery. After year 1, the initiative was associated with freeing a licensed vocational nurse to pursue other work but also income losses for LAC-USC; however, 3-year projections estimated a modest amount of cost savings for LAC-USC. Of importance, depending on the payment model, reductions in use resulted in varying financial outcomes: extrapolated income losses from a fee-for-service payment health system’s perspective and extrapolated savings from society’s perspective.

As noted above, a systematic review of the literature suggested that multicomponent initiatives (similar to this initiative) were more likely to succeed in reducing low-value care compared with single-component initiatives.29 We highlighted several important factors that have not been tested in combination, to our knowledge, and may have worked synergistically to decrease low-value care. First, a well-trained QI nurse used time-tested QI methods that favored iterative, bottom-up approaches tailored to local contexts as opposed to centralized, top-down planning.45,46 Specifically, these methods included plan-do-study-act cycles and Institute for Healthcare Improvement principles of data gathering and multidisciplinary clinician education and engagement, including obtaining critical buy-in from the clinical chiefs, removing non–evidence-based protocols, and convincing faculty, trainees, and staff to stop routinely ordering preoperative testing.23,32 Second, point-of-care potassium and glucose tests were available on the day of surgery, and none of the preoperative utilization rates declined to 0%, a finding underscoring the importance of preserving clinician judgment and autonomy to deliver appropriate care. Third, other factors (which although present at both the intervention and control groups, still likely augmented the outcomes of the intervention) included a capitated payment structure, meaning that reducing utilization would be more likely to save resources, and the director of LAC-DHS, who was committed to building a culture of high-value care.

The cost analysis provided important additional insights.30 The intervention was associated with revenue losses in the first 6 months after its rollout because of considerable upfront investment costs; such losses underscore the substantial challenges associated with reducing low-value care, even when incentives align. Of note, the cost analysis assumptions were conservative; for example, we did not count productivity gains from freeing the ophthalmology clinic licensed vocational nurse to pursue other clinical work. Moreover, many health systems employ QI nurses with similar training obtained before employment (which we counted as investments of both tuition and time); such arrangements would allow health systems to accrue savings even earlier. Nevertheless, 3-year projections estimated cost savings for LAC-USC.

Although fee-for-service revenue projections will vary by payer mix among other factors, under a fee-for-service payment model (using conservative assumptions), reductions in low-value care would yield revenue losses each year, representing a potential barrier to reducing low-value care and a tension between health system profits vs financial gains to society. The extent to which these relatively small revenue losses might be outweighed by addressing other contributors to persistent low-value preoperative care (eg, medico-legal concerns, non–evidenced-based protocols, and the status quo bias) will be important to investigate moving forward, particularly in light of Medicare’s recent proposal to loosen requirements for preoperative evaluations.47 Regardless, in our models, (1) savings and losses were relatively small; (2) even at financially capitated LAC-USC and Harbor-UCLA, most patients received low-value preoperative testing before the intervention; and (3) substantial multidisciplinary clinician education and engagement was required to reduce low-value testing. All these factors suggest that the capitated financial incentives as currently designed are important but insufficient alone to decrease low-value preoperative care.

Interventions in complex health systems will have trade-offs and unintended consequences, which should be monitored and reported.23,31,34,48 In our study, we measured many potential unintended consequences, such as reductions in preoperative eye examinations, increases in surgical no-shows or cancellations, and adverse postoperative events; however, none of these undesirable outcomes changed with exposure to the intervention. Although we were likely underpowered to detect differences in uncommon no-shows or cancellations and, in particular, rare postoperative adverse events, previous well-designed and adequately powered randomized clinical trials did not find that eliminating preoperative testing increased postoperative adverse events in patients undergoing cataract surgery.18 In addition, we were unable to report the intervention’s outcome on access to care because the EHR upgrade precluded our ability to measure time to surgery. Although we did not formally evaluate access to care, LAC-USC surgical volume did not decline after the intervention, suggesting that access to care did not worsen.

An unanticipated increase in preoperative medical visits during the follow-up period was observed, suggesting some unintended consequences of the intervention that our evaluation did not quantitatively capture. Our QI team reported that the increase in visits occurred in response to the team noticing that, after the intervention, some patients were arriving to undergo cataract surgeries after being late for the appointment, after having eaten, or having been otherwise unprepared. Recognizing that preoperative medical visits had served as an important educational and care-coordination visit for many safety-net patients with limited phone service and health literacy, the QI team reinstated preoperative medical visits in June 2016 (while still emphasizing eliminating routine preoperative testing). While it is reassuring that the resumption of visits was not associated with a concomitant increase in low-value testing in the follow-up period, such unintended consequences suggests that virtually all QI interventions require continuous monitoring, evaluation, and iterative modification, and such continuous QI is easier in the scale of a local health system than on the larger, national scale.23,49,50,51 Moreover, the importance of preoperative medical visits may be particularly salient for clinicians at other safety-net health systems who are interested in improving value without causing unintentional patient harm.

Ultimately, we believe these findings should be of interest to others across the nation who are seeking to reduce low-value preoperative testing.52 Although Medicare and Medi-Cal payment rates can underestimate the true cost of services, we used these publicly available reimbursement rates to standardize these analyses and help facilitate comparisons with other health systems.53,54 Moreover, the methods that we used were provided by an Institute for Healthcare Improvement QI course available to all interested in improving health care quality.32 Moving forward, it will be important for others to replicate the intervention (ideally, in a multicentered, randomized clinical trial) to report its effectiveness more broadly.

Limitations

Our study has several limitations. First, although declines in low-value care were swift and marked, this was a nonrandomized observational study, precluding causal inference. Second, our cost analysis was intentionally conservative and made several assumptions. For example, we assigned the entire cost of the Institute for Healthcare Improvement training for the QI nurse to this single intervention rather than amortizing the cost across other QI projects. Moreover, our cost analysis focused on observed costs associated with changes in our primary end points and did not account for other unmeasured costs, such as clerical or other nonphysician time or iatrogenic downstream consequences of overtesting. Third, we could not determine which elements of this multicomponent initiative were most important in reducing low-value care. Fourth, the intervention occurred in a single health system, which limited generalizability because local factors likely played an important role in this initiative.

Conclusions

A multipronged QI initiative was associated with substantial and sustained reductions in low-value preoperative testing among patients undergoing cataract surgery, as well as a modest amount of cost savings for the health system. The findings suggest that reducing low-value care would be associated with cost savings for financially capitated health systems and society but with losses for fee-for-service health systems, highlighting a potential barrier to eliminating low-value care.

Supplement.

eAppendix. Multicomponent Intervention Timeline

eTable 1. Medi-Cal Reimbursement Rates

eTable 2. Odds of Missed Surgeries/No Shows Among Patients Undergoing Cataract Surgery vs. Non-Cataract Ophthalmologic Surgeries and vs. Other Surgeries Over Time at LAC+USC Medical Center.

eTable 3. Adverse vents within 30 days of surgery

eFigure 1. Unadjusted Percentage of Patients Receiving Pre-Operative Medical Visits Over Time

eFigure 2. Unadjusted Percentage of Patients Receiving Pre-Operative Chest X-Rays Over Time

eFigure 3. Unadjusted Percentage of Patients Receiving Pre-Operative EKGs Over Time

eFigure 4. Unadjusted Percentage of Patients Receiving Pre-Operative Labs Over Time

eFigure 5. Unadjusted Percentage of Patients Receiving Pre-Operative Eye Exams Over Time

Click here for additional data file. (219KB, pdf)

References

  • 1.Centers for Medicare and Medicaid Services (CMS) National health expenditure fact sheet. 2018; https://www.cms.gov/research-statistics-data-and-systems/statistics-trends-and-reports/nationalhealthexpenddata/nhe-fact-sheet.html. Accessed November 27, 2018.
  • 2.Fisher ES, Wennberg DE, Stukel TA, Gottlieb DJ, Lucas FL, Pinder EL. The implications of regional variations in Medicare spending, part 1: the content, quality, and accessibility of care. Ann Intern Med. 2003;138(4):273-287. doi: 10.7326/0003-4819-138-4-200302180-00006 [DOI] [PubMed] [Google Scholar]
  • 3.Berwick DM, Hackbarth AD. Eliminating waste in US health care. JAMA. 2012;307(14):1513-1516. doi: 10.1001/jama.2012.362 [DOI] [PubMed] [Google Scholar]
  • 4.Institute of Medicine. Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. Washington, DC: The National Academies Press; 2013. [PubMed] [Google Scholar]
  • 5.Blumenthal D, Davis K, Guterman S. Medicare at 50—moving forward. N Engl J Med. 2015;372(7):671-677. doi: 10.1056/NEJMhpr1414856 [DOI] [PubMed] [Google Scholar]
  • 6.Schwartz AL, Landon BE, Elshaug AG, Chernew ME, McWilliams JM. Measuring low-value care in Medicare. JAMA Intern Med. 2014;174(7):1067-1076. doi: 10.1001/jamainternmed.2014.1541 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Colla CH, Morden NE, Sequist TD, Schpero WL, Rosenthal MB. Choosing wisely: prevalence and correlates of low-value health care services in the United States. J Gen Intern Med. 2015;30(2):221-228. doi: 10.1007/s11606-014-3070-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Reid RO, Rabideau B, Sood N. Low-value health care services in a commercially insured population. JAMA Intern Med. 2016;176(10):1567-1571. doi: 10.1001/jamainternmed.2016.5031 [DOI] [PubMed] [Google Scholar]
  • 9.Mafi JN, Russell K, Bortz BA, Dachary M, Hazel WA Jr, Fendrick AM. Low-cost, high-volume health services contribute the most to unnecessary health spending. Health Aff (Millwood). 2017;36(10):1701-1704. doi: 10.1377/hlthaff.2017.0385 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Mafi JN, Wee CC, Davis RB, Landon BE. Association of primary care practice location and ownership with the provision of low-value care in the United States. JAMA Intern Med. 2017;177(6):838-845. doi: 10.1001/jamainternmed.2017.0410 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Mafi JN, McCarthy EP, Davis RB, Landon BE. Worsening trends in the management and treatment of back pain. JAMA Intern Med. 2013;173(17):1573-1581. doi: 10.1001/jamainternmed.2013.8992 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Mafi JN, Edwards ST, Pedersen NP, Davis RB, McCarthy EP, Landon BE. Trends in the ambulatory management of headache: analysis of NAMCS and NHAMCS data 1999-2010. J Gen Intern Med. 2015;30(5):548-555. doi: 10.1007/s11606-014-3107-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Mafi JN, Wee CC, Davis RB, Landon BE. Comparing use of low-value health care services among US advanced practice clinicians and physicians. Ann Intern Med. 2016;165(4):237-244. doi: 10.7326/M15-2152 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Welch HG, Schwartz L, Woloshin S. Overdiagnosed: Making People Sick in the Pursuit of Health. Boston, Massachusetts: Beacon Press; 2011. [Google Scholar]
  • 15.Barnett ML, Linder JA, Clark CR, Sommers BD. Low-value medical services in the safety-net population. JAMA Intern Med. 2017;177(6):829-837. doi: 10.1001/jamainternmed.2017.0401 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Levinson W, Born K, Wolfson D. Choosing Wisely Campaigns: a work in progress. JAMA. 2018;319(19):1975-1976. doi: 10.1001/jama.2018.2202 [DOI] [PubMed] [Google Scholar]
  • 17.Sigmund AE, Stevens ER, Blitz JD, Ladapo JA. Use of preoperative testing and physicians’ response to professional society guidance. JAMA Intern Med. 2015;175(8):1352-1359. doi: 10.1001/jamainternmed.2015.2081 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Schein OD, Katz J, Bass EB, et al. ; Study of Medical Testing for Cataract Surgery . The value of routine preoperative medical testing before cataract surgery. N Engl J Med. 2000;342(3):168-175. doi: 10.1056/NEJM200001203420304 [DOI] [PubMed] [Google Scholar]
  • 19.Keay L, Lindsley K, Tielsch J, Katz J, Schein O. Routine preoperative medical testing for cataract surgery. Cochrane Database Syst Rev. 2012;(3):CD007293. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Chen CL, Lin GA, Bardach NS, et al. . Preoperative medical testing in Medicare patients undergoing cataract surgery. N Engl J Med. 2015;372(16):1530-1538. doi: 10.1056/NEJMsa1410846 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Matulis J, Liu S, Mecchella J, North F, Holmes A. Choosing Wisely: a quality improvement initiative to decrease unnecessary preoperative testing. BMJ Qual Improv Rep. 2017;6(1):bmjqir.u216281.w6691. doi: 10.1136/bmjquality.u216281.w6691 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.DeVoe JE, Stenger R. Aligning provider incentives to improve primary healthcare delivery in the United States. OA Fam Med. 2013;1(1):7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Mafi JN, Parchman M. Low-value care: an intractable global problem with no quick fix. BMJ Qual Saf. 2018;27(5):333-336. doi: 10.1136/bmjqs-2017-007477 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Schein OD, Pronovost PJ. A preoperative medical history and physical should not be a requirement for all cataract patients. J Gen Intern Med. 2017;32(7):813-814. doi: 10.1007/s11606-017-4043-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Morgan DJ, Leppin AL, Smith CD, Korenstein D. A practical framework for understanding and reducing medical overuse: conceptualizing overuse through the patient-clinician interaction. J Hosp Med. 2017;12(5):346-351. doi: 10.12788/jhm.2738 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Colla CH. Swimming against the curren—what might work to reduce low-value care? N Engl J Med. 2014;371(14):1280-1283. doi: 10.1056/NEJMp1404503 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Jha S. Uncertainty and the diagnostic leviathan. JAMA Intern Med. 2015;175(7):1085-1086. doi: 10.1001/jamainternmed.2015.1103 [DOI] [PubMed] [Google Scholar]
  • 28.Gupta R, Moriates C, Harrison JD, et al. . Development of a high-value care culture survey: a modified Delphi process and psychometric evaluation. BMJ Qual Saf. 2017;26(6):475-483. doi: 10.1136/bmjqs-2016-005612 [DOI] [PubMed] [Google Scholar]
  • 29.Colla CH, Mainor AJ, Hargreaves C, Sequist T, Morden N. Interventions aimed at reducing use of low-value health services: a systematic review. Med Care Res Rev. 2017;74(5):507-550. doi: 10.1177/1077558716656970 [DOI] [PubMed] [Google Scholar]
  • 30.Nuckols TK, Escarce JJ, Asch SM. The effects of quality of care on costs: a conceptual framework. Milbank Q. 2013;91(2):316-353. doi: 10.1111/milq.12015 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Grady D, Redberg RF, O’Malley PG. Quality improvement for quality improvement studies. JAMA Intern Med. 2018;178(2):187. doi: 10.1001/jamainternmed.2017.6875 [DOI] [PubMed] [Google Scholar]
  • 32.Institute for Healthcare Improvement. Improvement advisor professional development program. http://www.ihi.org/education/InPersonTraining/ImprovementAdvisor/Pages/default.aspx. Accessed August 1, 2018.
  • 33.Chen CL, Clay TH, McLeod S, Chang HP, Gelb AW, Dudley RA. A revised estimate of costs associated with routine preoperative testing in Medicare cataract patients with a procedure-specific indicator. JAMA Ophthalmol. 2018;136(3):231-238. doi: 10.1001/jamaophthalmol.2017.6372 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Kerr EA, Kullgren JT, Saini SD. Choosing Wisely: how to fulfill the promise in the next 5 years. Health Aff (Millwood). 2017;36(11):2012-2018. doi: 10.1377/hlthaff.2017.0953 [DOI] [PubMed] [Google Scholar]
  • 35.Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373-383. doi: 10.1016/0021-9681(87)90171-8 [DOI] [PubMed] [Google Scholar]
  • 36.Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol. 1992;45(6):613-619. doi: 10.1016/0895-4356(92)90133-8 [DOI] [PubMed] [Google Scholar]
  • 37.Transparent California. California's largest public pay and pension database. https://transparentcalifornia.com/. Accessed August 7, 2018.
  • 38.California Department of Health Care Services. Medi-Cal rates. https://files.medi-cal.ca.gov/pubsdoco/rates/rateshome.asp. Accessed February 15, 2018.
  • 39.Centers for Medicare and Medicaid Services. Physician fee schedule. https://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/PhysicianFeeSched/. Accessed August 1, 2018.
  • 40.Taheri PA, Butz D, Griffes LC, Morlock DR, Greenfield LJ. Physician impact on the total cost of care. Ann Surg. 2000;231(3):432-435. doi: 10.1097/00000658-200003000-00017 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Sageworks. Most profitable industries over past year. 2017. https://www.sageworks.com/most-profitable-industries-over-past-year/. Accessed December 1, 2017.
  • 42.ZipRecruiter. Average salary of quality improvement coordinator (RN) jobs. https://www.ziprecruiter.com/Salaries/Quality-Improvement-Coordinator-RN-Salary. Accessed November 14, 2018.
  • 43.United States Department of Labor, Bureau of Labor Statistics. May 2017 National occupational employment and wage estimates United States. https://www.bls.gov/oes/current/oes_nat.htm#00-0000. Accessed August 7, 2018.
  • 44.Weinstein MC, Siegel JE, Gold MR, Kamlet MS, Russell LB. Recommendations of the panel on cost-effectiveness in health and medicine. JAMA. 1996;276(15):1253-1258. doi: 10.1001/jama.1996.03540150055031 [DOI] [PubMed] [Google Scholar]
  • 45.Tsugawa Y, Mafi JN. Getting doctors to make better decisions will take more than money and nudges [published online June 18, 2018]. Harv Bus Rev. 2018. [Google Scholar]
  • 46.Khullar D, Wolfson D, Casalino LP. Professionalism, performance, and the future of physician incentives. JAMA. 2018. doi: 10.1001/jama.2018.17719 [DOI] [PubMed] [Google Scholar]
  • 47.Inserro A. CMS proposes cuts in hospital, surgical center regulations. AJMC Managed Markets Network. Published September 17, 2018. https://www.ajmc.com/newsroom/cms-proposes-cuts-in-hospital-surgical-center-regulations. Accessed February 20, 2019. [Google Scholar]
  • 48.Taleb NN. The Black Swan: Second Edition: The Impact of the Highly Improbable. New York, NY: Random House Books; 2007. [Google Scholar]
  • 49.Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q. 1966;44(3)(suppl):166-206. doi: 10.2307/3348969 [DOI] [PubMed] [Google Scholar]
  • 50.Donabedian A. Criteria and standards for quality assessment and monitoring. QRB Qual Rev Bull. 1986;12(3):99-108. doi: 10.1016/S0097-5990(16)30021-5 [DOI] [PubMed] [Google Scholar]
  • 51.Parchman ML, Henrikson NB, Blasi PR, et al. . Taking action on overuse: creating the culture for change. Healthc (Amst). 2017;5(4):199-203. doi: 10.1016/j.hjdsi.2016.10.005 [DOI] [PubMed] [Google Scholar]
  • 52.Buxbaum JD, Mafi JN, Fendrick AM. Tackling low-value care: a new “top five” for purchaser action. Health Affairs Blog; November 21, 2017. Accessed February 18, 2019. https://www.healthaffairs.org/do/10.1377/hblog20171117.664355/full/. [Google Scholar]
  • 53.Miller ME, Zuckerman S, Gates M. How do Medicare physician fees compare with private payers? Health Care Financ Rev. 1993;14(3):25-39. [PMC free article] [PubMed] [Google Scholar]
  • 54.Boccuti C, Moon M. Comparing Medicare and private insurers: growth rates in spending over three decades. Health Aff (Millwood). 2003;22(2):230-237. doi: 10.1377/hlthaff.22.2.230 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplement.

eAppendix. Multicomponent Intervention Timeline

eTable 1. Medi-Cal Reimbursement Rates

eTable 2. Odds of Missed Surgeries/No Shows Among Patients Undergoing Cataract Surgery vs. Non-Cataract Ophthalmologic Surgeries and vs. Other Surgeries Over Time at LAC+USC Medical Center.

eTable 3. Adverse vents within 30 days of surgery

eFigure 1. Unadjusted Percentage of Patients Receiving Pre-Operative Medical Visits Over Time

eFigure 2. Unadjusted Percentage of Patients Receiving Pre-Operative Chest X-Rays Over Time

eFigure 3. Unadjusted Percentage of Patients Receiving Pre-Operative EKGs Over Time

eFigure 4. Unadjusted Percentage of Patients Receiving Pre-Operative Labs Over Time

eFigure 5. Unadjusted Percentage of Patients Receiving Pre-Operative Eye Exams Over Time

Click here for additional data file. (219KB, pdf)

Articles from JAMA Internal Medicine are provided here courtesy of American Medical Association

RESOURCES