Longitudinal Content Analysis of the Characteristics and Expected Impact of Low-Value Services Identified in US Choosing Wisely Recommendations

Key Points

Question

What are the characteristics and expected impact of low-value services identified by Choosing Wisely recommendations by US physician societies?

Findings

In this qualitative study of 626 recommendations, the plurality identified low-value imaging and laboratory studies, services in the clinical contexts of healthy patients or those with a chronic condition, and services that were low cost and revenue neutral for the given society. Nearly half of identified low-value services had high direct harm potential, while 62% had high potential for cascades (ie, triggering downstream services).

Meaning

The study results suggest that current recommendations identify services with a range of expected impacts; prioritization based on impact may be helpful in developing future recommendations and targeting interventions.

Abstract

Importance

The US Choosing Wisely campaign has had substantial reach in mobilizing efforts to reduce low-value care, achieved largely by engaging physician specialty societies in stewardship. While some early recommendations were criticized for avoiding revenue-generating services, there is limited evidence of how the composition of recommendations shifted as more societies contributed.

Objective

To analyze the characteristics and expected impact of Choosing Wisely recommendations.

Design, Setting, and Participants

This qualitative study included content and trend analyses of all 626 Choosing Wisely recommendations by US physician societies as of March 1, 2021. Data were analyzed between March and May 2021.

Main Outcomes and Measures

Primary outcomes were proportions of identified low-value services by characteristics (society type, service type, indication, do vs avoid, and clinical context) and expected impact (effect on the revenue of society members, cost, number of individuals at risk, direct harm potential, and cascade potential).

Results

Low-value services identified in the 626 Choosing Wisely recommendations largely covered imaging (168 [26.8%]) and laboratory studies (156 [24.9%]) in the context of chronic conditions (169 [27.0%]) and healthy patients with risk factors alone (126 [20.1%]). Most of the identified low-value services were revenue neutral for the recommending society (402 [64.2%]) and the plurality were low cost (<$200; 284 [45.4%]); low-cost services represented a growing share of low-value services identified by Choosing Wisely recommendations (1.2 percentage points per year; P = .001). Nearly half (280 [44.7%]) of recommendations identified services with high direct harm potential, and 388 (62.0%) identified those with high potential for cascades (ie, triggering downstream services).

Conclusions and Relevance

The results of this qualitative study suggest that the Choosing Wisely recommendations identified services with a range of expected impacts. Stakeholders could explicitly set priorities for future recommendations, while clinical leaders and payers might target intervention efforts on recommendations with the greatest potential for impact based on spending across populations, direct harms, and cascades.

This qualitative study examines the characteristics and expected impact of 626 Choosing Wisely recommendations.

Introduction

Efforts to curb unsustainable growth in US health care spending have focused on reducing the estimated $76 to $101 billion spent on low-value care, which is defined as services that provide no or minimal benefit to patients in specific clinical situations.¹ To this end, the American Board of Internal Medicine Foundation and Consumer Reports launched the physician-led Choosing Wisely campaign in 2012 to encourage awareness, measurement, physician accountability, and intervention around this problem. Nine years later, the US Choosing Wisely compendium of recommendations for tests and treatments to avoid has grown from 45 to more than 600. The recommendations have been translated into physician, hospital, and system-level quality metrics and multiservice indices.^2,3 They have been used by clinical leaders to direct policies and interventions⁴ and payers to guide utilization management of low-value services.⁵ Choosing Wisely has also expanded to more than 20 countries in North America, Europe, Asia, and South America, becoming the prism through which efforts to reduce low-value care are viewed worldwide.^6,7,8

Despite the wide reach of the Choosing Wisely campaign and its success at raising awareness,^6,8 progress in reducing low-value care use and spending has been modest at best,^9,10 which may be partially explained by the characteristics and expected impact of the services identified in these recommendations. By design, the Choosing Wisely campaign did not focus on cost reduction and allowed individual societies to set their own priorities when writing recommendations to ensure physician autonomy and leverage shared professional values to spur necessary, previously absent conversations about overuse.^8,11 In the absence of formal guidance, some have raised concerns that the recommendations largely target services with low impact in terms of cost and prevalence that are not revenue generating for members of the recommending societies.^12,13,14 It is unclear what the direct harms and downstream consequences of the identified services may be.^15,16,17 Because Choosing Wisely recommendations must focus on easily definable services, they may also miss the full scope of medical services that constitute wasteful care.^{13,18,19,20,21,22}

Because deimplementation of low-value care is so challenging,²³ it is critical to target efforts to maximize improvement in health care value. To this end, it is important to understand the characteristics and potential impact of current Choosing Wisely recommendations and how their composition has changed over time to establish where current priorities lie and identify new opportunities to reduce low-value service use and spending. Using existing conceptual frameworks, we analyzed all Choosing Wisely recommendations by US-based physician societies from 2012 to 2021 to assess the services, clinical contexts, and consequences of the stated low-value services in aggregate and over time.

Methods

Data Source and Sample

We manually extracted the text and release dates of all published Choosing Wisely recommendations that were proposed by US-based physician societies on choosingwisely.org through March 1, 2021, to generate a data set of 626 recommendations. This study of publicly available text did not involve human participants and was exempt from institutional review board review.

Content Analysis

Using a literature review and iterative discussion with sample testing,^15,24 2 physician researchers (I.G. and D.K.) identified 10 mutually exclusive and comprehensively exhaustive variables to assess the (1) characteristics and (2) expected impact of low-value services described in the Choosing Wisely recommendations (Table 1). After developing the set of variables, each reviewer coded 20 recommendations independently and then convened to discuss the interpretation and application of these variables, revising them as needed. We repeated this process 3 times until consensus was reached. Each reviewer then coded all 626 recommendations independently with periodic sample checks. Finally, we conducted a series of joint iterative reviews to reach consensus on discrepant coding decisions.

Table 1. Distribution of the Characteristics and Expected Impact of Low-Value Services Identified in 626 Choosing Wisely Recommendations.

Domain	Variable	Categories	Recommendations, No. (%)
Characteristics of low-value service	Society type	Specialty	215 (34.3)
		Subspecialty	286 (45.7)
		Skill/procedure/setting based	80 (12.8)
		Disease/physiology/pathophysiology based	45 (7.2)
	Service type	Physical examination	15 (2.4)
		Laboratory test	156 (24.9)
		Imaging	168 (26.8)
		Noninvasive monitor	26 (4.2)
		Procedure	112 (17.9)
		Medication	142 (22.7)
		Blood product	22 (3.5)
		Other treatment	48 (7.7)
		Visit/hospitalization	10 (1.6)
	Indication	Screening	111 (17.7)
		Diagnosis	189 (30.2)
		Monitoring	72 (11.5)
		Prevention	34 (5.4)
		Treatment	246 (39.3)
	Do vs avoid	Do	590 (94.2)
		Avoid	36 (5.8)
	Clinical context	Healthy/risk factors only	126 (20.1)
		Undiagnosed signs/symptoms	114 (18.2)
		Minor acute condition	117 (18.7)
		Major illness/high-risk event	72 (11.5)
		Past major illness/high-risk event	21 (3.4)
		Periprocedure	59 (9.4)
		Chronic condition	169 (27.0)
		Complex comorbidities/frailty	9 (1.4)
		End of life	8 (1.3)
Expected impact	Effect on revenue of society member physicians	Revenue earned	209 (33.4)
		Revenue neutral	402 (64.2)
		Revenue lost	15 (2.4)
	Cost	<$200	284 (45.4)
		$200-$2000	237 (37.9)
		>$2000	105 (16.8)
	No. of individuals at risk	General population by age	83 (13.3)
		Common clinical scenario	274 (43.8)
		Uncommon clinical scenario	241 (38.5)
		Rare clinical scenario	28 (4.5)
	Direct harm potential	Low	346 (55.3)
		High	280 (44.7)
	Cascade potential	Low	238 (38.0)
		High	388 (62.0)

Each recommendation was counted once regardless of its similarity to other recommendations. For most variables, we coded a single category; for the 3 variables for which a recommendation might subsume more than 1 category (service type, indication, and clinical context), we allowed coding of up to 3 categories each. We categorized the effect of each identified low-value approach on the clinical revenue generated by physician members of the recommending society; a low-value approach was considered revenue generating if society physician members would be expected to receive fee-for-service reimbursement for taking the approach, revenue losing if the low-value approach entailed loss of fee-for-service reimbursement (by virtue of foregone services), and revenue neutral if neither of those conditions was met. For cost estimates, we took the individual service or a standard period as the unit of analysis (eg, for daily chest radiography, we used the cost of a single radiograph; for long-term medications, we estimated the cost of a month’s supply) and based the estimates on Medicare allowed rates and GoodRx. We assessed the potential for direct harms and cascades (defined as downstream medical services of uncertain value that follow from an initial test or treatment)^16,17,25 in reference to an individual patient (rather than to the health system or society) using an established conceptual model that incorporates nonphysical harms.¹⁵ Cascade potential was defined based on the likelihood of incidental findings, overdiagnosis, false positive results, or cascading adverse events (ie, second-order adverse effects and complications). The definition accounted for the frequency, severity, and duration of potential cascades (eTable 1 in Supplement 1).^17,25,26

Quantitative Analysis

We used descriptive statistics (counts and percentages) and plotted the distribution of the variables over time. For categories in which a recommendation fit into more than 1 response, we counted each response once such that the percentages added up to more than 100%. To assess time trends, we graphed the distribution of each variable by year. For variables with visually identifiable trends, we built linear proportion models in which the outcome was the proportion of all recommendations with a given code (eg, low-cost service) and the predictor was year. Statistical analyses were conducted using Stata SE, version 15.1 (StataCorp), and statistical significance was set at P < .05.

Results

We examined 626 recommendations by 93 societies. Each society offered a range of 5 to 35 recommendations (eTable 2 in Supplement 2).

Characteristics of Low-Value Services

Most recommendations came from specialty societies (215 recommendations [34.3%]; eg, American College of Obstetricians and Gynecologists) or subspecialty societies (286 [45.7%]; eg, Society for Maternal-Fetal Medicine), whereas fewer came from skill-, procedure-, or setting-based societies (80 [12.8%]; eg, Society of Cardiovascular Computed Tomography) or from disease-based, physiology-based, or pathophysiology-based societies (45 [7.2%]; eg, Heart Rhythm Society). Recommendations commonly named low-value services that were imaging studies (168 [26.8%]; eg, foot radiography), laboratory tests (156 [24.9%]; eg, serum vitamin K test), medications (142 [22.7%]; eg, benzodiazepines), or procedures (112 [17.9%]; eg, colonoscopy), while recommendations that named visits/hospitalizations as low-value services were rare (10 [1.6%]; eg, use of the intensive care unit for postoperative monitoring).

When considering the indication for the low-value service, the most common was treatment (246 [39.3%]; eg, antibiotics) followed by diagnosis (189 [30.2%]; eg, periodic fever syndrome genetic panel) and screening (111 [17.7%]; eg, cervical cytology), whereas prevention was least common (34 [5.4%]; eg, deep vein thrombosis prophylaxis). Nearly all recommendations specified low-value approaches that involved doing something (590 [94.2%]; eg, performing a stress echocardiogram) rather than the act of omitting or delaying care (36 [5.8%]; eg, delaying a palliative care consultation).

The most common clinical contexts for the low-value services were patients with chronic conditions (169 [27.0%]; eg, hypertension, insomnia), those who were healthy or had risk factors alone (126 [20.1%]; eg, pregnancy, contrast dye in the setting of seafood allergy), those with a minor acute condition (117 [18.7%], eg, otitis media), and those with undiagnosed signs/symptoms (114 [18.2%]; eg, amenorrhea). In contrast, services identified in the context of complex comorbidities/frailty (eg, advanced dementia, elderly individual with frailty, patient with polypharmacy) and end of life (eg, terminal cancer, patients at high risk of death) were rare (9 [1.4%] and 8 [1.3%], respectively).

Expected Impact

Most of the identified low-value approaches were revenue neutral for members of recommending societies (402 [64.2%]; eg, a recommendation from the American Academy of Family Physicians against screening for carotid artery stenosis). One-third (209 [33.4%]) of the named low-value approaches were revenue generating (eg, performing surgery for a bunion or hammer toes in asymptomatic patients, which the American Orthopedic Foot and Ankle Society recommends against). The remaining recommendations (15 [2.4%]) identified low-value approaches that represented lost revenue for the society by virtue of foregone services (eg, the Society of American Gastrointestinal and Endoscopic Surgeons recommended against discharging patients with acute cholecystitis without first offering laparoscopic cholecystectomy).

Nearly half (284 [45.4%]) of recommendations identified services that were low cost (<$200), such as serum vitamin D tests or electrocardiograms. The remaining identified services that were moderate cost ($200-$2000, 237 [37.9%]), such as coronary computed tomography angiography, while 105 (16.8%) identified high-cost services (>$2000), such as a positron emission tomography scan or hemodialysis. The share of recommendations representing low-cost services increased over time (1.2%, P = .001) such that there was a 1.2 percentage point increase in the proportion of low-cost services for each additional year (Figure 1).

Figure 1. Distribution of Costs of Low-Value Services in Choosing Wisely, 2012-2021.

The number and proportion of recommendations in 2012 through 2021 by the estimated cost of the identified low-value services.

When considering the number of individuals at risk, we found that most recommendations identified low-value services that covered common clinical scenarios, such as low back pain, pregnancy, or acute respiratory tract infection (274 [43.8%]), or uncommon clinical scenarios, such as pediatric nephrolithiasis (241 [38.5%]). Fewer low-value services applied to general populations (83 [13.3%]), such as ovarian cancer screening in healthy women, or to rare clinical scenarios (28 [4.5%]), such as snake envenomation. High-cost services applying to the general population were rare (4 [0.6%]), whereas 101 services (16.1%) were low cost and pertained to uncommon or rare clinical scenarios (Figure 2).

Figure 2. Cost of Service vs Percentage of Individuals at Risk.

The proportion of all recommendations (N = 626) by the cost of the identified low-value services (<$200, $200-$2000, and>$2000) and the percentage of individuals at risk of receiving the low-value services (rare clinical scenario, uncommon clinical scenario, common clinical scenario, and general population by age).

Nearly half (280 [44.8%]) of identified low-value services had high potential for direct harm (eg, central venous catheter placement), while 388 (62.0%) had high potential for cascades (eg, opioid treatment,²⁷ preoperative electrocardiogram,¹⁶ and prostate specific antigen test²⁸). Most services with low direct harm nevertheless had high cascade potential (Table 2), and 120 recommendations (19.2%) named services with high direct harm and high cascade potentials. Three recommendations (0.5%) identified services with the highest potential impact: high cost, applied to general populations, high direct harm, and high cascade potential (eg, percutaneous coronary intervention in asymptomatic patients with normal stress test results).

Table 2. Direct Harm Potential vs Cascade Potential^a.

Cascade potential	Direct harm potential, No. (%)
	Low	High
Low	78 (12.5)	160 (25.6)
High	268 (42.8)	120 (19.2)

^a The proportion of all recommendations (N = 626) by high vs low direct harm potential and high vs low cascade potential of the identified low-value services.

Discussion

In this analysis of US Choosing Wisely recommendations, identified low-value services largely covered imaging and laboratory studies in the context of chronic conditions and healthy patients with risk factors alone. Most services had low or moderate unit costs and were revenue neutral for members of the recommending society. Nearly half had high direct harms, while more than half had high cascade potential. While the full scope of low-value care is unknown, these results point to the success of Choosing Wisely in identifying services with a range of impacts based on expected cost, populations affected, direct harms, and cascades.

We found that recommendations covered a range of low-value service types and contexts, highlighting opportunities to prioritize current and future recommendations. While many recommendations focused on discrete laboratory and imaging studies, fewer than 20% of recommendations focused on procedures that are invasive and can contribute substantially to patient harms.²⁹ Fewer targeted potentially unnecessary visits/hospitalizations despite known harms and costs from preventable hospitalizations.²² Similarly, recommendations paid limited attention to complex patients and those at end of life, although there are well-described spending and harms from unnecessary services in these populations.³⁰ For instance, clinicians may do more for medically complex patients, such as caring for a low acuity condition in a hospital rather than in an outpatient setting, because of the perception (founded or not) of higher risk.³¹ The lack of recommendations addressing these contexts is understandable, because the appropriateness of services provided to patients with complex comorbid or terminal conditions is clinically nuanced and context specific.³² Future attention to these sources of overuse may increase the impact of efforts to reduce overuse.

While in aggregate, following the Choosing Wisely recommendations would theoretically result in lower health care spending for patients and payers, this study’s results regarding the financial effects on member physicians suggest possible room for improvement. Most recommendations identified low-value approaches that were revenue neutral for the members of the recommending society. Only one-third of identified low-value approaches were revenue generating, such that following recommendations to avoid these approaches would reduce revenue for member physicians. In some cases, the identified low-value approaches represented lost revenue such that following the recommendations would enhance member physician earnings. These findings mirror the Choosing Wisely analysis of Zadro et al,¹² although that study defined income-generating treatments as those that “attract a fee for service and are performed outside of a routine clinical encounter” to accommodate their international scope and is less relevant to the US context. The predominance of revenue neutral services is not surprising given the interest of professional societies in maintaining member income.^33,34 However, to the extent that recommendations from societies hold the most sway over their own members, the relative paucity of low-value services whose elimination would negatively affect the financial health of members suggests that policy makers and other stakeholders might take a larger role in prioritizing low-value services for identification and intervention.

The study’s results on the unit costs of identified low-value services should be interpreted carefully. Although cost is central to the definition of value, we found that the plurality of recommendations identified low-cost (<$200) services, with the proportion of low-cost services increasing each year. This may not be surprising given physician financial incentives and the Choosing Wisely campaign’s decision not to focus on cost reduction (in an analysis of recommendations as of 2018, only 2% cited cost-effectiveness evidence while 29% explicitly mentioned cost¹⁴). At the same time, because spending is the product of unit cost and utilization, the key financial metric is total spending rather than unit prices, and targeting low-cost services remains important when these services are frequently performed. We found that 21% of recommendations applied to common clinical scenarios (Figure 2), and low-cost, high-volume services have been found to contribute most to system-wide low-value spending in studies using Choosing Wisely–based measures.³⁵ In contrast, 17% of the recommendations we examined were for low-cost services in uncommon or rare clinical scenarios, suggesting low utility from targeting those services.

Beyond direct cost, low-value services can harm patients and the health care system through direct effects on physical, psychological, and social health, as well as through harms and costs from the care cascades of uncertain value that may follow.^15,17,36 We found that nearly half of identified services had high potential for direct harms, and even more had high potential for cascades (eg, preprocedural and screening electrocardiograms have low direct harm and high cascade potential).^16,37 Services that have high potential for direct harms and/or cascades should be a priority for deadoption, although optimizing health care value will also require targeting even beneficial services that are cost ineffective.¹⁴

Strengths and Limitations

To our knowledge, this is the first longitudinal content analysis of US Choosing Wisely recommendations; we examined detailed service types, clinical contexts, costs, direct harms, and cascades. However, this study has limitations. For instance, while we estimated the numbers of individuals at risk for identified low-value services, we did not examine current service prevalence (there are services, such as tonsillectomy, for which many people may be eligible yet may nonetheless be rarely or no longer performed). In addition, although we reviewed all recommendations independently, discussing and reaching consensus on any disagreements to optimize reliability, the classifications used in the study were inherently subjective, not externally validated, and may not be reproducible. This will be important to explore in future work.

Conclusions

Taken together, the results of this qualitative study point to the successes of Choosing Wisely to date as well as an opportunity to explicitly prioritize future recommendations and interventions to maximize impact across the dimensions of spending across populations, direct harms, and cascades. Current recommendations do identify many potentially high-impact low-value services; clinical leaders, payers, and policy makers might use the coded recommendations to target such services for intervention based on individual priorities. As the Choosing Wisely compendium evolves with professional societies contributing new and updated recommendations, a broader coalition of stakeholders could define recommendation standards based on these dimensions of impact, given the financial conflicts that affect these societies. Writing and updating these recommendations will also require robust evidence on the spending, utilization, direct harms, and cascades associated with the services.³⁸ Most importantly, eliminating low-value care must extend beyond better lists. We need scalable local, regional, and national multistakeholder efforts to leverage recommendations into meaningful improvements.^39,40 Choosing Wisely has been a helpful tool for raising awareness, but it cannot be the final word.¹¹

Supplement 1.

eTable 1. Variable and category definitions and examples

Supplement 2.

eTable 2. Coding of all recommendations