Appropriateness vs value: Echocardiography in primary care

Abstract

Background

Despite efforts targeting the growth of healthcare spending within the United States, the current increase in expenditures remains a widespread systemic issue. The overuse of healthcare testing has previously been identified as a modifiable contributing factor. One such test, echocardiography, has seen a continuous increase in its rate of use. This test is frequently ordered by primary‐care physicians.

Hypothesis

In the setting of a low likelihood of disease, echocardiography does not substantially change cardiac therapy, even if appropriately ordered.

Methods

We randomly identified 500 patients who received an echocardiogram ordered by a primary‐care physician between January 1, 2014, and December 31, 2014. Of these, 239 patient charts were reviewed and the following extracted: primary indication for the test, echocardiogram results, and changes in patient medical management. In addition, appropriateness of the test was assessed using the appropriate use criteria guidelines for echocardiography.

Results

Nearly 97% of the studies within the ambulatory primary‐care setting were appropriately ordered according to the appropriate use criteria. Among the 239 patients studied, only 52 had abnormalities and only 6 (2.5%) experienced a change in management that corresponded with the initial suspected diagnosis and echocardiographic findings.

Conclusions

To ensure the greatest value and optimize use of diagnostic testing, it may be necessary to develop a more comprehensive set of guidelines to assist clinicians to readily identify patient populations at low, moderate, and high risk for the presence of disease and provide educational interventions, including feedback regarding individual ordering behaviors.

1. INTRODUCTION

According to the US Centers for Medicare and Medicaid Services (CMS), healthcare spending in the United States is projected to grow at an average rate of 5.8% between 2014 and 2023.1 This substantial increase represents a rate 1.1% faster than the expected average growth in the country's gross domestic product and is projected to encompass 19.3% of gross domestic product by 2023.1 Despite the array of interventions created for the purpose of impeding healthcare spending and decreasing the predicted rate of growth, an increase of 6.1% per year in expenditures is projected for the remainder of this period (2017–2023).1

Ensuring improvement in quality of patient care while reducing healthcare‐related costs has been a long‐term goal for organizational leaders in cardiac health. In 2010, Howard Brody, MD, published “Medicine's Ethical Responsibility for Health Care Reform—The Top Five List” in the New England Journal of Medicine.2 The article targeted specialists and medical specialty societies, calling on them to pinpoint clinical tests and treatments that are habitually overused and of little meaningful benefit to patients. Soon after, the National Physicians Alliance (NPA) introduced the “Five Things” concept, which developed a set of steps for physicians in internal medicine, family medicine, and pediatrics in an effort to promote effective use of healthcare‐related resources.2 The top five lists for internal and family medicine include guidelines seeking to limit the ordering of cardiac screening.3

In 2007, the American College of Cardiology Foundation (ACCF) Task Force created a series of guidelines for the appropriate use of cardiac diagnostic services such as echocardiography, referred to as the appropriate use criteria (AUC).4 These criteria have been incorporated into education‐based strategies aimed at limiting unnecessary tests, treatments, and procedures,5 and are recommended for the purpose of decreasing the incidence of inappropriate testing, thus decreasing healthcare costs.6, 7

Despite these efforts, growth in healthcare spending remains a widespread systemic issue. Several modifiable expenses have been identified as major contributing factors, 2 of which include defensive medicine and overuse of diagnostic testing.6, 8, 9 One such test, echocardiography, has seen a continuous increase in its rate of use. This test is frequently ordered by primary‐care physicians (PCPs).10, 11

Western Connecticut Health Network (WCHN), which consists of Danbury Hospital, New Milford Hospital, and Norwalk Hospital, has previously studied the appropriateness of echocardiography‐ordering practices in the inpatient setting using AUC.12 Though the results of the study indicated appropriate use was present in >97% of cases, high rates of appropriateness do not reflect likelihood of disease state or suitability of test use. This is to say, even when appropriately ordered, if pretest likelihood of disease is very low, post‐test probability of disease will be low as well, regardless of the diagnostic value of the test. This has been reflected in previous literature regarding use of noninvasive testing.13 To further explore this phenomenon within the ambulatory primary‐care setting, investigators sought to refer to Bayes' theorem.14

Bayesian theory of diagnostic testing works to combine both test specificity and sensitivity and the prevalence of disease within a population. According to Bayes' theorem, pretest probability is critical in post‐test likelihood of disease. The theory has previously indicated that cardiac testing is valuable to predict post‐test probability when disease prevalence is moderate (30%) and when posttest probability is high (85%–90%.)15

To expand upon previous literature and assess diagnostic benefit, investigators examined post‐test physician intervention in response to echocardiography in a routine primary‐care ambulatory environment. The hypothesis addressed was, in the setting of a low likelihood of disease, echocardiography would not substantially change cardiac therapy, nor would it be diagnostically valuable even if appropriately ordered.

2. METHODS

A retrospective study was performed among a group of PCPs who serve a diverse patient population in western Connecticut. For the purposes of the study, 500 patients who received an echocardiogram ordered by a PCP between January 1, 2014, and December 31, 2014, were randomly identified. Patients were included if they were age ≥ 18 years and had an outpatient transthoracic echocardiogram. To limit the investigation to de novo echocardiography, patients who had received an echocardiogram within the past 10 years were excluded. Those without reports available for review were also excluded. After exclusions, the study population was 239.

For the remaining 239 patients, charts were reviewed and the following extracted: (1) primary indication for the test, (2) echocardiogram results, and (3) changes in medical management. Based on the documented primary indication for the echocardiogram, the appropriateness of the test was assessed using the AUC guidelines for echocardiography.7 Indications were initially selected by PCPs from a prepopulated dropdown list. To mitigate any potential for bias, charts were retrospectively reviewed by 2 board‐certified cardiologists (AMK and OP) to assess appropriateness independently of initial reviewers. Decisions regarding appropriateness were subsequently made based on consensus regarding documentation within the original clinical chart.

An abnormal echocardiogram was defined as the observation of severe, moderate, or mild abnormalities, with the exception that patients with mild or trace valvular regurgitation or septal aneurysm were considered normal. A change in management was defined as any documented alterations in treatment related to the ordering of the echocardiogram and its results. These changes in management included (1) referral to a specialist, (2) referral to emergency department, (3) change in medication, (4) order for additional testing, or (5) plan for a future echocardiogram. A change in medication was defined as the addition or removal of a medication, or change in a current medication's dose. Order of additional testing was defined as the order of any laboratory and/or noninvasive cardiac test, including 24‐h Holter monitoring, a regular stress test, single‐photon emission computed tomography, or a Cardiolite stress test by the PCP.

Charts were then further reviewed by AMK and OP, who utilized concordance decision‐making to classify all changes in management as either type I or type II decisions (Figure 1). Type I decisions were defined as changes in management made for the reason that the echocardiogram was initially ordered. More specifically, these changes reflected workflow in which the provider suspected the presence of a cardiac disease state, ordered an appropriate test, found the disease state, and then modified treatment based on that finding. Alternatively, type II decisions included changes in management unassociated with the initial order, and usually reflected management changes based on an unexpected incidental finding. Outcome‐based diagnostic value decisions were limited to those in the type I category.

Figure 1.

Methodology of patient classification

2.1. Statistical analysis

Differences in key outcomes were assessed using the Fisher exact test and considered statistically significant at a P value <0.05. All statistical analyses were performed using JMP 11.1.1 (SAS Institute, Inc., Cary, NC). This study was reviewed and approved by the Biomedical Research Alliance of New York's Institutional Review Board.

3. RESULTS

The sample population of patients (N = 239) had a mean age of 54.7 (± 17.76) years, and 38.4% were male. Nearly all (96.7%) had appropriate studies ordered based on current criteria for echocardiography (Table 1).7

Table 1.

AUC used for 239 patients

Criteria No.	Total Patients, n (%)
1	83 (34.7)
2	70 (29.3)
5	2 (0.8)
7	10 (4.2)
8	1 (0.4)
9	3 (1.3)
19	1 (0.4)
20	1 (0.4)
34	34 (14.2)
59	1 (0.4)
61	2 (0.8)
67	11 (4.6)
70	7 (2.9)
86	2 (0.8)
91	3 (1.3)
Unk/ND	8 (3.3)

Abbreviations: AUC, appropriate use criteria; ND, not documented; unk, unknown.

Of the patients, 52 (21.8%) had an abnormal echocardiogram (Table 2). Analysis indicated a statistically significant difference in change in management between patients with abnormal and normal results (P < 0.0001). Among patients who had an abnormal echocardiogram, 34.6% had a change in management. In comparison, only 4.3% of patients with normal echocardiogram results had a subsequent change in management. The most commonly found type of change in management was referral to a cardiologist.

Table 2.

Post‐echocardiogram PCP follow‐up with patient

	Abnormal	Normal	P Value
Total patients	52 (21.8)	187 (78.2)
Change in management?
Yes	18 (34.6)	8 (4.3)	<0.0001a
No	34 (65.4)	179 (95.7)
Referred to cardiologist?
Yes	13 (25.0)	7 (3.7)	<0.0001a
No	39 (75.0)	180 (96.3)
Another test ordered?
Yes	4 (7.7)	2 (1.1)	0.02a
No	48 (92.3)	185 (98.9)
Medication change
Yes	3 (5.8)	7 (3.7)	0.46
No	49 (94.2)	180 (96.3)
Plan for future echocardiogram?
Yes	3 (5.8)	6 (3.2)	0.41
No	49 (94.2)	181 (96.8)
Sent to ED?
Yes	1 (1.9)	0 (0.0)	N/Ab
No	51 (98.1)	187 (100.0)

Abbreviations: ED, emergency department; PCP, primary‐care physician.

Data are presented as n (%).

^aStatistical significance at P < 0.05.

^bN/A indicates that ≥1 cell contains a 0.

Among patients with abnormal results, there were 12 (11.2%) instances of valvular abnormalities, which included mitral regurgitation, mitral stenosis, aortic regurgitation, aortic stenosis, valvular/leaflet thickening, mitral valve prolapse, or bicuspid aortic valve. Other abnormalities identified (88.8%) included chamber enlargement or dilation, left ventricular hypertrophy, diastolic dysfunction, and dilation of aorta (Table 3).

Table 3.

Description of abnormalities observed among 52 patients with abnormal echocardiogram

Pathology	N	Total N (%)
Valvular abnormalities		12 (11.2)
Mitral regurgitation
Moderate	1
Severe	1
Mitral stenosis, severe	1
Aortic regurgitation, moderate	1
AS, severe	1
Valvular/leaflet thickening
Mitral valve leaflets	1
Aortic valve	1
Mitral valve prolapse	4
Bicuspid aortic valve	1
Other		95 (88.8)
Enlargement/dilatation
RA	17
LA	20
RV	10
LV	3
LVH	12
Diastolic dysfunction	23
Dilatation of aorta	10

Abbreviations: AS, aortic stenosis; LA, left atrium; LV, left ventricle; LVH, left ventricular hypertrophy; RA, right atrium; RV, right ventricle.

Of the 52 patients who were found to have abnormal studies, 34 (65.4%) patients had no change in management. Among the remaining 18 (34.6%) patients who experienced a management change, only 6 (11.5%) patients experienced a change in management that corresponded with the initial suspected diagnosis (type I). Primary indication for the order, result of the echocardiogram, type of decision made, and change in management for these 18 patients can be found in Table 4.

Table 4.

Indications, results, decision type, and description of management change among 18 patients (6 type I, 12 type II) with abnormal echocardiogram and change in management

Indication for Initial Order (AUC)	Patient	Echocardiography Results	Description of Change in Management
Type 1 Change in Management
Suspected cardiac etiology—general with TTE (2): Prior testing that is concerning for heart disease or structural abnormality, including but not limited to chest X‐ray, baseline scout images for stress echocardiogram, ECG, or cardiac biomarkers	1a	Moderately dilated LA	Sent to ED
	2a	Diastolic dysfunction, moderately dilated LA, mildly dilated RA, mildly dilated ascending aorta	Referred to cardiologist
	3a	Moderate LA enlargement, mild RA enlargement	Referred to cardiologist, future echocardiogram another test ordered
Murmur or click with TTE (34): Initial evaluation when there is a reasonable suspicion of valvular or structural heart disease	4a	Mild concentric LVH, moderate to severe valvular AS	Referred to cardiologist, another test ordered
HF with TTE (70): Initial evaluation of known or suspected HF (systolic or diastolic) based on symptoms, signs, or abnormal test results	5a	Severe LA dilatation, diastolic dysfunction, mild dilatation of RA	Medication change
	6a	Moderate LV dilatation, severe mitral regurgitation, moderate LA dilatation, moderate RA dilatation, mild RV dilatation	Referred to cardiologist
Type 2 Change in Management
Suspected cardiac etiology—general with TTE (1): Symptoms or conditions potentially related to suspected cardiac etiology, including but not limited to chest pain, shortness of breath, palpitations, TIA, stroke, or peripheral embolic event	1b	Mild concentric LVH, mild diastolic dysfunction	Medication change
Suspected cardiac etiology—general with TTE (2): Prior testing that is concerning for heart disease or structural abnormality, including but not limited to chest X‐ray, baseline scout images for stress echocardiogram, ECG, or cardiac biomarkers	2b	Mildly dilated RV	Referred to cardiologist
	3b	Diastolic dysfunction	Referred to cardiologist
	4b	Mildly dilated RV	Future echocardiogram
	5b	Mildly dilated ascending aorta	Referred to cardiologist, medication change
	6b	Mild mitral regurgitation, mild LV diastolic dysfunction	Referred to cardiologist, another test ordered
	7b	Bicuspid aortic valve, mild LV diastolic dysfunction	Referred to cardiologist
Lightheadedness/presyncope/syncope with TTE (7): Clinical symptoms or signs consistent with a cardiac diagnosis known to cause lightheadedness/presyncope/syncope, including but not limited to AS, HCM, or HF	8b	Thickening of mitral valve leaflets, probable mild bileaflet mitral valve prolapse	Referred to cardiologist, another test ordered
Murmur or click with TTE (34): Initial evaluation when there is a reasonable suspicion of valvular or structural heart disease	9b	Moderate thickening of the aortic valve	Referred to cardiologist
	10b	Moderate central aortic regurgitation	Future echocardiogram
	11b	Diastolic dysfunction	Referred to cardiologist
HTN with TTE (67): Initial evaluation of suspected hypertensive heart disease	12b	Mild concentric LVH, severely dilated LA, moderately dilated RA	Referred to cardiologist

Abbreviations: AS, aortic stenosis; AUC, appropriate use criteria; ECG, electrocardiogram; ED, emergency department; HCM, hypertrophic cardiomyopathy; HF, heart failure; HTN, hypertension; LA, left atrium/atrial; LV, left ventricle/ventricular; LVH, left ventricular hypertrophy; RA, right atrium/atrial; RV, right ventricle/ventricular; TIA, transient ischemic attack; TTE, transthoracic echocardiography.

These findings indicated the following:

1. Based on the AUC, PCPs appropriately ordered echocardiograms within WCHN's ambulatory primary‐care setting.

2. Pretest probability within this population was low, and as a result true positive findings were uncommon; thus, the value of the test was diminished, as were true interventions that occurred as a result of testing.

3. This suggested that interventions to refine guidelines for testing and provide feedback for physicians regarding their ordering habits may increase pretest probability, and thus the value of echocardiography in this setting.

4. DISCUSSION

The current investigation established that nearly 97% of the studies within WCHN's ambulatory primary‐care setting were appropriately ordered according to the AUC. Among the 239 patients studied, only 52 had abnormalities despite this appropriate use. Further, only 6 (2.5%) experienced a change in management that corresponded with the initial suspected diagnosis and subsequent echocardiographic findings. As the average cost of an individual echocardiogram to the healthcare system ranges between $1000 and $2000,16 for these patients, approximately $478 000 was spent on echocardiograms that were appropriate but had a low likelihood of resulting in a patient management change. The diagnostic cost for the 6 patients who experienced changes in management associated with the initial echocardiographic order was approximately $79 666 per patient.

Bayes' theorem of conditional probability states that outcomes are influenced by their preceding conditions.17 If the condition is a disease‐modifiable diagnosis, then instance of a low post‐test observation (6/239) defines a low pretest probability of disease. Said another way, in this population of ambulatory primary‐care patients, there was a low risk of modifiable disease state pretest. Based on Bayes' theorem, even if the predictive accuracy of a test is high, the post‐test likelihood of disease is low if pretest likelihood is low. In addition, if high likelihood of a normal test outcome and low clinical event rate is identified, both the benefit and value of testing are likely to be low; therefore, appropriateness is not necessarily linked to pretest probability.

Evidence throughout the literature supports the assumption that noninvasive testing is recommended for use in patients with intermediate (10%–90%) pretest likelihood of disease state,18 such as the presence of coronary artery disease.19 Conversely, patient populations with low pretest probability of disease have historically displayed low echocardiographic yield.20 These observations create several implications for practice, and further indicate the importance of adequately defining guidelines, which have been regarded as foundational in the effort to improve value‐based care,21 that speak to likelihood of disease rather than appropriateness of testing and likelihood of outcome diagnosis management.

In conjunction with efforts to refine current guidelines, educational interventions aimed at reducing overuse of echocardiography within this setting may additionally prove to be an effective strategy.22 Results of the Echo WISELY study indicated that PCPs in ambulatory care who received an educational intervention, which included a video lecture and monthly feedback summarizing individual ordering behavior, ordered significantly fewer “rarely appropriate” transthoracic echocardiograms.22 In line with this approach and supported by the results of this study, investigators assert that PCPs may benefit from receiving continuous feedback regarding ordering behavior to assist in developing improved pretest likelihood, thereby reducing unnecessary cost.

4.1. Study limitations

Potential limitations to the current study include that the investigation was both retrospective and single‐center in nature, as all patient records were obtained from PCPs under the umbrella of Western Connecticut Medical Group in 2014. Patients were only included in the study if they had not received an echocardiogram during the 10 years prior. This limited the capacity of the results from the studied echocardiograms to provide information based on previous diagnostic findings, which made assessing severity and compensation impossible. Additionally, the sample population included within the study was small; however, it is worth noting that despite this limitation, the network's overall patient population is both heterogeneous and well‐studied.

Causality associated with change of care for patients with normal echocardiographic outcomes was not inherently obvious in each of the involved cases. As such, instances during which change of care was associated with the episode of care initially triggering the echocardiogram as opposed to the outcome of the test itself were potentially present; however, the population of patients in which this could have occurred was small (8/239).

It was not the intent of investigators carrying out this study to assess absence of disease and provide disease exclusion, nor can value of reassurance or cost of care be determined from the findings presented. It can, however, be determined that the diagnostic cost for the 6 patients who experienced changes in management associated with the initial echocardiographic order was approximately $79 666.

Although this figure falls within the intermediate value range ($50 000–$150 000/quality‐adjusted life year [QALY]) with regard to the incremental cost‐effectiveness ratios selected by the American College of Cardiology and the American Heart Association,23 it is worth noting that QALYs have a variety of recognized limitations. In addition, cost‐effectiveness thresholds assume that benefits received by all patients are equal without taking into account characteristics such as disease stage and severity, which may be imperative in the process to save cost and add value at the organizational level.24

Alternatively, the cost for the 12 patients who experienced incidental, or type II, changes in management was $39 833 per patient. Combined, the cost for the 18 patients who experienced type I or type II changes in management was $26 555 per patient. Although, when including patients for whom type II decisions were made, the cost per patient is significantly lowered, it is worth noting that this is not attributable to the effectiveness of echocardiography in managing patient treatment. The QALY analysis included in this study was intended to be based solely on the process assessed. As the purpose of the analysis was to determine the effectiveness of echocardiography in patient management when appropriately ordered, only patients who received a change in management specifically related to the initial order were identified to have benefited as a direct result of the echocardiogram.

Results of similar studies will vary based on population assessed. Supplemental investigations to involve additional institutions and patient populations are warranted to ensure that findings may be generalized and expanded to include a more robust assessment of pretest probability.

5. CONCLUSION

As healthcare spending is on the rise, modifiable expenses such as overuse of diagnostic testing must be explored to curb unnecessary use of healthcare funds. Although results from the current study indicate that echocardiography is most valuable among patient populations within the medium range of pretest probability, at present echocardiography is frequently utilized within the primary‐care ambulatory environment to test low‐risk patients, among whom post‐test likelihood of disease is low.

To ensure the greatest value and optimize use of diagnostic testing, it may be necessary to develop a more comprehensive set of guidelines to assist clinicians in the process to identify patient populations at low, moderate, and high risk for the presence of disease, and provide educational interventions, including feedback regarding individual ordering behaviors.

Conflicts of interest

The authors declare no potential conflicts of interest.

Bethge A, Penciu O, Baksh S, Parve S, Lobraico J, Keller AM. Appropriateness vs value: Echocardiography in primary care. Clin Cardiol. 2017;40:1212–1217. 10.1002/clc.22810