Higher Emergency Physician Chest Pain Hospitalization Rates Do Not Lead to Improved Patient Outcomes

Abstract

Background:

Wide variation exists for hospital admission rates for the evaluation of possible acute coronary syndrome (ACS), but there is limited data on physician-level variation. Our aim is to describe physicians’ rates of admission for suspected ACS and associated 30-day major adverse events.

Methods:

We conducted a retrospective analysis of adult emergency department (ED) chest pain encounters from January 2016 to December 2017 across 15 community EDs within an integrated health system in Southern California. The unit of analysis was the ED physician. The primary outcome was the proportion of patients admitted/observed in the hospital. Secondary analysis described the 30-day incidence of death or acute myocardial infarction (AMI).

Results:

38,778 patients encounters were included among 327 managing physicians. The median number of encounters per physician was 123 (IQR 82, 157) with an overall admission/observation rate of 14.0%. Wide variation in individual physician admission rates were observed (unadjusted 1.5%−68.9%) and persisted after case-mix adjustments (adjusted 5.5%−27.8%). More clinical experience was associated with a higher likelihood of hospital care. There was no difference in 30-day death or AMI between high- and low-admitting physician quartiles (unadjusted 1.70% vs 0.82% and adjusted 1.33% vs 1.29%).

Conclusions:

Wide variation persists in physician-level admission rates for ED chest pain evaluation, even in a well-integrated health system. There was no associated benefit in 30-day death or AMI for patients evaluated by high-admitting physicians. This suggests an additional opportunity to investigate the safe reduction of physician-level variation in the use of hospital care.

Chest pain remains the second most common reason for adult emergency department (ED) visits in the United States, accounting for over 7 million annual encounters.¹ The minority of these visits are related to acute coronary syndrome (ACS).² However, stratifying this cohort is challenging with high clinical and medico-legal stakes.^3,4

Patients with suspected ACS are often hospitalized to facilitate early noninvasive cardiac stress testing and to mitigate the risk of sudden death or dangerous arrhythmias related to coronary artery disease.^5,6 As a result, evaluation of suspected ACS is the top reason for short-stay inpatient and observation admissions,^7,8 accounting for over $3 billion in hospital costs per year.⁹

The ED serves an increasingly prominent role as the source of hospital admissions.^10–12 Substantial hospital-level variation has been reported for overall admission rates.^13–16 This variation has been attributed to a combination of patient, hospital, local/regional, and physician factors. Admission rates for chest pain evaluation have among the widest hospital-level variation.^9,17–20 In a national sample of nearly 29 million ED visits at 961 hospitals, risk-standardized admission rates for chest pain differed by nearly seven-fold between top and bottom quartile EDs.⁹

Relatively less has been described about physician-level variation for admission rates.^14,20–24 The literature specifically for chest pain evaluation is limited.^20,23,24 A recent study at a single academic center found significant variation in the ED admission rate for chest pain at the individual physician level with an adjusted odds ratio ranging from 0.42 to 5.8 as compared to the average admission rate.²⁴ Further, no studies have examined the relationship of physician-level admission variation to subsequent clinical outcomes.

Our objective was to describe physician-level variation in admission rates for ED patients evaluated for chest pain and associated 30-day major adverse events. Our study aims to specifically address these knowledge gaps by examining all ED evaluations for suspected ACS in a large volume, multi-center, community setting.

Methods

Study Design and Setting

We conducted a retrospective analysis of prospectively collected data from 15 community EDs between January 1, 2016 to December 31, 2017. Study sites were all part of Kaiser Permanente Southern California (KPSC). KPSC is an integrated health system providing health care for over four million health plan members. KPSC hospitals provide care to over 1 million ED patients per year (study sites ranging from ≈25,000 to 95,000 ED visits per year). Of these ED visits, approximately 80% are health plan members. Our data set allows us to track detailed information for our members’ in-network encounters, claims data for out-of-network encounters, and death information from the California State Death Registry. Because of the sensitive nature of the data collected for this study, requests to access the dataset from qualified researchers trained in human subject confidentiality protocols may be sent to Kaiser Permanente Southern California at iris@kp-scalresearch.org.

Study Participants

We included ED encounters for adult (≥ 18 years) health plan members who had both a troponin lab test and a chest pain diagnosis (Supplemental Methods). All sites use the same troponin lab assay (Beckman Coulter Access AccuTnI+3). We only included health plan members because we do not have accurate follow up information for our outcomes for non-member patients. We excluded patients with a do not resuscitate (DNR) or hospice status (n=4,104), who had an acute myocardial infarction (AMI) identified in the ED, were transferred from another hospital, or expired in the ED (n=762). To ensure a minimum case threshold among providers, we excluded patients who had a visit with an ED provider that was a non-KP physician or who had fewer than 5 total visits during the study period (n=23,660) (Figure 1).

Figure 1.

Flow diagram of the study cohort used for analysis.

Outcome and Covariate Measurements

Covariates included patient demographic information. Age, gender, and race data were obtained from administrative records, while education was proxied by the percentage of college-educated individuals at the census block level based on a patient’s home zip code. Clinical patient variables and physician data were similarly obtained by querying the structured electronic health records. Cardiac risk factors such as hypertension and diabetes were defined using the Elixhauser index codes.²⁵ The ICD-9 and ICD-10 codes used to define dyslipidemia, coronary artery disease (CAD), stroke, percutaneous coronary intervention (PCI), and coronary artery bypass graft (CABG) can be found in the Supplemental Methods. Body mass index (BMI) was measured from ED intake documentation or the most recently available visit, while smoking and family history of CAD were self-reported fields in electronic health records. Those with a history of PCI or CABG were considered to have had prior coronary vascularization. The KPSC medical center was recorded at the time of the ED encounter. We also included the troponin-ordering physicians’ age, years in practice, and specialty. Part way through the study period in May 2016, all study sites implemented decision support to capture HEART scores and incorporate this standard risk tool into routine ED care.^26,27

The primary outcome was admission to the hospital, which included patients kept under ED observation and admitted to the hospital under observation status. The secondary outcome was 30-day AMI or all-cause mortality from the date of the ED encounter (Supplemental Methods).

Statistical Analysis

Patient, visit, physician, and facility characteristics were summarized using means and standard deviations for continuous variables, and frequencies and percentages for categorical variables. Due to low levels of missingness in our continuous variables, for missing variables we used either the missing indicator approach for continuous variables or included an “Unknown” category for categorical variables. We assessed whether physician- or hospital-level variation in admission rates exist by using a likelihood ratio test for the variance of the corresponding random effects, as obtained from random intercept generalized linear mixed models estimated via adaptive quadrature in SAS to make the Empirical Bayes Estimates (EBE) obtainable.²⁸

We extended the aforementioned generalized linear mixed models to include patient, physician, and hospital characteristics. Following the approach described by Abulainen and colleagues,¹⁴ we then used the EBE of the physician-level random intercepts (via posterior means) to obtain case-mix adjusted physician admission rates (i.e. the rates each physician would have if they all saw the same “average” patients). Reliability estimates were obtained for both the unadjusted and case-mix adjusted physician admission rates, using a previously described method.²⁹ We then used a scatterplot of the unadjusted and case-mix adjusted admission rates by the physician-level rates of 30-day AMI or all-cause mortality, the latter of which was computed as the ratio of total number of outcomes across all patients seen by a given physician, and the physician’s total number of ED encounters. Generalized estimating equations (GEE) were used to model whether quartiles of crude and case-mix adjusted physician admission rates were associated with the risk of 30-day major adverse outcomes for individual patients. As this analysis was exploratory in nature, and because we had already adjusted for case-mix variables in the calculation of the latter set of rates, we did not perform covariate adjustment in this analysis to avoid potential overfitting problems. All analyses were conducted using SAS 9.3 software (SAS Institute Inc, Cary, NC). This study was approved by the KPSC Institutional Review Board.

Results

In the study period there were a total of 38,778 ED encounters seen by 327 physicians included in analyses. The median number of encounters per physician was 123 (IQR 82, 157). The overall unadjusted admission/observation rate was 14.0% (Table 1). The mean age of the admitted group was 67.0 years (SD = 13.6), compared to 56.4 years (SD = 16.2) in the discharged group. There were substantially more female ED patients seen in total (56.8%); however, males comprised 51.5% of the admissions and 41.8% of the discharges. Men had a higher adjusted odds of admission than women (OR = 1.31, 95% CI: 1.22–1.41) (Table 2). Unsurprisingly, patients who were admitted were also more likely to have greater comorbidities and clinical risk factors for ACS (Supplemental Table I).

Table 1.

Emergency Department patient and physician characteristics for adults assessed for possible acute coronary syndrome. Study sample is stratified by disposition type. Percentages under Discharged and Admitted represent row percentages. Percentages under Total represent column percentages.

Patient and Physician Variables	Discharged (N=33356)	Admitted (N=5422)	Total (N=38778)
Age, Mean (SD)	56.4 (16.22)	67.0 (13.56)	57.8 (16.30)
Age Categories, N (%)
18–46	9408 (95.6)	430 (4.4)	9838 (25.4)
47–54	5836 (91.5)	545 (8.5)	6381 (16.4)
55–64	7282 (86.2)	1161 (13.8)	8443 (21.8)
65+	10830 (76.7)	3286 (23.3)	14116 (36.4)
Gender, N (%)
Female	19406 (88.0)	2630 (12.0)	22036 (56.8)
Male	13950 (83.3)	2792 (16.7)	16742 (43.2)
Race, N (%)
Alaska Native/Pacific Islander	562 (83.5)	111 (16.5)	673 (1.7)
Asian	3205 (84.7)	580 (15.3)	3785 (9.8)
Black	5461 (88)	745 (12)	6206 (16.0)
Others	7427 (89.1)	913 (10.9)	8340 (21.5)
White	16701 (84.5)	3073 (15.5)	19774 (51.0)
Elixhauser Index25, Mean (SD)	2.4 (2.07)	2.5 (2.14)	2.5 (2.24)
0	4176 (96.1)	168 (3.9)	4344 (11.2)
1–2	10496 (92.5)	854 (7.5)	11350 (29.3)
3–4	7819 (87.2)	1146 (12.8)	8965 (23.1)
5+	10865 (77.0)	3254 (23.0)	14119 (36.4)
Cardiac Risk Factors, N (%)
Coronary Artery Disease	5919 (69.3)	2621 (30.7)	8540 (22.0)
Heart Failure	2581 (68.2)	1205 (31.8)	3786 (9.8)
Diabetes	8235 (78.4)	2267 (21.6)	10502 (27.1)
Hypertension	17133 (80.3)	4213 (19.7)	21346 (55.0)
Liver Disease	3108 (84.3)	578 (15.7)	3686 (9.5)
Peripheral Vascular Disorders	8391 (75.4)	2743 (24.6)	11134 (28.7)
Body Mass Index, N (%)
Normal	7306 (84.1)	1383 (15.9)	8689 (22.6)
Overweight	11233 (85.8)	1860 (14.2)	13093 (34.0)
Obese	14527 (87)	2166 (13)	16693 (43.4)
Missing	290	13	303
Smoking Behavior, N (%)
Active	2277 (87.7)	319 (12.3)	2596 (6.7)
Missing	976 (98.8)	12 (1.2)	988 (2.5)
Never	20527 (87.6)	2899 (12.4)	23426 (60.4)
Passive	195 (94.7)	11 (5.3)	206 (0.5)
Quit	9381 (81.1)	2181 (18.9)	11562 (29.8)
Physician Age, Mean (SD)	43.1 (7.27)	43.6 (7.11)	43.2 (7.25)
Physician Practice Years, N (%)
≥20 years	9212 (84.4)	1703 (15.6)	10915 (28.1)
≥12 and <20 years	11493 (84.5)	2105 (15.5)	13598 (35.1)
≥8 and <12 years	9016 (88.7)	1151 (11.3)	10167 (26.2)
≥0 and <8 years	3635 (88.7)	463 (11.3)	4098 (10.6)

Table 2.

Adjusted odds ratio estimates for hospitalization among ED patients evaluated for possible acute coronary syndrome.

Effect	OR	95% CI
Age, y
65+ vs 18–46	2.93	(2.57, 3.33)
55–64 vs 18–46	2.34	(2.05, 2.66)
47–54 vs 18–46	1.67	(1.44, 1.92)
Men vs women	1.31	(1.22, 1.41)
Race
Alaska Native/Pacific Islander vs white	1.12	(0.89, 1.42)
Asian vs white	0.91	(0.82, 1.02)
Black vs white	0.9	(0.81, 1)
Others vs white	0.95	(0.86, 1.04)
Coronary Artery Disease: No vs Yes	2.52	(2.33, 2.71)
Elixhauser score25
1–2 vs 0	1.46	(1.22, 1.75)
3–4 vs 0	2.01	(1.67, 2.41)
5+ vs 0	2.77	(2.31, 3.32)
Body mass index
Normal vs overweight	0.91	(0.84, 1)
Obese vs overweight	0.92	(0.84, 1)
ED visit weekday vs weekend	1.06	(0.99, 1.14)
ED arrival hour
6am–3pm vs 12am–5am	1.06	(0.94, 1.18)
4pm–11pm vs 12am–5am	1.22	(1.09, 1.36)
Medical center
1 vs 15	2.2	(1.62, 2.98)
2 vs 15	0.64	(0.47, 0.86)
3 vs 15	0.07	(0.05, 0.11)
4 vs 15	1.89	(1.38, 2.58)
5 vs 15*	N/A	(N/A, N/A)
6 vs 15	0.18	(0.12, 0.28)
7 vs 15	0.49	(0.34, 0.71)
8 vs 15	0.28	(0.2, 0.39)
9 vs 15	0.98	(0.71, 1.36)
10 vs 15	1.26	(0.89, 1.77)
11 vs 15	1.43	(0.98, 2.1)
12 vs 15	0.37	(0.28, 0.5)
13 vs 15	0.64	(0.46, 0.89)
14 vs 15	0.44	(0.31, 0.62)
Post-HEART vs Pre-HEART	0.83	(0.77, 0.89)

Abbreviations: CI = confidence interval. ED = emergency department. OR = odds ratio.

^*= Values too large; not shown due to small sample size for Medical Center 5 (see Supplemental Table I)

With regard to characteristics of the ordering physician, we find encounter-level comparisons demonstrate that more experienced physicians are more likely to use hospital care (Years of experience ≥ 20 = 15.6%, 12–20 = 15.5% versus 8–12 = 11.3% and 0–8 = 11.3%) (Table 1). Physicians in the lower quartile of admission rates tended to be slightly younger (mean age of 42.7, SD = 7.4) than those in the upper quartile (mean age of 44.7, SD = 7.2). They also tended to be less experienced, with only 52.1% of physicians in the lower quartile having more than 12 years of experience compared to 81.3% of those in the upper quartile (Supplemental Table II). When comparing physicians in the upper and lower quartiles, the differences in quartiles were attenuated when using the case-mix adjusted admission rates, with respective mean ages of 43.1 (SD = 6.9) and 43.2 (SD = 6.9) and percent with at least 12 years of experience of 66.1% and 70.6% (Supplemental Table III, IV and V).

Figure 2 shows the unadjusted and case-mix adjusted physician admission rates in the upper and lower panels, respectively. Among individual clinicians, unadjusted admission rates ranged from 1.5% to 68.9% (mean 16.6%, median 12.6%, IQR 8.7–21.8%) and adjusted rates ranged from 5.5% to 27.8% (mean 13.3%, median 13.0%, IQR 11.4–14.6%). These results demonstrate how the variability in physician admission rates is overestimated when not accounting for case-mix differences between physicians, but >20% variability still persists after statistical adjustments.

Figure 2.

Individual physician unadjusted and adjusted admission rates. The caterpillar plot illustrates 95% confidence intervals around point estimates.

Figure 3 displays scatterplots of the unadjusted and case-mix adjusted physician admission rates, plotted against the physician-level rates of 30-day all-cause mortality or AMI. The overall 30-day death or AMI rate was low (1.2%) and appeared to vary by the unadjusted lowest versus highest quartiles of physician admission rates (0.82% vs 1.70%), but not after case-mix adjustment (1.29% vs 1.33%). The correlation between 30-day death/AMI and the unadjusted and case-mix adjusted physician admitting rates were 0.37 and 0.058, respectively, which reflects the patterns observed in Figure 3. In other words, there was no difference in the rate of 30-day death or AMI between low- and high-admitting physicians. The GEE model results indicated that patients seeing physicians in the lower quartile of the crude admission rates were less likely to have a 30-day outcome compared to both the middle quartiles (OR = 0.64, 95% CI: 0.51–0.80) and the upper quartile (OR = 0.48, 95% CI: 0.36–0.64). However, after case-mix adjustment, those purported associations were no longer evident compared to either the middle quartiles (OR = 1.13, 95% CI: 0.92–1.46) or the upper quartile (OR = 0.97, 95% CI: 0.74–1.27).

Figure 3.

Rates of death or acute myocardial infarction (AMI) per encounter compared to individual physician admission rate. Each data point represents an individual physician. The area of the data point is directly proportional to the total number of sample patients that the physician evaluated during the study period.

Discussion

Our study of 38,778 ED chest pain encounters evaluated for suspected ACS found a mean case-mix adjusted admission rate of 13% with a range of 5.5%−27.7%. After controlling for case-mix, there was no association between physician-level admission rate and subsequent 30-day death or AMI. Together, this variation of 1.7-fold in admission rates between the 10^th and 90^th percentile physicians without associated decrease in adverse events questions the current ED management strategy resulting in hospital admission for many patients.

Physician-level drivers remain a substantial source of variation in ED admission decisions, even after accounting for regional- and hospital-level factors.^13,15,30,31 Individual attitudes towards risk tolerance and fear of malpractice are frequently cited drivers of variable practice patterns^23,32, despite the mixed evidence of their actual determination of physician behavior,^33,34 The reality is more complex. As Schriger and colleagues point out, “local custom, physician personality, training, skill, malpractice experience, and larger cultural trends are some of the reasons why probability estimates, risk estimates, admission thresholds, and actual behavior vary among physicians.”³³ As expected, this variation is greatest for ED visits for intermediate severity (rather than emergent conditions or minor illness/injury), including exacerbations of chronic disease.^15,35 Frequently, these involve clinical categories that are diagnostically ambiguous (such as chest pain) or lack either clinical practice guidelines or established clinical pathways that incorporate patient preferences and outpatient access into decision-making.¹⁷

In previous work, we have demonstrated that the implementation of a care pathway using HEART to risk-stratify patients with suspected ACS safely reduced downstream hospital admissions and noninvasive cardiac testing.³⁶ Nevertheless, we find substantial variation in admission practices among ED physicians within an integrated health system, even after case-mix adjustment. We found that older and more experienced physicians tended to use more hospital-based care for similar patients. Perhaps younger physicians have grown accustomed to risk-stratification tools during training, or perhaps more experienced physicians have seen more patients sent home with negative outcomes. This warrants further study to better understand this difference.

To our knowledge, this is the first study to examine the relationship between physician-level admission rate variation for chest pain and subsequent patient outcomes. One study by Cotterill and colleagues assessed a similar question at the hospital level for Medicare patients, finding wide variation in ED admission rates for chest pain (38% to 81% in the lowest and highest quintiles) and an estimated 3.6 fewer cases of 30-day AMI and 2.8 fewer 30-day deaths per 1,000 chest pain patients for the higher admission (more conservative) quintile.¹⁹

In contrast, in our sample we find no apparent association between high versus low case-mix adjusted admitting rates and 30-day outcomes of death and AMI. With such a low event rate, this may be due to our study not being powered to detect differences in major adverse events across individual physicians. However, the flat distribution of adverse events against varying admission rates shows no signal of an underlying trend. Another major difference with the Cotterill study is our respective study populations. Our sample includes a more diverse range of patient clinical characteristics, including age and comorbidities, but also likely more homogeneity with regard to hospital and community settings. Our findings are consistent with studies showing lack of benefit to patients of early noninvasive testing or hospital-based evaluation.^37–41

With the continued emphasis on alternative payment models and limiting health care costs, greater scrutiny will be placed on admission decisions, particularly from the ED. Visits of intermediate severity that lead to high variation present the opportunity for greatest potential savings.^9,35 However, understanding the source of this variation will be important. In general, high variation does not necessarily reflect health care overuse. For higher-risk but diagnostically challenging conditions such as chest pain, where greater variation may reflect individual risk tolerance and limited outpatient follow-up care, further work is needed to understand the downstream impact of different disposition decisions on patient outcomes and health care use.¹⁷ Our work suggests that there is additional opportunity to safely reduce physician-level variation in admission rates for chest pain. Furthermore, while our study focused on physician clinical decision making, further research is needed to determine if these results would extend to other ED providers such as nurse practitioners or physician assistants.

Another important practice change to consider is the role of high-sensitivity cardiac troponin (hs-cTn) assays which can help rapidly and safely risk-stratify patients and reduce unnecessary hospital admissions. Future research will evaluate physician-level admission variation following the implementation of hs-cTn.

Limitations of our study include the retrospective analysis and restriction to a cohort of patients who presented with chest pain, as opposed to other atypical symptoms that were also evaluated for possible ACS. Additionally, perfect attribution of the admission decision to the individual physician is a challenge, given the nature of the ED and patient care handoffs. Patients were linked to the physician who ordered the initial troponin who may not always be the person who made the ultimate decision on admission. Furthermore, non-ED physicians such as the patient’s primary care physician or a cardiologist may provide input that affected the disposition decision. Other unmeasured variables, such as a patient’s social situation, may contribute to admission variation. We also do not capture ED-based protocols for chest pain evaluation that use ED-only care and do not involve an observation or admission status. Future research may expand on different levels of care. Another limitation is that our study population may not be representative of different types of US health systems that may not be as well integrated and where patients may have more limited access to follow-up. Furthermore, our 14% overall admission rate is far lower than what is to be expected in nonintegrated, fee-for-service systems. For comparison, the average admission rate for the Medicare population is 63%.¹⁹ Our health system also has lower rates of “missed MI” (0.6%) than what has been reported in other settings at 2.1%.^3,27,42 Finally, our study design cannot demonstrate a causal relationship between admission rate variation and adverse event rates.

Conclusions

In this cohort of ED patient encounters where an initial ED evaluation revealed no AMI or elevated troponin (> 0.5 ng/ml), wide variation persists in physician-level admission rates for chest pain evaluation, even in a well-integrated health system. There was no associated benefit in 30-day death or AMI for patients evaluated by high-admitting physicians. Despite clinical care pathways aiming to safely reduce overall admissions for chest pain, our results suggest an additional opportunity to investigate the safe reduction of physician-level variation in the use of hospital care when evaluating patients with suspected ACS.

What is Known

• Wide variation exists for hospital admission rates for the evaluation of possible acute coronary syndrome (ACS), but there is limited data on physician-level variation.

What the Study Adds

• In a retrospective analysis of 38,778 adult emergency department (ED) encounters for chest pain seen by 326 managing physicians over a 2-year period across 15 community EDs within an integrated health system in California, we found an overall admission/observation rate of 14.0%.

• Wide variation in individual physician admission rates were observed (unadjusted 8.8%– 93.7%) and persisted after case-mix adjustments (adjusted 24.8%–79.2%).

• There was no difference in 30-day death or acute myocardial infarction between high- and low-admitting physician quartiles (unadjusted 1.56% vs 0.86% and adjusted 1.27% vs 1.39%).

• Wide variation persists in physician-level admission rates for ED chest pain evaluation with no associated benefit in 30-day death or AMI for patients evaluated by high-admitting physicians, suggesting an opportunity to investigate the safe reduction of physician-level variation in the use of hospital care.

Abbreviations:

ACS

Acute coronary syndrome

ED

emergency department

AMI

acute myocardial infarction

IQR

interquartile range

KPSC

Kaiser Permanente Southern California

DNR

do not resuscitate

PCI

percutaneous coronary intervention

BMI

body mass index

HEART

History-Ekg-Age-Risk factors-Troponin score

EBE

empirical Bayes estimates

GEE

generalized estimating equations

SD

standard deviation

OR

odds ratio

ICD

international classification of diseases