Hospital Determinants of Emergency Department Left Without Being Seen Rates

Abstract

Study objective

The proportion of patients who leave without being seen in the emergency department (ED) is an outcome-oriented measure of impaired access to emergency care and represents the failure of an emergency care delivery system to meet its goals of providing care to those most in need. Little is known about variation in the amount of left without being seen or about hospital-level determinants. Such knowledge is necessary to target hospital-level interventions to improve access to emergency care. We seek to determine whether hospital-level socioeconomic status case mix or hospital structural characteristics are predictive of ED left without being seen rates.

Methods

We performed a cross-sectional study of all acute-care, nonfederal hospitals in California that operated an ED in 2007, using data from the California Office of Statewide Health Planning and Development database and the US census. Our outcome of interest was whether a visit to a given hospital ED resulted in left without being seen. The proportion of left without being seen was measured by the number of left without being seen cases out of the total number of visits.

Results

We studied 9.2 million ED visits to 262 hospitals in California. The percentage of left without being seen varied greatly over hospitals, ranging from 0% to 20.3%, with a median percentage of 2.6%. In multivariable analyses adjusting for hospital-level socioeconomic status case mix, visitors to EDs with a higher proportion of low-income and poorly insured patients experienced a higher risk of left without being seen. We found that the odds of an ED visit resulting in left without being seen increased by a factor of 1.15 for each 10-percentage-point increase in poorly insured patients, and odds of left without being seen decreased by a factor of 0.86 for each $10,000 increase in household income. When hospital structural characteristics were added to the model, county ownership, trauma center designation, and teaching program affiliation were positively associated with increased probability of left without being seen (odds ratio 2.09; 1.62, and 2.14, respectively), and these factors attenuated the association with insurance status.

Conclusion

Visitors to different EDs experience a large variation in their probability of left without being seen, and visitors to hospitals serving a high proportion of low-income and poorly insured patients are at disproportionately higher risk of leaving without being seen. Our findings suggest that there is room for substantial improvement in this outcome, and regional interventions can be targeted toward certain at-risk hospitals to improve access to emergency care.

INTRODUCTION

Chronicled by the Institute of Medicine,¹ lay press,^2–4 and researchers,^5–9 a decrease in access to emergency departments (EDs) and services has strained the acute care system to its breaking point. Patients who leave without being seen represent the failure of an emergency care delivery system to meet its goals of providing care to those most in need. The proportion of annual hospital-level left without being seen visits is the amalgamation of all individual ED visitors’ decisions to leave without being seen or not and is often used as a marker of ED crowding and is associated with longer waits.^10,11 Previous studies suggest that a nontrivial proportion of left without being seen patients are seriously ill, require immediate evaluation, and are at risk of poorer outcomes.^11,12 As such, the proportion of visits resulting in left without being seen have been proposed by The Joint Commission (TJC) and Centers for Medicare & Medicaid Services as hospital quality indicators.^13,14

The proportion of left without being seen has increased dramatically in the past 15 years⁵ as strains in the emergency care system have mounted. Although it is reasonable to suspect that the burden of left without being seen has decreased disproportionately on vulnerable populations and the hospitals that serve them, the evidence for this is not clear.

Contemporary attempts to study left without being seen have been limited by the scarcity of data reporting it. Most of the pertinent literature focuses on the patient-level or operational determinants at individual (or single) hospitals.^{10–12,15–23} A survey of Los Angeles County hospitals performed 20 years ago identified some facility-level determinants of left without being seen, including longer wait times, a higher proportion of uninsured patients, and teaching status.²³ One national survey of left without being seen analyzed hospital structural factors but did not include hospital socioeconomic status case mix.²⁴ This dearth of information has hindered the ability of policymakers to understand the effect of crowding on vulnerable communities and to design system-level interventions to improve access to emergency care.

The goal of the current study was to assess the association of socioeconomic status case mix and other hospital characteristics with left without being seen rates for patients presenting to California hospital EDs during 2007.

MATERIALS AND METHODS

Theoretical Model of the Problem

We study left without being seen rates as a measure of impaired health care access, in which patients’ behavior represents a failed attempt at entering the health care system.²⁵ Although literature on access spans aspects of potential measures of accessibility (eg, socio-organizational and geographic), as well as individuals’ willingness to seek care, left without being seen is the end product of a system’s inability to provide services to patients who have self-identified as needing care in a way that traditional access measures²⁶ often do not detect.^27,28 These patients have overcome significant barriers such as transportation and distance to seek what they deem as urgent or emergency care, which therefore makes their decision to leave without obtaining care all the more poignant.

Properly targeting interventions at the systems level requires understanding of the structural characteristics of facilities associated with increased left without being seen rates. Our primary hypothesis was that hospital-level, socioeconomic case mix would be associated with probability of left without being seen. For example, hospitals that serve a high proportion of vulnerable socioeconomic status patients may be at higher risk of increased left without being seen rates for all their patients. We a priori studied case-mix average age, sex, minority status, insurance, and income, which have been previously shown to be associated with ED use.^23,29

Our second aim was to study hospital structural factors that could be associated with increased probability of left without being seen. The literature on left without being seen has focused on public or teaching hospitals,^10,12,30,31 leaving left without being seen unstudied across other hospital types. We hypothesized that trauma centers, teaching program affiliation, county ownership, and low operating margin would be associated with decreased access to care and higher left without being seen rates.

Study Design and Selection of Participants

This is a retrospective cohort study of all ED visits to acute, nonfederal hospitals in California in 2007. We excluded visits to hospitals without an ED (either basic or comprehensive) and visits to hospitals that were only partially open during 2007, consistent with a previous analysis of California ED capacity.³² We excluded the 26 hospitals that reported left without being seen incidence of 0% because manual review of the data suggested that these data entries likely reflect missing information.

This study was approved by the California Committee for Protection of Human Subjects, the University of California at Los Angeles Institutional Review Board, and the University of California at San Francisco Committee on Human Research. Hospitals with zero left without being seen incidence were excluded after manual review because they were not believed to be accurate, given the authors’ knowledge of these hospitals and after consultation with Office of Statewide Health Planning and Development staff because these measures are not audited.

Data Collection and Processing

The California Office of Statewide Health Planning and Development mandates standardized reporting on all ED visits to nonfederal hospitals in California. Appendix E1 (Figure E1, available online at http://www.annemergmed.com) illustrates how we created the analytic data file. We obtained nonpublic use Office of Statewide Health Planning and Development ED visit data for 2007 from 2 data sources to count all ED visits: those that did not result in admission (Emergency Department Data) and those that did (Patient Discharge Data). Hospital structural and financial measures were obtained from the 2007 annual Office of Statewide Health Planning and Development hospital financial and utilization reports. We also linked Office of Statewide Health Planning and Development data files by patient zip code to the 2000 US census.³³

Our primary outcome was whether an ED visit resulted in left without being seen. For each facility, the Office of Statewide Health Planning and Development hospital annual utilization file provides the number of patients who leave without being seen and gives the number of ED visits for each hospital in the study cohort. Both values are by self-report for each hospital.

All candidate predictors are facility-level variables. Facility socioeconomic status case mix was defined as the 2007 demographic composition of ED visitors for each facility. We aggregated visit-level data by hospital for age, sex, race, and insurance. Race was first dichotomized into white/non-Hispanic versus any minority for each patient; then the percentage minority was calculated for each hospital.

For all of our predictor variables except for race/ethnicity, there was less than 0.03% missing in each category: sex (0.01%), age (0.007%), insurance (0.03%), and income (0.02%). We had 2.87% of our visits missing race/ethnicity data. For hospitals with greater than 15% missing race/ethnicity data (n=10, or 3.8% of our data set), we found that the majority of missing data occurred in the Office of Statewide Health Planning and Development ED file. Because racial distribution by hospital is highly correlated between individuals discharged from the ED versus admitted to the hospital (correlation 0.95; Appendix E1, Figure E2, available online at http://www.annemergmed.com), missing race/ethnicity was imputed according to Patient Discharge Data records alone.

Insurance categorized as expected payer sources was initially classified into 5 categories: (i) Medicare (individuals with Part A or Part B Medicare, even if also partially covered by Medi-Cal), (ii) Medi-Cal (including individuals under Medi-Cal managed care plans), (iii) private insurance (eg, preferred provider organization insurance, Blue Cross/Blue Shield, exclusive provider organization, health maintenance organization, or other commercial insurance), (iv) uninsured/self-pay, or (v) other (individuals with Champus/TriCare, a Veterans Affairs Plan, Title V, Worker’s Compensation, other federal programs, or automobile or disability coverage). We created a dichotomous variable by combining uninsured/self-pay patients and Medi-Cal patients and combining the other 3 categories.

Patient income was unknown; we therefore substituted the Census 2000 median household income within the patient’s residence zip code and then calculated a hospital mean for these income values.

For hospital structural characteristics, we obtained hospital-level information on hospital trauma status (yes for Level I or II trauma designation; otherwise not), ownership (3 categories: county, nonprofit, for profit), teaching status (yes/no), and operating margin (percentage) from the 2007 annual Office of Statewide Health Planning and Development hospital financial and utilization reports. Operating margin was defined as net from operations divided by total operating revenue, and this variable has previously been used to identify financially distressed hospitals.³⁴

Primary Data Analysis

We examined summary measures of each predictor by quartile of left without being seen for hospitals with left without being seen incidence greater than 0; ANOVA was used to assess continuous variables and χ², categorical variables.

A detailed description and justification for our analytic methodology is presented in the Appendix E1 (available online at http://www.annemergmed.com). In brief, we created a visit-level methodology to appropriately model data from hospitals with different numbers of left without being seen visits and different numbers of annual visits. We fit hierarchical logistic random effects models with hospital random effects and with ED visits clustered within hospitals. The outcome was whether an ED visit resulted in left without being seen (yes/no), with “yes” coded as a 1 and “no” coded as a 0. All predictors were facility level, including hospital-level demographic summaries and hospital structural characteristics.

Because the source data report left without being seen rates separately from visit-level socioeconomic data, we created a theoretical visit-level data set with a binary outcome of left without being seen. For example, if hospital A had 10,000 ED visits and a 5% left without being seen rate, then the portion of the theoretical data set devoted to hospital A would include 500 (5%) entries with an outcome of left without being seen coded as 1 and 9,500 (95%) entries with outcome 0, indicating no left without being seen. All facility-level predictors for these 10,000 entries would be identical and represent characteristics of hospital A. This visit-level approach correctly accounts for differences in ED visits among the hospitals contributing data to this analysis. Appendix E1 (available online at http://www.annemergmed.com) describes the limitations of possible alternative approaches, including hospital-level analyses that use facility left without being seen rates as the outcome.

We included hospital random effects to account for clustering of ED visits within facility and to model interfacility variance that was not explained by the predictors. In exploratory work, we considered county random effects. Because these did not improve model fit, we did not include county random effects in our final analyses.

To assess our hypotheses, we fit our models in 2 steps. To evaluate the link between left without being seen and facility-level demographic summaries, our first model includes hospital-level socioeconomic status determinants. To assess the incremental importance of structural characteristics, our second model adds hospital structural factors.

In pilot work, we found that we could not successfully model our multilevel data using widely available software based on the frequentist paradigm (eg, SAS PROC GENMOD). Therefore, we used Bayesian estimation procedures to fit our models.^35–40 Because we did not have preexisting information about the possible relationships between predictors and outcomes, we used a noninformative prior distribution for Bayesian model fitting. Appendix E1 (available online at http://www.annemergmed.com) provides greater detail about the Bayesian framework and our methodology.

We estimated the odds ratios and 95% equal-tail-probability credible intervals. The interpretation of the credible interval is that these values have a 95% probability of containing the true odds ratio conditioned on the Bayesian model.

To illustrate the absolute effects of predictors, we calculated predicted left without being seen rates for hypothetical new hospitals with specific predictor values. For each continuous predictor, we predicted the left without being seen rate for that predictor at its 10th and 90th percentiles in the California hospital population under study, holding all other variables fixed at their means for continuous predictors and at their modes for categorical predictors. For categorical variables, we estimated the left without being seen rate at each level of the categorical variable, holding continuous variables at their means. We assumed the same hypothetical hospital random effect at the 10th and 90th percentiles when calculating the standard error of the difference.

We used the SAS package (version 9.2, SAS Institute, Inc., Cary, NC) for data management and cohort description and WinBUGS software (publicly available; version 1.4.3) for Bayesian modeling.⁴¹

RESULTS

We identified 288 hospitals in our data set that operated an ED during the entire 2007 study period. Twenty-six facilities reported a left without being seen rate of 0%; those 262 hospital reporting a nonzero left without being seen rate represented a total of 9.2 million ED visits. The mean ED census for the remaining 262 hospitals reporting a nonzero left without being seen rate was 35,034 visits per year (median 31,079; range 5,721 to 133,968) compared with 28,126 visits (range 844 to 75,017) for the 26 hospitals with 0% left without being seen. None of the predictors significantly differed from the sample analyzed, except for income with hospitals reporting 0% left without being seen having a higher average patient household income (Table 1). The median left without being seen rate in the study cohort was 2.6%, with a range of 0.06% to 20%, as shown in the Figure. Characteristics of the study sample are presented in Table 1.

Table 1.

Characteristics of study hospitals by quartiles of left without being seen, including left without being seen=0 (n=288).

	Quartiles of % LBWS				All, 0.06–20.4	Hospitals Reporting 0 LWBS Patients*
	Q1, ≤1.4	Q2, >1.4–≤2.6	Q3, >2.6–≤4.4	Q4, >4.4
No. of hospitals	66	65	65	66	262	26
Mean (SD)
Annual ED visits, thousands	32.6 (21.0)	30.2 (15.3)	36.8 (16.9)	40.5 (22.0)	35.0 (19.3)	28.6 (13.8)
Age, y, mean (SD)	39.9 (7.9)	39.9 (7.2)	37.9 (7.9)	37.6 (5.1)	38.8 (7.2)	39.2 (8.8)
Male, No. (%)	46.0 (2.7)	46.2 (3.4)	46.2 (2.8)	46.8 (4.5)	46.3 (3.4)	46.6 (2.8)
Minority, %†‡§	44.8 (23.9)	44.9 (24.7)	49.5 (23.2)	60.3 (21.5)	49.9 (24.1)	47.9 (24.9)
Income, $1,000‡§	52.9 (13.8)	47.8 (11.6)	45.8 (10.3)	42.0 (9.4)	47.1 (12.0)	52.4 (14.2)
Insurance: Med-Cal+uninsured, %†‡§	34.8 (18.5)	38.0 (20.2)	40.9 (17.1)	53.6 (17.5)	41.8 (19.6)	36.2 (19.1)
Operating margin, No. (%)‡||	1.5 (10.2)	1.0 (13.2)	0.5 (12.1)	−5.2 (20.8)	−0.7 (15.0)	−1.8 (35.6)
Hospitals, No. (%)
Teaching status§¶
Yes	0	3 (5)	3 (5)	15 (23)	21 (8)	1 (4)
No	66 (100)	62 (95)	62 (95)	51 (77)	241 (92)	25 (96)
Ownership§¶
County	1 (2)	3 (5)	1 (2)	12 (18)	17 (6)	0
For profit	14 (21)	18 (28)	15 (23)	9 (14)	56 (21)	7 (27)
Nonprofit	51 (77)	44 (68)	49 (75)	45 (68)	189 (72)	19 (73)
Trauma center||¶
Yes	3 (5)	7 (11)	9 (14)	20 (30)	39 (16)	3 (12)
No	63 (95)	58 (89)	56 (86)	46 (70)	223 (85)	23 (88)

LWBS, Left without being seen; Q, quartile.

^*Nothing significant between “All” and “Hospitals reporting 0 LWBS patients” except income. For annual ED visits, the nonparametric Wilcoxon test indicated no significance and was also nonsignificant when a log transformation was used.

^†For these categorical variables, the numbers reported are mean proportion and SD of the proportion.

^‡ANOVA (comparing among Q1 to Q4).

^§P<.001.

^||P<.05.

^¶Chi-square test (comparing among Q1 to Q4).

Figure.

Left without being seen percentage across hospitals. Grey bar counts hospitals that reported an exact zero left without being seen percentage.

There were strong bivariate associations between probability of left without being seen and hospital-level percentage of minority patients, income, and insurance type. Higher income and higher operating margins were associated with a lower probability of left without being seen, whereas greater proportions of minority and more Medi-Cal/uninsured patients were associated with a higher probability of left without being seen. Teaching hospitals, county-owned hospitals, and trauma centers were associated with a higher probability of left without being seen compared with nonteaching, non–county-owned, and nontrauma centers.

Model 1 in Table 2 presents adjusted relationships between hospital-level socioeconomic status profiles and left without being seen rates. Hospitals with a higher proportion of poorly insured residents experienced a higher probability of left without being seen (with each 10-percentage-point increase associated with an odds ratio of a patient leaving without being seen of 1.15; 95% confidence interval [CI] 1.07 to 1.24). EDs serving patients who reside in areas with higher incomes were associated with a lower probability of left without being seen, with each $10,000 increase in income associated with an odds ratio of 0.86 (95% CI 0.78 to 0.96).

Table 2.

Results of hierarchical mixed-effects regression models of probability of left without being seen with hospital-level socioeconomic and hospital structural characteristics.

	Model 1: Mean Hospital-Level SES Characteristics			Model 2: Mean Hospital-Level SES+Hospital Structural Characteristics
	Odds Ratio	95% Credible Interval		Odds Ratio	95% Credible Interval
Age per 10 y	1.18	0.99	1.43	0.97	0.80	1.18
Men per 10%	1.31	0.92	1.99	0.80	0.52	1.15
Minority per 10%	1.04	0.99	1.10	1.04	0.98	1.10
Income per $10,000	0.86	0.78	0.96	0.79	0.71	0.88
Insurance per 10%; Medi-Cal+uninsured	1.15	1.07	1.24	1.04	0.96	1.16
Trauma center				1.62	1.13	2.24
Teaching hospital				2.14	1.25	3.50
Ownership (ref=not for profit)
County				2.09	1.16	3.46
For profit				1.07	0.80	1.41
Operating margin per 10%				1.89	0.11	31.6

SES, socioeconomic status.

Model 2 includes hospital structural features in addition to hospital socioeconomic status case mix measures. The effect of hospital-level insurance status is attenuated, whereas the effect of income continues to be predictive of left without being seen. Model 2 also demonstrates that county-owned facilities (reference group: not-for-profit facilities), teaching hospitals, and trauma centers experience a much higher rate of left without being seen (odds ratio 1.62, 95% CI 1.13 to 1.99; odds ratio 2.14, 95% CI 1.25 to 3.5; and odds ratio 2.09, 95% CI 1.16 to 3.46, respectively). These findings suggest that hospital-level insurance profiles are associated with structural characteristics. In supplementary analyses, we found that facilities serving more heavily uninsured or Medi-Cal populations are more likely to be county owned, teaching hospitals, and trauma centers.

To aid interpretation of Model 2, we present predicted effects of each variable on left without being seen in Table 3, holding all other continuous variables at their mean and discrete variables at their mode. For hospitals serving areas at the lowest 10th percentile of income (median $33,150), the left without being seen rate is 3.4% compared with 1.7% in hospitals serving populations at the 90th income percentile (median $65,110). Similarly, the predicted left without being seen for visitors to county-owned hospitals is 5.0% compared with 2.5% for not-for-profit hospitals, 5.1% for teaching hospitals compared with 2.5% for nonteaching hospitals, and 3.9% for trauma centers compared with 2.5% for nontrauma centers.

Table 3.

Posterior predicted probability of left without being seen for model 2: hospital-level socioeconomic status and hospital structural factors.*

Continuous Predictor	10th Percentile	Predicted Probability of LWBS	90th Percentile	Predicted Probability of LWBS	Mean Difference in % LBWS (=90%−10%; [SD])
Age, y	31.3	2.52	45.8	2.43	−0.10 (0.43)
Men, %	43.0	2.67	50.1	2.27	−0.41 (0.53)
Minority, %	16.8	2.21	80.4	2.75	0.54 (0.70)
Income, $1,000	33.2	3.38	65.1	1.65	−1.73 (1.39)
Insurance, % (Med-Cal+uninsured)	15.5	2.24	68.7	2.77	0.53 (0.87)
Operating margin, %	−12.9	2.29	12.2	2.68	0.39 (0.47)

Binary Predictor	Predicted Probability of LWBS	Difference Assessed	Mean Difference (SD)
Base case (not for profit, nonteaching, nontrauma center)	2.47
Ownership		County: for profit	2.39 (2.28)
County	5.02	County: not for profit	2.55 (2.35)
For profit	2.63	For profit: not for profit	0.16 (0.47)
Teaching	5.13	Yes/No	2.66 (2.34)
Trauma center	3.93	Yes/No	1.46 (1.29)

^*Means of the continuous predictors: Age: 38.82 years; men: 46.31%; nonwhite: 49.88%; income (per $1,000): 47.14; insurance (Med-Cal+uninsured: 41.83%; operating margin: −0.67%. Binary predictors are set: ownership=not for profit, teaching=no, trauma center=no.

LIMITATIONS

The findings in this study may not be generalizable to the entire United States because it is limited to a geographically distinct state with a number of characteristics different from the rest of the United States. However, California represents 12% of the US population and 7% of the US hospital market⁴² and itself can provide insight to the scale of the problem. In fact, California has a lower number of ED visits per 1,000 population compared with national estimates (274 in 2007 in California compared with 401 nationally), as well as a reduced number of inpatient days (469 per 1,000 population in California and 645 nationally).⁴³ Our findings of left without being seen are therefore likely conservative, given that utilization of the ED and inpatient services is therefore already significantly reduced in California compared with the national average.

Second, federal hospitals and partially open EDs were not included in our analyses. Visits to federal facilities are not reported by the Office of Statewide Health Planning and Development, and we excluded partially open hospitals to be consistent with previous analyses of California ED capacity.^32,44 Thus, our analyses may not be generalizable to such facilities.

Third, we extrapolated income data from the zip code of the patient’s residence rather than the actual patient because these data were not available. Although we would not expect this to change our results significantly, it is possible patients of a certain income within that zip code had higher use of emergency services than others. Similarly, because homeless patients in some hospitals are assigned the zip code of the hospital, the income distribution of these hospitals may be somewhat affected. Other individual patient characteristics that were not considered in this study, such as triage acuity, may also affect left without being seen rates. This study focused on hospital-level characteristics, including socioeconomic status and hospital structural characteristics, as potential predictors of ED left without being seen. Unfortunately, the Office of Statewide Health Planning and Development data do not have reliable information about illness severity, and analysis of acuity is beyond the scope of our study. Future studies should assess whether illness severity is an explanatory pathway for the associations that we report.

Fourth, drawing patient-level conclusions with the results from this study should be done with caution at best because it remains unknown whether distributions of individuals who leave without being seen are similar across age, sex, race, and insurance status to those of individuals who stay within a particular hospital, and any conclusion along these lines could well suffer from the ecologic regression fallacy. Acquiring data that specifically show which individuals within a hospital actually leave without being seen would reveal more reliable patient-level trends in future studies.

Finally, left without being seen data in the Office of Statewide Health Planning and Development data sets are self-reported by hospitals. These data are not independently validated and may not be reliably measured across hospitals. We acknowledge that differential reporting by facility-level characteristics could be possible, therefore introducing bias into our findings. For example, poorer hospitals might more conscientiously report their left without being seen rates in an attempt to document need for increased resources, or perhaps poorer hospitals are less able to document well because of a lack of administrative resources. Our estimates, however, are similar to those reported from previous studies.^5,23 Because we excluded hospitals reporting zero left without being seen rates, as described in the “Materials and Methods,” we conducted a sensitivity analysis as suggested to include these hospitals and found that our central policy conclusions did not change: our finding that low-income patients are differentially affected by left without being seen is still robust.

DISCUSSION

We analyzed a large, statewide cohort of hospital EDs to identify variation and facility-level determinants of annual left without being seen rates. We found that there was a wide range of left without being seen rates and that visits to EDs serving a high proportion of poorly insured individuals and patients residing in areas with low income were associated with a higher probability of left without being seen. Hospitals with high proportions of such vulnerable populations are much more likely to be county owned, teaching hospitals, or trauma centers. It is difficult to say whether socioeconomic status profile or these structural characteristics are primary in the chain of events that leads to left without being seen; for example, the structural characteristic of being a trauma center could oblige redistribution of resources such as staff during traumas, leaving fewer staff to treat nontrauma patients. Yet the cluster of these characteristics clearly invites policymakers’ attention. Specifically, our findings demonstrate that left without being seen and impaired access to emergency health care disproportionately affects hospitals serving populations at highest risk for poor health outcomes.^12,45

There is growing literature indicating that vulnerable populations have reduced ED visit rates, impaired access to care, and worse health outcomes.^46,47 These findings provide further evidence that, when access is measured not as a process but as an outcome,²⁷ as we have done with left without being seen, the current health care market does not serve vulnerable populations equitably, particularly those who live in lower-income areas or are poorly insured.

Our analysis of hospital-level determinants of left without being seen adds to current literature to show that certain populations may be at higher risk for left without being seen because of the institutions that serve them. Stock et al²³ performed a convenience sample survey of ED directors at 30 Los Angeles County hospitals in 1990. Independent predictors of increasing rates of left without being seen included teaching status, estimated wait times, and proportion of uninsured patients. Trauma center designation was not predictive, and public ownership appeared to be related to reduced rates of left without being seen. The majority of studies on left without being seen concentrate on single-center, operational determinants^{11,12,15,16,48–51} or patient-level factors.^5,24 Most studies identify proxies of ED crowding, such as wait time, ED capacity, and number of patients in the waiting room, as predictors of left without being seen. Although these factors no doubt play a role, our study suggests that they may be informed by larger socioeconomic and structural factors as well.

The most likely and commonly proffered explanation for decreased access and increasing left without being seen rates has been hospital restructuring and market factors, including increased ED closures.^52,53 It is possible that hospitals in lower-income areas may be closing preferentially, leaving these vulnerable populations at higher risk of decreased access to care. Other studies point to the systems factors related to increases in wait times and ED crowding, suggesting that financial pressures often lead hospital administrators to preferentially admit elective (and better insured) patients over ED patients and that market mechanisms may not correct themselves in a way to benefit vulnerable population groups such as the poorly insured.^54–56

Our findings suggest that a number of policies could be important in reducing system-level disparities in access to emergency care for vulnerable populations. Workflow analyses to identify specific reasons behind the increased left without being seen rates at certain facilities compared with others would be a first important step. Other funding mechanisms could then follow; for example, increasing subsidies to hospitals providing a high proportion of uncompensated care. This could be done by targeting these at-risk hospitals with increased disproportionate share payments, or at least ensuring that reimbursements from public insurers to hospitals are sufficient to cover the costs of care. If not, these at-risk hospitals could find themselves in an even worse condition than before. Other potential policies include establishing preferential ambulance diversion policies for certain facilities, creating emergency medical systems policies to equitably distribute patients from poor payer mix neighborhoods to different hospitals, or even integrating left without being seen as an avoidable outcome or patient safety measure, as is already being proposed by groups such as TJC.¹³ The last recommendation would need to be implemented cautiously, however, to avoid unintended consequences of penalizing hospitals that serve these vulnerable populations.

Patients who leave without being seen from an ED are a glaring measure of impaired health care access. Their observed behavior represents individual attempts to enter the health care system without success. Our study provides descriptive data about left without being seen from a large statewide cohort of hospital EDs. We demonstrate wide variation in left without being seen rates across hospitals and that ED visits to hospitals with large proportions of low-income and poorly insured patients are at higher risk of left without being seen.

The increasing phenomenon of left without being seen patients differentially affects those at hospitals that tend to serve the most vulnerable. Real action and resources should be applied to address these disparities on a systems level.

APPENDIX E1

Hospital determinants of ED left without being seen rates.

THE DATA SET

Figure E1 shows the data sources used and methods in which the final database was compiled. The data from the Emergency Department Database from the California Office of Statewide Health, Planning, and Development is separated into 2 volumes for 2007. We compiled these to form the entire ED data set of 8,785,265 visits for 2007. We merged this with our census demographic data set to match people by zip codes and concatenated this data set with that of the inpatient, or Patient Discharge Database. This aggregated data set was then matched by hospital ID to the facility-level information from the utilization, case mix, and financial databases. Data from the Area Resource File were then added to complete the data set. Our final data set represents 288 hospitals, with 262 of them having nonzero left without being seen rates, representing 9.2 million visits.

A BASIC DISCUSSION OF OUR MODEL

Each ED visit is a single observation in our analysis. In a theoretical version of our data set, there is 1 line per ED visit. There are columns for the outcome left without being seen and for the various predictors. Suppose hospital i has N_i ED visits, with m_i left without being seen and N_i−m_i non–left without being seen visits. Then the data set will have N_i rows devoted to hospital i. There will be m_i lines for the patients who left without being seen, with a 1 in the left without being seen column and N_i−m_i lines with a 0 in the left without being seen column for those who stayed and were treated in the ED. Regrettably, the source data base with subject-level predictor variables is separately reported from left without being seen data. Because of this, we were not able to use subject-level predictor variables in our analysis. We aggregated subject-level predictor variables to the hospital level: predictors are identical for all N_i cases for hospital i. This means that our analysis is about hospital case mix and hospital characteristics and does not provide statements about whether individuals with the various characteristics (sex, race) are more prone to left without being seen.

When an outcome is a 0 to 1 failure/success outcome, the typical statistical model used is a logistic regression model, and that is what we used here as well. In our case, we treat left without being seen as 1 (success) and not left without being seen as 0 (failure). The names success/failure are generic within statistics and do not connote a value judgment. Because all predictors are hospital level, it is possible to aggregate the visits to the hospital level, and we can think of the m_i left without being seen visits of N_i total visits as a binomial random variable. (This is basically a theorem of mathematical statistics; it uses the concept of a sufficient statistic.) A binomial random variable is the number of successes m_i out of N_i trials. Either way of thinking of the data as (1) ED visit level with left without being seen 0 or 1, or (2) hospital level with number of successes m_i out of N_i trials at the hospital leads to identical statistical models because the covariates are identical for all ED visits within the hospital.

The left without being seen rate is the fraction R_i=m_i/N_i, often expressed as a percentage. It is an abbreviated description of our binomial outcome to say that our outcome is the left without being seen rate. However, we do in fact know the numerator m_i and denominator N_i of the rate and can model m_i as binomially distributed out of N_i trials.

Why not use left without being seen rate as the outcome? In essence we did using the logistic model. We use the logistic model to get the information in the data set specified correctly so that we do not under- or overvalue an individual hospital’s contribution to the inferences. Almost any simpler model that one could use will not model these data correctly.

If we directly analyze rate R_i=m_i/N_i as our outcome, then we misinterpret our data. Suppose we assume “a hospital-level analysis” in which we take left without being seen rate R_i as the outcome and our same predictors as predictors and run a least squares analysis (ie, normal analysis).

Consider 4 hospitals, A, B, C, and D, with 0, 0, 5, and 2,500 left without being seen visits and 10, 1,000, 10, and 5,000 total ED visits, respectively (Table E1).

Table E1.

Four hypothetical hospitals.

	Left Without Being Seen Visits	Total Visits	Left Without Being Seen Rate, %	95% CI, %
				Lower	Upper
A	0	10	0	0	26
B	0	1,000	0	0	0.3
C	5	10	50	19	81
D	2,500	5,000	50	47	53

If we conducted the analysis “at the hospital level” only, we would take the left without being seen rate in the fourth column as our outcome measure. Hospitals A and B have the same rate and contribute the same information to the analysis; they are weighted equally even though A has 10 total visits and B has 1,000 total visits. Similarly, hospitals C and D have the same rate outcome and are weighted equally in the analysis. But within each pair of hospitals, A and B or C and D, the total number of visits is very different: hospitals A and C have few visits, whereas B and D have many visits. We need our model to treat A and C as contributing little information to the analysis, whereas B and D contribute much information.

Now you might ask, why not then run a “weighted” analysis in which we weight cases according to total visits? And the answer to this is found in the fifth and sixth columns, which are 95% CIs for the underlying probability that a given ED visit at the hospital resulted in left without being seen. The 95% CIs for hospitals A and B are 1-sided exact 95% CIs (up to rounding error), whereas the intervals for hospitals C and D are the usual large sample +/−1.96×SE(), where is 5/10 or 2,500/5,000 and SE() is sqrt(×(1−)/n).

If we weighted the hospitals’ contributions solely by the number of ED visits, then hospital D would contribute more than hospital B because D has more visits. But the 95% CI for hospital D is much wider (0.53–0.47=0.06 wide), whereas hospital B’s interval is quite narrow (0.003–0=0.003 wide). What this means is that we know hospital B’s left without being seen rate more accurately than we know hospital D’s left without being seen rate. How can we get this information into our statistical analysis? The answer is, the model that we set out does this exactly for us, without us having to work at it.

Merely using weighting options in software is not sufficient to analyze these data correctly. In contrast, the model we fit, a hierarchical logistic regression, correctly models the contributions of these 4 hospitals to the analysis.

Another issue occurs when we have 2 hospitals with identical covariates. These hospitals still might have different probabilities of left without being seen. We included in our model a random hospital effect. This random effect adds several benefits to our model: (1) it allows 2 hospitals with identical covariates to have different probabilities of left without being seen at each visit; (2) it allows individual ED visits within a hospital to be correlated: the hospital is the same for all ED visitors in that hospital, and one hospital may subtly encourage left without being seen, whereas another hospital treating a similar mix of patients may do a better job controlling their left without being seen rate; and (3) it makes an allowance in the model for unmeasured hospital predictors.

THE BAYESIAN PARADIGM

The Bayesian paradigm offers an alternative to conventional frequentist statistics.^1–6 In the Bayesian framework, uncertainty about the true values of population parameters (eg, coefficients in a regression model) is summarized by a probability distribution. Before the data are seen, this probability distribution is specified by the analyst with what is called the prior distribution. The prior is an assumption; it is an input into the analysis. Bayes’ theorem provides the mechanism to update the preexisting prior beliefs about population parameters (ie, the prior probability distribution) with information from the observed data to obtain an updated uncertainty estimate of the parameters, ie, the posterior probability distribution. The posterior probability distribution is the conclusion of a Bayesian analysis. Bayesian model inferences are commonly summarized as a point estimate and SD or an interval containing the true parameter with some probability (eg, the 95% credible interval). For regression coefficients, a calculation to replace a 1-sided P value is in common use: the probability that the coefficient is negative, if the point estimate is positive. Doubling this gives a classic 2-sided P value, though it no longer retains a Bayesian interpretation.

Bayesian statistics do not require calculations to be performed conditional on unknown parameters; in contrast, the conventional frequentist paradigm conditions on population parameters. In both paradigms, population parameters are “fixed but unknown values.” Prior information and information from data are combined to estimate population parameters. Prior information may not contribute to conclusions, in which case we say the prior is vague, flat, or uninformative. Frequentist hypothesis testing uses the P value, which assesses the probability of observing a statistic as extreme or more extreme than the test statistic actually observed from the data under repeated sampling from a population in which the null hypothesis is true. The Bayesian approach would report directly the probability that the alternative hypothesis is true. Bayesian inference mimics clinical reasoning, and the interpretation of Bayesian intervals may be more intuitive to clinical-investigators than P value–based hypothesis testing.⁷ For example, Bayesian 95% credible intervals are statements about the unknown parameter in the current problem at hand. Classic 95% CIs are statements about the statistical procedure and not specifically about the current analysis at hand. Still, with large samples, Bayesian intervals can be given the classic interpretation as well.

As currently implemented, the Bayesian approach is often computationally intensive; it also offers practical advantages in the modeling of complex data. Bayesian parameter estimates are generated with Markov chain Monte Carlo methods. Briefly, Markov chain Monte Carlo algorithms generate model inferences through iterative sampling from the posterior density of the parameters. Compared with frequentist maximum likelihood estimation procedures, the Bayesian Markov chain Monte Carlo approach is very flexible and allows fitting of complex models to complex data structures, such as those with multiple levels of nesting and with complex correlation structures.

For this study, we used a Bayesian approach to data modeling. In pilot work, we found that we could not successfully fit our multilevel data by using widely available software based on the frequentist paradigm (eg, SAS PROC GENMOD) and an appropriate model. Because we did not have preexisting information about the possible relationships between predictors and outcomes, we used uninformative prior distributions.

In this study, we fit hierarchical random-effects logistic regression models with hospital-level random effects to our data.

MODEL SPECIFICATION

The primary unit of analysis is a single visit to a hospital ED. The outcome of this jth visit to hospital i is denoted y_ij, which is 1 if the subject left without being seen or 0 if not. The probability that y_ij=1 is denoted p_ij

$$P(Y_{ij}=1)=p_{ij}.$$

We adopt the usual logistic regression model for p_ij, with a hospital-level random effect. Define the logit function logit(x)=log(x/(1−x)); then the logistic random effects model assumes

$$\text{logit}(p_{ij})=m_{ij}+b_i$$

where m_ij=x_ij’β is the sum of the regression coefficients β_k times predictors x_ijk where x_ijk is the value of the kth predictor for visit j in hospital i. The hospital random-effect term b_i is assumed normally distributed mean zero with variance D

$$b_i\sim \text{N}(0,D).$$

The purpose of including the random effect b_i in the model is to allow 2 visits with identical covariates but in different hospitals to have different probabilities of left without being seen rates; it also induces a positive association among visits within a hospital.

Because of the way the data come to us, there is no link between individual left without being seen y_ij and individual visit predictors x_ij. Thus, we aggregated visit-level predictors across hospital, and covariates x_ij and the probabilities p_ij are identical for all visits within hospital i. It is possible to write our model as

$$y_i\sim \text{binomial}(n_i,p_i),$$

where y_i=Σ y_ij is the total number of left without being seen at hospital i, n_i is the total number of ED visits for hospital i, and p_i=p_ij as mentioned because all predictors are the same for all visits within a hospital (i.e., x_ij=x_i).

The specific predictors we used include (1) aggregated SES factors: average age, sex (percentage male subjects), race (percentage white/non-Hispanic versus minority), average of median zip code income, insurance percentage uninsured/self-pay/Medi-Cal versus others), and (2) hospital-level structural factors: trauma center yes/no, 2 indicator variables for the tripartite indicator variable county hospital/for-profit hospital/not for profit, teaching status (yes/no), and operating margin (%). We illustrate 2 aggregated SES factors: for average age, we averaged all ages of patients within the facility; the same value of average age is used as the predictor for all patients within a facility. Race was first dichotomized into white/non-Hispanic versus any minority for each patient, and then the percentage minority (ie, percentage either nonwhite or Hispanic) was calculated for each hospital. This minority (%) predictor is used in the analysis; again, the same value of minority is used as the predictor for all patients within a particular facility.

PRIOR SPECIFICATION

The Bayesian approach requires prior distributions for all unknown parameters. We used uninformative or vague priors for all regression coefficients. The intercept was assumed to follow a normal distribution, with mean zero and variance 1,000. Each of the others is assumed to be distributed as a normal distribution, with mean zero and variance 10, which is relatively flat. The variance parameter D follows a uniform distribution on the range (0, 6); if the posterior of D were to have support near 6 (ie, gives probability to values of D near 6), it would be an indication of model misspecification.

COMPUTATION

The Markov chain Monte Carlo algorithm was run for 1,005,000 iterations, with the first 10,000 iterations excluded as a burn-in period. We stored every 50th sample, yielding a total of 19,900 realizations. We also ran multiple chains with different starting values and graphically examined the convergence.