Abstract
Objectives
While hospital length of stay (LOS) has been used as a surrogate injury outcome when more detailed outcomes are unavailable, it has not been validated. This project sought to validate LOS as a proxy measure of injury severity and resource use in heterogeneous injury populations.
Methods
This observational study used four retrospective cohorts: patients presenting to 339 California emergency departments (EDs) with a primary International Classification of Diseases, Ninth Revision (ICD-9), injury diagnosis (years 2005–2006); California hospital injury admissions (a subset of the ED population); trauma patients presenting to 48 Oregon EDs (years 1998–2003); and injured Medicare patients admitted to 171 Oregon and Washington hospitals (years 2001–2002). In-hospital deaths were excluded, as they represent adverse outcomes regardless of LOS. Duration of hospital stay was defined as the number of days from ED admission to hospital discharge. The primary composite outcome (dichotomous) was serious injury (Injury Severity Score [ISS] ≥ 16 or ICD-9 ISS ≤ 0.90) or resource use (major surgery, blood transfusion, or prolonged ventilation). The discriminatory accuracy of LOS for identifying the composite outcome was evaluated using receiver operating characteristic (ROC) analysis. Analyses were also stratified by age (0–14, 15–64, and ≥65 years), hospital type, and hospital annual admission volume.
Results
The four cohorts included 3,989,409 California ED injury visits (including admissions), 236,639 California injury admissions, 23,817 Oregon trauma patients, and 30,804 Medicare injury admissions. Composite outcome rates for the four cohorts were 2.1%, 29%, 27%, and 22%, respectively. Areas under the ROC curves for overall LOS were 0.88 (California ED), 0.74 (California admissions), 0.82 (Oregon trauma patients), and 0.68 (Medicare patients). In general, the discriminatory value of LOS was highest among children, tertiary trauma centers, and higher volume hospitals, although this finding differed by the injury population and outcome assessed.
Conclusions
Hospital LOS may be a reasonable proxy for serious injury and resource use among injury survivors when more detailed outcomes are unavailable, although the discriminatory value differs by age and the injury population being studied.
Keywords: injury severity, trauma, hospital stay, health services
Representative sampling and meaningful health outcomes are essential components of health services injury research. However, adding both components to a given project may not be logistically or financially feasible. Population-based sampling frequently requires data collection from multiple sources, approval from a variety of regulatory and governing bodies, and intensive data collection efforts, all of which can be cumbersome. Collecting detailed health outcomes can be equally resource-intensive and expensive. These challenges are particularly relevant with population-based emergency medical services (EMS) injury research,1–5 which requires linking the out-of-hospital phase of care (frequently with many different EMS providers from multiple agencies) to in-hospital and later phases of care. Recent efforts to develop a large, population-based, out-of-hospital trauma registry matched to hospital survival has illustrated such challenges.6 There is an inevitable trade-off in selecting the desired rigor of sampling design versus the volume of information to be captured (including outcomes), while operating with finite resources.
Injury measures of potential interest (e.g., injury severity, operative interventions, mechanical ventilation, blood transfusion, intensive care) may not be available from all hospitals, either due to research resource constraints or the type of hospital (e.g., nontrauma centers, nonteaching hospitals) from which data are being sought. Especially for projects involving large numbers of hospitals across broad geographic regions, simplifying the variables required as outcome measures may greatly increase the feasibility of such studies. One alternative to resolve this dilemma is to identify and validate surrogate measures of injury severity or trauma center need that are more readily available and less resource-intensive. Hospital length of stay (LOS) has been used as such an outcome measure in previous injury research,7–11 yet it has never been validated as a proxy for more meaningful injury outcomes. This study sought to validate LOS as a surrogate measure of injury severity and resource use among heterogeneous populations of injury survivors.
Methods
Study Design
This was an observational study using four retrospective injury cohorts representing diverse injury populations. The Oregon Health & Science University Institutional Review Board waived the requirement for informed consent and approved this project.
Study Setting, Populations, and Protocol
Patients in four distinct data sources were analyzed separately and independently to evaluate the utility of LOS as a surrogate measure for serious injury and resource use across a variety of injury populations.
California Patient Discharge and California Emergency Department Databases
The State of California Office of Statewide Health Planning and Development (OSHPD) maintains two unique databases for patients treated in California hospitals: the California Patient Discharge Database (admitted patients) and the California Emergency Department (ED) Database (visits not requiring admission). The California ED Database was first created in 2005. These databases capture all acute care visits (rather than just a sampling of visits), represent a large number of diverse hospitals, and include all ED visits (information lacking in most other state databases). These data sources were combined to generate a comprehensive sample of all injured patients (pediatric and adult) presenting to 339 California EDs from January 1, 2005, through December 31, 2006 (referred to in this paper as the “all ED” California data set). We restricted the sample to patients evaluated in acute care hospitals for nonscheduled visits with a primary International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis of injury (ICD-9-CM code 800–959), excluding those patients with foreign body (codes 930–939) or late effects of injury (code 905–909). Interhospital transfers with incomplete LOS or missing outcome information were excluded. Because we sought to evaluate the ability of LOS to identify survivors with serious injury or resource use, we excluded in-hospital deaths (mortality represents an adverse outcome regardless of LOS). The all-ED visit (including admissions) and admission-only data sets were analyzed separately to assess how altering the prevalence of disease (e.g., serious injury) in a given population may affect the use of LOS as an outcome measure.
Oregon State Trauma Registry
The sample obtained from the Oregon State Trauma Registry included consecutive injured children and adults meeting state trauma criteria (Table 1) and presenting to 48 acute care hospitals (Level 1–4) from January 1, 1998, through December 31, 2003. Many of these hospitals are located in rural areas. The majority of patients included in the Oregon State Trauma registry are identified prospectively in the out-of-hospital setting by EMS providers using standardized trauma criteria developed by the American College of Surgeons Committee on Trauma (ACSCOT).12 Patients meeting these criteria are included in the registry, regardless of disposition (admission, transfer, or discharge from the ED), injury type (e.g., patients with isolated hip fractures are included), or injury severity. Trained data abstractors at each hospital conduct structured chart reviews to collect the data, which are submitted at regular intervals to the Emergency Medical Services & Trauma Systems section of the Department of Human Services for Oregon. Standard procedures are in place to ensure reliable and consistent chart abstraction. For interhospital transfer patients, records from multiple acute care hospitals were matched using a combination of deterministic and probabilistic linkage (LinkSolv v.5.0, Strategic Matching, Inc., Morrisonville, NY), a process we have previously validated using similar data sets.13 Patients who left against medical advice (AMA), were direct admissions, had missing ED disposition, or were transferred to a second hospital without records available were excluded from the analysis. As with the California sample, only patients who survived to hospital (or ED) discharge were included.
Table 1. Out-of-hospital Criteria for Entry Into the Oregon Trauma System and Oregon State Trauma Registry.
| Physiologic criteria: |
| Systolic blood pressure < 90 mm Hg |
| Respiratory rate < 10, >29 breaths/min, or need for airway management |
| Glasgow Coma Scale score ≤ 12 |
| Anatomic criteria: |
| Flail chest |
| Two or more long bone (humerus/femur) fractures |
| Penetrating injury to the head, neck, torso, or groin |
| Amputation proximal to the wrist or ankle |
| Suspected spinal cord injury with paralysis |
| Mechanism of injury: |
| Extrication time > 20 minutes |
| Death of occupant in same vehicle |
| Ejection from vehicle |
| Discretionary criteria: |
| High-energy mechanism (fall > 20 feet, pedestrian vs. auto, motorcycle/ all-terrain vehicle /bicycle crash, rollover, passenger space intrusion) |
| Comorbid conditions (age < 5 or > 55 years, anticoagulation, medical illness, pregnancy, intoxication, hostile environment) |
| Paramedic discretion |
Oregon and Washington Medicare Injury Admissions
The third data set consisted of fee-for-service Medicare recipients aged 67 or older hospitalized in 171 Oregon and Washington hospitals with an injury diagnosis from January 1, 2001, through December 31, 2002. Qualifying injury diagnoses were identified from Medicare claims data using ICD-9 discharge diagnosis codes 800–959, excluding foreign bodies, complications, and late effects of injury. These records had been previously purchased from the Centers for Medicaid and Medicare Services. For patients with multiple injury hospitalizations, we included only the initial (index) hospitalization. When multiple acute care hospitalizations appeared to be linked to the same initial visit (i.e., interhospital transfers), records were matched where feasible and LOS was calculated for the entire duration of hospitalization. As with the other sources of data, the sample was restricted to patients surviving their hospital stay.
Measures
The primary predictor variable was hospital LOS, defined as the number of days from the date of admission to date of hospital discharge. Hour of admission was not available in these data, so patients admitted late on Day 0 (i.e., before midnight), and discharged the following calendar day (i.e., between 00:00 and 23:59), were counted as a 1-day hospital stay. Patients who were admitted and discharged on the same calendar day were considered to have an LOS of 0. Additional variables considered in the analysis included: age (years), age categories (0–14, 15–64, and ≥65 years), hospital trauma level (1–4), and total hospital annual admissions.
The primary outcome was a composite measure (dichotomous) of serious anatomic injury or in-hospital resource use. “Serious injury” was defined as Injury Severity Score (ISS)14 ≥ 16 or ICD-9-CM Injury Severity Score (ICISS)15 ≤ 0.90, depending on the availability of each measure. The cut-point in defining “serious” injury for ICISS was based on a mortality rate comparable to ISS ≥ 16.16 The ISS was calculated by trained data abstractors for the Oregon Trauma Registry, while ICISS was ascertained using ICD-9-CM diagnosis codes. Resource use consisted of major nonorthopedic surgical procedures in specific anatomic regions (brain, spine, neck, thorax, abdomen, vascular), prolonged ventilation (>96 hours), or blood transfusion, as ascertained by ICD-9-CM procedure codes. Similar definitions of “resource use” have been used in previous injury research.7,17–26 We combined measures of injury severity and resource use for the primary outcome because use of either measure individually has been shown to miss high-risk injured patients.27 We also assessed the outcomes of serious injury and resource use separately (rather than combined) in secondary analyses.
Data Analysis
We used descriptive statistics to characterize the demographics of each sample, including outcome measures. We then evaluated the discriminatory accuracy of LOS for identifying the injury outcomes using nonparametric receiver operating characteristic (ROC) analysis. The area under the ROC curve (AUC) was used to evaluate the diagnostic accuracy of LOS relative to the injury outcomes. The sensitivity and specificity of LOS cut-points were also compared within each sample to further evaluate the balance between the accuracy measures when selecting a specific LOS duration, as would be done when using LOS as a surrogate injury outcome.
Because we believed that patient age may modify the discriminatory value of LOS, we also stratified analyses by age categories. The age-based definition for children (0–14 years) was based on regional age-specific criteria for pediatric trauma patients in the Oregon Trauma System. Elders were defined as ≥65 years to approximate comparison to patients in the Medicare database and to isolate a population with a higher comorbidity burden and slower clinical recovery from injury. “Adults” were broadly defined by the age range above that of children and below that of elders (15–64 years). The ROC analysis was also stratified by level of trauma hospital (Level 1 and 2 tertiary trauma centers vs. Level 3, 4, and nontrauma hospitals), and total annual hospital admission volume (by quartile). These strata were used to assess whether the discriminatory value of LOS would vary by hospital resource level or experience base with admissions. Database management and analysis was performed using both SAS (SAS 9.1, SAS Institute, Cary, NC) and Stata (v.10, StataCorp, College Station, TX).
Results
The California sample included 3,989,409 injured patients who presented to 339 EDs and survived their hospital stay (or discharge from the ED) over the 2-year period (years 2005–2006). Of the California all-ED injury sample, 236,639 (5.9%) were admitted, 52,579 (1.3%) were seriously injured, and 44,937 (1.1%) fit our definition for resource utilization. There were 83,937 (2.1%) persons presenting to California EDs with either serious injury or resource use (the primary composite outcome). Of injured patients admitted to California hospitals, 67,551 (28.6%) were seriously injured or had resource use. Hospital LOS ranged from 0 to 652 days. The Oregon Trauma Registry included 23,817 patients presenting to 48 hospitals who survived to hospital or ED discharge over the 6-year period (years 1998–2003). Within this sample, 5,296 (22.2%) had serious injury, 2,931 (12.3%) required resource use, and 6,323 (26.6%) had serious injury or resource use. LOS ranged from 0 to 366 days. There were 30,804 injured Medicare patients hospitalized in 171 Oregon and Washington facilities who survived to hospital discharge over the 2-year period (years 2001–2002); 1,256 (4.1%) were seriously injured, 5,790 (18.8%) had resource use, and 6,639 (21.6%) had the primary composite outcome. Hospital LOS among Medicare patients ranged from 0 to 109 days. Detailed characteristics of the four injury samples are provided in Table 2.
Table 2. Demographics of Injured Patients Surviving to Hospital or ED Discharge*.
| California ED patients (including admissions), n = 3,989,409 | California Admitted Patients, n = 236,639 | Oregon Trauma Registry, n = 23,817 | Oregon and Washington Medicare, n = 30804 | |
|---|---|---|---|---|
| Median age, yr (IQR) | 28 (14–48) | 76 (48–85) | 34 (21–49) | 82 (77–88) |
| Age categories (yr) | ||||
| 0–14 | 878,690 (24.7) | 13,566 (7.1) | 2,302 (9.7) | — |
| 15–64 | 2,257,621 (63.4) | 97,874 (51.0) | 18,607 (78.2) | — |
| ≥65 | 424,258 (11.9) | 80,553 (42.0) | 2,876 (12.1) | 30,804 (100) |
| Female | 1,886,213 (55.0) | 84,995 (48.8) | 7,675 (32.2) | 21,088 (68.5) |
| Median hospital stay, days (IQR) | 0 (0–0) | 3 (2–5) | 2 (1–5) | 4 (3–6) |
| Serious injury | 52,576 (1.3) | 37,245 (15.7) | 5,296 (22.2) | 1,256 (4.1) |
| Major surgery | ||||
| Brain or spine | 6,731 (0.2) | 6,723 (2.8) | 1,106 (4.6) | 606 (2.0) |
| Thoracic | 4,191 (0.1) | 4,048 (1.7) | 307 (1.3) | 477 (1.6) |
| Abdominal | 5,428 (0.1) | 5,357 (2.3) | 1,082 (4.5) | 136 (0.4) |
| Neck | 5,64 (<0.1) | 486 (0.2) | 10 (0.1) | 2 (0) |
| Vascular | 4,384 (0.1) | 3,883 (1.6) | 525 (2.2) | 179 (0.6) |
| Blood transfusion | 29,234 (0.7) | 28,811 (12.2) | 734 (3.1) | 4,573 (14.7) |
| Ventilation > 96 hr | 3,319 (0.1) | 3,319 (1.4) | 26 (0.1) | 124 (0.4) |
| Resource use† | 44,937 (1.1) | 43,744 (18.5) | 2,931 (12.3) | 5,790 (18.8) |
| Resource use† or orthopedic surgery | 120,974 (3.0) | 117,878 (49.8) | 6,378 (26.8) | 20,816 (67.6) |
Values in parentheses are percentages unless otherwise noted.
ISS = Injury Severity Score; ICISS = ICD-9 Injury Severity Score; LOS = length of stay.
Serious injury was defined as ISS ≥ 16 (Oregon Trauma Registry) or ICISS ≤ 0.90 (California all-ED patients, California admitted patients, and Medicare patients). All percentages refer to column percentages and are based on observed (i.e., nonmissing values). In the California data sets, a proportion of certain demographic fields (e.g., age, sex) are set to missing to maintain the deidentified status of these data. Age was missing in 11% of the all-ED and 19% of admission-only California data, and sex was missing in 14% of the all-ED and 26% of admission-only California data. There were no missing values for hospital LOS, diagnosis, or procedure codes.
Major surgery, transfusion, or prolonged ventilation.
The ROC analysis assessing discriminatory accuracy of LOS for identifying patients with serious injury or resource use is demonstrated in Figure 1. For patients of all ages, ROC AUC varied by the outcome being assessed and between injury populations, ranging from 0.66 to 0.97 (Table 3). Age-based ROC analyses are demonstrated in Figures 2A (children), 2B (adults), and 2C (elders). Based on these results, LOS appears to have moderate to high discriminatory value in identifying children and adults with serious injury or resource use, although discriminatory accuracy varied by the outcome being assessed and prevalence of the outcome (e.g., all-ED visits versus admitted patients). Hospital stay had less discriminatory value among elders (particularly Medicare patients), although it was still a fair measure. Duration of hospital stay generally had the highest discriminatory value for resource use.
Figure 1.
ROC curves for the discriminatory accuracy of hospital LOS to identify patients of all ages with serious injury or resource use. Serious injury was defined as ISS ≥ 16 (Oregon Trauma Registry) or ICISS ≤ 0.90 (California all-ED patients, California admitted patients). Resource use was defined as any major surgery (brain or spine, neck, thoracic, abdominal, vascular), ventilation > 96 hours, or blood transfusion. ROC AUC for each of the data sets was 0.88 (California all-ED patients), 0.74 (California all admissions), and 0.82 (Oregon Trauma Registry). AUC = area under the curve; ISS = Injury Severity Score; ICISS = ICD-9 Injury Severity Score; LOS = length of stay; ROC = receiver operating characteristic.
Table 3. Accuracy Measures From ROC Analyses of Hospital LOS*.
| All ages | Children (0–14 yr) | Adults (15–64 yr) | Elders (≥65 yr) | |
|---|---|---|---|---|
| California ED patients | ||||
| Serious injury or resource use | 0.884 (0.883–0.885) | 0.669 (0.662–0.676) | 0.858 (0.856–0.860) | 0.913 (0.911–0.915) |
| Serious injury | 0.828 (0.826–0.830) | 0.643 (0.636–0.650) | 0.834 (0.831–0.836) | 0.789 (0.784–0.794) |
| Resource use | 0.974 (0.973–0.975) | 0.905 (0.892–0.918) | 0.966 (0.964–0.968) | 0.954 (0.953–0.955) |
| California admitted patients | ||||
| Serious injury or resource use | 0.736 (0.734–0.738) | 0.814 (0.803–0.826) | 0.739 (0.735–0.742) | 0.693 (0.689–0.697) |
| Serious injury | 0.659 (0.656–0.662) | 0.797 (0.783–0.811) | 0.705 (0.701–0.709) | 0.565 (0.557–0.574) |
| Resource use | 0.799 (0.797–0.802) | 0.875 (0.860–0.889) | 0.816 (0.812–0.820) | 0.722 (0.719–0.726) |
| Oregon Trauma Registry | ||||
| Serious injury or resource use | 0.824 (0.818–0.830) | 0.816 (0.794–0.838) | 0.837 (0.831–0.844) | 0.754 (0.733–0.774) |
| Serious injury | 0.812 (0.806–0.819) | 0.788 (0.763–0.812) | 0.828 (0.821–0.835) | 0.734 (0.711–0.756) |
| Resource use | 0.838 (0.830–0.846) | 0.848 (0.816–0.880) | 0.843 (0.834–0.851) | 0.808 (0.780–0.836) |
| Oregon and Washington Medicare patients | ||||
| Serious injury or resource use | — | — | — | 0.680 (0.673–0.687) |
| Serious injury | — | — | — | 0.656 (0.638–0.674) |
| Resource use | 0.687 (0.680–0.694) |
Data are reported as AUC (95% CI).
AUC = area under the curve; ISS = Injury Severity Score; ICISS = ICD-9 Injury Severity Score; LOS = length of stay; ROC = receiver operating characteristic.
Serious injury was defined as ISS ≥ 16 or ICISS ≤ 0.90. Resource use was defined as any major surgery (brain or spine, neck, thoracic, abdominal, vascular), ventilation > 96 hours, or blood transfusion.
Figure 2.
ROC curves for the discriminatory accuracy of hospital LOS to identify patients with serious injury or resource use, by age group. (A) ROC curves for overall LOS in children (0–14 years). ROC AUC for each of the data sets was 0.67 (California all-ED patients), 0.81 (California all admissions), and 0.82 (Oregon Trauma Registry). (B) ROC for overall LOS in adults (15–64 years). ROC AUC for each of the data sets was 0.86 (California all-ED patients), 0.74 (California all admissions), and 0.84 (Oregon Trauma Registry). (C) ROC for overall LOS in elders (≥65 years). ROC AUC for each of the data sets was 0.91 (California all-ED patients), 0.69 (California all admissions), 0.75 (Oregon Trauma Registry), and 0.68 (Medicare). The Medicare data set included patients ≥67 years. Serious injury was defined as ISS ≤ 16 (Oregon Trauma Registry) or ICISS ≤ 0.90 (California all-ED patients, California admitted patients, Medicare patients). Resource use was defined as any major surgery (brain or spine, neck, thoracic, abdominal, vascular), ventilation > 96 hours, or blood transfusion. AUC = area under the curve; ISS = Injury Severity Score; ICISS = ICD-9 Injury Severity Score; LOS = length of stay; ROC = receiver operating characteristic.
Accuracy measures (sensitivity and specificity) for specific LOS cut-points, by age group, are presented in Tables 4A (children), 4B (adults), and 4C (elders). When evaluated by age group, the LOS cut-points that balanced sensitivity and specificity increased with increasing age between children, adults, and elders. The balance between sensitivity and specificity was also different between primarily outpatient populations (California ED visits) and inpatient populations (California admissions, Oregon Trauma Registry, and Medicare patients). Among injury patients presenting to EDs, simply being admitted (LOS ≥ 1 day) was the strongest determinant of discriminatory accuracy for adults and elders. However, this was not the case for children; a substantial portion of whom had the primary outcome without hospital admission (or less than 24-hour admission).
Table 4. Sensitivity and Specificity Estimates for Identifying Patients With Serious Injury or Resource Use With Different Hospital LOS Cut-points, by Age Group.
| LOS (days) | Sensitivity (%) | Specificity (%) | |
|---|---|---|---|
| A. Children (0–14 yr) | |||
| California ED patients | ≥1 | 34.8 | 98.8 |
| ≥2 | 29.7 | 99.5 | |
| ≥3 | 24.6 | 99.8 | |
| ≥4 | 20.1 | 99.9 | |
| ≥5 | 15.9 | 99.9 | |
| California injury admissions | ≥1 | 98.4 | 9.4 |
| ≥2 | 84.2 | 60.9 | |
| ≥3 | 69.7 | 82.3 | |
| ≥4 | 57.0 | 90.4 | |
| ≥5 | 45.1 | 93.9 | |
| Oregon Trauma Registry | ≥1 | 97.1 | 8.9 |
| ≥2 | 85.4 | 63.5 | |
| ≥3 | 73.2 | 80.2 | |
| ≥4 | 61.5 | 87.6 | |
| ≥5 | 49.6 | 91.9 | |
| B. Children (15–64 yr) | |||
| California ED patients | ≥1 | 73.5 | 97.1 |
| ≥2 | 66.0 | 98.0 | |
| ≥3 | 57.4 | 98.7 | |
| ≥4 | 48.9 | 99.1 | |
| ≥5 | 41.0 | 99.4 | |
| California injury admissions | ≥1 | 97.9 | 7.8 |
| ≥2 | 87.8 | 36.2 | |
| ≥3 | 76.4 | 57.1 | |
| ≥4 | 65.1 | 71.9 | |
| ≥5 | 54.6 | 81.1 | |
| Oregon Trauma Registry | ≥1 | 98.2 | 11.1 |
| ≥2 | 92.6 | 50.3 | |
| ≥3 | 84.4 | 67.4 | |
| ≥4 | 76.1 | 77.7 | |
| ≥5 | 67.0 | 84.3 | |
| C. Elders (≥ 65 yr) | |||
| California ED patients | ≥1 | 91.6 | 86.2 |
| ≥2 | 89.3 | 87.5 | |
| ≥3 | 85.3 | 89.2 | |
| ≥4 | 74.4 | 92.4 | |
| ≥5 | 59.4 | 95.0 | |
| California injury admissions | ≥1 | 99.7 | 1.5 |
| ≥2 | 97.2 | 10.9 | |
| ≥3 | 92.8 | 23.0 | |
| ≥4 | 80.9 | 45.5 | |
| ≥5 | 64.7 | 64.1 | |
| Oregon Trauma Registry | ≥1 | 98.8 | 3.0 |
| ≥2 | 93.1 | 28.5 | |
| ≥3 | 85.9 | 42.2 | |
| ≥4 | 77.7 | 56.8 | |
| ≥5 | 69.5 | 67.4 | |
| Oregon and Washington Medicare patients | ≥1 | 99.8 | 0.7 |
| ≥2 | 97.1 | 10.6 | |
| ≥3 | 92.5 | 22.1 | |
| ≥4 | 80.4 | 45.0 | |
| ≥5 | 62.7 | 63.5 | |
LOS = length of stay.
Receiver operating characteristic curve analysis stratified by level of trauma hospital (Level 1 or 2 trauma centers vs. Level 3, 4, and non-trauma hospitals) suggested that LOS may have higher discriminatory accuracy for identifying patients with serious injury/resource use among tertiary trauma centers (California ED patients ROC AUC 0.92 vs. 0.86; California admissions 0.75 vs. 0.72; Oregon Trauma Registry 0.85 vs. 0.75). When stratified by total annual admissions (California hospitals only), the ROC AUC of hospital stay decreased with decreasing quartile of annual hospital admissions among ED patients (0.91, 0.89, 0.85, 0.79), but not among admitted patients (ROC AUC 0.74, 0.73, 0.73, 0.73).
Discussion
In this study, we demonstrate that hospital LOS is a reasonable proxy for important injury outcomes, although the discriminatory value appears to differ by age group, outcome prevalence in a population, and type of outcome being assessed. There may also be some differences in the ability of LOS to serve as a surrogate outcome by type of hospital (e.g., tertiary trauma center vs. nontertiary hospitals, annual admission volume). Selection of a specific LOS cut-point will also differ depending on the desired sensitivity for identifying patients with the outcomes of interest versus the costs of including patients without such outcomes (specificity).
In an ideal setting, detailed health outcomes would be available for all study subjects without the need for surrogate outcome measures. Unfortunately, this is often not the case. An investigator may be faced with the choice of restricting the population of interest to those cared for in facilities where detailed outcomes are readily available (e.g., trauma centers) or pursuing broader and more generalizable injury populations cared for in diverse health care settings that present more difficulty in tracking outcomes. The former has been typical of previous injury research, at the expense of introducing potential selection bias and reducing generalizability. Our results suggest that hospital LOS may be a reasonable proxy outcome measure when more detailed outcomes are not available, although misclassification costs must be considered. Validation of a surrogate outcome measure (i.e., LOS) that is feasible to capture from a wide variety of hospitals provides a potentially less expensive alternative to detailed chart review in injury research involving a large number of diverse institutions.
The discriminatory accuracy of LOS varied by age group and the injury population evaluated. These findings have several potential explanations. Hospital stay had the highest discriminatory value in children and adults, but was lower for elders. Among elders, such a finding may be reflective of a higher comorbidity burden, medical fragility, less tolerance to injury, and clinical practice variability in determining the duration of hospital stay relative to younger patients, any of which may increase variability (variance) in LOS, and thus reduce the utility of LOS as a surrogate outcome measure. In younger patients, with a lower prevalence of comorbidities and greater physical resilience, the injury admission and resulting hospital stay may be driven primarily by the injury itself and therefore have less variability. Children were also the only population where a portion of ED patients with the composite outcome were admitted for less than 24 hours.
While sensitivity and specificity (and therefore ROC AUC results) should not fluctuate based on the prevalence of disease in a population, several factors may explain the differences in results between injury populations. First, there is heterogeneity between injured outpatient (e.g., ED), inpatient (e.g., admitted patients), and trauma system populations. The injury populations used in this study were diverse, allowing greater generalizability of our findings and detailed assessment of how LOS may perform as a surrogate measure across different types of injured patients. Second, there is likely to be spectrum bias between the different injury populations, whereby certain patients (e.g., those included in the Oregon Trauma Registry) are further along the continuum of injury severity and physiologic insult, resulting in a higher acuity injury population with different LOS discriminatory value. In broad ED populations, whether or not a patient is admitted provides a strong discriminator for outcome and generally demonstrates greatest discriminatory value with any hospital stay beyond zero (except in children).
The finding that a subset of children with serious anatomic injury or resource use will not be hospitalized longer than 24 hours suggests that this population may be managed differently than older patients. However, this finding appeared to be primarily driven by short stays among children with serious anatomic injury without the need for intervention. It is also possible that many such children were listed under hospital (or ED) “observation” status (i.e., requiring a 1- to 2-day stay), which was recorded as zero duration of stay in the database. Because the ICISS calculation was developed from an inpatient population, it may have underestimated the probability of survival among injured children presenting to an ED (primarily outpatient population). That is, ICISS scores for many pediatric ICD-9-CM diagnosis codes in the California ED database may have been lower than the true survivability, resulting in a probability of survival lower than our threshold of 90%, and therefore declared “serious” injury, yet still managed as outpatients or with short observation stays. For children meeting a higher level of acuity (e.g., admitted children, those entered into a trauma system, or ED patients requiring major resource use), duration of hospital stay appeared to have good discrimination for identifying specific pediatric injury outcomes. These findings demonstrate that use of LOS as a surrogate outcome measure should be matched to the injury population under study by age and acuity, plus the type and prevalence of the injury outcome being approximated.
The trade-off in using LOS as a proxy measure for more detailed outcomes is misclassification. If LOS is used as part of a composite outcome for out-of-hospital trauma triage studies or projects assessing the distribution of “seriously injured” patients throughout hospitals in a regional trauma system, then the costs and prevalence of disease in the population being examined must be taken into account when selecting a cut-point. That is, simply selecting the LOS duration that maximizes both sensitivity and specificity (i.e., maximizing AUC) may not be ideal in all studies. An investigator may intend to evaluate factors necessary to capture all high-risk patients in a system and therefore favor high sensitivity at the expense of specificity when selecting an LOS cut-point. Conversely, if the “costs” of misclassifying a large number of minimally injured patients as seriously injured are large, then an LOS cut-point that favors specificity might be considered. The costs of misclassification should be considered in the context of potential impact on research results, interpretations, and the implications of such findings.
Limitations
Although we used independent data sources from diverse injury populations to validate the use of LOS as an outcome measure, the patients used in this analysis were from West Coast regions of the United States, and thus the findings may not be generalizable to other areas. In addition, the premise for this study was that both in-hospital mortality and hospital LOS are outcomes that can be more readily obtained than detailed anatomic injury measures and specific procedures, although capturing these outcomes still requires substantial resources and effort. We also attempted to evaluate potential differences in the utility of LOS by types of hospitals using the metrics of trauma level and annual admissions (surrogate measures for hospital size, volume, resources, and experience base); however, these strata may not have completely accounted for hospital-based variability in LOS. While we believe the broad base of hospitals represented in these data suggests that LOS is a reasonable surrogate outcome marker in population-based injury research, these findings may not be true for research isolated to a single hospital.
Selection of appropriate injury outcomes may vary depending on the study question and topic being assessed. We used a composite outcome of injury severity and resource use; however, this outcome may not be ideal for all types of injury research. We did not assess the ability of LOS to approximate functional outcomes among survivors, so these results may not apply to such measures. In addition, some investigators may elect to define measures of injury severity and resource use differently, which could alter the discriminatory accuracy of LOS for approximating such measures. We are unaware of a widely accepted, validated definition for resource use in injury research. A variety of definitions have been used in previous studies based on type of operative intervention, timing of interventions, intensive care unit (ICU) stay, and other treatments and procedures (e.g., blood transfusion and duration of mechanical ventilation).7,17–26,28 Some studies have combined resource use measures with ISS,20,22,24,29 while others have not. Our definition for resource use was based on the most commonly used criterion (major nonorthopedic surgery), plus transfusion or mechanical ventilation, which have been shown to be markers of both serious injury and the need for specialized clinical management to optimize outcomes.25,26,30–32 We used ICD-9-CM procedure codes to generate the measure of resource use, which did not allow for calculation of ICU duration of stay (a resource included in some previous composite outcome definitions). A sensitivity analysis that integrated ICU LOS ≥ 2 days to the composite outcome from the one database that included this measure (Oregon Statewide Trauma Registry) did not qualitatively change our findings. Nonetheless, without an accepted and validated definition for resource use, our findings may be different when using another definition of resource use. In addition, these findings only apply to patients who survive to hospital (or ED) discharge.
Hospital LOS was calculated as the difference in days between the dates of admission and discharge, although the actual time in-hospital using this definition could have varied by as much as one day. Although a limitation, this is similar to LOS information and calculations that are likely to be used in larger health services injury studies lacking more detailed outcomes (e.g., using administrative hospital data). There were also slightly higher numbers of major surgical procedures listed in the California all-ED sample when compared to the admission-only sample, suggesting either that a proportion of patients had surgical intervention and were discharged in less than 24 hours or that a hospital observation status was not counted as “admitted” in these data and therefore did not generate an LOS beyond zero (even if a portion of such patients stayed in the hospital 1–2 days). We believe that these possibilities would be unlikely to substantially alter our findings.
Conclusions
Hospital length of stay may be a reasonable proxy for serious injury and resource use among injury survivors when more detailed outcomes are unavailable. However, the discriminatory value of LOS differs by age group, prevalence of disease, acuity, and possibly by the types of hospital to which patients are admitted. The discriminatory value of LOS was highest for children and adults and was slightly lower for injured elders. Duration of hospital stay may provide a complementary outcome to mortality for population-based health services injury research.
Acknowledgments
This project was supported by a grant from the Robert Wood Johnson Foundation Physician Faculty Scholars Program.
Footnotes
An abstract of this project was presented at the 2009 Society for Academic Emergency Medicine annual meeting, New Orleans, LA, May 2009.
References
- 1.Committee on the Future of Emergency Care in the United States Health System, Board on Health Care Services Future of Emergency Care Series: Emergency Medical Services, At the Crossroads Institute of Medicine of the National Academies. Washington, DC: The National Academies Press; 2006. [Google Scholar]
- 2.Keim SM, Spaite DW, Maio RF, et al. Establishing the scope and methodological approach to out-of-hospital outcomes and effectiveness research. Ann Emerg Med. 2004;11:1067–73. doi: 10.1197/j.aem.2004.04.014. [DOI] [PubMed] [Google Scholar]
- 3.McLean SA, Maio RF, Spaite DW, Garrison HG. Emergency medical services outcomes research: evaluating the effectiveness of prehospital care. Prehosp Emerg Care. 2002;6(2 Suppl):S52–6. doi: 10.3109/10903120209102683. [DOI] [PubMed] [Google Scholar]
- 4.Maio RF, Garrison HG, Spaite DW, et al. Emergency medical services outcomes project I (EMSOP I): prioritizing conditions for outcomes research. Ann Emerg Med. 1999;33:423–32. doi: 10.1016/s0196-0644(99)70307-0. [DOI] [PubMed] [Google Scholar]
- 5.Spaite DW, Maio R, Garrison HG, et al. Emergency medical services outcomes project (EMSOP) II: developing the foundation and conceptual models for out-of-hospital outcomes research. Ann Emerg Med. 2001;37:657–63. doi: 10.1067/mem.2001.115215. [DOI] [PubMed] [Google Scholar]
- 6.Newgard CD, Sears GK, Rea TD, et al. The Resuscitation Outcomes Consortium Epistry-Trauma: design, development, and implementation of a North American epidemiologic prehospital trauma registry. Resuscitation. 2008;78:170–8. doi: 10.1016/j.resuscitation.2008.01.029. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Henry MC, Hollander JE, Alicandro JM, Cassara G, O'Malley S, Thode HC., Jr Incremental benefit of individual American College of Surgeons trauma triage criteria. Acad Emerg Med. 1996;3:992–1000. doi: 10.1111/j.1553-2712.1996.tb03340.x. [DOI] [PubMed] [Google Scholar]
- 8.Newgard CD, Rudser K, Hedges JR, et al. the ROC Investigators. A critical assessment of the out-of-hospital trauma triage guidelines for physiologic derangement. J Trauma. doi: 10.1097/TA.0b013e3181ae20c9. in press. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Norwood SH, McAuley CE, Berne JD, Vallina VL, Creath RG, McLarty J. A prehospital Glasgow Coma Scale score ≤ 14 accurately predicts the need for full trauma team activation and patient hospitalization after motor vehicle collisions. J Trauma. 2002;53:503–7. doi: 10.1097/00005373-200209000-00018. [DOI] [PubMed] [Google Scholar]
- 10.West JG, Murdock MA, Baldwin LC, Whalen E. A method for evaluating field triage criteria. J Trauma. 1986;26:655–9. [PubMed] [Google Scholar]
- 11.O'Rourke B, Bade RH, Drezner T. Trauma triage: a nine-year experience. Ann Emerg Med. 1992;21:680–7. doi: 10.1016/s0196-0644(05)82779-9. [DOI] [PubMed] [Google Scholar]
- 12.Committee on Trauma. Resources for Optimal Care of the Injured Patient. Chicago, IL: American College of Surgeons; 2006. [Google Scholar]
- 13.Newgard CD. Validation of probabilistic linkage to match de-identified ambulance records to a state trauma registry. Acad Emerg Med. 2006;13:69–75. doi: 10.1197/j.aem.2005.07.029. [DOI] [PubMed] [Google Scholar]
- 14.Baker SP, O'Neill B, Haddon W, Jr, Long WB. The injury severity score: a method for describing patients with multiple injuries and evaluating emergency care. J Trauma. 1974;14:187–96. [PubMed] [Google Scholar]
- 15.Osler T, Rutledge R, Deis J, Bedrick E. ICISS: an international classification of disease-9 based injury severity score. J Trauma. 1996;41:380–6. doi: 10.1097/00005373-199609000-00002. [DOI] [PubMed] [Google Scholar]
- 16.Diggs BS, Mullins RJ, Hedges JR, Arthur M, Newgard CD. Proportion of seriously injured patients admitted to hospitals in the US with a high annual injured patient volume: a metric of regionalized trauma care. J Am Coll Surg. 2008;206:212–9. doi: 10.1016/j.jamcollsurg.2007.08.019. [DOI] [PubMed] [Google Scholar]
- 17.Baxt WG, Jones G, Fortlage D. The trauma triage rule: a new, resource-based approach to the out-of-hospital identification of major trauma victims. Ann Emerg Med. 1990;19:1401–6. doi: 10.1016/s0196-0644(05)82608-3. [DOI] [PubMed] [Google Scholar]
- 18.Engum SA, Mitchell MK, Scherer LR, et al. Prehospital triage in the injured pediatric patient. J Pediatr Surg. 2000;35:82–7. doi: 10.1016/s0022-3468(00)80019-6. [DOI] [PubMed] [Google Scholar]
- 19.Qazi K, Kempf JA, Christopher NC, Gerson LW. Paramedic judgment of the need for trauma team activation for pediatric patients. Acad Emerg Med. 1998;5:1002–7. doi: 10.1111/j.1553-2712.1998.tb02780.x. [DOI] [PubMed] [Google Scholar]
- 20.Newgard CD, Cudnik M, Warden CR, Hedges JR. The predictive value and appropriate ranges of prehospital physiologic parameters for high-risk injured children. Pediatr Emerg Care. 2007;23:450–6. doi: 10.1097/01.pec.0000280508.90665.df. [DOI] [PubMed] [Google Scholar]
- 21.Zechnich AD, Hedges JR, Spackman K, Jui J, Mullins RJ. Applying the trauma triage rule to blunt trauma patients. Acad Emerg Med. 1995;2:1043–52. doi: 10.1111/j.1553-2712.1995.tb03148.x. [DOI] [PubMed] [Google Scholar]
- 22.Simmons E, Hedges JR, Irwin L, Maassberg W, Kirkwood HA. Paramedic injury severity perception can aid trauma triage. Ann Emerg Med. 1995;26:461–8. doi: 10.1016/s0196-0644(95)70115-x. [DOI] [PubMed] [Google Scholar]
- 23.Fries G, McCalla G, Levitt MA, Cordova R. A prospective comparison of paramedic judgment and the trauma triage rule in the prehospital setting. Ann Emerg Med. 1994;24:885–9. doi: 10.1016/s0196-0644(94)70207-1. [DOI] [PubMed] [Google Scholar]
- 24.Kane G, Engelhardt R, Celentano J, et al. Empirical development and evaluation of prehospital trauma triage instruments. J Trauma. 1985;25:482–9. doi: 10.1097/00005373-198506000-00002. [DOI] [PubMed] [Google Scholar]
- 25.Newgard CD, Hedges JR, Mullins R. Secondary triage: early identification of high risk injured patients presenting to non-tertiary hospital emergency departments. Prehosp Emerg Care. 2007;11:154–63. doi: 10.1080/10903120701205778. [DOI] [PubMed] [Google Scholar]
- 26.Newgard CD, Hedges JR, Stone J, Mullins RJ. Derivation of a clinical decision rule to guide the interhospital transfer of patients with blunt traumatic brain injury. Emerg Med J. 2005;22:855–60. doi: 10.1136/emj.2004.020206. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Newgard CD, Hedges JR, Diggs B, Mullins RJ. Establishing the need for trauma center care: anatomic injury or resource use? Prehosp Emerg Care. 2008;12:451–8. doi: 10.1080/10903120802290737. [DOI] [PubMed] [Google Scholar]
- 28.Steele R, Gill M, Green SM, Parker T, Lam E, Coba V. Do the American College of Surgeons' “Major Resuscitation” trauma triage criteria predict emergency operative management? Ann Emerg Med. 2007;50:1–6. doi: 10.1016/j.annemergmed.2006.09.007. [DOI] [PubMed] [Google Scholar]
- 29.Simon BJ, Legere P, Emhoff T, Fiallo VM, Garb J. Vehicular trauma triage by mechanism: avoidance of the unproductive evaluation. J Trauma. 1994;37:645–9. [PubMed] [Google Scholar]
- 30.Kneyber MC, Hersi M, Twisk JW, Markhorst DG, Plotz FB. Red blood cell transfusion in critically ill children is independently associated with increased mortality. Inten Care Med. 2007;33:1414–22. doi: 10.1007/s00134-007-0741-9. [DOI] [PubMed] [Google Scholar]
- 31.Sperry JL, Ochoa JB, Gunn SR, et al. Inflammation the Host Response to Injury Investigators. An FFP:PRBC transfusion ratio ≥ 1:1.5 is associated with a lower risk of mortality after massive transfusion. J Trauma. 2008;65:986–93. doi: 10.1097/TA.0b013e3181878028. [DOI] [PubMed] [Google Scholar]
- 32.Zink KA, Sambasivan CN, Holcomb JB, Chisholm G, Schreiber MA. A high ratio of plasma and platelets to packed red blood cells in the first 6 hours of massive transfusion improves outcomes in a large multicenter study. Am J Surg. 2009;197:565–70. doi: 10.1016/j.amjsurg.2008.12.014. [DOI] [PubMed] [Google Scholar]