Abstract
Background
Patients with active cancer account for a growing percentage of all emergency department (ED) visits and have a unique set of risks related to their disease and its treatments. Effective triage for this population is fundamental to facilitating their emergency care.
Objectives
To evaluate the validity of the Emergency Severity Index (ESI; version 4) triage tool to predict ED-relevant outcomes among adult patients with active cancer.
Methods
We conducted a pre-specified analysis of the observational cohort established by the National Cancer Institute-supported Comprehensive ONCologic Emergencies Research Network’s (CONCERN) multicenter (18 sites) study of ED visits by patients with active cancer (N=1075). We used a series of X2 tests for independence to relate ESI scores with (a) disposition, (b) ED resource utilization, (c) hospital length of stay, and (d) 30-day mortality.
Results
Among the 1008 subjects included in this analysis, the ESI distribution skewed heavily towards high acuity (>95% were ESI 1, 2, or 3). ESI was significantly associated with patient disposition and ED resource use (p’s < 0.05). No significant associations were observed between ESI and the non-ED based outcomes of hospital length of stay or 30-day mortality.
Conclusion
ESI scores among ED patients with active cancer indicate higher acuity than the general ED population and are predictive of disposition and ED resource utilization. These findings demonstrate that the ESI is a valid triage tool for use in this population for outcomes directly relevant to ED care.
INTRODUCTION
Emergency department (ED) triage aims to first identify patients who cannot wait to be seen and then prioritize among patients who do not require immediate life-saving interventions. Triage tools, such as the Emergency Severity Index (ESI), are used to standardize the approach to this important and challenging process and to predict patient disposition and ED resource use. The concept for the ESI was established in 1998 and first codified in an Implementation Handbook in 2002 (1). Over the last two decades the ESI has been validated several times among general ED patient cohorts (2-4) and compared favorably to other emergency department triage systems (5, 6). Now in its fourth version, the ESI is the most commonly used ED triage system in the United States (7).
The ESI is used by triage nurses to categorize patients into 5 mutually exclusive levels – with 1 being the most acute and 5 being the least. Level 1 is for patients who require immediate life-saving intervention (e.g. apnea). Level 2 is for patients who should not wait for care based on assessment of risk, patient mental status, and severe pain/distress. Levels 3 – 5 are determined based on vital sign abnormalities and anticipated ED resource utilization. The ESI does not specify time standards for each level (1).
In addition to several validations of the ESI for the general ED population based on its ability to predict patient disposition and ED resource use (2-4), the validity of the ESI has also been demonstrated in several important subgroups including pediatrics (both ESI versions 3 (8) and 4 (9, 10)) and older adults (11) (ESI version 3) using the same outcome measures as the present study. These subgroup validations have been conducted in recognition of fundamental clinical differences among definable patient subgroups with distinct risk profiles. An important such subgroup for whom the ESI has not been previously validated is patients with active cancer .
Approximately 15 million people suffer from cancer in the United States (12). These patients comprise over 4% of ED visits nationally (12) and have a unique set of risks related to their disease and its treatments. In March 2015, the National Institutes of Health (NIH) convened a workshop co-sponsored by the National Cancer Institute (NCI) and the Office of Emergency Care Research entitled “Cancer and Emergency Medicine: Setting the Research Agenda”. Conference attendees – policymakers, practitioners, and advocates – recognized and prioritized the imperative to advance knowledge related to the emergency care of patients with cancer (13). A subsequent analysis of ED utilization among adult cancer patients based on the Nationwide Emergency Department Sample (2006-2012) highlighted the salient characteristics of cancer-specific ED presentations (including a high rate of inpatient admission) and affirmed the need for evidence based tools to facilitate emergency care for this large and growing patient population (12). Likewise, the Comprehensive ONCologic Emergencies Research Network’s (CONCERN) multicenter prospective cohort analysis of ED visits by patients with active cancer identified high rates of hospital admission and antibiotic administration, as well as a preponderance of symptom control issues related to pain, dyspnea, and nausea (14).
This study assesses the validity of the ESI (version 4) for adult patients with active cancer. We performed a pre-specified analysis of the database established by the NCI-supported CONCERN multicenter observational cohort analysis of ED visits by patients with active cancer in order to assess the predictive value of ESI (version 4) for the ED-relevant outcomes of resource utilization and disposition. Predictive validity for non-ED outcomes including hospital length of stay (LOS), and 30-day mortality were also assessed. Forthcoming references to ESI in this article will be to version 4 unless specifically stated otherwise.
MATERIALS AND METHODS
Design, Setting, Participants
Our research network conducted a prospective observational cohort study in 18 CONCERN-affiliated EDs from March 1, 2016 to January 30, 2017. Enrollment details for participating sites are presented in Table 1. The initial study protocol has been previously reported (14). All study sites were at academic centers with ED annual volumes averaging 71,886 (SD: 31,157) and admission rates averaging 30% (SD: 7.2%). All sites use the ESI triage tool and received approval from their Institutional Review Board to participate in this study. We followed STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) guidelines in the conduct of this study (15).
Table 1.
Participating Emergency Departments and Number of Subjects Enrolled
Note: A more detailed version of this Table has been previously published (14).
| Emergency Department | City, State | Number of subjects enrolled |
|---|---|---|
| The Ohio State University Wexner Medical Center | Columbus, OH | 70 |
| NYU Bellevue Hospital Center | New York, NY | 18 |
| NYU Langone Medical Center | New York, NY | 69 |
| UT MD Anderson Cancer Center | Houston, TX | 70 |
| Tampa General Hospital | Tampa, FL | 45 |
| Beaumont Royal Oak | Royal Oak, MI | 70 |
| Beaumont – Troy Hospital | Troy, MI | 42 |
| Brigham and Women’s Hospital | Boston, MA | 71 |
| Yale University | New Haven, CT | 72 |
| Allegheny General Hospital | Pittsburgh, PA | 22 |
| University of California, San Diego | San Diego, CA | 48 |
| Memorial Sloan Kettering Cancer Center | New York, NY | 71 |
| Beth Israel Deaconess Medical Center | Boston, MA | 60 |
| Saint Vincent Hospital | Worcester, MA | 70 |
| University of Washington | Seattle, WA | 70 |
| University of Rochester Medical Center | Rochester, NY | 71 |
| University of Utah | Salt Lake City, UT | 66 |
| University of Cincinnati | Cincinnati, OH | 70 |
We enrolled a convenience sample of adult (≥18 years) ED patients with active cancer. Triage, and associated ESI assignment, occurred prior to screening and enrollment in the study. Informed consent was obtained from all study participants. We defined active cancer, in accordance with prior published work, as a patient with ongoing anti-cancer therapy (radiation, chemotherapy, or other), known cancer recurrence or metastasis, or cancer-related symptoms (16). Exclusion criteria included: pregnancy, incarceration, psychiatric chief complaint, trauma response patients, non-English speaking, or inability to complete the survey for any reason. Among the 1075 participants in the parent study, 67 were excluded due to no ESI score being reported. No exclusions were made based on chief complaints regardless of whether the complaint was potentially related or unrelated to the patient’s cancer.
Procedures, Measures, Outcomes
Trained research staff administered a questionnaire to study participants confirming details of their active cancer, current and past cancer treatments, active symptoms, co-morbidities, and demographic information. We conducted structured chart reviews (including the ED, inpatient and outpatient records) at 30 days after enrollment. As previously reported (14), chart reviews were conducted by trained reviewers using a data dictionary and standardized electronic REDCap forms (17). We measured interrater reliability in a subset of subject charts and observed high reliability (14). Variables collected for the current analysis included demographics, ESI score, disposition, ED resource utilization, 30-day mortality, and hospital length of stay.
ESI guidelines require the estimation of number of resources only for those patients not categorized as ESI levels 1 or 2. Per ESI guidelines, we categorized resource utilization as none (ESI 5), one (ESI 4), or ≥ 2 (ESI 3) of the resources among the 7 possible categories of resources: 1) laboratory tests (blood, urine), 2) ECG, X ray, CT, MRI, ultrasound, angiography, 3) IV fluids, 4) IV, IM or nebulized medications, 5) specialty consultation (including admission to an inpatient service), 6) simple procedures (not exhaustively defined), and 7) complex procedures (not exhaustively defined) (1). Counting the expected number of resources beyond 2 is not necessary because any number greater than 2 remains an ESI 3. ESI guidelines lack perfect precision in determining resource use as exhaustive lists of simple and complex procedures are not provided. Moreover, within a single resource category multiple resources can be attributed. For example, although two separate blood tests count as one resource, a blood test and a urine test count as two resources. Likewise, two plain x-rays count as one resource but one x-ray and one CT scan count as two resources. In determining resource use we restricted identification of complex procedures to those that we recorded: intubation, central line placement, and CPAP/BiPAP. The study database did not include minor procedures and thus we did not include them in our determination of resource use.
The primary outcome measures for this study were patient disposition (discharge, observation level of care, regular inpatient admission, ICU/step-down) and ED resource use across ESI categories among ED patients with active cancer. Collectively, these outcomes reflect the predictive validity of ESI for this ED patient population and are consistent with prior validations of the ESI for both the general ED population (2-4) as well as specific sub-populations including pediatric (8-10) and geriatric (11) patients. Secondary outcomes include hospital length of stay (LOS) and 30-day mortality.
Data Analysis
We analyzed data using a series of X2 tests for goodness of fit andS X2 tests for independence with Yates corrections (when necessitated by cell sizes). We examined X2 tests for linear trend following an initial significant finding. We used one-way Analysis of Variance to compare ESI categories on hospital length of stay (for admitted patients). We used an a priori alpha level of 0.05 for all analyses.
RESULTS
Study enrollment and study subject characteristics for the cohort have been previously described (14). In brief, we enrolled 1075 study subjects from 2337 screened and 1562 eligible ED patients. We excluded 67 patients from this analysis due to no ESI being recorded resulting in a final N of 1008. Table 2 summarizes the characteristics of the study population. Of the patients retained, 10 (1%) had an ESI of 1, 430 (43%) had an ESI of 2, 542 (54%) had an ESI of 3, 20 (2%) had an ESI of 4, and 6 (1%) had an ESI of 5. Given the small cell sizes at the extremes of the scale and the conceptual meanings of scores, several of our analyses combine ESI 1 with ESI 2 and ESI 4 with ESI 5.
Table 2.
Characteristics of 1,008 emergency department patients with active cancer
| Characteristic | Frequency | % | |
|---|---|---|---|
| Gender | |||
| Male | 485 | 48% | |
| Female | 521 | 52% | |
| Unspecified | 2 | <1% | |
| Ethnicity | |||
| Hispanic/Latino | 73 | 7% | |
| Not Hispanic/Latino | 921 | 91% | |
| Missing | 14 | 1% | |
| Race | |||
| White | 798 | 79% | |
| Black/African American | 123 | 12% | |
| Other | 35 | 4% | |
| Missing | 51 | 5% | |
| Age | |||
| 18-64 | 528 | 52% | |
| 65-79 | 389 | 39% | |
| ≥80 | 91 | 9% | |
| ESI | |||
| 1 | 10 | 1% | |
| 2 | 430 | 43% | |
| 3 | 542 | 54% | |
| 4 | 20 | 2% | |
| 5 | 6 | 1% | |
| Disposition a | |||
| Discharged | 337 | 34% | |
| Hospitalized for observation level of care | 72 | 7% | |
| Admitted as inpatient to regular floor | 481 | 48% | |
| Admitted to stepdown/progressive care unit or ICU | 111 | 11% | |
| 30-day mortality | |||
| Yes | 61 | 6% | |
| No | 943 | 94% | |
Patients missing disposition values or transferred (n=7) excluded from ESI comparisons
ESI Distribution
The distribution of ESI scores in our cohort reflect the high acuity of this patient population. The distribution observed significantly differed from the published national distribution for all ED patients (18), X2 (4) = 1358.81, p < 0.001 (Table 3). The largest differences observed were in the proportion of ESI 2 patients, with 11% of general population rated a 2 compared to 43% in the current active cancer sample. ESI 4 patients were also considerably underrepresented among our cohort (2% vs 38% in the general population). Differences were also observed in comparison to a 2007 single-center validation study of ESI (version 3) for geriatric (>65 years) ED patients (11), X2 (4) = 207.86, p < 0.001. The primary difference in these two populations was again the underrepresentation of ESI 4 patients in active cancer sample (2% vs. 14% in the geriatric population).
Table 3.
Comparison of ESI distribution in the current study to the general ED population and geriatric ED population
Disposition
A full 5 X 4 analysis demonstrated a statistically significant relationship between ESI score and disposition, Yates X2 (12) = 89.29, p < 0.001 (Table 4), though sparseness in several cells limits the interpretability of this finding. When cells at the extremes were combined as described above, the associated X2 (ESI 1-2, 3, and 4-5 by disposition status) remained statistically significant, X2 (6) = 93.25, p < 0.001. We observed similar findings when we limited our analyses to only individuals in the two modal ESI categories (i.e., score of 2 or 3; n=963), with statistically significant differences in the disposition distribution between ESI 2 versus ESI 3 (X2 (3) = 71.97, p < 0.001). Across each comparison, a higher acuity ESI score was associated with higher acuity dispositions. Specifically, 20% of patients with an ESI score of 1 or 2 were discharged compared to 43% of patients with an ESI score of 3, and 68% of patients with an ESI score of 4 or 5 (discharge vs. admit comparison Yates X2 (4) = 69.066, p < 0.001). Furthermore, the rate of admission to a stepdown unit or ICU was more than twice as high in the ESI 2 group than the ESI 3 group or the ESI 4/5 group (admit stepdown or ICU vs. discharge or admit other comparison Yates X2 (3) = 18.74, p < 0.001).
Table 4.
ESI score by outcome frequencies and percentages or means and standard deviations
| ESI Classification | |||||
|---|---|---|---|---|---|
| Outcomes | 1 | 2 | 3 | 4 | 5 |
| Disposition | |||||
| Discharged | 2 (20%) |
85 (20%) |
231 (43%) |
16 (80%) |
3 (50%) |
| Hospitalized for observation level of care | 0 (0%) |
23 (5%) |
45 (8%) |
2 (10%) |
2 (33%) |
| Admitted as inpatient to regular floor | 7 (70%) |
248 (58%) |
223 (41%) |
3 (15%) |
0 (0%) |
| Admitted to stepdown/progressive care unit or ICU | 1 (10%) |
69 (16%) |
39 (7%) |
1 (5%) |
1 (17%) |
| ED Resource Utilization | |||||
| 1 | 0 (0%) |
4 (1%) |
12 (2%) |
3 (15%) |
1 (17%) |
| 2 or more | 10 (100%) |
426 (99%) |
530 (98%) |
17 (85%) |
5 (83%) |
| Hospital Length of Stay | 4.00 (2.00) |
5.46 (4.75) |
4.97 (4.58) |
2.60 (1.52) |
2.33 (1.52) |
| 30 Day Mortality | 1 (10%) |
27 (6%) |
32 (6%) |
1 (5%) |
0 (0%) |
ED Resource Utilization
All patients in the dataset used at least 1 clinical resource in the ED, and only 20 patients used a single resource. As such, our analysis compared ESI categories combined into (a) ESI 1 and 2, (b) ESI 3, and (c) ESI 4 and 5, on whether a patient utilized 1 or 2+ resources in the ED. This comparison was statistically significant, Yates X2 (2) = 19.76, p < 0.001, but cell sparseness limits interpretability. A X2 test for linear trend was significant (X2 = 13.02, p < 0.001), as rates of resource use consistently increased across categories.
Hospital Length of Stay
The mean LOS for patients who were not discharged home from the ED (i.e. dispositions of observation level of care, regular admission, and ICU/stepdown admission), was 5.18 days (median = 4.00). Among this group, there was no association between ESI and LOS, F (4, 645) = 1.24, p = 0.29. This null finding persisted when deceased patients were retained or excluded, as well as when a death was recoded as the longest possible LOS (i.e., 30 days).
30-Day Mortality
Using the above mentioned ESI combinations due to sparseness in cells, ESI score was not statistically related to 30-day mortality, X2 (2) = 0.28, p =0.28. In general, rates of mortality remained fairly even across ESI.
DISCUSSION
Patients with active cancer constitute a large and growing portion of the national ED population (12). As recently affirmed by the National Cancer Institute and the NIH Office of Emergency Care Research, there is a need for evidence-based tools to facilitate the emergency care of this complex and ill subgroup of ED patients. Our analysis of the validity of ESI for ED patients with active cancer found that ESI strongly predicts disposition and ED resource utilization. However, ESI was not significantly predictive of the non-ED outcomes of hospital LOS or 30-day mortality. This speaks to the discriminant validity of the measure, as it is designed to capture expected activity/clinical care in the ED and not comprehensively assess the care continuum across the length of hospital treatment.
The predictive validity of ESI for disposition and ED resource use among patients with active cancer supports its value as a triage tool for this population since these factors impact temporal prioritization among patients, bed/zone assignments, deployment of ED personnel and equipment, and management of ED-to-hospital patient flow. These outcomes mirror those used in past validations of ESI for general ED populations (2-4) as well as specific sub-populations such as pediatric patients (8-10) and geriatric patients (11). This demonstrated value of ESI aligns with the American College of Emergency Physician’s recommendation that EDs use a five-level triage scale (19).
Our finding that ESI was not predictive of hospital LOS or 30-day mortality may be related to selection bias due to ineligibility of patients unable to consent (i.e. too ill or in distress to consent). This lack of association, however, does not diminish the value of ESI as a triage tool in the ED since hospital LOS and 30-day mortality do not directly impact ED care. Some ESI validation studies have used ED LOS as an outcome of interest (11, 20). We did not include ED LOS for several reasons. First, the sickest patients often have shorter ED LOS as they are rapidly transferred from the ED to an ICU, procedure suite, OR, or morgue – thus inverting the expected association between severity of illness and time in the ED. Second, ED LOS is often largely driven by non-patient factors such as hospital occupancy, ED crowding, and other issues related to ED flow and boarding. We elected to include hospital LOS as a secondary outcome of interest because it is a surrogate indicator of illness severity that is less influenced by organizational factors.
Our validation of ESI for patients with active cancer is also the first prospective multicenter descriptive assessment of the distribution of ESI in this population. Our data demonstrate that, even with our eligibility criteria excluding some of the most ill or distressed patients (due to inability to provide consent), the ESI distribution of patients with active cancer reflects higher acuity than the general ED patient population and even the complex geriatric ED population. These comparisons underscore the unique presentations of ED patients with active cancer.
Limitations
There are several important limitations of our study. Our cohort was drawn from an academic and largely urban setting, thereby potentially limiting the generalizability of our findings. In addition, 21% of our total ineligible patients were excluded due to being too ill or otherwise unable to consent. This likely resulted in lower numbers of patients in the highest acuity ESI categories (i.e. ESI 1 and 2) and may bias our findings towards a weakened association between ESI and disposition, hospital LOS, and 30-day mortality.
As discussed above, resource utilization is not exhaustively defined in the ESI handbook (1). Nor would it have been practicable to collect data on every possible resource. Our approach to counting resources, therefore, was conservative and likely resulted in an underestimation of resource use. This strategy could also have resulted in an underestimation of the difference in resource utilization between subjects assigned ESI levels 3, 4, and 5 (assignment to ESI categories 1 and 2 is not made based on expected resource use). However, we suspect this impact was probably quite small as relatively few of the subjects in our cohort were categorized as ESI 4 or 5.
We did not include non-English-speaking patients in our study, again, potentially limiting the generalizability of our findings.
CONCLUSIONS
In summary, ESI is predictive of disposition and ED resource use for patients with active cancer. ESI should therefore be considered a validated triage tool for this population. Our analyses did not show that ESI is associated with hospital LOS (among admitted patients) or 30-day mortality. We encourage future research to consider ESI’s capacity to provide insights into other components of the care continuum that are linked to the ED care of patients with active cancer.
ARTICLE SUMMARY.
Why is this topic important?
Patients with active cancer constitute a large and growing portion of emergency department (ED) visits. Assessing the validity of the most commonly used triage tool, the Emergency Severity Index (ESI), for this patient subgroup is important because patients with active cancer have unique clinical characteristics related to their disease and treatments, making them distinct from the general ED population.
What does this study attempt to show?
This study evaluates the validity of the Emergency Severity Index (version 4) to predict Emergency Department-relevant outcomes among adult patients with active cancer.
What are the key findings?
The ESI is predictive of patient disposition and resource use in the ED. This study did not demonstrate that the ESI is predictive of the non-ED outcomes of hospital length of stay or 30-day mortality.
How is patient care impacted?
Based on this analysis, the ESI is predictive of ED-relevant outcomes (disposition and ED resource use) for patients with active cancer. It should be considered a valid triage tool for this patient subgroup.
Acknowledgments
GRANT FUNDING
This project did not receive grant funding from any source.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
COMPETING INTERESTS
The authors have no competing interest to declare.
REFERENCES
- [1].Gilboy N, Tanabe P, Travers D, Rosenau AM. Emergency Severity Index (ESI): A Triage Tool for Emergency Department Care Version 4 Implementation Handbook 2012 Edition. 2012 ed: Agency for Healthcare Research and Quality (AHRQ); 2011. [Google Scholar]
- [2].Wuerz RC, Milne LW, Eitel DR, Travers D, Gilboy N. Reliability and validity of a new five-level triage instrument. Acad Emerg Med. 2000;7:236–42. [DOI] [PubMed] [Google Scholar]
- [3].Tanabe P, Gimbel R, Yarnold PR, Kyriacou DN, Adams JG. Reliability and validity of scores on The Emergency Severity Index version 3. Acad Emerg Med. 2004;11:59–65. [DOI] [PubMed] [Google Scholar]
- [4].Eitel DR, Travers DA, Rosenau AM, Gilboy N, Wuerz RC. The emergency severity index triage algorithm version 2 is reliable and valid. Acad Emerg Med. 2003;10:1070–80. [DOI] [PubMed] [Google Scholar]
- [5].Chi CH, Huang CM. Comparison of the Emergency Severity Index (ESI) and the Taiwan Triage System in predicting resource utilization. J Formos Med Assoc. 2006;105:617–25. [DOI] [PubMed] [Google Scholar]
- [6].van der Wulp I, Schrijvers AJ, van Stel HF. Predicting admission and mortality with the Emergency Severity Index and the Manchester Triage System: a retrospective observational study. Emerg Med J. 2009;26:506–9. [DOI] [PubMed] [Google Scholar]
- [7].McHugh M, Tanabe P, McClelland M, Khare RK. More patients are triaged using the Emergency Severity Index than any other triage acuity system in the United States. Acad Emerg Med. 2012;19:106–9. [DOI] [PubMed] [Google Scholar]
- [8].Baumann MR, Strout TD. Evaluation of the Emergency Severity Index (version 3) triage algorithm in pediatric patients. Acad Emerg Med. 2005;12:219–24. [DOI] [PubMed] [Google Scholar]
- [9].Durani Y, Brecher D, Walmsley D, Attia MW, Loiselle JM. The Emergency Severity Index Version 4: reliability in pediatric patients. Pediatr Emerg Care. 2009;25:751–3. [PubMed] [Google Scholar]
- [10].Travers DA, Waller AE, Katznelson J, Agans R. Reliability and validity of the emergency severity index for pediatric triage. Acad Emerg Med. 2009;16:843–9. [DOI] [PubMed] [Google Scholar]
- [11].Baumann MR, Strout TD. Triage of geriatric patients in the emergency department: validity and survival with the Emergency Severity Index. Ann Emerg Med. 2007;49:234–40. [DOI] [PubMed] [Google Scholar]
- [12].Rivera DR, Gallicchio L, Brown J, Liu B, Kyriacou DN, Shelburne N. Trends in Adult Cancer-Related Emergency Department Utilization: An Analysis of Data From the Nationwide Emergency Department Sample. JAMA Oncol. 2017;3:e172450. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [13].Brown J, Grudzen C, Kyriacou DN, et al. The Emergency Care of Patients With Cancer: Setting the Research Agenda. Ann Emerg Med. 2016;68:706–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [14].Caterino JM, Adler D, Durham DD, et al. Analysis of Diagnoses, Symptoms, Medications, and Admissions Among Patients With Cancer Presenting to Emergency Departments. JAMA network open. 2019;2:e190979. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [15].von Elm E, Altman DG, Egger M, et al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. J Clin Epidemiol. 2008;61:344–9. [DOI] [PubMed] [Google Scholar]
- [16].Richards CT, Gisondi MA, Chang CH, et al. Palliative care symptom assessment for patients with cancer in the emergency department: validation of the Screen for Palliative and End-of-life care needs in the Emergency Department instrument. J Palliat Med. 2011;14:757–64. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [17].Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42:377–81. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [18].Rui P KK. National Hospital Ambulatory Medical Care Survey 2015: Emergency Department Summary Tables. In: https://www.cdc.gov/nchs/data/nhamcs/web_tables/2015_ed_web_tables.pdf, editor.2015.
- [19].American College of Emergency Physicians. Triage Scale Standardization. 2010, revised 2017. [Google Scholar]
- [20].Green NA, Durani Y, Brecher D, DePiero A, Loiselle J, Attia M. Emergency Severity Index version 4: a valid and reliable tool in pediatric emergency department triage. Pediatr Emerg Care. 2012;28:753–7. [DOI] [PubMed] [Google Scholar]