The Association of Emergency Severity Index Score and Patient and Family Experience in a Pediatric Emergency Department

Abstract

The study aim was to determine the relationship between a patient's Emergency Severity Index (ESI) score and their or their family's response to the key performance indicator (KPI) question on the post-visit patient and family experience (PFE) survey. Retrospective review of patients presenting to the Pediatric Emergency Department between July 1, 2021, and June 30, 2022, who completed the KPI question on an associated post-visit survey. We performed univariate analyses on all candidate variables; multivariable linear regression identified independent predictors of KPI on the PFE survey. A total of 8136 patients were included in the study. Although ESI score was significantly associated with PFE in univariate analysis, this association was lost in the multivariable model. Independent associations were appreciated with race/ethnicity, time to provider, length of stay, and procedure performance during the visit. Although ESI is not independently associated with PFE in this study, its interaction with factors such as time to provider, length of stay, and procedure performance may be important for emergency department providers creating interventions to impact experience during low acuity visits.

Introduction

Patients and their families seek care in Pediatric Emergency Departments (PEDs) for emergencies real and perceived at all hours of the day. Multiple studies have investigated the motives of families seeking care in the PED.^1–5 Commonly, caregivers view their child's illness or injury with higher perceived urgency than providers.^2,5 Caregivers also seek empathy for a child's suffering and maintain expectations around being a responsible parent for their ill child. ¹ Overnight, an emergency department may be the only place to seek care.^2,3 The PED may be the closest site of care for the family and, in the event of a perceived emergency, viewed as the quickest way to have their child seen.^4,5 The PED offers a perceived immediate fix to a problem and an opportunity to have a sick child feel better versus having to delay treatment while awaiting an ambulatory appointment.^1,3

When a patient arrives at the PED, they are triaged using the Emergency Severity Index (ESI) system, which is a validated 5-level triage scale that applies a clinical assessment to the patient and assigns a triage level 1 to 5, with 1 denoting a patient who requires immediate emergency lifesaving medical care and 5 identifying the least severe patients with less urgent needs. The ESI considers the patient's illness, vital signs, and potential need for PED resources when assigning a triage level. The American College of Emergency Physicians and the Emergency Nurses Association support the use of the ESI as a tool in emergency care. ⁶ Most patients in the United States who seek emergency care are triaged via the ESI system. ⁷ Specifically in pediatrics, the current version of ESI has been shown to be a reliable tool in predicting hospitalization, PED length of stay, and resource utilization. ⁸ Overall, ESI is a crucial tool in the PED to determine which patients need expedited care. However, while prioritizing those patients with lower ESI scores, those with higher scores may wait longer for definitive care.

Wait times have been associated with overall patient and parent satisfaction in PED visits.^9–12 Wait times were identified as the second strongest predictor in overall patient satisfaction only behind how well physicians and nurses work together. ⁹ In a survey of patients and parents in a PED, long wait times were highlighted as an aspect of their care that was described as poor. ¹² Notably, children being treated had poorer experiences with wait time when compared to their parents. ¹² Making waiting a positive experience was noted to be uniquely important contributor to the patient and family experience (PFE) in the PED.^13,14 One survey of PED families noted that longer wait times were identified as a problem more often in the night or overnight shifts. Of note, most respondents in this study who completed the survey overnight were completed by parents of patients with lower acuity triage scores. ¹⁴

PFE surveys are frequently given after PED visits asking families to rate their experience. PFE scores have been shown to be inversely related to wait times and door-to-room times. ¹¹ Abbreviated wait times and optimized waiting room experiences can improve the PED experience for patients and families.^12,15 Given the connection between ESI and wait times, ESI scores may be correlated with PFE survey results. The purpose of this study is to identify any association between a patient's ESI level and their or their family's experience with that encounter, as identified through the key performance indicator (KPI) on our post-visit PFE survey.

Methods

Study Design and Setting

This is a retrospective cohort study of patient encounters resulting in discharge from either of 2 freestanding PEDs between July 1, 2021, and June 30, 2022, in which a PFE survey was completed after the visit. This single-center study was performed across 2 campuses in which freestanding PEDs are present. Our institution has >650 inpatient beds and sees approximately 100 000 PED visits annually, resulting in approximately 35 000 admissions annually. Our institution uses the National Research Corporation (NRC) Health survey platform for soliciting feedback from patients and families, with results accessible through the NRC Health Portal, which has been used for the last 3 years. Around 8000 survey responses are received annually following PED visits. Our institutional review board deemed this study exempt from review. Given this exemption and the minimal risk to included patients in this retrospective study, consent was not obtained for those included in this study.

Subject Selection

Inclusion Criteria. All encounters for patients <21 years of age who were treated in and discharged from either of our PEDs with a completed PFE survey after the visit were eligible for inclusion. This includes patients of all genders, races, and ethnicities, regardless of presenting complaint. Patient encounters were included if, in the post-visit PFE survey, there was a response provided to the KPI question: “How would you rate our facility?”

Exclusion Criteria. Encounters for patients 21 years of age or older, those who left prior to provider assessment and standard discharge, those admitted to the hospital, and those transferred to another institution for definitive care were excluded from this study. Patients were excluded if there was no corresponding PFE survey to their PED visit, or if there is no response to the KPI “rate facility” question. Lastly, a patient who visits one of our PEDs within 7 days of a prior PED visit and is discharged on both occasions will not be issued a PFE survey on their second visit and thus is not eligible for this study.

Data Collection

Preexisting data from 2 sources were used to complete this study. Data were collected on all patients who completed a PFE survey during the defined study period. Single data extraction from the NRC health portal generated a list of all patients eligible for inclusion in the study. The NRC health portal is not directly connected to our institutional electronic health record (EHR), and thus the contact serial number (CSN) from each PFE survey was used for a second data extraction from our EHR. The CSN is a unique identifier that ties data to the specific visit and is consistent across both platforms. We queried EPIC (Verona, WI, 2022) to pull clinical data related to each encounter to generate a final database, which included experience-level data cross-referenced with clinical elements from the EHR.

Data Collected

Data collected from the NRC Health Portal included: patient name, medical record number (MRN), CSN, encounter date, PFE survey response date, location of encounter, and responses to the following prompts: How would you rate our facility (KPI)? How likely would you be to recommend our emergency department to friends or family? I/my child was seen in a timely manner. Data collected from our EHR included: patient name, MRN, CSN, date of birth, encounter date, race, ethnicity, and ESI score. We also extracted whether a prescription was written at the time of discharge, if any lab work was ordered during the encounter, if any imaging studies were ordered during the encounter, or if a procedure had been performed during the visit. We chose these a priori based on the theory that the occurrence of any of these interventions during a patient encounter might impact PFE. We identified procedures in our EHR review as any intravascular catheter placement, ketamine administration, or laceration repair/abscess drainage documentation. We chose this specifically to align with our definition of procedure in the NRC health portal and felt that this identified our most common procedures and sedative used in nearly all other procedures requiring sedation. Lastly, we collected time to provider and length of stay. We defined time to a provider as the time of patient arrival to the time they were first seen by a provider (defined as a medical student, resident, Pediatric Emergency Medicine fellow or attending, staff pediatrician, or advanced nurse practitioner); we defined length of stay as the time of patient arrival to the time of patient discharge.

Data Analysis

We generated frequencies and proportions for all categorical variables and means and standard deviations for all continuous variables. Statistical associations with the continuous facility rating were assessed using the Wilcoxon rank-sum or Kruskal-Wallis tests for categorical variables and the Pearson correlation coefficient for continuous variables in the univariate analyses. For our primary outcome, we used multivariable linear regression to identify independent predictors of KPI on the PFE survey adjusting for other factors. Candidate variables included: sex, race, ethnicity, ESI score, prescription written, labs ordered, imaging ordered, procedure performed, time to provider, and length of stay. We did not include responses to the following prompts: How likely would you be to recommend our emergency department to friends or family; I/my child was seen in a timely manner. Our multivariable model included all objective clinical variables included in univariate analysis. SAS® version 9.4 was used for all analyses (SAS Institute Inc).

Results

There were 9795 PFE surveys completed during the study period, of which 8136 survey respondents answered the KPI question and were included in our analysis (Table 1). The mean (SD) age of patients included in our analysis was 7.7 (5.9) years, and approximately half were male (51%). Most respondents were White (66%), with Black (16%) respondents comprising the second highest race. Three-quarters of patients were triaged as ESI 3 or 4 (74%), and most did not receive prescriptions (62%), have labs or imaging ordered (64%), or undergo a procedure (84%).

Table 1.

Demographic Data for Patients Discharged From the Pediatric Emergency Department During the Study Period Who Completed the Post-Visit PFE Survey. ^a

Characteristic	Answered KPI, n = 8136	Did Not Answer KPI, n = 1659
Age (years, SD)	7.7 (5.9)	7.3 (6.0)
Sex (N, %)
Male	4130 (51)	863 (52)
Female	4006 (49)	796 (48)
Race (N, %)
Asian	245 (3)	112 (7)
Black	1317 (16)	407 (25)
Middle Eastern	42 (1)	19 (1)
Multiple	329 (4)	75 (5)
Other	96 (1)	53 (3)
Preferred category not available	496 (6)	192 (12)
Unknown	227 (3)	59 (4)
White	5383 (66)	742 (45)
Ethnicity (N, %)
Hispanic	895 (11)	344 (21)
Non-Hispanic	7153 (88)	1300 (78)
Refused/Unknown	87 (1)	15 (1)
ESI (N, %)
1	6 (0)	1 (0)
2	1732 (21)	293 (18)
3	3505 (43)	655 (40)
4	2508 (31)	605 (36)
5	381 (5)	104 (6)
Prescription written (N, %)
Yes	3068 (38)	706 (43)
No	5068 (62)	953 (57)
Labs ordered (N, %)
Yes	2935 (36)	639 (39)
No	5201 (64)	1020 (61)
Imaging ordered (N, %)
Yes	2716 (33)	472 (28)
No	5420 (67)	1187 (72)
Procedure performed (N, %)
Yes	1324 (16)	198 (12)
No	6812 (84)	1461 (88)

Abbreviations: KPI, Key Performance Indicator; PFE, patient family experience; ESI, Emergency Severity Index.

^a Value assigned at triage to indicate anticipated resource needs as well as illness or injury severity; Prescription written, labs ordered, imaging ordered, and procedure performed refer to any of those elements occurring during the patient encounter.

Table 2 presents the univariate analysis comparing PFE survey responses or specific clinical aspects of the encounter with responses to the KPI question on the post-visit PFE survey. Except for prescriptions, imaging, and procedure performance, all variables were statistically significant in univariate analysis (P < .05). ESI levels were significantly associated with the KPI response, with higher acuity patients (ESI 1 and 2) providing more positive responses than lower acuity patients (ESI 3 and 4). Notably, ESI level 5 patients provided scores similar to ESI level 2 patients; however, there were far less responses in this triage category than in ESI 2, 3, or 4. Both time to provider and length of stay demonstrated statistically significant negative associations with the KPI question on our PFE survey. This suggests that with longer time to providers and length of stays, responses to the KPI are less positive.

Table 2.

Univariate Association of PFE Survey Responses and Specific Aspects of Clinical Care With Facility Rating on PFE Survey. ^a

PFE or clinical aspect	How would you rate our facility? (mean [SD], N)	P
Sex
Male	8.5 (2.5), 4130	.035
Female	8.4 (2.6), 4006
Race
Asian	8.2 (2.7), 245	<.0001
Black	8.4 (2.6), 1317
Middle Eastern	8.8 (2.6), 42
Multiple	8.5 (2.6), 329
Other	9.1 (1.8), 96
Preferred category not available	9.3 (1.7), 496
Unknown	8.8 (2.3), 227
White	8.4 (2.5), 5383
Ethnicity
Hispanic	9.1 (2.0), 895	<.0001
Non-Hispanic	8.4 (2.6), 7153
Refused/Unknown	8.4 (2.6), 87
ESI
1	9.5 (0.8), 6	<.0001
2	8.8 (2.2), 1732
3	8.3 (2.7), 3505
4	8.5 (2.5), 2508
5	8.8 (2.3), 381
Prescription written
Yes	8.5 (2.4), 3068	.83
No	8.4 (2.6), 5068
Labs ordered
Yes	8.4 (2.6), 2935	.043
No	8.5 (2.5), 5201
Imaging ordered
Yes	8.5 (2.5), 2716	.84
No	8.4 (2.5), 5420
Procedure performed
Yes	8.5 (2.4), 1324	.55
No	8.5 (2.5), 6812
How likely would you be to recommend our emergency department to friends or family?
0-6	4.3 (2.8), 1526	<.0001
7-8	8.0 (1.1), 992
9-10	9.7 (0.8), 5618
I/my child was seen in a timely manner.
Yes	9.1 (1.8), 6605	<.0001
No	5.7 (3.2), 1529
Time to provider b	-0.3	<.0001
Length of stay b	-0.2	<.001

Abbreviations: PFE, patient family experience; ESI, Emergency Severity Index.

^a Facility rating is on a scale of 0 to 10. Value assigned at triage to indicate anticipated resource needs as well as illness or injury severity; prescription written, labs ordered, imaging ordered, and procedure performed refer to any of those elements occurring during the patient encounter.

^b Measure represents Pearson correlation coefficient between measure and question, “How Would you Rate our Facility?.”

Results from our multivariate analysis are displayed in Table 3. The point estimate represents the effect of the clinical aspect on KPI response in a linear regression model. There was no independent association between ESI score and the patient or family response to the KPI question on our post-visit PFE survey. There is a statistically significant association between race and response, driven by the “preferred category not available” group. Similarly, patients and families identifying as Hispanic race responded more positively to the KPI question on the post-visit PFE survey. Lastly, patients with procedures performed had an independent positive association with the KPI question response, while length of stay and time to provider had independent negative associations.

Table 3.

Multivariable Regression Model of Specific Aspects of Clinical Care Predicting KPI Response on PFE Survey. ^a

Clinical aspect	Estimate	P
Sex		.6
Male
Female	−0.03
Race		.02
Asian	−0.2	.1
Black	−0.04	.6
Middle Eastern	0.3	.4
Multiple	0.05	.7
Other	0.3	.3
Preferred category not available	0.5	.0006
Unknown	0.3	.09
White	0
Ethnicity		.0005
Hispanic	0.5	.0002
Refused/Unknown	−0.1	.6
Non-Hispanic	0
ESI		.4
1	0.5	.6
2	0.07	.6
3	−0.05	.7
4	−0.07	.6
5	0
Prescription written	0.08	.1
Labs ordered	−0.1	.1
Imaging ordered	0.1	.06
Procedure performed	0.3	<.0001
Time to provider	−0.008	<.0001
Length of stay	−0.002	<.0001

Abbreviations: KPI, Key Performance Indicator; PFE, patient family experience; ESI, Emergency Severity Index.

^a Value assigned at triage to indicate anticipated resource needs as well as illness or injury severity; prescription written, labs ordered, imaging ordered, and procedure performed refer to any of those elements occurring during the patient encounter. Time to provider is the time of arrival to the first provider assessment; Length of stay is the time between patient arrival and discharge.

Discussion

In this study, we hypothesized that ESI triage levels might serve as an independent predictor for PFE during PED visits. Although there is no independent association between these variables, the results of this study demonstrated several key findings. In our multivariable model, clinical factors, including time to provider, total length of stay, and procedure performance are associated with the KPI score, with abbreviated time to provider, shorter length of stay, and any procedure performance positively impacting experience. This is consistent with other studies, which have demonstrated that increased door-to-room time is associated with lower patient satisfaction, supporting the concept of waiting room interventions to improve overall patient experience.^9–13 In the wake of a global pandemic, as institutions continue to endure increased stress on existing healthcare and staffing resources, experience-driven efforts should primarily focus on these metrics.^16,17

We did find univariate associations between ESI scores and KPI, an association that was lost in the multivariate model. It is likely that this association is lost because of the inherent interaction between ESI and time to provider and total length of stay. A more ill or injured child may also be more likely to have a procedure performed. However, recognizing these factors, providers in the front of house, such as waiting room or triage, might focus experience-driven interventions on patients with higher ESI triage scores or in whom procedures and interventions are unlikely to be performed to help optimize their experience when times to providers are prolonged. Considerations here could include accurate and visible updates on expected wait times, positive distractions during the waiting room experience, and liberalization of diet in low-risk patients. In one quality improvement study, family experiences in the ED were improved through communication around wait times and waiting room rounds. ¹⁵ This study did not address the illness and injury spectrum captured through ESI levels, which might have promoted targeted expectation setting for families of patients with higher ESI levels, where longer waits and times to providers are more likely to be endured.

Although prescribing medications or performing labs and imaging would not likely lead to higher KPI scores, procedure performance has an independent association with PFE. More research into the relationship between procedures and PFE is needed as there is a relative lack of literature in this area. Although we did not directly study the interaction between procedure performance and ESI, it can be postulated that more ill or injured children (with resultant lower ESI scores) may be more likely to have a procedure performed. This interaction may have also contributed to the lack of association between ESI and PFE scores in our multivariate model.

In this study, we found that patients and families who identified as Hispanic race responded more positively to the KPI question in the PFE survey. This is consistent with a similar study from 2018 that looked at differences in patient experience between Hispanic and non-Hispanic white patients, which found that Hispanics tended to be more satisfied with their care than their non-Hispanic counterparts ¹⁷ . Similar results have been noted in other settings.^18,19 This interaction deserves further exploration, especially given the frequently reported disparate care for Hispanic patients in the PED.^20–23

Limitations

As this study is retrospective, it is limited by factors inherent to this study design. Based on its retrospective nature and data collection method, there may be independent predictors of PFE that were not assessed in this study. Also, due to the voluntary nature of the PFE post-visit response survey, our results are based only on families who filled out the survey after their visit. This may bias our results toward those families who had a particularly positive or negative experience. Similarly, our department only receives post-visit surveys from families discharged, and the experience of admitted patients is not considered in this study. Although admission might suggest a higher likelihood of certain PED tasks (imaging, labs, procedures, etc) and lower ESI levels (higher illness or injury severity), the experience of these patients is not included in this study. We believe this is mitigated by the sheer number of patients in this study. This study was performed in a tertiarily care pediatric ED with a high focus on PFE, and its results may not be generalizable to all care centers.

Conclusion

In the PED, patients and families experiencing shorter wait times, time to providers, and those in whom a procedure was performed are likely to have a more positive experience during their visit. Although ESI triage score is not an independent predictor of experience, it is associated with variation in wait time and can be used as a proxy for these measures to design interventions aimed at improving the experience of patients and their families presenting to the PED.