Validity of the Electronic Triage System in Predicting Patient Outcomes in Tabriz, Iran: A Cross-Sectional Study

Abstract

Objective:

To validate the triage ratings performed by the Electronic Triage System (ETS) using hospitalization, length of stay, resource use, in-hospital mortality and patient bills as outcome measures.

Methods:

In this retrospective cross-sectional study the medical records of 387 patients were reviewed in a one-week period. The data included triage category and the outcome measures were hospitalization, length of stay, in-hospital mortality, patient bill, and used resources. The association between the triage category and hospitalization and death was assessed. The association between the triage category and the number of resources, length of stay, and the bill was also assessed.

Results:

The mean age of the patients was 43.65 ±21.17 years. Women comprised 40% (n=155) of 387 people who were included in the study. The frequency of Emergency Severity Index (ESI) 1, 2, 3, 4 and 5 categories were 18, 61, 127, 181 and 0 respectively. Phi and Cramer’s V for hospitalization and death were 0.365 (p<0.001) and 0.305 (p<0.001). Spearman's rho for bill, length of stay, and resource use were -0.483 (p<0.001), -0.228 (p<0.001) and -0.490 (p<0.001). The association between triage category and resource consumption was stronger than other outcomes.

Conclusion:

The ETS was valid in predicting all studied patient outcomes. The ETS has also the advantages of providing quick reports, giving feedback and providing data for research purposes.

Introduction

Immediate care of patients who visit the emergency department (ED) is a necessity [1] which mandates having a valid triage system [2]. Among several triage scales that have been developed to date, the Emergency Severity Index (ESI) triage has been introduced as suitable for the hospitals of Iran by the Ministry of Health and Medical Education (2011) [3]. Nowadays the use of information technology is increasingly growing in the health sector. Some countries have developed electronic triage systems based on existing or new algorithms [4-6] . The ability of such systems in accurately triaging the patients is proved in published studies [4, 7, 8] . In Iran the Electronic Triage System (ETS) has been developed in Tabriz University of Medical Sciences for the first time in the country. It is designed based on ESI algorithm and can determine the triage category and treatment area within the ED based on the patient data [9].

One of the most important pre-requisites of using any triage system is its validity. A valid triage system should be able to predict hospitalization, death, ED length of stay (LOS), and the number of resources [10]. Validity of traditional ESI triage is proved in a variety of studies [2, 11-14] .

The validity of its electronic version -the ETS- also needs to be assessed. Thus this study has been conducted to investigate the validity of triages performed by the ETS. The ESI triage was first implemented in the studied ED in 2008 and was at work continuously. Then the ETS was developed and it was implemented in December 2012 [9]. The ETS is a web-based computer application developed using PHP programming language and MySQL database which both of them are open source. To make triage nurses feel familiar with the system, the ETS was designed in a way that its user interface was similar to the paper forms used in the participating ED. The overall logic of the ETS was based on the ESI triage algorithm [15]. According to the ESI triage algorithm, patients are evaluated for life threating conditions then for instant unconsciousness. If those threat are rolled out then the vital signs are assessed for patients who at first do not have the criteria for level 1, 2, 4 and 5. The vital signs that can be recorded on the system are as pulse rate (PR), respiratory rate (RR), blood pressure (BP), temperature (T), and the O2 saturation (SAO2). If the PR for a particular patient is more than 100 beats per minute, he/she would be classified as ESI 2 category and if less than that, the patient falls in ESI 3 category. But if the PR is low than 50 or higher than 120, then the patient's triage index will be upgraded to ESI 1. The RR more than 20 per minute moves the patient into ESI 2 category and otherwise ESI 3, unless the RR became more than 30 or less than 10, which in this case the assigned category changes to ESI 1. The systolic BP less than 90 and upper than 180 mm Hg makes the patient to be classified as ESI 1. The SAO2 is considered as more than 92%: ESI 3; less than 92% and more than 90%: ESI 2; less than 90%: ESI 1. To be conservative about the vital signs, it was determined that if the triage category varies according to different vital signs, the system takes into account the highest acuity one.

The participating ED consists of four treatment area including the cardiopulmonary resuscitation room (CPR), trauma room, medical room (non-trauma), and the fast track. After acquiring sufficient information, the ETS determines the triage category and the treatment area [16]. Like many other EDs that use the ESI triage, the triage is performed by nurses. Using the ETS, the physicians in each treatment room are able to see the basic information of triaged patients immediately after triage ends- even before the patient enter the room. By clicking on "Details", the complete triage form will be shown. When logging in the system the physicians can choose either to see all patients of the ED or only the patients who are assigned to the particular treatment room that the physician is there. Physicians, after examining the patient, decide to confirm the accuracy of triage done by nurse or not. If the physician disagrees with the nurse about the assigned triage category, he/she chooses the over-triage or under-triage. Then a box becomes active and asks the physician for new triage category and treatment room. There is also a dedicated place where physicians can write down the reason for deciding over/under triage or any comment that may help others in the next treatment room.

The present study is aimed at investigating the extent to which the ETS is able to predict the outcomes of ED patients. Measuring predictive validity of the triage ratings was done using hospitalization, in-hospital mortality, incurred patient costs in ED, LOS in the ED, and the number of used resources.

Materials and Methods

Study Design

 In this retrospective cross-sectional study medical records of patients presented to the ED of Tabriz Imam Reza hospital during a one-week course in March 2013 were reviewed. The institutional review board, which is the Ethics Committee of Tabriz University of Medical Sciences, approved the study protocol.

 Study Setting and Population

 The participating ED is placed in a university tertiary care large hospital that serves as referral center for other hospitals of the city and the surrounding cities. Since specialist hospitals for psychiatry, obstetrics, children, and burnt patients exist in the city these patients usually are not seen in this ED except for complex cases. Children under age 15 are treated only for injury and trauma at this ED. The triage is done for all who seek care from the ED.

 Study Protocol

 All patients who their triage was done electronically were potentially eligible to include in the study. Some patients were excluded from study. The first group was those who left the ED without being seen by a physician. In the medical records of those patients there was no information. So they excluded from the study. The second group to exclude was those who left the ED against medical advice. The information about these patients was not complete and based on that information it is not possible to calculate the length of stay, the number of resources used and also the cost of services. Of 491 reviewed records, 104 people were left the ED without being seen or against medical advice, leaving 387 patients for the analysis. The data were abstracted by one abstractor to prevent the inter-reviewer variation. The abstractor was trained and then practiced with some records to confirm his ability just before the data collection began. The inclusion and exclusion criteria were explained to him in an explicit manner. The study variables were defined. Standardized data collection form was used to guide the abstractor and to ensure uniform handling of the data. Several meetings were hold during the data collection period to ensure the quality of data and to monitor the performance of the abstractor.

 Measures

 Medical records of remaining patients were reviewed for extracting the required data. The data included the triage category, hospitalization, in-hospital mortality, the total amount of patient bill, ED length of stay, and the number of used resources. Those who transferred to another hospital or died after being triaged also considered as hospitalized. To calculate the ED length of stay the time interval between the triage and the disposition of the patients were calculated. The used resources were counted according to the ESI Implementation Handbook [15]. Dividing patients in work shifts was done based on the routine nursing shifts (morning: 8 to 14, afternoon: 14 to 20, and the night: 20 to 8 next day). The time of patient disposition was not documented in records of 33 patients then they were excluded in the analysis of LOS.

 Data Analysis

 To examine the association between the triage category as an ordinal variable and the nominal ones (hospitalization and death) the Phi and Cramer's V was used. The Spearman's Correlation Coefficient (rho) was used to examine the similar association between the triage category and the quantitative data (number of resources, LOS, and the bill). The correlation in each of the two tests is considered as poor if the coefficient is less than 0.3; moderate if greater than 0.3 and lesser than 0.5; and strong if greater than 0.5. All statistical tests were performed using statistical package for social sciences (SPSS Inc., Chicago, Illinois, USA) version 16.0 software. A 2-sided p-value of less than 0.05 was considered statistically significant.

Results

The mean age of the patients was 43.65 ±21.17 and 40.38 ± 22.28 years in the included and excluded patients, respectively. Although the age of included patients range from 1 to 90, only 20 (5.1%) of them were under 15 years old. Women comprised 40% (n=155) of 387 people who were included in the study. The similar percent in those who excluded from the study was %38. The triage of 99 people (25.6%) in the morning shift, 229 people (59.2%) in the afternoon, and 59 (15.2%) in the night was performed by the ETS. The frequency (and relative frequency) of ESI 1, 2, 3, 4 and 5 categories were 18 (4%), 61 (16%), 127 (33%), 181 (47%) and 0 (0%) respectively. Table1 shows the distribution of hospitalization, death and LOS in ESI categories. A statistically significant association was observed between the hospitalization and the triage category (Phi and Cramer’s V= 0.365, p value <0.001). These values were (0.288, p=0.005) and (0.418, p<0.001) for women and men respectively. Comparing these values shows that the association was stronger in men rather than the women. The test was repeated for the shifts separately and the values of (0.377, p=0.003), (0.420, p<0.001) and (0.284, p=0.189) was obtained for the morning, afternoon, and the night shift respectively. The P values ​​show that the association did not exist in the night shift.

Table1.

Distribution of hospitalization, in-hospital mortality, and ED^a length of stay among ESI categories.

ESI b category	Hospitalization N (%)	Death N (%)	LOS c (minute) Median (IQRd)
	15 (83.3)	3 (16.7)	261 (166)
	28 (45.9)	1 (1.6)	186 (180)
	49 (38.6)	1 (0.8)	157 (179)
	29 (16)	0	120 (136)
	0	0	0

^aED: Emergency Department;

^bESI: Emergency Severity Index;

^cLOS: Length Of Stay;

^dIQR: Inter-Quartile Range

As it is seen in Table1, five people died in the study cohort which significantly were in higher acuity categories of the ESI triage (Phi and Cramer’s V= 0.305, p<0.001). There was a relation between the costs and the triage categories (Spearman's rho= -0.483, p<0.001). The separate analysis for men, women, morning, afternoon, and the night produced the values of (-0.506, p<0.001), (-0.433, p<0.001), (-0.379, p<0.001), (-0.542, p<0.001), and (-0.402, p=0.002) respectively. There was also an association between the triage category and the ED length of stay (Spearman's rho= -0.228, p<0.001). The values of rho and P for women, men, morning, afternoon, and the night were (-0.205, p=0.016), (-0.238, p<0.001), (-0.108, p=0.301), (-0.238, p=0.001) and (-0.289, p=0.031) respectively. The value of P indicates that the association was not statistically significant in the morning shift. 

The number of used resources for the patients correlate with the triage categories (Spearman's rho= -0.490, p<0.001). The values for women, men, and morning, afternoon and night shifts were (-0.424, p<0.001), (-0.529, p<0.001), (-0.435, p<0.001), (-0.564, p<0.001) and (-0.230, p= 0.088) respectively. The value of P shows no statistically significant correlation in the night shift.

Discussion

Findings of this study showed that the ETS was valid in predicting hospitalization, ED length of stay, resource use, in-hospital mortality, and bills of the patients. The association between the triage category and the number of resources was stronger than other outcome measures.

Medical condition of the patients who became hospitalized is more serious than those who are discharged from ED [10]. Thus the association of the triage category and the hospitalization means that the ETS has correctly detected them. The correlation was stronger in men. This means that men compared to women who are in the same triage category, are more likely to be hospitalized. Yet, another study investigating validity of triages in the participating ED reported that the ETS was unable to predict admission to critical/intensive care unit [17]. Beside the patients’ medical condition, there are some confounding factors affecting admission to such units like variation in the opinion of the physicians whether to admit a particular patient in critical or non-critical ward, availability of beds in the critical care units, and also personal relations and recommendation of friends or upper hand officials to admit the patient in the ICU/CCU [17]. An association was observed between the ETS triage category and the in-hospital mortality. Although, the number of deaths was not high enough (5 out of 387) and it is better to be studied with a bigger study sample.

It is assumed that patients with more serious condition tend to use more resources and thus have higher bills [4]. The association between the triage category and the amount of bills indicates that the ETS was valid in predicting the bills. This is in line with the finding of the study on traditional triage in the participating ED [2]. Similar studies performed on the electronic version of the Canadian Triage and Acuity Scale (eTRIAGE) and the Soterion Rapid Triage System found an association between triage category and the bills [4, 7] . Wiler et al. reported an association between ESI triage category and the ED professional charges. They stated that the triage categories can be used to estimate the lost revenue of left without being seen (LWBS) patients [18].

A statistically significant association was observed between the triage category and the ED length of stay. It means that the patients who were triaged by the ETS spent different amount of time according to the acuity of their triage categories. This shows that the ETS has correctly detected those who need more services and thus need to stay longer in the ED. While no significant association was reported in the study of validity of traditional triage in the participating ED [2].

The association between the triage category and the number of resources was stronger than the other outcomes. It may be due to the fact that despite all other outcome measures used for validating triage ratings, predicting the number of resources is a part of ESI triage algorithm [15]. Another possible cause is the ability of triage nurses in predicting the number of resources that a particular patient would need [19]. Yet the study of Dong et al. stated that the seriousness of a patient's medical condition does not always correlate with the number of resources that he/she receives [7].

At the first glance it may seem that the high rate of exclusion can potentially bias the results of the study. But excluding those people for which the data is incomplete has been done for the precision of the study. Since we cannot determine if they did not leave the ED against medical advice, how the outcomes would change. In fact the high rate of left without being seen and/or discharge against medical advice (21%) that was observed in this study is similar to findings of the study on another tertiary care emergency department in Iran (20%) [20]. And most probably it was not an issue relating to the ETS, study design, or study period.

A limitation of this study was low sample size in the night shift. Yet the total number of studied patients was sufficient enough to observe the statistical correlations. Another point is that the present study was conducted only three months after implementing the ETS. Then may be some nurses were still learning to use the system. It can be assumed that if the study was performed later, the triage ratings might show more validity [7]. Although, we cannot certainly specify a time period after which the triage nurses arrive the maximal competency with the system.Although the main purpose of the triage is to detect those patients who are in need of immediate attention [21], it can be used as a tool for determining the acuity level of EDs' case mix [4, 22] and then for comparing and benchmarking EDs [7]. Electronic triage systems can accelerate this by providing managers and administrators with fast and timely reports [23]. The Electronic Triage System (ETS) developed at Tabriz University of Medical Sciences has the advantage of providing such quick reports like the over-triage and under-triage rate [9]. The ETS also makes it possible for triage nurses to get feedback about the triages that each nurse has performed whether they are confirmed by the physician or not [9]. Giving such feedback can theoretically improve the triage over time. It can also be a basis for assessing the need for retraining courses. Also the database that ETS provides can be a valuable source for research purposes [23]. Computer skills of the triaging nurses and the existence of the IT infrastructure are two important prerequisites to use the ETS [16]. Another issue to consider is the reliability of triage ratings performed by the ETS which is proven [8].

In conclusion, the ETS was valid in predicting all studied patient outcomes. The ETS has also the advantages of providing quick reports, giving feedback and providing data for research purposes.

Conflict of Interest: None declared.

Funding/Support

This is part of an MSc dissertation funded by Tabriz University of Medical Sciences (grant number: A142).